<p>Say you have 3 algorithms, A B and C.  You want to present a comparison of A B and C based on:</p>

<ul>
<li>Precompute overhead (say tree building etc that occurs prior to runtime)</li>
<li>Runtime speed (say query response time)</li>
</ul>

<p>The problem is, however, A B and C <strong><em>each have their own unique set of parameters that will dictate the quality of the results</em></strong>.  </p>

<ul>
<li>A has 3 parameters x, xx and xxx</li>
<li>B has 1 parameter y</li>
<li>C has 2 parameters z and zz</li>
</ul>

<p>If A has it's parameter x increased from 10 to 50, say, then A's precompute overhead doubles, and A's runtime speed is halved.</p>

<p>If B has it's parameter y increased from 0 to 0.001 say, it's precompute time stays the same but it's runtime speed becomes 1/3 of it's former speed.</p>

<p>So each algorithm has it's own set of quirks and behaves differently, depending on how you set each algorithm's parameters.</p>

<p><em>But we are interested in comparing A B and C based on the two categories (precompute and runtime) listed above!</em></p>

<p>How can you compare algorithms A B and C?</p>

CC BY-SA 3.0
2012-07-16T16:18:47.180Z
[en.wikipedia.org/wiki/Pareto_efficiency](http://en.wikipedia.org/wiki/Pareto_efficiency)
Radu Curticapean
CC BY-SA 3.0
2012-07-16T21:33:36.790Z
Is there some distribution over the set of parameters for each algorithms? For example, in 50% of the cases you use algorithm A with one set of parameters, and in the other 50% you use it with another set? If that's the case, you can compute the expected "precompute" and "runtime" and compare these. Alternatively, you can also compare the worst-case (i.e. find the parameters for which all algorithms run the most). I personally prefer the average case.
inquirer
<ds.algorithms>
Making algorithm comparison when algorithms use parameters
bobobobo
# Making algorithm comparison when algorithms use parameters

(Asked by: username_0 on 2012-07-16)

Say you have 3 algorithms, A B and C. You want to present a comparison of A B and C based on:

* Precompute overhead (say tree building etc that occurs prior to runtime)
* Runtime speed (say query response time)

The problem is, however, A B and C ***each have their own unique set of parameters that will dictate the quality of the results***.

* A has 3 parameters x, xx and xxx
* B has 1 parameter y
* C has 2 parameters z and zz

If A has it's parameter x increased from 10 to 50, say, then A's precompute overhead doubles, and A's runtime speed is halved.

If B has it's parameter y increased from 0 to 0.001 say, it's precompute time stays the same but it's runtime speed becomes 1/3 of it's former speed.

So each algorithm has it's own set of quirks and behaves differently, depending on how you set each algorithm's parameters.

*But we are interested in comparing A B and C based on the two categories (precompute and runtime) listed above!*

How can you compare algorithms A B and C?

**Tags:** ds.algorithms

> **Comments:**
> **username_1**: [en.wikipedia.org/wiki/Pareto_efficiency](http://en.wikipedia.org/wiki/Pareto_efficiency)  
> **username_2**: Is there some distribution over the set of parameters for each algorithms? For example, in 50% of the cases you use algorithm A with one set of parameters, and in the other 50% you use it with another set? If that's the case, you can compute the expected "precompute" and "runtime" and compare these. Alternatively, you can also compare the worst-case (i.e. find the parameters for which all algorithms run the most). I personally prefer the average case.  

---

## 0 answers

