<p>There is a discussion of this issue in the recent book by Fedor V. Fomin and Dieter Kratsch "<a href="http://books.google.com/books?id=LXDe1XHJCYIC&amp;lpg=PA195&amp;vq=traveling&amp;pg=PA6#v=onepage&amp;q&amp;f=false" rel="nofollow">Exact Exponential Algorithms</a>" where they specify the running time in the unit-cost RAM model and the log-cost RAM model ($W$ - the maximum distance between the cities and $f(n)=\mathcal{O}^{\ast}(g(n))$ if $f(n)=\mathcal{O}(g(n)poly(n))$):</p>

<blockquote>
  <p>$\mathcal{O}^{\ast}(2^n)$ and $2^n\log Wn^{\mathcal{O}(1)}$ (note, $2^n\log Wn^{\mathcal{O}(1)}\notin\mathcal{O}^{\ast}(2^n)$ ), respectively. </p>
</blockquote>

CC BY-SA 2.5
2010-12-09T14:33:54.763Z
So they dodge the issue by hiding the polynomial factor.  I want to know what the polynomial factor is!
Jeffε
CC BY-SA 2.5
2010-12-09T15:07:04.180Z
They assume that the polynomial factor is $n^2$ (see the link in my comment).
Oleksandr Bondarenko
CC BY-SA 3.0
2010-12-09T09:57:05.037Z
2011-05-13T20:17:42.670Z
2011-05-13T20:17:42.670Z
Oleksandr Bondarenko
<p>This is less of a mathematical answer than a philosophical one, but I prefer to think of a RAM model that allows constant-time manipulation of integers with some number B of bits that is unknown but at least as large as $\log_2 S$, where S is the amount of space that the algorithm requires. Because, if the integers weren't that big, how could you even address your memory? For polynomial time and space algorithms it's the same as O(log n) bits, but for exponential space algorithms it avoids the problem.</p>

<p>Of course, if S exceeds the amount of memory you actually have, your algorithm won't run at all. Or, it will run by paging information into and out of memory and you should be using a memory hierarchy model instead of the RAM model.</p>

CC BY-SA 2.5
2010-12-10T04:47:05.673Z
I'm used to the idea that the machine model should depend on the input size $n$, but there's something a little wonky about letting the machine model depend on the algorithm.  Do you really want to let your machine solve any problem in PSPACE in constant time, as long as you're already using exponential space?
Jeffε
CC BY-SA 2.5
2010-12-12T23:29:42.093Z
To me it's less a question of making the model vary depending on the algorithm, and more a question of having a model that is fixed but not capable of running all algorithms (because it runs out of space). That doesn't sound so different from real computers, to me.
David Eppstein
CC BY-SA 3.0
2011-09-13T19:22:06.067Z
I am not convinced by David's answer. There are two issues here. One is theoretical, the other practical. In the theoretical setting it is more natural to be precise about the model and analyze the running time appropriately. In the practical setting it is not easy to know whether one would actually run out of memory on any particular instance because of various optimizations that one can do (and do partial memoization etc), however, when implementing the algorithm we will have to deal with how we store the sets and index into them. The above model does not help in this regard.
Chandra Chekuri
CC BY-SA 2.5
2010-12-09T06:27:14.360Z
2010-12-09T06:27:14.360Z
David Eppstein
<p><a href="https://cstheory.stackexchange.com/questions/3581/time-complexity-of-held-karp-algorithm-for-tsp">A recent question</a> discussed the now-classical dynamic programming algorithm for TSP, due independently to <a href="http://portal.acm.org/citation.cfm?doid=321105.321111" rel="nofollow noreferrer">Bellman</a> and <a href="http://dx.doi.org/10.1137/0110015" rel="nofollow noreferrer">Held-Karp</a>.  The algorithm is <a href="http://xkcd.com/399/" rel="nofollow noreferrer">universally</a> reported to run in $O(2^n n^2)$ time.  However, as one of my students recently pointed out, this running time may require an unreasonably powerful model of computation.</p>

<p>Here is a brief description of the algorithm.  The input consists of a directed graph $G=(V,E)$ with $n$ vertices and a non-negative length function $\ell\colon E\to\mathbb{R}^+$.  For any vertices $s$ and $t$, and any subset $X$ of vertices that excludes $s$ and $t$, let $L(s,X,t)$ denote the length of the shortest Hamiltonian path from $s$ to $t$ in the induced subgraph $G[X\cup\{s,t\}]$.  The Bellman-Held-Karp algorithm is based on the following recurrence (or as economists and control theorists like to call it, “Bellman's equation”):</p>

<p>$$
    L(s,X,t) = \begin{cases}
       \ell(s,t) &amp; \text{if $X = \varnothing_{\strut} $} \\
       \min_{v\in X}~  \big(L(s, X\setminus\lbrace v\rbrace, v) + \ell(v,t)\big)
              &amp; \text{otherwise}
    \end{cases}
$$</p>

<p>For any vertex $s$, the length of the optimal traveling salesman tour is $L(s,V\setminus\{s\}, s)$.  Because the first parameter $s$ is constant in all recursive calls, there are $\Theta(2^n n)$ different subproblems, and each subproblem depends on at most $n$ others.  Thus, the dynamic programming algorithm runs in $O(2^n n^2)$ time.</p>

<p>Or does it?!  </p>

<p>The standard integer RAM model allows constant-time manipulation of integers with $O(\log n)$ bits, but at least for <em>arithmetic</em> and <em>logical</em> operations, larger integers must be broken into word-sized chunks.  (Otherwise, <a href="http://www.computational-geometry.org/mailing-lists/compgeom-announce/2003-December/000852.html" rel="nofollow noreferrer">strange things</a> can happen.)  Is this not also true of access to longer memory addresses?  <strong>If an algorithm uses superpolynomial space, is it reasonable to assume that memory accesses require only constant time?</strong></p>

<p>For the Bellman-Held-Karp algorithm in particular, the algorithm must transform the description of the subset $X$ into the description of the subset $X\setminus\{v\}$, for each $v$, in order to access the memoization table.  If the subsets are represented by integers, these integers require $n$ bits and therefore cannot be manipulated in constant time; if they are not represented by integers, their representation cannot be used directly as an index into the memoization table.</p>

<p>So: <strong>What is the actual asymptotic running time of the Bellman-Held-Karp algorithm?</strong></p>

CC BY-SA 3.0
2011-09-12T15:27:20.610Z
Your "strange things" link is broken.
Tyson Williams
CC BY-SA 3.0
2011-09-13T12:00:48.680Z
I fixed the link.
Jeffε
<ds.algorithms><machine-models><computing-over-reals>
Time complexity of Bellman-Held-Karp algorithm for TSP, take 2
Jeffε
# Time complexity of Bellman-Held-Karp algorithm for TSP, take 2

(Asked by: username_0 on 2010-12-09)

[A recent question](https://cstheory.stackexchange.com/questions/3581/time-complexity-of-held-karp-algorithm-for-tsp) discussed the now-classical dynamic programming algorithm for TSP, due independently to [Bellman](http://portal.acm.org/citation.cfm?doid=321105.321111) and [Held-Karp](http://dx.doi.org/10.1137/0110015). The algorithm is [universally](http://xkcd.com/399/) reported to run in $O(2^n n^2)$ time. However, as one of my students recently pointed out, this running time may require an unreasonably powerful model of computation.

Here is a brief description of the algorithm. The input consists of a directed graph $G=(V,E)$ with $n$ vertices and a non-negative length function $\ell\colon E\to\mathbb{R}^+$. For any vertices $s$ and $t$, and any subset $X$ of vertices that excludes $s$ and $t$, let $L(s,X,t)$ denote the length of the shortest Hamiltonian path from $s$ to $t$ in the induced subgraph $G[X\cup\{s,t\}]$. The Bellman-Held-Karp algorithm is based on the following recurrence (or as economists and control theorists like to call it, “Bellman's equation”):

$$
L(s,X,t) = \begin{cases}
\ell(s,t) & \text{if $X = \varnothing\_{\strut} $} \\
\min\_{v\in X}~ \big(L(s, X\setminus\lbrace v\rbrace, v) + \ell(v,t)\big)
& \text{otherwise}
\end{cases}
$$

For any vertex $s$, the length of the optimal traveling salesman tour is $L(s,V\setminus\{s\}, s)$. Because the first parameter $s$ is constant in all recursive calls, there are $\Theta(2^n n)$ different subproblems, and each subproblem depends on at most $n$ others. Thus, the dynamic programming algorithm runs in $O(2^n n^2)$ time.

Or does it?!

The standard integer RAM model allows constant-time manipulation of integers with $O(\log n)$ bits, but at least for *arithmetic* and *logical* operations, larger integers must be broken into word-sized chunks. (Otherwise, [strange things](http://www.computational-geometry.org/mailing-lists/compgeom-announce/2003-December/000852.html) can happen.) Is this not also true of access to longer memory addresses? **If an algorithm uses superpolynomial space, is it reasonable to assume that memory accesses require only constant time?**

For the Bellman-Held-Karp algorithm in particular, the algorithm must transform the description of the subset $X$ into the description of the subset $X\setminus\{v\}$, for each $v$, in order to access the memoization table. If the subsets are represented by integers, these integers require $n$ bits and therefore cannot be manipulated in constant time; if they are not represented by integers, their representation cannot be used directly as an index into the memoization table.

So: **What is the actual asymptotic running time of the Bellman-Held-Karp algorithm?**

**Tags:** ds.algorithms, machine-models, computing-over-reals

> **Comments:**
> **username_1**: Your "strange things" link is broken.  
> **username_0**: I fixed the link.  

---

## 2 answers

### Answer by username_3 on 2010-12-09. Score: 12 | Accepted answer

This is less of a mathematical answer than a philosophical one, but I prefer to think of a RAM model that allows constant-time manipulation of integers with some number B of bits that is unknown but at least as large as $\log\_2 S$, where S is the amount of space that the algorithm requires. Because, if the integers weren't that big, how could you even address your memory? For polynomial time and space algorithms it's the same as O(log n) bits, but for exponential space algorithms it avoids the problem.

Of course, if S exceeds the amount of memory you actually have, your algorithm won't run at all. Or, it will run by paging information into and out of memory and you should be using a memory hierarchy model instead of the RAM model.

> **Comments:**
> **username_0**: I'm used to the idea that the machine model should depend on the input size $n$, but there's something a little wonky about letting the machine model depend on the algorithm.  Do you really want to let your machine solve any problem in PSPACE in constant time, as long as you're already using exponential space?  
> **username_3**: To me it's less a question of making the model vary depending on the algorithm, and more a question of having a model that is fixed but not capable of running all algorithms (because it runs out of space). That doesn't sound so different from real computers, to me.  
> **username_4**: I am not convinced by David's answer. There are two issues here. One is theoretical, the other practical. In the theoretical setting it is more natural to be precise about the model and analyze the running time appropriately. In the practical setting it is not easy to know whether one would actually run out of memory on any particular instance because of various optimizations that one can do (and do partial memoization etc), however, when implementing the algorithm we will have to deal with how we store the sets and index into them. The above model does not help in this regard.  

---

### Answer by username_2 on 2010-12-09. Score: 8

There is a discussion of this issue in the recent book by Fedor V. Fomin and Dieter Kratsch "[Exact Exponential Algorithms](http://books.google.com/books?id=LXDe1XHJCYIC&lpg=PA195&vq=traveling&pg=PA6#v=onepage&q&f=false)" where they specify the running time in the unit-cost RAM model and the log-cost RAM model ($W$ - the maximum distance between the cities and $f(n)=\mathcal{O}^{\ast}(g(n))$ if $f(n)=\mathcal{O}(g(n)poly(n))$):

> $\mathcal{O}^{\ast}(2^n)$ and $2^n\log Wn^{\mathcal{O}(1)}$ (note, $2^n\log Wn^{\mathcal{O}(1)}\notin\mathcal{O}^{\ast}(2^n)$ ), respectively.

> **Comments:**
> **username_0**: So they dodge the issue by hiding the polynomial factor.  I want to know what the polynomial factor is!  
> **username_2**: They assume that the polynomial factor is $n^2$ (see the link in my comment).  

---

