Solutions: Analysis of data structures

Solutions to the practice problems at the end of Analysis of data structures. Line numbers count from the first line of each problem’s code.

Problem 1: Nested loops

def f(n):
    i = 0
    while i < n:
        j = 0
        while j < n:
            print(i, j)
            j += 1
        i += 1

First, group the lines by how often they run, and give each group a constant for one run:

• Line 2 (i = 0): $c_1$, run once.
• Lines 3, 4, and 8 (the outer loop test, j = 0, and i += 1): $c_2$, run once per outer iteration, so $n$ times.
• Lines 5, 6, and 7 (the inner loop test, the print, and j += 1): $c_3$, run $n$ times per outer iteration, and the outer loop iterates $n$ times, so $n^2$ times.

The total running time is $c_1 + c_2 n + c_3 n^2$. Keeping the highest-order term and dropping its constant gives $\Theta(n^2)$.

Problem 2: A fixed inner loop

The sections are the same as in Problem 1: line 2 costs $c_1$ and runs once; lines 3, 4, and 8 cost $c_2$ and run $n$ times. But lines 5, 6, and 7 now run 1000 times per outer iteration, however large $n$ is, so they run $1000n$ times. The total is

$c_1 + c_2 n + 1000\, c_3 n,$

which is $\Theta(n)$. The 1000 is large, but it is a constant, and it doesn’t grow with $n$.

Problem 3: A loop that calls a function

In f, line 6 (i = n) costs $c_1$ and runs once. Lines 7 and 9 (the loop test and i -= 1) cost $c_2$ and run $n$ times.

Line 8, g(n), is not a constant-time line: it calls g, whose running time depends on its argument. So analyze g first. Its loop runs $x$ times, at constant cost $c_3$ each, so g(x) takes time $c_3 x$, and each call g(n) costs $c_3 n$. Line 8 runs $n$ times, for a total of $c_3 n \cdot n$.

The total is $c_1 + c_2 n + c_3 n^2$, which is $\Theta(n^2)$.

Problem 4: A doubling loop variable

Line 2 costs $c_1$ and runs once. Lines 3, 4, and 5 cost $c_2$ per iteration. Because i doubles on every iteration, starting at 1, the loop stops once i reaches $n$, after about $\log_2 n$ iterations. The total is $c_1 + c_2 \log_2 n$, which is $\Theta(\log n)$.

Problem 5: Factors up to the square root

Line 4 (i = 1) and line 9 (the final print) cost $c_1$ together and run once. Lines 5 through 8 (the loop test, the if and its print, and i += 1) cost at most $c_2$ per iteration, and the loop runs while i is at most $\sqrt n$: $\lfloor \sqrt n \rfloor$ times. The total is $c_1 + c_2 \sqrt n$, which is $\Theta(\sqrt n)$.

Problem 6: Linked-list operations

Let the list have $n$ items, and so $n + 1$ nodes, counting the sentinel. The $+1$ doesn’t matter in big-Θ terms.

• __init__, insert_after, prepend, append, and deleting a given node each take $\Theta(1)$ time: each looks at and relinks only a constant number of nodes. prepend and append reach their positions directly, through sentinel.next and sentinel.prev.
• get(i) has to walk $i$ nodes from the sentinel, since a linked list cannot jump to a position. In the worst case, $i$ is near the end of the list: $\Theta(n)$. (Walking from whichever end is closer halves the worst case, but it is still $\Theta(n)$.)
• length(), written to walk the list and count, visits every node once in every case: $\Theta(n)$, best and worst. (Keeping a running count, updated on every insertion and deletion, makes it $\Theta(1)$.)

Problem 7: Growing by a constant

The block starts with capacity 10, so the first 10 appends need no copying. The 11th append finds the block full and copies 10 items into a block of capacity 20; the 21st copies 20 items; the 31st copies 30; and so on. Over $n$ appends, the resizes copy

$10 + 20 + \cdots + (n - 10) = 10\left(1 + 2 + \cdots + \left(\frac{n}{10} - 1\right)\right) = 10 \cdot \frac{\left(\frac{n}{10} - 1\right)\frac{n}{10}}{2} = \frac{n(n - 10)}{20}$

items in total, using the arithmetic-series formula. That is $\Theta(n^2)$ copies for $n$ appends, or $\Theta(n)$ per append on average. Growing by 10 slots instead of 1 makes the resizes ten times rarer, but that is only a constant factor.

With doubling, the copies are $1 + 2 + 4 + \cdots$, up to the largest power of 2 below $n$, a total of less than $2n$. That is $\Theta(n)$ copies for $n$ appends, or $\Theta(1)$ per append on average, as the notes’ argument about one block at a time showed.

Problem 8: Counting digits

Line 2 (count = 0) costs $c_1$ and runs once. Lines 3, 4, and 5 cost $c_2$ per iteration. Each iteration removes one decimal digit from n, so the loop runs once per digit of the original $n$: $\lfloor \log_{10} n \rfloor + 1$ times. The final loop test and the return cost $c_3$. The total is

$c_1 + c_2 (\lfloor \log_{10} n \rfloor + 1) + c_3,$

which is $\Theta(\log n)$. Dividing by 10 instead of 2 changes only the base of the logarithm, which is a constant factor.

Problem 9: An inner loop that starts at i

The outer loop’s own step costs $c_1$ and runs $n$ times. On outer iteration $i$, the inner loop runs $n - i$ times: $n$ times when $i = 0$, $n - 1$ times when $i = 1$, and so on down to once. So the print runs

$n + (n - 1) + \cdots + 1 = \frac{n(n+1)}{2}$

times, at $c_2$ each, for a total of $c_1 n + c_2 \frac{n(n+1)}{2}$, which is $\Theta(n^2)$. The inner loop is shorter on later iterations, but on average it still runs about $n/2$ times, and a constant factor of $1/2$ doesn’t change the growth rate.

Problem 10: Looking for a repeated value

Line 2 costs $c_1$ and runs once. The outer loop’s step costs $c_2$ per iteration. The inner loop’s test and comparison cost $c_3$ per iteration. The final return False costs $c_4$.

• Best case: the first two items are equal. The outer loop runs once, the inner loop runs once and returns True, for $c_1 + c_2 + c_3$, which is $\Theta(1)$. This is a list of any length whose first two items match, not a short list.
• Worst case: no value repeats, so every pair is compared. On outer iteration $i$, the inner loop runs $n - 1 - i$ times, for $(n-1) + (n-2) + \cdots + 0 = \frac{n(n-1)}{2}$ comparisons in all. The total is $c_1 + c_2 n + c_3 \frac{n(n-1)}{2} + c_4$, which is $\Theta(n^2)$.

Problem 11: Building a list at the front

The loop’s own step costs $c_1$ and runs $n$ times. The call result.insert(0, i) is not constant time: when result holds $i$ values, it shifts all $i$ of them, at a cost of $c_2$ per value. Over the whole loop, the shifts cost

$c_2 (0 + 1 + \cdots + (n-1)) = c_2 \frac{n(n-1)}{2}.$

(The occasional resize of the array adds only $\Theta(n)$ in total, as in the amortized analysis of append.) The total is $\Theta(n^2)$. Appending instead, result.append(i), takes $\Theta(1)$ amortized time per value, and then reversing the list once, or building it with range(n - 1, -1, -1) to begin with, takes $\Theta(n)$ time in all.

Problem 12: A doubling inner loop

The outer loop’s step, together with j = 1, costs $c_1$ and runs $n$ times. The inner loop doubles j from 1 until it reaches $n$, so it runs $\lceil \log_2 n \rceil$ times for each outer iteration, at $c_2$ each. The total is

$c_1 n + c_2 n \lceil \log_2 n \rceil,$

which is $\Theta(n \log n)$. It is the same on every input of size $n$.

Problem 13: Cases and bounds

1. True. The worst case is $\Theta(n^2)$, which is at least a constant times $n$ for large $n$. It is a lower bound, but not a tight one.
2. False. The best case, with the first two items equal, is $\Theta(1)$: it does not grow with $n$ at all.
3. True. No input takes longer than the worst case, and the worst case is $O(n^2)$.
4. False. A list whose first two items are equal takes constant time. $\Omega(n^2)$ holds for the worst case, not for every input.
5. True. For each $n$, the best case is no slower than the worst case, so any upper bound on the worst case is also an upper bound on the best case.
6. True. For each $n$, every input takes at least as long as the best case, so any lower bound on the best case holds for every input. Statements 5 and 6 are why upper bounds are usually stated for the worst case and lower bounds for the best case: each then applies to every input.

Problem 14: Shrinking a growing array

1. Alternate appends and pops. The first append finds the block full, so it copies $m$ values into a block of capacity $2m$ and then holds $m + 1$ values. The next pop leaves $m$ values in a block of capacity $2m$, exactly half full, so it copies $m$ values into a block of capacity $m$, which is full again. The next append doubles again, the next pop shrinks again, and so on: every operation copies $m$ values. A sequence of $k$ such operations costs $\Theta(km)$, or $\Theta(m)$ per operation, so appends and pops are no longer $\Theta(1)$ amortized. The trouble is that each copy leaves the block either completely full or exactly at the shrinking threshold, so no cheap operations come between one copy and the next to pay for it.

2. After the append doubles the block to capacity $2m$, it holds $m + 1$ values, just over half full. It won’t shrink until it is a quarter full, at $m/2$ values, which takes about $m/2$ pops, and it won’t grow again until it is full, which takes about $m$ appends. In general, right after any resize the block is half full, so it takes at least about a quarter of its capacity in operations to reach either threshold. Those cheap operations pay for the next copy, whose size is proportional to the capacity, so appends and pops are again $\Theta(1)$ amortized.