Balance between informal and formal

• Include enough details to be correct.
• Try not to include details that obfuscate.
• Strive for an argument that convinces someone that the algorithm really works.
  • A description of what the algorithm does is not a proof that the algorithm is correct.

Cavarretta Sort Correctness

CavarrettaSort(A[0 .. n - 1]):
    if n = 2 and A[0] > A[1]
        swap A[0] and A[1]
    else if n > 2
         m = ceiling(2n/3)
         CavarrettaSort(A[0 .. (m - 1)])
         CavarrettaSort(A[(n - m) .. (n - 1)])
         CavarrettaSort(A[0 .. (m - 1)])

Clear proof, with a couple details glossed over

Base case: If n = 2, the two elements of the list are swapped if out of order, resulting a correctly sorted list.

Suppose as inductive hypothesis that CavarrettaSort(A[0 .. k-1]) correctly sorts its input array for all \(k<n\).

Then CavarrettaSort(A[0...n-1]) takes the following steps: 1. Correctly sorts the first 2/3 of the array, according to the inductive hypothesis 2. Correctly sorts the last 2/3 of the array, according to the inductive hypothesis 3. Correctly sorts the first 2/3 of the array again, according to the inductive hypothesis.

After step 2, the last 1/3 of the array will contain the highest 1/3 of numbers, because any of these numbers that may have been in the first 1/3 were moved into the middle 1/3 in step 1, and the last 1/3 in step 2. After step 3, the first 2/3 of the array will be sorted correctly as well, because all numbers in the first 2/3 at this point are less than all numbers in the last 1/3, resulting in a complete solution.

Alternative inductive step (more details)

First, since \(m\) is the ceiling of \(2n/3\), \(n-m\) is never larger than \(2m-n\).

After the first CavarrettaSort(A[0 .. (m - 1)]) call, all of the \(n-m\) elements of A[(2m-n)..(m-1)] (the second “1/3”) are greater than all of the \(2m-n\) elements of A[0..(2m-n-1)] (the first “1/3”), by inductive hypothesis. Therefore, none of the largest \(n-m\) elements of the entire array can be in A[0..(n-m-1)], and so the largest \(n-m\) elements must be in A[(n-m)..(n-1)] (the last “2/3”).

After the call CavarrettaSort(A[(n - m) .. (n - 1)]), these largest \(n-m\) elements must be in order in A[m..(n-1)] (the last “1/3”), by inductive hypothesis. The final recursive call puts the smallest \(m\) elements of the array in order in A[0..(m-1)], by inductive hypothesis. Therefore, the entire array is correctly sorted.

Recurrence

CavarrettaSort(A[0 .. n - 1]):
    if n = 2 and A[0] > A[1]
        swap A[0] and A[1]
    else if n > 2
         m = ceiling(2n/3)
         CavarrettaSort(A[0 .. (m - 1)])
         CavarrettaSort(A[(n - m) .. (n - 1)])
         CavarrettaSort(A[0 .. (m - 1)])

\[ C(n) = \left\{\begin{array}{lr} 1, & \text{for } n=2,\\ 3C\left(\left\lceil\frac{2}{3}n\right\rceil\right), & \text{for } n>2.\\ \end{array}\right. \]

Solving the recurrence

Ignoring the ceiling,

\[\begin{align} C(n) &= 3C\left(\frac{2}{3}n\right) = 3^2C\left(\left(\frac{2}{3}\right)^2n\right) = 3^3C\left(\left(\frac{2}{3}\right)^3n\right) \\ & = \cdots = 3^LC\left(\left(\frac{2}{3}\right)^Ln\right) = 3^L C(2) = 3^L \cdot 1 = 3^L \end{align}\]

Where \(\left(\frac{2}{3}\right)^Ln = 2\). Solving for \(n\) gives \(L = \log_{1.5}(n/2)\).

Therefore, \(C(n) = 3^{\log_{1.5}(n/2)} = (n/2)^{\log_{1.5} 3} \in O(n^{\log_{1.5} 3}) = O(n^{2.7095\ldots})\)

Recall: Merge (Quick?) Sort Recurrence

\[ C(n) = \left\{\begin{array}{ll} 0 & \mbox{if } n=1 \\ 2C(n/2) + O(n) & \mbox{if } n>1 \end{array} \right. \]

Total up each level in the recursion tree:

                                                            level total
                             n                                   n
                    /                 \
                n/2                      n/2                     n
            /        \              /          \ 
          n/4        n/4           n/4          n/4              n
        /    \       /    \       /   \        /    \
      n/8    n/8   n/8   n/8    n/8   n/8    n/8    n/8          n
    ... and so on

There are \(\log n\) levels in this tree, so \(C(n) \in O(n \log n)\).

General divide and conquer relations

• Each step costs \(O(f(n))\), plus
• the cost of \(r\) subproblems, each of size \(n/c\).
• We want an asymptotic (big-O) solution

\[ T(n) = \left\{\begin{array}{ll} \mbox{doesn't matter} & \mbox{if } n \leq n_0 \\ rT(n/c) + O(f(n)) & \mbox{if } n > n_0 \end{array} \right. \] Recursion tree:

• \(r\)-way branches (\(r\)-ary tree)
• Level of leaves: \(L = \log_c n\)
• Number of leaves: \(r^L = r^{\log_c n} = n^{\log_c r}\)

See Figure 1.9.

Three cases you can solve

Recursive case of divide and conquer recurrence:

\[ T(n) = rT(n/c) + O(f(n)) \]

1. Root dominates: \(T(n) = O(f(n))\)
   • happens when row sums decrease geometrically.
2. Internal nodes dominate: \(T(n) = O(f(n)\log n)\)
   • happens when row sums are constant.
3. Leaves dominate: \(T(n) = O(n^{\log_c r})\)
   • happens when row sums increase geometrically.

[CLRS] calls this the Master Theorem for solving d&c recurrences.

Examples

• \(C(n) = 2C(n/2) + O(n)\), (Merge Sort)
  • row sums all equal \(n\). Case 2: \(C(n) = O(n \log n)\)
• \(D(n) = 4D(n/2) + O(n^3)\)
  • row sums decrease geometrically. Case 1: \(D(n) = O(n^3)\)

Table Groups

Recursion Tree Practice

1. Draw a recursion tree for the recurrence \(F(n) = 3F(n/4) + O(\sqrt{n})\).

2. Write down the first four terms in the sequence of row sums.

3. Apply the Master Theorem to solve the recurrence.

Cavarretta recurrence, revisited.

4. Consider adding a constant term \(K = O(1)\) to the Cavarretta recurrence:

\[ C(n) = \left\{\begin{array}{lr} 1, & \text{for } n=2,\\ 3C\left(\left\lceil\frac{2}{3}n\right\rceil\right) + K, & \text{for } n>2.\\ \end{array}\right. \]

Use a recursion tree (with root \(K\)) to solve this recurrence. What effect did adding the constant term have?