More tips on describing algorithms (from pp. 11ff)

• Algorithms are not programs. Please don’t write code (e.g., C, Python, Java, Haskell, R, …).
  • Use a combination of pseudocode and structured English.
    • Pseudocode uses the structure of formal programming languages to break algorithms into primitive steps.
    • The primitive steps themselves can be written using math, English, or a mixture of the two, whatever is clearest.
  • A good algorithm description is closer to what we should write in the comments of a real program than the code itself. Code is a poor medium for storytelling.
  • On the other hand, conditionals, loops, function calls, and recursion should be stated explicitly.

Who is the audience?

• Imagine you are writing pseudocode for a competent but skeptical programmer who is not as clever as you are.
  • This programmer should be able to quickly and correctly implement the algorithm in their favorite programming language.
• Goal: include every detail necessary to fully specify the algorithm, prove its correctness, and analyze its running time.

Baguenaudier solver

Pseudocode and English: It is clear what it’s doing and why it works:

solve(n):

If n=1, 
    flip(1)
If n=2
    flip(2)
    flip(1)
else (when n>2)
    solve(n-2) on the rightmost n-2 bits
    flip(n), the nth bit                            << legal, b/c there are n-2 0's at the end >>
    solve(n-2) in reverse on the rightmost n-2 bits << makes Puzzle look like 0111...111 >>
    solve(n-1) on rightmost (n-1) bits

Baguenaudier solver, but a little too code-y

Psuedocode only. It’s correct, but it doesn’t communicate as well:

B(int n){
        if(n == 0){
            print nothing
        }
        else if (n==1){
            print 1
        }
        else {
            B(n-2)
            print n
            B(n-2), but printed backwards 
            B(n-1)
        }
    }

Recurrence Relation

The recurrence relation should match the algorithm:

solve(n):

If n=1, 
    flip(1)
If n=2
    flip(2)
    flip(1)
else (when n>2)
    solve(n-2) on the rightmost n-2 bits
    flip(n), the nth bit                            << legal, b/c there are n-2 0's at the end >>
    solve(n-2) in reverse on the rightmost n-2 bits << makes Puzzle look like 0111...111 >>
    solve(n-1) on rightmost (n-1) bits

\[\begin{equation} M(n)=\begin{cases} 1 & \text{if $n=1$}.\\ 2 & \text{if $n=2$}.\\ 2M(n-2)+M(n-1)+1 & \text{if $n > 2$}. \end{cases} \end{equation}\]

Another Recurrence Relation

If the algorithm is different, the recurrence is different.

B(int n){
        if(n == 0){
            print nothing
        }
        else if (n==1){
            print 1
        }
        else {
            B(n-2)
            print n
            B(n-2), but printed backwards 
            B(n-1)
        }
    }

\[\begin{equation} M(n)=\begin{cases} 0 & \text{if $n=0$}.\\ 1 & \text{if $n=1$}.\\ 2M(n-2)+M(n-1)+1 & \text{if $n > 2$}. \end{cases} \end{equation}\]

Closed-form formula from OEIS

At least two options:

\[ 2^{n+1}/3 - 1/2 - (-1)^n/6 \] or

\[ M\left(n\right) = \left\lceil2\cdot\frac{2^n-1}{3}\right\rceil \] The former is easier; the latter requires cases in the induction argument.

Inductive verification

Base case: \(M(1)=1=\frac{2^{2}}{3}-\frac{1}{2}-\frac{(-1)^{1}}{6}\) and \(M(2)=2=\frac{2^{3}}{3}-\frac{1}{2}-\frac{(-1)^{2}}{6}\).

Suppose as inductive hypothesis that \(M(k)=\frac{2^{k+1}}{3}-1/2-\frac{(-1)^{k}}{6}\) for all \(k<n\).

(When in doubt, always use strong induction.)

Inductive step

\[\begin{align} M(n)&=2M(n-2)+M(n-1)+1\\ &=2\left(\frac{2^{n-1}}{3}-\frac{1}{2}-\frac{(-1)^{n-2}}{6} \right) + \left(\frac{2^{n}}{3}-\frac{1}{2}-\frac{(-1)^{n-1}}{6}\right)+1\\ &=\frac{2^{n}}{3}-1-\frac{2(-1)^{n-2}}{6}+\frac{2^{n}}{3}-\frac{1}{2}-\frac{(-1)^{n-1}}{6}+1\\ &=\frac{2^{n}}{3}+\frac{2^{n}}{3}-1+1-\frac{1}{2}-\frac{2(-1)^{n-2}}{6}-\frac{(-1)^{n-1}}{6}\\ &=\frac{2^{n+1}}{3}-\frac{1}{2}+\frac{2(-1)^{n-1}}{6}-\frac{(-1)^{n-1}}{6}\\ &=\frac{2^{n+1}}{3}-\frac{1}{2}+\frac{(-1)^{n-1}}{6}\\ &=\frac{2^{n+1}}{3}-\frac{1}{2}-\frac{(-1)^{n}}{6} \end{align}\]

Recall: Writing recursive functions

Your only task is to simplify the original problem, or to solve it directly when simplification is either unnecessary or impossible; the Recursion Fairy will solve all the simpler subproblems for you, using Methods That Are None Of Your Business…

Jamboard Questions

Consider the usual Towers of Hanoi puzzle (pp. 24ff) with \(n\) disks and 3 pegs, numbered 0, 1, and 2. Now suppose you are forbidden to move any disk directly between peg 1 and peg 2; every move must involve peg 0.

1. Describe a recursive algorithm to solve this puzzle.

2. Explain why (prove that) your algorithm never moves a disk between peg 1 and peg 2. (Use a strong induction hypothesis.)

Table Groups

Divide and Conquer paradigm

• When the problem domain partitions into smaller subproblems, the recursive paradigm is called divide and conquer.
  • Towers of Hanoi isn’t really divide and conquer.
  • Binary search is divide and conquer. The search space partitions into two halves.

Merge Sort

Merge(A[1..n], m):                        MergeSort(A[1..n]):
    i <- 1; j <- m + 1                        if n > 1
    for k <- 1 to n                               m <- floor(n/2)
        if j > n                                  MergeSort(A[1..m])
            B[k] <- A[i]; i <- i + 1              MergeSort(A[(m+1)]..m)
        else if i > m                             Merge(A[1..n],m)
            B[k] <- A[ j]; j <- j + 1
        else if A[i] < A[j]
            B[k] <- A[i]; i <- i + 1
        else
            B[k] <- A[j]; j <- j + 1
    for k <- 1 to n
        A[k] <- B[k]

Merge Sort recurrence

Merge(A[1..n], m):   # O(n) comparisons   MergeSort(A[1..n]):  # C(n) comparisons
    i <- 1; j <- m + 1                        if n > 1
    for k <- 1 to n                               m <- floor(n/2)
        if j > n                                  MergeSort(A[1..m])      # C(n/2) comparisons (approx)
            B[k] <- A[i]; i <- i + 1              MergeSort(A[(m+1)]..m)  # C(n/2) comparisons (approx)
        else if i > m                             Merge(A[1..n],m)        # O(n) comparisons
            B[k] <- A[ j]; j <- j + 1
        else if A[i] < A[j]
            B[k] <- A[i]; i <- i + 1
        else
            B[k] <- A[j]; j <- j + 1
    for k <- 1 to n
        A[k] <- B[k]

\[ C(n) = \left\{\begin{array}{ll} 0 & \mbox{if } n=1 \\ 2C(n/2) + O(n) & \mbox{if } n>1 \end{array} \right. \]

Merge Sort time

\[ C(n) = \left\{\begin{array}{ll} 0 & \mbox{if } n=1 \\ 2C(n/2) + O(n) & \mbox{if } n>1 \end{array} \right. \]

As a tree, we can model \(C(n)\) as:

          O(n)
         /    \
     C(n/2)   C(n/2)

Expanding recursively, we get the following recursion tree:

                             n
                    /                 \
                n/2                      n/2
            /        \              /          \
          n/4        n/4           n/4          n/4
        /    \       /    \       /   \        /    \
      n/8    n/8   n/8   n/8    n/8   n/8    n/8    n/8
    ... and so on

Merge Sort time

\[ C(n) = \left\{\begin{array}{ll} 0 & \mbox{if } n=1 \\ 2C(n/2) + O(n) & \mbox{if } n>1 \end{array} \right. \]

Total up each level in the recursion tree:

                                                            level total
                             n                                   n
                    /                 \
                n/2                      n/2                     n
            /        \              /          \ 
          n/4        n/4           n/4          n/4              n
        /    \       /    \       /   \        /    \
      n/8    n/8   n/8   n/8    n/8   n/8    n/8    n/8          n
    ... and so on

There are \(\log n\) levels in this tree, so \(C(n) \in O(n \log n)\).

(More to come.)