Lecture Ten Notes

In this lecture we will talk further about induction: including strong induction, the need to strengthen the statements for induction and talk about induction over programs.

This lecture corresponds to section 2.3 of Ensley and Crawley's book. Some of the material is not available in Ensley-Crawley. Please refer to these notes.

The General (Strong) Induction.
- Fallacies with Strong Induction.
Induction on Programs

Strong Induction

To prove a statement (forall n in mathbb{N}) P(n) by strong induction.

Base Case: Establish (or in general the smallest number for which the theorem is claimed to hold.).
Inductive hypothesis: For all , Assuming hold, prove .

Strong induction is the “mother” of all induction principles. We can formulate many “baby” induction principles that are all just restatements of strong induction. In fact, mathematically they are all the same principle in different guises. However, seeing that is a little beyond the scope of this class.

Weak Induction

To prove a statement (forall n in mathbb{N}) P(n) by weak induction.

Base Case: Establish (or in general the smallest number for which the theorem is claimed to hold.).
Inductive hypothesis: For any , assuming holds, prove .

Step-By-Two Induction

To prove a statement (forall n in mathbb{N}) P(n) by step-by-two induction.

Base Case: Establish (or in general the smallest number and its successor).
Inductive hypothesis: Assuming holds, prove .

Q: Why does step-by-two induction need two base cases?

Look-Back-By-Two Induction

To prove a statement (forall n in mathbb{N}) P(n) by Look-Back-By-Two induction.

Base Case: Establish (or in general the smallest number and its successor).
Inductive hypothesis: For , assuming holds, prove .

Q: Why does look-back-by-two induction need two base cases?

Step-By-Three Induction

To prove a statement (forall n in mathbb{N}) P(n) by step-by-three induction.

Base Case: Establish (or in general the smallest number and its next two successors).
Inductive hypothesis: Assuming holds, prove .

Q: Why does step-by-three induction need three base cases?

We can continue with a cottage industry that produces induction principles, but we will stop here!

Why Strong Induction?

Let us try to prove a couple of theorems that will require strong induction.

Theorem-1: Every number n geq 2 is divisible by a prime number.

Proof Attempt # 1 (Weak Induction)

We will attempt prove by weak induction first.

Base Case: This holds for , which is a prime number and is divisible by itself.
Induction Hypothesis: If is divisible by a prime number then so is . The difficulty of proving this should be clear. Knowing something about divisibility of does not let us conclude the divisibility for .

Proof Failed

Proof By Strong Induction

We will now try strong induction.

Base Case: This holds for .
Induction Hypothesis: Assume that hold for any and prove that holds.

Notes: (a) P(k) here means is divisible by a prime number. (b) The theorem only applies for n geq 2 . We can start induction from there.

Let us assume that for all 2 leq k leq n , is divisible by a prime number. We wish to prove that n+1 is divisible by a prime number. Since n+1 geq 2 , we know that n+1 is either prime or composite. We are allowed to split two cases:

Case-1 is a prime number itself. So in this case, it is trivially divisible by itself.
Case-2 is composite. Therefore there is a number such that is divisible by . is divisible by some prime , by induction hypothesis. Therefore is divisible by some prime .

QED

Faulty Applications of Induction

Miss out on the base case(s).

Claim: Every number n geq 1 is odd.

Sproof by “strong induction”

Base Case: We know that is odd. So base case is verified.

Induction Hypothesis: Assuming that all numbers are odd, we wish to prove that is also odd.
- Let us assume all numbers are odd.
- We wish to prove that is odd.
- By induction hypothesis, is odd.
- Therefore for some .
- Therefore, we conclude that .
- Therefore, is also odd.

QED??

The proof failed because the Induction hypothesis proof is flawed. Let us split the proof step by step.

Induction Hypothesis:
- Let us assume that all numbers are odd. ( Perfectly within your rights to assume because of the strong induction )
- We wish to prove that is odd. ( good luck! )
- By induction hypothesis is odd. ( Wait a minute, this does not work for . For , we have which falls outside the scope of the induction hypothesis! ) FAIL!!

Here is a funny version of the same proof.

Claim: All horses are of the same color.

Sproof??

Proof by weak induction on number of horses.

Base case: Any single horse is of the same color as itself.
Induction: Let us assume that for every set of horses have the same color.
- We wish to prove the same for a set of horses.
- Let us take any set of horses and call them ${H_1,ldots, H_{n+1}}$ .
  - We can split the set into two parts $S_1: { H_1,ldots, H_n }, ;;;; S_2: { H_1,ldots,H_{n-1}, H_{n+1} }$
  - has horses in it. By induction hypothesis, they all have the same color.
  - Again is a set of horses. So they all have the same color.
  - How do we know that both have the same color. Becase they have horses $H_1,ldots,H_{n-1}$ in common.
  - Therefore $H_{n+1}$ must have the same color as .
- All set of horses have the same color. Ergo, all horses indeed have the same color.

The flaw behind this failed proof was explained in this class.

Application: Divisibility Test for Binary Numbers

In the very first lecture, we derived a divisibility test for binary number division by . As part of that test, we claimed the following theorem:

Theorem: For all natural numbers , 2^n mod 3 = 1 if is even and 2^n mod 3 = 2 if is odd.

Proof (Step-By-Two Induction)

Proof is by step-by-two induction on .

Base Cases We verify that for n=0 , we have 2^n mod 3 = 1 and likewise, for n= 1 , we have 2^n mod 3 = 2 .

Induction-Hypothesis (Step by Two) For any , assume that

$2^n mod 3 = left{ begin{array}{cc} 1 & mbox{if} n mbox{even} 2 & mbox{if} n mbox{odd} end{array}right.$ ,

we wish to prove that for mathbf{n+2} ,

$2^{n+2} mod 3 = = left{ begin{array}{cc} 1 & mbox{if} n+2 mbox{even} 2 & mbox{if} n+2 mbox{odd} end{array}right.$ .

Proof of I.H. Let be any given number such that

$2^n mod 3 = left{ begin{array}{cc} 1 & mbox{if} n mbox{even} 2 & mbox{if} n mbox{odd} end{array}right.$

Consider n+2 . We have $2^{n+2} mod 3 = 4 times 2^n mod 3 = ((4 mod 3) times (2^n mod 3))mod 3$ .

Therefore, $2^{n+2} mod 3 = 2^n mod 3$ . Thus, if the property holds for it also holds for n+2 .

This same proof can be done by strong induction or even weak induction, if you wish.

Inductive proofs of programs

Often, we are interested in proving that a program we wrote is correct. For instance, let us take a simple program that adds the elements of an array to find the total. We wish to prove that it is correct.

Program to Sum Elements Of Array

def sumArray(a,n):
    # Assume a is an array of size n
    # Assume n > 0
    # Array elements are a[0]..,a[n-1]

    sum = 0
    j = 0
    while (j < n):  # LOOP HEAD
    	  sum = sum + a[j]
	  j = j +1

We note that whenever we reach the loop head, the following facts always hold:

(A) 0 leq mbox{texttt{j}} leq n (The value of j remains between 0 and n )

(B) $mbox{texttt{sum}} = sum_{i=0}^{mbox{texttt{j-1}}} a[i]$ (The value of sum is the partial sum of the first j-1 array elements ).

These facts are called loop invariants. Establishing a loop invariant is a fundamental task in proving that a program is correct.

Loop Invariant

A condition involving the program variables is a loop invariant if it is true whenever the LOOP HEAD is reached.

Let us try to state the loop invariant as a theorem involving the sumArray program.

Theorem Whenever control reaches LOOP HEAD the value of j satisfies the condition 0 leq mbox{texttt{j}} leq mbox{texttt{n}} .

Loop Invariant Proof

Proof is by weak induction on the number of times the loop body has been executed. We use here to distinguish it from texttt{n} the program variable.

Base Case:

N= 0 . This corresponds to the very first time, the loop is reached. We know by looking at the program that mbox{texttt{j}} = 0 . This satisfies the property 0 leq mbox{texttt{j}} leq mbox{texttt{n}} , since we are given that is the size of array texttt{a} which cannot be negative.

Induction Hypothesis:

Let us assume that the property holds after executions of the loop body. We wish to prove that it holds after N+1 executions.

Let texttt{j},texttt{n} be values of the variables at the loop head after the loop has been executed times. We assume that the inductive hypothesis holds. In other words, 0 leq mbox{texttt{j}} leq mbox{texttt{n}} is true. We wish to prove that if we execute the loop body once more, the resulting values for the variables texttt{j},texttt{n} will also satisfy the invariant.

Since are such that the loop can be executed one more time, it must be the case that must be true.
After the loop is executed, we have and where the prime ’ next to the variables are used to separate the values after the loop is executed once from the values before.
1. We therefore have that if then .
2. Similarly, since (due to the loop condition), we have .

Together, we have texttt{j}' geq 0 and texttt{j}' leq texttt{n}' . Thus, the loop condition holds even after the loop has been executed once.

Let us now prove the loop invariant property of partial sums for the loop:

$mbox{texttt{sum}} = sum_{i=0}^{mbox{texttt{j-1}}} a[i]$

Note that here is the index variable of the summation notation sum and not a program variable. texttt{j} on the other hand, is very much a program variable.

Theorem Whenever the control reaches LOOP-HEAD, we have

$mbox{texttt{sum}} = sum_{i=0}^{mbox{texttt{j-1}}} a[i]$

Proof

Once again, we prove by induction on , the number of times the loop has been executed.

Base-Case For N=0 , we have that the loop body has never been executed. Therefore, texttt{j}= 0 and texttt{sum} = 0 . We therefore have

mbox{texttt{sum}} = 0

$sum_{i=0}^{mbox{texttt{j-1}}} a[i] = underset{mbox{empty summation is zero}}{underbrace{sum_{i=0}^{-1} a[i]}} = 0$

Therefore, we conclude that

$mbox{texttt{sum}} = sum_{i=0}^{mbox{texttt{j-1}}} a[i]$

Ind.Hyp. Assume that the property holds after N loop executions, we wish to prove that it holds after N+1 loop executions.

Let (texttt{j},texttt{sum},texttt{n}) be the values of the corresponding variables after the loop has executed times. We assume that the loop executes one more time and gives us the values (texttt{j}', texttt{sum}', texttt{n}') . Examining the loop, we conclude the following relevant facts:

Assuming the induction hypothesis, we have

$begin{array}{rcll} texttt{sum}' &=& texttt{sum} + a[texttt{j}] & textcolor{red}{mbox{(From loop body )}} &=& sum_{i=0}^{mbox{texttt{j-1}}} a[i] + a[texttt{j}] &textcolor{red}{mbox{(From I.H for texttt{sum} )}} &=& sum_{i=0}^{mbox{texttt{j}}} a[i] & textcolor{red}{mbox{(Last term is pushed into the summation)}} &=& sum_{i=0}^{mbox{texttt{j}'-1}} a[i] & textcolor{red}{(mbox{substitute} texttt{j}' = texttt{j} + 1 )} end{array}$

In other words, we have established that

$texttt{sum}' = sum_{i=0}^{mbox{texttt{j}'-1}} a[i]$

QED

To understand the value of inductive invariants, can you explain how the proof will fail if the program were modified slightly as

Faulty program to Sum Elements Of Array

def sumArray(a,n):
    # Assume a is an array of size n
    # Assume n > 0
    # Array elements are a[0]..,a[n-1]

    sum = 0
    j = 0
    while (j < n):  # LOOP HEAD
	  j = j +1  # Fault: incrementing j first and then adding
	  sum = sum + a[j] # Fault: buffer overflow possible

Program to compute logarithm

We now consider a program that given a number n geq 1 , returns a number such that n leq 2^k < 2n . In other words k = lceil log_2(n) rceil .

Log Program

def logtwo(n):
    # Assume n >= 1

    lg = 0
    j = 1
    while (j < n):  # LOOP HEAD
    	  lg = lg +1
	  j = j *2
    return lg

Please take a second to understand how the program works. You can try it in a python interpreter.

To prove that this program is correct, we will provide the following loop invariants for the while loop.

$texttt{j} geq 0 mbox{AND} texttt{j} < 2*texttt{n} mbox{AND} texttt{j} = 2^{texttt{lg}}$

Proof

Proof is by induction on , the number of steps the while loop is executed.

Base Case For N=0 , we know that texttt{n} geq 1 , texttt{lg}= 0 and texttt{j} = 1 . We are also assuming that n geq 1 .

Therefore, it is easy to check that the invariant

$texttt{j} geq 0 mbox{AND} texttt{j} < 2*texttt{n} mbox{AND} texttt{j} = 2^{texttt{lg}}$

is satisfied.

I.H If the loop invariant is true after the loop has been executed times then it remains true after the loop has been executed N+1 times.

Again, let us assume the loop has been executed times and texttt{j},texttt{lg} and texttt{n} be the values for these variables. Let us assume that the loop executes one more time and the resulting values are texttt{j}',texttt{lg}' and texttt{n}' .

Here is a summary of the facts we know about these:

$texttt{j} geq 0 mbox{AND} texttt{j} < 2*texttt{n} mbox{AND} texttt{j} = 2^{texttt{lg}}$ (from the induction hypothesis )
( because, the loop condition should be satisfied )
( body of the loop )
.

Let us now prove that $texttt{j}' geq 0 mbox{AND} texttt{j'} < 2*texttt{n}' mbox{AND} texttt{j}' = 2^{texttt{lg}'}$ going conjunct by conjunct.

(a) Proof of texttt{j}' geq 0

(from the induction hypothesis )
( body of the loop )
.

(b) Proof of texttt{j}' < 2* texttt{n}'

( because, the loop condition should be satisfied )
( body of the loop )
( chaining the inequalities together )

$texttt{j} = 2^{texttt{lg}}$ (from the induction hypothesis )
, ( body of the loop )
$texttt{j}' = 2 * texttt{j} = 2 * 2^{texttt{lg}} = 2^{texttt{lg} +1} = 2^{texttt{lg}'}$ .

Therefore, we have proved that

$texttt{j}' geq 0 mbox{AND} texttt{j'} < 2*texttt{n}' mbox{AND} texttt{j}' = 2^{texttt{lg}'}$

Now, it remains to put together the inductive invariant to prove the the log function “works”.

Note that the while loop finishes when texttt{j} geq n (the negation of the loop condition holds). However, if the control is at the LOOP-HEAD the loop invariant holds.

Therefore, we have the following facts:

$texttt{j} = 2^{texttt{lg}}$ .

Putting them together, we conclude that when the loop terminates,

$texttt{n} leq 2^{texttt{lg}} < 2texttt{n}$ .