CSCI 2824: Lecture #2 Notes

In this lecture we will cover the basic notation for sequences and summation.

Concepts learned:

Representing sequences: closed form, recurrence relation. Ordinal notation for sequences.
Converting recurrences to closed forms (very basics).
Summations and summation notation.
Extra: Alphabet sequences.

Sequences

We have seen this puzzle. Some one gives you a sequence of numbers and asks you to guess the next number. Often it is easy enough to do that. Sometimes, it can get really tricky!

Guess the next number(s) in the following sequence:

A common trick (a lot of IQ tests use) is to mix two simple ones into a seemingly complex one:

How about this one?

Working hard on a sequence, try this cool site: Online Encylopedia of Integer Sequences! :-)

How do we talk about sequences mathematically?

We use the notation a_1,a_2,ldots,a_n,ldots to denote a sequence. a_1 is the $1^{st}$ element. Likewise, the $n^{th}$ element is written a_n (pronounced ’’ a sub n’’).

A sequence can be specified in two ways:

Closed-Form Representation: We write $a_n = mbox{function involving } n$ that holds for all .
Recurrence Relation: This is also called a Recursive Formula or simply a recurrence. A sequence can be written as a recurrence by writing in terms of $a_{n-1}, a_{n-2},ldots$ and specify a base case for and so on.

We will now give examples to illustrate both ways in detail starting with the recurrence for a sequence.

Recursive Formulae

Each recursive formula or recurrence has two parts to it.

written as a function of $a_{n-1},ldots,a_1$ .
A starting value or a base case for .

This will be clear once we work through some examples. Let us start with some examples of recurrences.

Example-1 (Even Numbers)

We have the following recurrence for even numbers:

$a_n = a_{n-1} + 2, mbox{for all} n geq 1 ,.$

with the base case:

a_1 = 2

Do not forget the base case and note that recurrence holds only for n >= 1

We can think of a recurrence as a simple recursive program. Here is how the recurrence above translates into C/C code:

Code for Recurrence Relation

 /* Recursive formula is implemented by a recursive function */
 int A( int n) {
   assert(n >= 0);
   if ( n == 1) return 2;
   return A(n-1) + 2;
 }

Question: What happens to the code above, if we forgot the base case?

We saw an example where a_n was related just to $a_{n-1}$ . However, recurrences can be more general.

Example 2: Fibonacci Numbers

Fibonacci Numbers were invented by the famous Italian mathematician Leonardo Fibonacci. Interestingly these numbers turn up all over the place, often for reasons that are not well understood. They are closely related to the golden ratio. For more Fibonacci-ology see the Wikipedia Article.

Fibonacci numbers can be written as the recurrence:

$F_n = F_{n-1} + F_{n-2}, mbox{for all} n geq 3 ,.$

We have to give two base cases:

F_1 = 1, F_2 = 1 ,.

Question to class: Why do we need to specify F_2 ?

Notice that since we have to give base cases for n=1,2 , the recurrence is applied for n geq 3 .

If you think in code (like me), then consider this very simple piece of code to compute the Fibonacci numbers:

Fibonacci Number Code

/* CS majors: Coding up Fibonacci numbers is a common job interview (trap) question */
/* If you are ever challenged to do so, do not write this code below. */
 int F ( int n){
   assert ( n >= 1);
   if (n == 1) return 1;
   if (n == 2) return 1;
   return F(n-1) + F(n-2);
 }

Delete the ’'if’’ statement that handles the base case for n=2 and see what happens.

Coding Question: The code above is a really really inefficient way to compute Fibonacci numbers. Can you say why? Hint Try computing F(10000) and go take a long walk in the woods :-)

A More Complex Recurrence

Let us try a more complex recurrence (more complex than the ones we have seen so far, at least):

$s_n = left{ begin{array}{ll} s_{n-2} + 5 & mbox{if} n mbox{is even}, n geq 3 s_{n-2} + 3 & mbox{if} n mbox{is odd}, n geq 3 end{array} right.$

Base Cases for the Recurrence

What bases cases should be specified for this recurrence?

To answer this, let us ’'run’’ this recurrence for some values of .

Compute s_5 : Since is odd, we use the recurrence for odd numbers:

s_5 = s_3 + 5

Now let us expand s_3 further:

$begin{array}{rcl} s_5 &=& s_3 + 5 &=& underset{s_3}{underbrace{s_1 + 5}} + 5 end{array}$

We have no rule for expanding s_1 , so we have to specify it as a base case.

s_1 = 0

Compute s_6 : Following the definition,

$begin{array}{rcl} s_6 &=& s_4 + 3 &=& underset{s_4}{underbrace{s_2 + 3}} + 3 end{array}$

Again, we have no rule for expanding s_2 . So we have to specify it as a base case:

s_2 = 1

Putting it together

$s_n = left{ begin{array}{ll} s_{n-2} + 5 & mbox{if} n mbox{is even}, n geq 3 s_{n-2} + 3 & mbox{if} n mbox{is odd}, n geq 3 end{array} right. ,,$

with the base cases given by s_1 = 0, s_2 = 1 .

We obtain the sequence:

0, 1, 5, 4, 10, 7, 15, 10, ldots

Looks complicated, but it is actually two sequences ’'zipped’’ together.

Collatz Recurrence

This is one more famous (and crazy!!) recurrence defined as follows:

$s_n = left{ begin{array}{ll} s_{3 n +1} & mbox{if} n mbox{is odd}. s_{frac{n}{2}} & mbox{Otherwise}. end{array} right.$

Base case is: s_1 = 1 .

Claim: s_n = 1 for all n geq 1 .

No one has been able to prove it. To see why let us try an example:

$s_{10} = s_5 = s_{16} = s_8 = s_4 = s_2 = s_1 = 1$

Let us try one more:

$s_{15} = s_{46} = s_{23} = s_{70} = s_{35} = s_{106} = s_{53} = s_{160} = s_{80} = s_{40} = s_{20} = s_{10} = cdots = 1$

Here is the code for printing out the the sequence of indices encountered through the recurrence above:

Collatz's Recurrence

 #include <stdio.h>
 int C (int n) {
   assert ( n >= 1);
   printf ("Collatzing through %d \n", n);
   if (n == 1) return 1;
   /* n is even? */
   if (n %2 == 0) return C(n/2);
   /* n is odd */
   return C( 3 * n + 1);

 }

The claim above can be interpreted as saying that Collatz's recurrence code above always halts and outputs for any integer . Can you try proving that the code above terminates?

Exercise: Try running the code above for C(100) , C(1000) and so on. You will see the sequence go up to really large numbers before coming back down to , magically. At some point, however, an integer overflow on your machine could happen and the result could be meaningless.

Random Number Generator

Here is an interesting fact: sequences defined using recurrence relations are used inside computers to generate pseudo random numbers. The most common sequence used is a Linear Congruential Generator.

Ill Defined (Nonsensical) Recurrences

Note that we have carefully avoided the topic of “well-definedness” of a recurrence. We provide a quick introduction here.

It turns out that not all recurrences or recursive formulae are well defined. A formula defining s_n recursively is well defined if for every it can be used to compute s_n uniquely.

Some of them are plain ’'nonsense’’.

Here are some examples. Can you spot the problems?

1. s_n = s_n , with base case s_1 = 0 . This does not define a unique sequence. Every sequence beginning with a can fit this definition. If you implemented this in code, it will just loop forever.

2. $s_n = s_{lfloor frac{n}{2} rfloor } + s_{2n}$ , with s_1 = 0 . Once again, if you implemented this in code, it will go into an infinite loop or number overflow.

As a good rule of thumb: a recurrence is ill defined if writing it out as a program in CCJava or whatever, causes the code to go into an infinite loop for some number .

Diversionary Note: Finding out if a recurrence is well-defined is a really hard problem (it is equivalent to the halting problem for computers).

Closed Form Representation

Another way of representing sequences is through the closed form. In the closed form representation, we simply express a_n as an expression over for all n geq 1 .

Here is the closed form representation for the sequence of even numbers:

$a_n = 2 n, mbox{for all} n geq 1$

Simple, no fuss.

However, closed form representations are not that easy to get always. Let us try some examples.

Squares

Consider the recurrence for s_n given by

$s_n = s_{n-1} + 2 n -1, mbox{for all} n geq 2,.$

with base case

s_1 = 1,.

Let us try computing some values.

$begin{array}{rclcl} s_3 &=& s_2 + 2 cdot 3 - 1 &=& s_2 + 5 &=& s_1 + 2 cdot 2 -1 + 5 &=& s_1 + 8 &=& 9 end{array}$

s_4 = s_3 + 2 cdot 4 -1 = 9 + 7 = 16 ,.

s_5 = s_4 + 9 = 25

The closed form for this recurrence is

s_n = n^2

In a couple of weeks, we will look at induction proofs of this fact

Fibonacci Numbers

Closed forms are not always ’'simpler’’ than the original recurrences.

Recall the recurrence for Fibonacci numbers:

$F_n = F_{n-1} + F_{n-2}, mbox{for all} n geq 3$

with base case F_2 = F_1 = 1 .

The closed form for the $n^{th}$ Fibonacci number is given by the Binet formula:

$F_n = frac{1}{sqrt{5}} left( left( frac{ 1 + sqrt{5} } {2} right)^n - left( frac{ 1- sqrt{5}}{2} right)^n right) ,.$

You will be asked to prove this fact. In fact, you will find that it is not all that hard once we really get going in this course!!

Here is the cool fact. The number

$phi = left( frac{1 + sqrt{5}}{2} right)$

has a special name. It is called the Golden Ratio. Among other things, the rectangle whose sides are in the golden ratio seems to be the most ’'aesthetically pleasing’’ to the human eye!!

Using this fact, we can write Fibonacci numbers as:

$F_n = frac{1}{sqrt{5}} left( phi^n + frac{1}{phi^n} right) ,.$

Diversionary Note: We can use this formula to write a computer program that is truly efficient for computing F_n . Such a program can compute a F_n for 100 digit in roughly 1000 or so arithmetic operations (if you assume that overflows do not happen). Can you attempt to write it? As a hint, you do not need to know the value of sqrt{5} , to code this up.

Sequences that Cannot be Represented Easily

It needs to be mentioned that there are many sequences that are not easily represented either as a closed form or as a recurrence.

The sequence of prime numbers is one such sequence.

2,3,5,7,11,13,17,19,23,27,31, ldots

We do not have a ’'easy’’ closed form for the $n^{th}$ prime number (there are some really complicated ones that are not worth going into). Same holds for recurrences: the known ones are quite complicated.

Summation

Given a sequence of numbers a_1,a_2,ldots, (defined using closed form or a recurrence), we define the summation of the first terms of the sequence as

$s_n = a_1 + a_2 + a_3 + ldots + a_{n-1} + a_n$

We write this summation succinctly as:

$s_n = sum_{j=1}^n a_j$

The symbol sum is the summation operator. The notation j=1 at the bottom of the summation is the lower limit of the sum and the at the top is the upper limit.

Once again we think in terms of code:

Adding up a Sequence

int SumSequence(int n, int [] A){
  int j;
  int sum = 0;
  for (j=1; j <= n; ++j){
     sum = sum + A[j];
  }
  return sum;
}

If the sequence a_n is given, then the summation of its first terms s_n itself is a sequence given by the following recurrence:

$s_{n} = s_{n-1} + a_n, mbox{for} n geq 2$

with the base case

s_1 = a_1

Take the sequence a_n = n written in the closed form for convenience. What about the summation of the first terms of this sequence?

$s_{n} = s_{n-1} + a_n = s_{n-1} + n, mbox{for} n geq 2$

The base case is s_1 = a_1 = 1 .

The closed form for s_n (the summation) is given by

$s_n = frac{n (n+1)}{2} ,.$

It is really the summation of first numbers. Here is how the year old Gauss supposedly (and famously) reasoned this out:

$begin{array}{r c c c c c c c c c c c c c c } s_n &=& 1 &+& 2 &+& 3 &+& ldots &+& (n-1) &+& n & leftarrow mbox{Original Summation} s_n &=& n &+& (n-1)&+& (n-2) &+& ldots &+& 2 &+& 1 & leftarrow mbox{Rewriting it backwards} hline 2 s_n &=& (n+1) &+& (n+1) & +& (n+1) &+& ldots &+& (n+1) &+& (n+1) & leftarrow mbox{Add the two ways of writing the same summation} end{array}$

The last equation gives us

2 s_n = n ( n + 1)

which gives us the required closed form for s_n .

Observation Getting closed form representations for sequences and summations is often a difficult art in itself. Carl Fredrich Gauss was one of the unrivalled master at this!!

Beyond Number Sequences

We just noticed how sequences of numbers can be represented either by a closed form or by a recurrence. In computer science, we often represent other sequences using such recursive formulae or recurrences. We will keep seeing examples of these throughout this course. The generalization of sequences are called ’'inductive definitions’’ and sometimes they are called ’'grammars’’. Here is a quick primer on grammars. CSCI 3434 (Mathematical Theory of Computation )and the compilers course cover a lot more on this.

Sequences of Letters

Let us assume that there are two letters and . We can define sequences of ’'words’’ over these letters:

B, BAB, BABAB, BABABAB, ldots

The recurrence for such sequences is often called a ’'grammar’’ and written slightly differently (after the work of computer scientists and linguists such as Chomsky, Naur, …).

The recurrence is given by

$a_n = a_{n-1} AB$

which means, that the $n^{th}$ element of the sequence is the $n-1^{th}$ element with the characters AB tacked on to the end. The base case is given by

a_1 = B

Here is a more complex (context-free) recurrence relation:

$b_n = B b_{n-1} B$

with b_1 = A as the base case.

What is the sequence obtained here?

Answer: A, BAB, BBABB, BBBABBB, ldots .

Interestingly, the theory behind formal grammars is the basis for how we write compilers for programming languages. It is one of the most important applications of theoretical computer science or discrete mathematics to computer science.

Simultaneous Recurrences

Not covered in class. For your own interest.

Simultaneous recurrences involve two or more sequences that mutually depend on each other. Perhaps, an example is the best way of explaining.

We define two sequences s_1,ldots,s_n, ldots and simultaneously t_1,ldots,t_m,ldots in terms of each other. Here is an example:

$begin{array}{rcl} s_{n} &=& s_{n-1} + 2 t_{n-1} + 2, t_n &=& -2 s_{n-1} + t_{n-1} + n, end{array},;; mbox{for} n geq 2 ,.$

with base case:

s_1 = 1, t_1 = 2

The code for such a recurrence involves mutual recursive functions.

Simultaneous Recurrence

extern int S( int n); /* Declare S,T in advance */
extern int T( int n);

int S( int n) {
  assert( n >= 1);
  if (n==1) return 1;
  return S(n-1) + 2 * T(n-1) + 2;
}

int T(int n) {
  assert( n >= 1);
  if (n==1) return 2;
  return T(n-1) -2 * S(n-1) + n;
}

The study of these recurrences is beyond the scope of this class. Yet, they produce some very interesting phenomena called chaotic dynamics. It turns out that some of these recurrences (eg., see the Henon map) can have very complex behaviours as the base case for the recurrence is modified. For more, you will have to take the course CSCI 4446 on Chaotic Dynamics or equivalent applied maths/physics class.