Math 480, Fall 2016, Assignment 5

I was at the mathematical school, where the master taught his pupils after a method scarce imaginable to us in Europe. The proposition and demonstration were fairly written on a thin wafer, with ink composed of a cephalic tincture. This the student was to swallow upon a fasting stomach, and for three days following eat nothing but bread and water. As the wafer digested the tincture mounted to the brain, bearing the proposition along with it.

- Jonathan Swift, Gulliver's Travels

Carefully define the following terms, then give one example and one non-example of each:

1. Memoryless channel.
2. Source.
3. Memoryless source.
4. Stationary source.
5. Character frequencies (of a source).
6. Information rate (of a memoryless, stationary source paired with a memoryless channel).
7. Channel capacity.

Carefully state the following theorems (you do not need to prove them):

1. Formula for output frequencies (in terms of input frequencies and the transition matrix).
2. Capacity equation (giving optimal input frequencies for a given channel, and hence the channel capacity; see Theorem 3.4.3).

Solve the following problems:

1. Section 3.2, problem 1.
2. Section 3.3, problem 3.
3. Section 3.4, problems 3, 5, and 8.

--------------------End of assignment--------------------