\documentstyle[12pt,twoside]{article}
\input{defs-6045}

\begin{document}

\handout{lecture25}{Notes on Lecture 25 (4/6/92)}{April 8, 1992}

We now begin the last chapter of the class: Complexity Theory. So far
in the class we did not concern ourselves with the issue of performance:
languages were computable or not, and we disregarded completely
the various levels of difficulty involved in different problems.
But there are problems which are theoretically computable (in the sense
of the first part of this class) but for which all the atoms of our galaxy
working in a coordinate way could not reach the solution in a million years!
We now develop a theory allowing us to quantify these intrinsic different
complexity levels of computation.

Most of the results presented in class are contained in the ones of Handout~33
so that the minutes covering the topics covered in class will be more sketchy
(and the TAs happier!). Today's lecture covers the first 4 pages of this handout.


\section{Complexity Theory}
For the rest of the term we will restrict ourselves to
recursive sets. Complexity theory studies the costs involved in computations
and classifies the different computation problems by their difficulty.
The cost measures can be of different nature: the {\em time} required
in the computation, the {\em space} required by this computation (when performed
for instance by a Turing machine), the {\em communication} required when this computation
is performed by different ``communicating processes'', the amount of {\em random inputs}
that has to be used when the language is ``explored randomly'', the number of
{\em logical gates} that have to be used when the language is recognized by some
special classes of circuits... All these (so far loosely described) 
measures are useful to investigate
the intrinsic difficulty of computations. We just quoted a few of them here 
to make you feel the ``complexity'' of complexity theory. We will confine ourselves
here mostly to the {\em time} measure and will consider space measures only shortly 
by the end of the class. 


We provide some examples to introduce and illustrate  the discussion carried
out in  Handout~33.

\begin{newexamp}[Regular sets are in TIME($O(n)$)]
Let $L$ be a regular set. Consider a TM $M$ that 
simulates in some sense a DFA deciding $L$: it scans the input
(this takes time $n$), and then moves the head back to its original position...
\end{newexamp}

\begin{newexamp}
Consider $L=\{w\in \{a,b,c\}^* : N_a(w) = N_b(w) = N_c(w)\}$.
A TM deciding this language is one that scans across, looking first
for an $a$, a $b$ and a $c$, changing them into $d$'s. It then scans back and iterates
until it cannot do that anymore. If only $d$'s are left it then decides
``yes'' and else it decides ``no''. How long does this machine work
for? Well, each scan takes time $n+k$ (for some constant $k$ that depends
only  on the little overhead required to initiate and finish the scan...)
The number of scans before termination is at most  $n/3$. Hence
$M$ accepts $L$ in time $T$ where 
$$T(n) = (n+k)\cdot n/3 \sim 1/3 n^2.$$
\end{newexamp}

Note that the equivalence $\sim$ above has a precise meaning: it means
very precisely that the relative error $\frac{T(n) - n/2}{T(n)}$ 
is asymptotically zero i.e. that $\frac{T(n) - 1/3 n^2}{T(n)}$ tends to zero
as $n$ grows to infinity. The significance of this definition is that,
up to a negligible correction, $T(n)$ is equal to $1/3 n^2$. This means
that we simplify $T(n)$ so as to retain only its meaningful asymptotic
expression. 

Note also that even if we wanted to, computing the exact value $T(n)$
might be very hard. What we are most of the time after is 
a tractable  {\em upper bound} of the time of computation:
For all input of size $n$ we want to make sure that by time $T(n)$
(and possibly before!) the computation has ended. Sometimes we can find
an upperbound by considering some procedure. Optimizing this procedure
might then give a better (i.e. a smaller) bound. The following example is an illustration 
of this fact.

\begin{newexamp}[Optimization of the previous example]
In the previous example we were implicitly working with
an unsophisticated one tape Turing machine, of the type
that you see in any hardware store. What about using
the latest four tape TM? On the first tape we simply scan
the input, recording (in unary) on the second tape  the number of $a$'s
encountered, and recording on the third and on the fourth the number of $b$'s and $c$'s.
We then compare the three recording tapes by simultaneously
moving the three heads back to the left. The time spent in this
simulation is  linear, which is an improvement with respect 
to the quadratic cost that we had derived in the previous paragraph.
\end{newexamp}

Improvements of the previous type are the raison d' \^{e}tre of all theoreticians
and require subtle analysis to get better algorithms.

\begin{newexamp}
COMPOSITE and UCOMPOSITE are analyzed in Handout 33. Here are some quick
explanations. 

A number $N$ expressible with $n$ bits (in base 2, say) is at
most equal to $2^n$.  It is composite iff there exists some integer less then
$\sqrt{N}= 2^{n/2}$ which divides it. On the other hand a division between
two $n$ digits numbers takes at most time $n^2$.
This explains the upper bound of $O(n^2 2^{n/2})$ when the input $N$
is expressed in $k$-ary for $k\geq 2$.

But if the input $N$ is written in unary its length becomes $N$\ !
An algorithm checking the compositness of $N$ would then proceed as follows.
First convert $N$ in base 2: this takes time $O(N)$. Then run the previous
algorithm on the base~2 number derived. This takes, as we saw, time
at most $O(n^2 2^{n/2})$. But, in terms of $N$ (recall that 
$N$ {\em is} our current input length!),
this is $O({\mbox{log} N}^2 \sqrt{N})$. We then obtain a total
time equal to $O(N + {\mbox{log} N}^2 \sqrt{N}) \sim O(N)$ which seems
much better!! Of course there are no miracles and this is
not truly better. This is just an artificial improvement due
to the convention used in the representation of the input.
(Artificial improvements are well known to politicians quoting
statistics!)

\end{newexamp}

The linear speedup theorem is presented in Handout~33. The idea behind
its proof is to begin with a machine $M$, and to derive another machine $M'$ using
more tape symbols, allowing to condense several squares of the input tape
 into one. This compressed encoding will permit simulating
$m$ steps of the original machine $M$ with a fixed small number of steps of the new machine $M'$.
Let's analyze the time the whole procedure takes:

{\em Initialization:} The input must be read and compressed. This takes $O(n)$ time.

{\em Simulation:} One step of $M'$ simulates $\Omega(m)$ steps of $M$.

{\em Termination:} The output is decoded back from its compressed version into the
original one.

For the curious minds we attach here a precise description of the procedure
as described in Lewis and Papadimitriou's book. (But the reading of this
is {\em only} optional !)











\end{document}
