\documentstyle[11pt,twoside,psfig]{article}
\input{defs-6045}

\begin{document}

\handout{lecture26}{Notes on Lecture 26 (4/8/92)}{April 13, 1992}

These lecture notes should be read in conjunction with Handout~33.

{\bf erratum:} There is a  little slip in Handout~36. In example~2
we actually proved that $\frac{T(n) - 1/3 n^2}{T(n)}$ 
(and not  $\frac{T(n) - n/2}{T(n)}$) tends to 0.



\section{Big $O$, little $o$ and $\Omega$ notations}

As we noticed last time, computing {\em exact} bounds can be very hard.
Also, as we noted with the linear-speedup theorem, the constants (appearing in the 
evaluation of the time required to solve a problem) are meaningless anyway.
These considerations justify the following definitions

\begin{definition}
\label{def:bigO} Let $f$ and $g$ be functions from $\Nat$ to $\Nat$.
\begin{enumerate}
\item We say that $f$ is a $O(g)$ (read ``$f$ is a big-oh of $g$'') 
and will then write $f = O(g)$ if there is a
constant $c>0$ and an integer $n_0$ such that $f(n) \leq c \cdot g(n)
\hbox{ for all } n \geq n_0$.
\item We say that $f$ is an $\Omega(g)$ 
and will then write $f =\Omega(g)$
if there is a constant $c > 0$
and an integer $n_0$ such that
$f(n) \geq c \cdot g(n) \hbox{ for all } n \geq n_0$.
\item We say that $f$ is a $o(g)$ (read ``$f$ is al little-oh of $g$'') 
and will then write $f = o(g)$ if
for all $\epsilon \geq 0$ there is an integer $N(\epsilon)$ such that
 $f(n) \leq \epsilon \cdot g(n)$  for all  $n\geq N(\epsilon)$.
\item We say that $f \sim g$ (read ``$f$ is equivalent to $g$'') if
$f-g$ is a $o(g)$. 
\end{enumerate}
\end{definition}

A few important explanations:
\begin{itemize}
\item $f = O(g)$ means that $f$ is asymptotically {\em upper}
bounded (up to a multiplicative constant) by $g$.
\item Note that $f = O(g)$ is equivalent to $g = \Omega(f)$.
\item $f = o(g)$ means that $f$ is asymptotically negligible 
with respect to $g$. Provided that $g$ is not zero for $n$ 
sufficiently big this is equivalent to $f/g$ tending to 0
as $n$ tends to $\infty$. 
\item $f \sim g$ means that $f$ is asymptotically equivalent to
$g$. Provided that $g$ is not zero for $n$ 
sufficiently big, this is equivalent to $f/g$ tending to 1
as $n$ tends to $\infty$. 
\end{itemize}

\begin{newexamp}
\begin{itemize}
\item
$25 n^2 +2n + 6 = O(n^2)$. Actually 
$25 n^2 +2n + 6  \sim n^2$. 
\item
$n^2 = O(n^3)$. Actually $n^2 = o(n^3)$. 
\end{itemize}
\end{newexamp}

Note that $f$ is $O(g)$ if and only if $g$ is $\Omega(f)$.
Some other important facts about $O$, $o$, $\Omega$ and $\sim$ are:

\begin{itemize}
\item $\sim$ is an equivalence relation.
\item $ f = O(g)$ and $ f = o(g)$ are  transitive relations.
\item $f \sim g \Rightarrow f =O(g)$. This
explains the  ``actually'' that we used in the first previous example
\item Also $f =o(g) \Rightarrow f =O(g)$. (The converse is wrong: if
$f$ is (asymptotically) negligible with respect to $g$,
then it
is (asymptotically) bounded by $g$. But is not true that if $f$
is (asymptotically) bounded by $g$ then it 
is (asymptotically) negligible with respect to $g$!)

\item if $p_1$ and $p_2$ are univariate
polynomials with positive leading coefficients and are
of degree $d_1$ and $d_2$, respectively,
then $p_1 = O(p_2)$ if and only if $d_1 \leq d_2$,

\item $n^\epsilon = O(n^\delta)$ if and only if $\epsilon
\leq \delta$.

\item  $p(n)= O(c^{n^\epsilon})$, for all
$c>1, \epsilon>0$, and polynomials $p$.

\item  $\lceil \log_a(n) \rceil=
O(\lceil \log_b(n) \rceil)$
for all $a,b>1$,

\item  $p(\log n)=O(n)$ for any polynomial $p$.   
\end{itemize}

We now re-express the examples seen in Handout~36 in
the precise terms of our new terminology

\begin{newexamp}
\begin{itemize}

\item Any regular set can be decided in linear time by some Turing machine.
We rewrite this as:
$\{$Regular sets$\} \subseteq $  TIME($O(n)$)
\item Let $L=\{w\in \{a,b,c\}^* : N_a(w) = N_b(w) = N_c(w)\}$.
Then $L \in $  TIME($O(n^2)$). 
\item We then refined our analysis and got that actually
Then $L \in $  TIME($O(n)$). 
\item COMPOSITE $\in $  TIME( $O(n^2 2^{n/2} )$).

\end{itemize}
\end{newexamp}

\section{Analysis of the Running Time according to the Model of Computation}
The problem that we address here is to understand how the time bounds 
vary when we add a few belts and whistles to our Turing machine: for
instance what is the speedup that we get by considering a multitape
Turing machine instead of a good classical one tape machine?

\begin{newexamp} ({\bf 2}-way infinite, 1 tape, 1 head TM versus 
		{\bf 1}-way infinite, 1 tape, 1 head TM)\\

The last model is the one that we used mostly for the computability section
of the course.

\begin{claim}
If $L$ is decided by a 1-head 1-tape (in short 1H1T) Turing machine with a 2 
way infinite tape in time $T(n)$, then it can also be decided by 
a 1H1T with a 1 way infinite tape Turing machine in time $O(T(n))$. (Provided 
that $T(n) = \Omega(n)$.)
\end{claim}

\begin{proof} We analyze more carefully the same proof that the one that we gave
in the computability part of the course...
Actually, the construction involved a pre-pass to re-code the input as 
shown in Figure~\ref{fig:encode-2way-tape}


\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand40-fig1.ps,height=5cm}}}
\caption{The input of the 1-way tape is re-encoded in its 2 way version}
\label{fig:encode-2way-tape}
\end{figure}

The time spent in the simulation of 2-way infinite TM is then 
equal (up to a multiplicative factor) to the time spent
in the simulation with the one way tape TM + the time spent in
the pre-pass. But this one is an $O(T(n))$ provided
that  $O(T(n)) = \Omega(n)$. We will constantly assume this 
implicitly in the sequel, and will most of the time forget
to recall this assumption. (Basically this assumption means that
the TM has at least to read the input before halting. This
a very reasonable assumption which is contradicted
only in  exceptional instances, as for instance
the cases of
$L = \emptyset$ or $L= \{ x : |x| \leq 100\}$ where
the input does not need to be read, or need to be scanned up to only
its first 101 characters.)
\end{proof}
\end{newexamp} 


\begin{newexamp} ({\bf multi-tape} TM versus 
             {\bf multi-head, 1 tape} TM versus {\bf 1 head, 1 tape} TM)\\

\begin{claim}
If $L$ is decided by a $k$-tape  (in short $k$T) Turing machine,
 then it can also be decided by 
a 1H1T  Turing machine in time $O(T^2(n))$. (Provided 
that $T(n) = \Omega(n)$) 
\end{claim}

\begin{proof}
\begin{itemize}

\item (From multi-tape to 1-tape multi-head TM's):\\
Multi-tapes ($k$T) Turing machines are more powerful then
multi-head one tape  TM so that the running
time on the later is bigger then the one on the former.

On the other hand notice that we can very code simply several tapes into
one with different ``tracks'', as shown in
Figure~\ref{fig:encode-multitape}.

\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand40-fig2.ps,height=3cm}}}
\caption{How to encode a multitape TM with a 1-tape TM}
\label{fig:encode-multitape}
\end{figure}

Hence  multitape TM's and 1-tape multihead TM's have
the ``same'' running time (a precise formulation of ``same''
is they have an {\em equivalent} running time 
(recall that the notion of equivalence 
is defined  in Definition~\ref{def:bigO})).


\item (From  1-tape multi-head to 1-tape 1-head  TM's)\\
Recall that the simulation of a
1-tape $k$-head by 1-tape 1-head  TM consists of scanning all
the work tape from left to right, then extending (by at most $k$!) if
necessary the length of (the significant part of) the work tape
and  making all other  changes on a second sweep of the head back to 
the leftmost position. Hence the simulation of the $t$th step of the
multihead TM requires $O(T(n))$  steps. (Because, by the $t$th step, the input
can be of length  at most $n + tk \leq (k+1) T(n)$.) Hence the simulation
of the $T(n)$ steps of the total execution require at most time $O(T^2(n))$.

\end{itemize}
\end{proof}

\end{newexamp}

From now on in this course we will just consider the more flexible multitape
Turing Machines as our model of computation. This gives nice properties as 
we can simulate for instance $k$ computations in {\em parallel} by
just considering a TM with $k$ tapes.

The rest  of this class is covered in the notes of Handout~33 (up to page~5).
\end{document}

