\documentstyle[12pt,twoside,psfig,amssymb]{article}
\input{/a/class/6.045/6045-spring-92/handouts/defs-6045}






\newcommand{\poly}{{\Bbb P}}
\newcommand{\Np}{{\Bbb NP}}
\newcommand{\RP}{{\Bbb RP}}
\newcommand{\nat}{{\Bbb N}}
\newcommand{\fa}{\forall}
\newcommand{\sig}{\Sigma}
\newcommand{\arr}{\rightarrow}
\newcommand{\pless}{\leq_{P}}
\newcommand{\mless}{\leq_m}


\begin{document}

\handout{lecture33}{Notes on Lecture 33 (4/27/92)}{May 4, 1992}

\section*{Coping with NP-completeness}

Just because a problem is NP-complete doesn't make it go away.  What
do people do with NP-complete problems in practice?  The people who
want to solve these kinds of problems in practice tend not to think
about {\it decision problems} (e.g.\ does graph $G$ have a vertex
cover of size $\leq k$?) so much as {\it optimization problems} (e.g.\
what is the smallest vertex cover in this graph?).  Given this fact,
practitioners are often satisfied with {\it approximation algorithms}:
with polynomial-time algorithms that are guaranteed to produce
near-optimal solutions (e.g.\ a vertex cover that is guaranteed not to
be much larger than the smallest possible).
 
As an example, here is an approximation algorithm for VC:
\begin{itemize}
\item[] While there exists some edge in the graph:
\item[] ~~~~~Pick an edge
\item[] ~~~~~Add both endpoints to the vertex cover
\item[] ~~~~~Remove all edges in the graph covered by these endpoints
\end{itemize}
\noindent We claim that this algorithm yields a vertex cover that is
no larger that twice the optimal (smallest possible).  To see this,
notice first that the edges chosen by the algorithm will not be
adjacent to each other.  In the minimum vertex cover each of these
edges must have one of its endpoints covered; since these edges are
not adjacent, each of the edges must be covered by a different vertex
from the vertex cover.  Hence if the algorithm chose $k$ edges, there
must be at least $k$ vertices in the minimum vertex cover.  The
algorithm constructs a vertex cover of size $2k$, so that the vertex
cover it constructs is no more than twice as large as the smallest
possible.

\section*{Space complexity}

We now turn from our study of time complexity to a study of space
complexity.  Time is not the only real constraint on computing.
Memory is, too (RAM, disk space, etc.)  Hence we will look at
computing problems in terms of the amount of space (memory) that they
use.  In addition to being an interesting complexity problem, space
also turns out to have an important connection to parallel
computation.  We will see in later lectures that the amount of
sequential space a problem needs is roughly equal to the amount of
parallel time it needs.

To study space complexity, we change the model that we study slightly
so that we can consider sublinear space bounds.  Our model of a space
bounded Turing Machine will have three tapes.  The first tape will be
a finite-size, read-only input tape.  The third tape will be a
write-only output tape set up so that the tape head can only move one
way.  The second tape will be a read/write work tape.  The idea is
that only the information written to the second tape will count as
usage of space.  This way the size of the input does not affect the
space bound.  More formally, we say that $L \in SPACE(S(n))$ if there
exists a TM $M$ such that for all inputs $x$, $M$ decides where $x \in
L$ or not in space $S(n)$ (where $n = |x|$); that is, the head on the
work tape never moves to the right of the $S(n)$th square.  

Here are some examples of space bounds.  Every regular language can be
decided in $SPACE(0)$, since we can recognize it with a TM that
doesn't need to write anything on the work tape.  The balanced
parentheses language can be recognized in $SPACE(\log_2 n)$.  To do
this, we build a TM that keeps a count, adding 1 to the count for open
parens, and subtracting 1 from the count for close parens.  The TM
makes sure that the count is always non-negative, and that the count
is zero at the end of the input.  The TM only uses $\log_2 n$ space
since the count can never get larger than $n$, and it only takes
$\log_2 n$ bits to represent $n$ in binary.

There are lots of interesting problems in computer science that are in
fact in $SPACE(\log_2 n)$.  There are also some problems that don't
seem to be in this class, though we haven't been able to prove it.
One of these problems is known as the Circuit Value Problem.  In this
problem we are given a description of a circuit (as AND, OR, and NOT
gates) and values for the input wires, and we must determine the value
of the output wires.  The intuitive reason that this problem is not in
log-space is that we need to store the intermediate results of each
gate somewhere, and this needs more than log-space.  However, no one
has been able to prove that there isn't some more clever way to solve
the problem that doesn't need all the intermediate storage.

\section*{Time vs. Space}

We now look at the relationship between the $TIME$ and $SPACE$
complexity classes.  To do this, it first proves useful to look at the
machine configurations of a space-bounded TM.  To describe the state
of a space-bounded TM at any given point in time, we need to describe:
\begin{itemize}
\item the state of the Finite State Control;
\item the head position on the input and work tapes;
\item the work tape contents;
\item the output tape contents (which can only be $\Delta$, 0, or 1
for language decision problems).
\end{itemize}
How many of these configurations can there be for an $S(n)$
space-bounded machine?  There are:
\begin{itemize}
\item $|Q|$ possible states;
\item $(n+2)$ possible head positions on the input tape;
\item $S(n)$ possible head positions on the work tape;
\item $|\Gamma|^{S(n)}$ possible work tape contents;
\item 3 possible output tape contents.
\end{itemize}
The TM always halts (with output 1 or 0), so it never loops.
Therefore it must never repeat a configuration, otherwise it would go
into an infinite loop.  Hence the total number of steps the TM can
take is bounded by the total number of possible configurations, $|Q|
\cdot (n+2) \cdot S(n) \cdot |\Gamma|^{S(n)} \cdot 3$.  When $S(n)
\geq \log(n+2)$, this product can be bounded above by $2^{cS(n)}$ for
some constant $c > 1$.  Hence we have the following theorem.

\begin{theorem}
When $S(n) \geq \log (n+2)$, then $SPACE(S(n)) \subseteq
TIME(2^{cS(n)})$ for some constant $c > 1$.
\end{theorem}

\section*{Nondeterministic Space}

We can also think about nondeterministic space-bounded machines.  A
nondeterministic TM has space bound $S(n)$ if no possible sequence of
nondeterministic moves causes the TM to use more than $S(n)$ space on
its work tape.  Although nondeterministic time seems to be quite
different from deterministic time, this does not seem to be true of
non-deterministic and deterministic space.  In fact, we have the
following theorem.

\begin{theorem}[Savitch]
$NSPACE(S(n)) \subseteq SPACE(S^2(n))$.
\end{theorem}

\begin{proof}
We will again use machine configurations to prove this.  Without loss
of generality, we will assume that there is a unique accepting
configuration.  That is, we will say that a TM has accepted input $x$
only if the work tape is filled with blanks and all three heads are on
the leftmost square of their respective tapes.  If there is a TM that
halts in some other configuration, we can modify it to clean up the
work tape and move all its heads to the leftmost tape squares.

We now write a program $Q(a,b,2^k)$ that decides the following: if the
machine is started in configuration $a$, is there some sequence of
no more than $2^k$ nondeterministic moves such that the machine
arrives in configuration $b$?  Given this program, we could answer the
question of whether the TM accepts an input by running the program
$Q(Start, Accept, 2^{cS(n)})$, where $Start$ is the starting
configuration and $Accept$ is the unique accepting configuration.  We
know from previous discussion that the machine must halt within
$2^{cS(n)}$ steps.

Here is the program $Q(a,b,2^k)$:
\begin{itemize}
\item[] If $k = 0$
\item[] ~~~~If $a = b$ or $b$ is a possible successor configuration of
$a$ then return True
\item[] ~~~~else return False
\item[] For all configurations $m$
\item[] ~~~~If $Q(a,m,2^{k-1})$ and $Q(m,b,2^{k-1})$ are True then return True
\item[] ~~~~else return False
\end{itemize}
Note first that it only takes $cS(n)$ space to describe a
configuration: because there are at most $2^{cS(n)}$ configurations
total, we can assign each configuration a number, and writing down the
number will take at most $\log_2 2^{cS(n)} = cS(n)$ bits.  How much
space does the program need?  It turns out that it only needs space to
write down the current configuration $m$ under consideration in the
main loop, plus whatever space is required for one recursive call.  So
the total space required will be the space taken for one configuration
times the depth of the recursion (i.e.\ the number of recursive calls
required to reach the base case of the program).  Since $k = cS(n)$
for our top-level call of the program and $k$ decreases by 1 for each
recursive call, the depth of the recursion is $cS(n)$.  Hence the
total space needed is $cS(n) \cdot cS(n) = O(S^2(n))$.
\end{proof}

\end{document}



