\documentstyle[12pt,twoside,psfig,amssymb]{article}
\input{/a/class/6.045/6045-spring-92/handouts/defs-6045}



\newcommand{\defi}{\stackrel{\mbox{\tiny def}}{=}}

\newcommand{\poly}{{\Bbb P}}
\newcommand{\npoly}{{\Bbb NP}}
\newcommand{\Lsp}{{\Bbb L}}
\newcommand{\NL}{{\Bbb NL}}
\newcommand{\pspace}{{\Bbb PSPACE}}
\newcommand{\npspace}{{\Bbb NPSPACE}}

\newcommand{\RP}{{\Bbb RP}}
\newcommand{\nat}{{\Bbb N}}

\newcommand{\fa}{\forall}
\newcommand{\sig}{\Sigma}
\newcommand{\arr}{\rightarrow}
\newcommand{\pless}{\leq_{P}}
\newcommand{\mless}{\leq_m}
\newcommand{\Lless}{\leq_L}

\newcommand{\spa}{\mbox{SPACE}}
\newcommand{\nspa}{\mbox{NSPACE}}
\newcommand{\ttime}{\mbox{TIME}}
\newcommand{\llog}{\mbox{log}}
\newcommand{\sllog}{\mbox{{\tiny log}}}


\begin{document}

\handout{lecture34}{Notes on Lecture 34 (4/29/92)}{May 4, 1992}

We will pursue in this lecture the study of the SPACE complexity 
classes and of the their connections with the TIME complexity classes.

We defined in last lecture $\spa(S(n))$ to be the set of languages computable
by a Turing machine $M$ using at most $O(S(n))$ work space. 
Such
a Turing machine is  called a $S(n)$-space transducer in the literature.
(For simplicity we will often say that the transducer ``uses  $S(n)$'' work space
instead of saying ``uses at most $O(S(n))$'' work space).


\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand57-transducer.ps,height=4cm}}}
\caption{A $S(n)$-space transducer $M_f$ computing $f$}
\end{figure}

Let us emphasize once more that a $S(n)$-space transducer is a Turing machine
having {\em three} tapes: one is the input tape over which the input is written
(as its name indicates!) and which is a {\em read-only} tape. The second
tape is  the $S(n)$-size work tape. You can visualize this tape as being the 
``scratch'' tape where the machine performs all its local computations.
The third tape is the output tape through which the TM let the outside world know
what is the output. There is a subtle point here: we want to think of this
tape as a {\em write-only} tape. If the TM $M$ was for instance allowed to scan this tape 
before appending another output symbol, then $M$ could conceivably save some
work space: indeed, assume that in order to perform its local computations $M$ needs
to ``remember'' what has been the output string so far. If $M$ is not allowed to read
its output tape it must then store this information on its work tape,
whereas if  $M$ were allowed to read
its output tape it could then save (``in an unfair way'')
all this space from its work tape. This is why we require the head of the output
tape to move only towards the right. 

Let's recall  the major results that we derived in the previous class. To begin with
we look at how \spa \ and \ttime\ relate to each other.

Let $M$ be a 
$S(n)$-space transducer. Then the total number of possible different 
configurations of the work-tape is upper-bounded by $2^{cS(n)}$ for some constant
$c$.

This remark allows us to deduce that 
\begin{equation}
\label{eq:space-time}
\spa(S(n))\subseteq\ttime(2^{cS(n)}).
\end{equation}

On the other hand,  it is obvious that a TM can  use only
up to $T(n)$ space in time $T(n)$!
This very simple remark yields the converse result:
\begin{equation}
\label{eq:time-space}
\ttime(T(n))\subseteq \spa(T(n)).
\end{equation}


We then look at how non-determinism affects the power of computation.
Recall that $\nspa(S(n))$ denotes the set of languages  decided by a non 
deterministic $S(n)$-pace transducer.
We then let $\npspace$ denote the set of languages  decided by a {polynomial}-space
{\em non-deterministic}
transducer: $$\npspace \defi \cup_k \nspa(n^k).$$

Savitch's Theorem states that
\begin{equation}
\label{eq:savitch}
\nspa(S(n))\subseteq \spa(S^2(n)),
\end{equation}
which in turn  implies that $$\npspace = \pspace.$$

This last result shows how fundamentally different the time
and space  complexity measures are. Indeed, for time,
the corresponding  result would be $\npoly=\poly$. But, as
we already saw, no one knows so far whether this fact is true or not, even
though most of the theoreticians agree that the answer should be no. 
The question is so hard to analyze that some logicians wonder whether
this question is within the realm of our logic!


\section{The class $\spa(\llog(n))$}
We now introduce and study the class $\Lsp \defi \spa(\llog(n))$.
Note that $2^{c\sllog(n)}= n^c$ so that Equation~\ref{eq:space-time}
shows that $\Lsp\subseteq \poly$. 

Let $\NL$ denote the class $\nspa(\llog(n))$. The results that we recalled 
above allow us to write that
$$\NL = \nspa(\llog(n)) \subseteq \spa(\llog^2(n))  \subseteq
 \ttime(2^{c\sllog^2(n)}) = \ttime(n^{c\sllog(n)}).$$
Note that, as $\llog(n)$ tends to  $\infty$, $\poly$ is strictly included 
into  $\ttime(n^{c\sllog(n)})$ so that the previous argument does not
allow to say that $\NL \subseteq \poly$. Nevertheless a more careful analysis allows
us to derive that actually:

\begin{theorem}
$$\NL \subseteq \poly.$$
\end{theorem}

\begin{proof}
Let $L$ be a language in $\NL$ and let $M$ denote
a $S(n)$-space non-deterministic transducer.
Note that the notions of determinism or non-determinism
affect the way we move from a configuration to another 
but { not} the {\em number} of configurations. Hence
for the non-deterministic machine $M$ it is also true that  
the total number of possible different 
configurations of the work-tape is upper-bounded by $2^{cS(n)} = n^c$.

We then define the following directed graph $G=(V,E)$. The set of vertices is 
the set of configurations (of the work tape of $M$). The edge $ij$ is
in $E$ iff configuration $j$ is reachable in one step from configuration $i$.
Without loss of generality we can assume that there is only one final configuration $t$
and we will let $s_x$ denote the initial configuration corresponding to input $x$.
In these terms $x\in L$ iff there is a path
from $s_x$ to $t$. We describe a polynomial time depth-first-search algorithm testing for
the existence of such a path. 


In order to  implement the algorithm we  maintain the following data structures:
a list ``{\tt Seen}'' 
 describing the set of nodes
encountered so far in the algorithm,  a stack 
 allowing
to move through the graph  in a depth-first fashion, 
and for each node $a$ the list {\tt Marked-successors}$(a)$ of the successors $b$ of $a$ 
(i.e., of the edges $ab$) already visited.
The algorithm is presented in Figure~\ref{fig:DFS-alg}.


\begin{figure}[ht]
\begin{center} 
\fbox{
\begin{minipage}{\textwidth}
{\tt Seen} $\leftarrow \{s_x\}$\\
{\tt Stack} $\leftarrow s_x$\\
return {\tt DFS}$(s_x)$

\begin{tabbing}
{\tt DFS}$(a)$\hspace{17mm}: \ \= {\bf if} $a=t$ return {\tt true} and stop\\
(The main loop) \> {\bf while} {\tt Marked-successors}$(a) \not =$  {\tt Successors}$(a)$ do\\
		\> \hspace*{5mm} \= choose  $b \in $ {\tt Successors}$(a) - $ {\tt Marked-successors}$(a)$ \\
		\>     \>    {\tt Marked-successors}$(a) 
		                      \leftarrow$ {\tt Marked-successors}$(a)\ \cup \{b\}$ \\
		\>     \> {\bf if} $b \not \in$  {\tt Seen} do\\
		\>     \>  \hspace*{5mm} \=  {\tt Seen} $ \leftarrow$ {\tt Seen} $\cup\ \{b\}$ \\
		\>     \>               \>  Put $b$ on top of {\tt Stack}\\
		\>     \>               \>  {\tt DFS}$(b)$ \\
		\> remove top element from {\tt Stack}\\
		\> {\bf if} {\tt Stack} $= \emptyset$ return {\tt false} and stop
\end{tabbing}
\end{minipage}
}

\caption{DFS-algorithm}
\end{center}
\label{fig:DFS-alg}
\end{figure}


Note  that 
in each pass of the main loop, a vertex is removed from the top of the stack 
{\em or} a new edge added to a {\tt Marked-successors} list.
Note also that each vertex is (added to and)  removed at most once from
the stack (exactly once if the graph is connected!) and that similarly
each edge is marked at most once. Hence the number of passes
through the main loop is at most $|V| + |E|$ and hence linear. 
Each of these passes involves
searching through linear size data structures. Hence the total time of the algorithm
is quadratic (and therefore polynomial!).
\end{proof}

Our knowledge of the correspondence between the different complexity classes is summarized in
Figure~\ref{fig:complxty-classes}.


\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand57-hierarchy.ps,height=5cm}}}
\caption{The relation among the different complexity classes}
\label{fig:complxty-classes}
\end{figure}

All these classes are conjectured to be  different, but at this point it is 
not even know whether $\Lsp = \npoly$!!

\section{Log-space Reductions}
The following notion is the space analogue of polynomial-time reduction.
\begin{definition}
\label{def:L-reduction}
$A \Lless B$ if there is a $\llog(n)$-space transducer $f$ such that
$x\in A$ iff $f(x)\in B$.
\end{definition}

\begin{theorem}
\label{th:L-implies-P}
$A \Lless B \Rightarrow A \pless B.$
\end{theorem}

\begin{proof}
$\spa(\llog(n))\subseteq \ttime(2^{c\llog(n)})$.
\end{proof}

\begin{theorem}
$[A \Lless B \mbox{ and } B \Lless C] \Rightarrow A \Lless C.$
\end{theorem}

\begin{proof}
Let $M_f$ and $M_f$ denote the transducer computing
the functions $f$ and $g$ through which $A$ is reduced to $B$ and $B$ reduced
to $C$.

It seems at first that the result is trivial and it actually requires a
good understanding of the situation to realize where the problem is!
In effect the ``natural'' simulation described by Figure~\ref{fig:transtivty-false}
seems to solve the question. In this simulation we run the
transducers $M_f$ and $M_g$ in series: $M_f$ computes $f(x)$
which  then becomes as the input of $M_g$.


\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand57-transitivity.ps,height=5cm}}}
\caption{A  wrong solution to the transitivity problem of log-space reduction}
\label{fig:transtivty-false}
\end{figure}

In this solution  the three middle tape are thought of as
being the work tape of the composed transducer.
The problem with this is that this work tape could very well be 
of size bigger then logarithmic (if the output $f(x)$ is too-big).

The idea is to feed the output of $M_f$  directly into $M_g$, {\em one symbol at a
time}. This works as long as long as $M_g$ moves right on its input. 
Should $M_G$ move left, $M_f$ must be restarted to determine the input symbol
for $M_g$, since the input of $M_f$ is {\em not} saved. We will let $M'$ denote the
transducer thus built.

Some care has to be devoted into the book-keeping. As $M_f$ works in logarithmic space
we can also assume that it works in polynomial time $n^c$ (see Equation~\ref{eq:time-space}).
Hence the output of $M_f$ has also size at most $n^c$. Remark that $O(\llog(n))$ bits are sufficient
to represent (in base 2) any number less then $n^c$. Assume that in the previous simulation
$M'$ needs to move the input head of $M_g$ to the left from position $i$ position $i-1$.
To achieve this $M'$ just needs to write the number  $i-1$ on its work tape (this takes at most
 log-space)
 and then simulates $M_g$ until $M_g$ outputs its $i-1$-th symbol: this is indeed
equivalent (modulo the waste of time...but time is of
no concern here!) to moving the input head of $M_g$ to the left.
\end{proof}

The same proof gives us the following result:

\begin{theorem}
\label{th:L-reducibility}
$[ A \Lless B \mbox{ and } B\in \Lsp ] \Rightarrow A \in \Lsp.$
\end{theorem}


\begin{definition}
\label{def:space-comlpete}
$B$ is $\Lless$-complete for $\poly$ iff $B\in \poly$ and if $A\Lless B$ for all $A\in \poly$.
\end{definition}

\begin{theorem}
\label{th:L-complete}
[$B$ is $\Lless$-complete for $\poly$ and $B\in \Lsp$] $\Rightarrow \poly = \Lsp.$
\end{theorem}
\begin{proof}
Theorem \ref{th:L-reducibility}  tells us that $\poly\subseteq \Lsp$. On the
other hand we know that  $\Lsp\subseteq \poly$ is true.
\end{proof}

This result is to be interpreted as saying that any language $B$
$\Lless$-complete for $\poly$ is unlikely to be in $\Lsp$.

We now provide the example of a language that is
$\Lless$-complete for $\poly$.


\begin{definition}
\label{def:CVP}
$\mbox{CVP}\defi\{\mbox{circuits that evaluate to 1}\}.$
\end{definition}

The circuit $C$ shown in Figure~\ref{fig:cuircuit-1} evaluates to 1.


\begin{figure}[ht]
\centerline{\mbox{\psfig{figure=hand57-CVP-ex.ps,height=3cm}}}
\caption{A  cuircuit $C$ evaluating to 1}
\label{fig:cuircuit-1}
\end{figure}


\begin{theorem}
\label{th:CVP-is-L-complete}
$\mbox{CVP is } \Lsp-\mbox{complete for } \poly.$
\end{theorem}
\begin{proof}
The proof is very similar to the one that we gave 
in Handout~48 (or covered in the book on page 437) and showing that
SAT is $\npoly$-complete, except that here we deal with 
a  deterministic computation whereas 
there we were considering nondeterministic computations.


Let $B$ be a language in $\poly$ recognized by a TM $M_B$.
As in the proof of Cook's theorem we can build  a cuircuit 
that outputs 1 {\em iff}
$x \in B$. We introduce the boolean   variables 
\begin{itemize}
\item $H_{i,t}$: $H_{i,t} =1$ exactly when the head is in position $i$
    at time $t$.
\item $S_{i,j,t}$: $S_{i,j,t} =1$ exactly when the symbol on square $i$
	at time $t$ is $j$. 
\item $Q_{k,t}$: $Q_{k,t} =1$ exactly when the state at time $t$ is $k$.
\end{itemize}

We claim that the construction of this cuircuit can be done in logarithmic
space and also that the circuit can be evaluated in logarithmic space.
For instance we write on the output tape:
$H_{i,t}\equiv (H_{i,t-1} \wedge Q_{k,t-1} \wedge S_{i,j,t-1}) \vee \ldots$
(Recall that $A\equiv B$ can be written as 
  $(B \wedge \neg A) \vee (A \wedge \neg B)$.) Writing this can be done in
constant space, except that $i$ can grow up to $t=p(n)$:
we need to keep a counter incrementing $i$ up
to $t=p(n)$, and this can be done in log-space (because $\llog(p(n))= O(\llog(n))$).
\end{proof}

\section{Other complete problems}
\begin{itemize}
\item {\bf MCVP} (Monotone Circuit Value Problem): this is the same as CVP except that
no $\neg$ is allowed.
\item {\bf Linear programming}. This problem is  genuinely very important in applications:
most scheduling problem encountered for instance by airline companies
 involve solving linear programming problems of huge size. The problem is described as  follows:

{\bf Input}: 
\begin{tabbing}
\hspace*{5mm}\= An $m\times n$ matrix $A$ \\
  	\> an $n$-vector $C$ of integers\\
	\> an $m$-vector $y$ of integers\\
	\> an integer $B$
\end{tabbing}        

{\bf Decide}: Does there exist an $n$-vector $x$ of rationals such that
$$\begin{array}{rll}
 & Ax \leq y & \ \mbox{component-wise}\\
\mbox{and } & C\cdot x \geq B & \mbox{ (here $\cdot$ denotes the dot-product)\ ?}
\end{array}
$$
\end{itemize}  

\end{document}



