\documentstyle[12pt,twoside,psfig,amssymb]{article}
\input{defs-6045}


\newcommand{\poly}{{\Bbb P}}
\newcommand{\RP}{{\Bbb RP}}
\newcommand{\nat}{{\Bbb N}}
\newcommand{\fa}{\forall}
\newcommand{\sig}{\Sigma}
\newcommand{\arr}{\rightarrow}
\newcommand{\pless}{\leq_{\poly}}


\begin{document}

\handout{lecture29}{Notes on Lecture 29 (4/15/92)}{April 22, 1992}



In this lecture we will finish the study of the material introduced in 
 Handout~33: these lecture notes are  designed to make easier the reading
of Handout~33. 

We will introduce and justify the existence of a complexity 
hierarchy. We will then give the example of a ``natural'' language that
is not in $\poly$. By natural we mean that it might seem that the halting problem 
$K_1^T$ that we introduced is a bit artificial, or at least far fetched.
The example NEREGSQ that we introduce here deals with regular expressions,
and shows that hard problems (i.e., outside of the realm of polynomial time
algorithms) are indeed part of our world.



\section{The polynomial hierarchy}
We last time introduced the general time self halting problem

$$K_1^T := \{  M : \hbox{ Turing machine } M \hbox{ accepts }
e(M) \hbox{ in } \leq T(|x|) \hbox{ steps}\},$$
and showed that 
$K_1^T \not \in $ TIME$(T)$. 
We proved this by a  diagonalization argument. A natural question is then:
if time~$T$ is not enough to decide $K_1^T$, what time would then  be enough?
This will be answered in Theorem~\ref{th:t-square-time}.

A natural algorithm for deciding $K_1^T$ is:
\begin{enumerate}
\item compute $T(n)$
\item simulate $M$ on $e(M)$ for time $T(n)$
	\begin{itemize}
	\item if $M$ halts output 1,
	\item else output 0.
	\end{itemize}
\end{enumerate}

Part (b) only takes $T(n)$ time to compute. What about part (a)?
Well, it can be pretty horrendous in some cases. Consider for instance the following
examples.
\begin{newexamp} We define $T_1(n)$  as follows. Let $M_n$ be the $n$th
TM (for some predefined ordering of Turing Machines). If $M\in K_1^{2^{2^n}}$
then $T_1(n) = n$, else $T_1(n) = n+1$. 

Even though the {\em value} of $T_1(n)$ is ``small'', the time required to compute
it is huge. 
\end{newexamp}

\begin{newexamp}
 We define $T_2(n)$  as follows. $T_2(n)$ is the $n$th digit of the binary decimal expansion
of $\pi^n$. Here also the values taken by the
function $T_2$ are small: there are actually bounded! But $T_2(n)$ cannot be computed
in constant time (constant means independent of the input).
\end{newexamp}

The natural algorithm for $K_1(T)$ along with the two previous examples
should have convinced you that the time measuring functions $T$ that
we want to consider should not be too wild,
 and this justifies the introduction of the following definition.

\begin{definition}
$T:\;\nat\rightarrow\nat$ is a 
{\it step-counting function\/} if there is a TM which computes
$T$ in time $O(T)$, using unary representation of integers for input and
output.
\end{definition}

In Homework~9, you prove that all the usual functions
are  encompassed by this  definition.

\begin{theorem}
\label{th:t-square-time}
Assume that $T$ ia a step counting function. Then
$K_1^T$ is decidable in time $T^2(n)$, i.e.,
$K_1^T \in$ TIME$(T^2(n))$.
\end{theorem}




{\em Note:} This is a little improvement on Theorem~3 of
Handout~33.
\begin{proof}
Look at the natural algorithm described above. Step~(a) takes
$O(T(n))$ steps. Step~(b) takes $T(n)$ ``phases'': each of these phases
involves the simulation of a single step (call it step $k$) of machine $M$.
In the $k$th step of machine $M$ the input held on the tape is
of length at most $k$. The simulation of this step involves screening the tape and
doing some local modifications: this requires $O(k)= O(T(n))$ steps. 
Hence the simulation
of all the $T(n)$ steps require at most $T^2(n)$ steps.
\end{proof}

This result is very tight: 
$K_1^T \in$ TIME$(T^2(n))$ but 
$K_1^T \not \in$ TIME$(T(n))$\ !


\begin{corollary}
$$\TIME(n)\psubset\TIME(n^2) \psubset\TIME(n^4)\psubset\ldots$$
\end{corollary}

\begin{corollary}
$$\poly\psubset\TIME(2^n) \psubset\TIME(4^n) 
\psubset\ldots$$
\end{corollary}

These results are the formalization of a very intuitive notion: they say that
we can compute more with more time! In the face of this hierarchy, 
a natural concern is  to  locate precisely
what is the precise complexity of all the problems that we know. For
instance where would we locate CLIQUE or HAMILTONIAN-PATH in this hierarchy?
This type of question  involves two different types of sub-questions:
the upper bound is usually obtained by constructing a specific algorithm that
solves the problem investigated in provably ``little'' time. The lower bound
is usually much harder to establish and our current panoply for proving
that type of result is still very poor!

\section{Polynomial-time reducibility, $\leq_\poly$-Hardness}
Read Section 6 and 7 of Handout~33.

\section{A Provably Infeasible Problem from Automata Theory}
We saw in the first part of this course how to define regular expressions
over $\Sigma = \{0,1\}$. They were
(bracket balanced) expressions over $\Sigma \cup \{(, ), +, \cdot, *,\}$. 
Define the following languages:
$$\hbox{EMPTYREG} ::= \{E : \ E \hbox{ is a regular expression over alphabet
$\Sigma$ and } L(E) = \emptyset\},$$

$$\hbox{NEREG} ::= \{E : \ E \hbox{ is a regular expression over alphabet
$\Sigma$ and } L(E) \not= \Sigma^*\}$$

\begin{theorem}
{EMPTYREG} is in $\poly$ and \hbox{NEREG} is decidable in exponential time.
\end{theorem}

\begin{proof}
\begin{itemize}
\item {EMPTYREG}: We can describe a procedure that decides bottom up
whether a regular expression is empty. We let 
$N$ denote the set of non-empty regular expressions.
The operations on regular expressions induce the following equalities:
\begin{itemize}
\item $\emptyset \cdot N = \emptyset$
\item $N \cdot N = N$
\item $\emptyset + \emptyset = \emptyset$
\item $\emptyset + N = N$
\item $N + N = N$
\item $N^* = N$
\item $\emptyset^* = N$ (because $\emptyset^* = \Lambda$)
\end{itemize}
Hence, given a regular formula we can decide in polynomial time whether 
it is empty or not.

\item {NEREG}: We can convert the regular expression $E$ to
an NFA in polynomial time, and then the NFA into a DFA in exponential
time. Then $L(E)= \Sigma^*$ iff  the states of the DFA are all accepting states.

This decision procedure runs in exponential time. But it is still
an open question to know whether it is optimal, i.e., whether
$ \hbox{NEREG} \in $ TIME$(T)$ with $T(n)= o(e^n)$.
\end{itemize}
\end{proof}

$\hbox{NEREG} ::= \{E : \ E \hbox{ is a regular expression over alphabet
$\Sigma$ and } L(E) \not= \Sigma^*\}$ is well studied in Handout~33.
The main result of this study, expressed in Theorem~7, is that
NEREGSQ is not in $\poly$.

Note that $\Lambda$ is denoted $e$ in page 15 of Handout~33.

In order to truly understand this proof you should 
be able to answer the following  questions:
\begin{itemize}
\item Where is exponentiation used in the proof?
\item Why does the use of exponentiation allow us to conclude
that NEREGSQ is not in $\poly$ (whereas this result is not known
for NEREG)?
\item What does it mean that the legality function $legal_M$ is {\em local}?
\item What does the condition described by  
 $$\bigcup_{a,b,c,d\in\Sigma_c}\;\Sigma_c^*abcd\Sigma_c^{2^{kn}+n+1}
  \left(\Sigma_c-{\it legal}_M(a,b,c,d)\right)\Sigma_c^*.$$
in case~2. of page 16 really express? Why is there $ \Sigma_c^{2^{kn}+n+1}$
in this formula?
\item Why does this {\em finite} union $\cup_{a,b,c,d\in \Sigma_c}\ldots$ 
account for {\em all} the strings that are not the concatenation
of consecutive  configuration 
words? (After all, there are {\em more} then
a {\em constant} number of locations to check to make sure that
a string is such a legal concatenation of  configuration words interspaced with \$ signs!!)
\end{itemize}

Here are some quick answers:

\begin{itemize}
\item Exponentiation is used to say (in a concise way) that the configuration
words considered have length ${2^{kn}+n+3}$. We ensure that all configuration
words have this length by padding if necessary with \#-signs.
The number ${2^{kn}+n+3}$ represents
the length $n$ of the initial input, plus 1 for the state symbol,
plus $2^{kn}$ for everything that might need  to be written in time $2^n$,
 plus 1 for the \#-sign that always begins the configuration word, plus 1
for the final answer ``Y'' or ``N'' telling whether $w$ is in the language $L$
or not. Remember also that in the computation word 
$\$C_1\$C_2\$...\$C_l\$ $ the 
configuration words $C_l$ are separated with \$-signs. As a consequence there are
actually  ${2^{kn}+n+4}$ symbols between a location $b$ in a computation word
$C_l$ and the  corresponding  location $b'$ in the subsequent  computation word
$C_{l+1}$. Hence there are 
${2^{kn}+n+1}$ symbols between a location $d$ (of a substring $abcd$) in a computation word
$C_l$ and the location $b'$ in the subsequent  computation word.
 The factor
$k$ in $2^{kn}$ comes from the reduction of a $k$-tape TM to a 1-tape TM.

\item We already know that TIME$(2^n) \not \subseteq \poly$.
(Actually we even know that $\poly \psubset\TIME(2^n)$~!) Showing that
NEREGSQ is TIME$(2^n)$-hard then insures that 
NEREGSQ $\not \in \poly$.

\item  $legal_M$ is {\em local} in the following sense. To check that $C_l$ and $C_{l+1}$
are two consecutive computation words we just need to write 
$C_l$ on top of $C_{l+1}$ and do local checks on all length 4 (and height 2) windows:
if in the current configuration we are in state
$q_l$ and scanning $\sigma_l$, in the next configuration
we are in state $q_{l+1}$, having replaced $\sigma_l$ by $\sigma_{l+1}$ with the head
positioned to the left, to the  right or in the same position.

\item The condition described by  $\cup_{a,b,c,d\in \Sigma_c}\ldots$ in 
case~2. of page 16 expresses precisely that two  words $C_l$ and $C_{l+1}$
differ
only by these legal changes.
But the two words $C_l$ and $C_{l+1}$ are not written on top of each other
as in the method that I just suggested: $C_l$ is written {\em on the left of } $C_{l+1}$.
Hence the two windows $a_l,b_l,c_l,d_l$ and $a_{l+1},b_{l+1},c_{l+1},d_{l+1}$ are
${2^{kn}+n+4}$ apart. 

\item The union $\cup_{a,b,c,d\in \Sigma_c}\ldots$ is  {\em finite} because,
for a given TM, 
the different possible moves  are uniquely 
characterized by the current state and
the symbol being currently scanned.
\end{itemize}
	
To make sure that you understood all this let me finish with
a last question. A configuration word
$C_{l+1}$ follows 
the configuration word $C_{l}$ only if $C_{l+1} = C_{l}$ except
for {\em only one} size-4 window $abcd$ (which is changed into 
the legal modification $a'b'c'd'$).
But
consider once again the formula
of part 2., page 16 of handout~33:
$$\bigcup_{a,b,c,d\in\Sigma_c}\;\Sigma_c^*abcd\Sigma_c^{2^{kn}+n+1}
  \left(\Sigma_c-{\it legal}_M(a,b,c,d)\right)\Sigma_c^*.$$
This formula expresses that, if for {\em all} substrings $abcd$
(i.e., for all size 4 windows)
of the computation word, the corresponding subsequent $b'$ ($2^{kn}+n+1$ away)
is equal to $legal_M(a,b,c,d)$ then the computation word is correct.
But, once more, this does not precludes a priori the possibility of having
$C_{l+1}$ and $C_{l}$ differing in {\em two} size-4 windows.
Why is this not a problem?

\end{document}



