\documentstyle[twoside,psfig]{article}
\input{defs-6045}
\newcommand{\mless}{\leq_m}
\newcommand{\tless}{\leq_T}
\newcommand{\meq}{\equiv_m}
\newcommand{\cmp}{\overline}
\newtheorem{prop}[theorem]{Proposition}


\begin{document}

\handout{lecture24}{Notes on Lecture 24 (4/3/92)}{April 6, 1992}


\section{Advanced Recursion Theory}

In the past few weeks we've defined the concepts of T-acceptable/r.e.\
languages, recursive/decidable languages, partial-recursive functions,
and total recursive functions using Turing Machines.  We've shown
that many problems fit into these various classes.  We've also shown
ways to prove that languages don't fit into these classes, using
diagonalization and reducibility arguments.

Today, we'll finish up our look at general computability theory by
considering a few final concepts:

\begin{enumerate}
\item We'll show that an interesting language is neither r.e.\ nor
co-r.e. (where ``$L$ is co-r.e.'' means ``the complement of $L$ is
r.e.'').  So in a sense, this language is even harder than 
undecidable.  

\item We'll define an important concept of $\mless$-completeness to
help us classify various problems.  This concept of completeness
foreshadows some important ideas in complexity theory that will be
appearing soon.

\item We'll discuss ``higher degrees of unsolvability''.  Some
problems are ``worse'' than undecidable!

\item We will state the amazing Recursion Theorem, which can be used
to prove deep results in a simple manner.
\end{enumerate}

\section{A Non-r.e. and Non-co-r.e. Language}

We have shown many problems to be non-recursive.  We showed that $K_0
= \{<M,x>: M$ halts on $x \}$ is r.e., but not recursive, and we also
showed that $K_0 \mless TOT$, where $TOT = K_4 = \{M : M$ halts on all
inputs $\}$, which proves that $TOT$ is also not recursive.  The proof
follows from the fact that if $A \mless B$ and $A$ is not recursive,
then $B$ is not recursive.

But note that since $K_0$ is r.e.\ but not recursive, then $\cmp{K_0}$
must not be r.e.  If both $K_0$ and $\cmp{K_0}$ were r.e., then we have
a theorem that states that $K_0$ must be recursive.  Since this
contradicts the non-recursiveness of $K_0$, it must be the case that
$\cmp{K_0}$ is not r.e.  Now $\cmp{K_0} \mless \cmp{TOT}$, since when $A
\mless B$ then $\cmp{A} \mless \cmp{B}$.  This implies that
$\cmp{TOT}$ is also not r.e., because if $A \mless B$ and $A$ is not
r.e., then $B$ is not r.e.  (see the notes from lecture 20).

We claim that $TOT$ is also not r.e.  We will prove this by showing
that $\cmp{K}_0 \mless TOT$.  So we need a total recursive function
$f$ such that $<M,x> \in \cmp{K}_0$ iff $f(<M,x>) = N_{M,x} \in TOT$;
that is, $M$ does not halt on $x$ if and only if $L(N_{M,x}) =
\Sigma^*$.  Recall that we have a canonical ordering on the strings of
$\Sigma^*$.  We will call a string $y_i$ if it is the $i$th string in
the canonical ordering.  We define machine $N_{M,x}$, on input $y_i$,
to simulate $M$ on $x$ for $i$ steps.  If $M$ halts on $x$ within $i$
steps, then $N_{M,x}$ will go into an infinite loop.  Otherwise let
$N_{M,x}$ halt.  Now notice that if $M$ never halts on $x$, then
$N_{M,x}$ halts on all inputs; hence $L(N_{M,x}) = \Sigma^*$.  If $M$
halts on $x$, however, then there exists some number $n$ such that $M$
halts on $x$ within $n$ steps.  Thus $N_{M,x}$ will go into an
infinite loop on inputs $y_n, y_{n+1}, \ldots$, so that $L(N_{M,x})
\neq \Sigma^*$.  Because $M$ does not halt on $x$ iff $L(N_{M,x}) =
\Sigma^*$, we have shown that $\cmp{K}_0 \mless TOT$.  Therefore,
$TOT$ is not r.e.

So we see that $TOT$ is neither r.e., nor co-r.e.  There are many
problems like this, as we will see in a few minutes.

\section{$\mless$-Completeness}

We have now seen languages that are recursive, r.e.\ but not recursive
(which is the same as r.e.\ but not co-r.e.), co-r.e.\ but not
recursive (the same as co-r.e.\ but not r.e.), and neither r.e.\ nor
co-r.e.  In some sense, the last kind of language is ``harder'' or
``more impossible'' than the others.  We now define a way to classify
how hard a language is.

\begin{definition}
If ${\cal L}$ is a class of languages, then a language $A$ is {\it
$\mless$-hard for ${\cal L}$} (that is, at least as hard as all
languages in ${\cal L}$ with respect to $\mless$) provided that $B
\mless A$ for all $B \in {\cal L}$.  We sometimes say that $A$ is
${\cal L}$-hard with respect to $\mless$.
\end{definition}

\begin{definition}
$A$ is {\it $\mless$-complete for ${\cal L}$} (or ${\cal L}$-complete with
respect to $\mless$) provided that $A$ is $\mless$-hard for ${\cal L}$
and $A \in {\cal L}$.
\end{definition}

\begin{newexamp}[$K_0$ is $\mless$-complete for the r.e.\ languages]
~~\\
We've already seen that $K_0$ is r.e.  All we have to do is show that
it is $\mless$-hard for the r.e.\ languages.  Let $B$ be any r.e.\
language; we will show that $B \mless K_0$.  Since $B$ is r.e., there
is some Turing Machine $M_B$ that accepts $B$.  Then $x \in B$ iff $M_B$
halts on $x$ iff $<M_B,x> \in K_0$.  So our total recursive function
mapping $B$ to $K_0$ will simply be $f(x) = <M_B,x>$.  This proves
that $B \mless K_0$.
\end{newexamp}

\begin{newexamp}[$K_1, K_2, K_3$ are also $\mless$-complete for the
r.e.\ languages] ~~ \\
We've already seen that all of these languages are r.e.  To prove that
they are complete, all we have to do is show that they are
$\mless$-hard for the r.e.\ languages.  For example, for $K_1 = \{M :
M$ halts on $e(M) \}$, we need to show that $B \mless K_1$ for all
r.e.\ languages $B$.  We already know that for all r.e.\ $B$, $B
\mless K_0$.  We also know that if $B \mless K_0$ and $K_0 \mless K_1$, then
$B \mless K_1$, by the transitivity of $\mless$.  So if we simply show
that $K_0 \mless K_1$, then we will have shown that $K_1$ is
$\mless$-hard for the r.e.\ languages.  Define $f(<M,x>) = N_{M,x}$ as
we have in other problems, so that $N_{M,x}$ on input $w$ first
simulates $M$ on $x$ and halts if $M$ halts on $x$.  Then $N_{M,x}$
halts on all inputs if $M$ halts on $x$, and halts on no inputs if $M$
does not halt on $x$.  Then certainly $<M,x> \in K_0$ iff $N_{M,x}$
halts on $e(N_{M,x})$, or $<M,x> \in K_0$ iff $N_{M,x} \in K_1$.  Thus
$K_0 \mless K_1$.


The discussion above essentially proves the following theorem.

\begin{theorem}
If $A$ is $\mless$-complete for ${\cal L}$, $B \in {\cal L}$ and $A
\mless B$, then $B$ is also $\mless$-complete for ${\cal L}$.
\end{theorem}

Similarly, $K_2$ and $K_3$ are also $\mless$-complete for r.e.\
languages since we've shown in previous lectures that $K_0 \mless K_2$
and $K_0 \mless K_3$.  $K_4 = TOT$ is $\mless$-hard for r.e.\
languages but not $\mless$-complete, since it is not r.e.
Furthermore, $TOT$ is $\mless$-hard for the co-r.e.\ languages since
$K_0 \mless \cmp{TOT}$ implies that all r.e.\ languages reduce to
$\cmp{TOT}$, which implies that all co-r.e.\ languages reduce to
$TOT$.
\end{newexamp}

\begin{newexamp}[Rice's Theorem revisited] ~~ \\
Recall that if $P$ is a non-trivial property of r.e.\ languages then
$M_P = \{M: L(M)$ has property $P \}$ is undecidable.  The proof of
Rice's Theorem actually showed that:
\begin{enumerate}
\item If $\emptyset$ doesn't satisfy $P$, then $K_0 \mless M_P$.
\item If $\emptyset$ does satisfy $P$, then $K_0 \mless \cmp{M}_P$.
\end{enumerate}
\noindent Since $K_0$ is $\mless$-complete for the r.e.\ languages, we
have also shown that if $P$ is a non-trivial property, then:
\begin{enumerate}
\item If $\emptyset$ doesn't satisfy $P$, then $M_P$ is $\mless$-hard
for the r.e.\ languages.
\item If $\emptyset$ does satisfy $P$, then $\cmp{M}_P$ is
$\mless$-hard for the r.e.\ languages.
\end{enumerate}
\end{newexamp}

One might wonder if all languages that are r.e.\ but not recursive are
necessarily $\mless$-complete for the r.e.\ languages.  It turns out
that this is not true.  A very complicated diagonalization argument
proves that there are languages which are r.e., not recursive, and
not $\mless$-complete for the r.e.\ languages (see Hartley Rodgers,
{\it Theory of Recursive Functions and Effective Computability}).

\section{Higher Degrees of Unsolvability}

We can think of reducibility ($A \mless B$) as one program ($A$) using
another program ($B$) as a subroutine to solve a particular problem.
However, $B$ gets used as a subroutine in a very limited way: $A$ can
call $B$ only once, and whatever answer to the problem that $B$ gives
is the answer that $A$ must give.  We might rather want to call $B$
several times, and do some computation with the results before $A$
returns its answer.  To be able to do this, we will informally define
a new type of reducibility, called {\it Turing reducibility}, which is
denoted by $\tless$.  We say that $A \tless B$ if there exists a
procedure to decide $A$ given a subroutine that decides $B$.  This
notion can be defined formally using an {\it oracle Turing Machine}
that has a separate {\it oracle tape}.  Whenever the oracle TM wants
to call the subroutine for $B$, it writes question $x$ on the oracle
tape and magically, in 1 step, receives the answer of whether or not
$x \in B$.  Instead of $A \tless B$, we also say {\it $A$ is recursive
relative to $B$}, or {\it $A$ is recursive in $B$}.

A couple of facts follow straightforwardly from these definitions.
First, $A$ is recursive iff $A$ is recursive in $\emptyset$.  Second,
if $A \tless B$ and $B$ is recursive, then $A$ is recursive.  Third,
$\cmp{K}_0 \tless K_0$ even though it is not true that $\cmp{K}_0
\mless K_0$.

Given an arbitrary oracle set $B$, there are still problems that are
not recursive relative to $B$.  An example is the halting problem
relative to $B$, $K_0^B = \{<M,x> : M$ with oracle $B$ halts on input
$x \}$.   This problem can be shown to be non-recursive relative to
$B$ using a diagonalization argument similar to the one we used
before.  Using oracle TMs, we can construct a hierarchy of harder and
harder (or ``more and more undecidable'') problems: $K_0, K_0^{K_0},
K_0^{K_0^{K_0}}, \ldots$.  $K_0^{K_0}$ is the halting problem
relative to the halting problem; that is, the problem of whether a
given oracle TM with a $K_0$ oracle halts on input $x$.  Each problem
in the sequence above is Turing reducible to the ones after it, but is
not Turing reducible to the ones before it (e.g. $K_0^{K_0} \not\tless
K_0$). 

We can classify natural problems according to whether they are Turing
reducible to various sets in this sequence.  For example, $K_0, K_1,
K_2, K_3$ are all Turing reducible to $K_0$.  $TOT$, however, is not
Turing reducible to $K_0$ (this does not follow straightforwardly from
the fact that $TOT$ is neither r.e.\ nor co-r.e.).  To prove this, one
can show that $K_0^{K_0} \tless TOT$, a reduction that we will not go
through here.  The classification of $TOT$ is completed by showing that
$TOT \tless K_0^{K_0}$, which we will prove by showing that $\cmp{TOT}
\mless K_0^{K_0}$.  Given a TM $M$ from $TOT$, we construct an oracle
TM $N_M$ such that $L(M) \neq \Sigma^*$ iff $N_M^{K_0}$ accepts $x$.
$N_M^{K_0}$ works by ignoring its input, then asking its oracle $K_0$
if $<M,\Lambda> \in K_0$, $<M,y_1> \in K_0$, $<M,y_2> \in K_0$, and so
on, where $y_1, y_2, \ldots$ are the strings in the canonical ordering
of strings.  If the oracle ever answers ``no'', then $N_M^{K_0}$ halts
and accepts, otherwise it continues asking questions forever.  Thus if
$L(M) \neq \Sigma^*$, then the oracle will answer ``no'' for some
$<M,y_k>$, so $N_M^{K_0}$ will accept input $x$.  If $L(M) =
\Sigma^*$, then the oracle will always answer yes, so that $N_M^{K_0}$
will never halt on input $x$.  Since $L(M) = \Sigma^*$ iff
$<N_M^{K_0},x> \in K_0^{K_0}$, we have that $\cmp{TOT} \mless
K_0^{K_0}$.

\section{The Recursion Theorem}

The Recursion Theorem is a rather amazing, counterintuitive result
that can be used to devise clever and mystifying proofs.  

\begin{theorem}[Recursion Theorem]
Given {\it any} total recursive mapping $f:M \rightarrow N_M$ from TMs
to TMs over a common input alphabet $\Sigma$, there is some $M$ such
that $L(M) = L(N_M)$.
\end{theorem}

Intuitively, the theorem says that given some total recursive mapping
of TMs to TMs, there is some ``fixed point'' of the mapping.  A
similar theorem holds on TMs that compute partial recursive functions
(instead of accepting languages).

\begin{example}
Suppose $f$ maps $M$ to $N_M$, where $N_M$ halts on all inputs iff $M$
halts on $\Lambda$ (this $f$ could be used to show that $K_2 \mless
TOT$).  What is the fixed point here?  For any $M$ such that $L(M) =
\Sigma^*$, then $L(M) = L(N_M) = \Sigma^*$.
\end{example}

\begin{example}
Suppose $f$ maps $M$ to $N_M$ where $N_M$ simulates $M$, and if $M$
halts with output $y_i$, then $N_M$ halts with output $y_{i+1}$.  The
recursion theorem for partial recursive functions would say that there
exist some $M$ for which $M$ and $N_M$ compute the same function.
The only $M$ for which that is true in this case is an $M$ which never
halts.
\end{example}

We can use the recursion theorem to come up with another proof of
Rice's Theorem.

\begin{proof}  We want to show that if $P$ is a non-trivial property
of r.e.\ languages, then $M_P$ is not decidable.  Suppose that $M_P$
is decidable.  Let $M_1$ be any machine accepting a language that
satisfies property $P$, and let $M_2$ be a machine that accepts a
language not satisfying $P$.  We know that both machines exist by the
non-triviality of $P$.  Define a function $f$ mapping $M$ to $N_M$
where if $M \in M_P$, then $N_M = M_2$, while if $M \not\in M_P$, then
$N_M = M_1$.  Given our assumption that $M_P$ is decidable, $f$ is
total recursive: we simply take the TM that decides $M_P$ and make it
output $M_2$ instead of ``1'' and $M_1$ instead of ``0''.  By the
recursion theorem, we know there is a fixed point, a machine $M$ such
that $N_M$ and $M$ accept the same language.  But if $M \in M_P$ then
$N_M = M_2 \not\in M_P$, while if $M \not\in M_P$, then $N_M = M_1 \in
M_P$.  Since if $N_M$ and $M$ are the same language they can't both
have and not have property $P$, so we have reached a contradiction.
Hence our assumption that $M_P$ is decidable must be wrong.
\end{proof}



\end{document}
