
\documentclass{article}

\usepackage{xspace,graphicx,hyperref,paralist,subfigure}

\newtheorem{illustration}{Illustration}[section]
\newtheorem{theorem}{Theorem}[section]


\newcounter{exer}
\renewcommand{\theexer}{\thechapter:\arabic{exer}}

\newcommand{\FIXIT}[1]{\typeout #1 -- {\LARGE\sf #1}}

%---
\newcommand{\TM}{{Turing machine}\xspace}
\newcommand{\TMs}{{Turing machines}\xspace}
\newcommand{\NPCness}{{NP-completeness}\xspace}
\newcommand{\NPC}{{NP-complete}\xspace}
\newcommand{\NPH}{{NP-hard}\xspace}
\newcommand{\sfP}{{\sf P}\xspace}
\newcommand{\sfNP}{{\sf NP}\xspace}
\newcommand{\sfNPC}{{\sf NPC}\xspace}
\newcommand{\sfNPH}{{\sf NPH}\xspace}
\newcommand{\sfCoNP}{{\sf CoNP}\xspace}
\newcommand{\sfCoNPC}{{\sf CoNPC}\xspace}
\newcommand{\BA}{{B\"{u}chi automata}\xspace}
\newcommand{\BAon}{{B\"{u}chi automaton}\xspace}
\newcommand{\TL}{{Temporal Logic}\xspace}
\newcommand{\LTL}{{linear-time temporal logic}\xspace}
\newcommand{\CTL}{{Computational Tree Logic}\xspace}
\newcommand{\KS}{{Kripke structure}\xspace}
\newcommand{\KSs}{{Kripke structures}\xspace}
\newcommand{\OR}{{$\omega$-regular}\xspace}
%---


\oddsidemargin 0cm
\textwidth 17cm

\begin{document}
{\hfill \large\bf CS 3100 -- Models of Computation -- Fall 2011 --  Notes for L27 \hfill}

{\hfill \large\bf Halting Problem \hfill}

{\hfill \large\bf Last Updated 10:01am, 12/01/11 \hfill }

\paragraph{Acceptance ($A_{TM}$) is undecidable (important!):\/}
We now prove this set to be {\em undecidable}.
{\bf Suppose} there exists a decider $H$ for $A_{TM}$. $H$ expects to be
 given a \TM $M$ and a string $w$. Notice that ``giving $H$ a \TM'' means
 ``giving it a character string representing a \TM program.''

Build a program called $D$ as follows:
  \begin{list}{$-$}{\addtolength{\itemsep}{-1.6\itemsep}}
  \item $D$ takes a single argument $M$.
  \item As its first step, $D$ invokes $H$ on $\langle M,M\rangle$.\footnote{Basically,
   we feed $\langle M\rangle$ twice over, just to `please' $H$ that expects two arguments.}
  \item If $H(\langle M,M\rangle)$ rejects, $D(\langle M\rangle)$ accepts.
  \item If $H(\langle M,M\rangle)$ accepts, $D(\langle M\rangle)$ rejects.
  \end{list}
% \footnotetext{Basically, we feed $\langle M\rangle$ twice over, just to `please' $H$ that expects two arguments.}
%
 Now we can ask what $D(\langle D\rangle)$ will result in (to preserve the clarity of our arguments,
the reader is invited to suppress any occurrence of $\langle\ldots\rangle$ in the text below):
%
\begin{list}{$-$}{\addtolength{\itemsep}{-1.6\itemsep}}
\item The $D(\langle D\rangle)$ ``call'' turns into an $H(\langle D,D\rangle)$ call.
\item Suppose $H(\langle D,D\rangle)$ rejects. In that case, {$D(\langle D\rangle)$ accepts.}
\item But, according to the advertised behavior of $H$ --- which is that it is a decider for $A_{TM}$ --- the
fact that $H(\langle D,D\rangle)$
 rejects means that $D$ is {\em not} a Turing machine that will accept $\langle D\rangle$, or
that {$D(\langle D\rangle)$ rejects!}
\item Suppose $H(\langle D,D\rangle)$ accepts. In that case, {$D(\langle D\rangle)$ rejects.}
\item But, according to the advertised behavior of $H$ --- which is that it is a decider for $A_{TM}$ --- the
fact that $H(\langle D,D\rangle)$
 accepts means that $D$ {\em is} a Turing machine that accepts $\langle D\rangle$, or
that {$D(\langle D\rangle)$ accepts!}
\end{list}
%
\noindent Therefore, we obtain a contradiction {\em in both cases}.
Hence, the claimed decider $H$ for $A_{TM}$ cannot exist, or
{\em the acceptance problem for Turing machines is undecidable.}

%\footnotetext{Please recall our discussions of Section~\ref{sec:russells-paradox}.}%

%

\paragraph{What does this have to do with diagonalization?\/}
We can conceptually build a table of machines that accept other machine descriptions (supposedly filled
by the checker program {\tt H}. 
%
Since {\tt D} is a legal machine, it must exist in this table.
%
Now, at the spot marked with {\tt ?}, {\tt H} will run into a contradiction.

\begin{scriptsize}
\begin{verbatim}
     <M1>  <M2>  <M3>  <M4>  ..   <D>

M1    Y     N     N     Y

M2

M3

M4

..

D                                 ?

\end{verbatim}
\end{scriptsize}

\paragraph{Halting ($Halt_{TM}$) is undecidable (important!)\/}
%
\label{sec:reduction}
\index{$Halt_{TM}$}
%
\begin{figure}[tbhp]
\centering
  \subfigure[{$A_{TM}$ to $Halt_{TM}$ reduction}]
  {\includegraphics[scale=0.25]{atm_to_halt_reduction.pdf}
  \label{fig:atm_to_halt_reduction}  
  }
\qquad
  \subfigure[{Illustration of mapping reduction from $A$ to $B$}]
   {\includegraphics[scale=0.4]{mapping-redn.pdf}
   \label{fig:mapping-redn}
   } 
\end{figure}

The golden rule of reduction is:
{\bf Reduce an existing (``old'') undecidable problem to the given (``new'') problem.}
If we assume that the new problem is decidable, we would 
conclude that the existing undecidable problem is decidable (a contradiction,
Figure~\ref{fig:atm_to_halt_reduction} where the
``{\em new problem}'' is $Halt_{TM}$).
%
By assuming that it is decidable, we can
assume the existence of the decider $D_{Halt_{TM}}$, and using it, build a
decider for $A_{TM}$, the ``{\em old problem},'' known undecidable;
this obtains a contradiction. Hence, $D_{Halt_{TM}}$ cannot exist.

We should not go by the English language meaning
 of the term ``reduction'' that can lead us astray.
 %
To see why, assume this wrong direction and see what it leads to.
%
It will lead us to prove, by contradiction, the following
futile fact: ``IF the existing undecidable problem is decidable, THEN
we would have shown the new problem to be decidable.'' However,
this is a statement of the form ``IF false, THEN assert X.''
Clearly, in this case, we cannot assert X.
%
{\em So we must always reduce the known undecidable problem to a new problem.}
%
Then the new problem, if shown decidable, will trigger a contradiction.
%
Always draw a ``boxes within boxes'' diagram as in Figure~\ref{fig:atm_to_halt_reduction}.
%
Here, the new problem is used to build a box to ``solve'' the old impossible-to-solve problem.

\paragraph{An introduction to mapping reduction:\/}
%
Define
\( Halt_{TM} = \{ \langle M,w\rangle \;\mid\; TM\; M\; halts\;on\; w \}. \)
%
We show $Halt_{TM}$ to be undecidable as follows:
%
\begin{quote}
  Suppose not; i.e., there is a
  decider for $Halt_{TM}$. Let's now build a decider for
    $A_{TM}$ (call it $D_{A_{TM}}$).
  $D_{A_{TM}}$'s design will be as follows:
    %--
    \begin{list}{$-$}{\addtolength{\itemsep}{-1.7\itemsep}}
    \item $D_{A_{TM}}$ will first feed $M$ and $w$ to
      $D_{Halt_{TM}}$, the claimed decider for $Halt_{TM}$.
    \item If $D_{Halt_{TM}}$ goes to $accept_{\textstyle D_{Halt_{TM}}}$,
      $D_{A_{TM}}$ knows that it can safely run $M$ on $w$, which it does.
    \item If $M$ goes to $accept_{\textstyle M}$, $D_{A_{TM}}$ will go to $accept_{\textstyle D_{A_{TM}}}$.
    \item If $M$ goes to $reject_{\textstyle M}$, or if
      $D_{Halt_{TM}}$ goes to $reject_{\textstyle D_{Halt_{TM}}}$,
      $D_{A_{TM}}$ will go to $reject_{\textstyle D_{A_{TM}}}$.
    \end{list}
\end{quote}
%
Notice that we have labeled the accept and reject states of
the two machines $D_{Halt_{TM}}$ and $D_{A_{TM}}$.
%
After one becomes familiar with these kinds of proofs,
higher-level proof sketches are preferred.
%
{\sl Here is such a higher-level proof sketch.
By way of contradiction, assuming $D_{Halt_{TM}}$ exists, build $D_{A_{TM}}$.
Accepts input $\langle M,w\rangle$
and run $D_{Halt_{TM}}$ on it. If this run accepts,
then we can safely (without the fear of looping) deploy $M$ on $w$,
and return the accept/reject result that this run returns;
else return ``reject.''}


\paragraph{Mapping reductions:\/}
%--


\noindent{\bf Definition:\/}
A {\em computable} function $f:\Sigma^{*}\rightarrow \Sigma^{*}$ is a mapping reduction
from $A\subseteq \Sigma^*$
into $B\subseteq \Sigma^*$ if for all $x\in\Sigma^*$, $x\in A \Leftrightarrow f(x)\in B$.
%


%
% \footnotetext{Using the familiar notation ${\cal O}(\ldots)$
%  for asymptotic upper-bounds,
%  polynomial-time means ${\cal O}(n^k)$ for an input of length $n$, and $k>1$.
%  See
% \cite{hop-ull-algo,moret-shapiro,cormen-rivest-leicerson-algo} for details.}

See Figure~\ref{fig:mapping-redn} which illustrates the general situation that $A$
maps into a subset denoted by $f(A)$ of $B$, and members of $A$ map into $f(A)$
while non-members of $A$ map outside of $B$ (that means they map outside of even
$B\setminus f(A)$).
%
Also note that $A$ and $B$ need not be disjoint sets, although they often are.
%
A mapping reduction can be (and usually is) a non-injection and non-surjection; {\em i.e.},
it can be many to one and not necessarily onto.
%
It is denoted by $\leq_m$.
%
By asserting $A\leq_m B$, the existence of an $f$ as described above
is also being asserted.

\noindent Typically mapping reductions are used as follows:
%
\begin{list}{$\bullet$}{\addtolength{\itemsep}{-1.6\itemsep}}
\item Let $A$ be a language known to be undecidable (``old'' or ``existing'' language).
\item Let $B$ be the language that must be shown to be undecidable (``new'' language).
\item Find a mapping reduction $f$ from $A$ into $B$.
\item Now, if $B$ has a decider $D_B$, then we can decide membership in $A$ as
 follows:
 \begin{list}{$-$}{\addtolength{\itemsep}{-1.6\itemsep}}
 \item On input $z$, in order to check if $z\in A$, find out if $D_B(f(z))$
    accepts or not. If it accepts, then $z\in A$, and if it rejects, then
    $z\notin A$.
 \end{list}
\item The PCP is shown undecidable by exhibiting a mapping reduction
  from $A_{TM}$ to PCP instances (``tiles'').
\item Grammar ambiguity is shown undecidable through a mapping reduction
  from PCP instances to CFGs.
\item We can see how arguments chain: $A_{TM}$ is reduced to $PCP$ which in turn
  is reduced to $CFG\_ambiguity$. Thus $CFG\_ambiguity$ being decidable makes $PCP$
decidable which makes $A_{TM}$ decidable.
\end{list}

\subsubsection*{Rice's Theorem}

Every non-trivial partitioning of TM codes based on the language of the TMs is undecidable.
A trivial partitioning is either ``all TM codes'' or ``no TM code.'' These are decidable sets.
%
A non-trivial partitioning attempts to classify $\langle M\rangle$ based on the {\em language}
that the TM accepts.
%
Here are some non-trivial partitionings (any such language is undecidable):
%
\begin{compactitem}
\item The set of all TMs (TM codes) that accept a particular string
\item The set of all TMs whose language is regular
\item The set of all TMs which accept $\epsilon$
\item The set of all TMs which accept a given string $w$ 
\end{compactitem}

\subsubsection*{A brief overview of NP-completeness}

\noindent{\bf Class NP:\/} The
class NP is those problems for which there are non-deterministic
algorithms that ``perform'' in polynomial time.
%
That is, there is an NDTM that has one execution path (among the potential
exponentially branching set of non-deterministic selections) that takes a
polynomial number of steps.

Alternately, an NP problem is one for which given a ``certificate'' (a claimed answer)
which is also polynomially long, a DTM
can check the answer for correctness in polynomial time.


Examples of ND languages:
\begin{compactitem}
\item \( SAT = \{ \Gamma \;\mid\; \Gamma\; is\; a\; Boolean\; formula\;
  that\; is\; satisfiable\}\). This is in NP because of either reason below:
  \begin{compactitem}
    \item One can develop a SAT checker that picks one assignment for the
      Boolean variables (the ``guess generation phase'') and checks the
      formula for satisfiability.
    \item Given a formula and a claimed assignment, one can check whether
      the formula is true under the assignment.
  \end{compactitem}
\item \( 3SAT= \{ \Gamma \;\mid\; \Gamma\; is\; a\; 3CNF \; Boolean\; formula\;
  that\; is\; satisfiable\}\). This is in NP for the same reasons.
\item \( Clique= \{ G \;\mid\; G\; is\; a\; graph\; with\; a\; \; clique\; of\; size\; k\}\). 
  This is in NP because
  of either reason below:
  \begin{compactitem}
  \item The NDTM can generate a collection of $k$ nodes and then check.
  \item One can present a collection of $k$ nodes and design a DTM that checks whether
    these form a clique.
  \end{compactitem}
\end{compactitem}

\begin{figure}[tbhp]
\centering
  \subfigure[{Show problem in NP; then do a mapping reduction.}]
  {\includegraphics[scale=0.35]{funnel-diagram.pdf}
  \label{fig:funnel-diagram}
  }
\qquad
  \subfigure[{The Proof that $Clique$ is \sfNPH using an example formula
 $\varphi = (x_1\vee x_1\vee x_2)\wedge
            (x_1\vee x_1\vee \neg x_2)\wedge
            (\neg x_1\vee \neg x_1\vee x_2)\wedge
            (\neg x_1\vee \neg x_1\vee \neg x_2)$}]
   {\includegraphics[scale=0.4]{clique-npc.pdf}
   \label{fig:clique-npc}
   } 
\end{figure}

\noindent{\bf {NP-Completeness}:\/}
We don't know whether all ND languages have deterministic polynomial
time algorithms. That is whether the language family NP and P are
the same.
%
To attack this annoying ``gap'' between P and NP,
scientists have come up with the {\em hardest} members of NP
called NP-complete.

A language $L$ is NP-complete if
\begin{compactitem}
  \item (In NP) $L$ is in NP
  \item (NP-hard) 
    Every other language in NP can be poly-time mapping reduced to $L$
\end{compactitem}

Note that
there are undecidable NP-hard languages (e.g. Diophantines language!).
%
Thus we must show the ``In NP'' part for any meaningful dialog in this space!!

The first problem shown to be NP-complete is 3SAT.
%
This is done by showing that the ``work of any NDTM can be
represented as Boolean formula satisfaction.''

Now for any other language (e.g. Clique), we show it NP-complete by
\begin{compactitem}
  \item Showing that Clique is in NP
  \item Showing that there is a mapping reduction from 3SAT to Clique
\end{compactitem}

A diagram showing this mapping reduction chain is in Figure~\ref{fig:funnel-diagram}.
Here, all of NPC was mapping-reduced to 3SAT.
%
Then problems such as Clique are shown NPC by the second mapping-reduction whose
details are illustrated in Figure~\ref{fig:clique-npc}.
%
If Clique were to have a P-time algorithm, then {\em all of NP} will have a P-time
algorithm---thought to be highly unlikely!


\subsubsection*{Significance of NPC}

While NP-complete problems are solved using algorithms with exponential
time complexity, with well-chosen heuristics, they often run in polynomial
time.
%
Complexity classifications are essential for theoretical CS advance by linking problems together
into complexity classes, and developing a common understanding.

\end{document}


%--\end{compactitem}
%--\end{document}