\documentclass{article}

\usepackage{hevea,xspace,graphicx,hyperref}

\newtheorem{illustration}{Illustration}[section]


\newcounter{exer}
\renewcommand{\theexer}{\thechapter:\arabic{exer}}

\newcommand{\FIXIT}[1]{\typeout #1 -- {\LARGE\sf #1}}

%---
\newcommand{\TM}{{Turing machine}\xspace}
\newcommand{\TMs}{{Turing machines}\xspace}
\newcommand{\NPCness}{{NP-completeness}\xspace}
\newcommand{\NPC}{{NP-complete}\xspace}
\newcommand{\NPH}{{NP-hard}\xspace}
\newcommand{\sfP}{{\sf P}\xspace}
\newcommand{\sfNP}{{\sf NP}\xspace}
\newcommand{\sfNPC}{{\sf NPC}\xspace}
\newcommand{\sfNPH}{{\sf NPH}\xspace}
\newcommand{\sfCoNP}{{\sf CoNP}\xspace}
\newcommand{\sfCoNPC}{{\sf CoNPC}\xspace}
\newcommand{\BA}{{B\"{u}chi automata}\xspace}
\newcommand{\BAon}{{B\"{u}chi automaton}\xspace}
\newcommand{\TL}{{Temporal Logic}\xspace}
\newcommand{\LTL}{{linear-time temporal logic}\xspace}
\newcommand{\CTL}{{Computational Tree Logic}\xspace}
\newcommand{\KS}{{Kripke structure}\xspace}
\newcommand{\KSs}{{Kripke structures}\xspace}
\newcommand{\OR}{{$\omega$-regular}\xspace}
%---


\oddsidemargin 0cm
\textwidth 17cm

\begin{document}


\title{\large\bf CS 3100 -- Models of Computation -- Fall 2011 --  Notes for L3}

\maketitle

\section{Roadmap for this Course}

\noindent This course will examine the following foundational principles

\begin{itemize}
\item Languages, Grammars, and Machines
  \begin{itemize}
  \item Languages and why study them?
  \item Regular, context-free and recursively enumerable languages
  \item Other languages: Recursive, NP-complete, etc.
  \item Machines to describe languages
    \begin{itemize}
    \item Machines with and without non-determinism
    \end{itemize}
  \item Grammars to describe languages
  \end{itemize}
\item{\em Midterm 1}
\item Mathematical Logic
  \begin{itemize}
  \item Propositional Logic
    \begin{itemize}
    \item Building propositional solvers using DFA: the BDD approach
    \item Solving puzzles using BDDs
    \end{itemize}
  \item First-order Logic
    \begin{itemize}
    \item A taste of first-order logic: Solving Ken-ken
    \end{itemize}
  \end{itemize}

\item Schr\"{o}der-Bernstein Theorem, knowing how to put sets into various
  cardinalities 
\item Diagonalization 
\item Turing Machines and Undecidability; how to show the undecidability of
  the Halting Problem
\item{\em Midterm 2}
\item Launch Projects
\item Complexity: NP-completeness
\item Showing simple problems NP-complete
\item{\em Finals}
\end{itemize}


\section{Formal Languages, Grammars, Machines, Non-determinism}

\noindent {\bf Formal} Languages (``languages'') have many roles, some important ones being these:
%
\begin{itemize}
\item Describe desired strings (to help filter out undesirable ones out). Some uses:
  \begin{itemize}
  \item In spreadsheets
  \item Compilers, GUIs, other text-oriented information processing systems
  \item Network malware detection
  \item Biology, Identifying Genomes
  \end{itemize}
\item Help encode algorithms (problems to be solved) and help analyze
  the complexity (time and space) of machines deployed as {\em recognizers}
  for these languages.
\end{itemize}

\noindent {\bf Machines} help specify languages:
%
\begin{itemize}
\item Gives us an {\em operational} flavor (plug and chug along a string)
\item Machines can be programmed directly to give us {\em algorithms}
  to recognize languages (filter bad strings out)
\end{itemize}

\noindent {\bf Grammars} help specify languages:
%
\begin{itemize}
\item Sometimes machines are tedious to draw out (are huge)
\item Machines are not ``that machine-readable''---so {\em grammars}
  serve the purpose of a {\em machine-readable machine notation}
\end{itemize}

\noindent {\bf Equivalence, the power of alternative notations, non-determinism}
\begin{itemize}
\item Humans are fallible!
\item Humans {\em can't be trusted} to program correctly or specify anything correctly
\item Humans are poor at checking other humans' work!
\item So we need {\em machines to keep us honest}
\item If you specify a machine and a grammar, you'll be thinking differently in arriving
  at each, and so
  by comparing the machine and the grammar and showing that they denote the same
  language, you increase your confidence in having specified the desired language.
\item {\em Compactness, and notational economy, is a virtue}
  \begin{itemize}
  \item Compare Java and purely functional notations (such as Python) for expressing
    intricate ideas compactly
  \item The less the lines of ``code'' the less the maintenance headache
  \item More natural notations are needed often!
  \item Sometimes grammars are the most compact/natural to use
  \item Sometimes machines are the most compact/natural
  \item Ideally, you must try doing it both ways and comparing for agreement
  \item But in general, for some languages we prefer grammars, for some we
    prefer machines, etc.
    \begin{itemize}
    \item Knowing what to use comes with experience!
    \item Sometimes you can have ``rules of the thumb''; example: for a language
      of strings over $0$ and $1$ where the pattern of interest is ``the third
      from the end is a $1$, {\em don't even try to build a DFA}!
    \end{itemize}
  \end{itemize}
\item The reason we are using Python: sometimes, math is too non-obvious; by
  programming something, you understand it much better!

\item The idea of ``non-determinism'' is a {\em fundamental} invention
  of computer science. It makes many descriptions far more compact and more natural!

\item It is almost like the invention of {\em complex number}: it makes many tasks easy
  (and finally we know how to ``take the real part'' and descend into reality)

\item The same way, we use non-determinism for a ``mental high'' that allows us
  to get near the ``final answer'' quickly; once we do, we can always ``descend to
  reality'' by ``boiling away'' non-determinism

\item Oops, but sometimes, when you boil away non-determinism, you pay a big price: things
  blow up into exponential space/time on your face! You should learn when to expect this
  to happen!

\item In some machines, adding non-determinism fundamentally alters their power; in some machines,
  it only makes machines smaller / easier to understand (it does not fundamentally alter them).
  Know when this happens.

\item After all, non-determinism is very close to the lazy approach:
  \begin{itemize}
  \item Instead of solving a problem the hard way, guess an answer; then check it
  \item Instead of trying to be smart about guessing, invent the notion of a {\em free guess}
  \item Then all we need to do is {\em check}!
  \item {\bf But the real world is cruel}. Even if you assume free guesses, some problems
    are too hard to solve. 
  \end{itemize}

\item The structure of strings packs so much ``power'' in it that by encoding problems well, you
  can use machines for tasks they were not intended to do!
  Example:
  \begin{itemize}
  \item By encoding the truth of Boolean formulae as languages in a certain way, we can
    ``teach DFA'' to become Boolean solvers!
  \item This truly illustrates the CS approach:
 {\bf Computer scientists cannot depend on nature to give them the electrons or the Benzene ring, or anything; they have
   to pretty much do it all using their own imagination.}
  \end{itemize}

\item NFA, PDA, and TM represent three ways of programming in a language with ``if-then'' and ``goto'', and using zero stacks, one stack,  or two
  stacks respectively.

\item Another CS approach (Garey and Johnson's cartoon):
  \begin{itemize}
  \item If you don't know how to solve something, at least find enough things that look similar, then
    group them and tell your boss ``look I can't solve X, but at least I've shown that X belongs
    to a large class that all these other smart people have failed to solve.'' If anyone solves one
    member of this class some day, the whole class will have been solved.
  \end{itemize}

\end{itemize}

\section{Regular Languages}

\subsection{Why are Regular Languages Crucially Important?}

Regular languages capture the simplest and most prelavent forms of patterns.
You should not ask for any form of infinite memory to recognize these patterns.


\noindent Here are examples of languages (some identified to be regular/non-regular; ohers left to you; let $\#_1(x)$ mean ``number of $1s$ in $x$''; further,
let YAP stand for ``Your Answer Please''):
%
The way you have to identify whether a language is regular or not: {\bf try to write a conditional and goto-based program without using a stack}.
%
By the way, such programs are called DFAs.
%
In Assignment 2, you are being told how we are going to mathematically represent DFA as a Python {\tt dict} object.


\begin{itemize}
\item Regular: $L_{odd1s} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;has\;odd\;number\;of\;1s\}$
\item Regular: $L_{1sMinus0sOdd} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;\mid(\#_1(x) - \#_0(x))\mid\;is\;odd\}$
  Here $\mid\ldots\mid$ denotes the absolute value
\item Non-Regular: $L_{eq1s0s} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;has\;equal\;number\;of\;0s\;and\;1s\}$
\item Non-Regular: $ L_{0n1n} = \{ 0^n 1^n \;\mid\; n\ge 0\}$
\item[]
\item ? (YAP!): $L_{thirdIs1}  = \{ x\;\mid\; x\in\{0,1\}^*\;and\;the\; third\;char\;is\;a\;1\}$
\item ? (YAP!): $L_{thirdLastIs1} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;the\; third\;from\;the\;end\;is\;a\;1\}$
\item ? (YAP!): $L_{endsWith0101} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;ends\;in\;0101\}$
\item ? (YAP!): $L_{nonObvious} = \{ x\;\mid\; x\in\{0,1\}^*\;and\;has\;odd\;1s\;or\;even\;0s\;and\;fifth\;from\;right\;is\;0\}$
\item ? (YAP!): $L_{0n1n2n} = \{ 0^n 1^n 2^n \;\mid\; n\ge 0\}$
\end{itemize}

\subsubsection{Application 1: Scanners in Various Applications}

Think of parsing Python and separate identifiers, keywords, numbers, strings (especially those nice triply quoted strings with embedded
double-qoutes and back-slaskes). 
%
This is clearly a non-trivial task -- one that calls for the {\em full power of regular expressions}.
%
You had better not miss a keyword, confuse an identifier for a number, a comment for a string, etc.!

\subsubsection{Application 2: Malware Detection in Networks}

Read the article ``Use multicore flow processing to boost network router/security
appliance throughput.
%
A copy has been kept on the class webpage in the list of references for Lecture 3
as {\tt network-flow-regexp-article.pdf}.

\section{Three Ways to Specify Regular Languages}

\begin{quote}
{\bf Regular languages are those for which there exists a Deterministic Finite Automaton (DFA)} (of course,
you can't say so and walk away; you have to draw or program one!)
\end{quote}

\noindent But we will also allow you to take other approaches to do so.
\begin{itemize}
\item {\bf Describe a DFA Grammatically!}
  The ``grammar'' that we use to write down a  DFA is variously called:
  \begin{itemize}
  \item Regular expression (RE)
  \item Right-linear grammar
  \item Left-linear grammar
  \end{itemize}
  If you take this approach, you will be asked to write down a regular expression, and demonstrate that
  you can write a program to parse a regular expression and build a DFA out of it.

\item {\bf Describe a DFA through a non-deterministic finite automaton (NFA)!}
  If you take this approach, you must demonstrate that you can convert NFA to DFA
  by writing a Python program ({\em i.e.}, algorithmically).

\item For added reassurance, if you take any of these alternative approaches, you will
  be told to further show that you can
  \begin{itemize}
  \item turn an RE into an NFA
  \item NFA into a DFA
  \item DFA into an RE
  \item RE into gold (n.b. This part is optional.)
  \end{itemize}

\end{itemize}


\subsection{Which Regular Languages are Easily Described, and Which Are Not?}

\noindent The answer will be apparent once you answer these questions:
\begin{itemize}
\item Why are these ``easy'' to describe using a DFA: $L_{odd1s}$, $L_{thirdIs1}$, $L_{endsWith0101}$.
\item Why are these not so easy (!!) to describe using a DFA (watch out for possible leg-pull from me): 
  $L_{1sMinus0sOdd}$, $L_{thirdLastIs1}$, $L_{eq1s0s}$, $L_{0n1n}$, $L_{nonObvious}$.
\item In particular, why is  $L_{thirdLastIs1}$ harder than $L_{endsWith0101}$?
\item Is $L_{0n1n2n}$ in the same language category as $L_{0n1n}$? Explain!
\end{itemize}


Once you understand these issues, you have gotten half-way to somewhat understanding DFA (!) --- naah, you are
well off to understand them!

\subsection{How to Describe Difficult (Regular) Languages?} 

First, you have to know if they are even regular (they may not be!). 
%
If you are sure that they are regular, then you must pick the best method (using RE or NFA).
%
If they are not even regular, but nevertheless you are expected to describe them, then you
should know how to classify the language.

\subsection{How to Debug DFAs?}

You will be using several methods:
\begin{itemize}
\item Write a Python program to take a DFA and help you test it (feed strings to it, and
  your program tells whether the strings are accepted or not)
\item Write a Python program to convert the DFA into a regular expression (you'll 
  pick up all you need to know about REs in a jiffy)
\item Write an NFA, compare against your DFA 
\item Write an RE, compare against your DFA
\end{itemize}
%...................................





\end{document}

%--


