\documentclass{article}

\usepackage{hevea,xspace,graphicx,hyperref}

\newtheorem{illustration}{Illustration}[section]


\newcounter{exer}
\renewcommand{\theexer}{\thechapter:\arabic{exer}}

\newcommand{\FIXIT}[1]{\typeout #1 -- {\LARGE\sf #1}}

%---
\newcommand{\TM}{{Turing machine}\xspace}
\newcommand{\TMs}{{Turing machines}\xspace}
\newcommand{\NPCness}{{NP-completeness}\xspace}
\newcommand{\NPC}{{NP-complete}\xspace}
\newcommand{\NPH}{{NP-hard}\xspace}
\newcommand{\sfP}{{\sf P}\xspace}
\newcommand{\sfNP}{{\sf NP}\xspace}
\newcommand{\sfNPC}{{\sf NPC}\xspace}
\newcommand{\sfNPH}{{\sf NPH}\xspace}
\newcommand{\sfCoNP}{{\sf CoNP}\xspace}
\newcommand{\sfCoNPC}{{\sf CoNPC}\xspace}
\newcommand{\BA}{{B\"{u}chi automata}\xspace}
\newcommand{\BAon}{{B\"{u}chi automaton}\xspace}
\newcommand{\TL}{{Temporal Logic}\xspace}
\newcommand{\LTL}{{linear-time temporal logic}\xspace}
\newcommand{\CTL}{{Computational Tree Logic}\xspace}
\newcommand{\KS}{{Kripke structure}\xspace}
\newcommand{\KSs}{{Kripke structures}\xspace}
\newcommand{\OR}{{$\omega$-regular}\xspace}
%---


\oddsidemargin 0cm
\textwidth 17cm

\begin{document}


\title{CS 3100 -- Models of Computation -- Fall 2011 --  Notes for L2}
\date{August 27, 2011}

\maketitle

\section{Preliminaries}

Please get set up to learn (or refresh on) Unix; plenty of resources
are available on the web ({\em e.g.}, \\
\url{http://www.ee.surrey.ac.uk/Teaching/Unix/}).
%
Then please get set up to use Python (available as \verb|python3| on our CADE machines).


A very nice tutorial on Python (with plenty of good exercises) appears 
at \url{http://www.python-course.eu/index.php}.
%
In fact, this is one page where I found a lot of useful material about
Python3.
%
Some additional details are at \url{http://www.python.org}
and \url{http://docs.python.org/py3k/}.

\section{A Tour of the Python Subset of Interest}

We shall use a limited subset of Python initially, and we describe this
subset in the order that best matches upcoming material.

\subsection{Indentation}
\label{sec:Indentation}

The indentation rules of Python are very important to understand.
%
Read about recommended indentation styles (and coding-styles in
general) at \url{http://www.python.org/doc/essays/styleguide.html}.

\subsection{Characters}
\label{sec:Characters}

Characters are entered as follows against the Python prompt \verb|>>>|:


\begin{footnotesize}
\begin{verbatim}
>>> 'a'
'a'              >>> '"'
                 '"'
>>> "a"
'a'              >>> ord('a')
                 97
>>> """a"""
'a'              >>> chr(97)
                 'a'
>>> "'"
"'"              >>> chr(ord('a')+1)
                 'b'
\end{verbatim}
\end{footnotesize}

\subsection{Strings}
\label{sec:Strings}

Strings are entered as follows. Note how ``raw'' strings are entered and
the convention pertaining to \verb|\|, esp. at the end of a string.

\begin{footnotesize}
\begin{verbatim}
>>> t='abcdef'                    >>> r"a\a"
                                  'a\\a'                  >>> for x in "abcd":
>>> t                                                           print(x)
'abcdef'                          >>> s=r"a\a"            .. prints a, b, c, d
                                  >>> s
>>> t[::2]                        'a\\a'                  >>> len("abcd")
'ace'                                                     4
                                  >>> s=="a\a"
>>> t[::-1]                       False
'fedcba'                                                  s= """hello, this is
                                  >>> s==r"a\a"           ... a multi-line sentence.
>>> t='abcdef'                    True                    ... """

>>> t[2:]                         >>> s[0]                >>> s
'cdef'                            'a'                     s
                                                          'hello, this is \na multi-line sentence.\n'
>>> t[:2]                         >>> s[1]                >>>
'ab'                              '\\'

>>> t[:2] + t[2:]                 >>> s[2]
'abcdef'                          'a'

>>> t[:]                          >>> s[0:]
'abcdef'                          'a\\a'

>>> t[:2] + 'z' + t[3:]           >>> s[1:]
'abzdef'                          '\\a'

>>> "aaa"                         >>> s[2:]
'aaa'                             'a'

>>> "a\a"                         >>> t=r"a\b"
'a\x07'
                                  >>> t
>>> "a\a"                         'a\\b'
'a\x07'
                                  >>> t[::2]
>>> "a"*5                         'ab'
"aaaaa"



\end{verbatim}
\end{footnotesize}

About raw strings ending in backslash, see\\

\verb|http://stackoverflow.com/questions/647769/..|

\verb|why-cant-pythons-raw-string-literals-end-with-a-single-backslash|


\subsection{Lists}
\label{sec:Lists}

\begin{footnotesize}
\begin{verbatim}
>>> lst = ['a', 'b']
                                         >>> alias = n09
>>> lst[0]
'a'                                      >>> alias
                                         [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]           >>> n09[0] = 100

>>> n09 = [x for x in range(10)]         >>> alias
                                         [100, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> n09
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]           >>> n09 = [x for x in range(10)]

>>> n09[0]=100                           >>> n09[2:]
                                         [2, 3, 4, 5, 6, 7, 8, 9]
>>> n09
[100, 1, 2, 3, 4, 5, 6, 7, 8, 9]         >>> acopy = n09[:]  # Make a deep copy

>>> n09 = [x for x in range(10)]         >>> acopy
                                         [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> n09
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]           >>> n09[0] = 100
                                         >>> acopy
                                         [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> f = lambda x: x+1

>>> f
f
<function <lambda> at 0x1005eb5a0>

>>> list(map(f, acopy))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

>>> ["a"*i + "b"*j + "c"*k for i in range(3) for j in range(3)
        for k in range(3) if ((i != 1) or (j == k)) ]

['', 'c', 'cc', 'b', 'bc', 'bcc', 'bb', 'bbc', 'bbcc', 'a', 'abc', 'abbcc', 'aa',
   'aac', 'aacc', 'aab', 'aabc', 'aabcc', 'aabb', 'aabbc', 'aabbcc']

>>> f = lambda x, y: x+y

>>> acopy
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> functools.reduce(f, acopy)
functools.reduce(f, acopy)
45

>>> odd = lambda x: x%2 == 1

>>> odd(2)
False

>>> odd(3)
True

>>> list(filter(odd,acopy))
[1, 3, 5, 7, 9]

>>> [[1,2],3][0]
[1,2]

>>> [[1,2],3][0][1]
2

>>> [a,b] = [1,2]

>>> a
1

>>> b
2

\end{verbatim}
\end{footnotesize}

\subsection{Sequences}
\label{sec:Sequences}

\begin{footnotesize}
\begin{verbatim}
(1,2)
(1, 2)                   >>> ((1,2),3)[1]
                         3
>>> (1,2)[0]
1                        >>> (a,b) = (1,2)

>>> ((1,2),3)[0][0]      >>> a
1                        1

>>> ((1,2),3)[0][1]      >>> b
2                        2

>>> mixList = [ [], (1,2), (), "aa", ("aa", "bb", 2), 12, [1, (1,2)]]

>>> list(zip([1,2], [4,5]))
[(1, 4), (2, 5)]

\end{verbatim}
\end{footnotesize}

\subsection{Sets and Dicts (or Hash-maps)}
\label{sec:SetsDicts}

Set representation involves hashing (as with Dicts), and so the members of a set
must be ``hashable.''
%
Thus unfortunately we can't have sets of sets or sets of lists, but we can have lists of
sets or lists of lists.
%
It is also good to disambiguate whether we are making sets or dicts---for instance,
instead of just saying \verb|{}|, say \verb|set({})| or \verb|dict({})|.
%
We can also iterate over items of a set as well as build sets using comprehension.
%
However, the moment we

\begin{footnotesize}
\begin{verbatim}
>>> set({})
set()

>>> dict({})
{}

>>> {1,2,3,4,1,2,3}
{1, 2, 3, 4}

>>> {'a', (1,2), 'b', (1,2)}
{(1, 2), 'a', 'b'}

>>> (1,2) in {(1, 2), 'a', 'b'}
True

>>> d1 = {11 : 1122, 2234 : 33, 34343 : 'ee', "adf" : 222 }

>>> d1.keys()
dict_keys(['adf', 2234, 11, 34343])

>>> d1.values()
dict_values([222, 33, 1122, 'ee'])

>>> d1.items()
dict_items([('adf', 222), (2234, 33), (11, 1122), (34343, 'ee')])

>>> list(d1.items())
[('adf', 222), (2234, 33), (11, 1122), (34343, 'ee')]

>>> set(d1.items())
{('adf', 222), (11, 1122), (34343, 'ee'), (2234, 33)}

>>> d1['adf']
222

>>> d1.update({'aaaaa' : 11111, 'bbbbb' : 99999 })

>>> d1
d1
{'adf': 222, 34343: 'ee', 'aaaaa': 11111, 2234: 33, 11: 1122, 'bbbbb': 99999}

\end{verbatim}
\end{footnotesize}

\subsection{Variable Scoping, Function Definitions}

Most of your work for this class will consist of definition short functions in a file and importing
the files.
%
You can place files in sub-directories also.
%
To import everything from a file \verb|foo| say \verb|from foo import *|, and for importing from sub-directory
\verb|bar| of file \verb|foo|, say \verb|from foo.bar import.*|.
%
After that, you can do \verb|dir()| to list all the names in the scope (top-level global variables and function names, mainly)
or \verb|dir("foo")| to know the names in \verb|foo|, or \verb|dir(["foo", "bar"])| for all the names in \verb|foo| and \verb|bar|.
%
After declaring the function name and arguments, one has to (as a matter of style) provide a documentation string, which automatically
gets incorporated into the \verb|help(..)| command.
%
All these ideas are now illustrated.

\begin{scriptsize}
\begin{verbatim}
# Try all these commands yourself, assuming that you have a file called lang.py and nfa.py
# If not, put in some functions yourself. Also import ply-3.4

from lang import *             # Import all names from lang.py
from nfa import *              # Similar to the above
dir("lang")                    # List names in lang.py
dir()                          # List of all names available to Python
dir(["lang", "nfa"])           # Names in these two modules
import ply.yacc                # Import from ply/yacc.py
import ply.yacc as yacc        # Alias the module as 'yacc' here, including for dir(yacc)

def cat(L1,L2):
    """Concat two languages
    cat(a,b) --> set(['abab', 'bc22', 'ab11', 'ab22', 'bcab', 'bc11']) where a=set(['ab', 'bc']) b=set(['11', 'ab', '22'])
    """
    return set([x+y for x in L1 for y in L2])

# Now you can type help(cat) and see the above documentation listed. Using triple-quotes allows the newlines to be preserved.

\end{verbatim}
\end{scriptsize}

\subsection{Reading and Writing from Keyboard,  to Console,  from/to Files}
\label{sec:Printing}

There are so many ways to print things, either to files or to the standard output.
%
{\em Pretty much all methods of printing to the console also work with files.}
Here are examples of dealing with files. 
The rest of the document deals mostly with console I/O.


\begin{description}
\item[Reading from a file:] {\tt fHandle = open(fileNameString)}, then {\tt fHandle.read()}.
  You'll also find additional examples of console I/O as part of asg2.pdf which is already posted.

\item[Writing to a file:] {\tt fHandle = open(fileNameString, 'w')}, then {\tt print(stringToPrint, file=fHandle)}.
  You'll also find additional examples of console I/O as part of asg2.pdf which is already posted.
\end{description}


\begin{footnotesize}
\begin{verbatim}
print(33) - prints 33 and puts a \n

print(33, end='') - prints 33 and ends the line with a '' i.e. does not put a \n
\end{verbatim}
\end{footnotesize}

%
One could use \verb|print| statements accompanied by format specifiers, or one could form strings to be printed and print them.
%
Here is an example of a print with format involved.
%
This example prints two items---a string and a number.
%
The minimum field width is specified as 10 for the string and 9 for the number.

\begin{footnotesize}
\begin{verbatim}
print('{0:10} --> {1:9d}'.format("history", 12))

This gives
history    -->        12
\end{verbatim}
\end{footnotesize}

Another way to print things in a list separated by a given separator is shown below.
%
This takes a list of things and converts each to a string, with {\em each} item separated by a tab.
%
To that string, we attach two more tabs, and a tail string \verb|tailstr|.

\begin{footnotesize}
\begin{verbatim}
  print("separator".join(map(str, list_of_things)) + "\t\t" + tailstr)

For example,

print("~\b~".join(map(str, [{'a'}, 10, {'a':10}, {1:20}, []])) + " - " + " >>")

produces

  {'a'}~10~{'a': 10}~{1: 20}~[] -  >>
\end{verbatim}
\end{footnotesize}

The cool thing is that the \verb|format| modifier function applid to strings formats the string in a 
given way (much like Unix's \verb|sprintf|). 
%
Here is an interesting example putting these ideas together:


\begin{footnotesize}
\begin{verbatim}
fm = lambda x: '{0:3d}'.format(x)  # each item is the "zeroth" item, hence "0:"

list(map(fm, [1,2,30,400]))

produces

['  1', '  2', ' 30', '400']
\end{verbatim}
\end{footnotesize}

\subsection{Higher Order Formatter Functions}

\begin{footnotesize}
\begin{verbatim}

>>> num_in_width = lambda w: (lambda x: ('{0:' + str(w) + '}').format(x))

>>> num_in_width(4)
<function <lambda> at 0x1005f12f8>

>>> w4 = num_in_width(4)

>>> w5 = num_in_width(5)

>>> w3 = num_in_width(3)

>>> list(map(w3, [1, 20, 300, 4000]))

['  1', ' 20', '300', '4000']

>>> list(map(w4, [1, 20, 300, 4000]))

['   1', '  20', ' 300', '4000']

>>> list(map(w5, [1, 20, 300, 4000]))

['    1', '   20', '  300', ' 4000']

\end{verbatim}
\end{footnotesize}

Now we can write a function to print Pascal's triangle by adjusting the print
width depending upon the magnitude expected.

\subsection{Illustration}
\label{sec:Illustration}

Let us write a recursive program to generate the rows of Pascal's triangle. Then let us modify
this program to print the rows of the Pascal's triangle.

Let's study an example Pascal's triange:

\begin{footnotesize}
\begin{verbatim}
                                   1                                

                               1       1                            

                           1       2       1                        

                       1       3       3       1                    

                   1       4       6       4       1                

               1       5      10      10       5       1            

           1       6      15      20      15       6       1        

       1       7      21      35      35      21       7       1    
\end{verbatim}
\end{footnotesize}

From this, we can think of an algorithm to derive the rows, starting from the first row which is just \verb|1|
\begin{itemize}
\item Take the previous row and make two copies, one with a 0 to the left and one with a 0 to the right; for example \verb|0 1|
and \verb|1 0|. Add the rows. You now obtain the next row!

\item Do this \verb|N| times.
\end{itemize}

\noindent Example of how this idea can be developed:

\begin{footnotesize}
\begin{verbatim}

L = [1]

>>> La = [0]+L

>>> Lb = L + [0]

>>> zip(La, Lb)

>>> list(zip(La, Lb))
[(0, 1), (1, 0)]

>>> f = lambda x: x[0]+x[1]

>>> list(map(f, zip(La, Lb)))
[1, 1]
\end{verbatim}
\end{footnotesize}

Based on this study, we can develop a ``row to new row'' function:

\begin{footnotesize}
\begin{verbatim}
def row2nrow(row):
    return list(map(lambda x: x[0]+x[1], zip([0] + row, row + [0])))
\end{verbatim}
\end{footnotesize}

\noindent Based on this, we can finally present the whole solution:

\begin{footnotesize}
\begin{verbatim}
def pascal(N):
    return dopasc([[1]], N)

def dopasc(Ll, N):
    return Ll[::-1] if N==1 else dopasc( [ row2nrow(Ll[0]) ] + Ll, N-1)

def row2nrow(row):
    return list(map(lambda x: x[0]+x[1], zip([0] + row, row + [0])))    
\end{verbatim}
\end{footnotesize}

\noindent Let's see how things work:

\begin{footnotesize}
\begin{verbatim}
>>> pascal(3)

[[1], [1, 1], [1, 2, 1]]

>>> pascal(5)

[[1], [1, 1], [1, 2, 1], [1, 3, 3, 1], [1, 4, 6, 4, 1]]

\end{verbatim}    
\end{footnotesize}

OK things seem to be working; but how to nicely format things so that we can obtain

\begin{footnotesize}
\begin{verbatim}
def pascal(N):
    pascRows = dopasc([[1]], N)
    for line in list(map(lambda x: prRow(x, N), pascRows)):
        print(line)

def prRow(L, N):
    sp = 4 if N >= 5 else 3
    n = len(L)
    side_space = (" " * sp) * (N-n+1)
    mid_item = (" " * sp).join( list(map(num_in_w(sp), L)) )
    return "\n" + side_space + mid_item + side_space    

def dopasc(Ll, N):
    return Ll[::-1] if N==1 else dopasc( [ row2nrow(Ll[0]) ] + Ll, N-1)
    
def row2nrow(row):
    return list(map(lambda x: x[0]+x[1], zip([0] + row, row + [0])))

def num_in_w(w):
    return (lambda x: ('{0:' + str(w) + '}').format(x))
\end{verbatim}
\end{footnotesize}

\clearpage

\section{Strings and Languages}

With some functional programming under our belts, we can now embark on studying
one of the first topics---namely, {\em strings} and {\em languages}.
%
I'll try my best to introduce Python encodings of concepts in {\sf sf} font.

\subsection{Alphabet}

An {\em alphabet}, $\Sigma$, is a finite set of {\em symbols}.
%
In most cases, symbols will be characters.
%--
Alphabets are always non-empty.
%
\begin{sf}
\noindent In Python, we will employ sets of strings, each of length 1 to denote alphabets. 

\noindent Example: \verb|Sigma = {'a', 'b', 'c'}|
\end{sf}


\subsection{Strings and Languages}

Strings are a  {\bf finite} concatenation of symbols (a finite sequence of characters).
%
Strings are read {\em left-to-right}.
%
We write the empty string $\varepsilon$.


\begin{itemize}
\item Languages are sets of strings -- ({\bf finite}) strings for emphasis (for the last time!).
\item {\bf Languages can be infinite!} In fact, most languages that we study will be infinite.
\item So remember: 
  \begin{quote}
Almost all languages that we deal with are {\em infinite sets of finite strings.}    
  \end{quote}
\end{itemize}


\noindent Two examples:
\begin{itemize}
\item[] \( Mylang = \{\varepsilon,  aa, abc\} \)
\item[] \( La\_lt\_b = \{ a^i b^j \;\mid\; i, j \ge 0, and\; i < j \} \)
\end{itemize}
\noindent By saying $i, j > 0$ we will tacitly assume that $i$ and $j$ are 
natural numbers.
%
Since for every natural number $p$  there is a natural number $q > p$,
we can easily conclude that $La\_lt\_b$ is infinite.

\begin{bf}
Note that most languages discussed in Automata theory are infinitary. 
When we ``simulate'' these languages in Python, we will have to obtain
finite approximations. 
This is a reasonable compromise in my opinion.
%
The use of Python helps you understand {\em roughly} what 
is going on; for details, refer to the math!
\end{bf}


\begin{sf}
\noindent In Python, we will employ sets of strings to denote languages.
We will use \verb|''| to denote $\varepsilon$.

\begin{footnotesize}
\begin{verbatim}

Mylang = {'', 'aa', 'abc'} is a language over the above Sigma

La_lt_b_9 = { "a"*i + "b"*j for i in range(10) for j in range(10) if i < j }

>>> La_lt_b_9  # Approximated to <= 9 i's and j's

{'aaaaaabbbbbbbbb', 'abbbbb', 'aaabbbbbbbb', 'aabbbbb', 'aabbb',
'aaaabbbbbbbbb', 'bbbbbbbb', 'aaabbbbbbbbb', 'aaabbbbbbb', 'aaabbbbb',
'bbbbbbb', 'abb', 'aaaaaaaabbbbbbbbb', 'abbbbbb', 'aaaabbbbbb', 'bbb',
'aaaabbbbbbb', 'aabbbbbbb', 'aaaabbbbb', 'aaaaabbbbbbbbb',
'aabbbbbbbbb', 'abbbbbbbbb', 'aaaaaabbbbbbbb', 'abbb', 'bb',
'aaaaaabbbbbbb', 'abbbbbbb', 'aaaaaaabbbbbbbb', 'aabbbbbb',
'aaaaabbbbbbbb', 'aaaaabbbbbbb', 'bbbbbbbbb', 'bbbbb', 'aaabbbb', 'b',
'abbbb', 'abbbbbbbb', 'bbbbbb', 'bbbb', 'aaabbbbbb', 'aaaabbbbbbbb',
'aaaaabbbbbb', 'aabbbbbbbb', 'aaaaaaabbbbbbbbb', 'aabbbb'} 
\end{verbatim}
\end{footnotesize}
\end{sf}

Strings and languages are {\em fundamental} to computer
science theory.
%--
We will primarily be studying
{\em patterns} described by strings.
%
Many operations are defined on strings: (i)~One can take substrings of a given string;
(ii)~Strings may also be concatenated or reversed.
%
More operations on strings are defined later.

\vspace{1ex}

\noindent Drill problems:
\begin{itemize}
\item Write a one-line set comprehension to
  generate the set of all substrings of \verb|s="abc"|.
  You can assume that \verb|s="abc"| is a statement issued
  prior to your set comprehension.

\item Is this true in Python for two strings p and q? Explain!

\verb|p+q == (q[::-1] + p[::-1])[::-1]|

\end{itemize}

\subsection{Operations on Languages}

\noindent We now list some of the basic operations on languages.
%
Some of these are constant operations (meaning, they denote
constant languages).

\begin{list}{$\bullet$}{\addtolength{\itemsep}{-1.6\itemsep}}
\item {\em Empty Language (or Zero Language):} 
	\index{Language!empty}
	\( \emptyset\;\; or\;\; \{ \} \)

        We call it the ``zero'' language because it is like the
        $0$-element for concatenation.
        \begin{footnotesize}
\begin{verbatim}
def Phi():
    """This is the ZERO language for concatenation (viewed as multiplication)
    """
    return set() # Don't write it as {} because this is ambiguous - dict/set 
\end{verbatim}
        \end{footnotesize}

\item {\em Unit Language:} 
	\index{Language!unit}
	\( \{ \varepsilon \} \)

        We call it the ``unit'' language because it is like the unit element
        for concatenation.
        \begin{footnotesize}
\begin{verbatim}
def Unit():
    """This is the UNIT language for concatenation (viewed as multiplication)
    """
    return {""} # Set with epsilon
\end{verbatim}
        \end{footnotesize}

\item {\em Concatenation:}
  \index{Language!concatenation}
	\( L_1 L_2 = \{ x y \;\mid\; x\in L_1 \wedge y\in L_2 \} \)
        \begin{footnotesize}
\begin{verbatim}
def cat(L1,L2):
    """Concat two languages
    cat(a,b) --> set(['abab', 'bc22', 'ab11', 'ab22', 'bcab', 'bc11']) 
    where a=set(['ab', 'bc']) b=set(['11', 'ab', '22'])
    """
    return set({x+y for x in L1 for y in L2})
\end{verbatim}
        \end{footnotesize}

\item {\em Exponentiation:}
	\index{Language!exponentiation}
	\( L^0 = \{ \varepsilon \}\; and \; L^n = L L^{n-1} \)
        \begin{footnotesize}
\begin{verbatim}
def exp(L,n):
    """Exponentiate a language
    exp(a,2) --> set(['abab', 'bcab', 'bcbc', 'abbc']) where  a = set(['ab', 'bc'])
    """
    return Unit() if n == 0 else cat(L, exp(L, n-1))
\end{verbatim}
        \end{footnotesize}


\item {\em Union:}
	\index{Language!union}
	\( L_1 \cup L_2 = \{ x \;\mid\; x\in L_1 \vee x\in L_2 \} \)
        \begin{footnotesize}
\begin{verbatim}
def lunion(L1,L2):
    """Language union
    """
    return L1 | L2
\end{verbatim}
        \end{footnotesize}


\item {\em Star:}
  We introduce {\em Star} in stages. 

\noindent First let us introduce $L^{*}_{n}$:
	\index{Language!star}
	\( L^{*}_{n} = L^n \; \cup \; L^{*}_{n-1} \), with 	\( L^{*}_{0} = \{\varepsilon\} \)

        {\em i.e.},

	\( L^{*}_{n} = L^n \; \cup \; L^{n-1} \; \cup \; \ldots \; \cup \; L^1 \; \cup \; L^0 \)
\noindent The following encodes 	\( L^{*}_{n} \).
        \begin{footnotesize}
\begin{verbatim}
def star(L,n):
    """Star a language, bounding the iteration to the given n
    star(a,2) --> set(['abab', 'bcbc', 'ab', 'abbc', '', 'bc', 'bcab']), 
    where  a = set(['ab', 'bc'])
    """
    return Unit() if n == 0 else exp(L,n) | star(L,n-1)
\end{verbatim}
        \end{footnotesize}

\noindent Now $L^{*} = L^{*}_{\infty}$, {\em i.e.}, 
	\( L^{*} = L^0 \; \cup \; L^1 \; \cup \; L^2 \; \cup \ldots \)

\item[]


\item {\em Reverse:}
	\index{Language!reversal}
	\( rev(L) = \{ rev(s) \;\mid\; s\in L \} \)
        \begin{footnotesize}
\begin{verbatim}
def revl(L):
    """Reverse a language.
       revl(set(['ab', 'bc'])) --> set(['cb', 'ba'])
    """
    return set(map(lambda x: x[::-1], L))
\end{verbatim}
        \end{footnotesize}


\item {\em Complementation:}
Complementation of any set is with respect to a ``universe'' (or 
universal set). 
%--
For language complementation, the universe is $\Sigma^*$.
%
Now define the complementation of a language $L$ with respect to that universe:

\[ \overline{L} = \{ x \;\mid\; x\in (\Sigma^* \setminus L) \}. \]

Again, language complements can be (and usually are) infinitary.
%
For ``simulating it in Python,'' we need to bound complements:

\begin{footnotesize}
\begin{verbatim}
def lcomplem(L,alph,m):
    """Complement L relative to alphabset alph. alph is also given as a set of strings.
       We subtract from the "star up to m" of the alphabet alph, the language L.
    """
    return star(alph,m) - L
\end{verbatim}
\end{footnotesize}

\item {\em Homomorphism on a string:}
	\index{String!homomorphism}
        Given a string belonging to $\Sigma^{*}$ (a ``string
        over $\Sigma^*$''), a function $h$ from domain
        $\Sigma^*$ to range $\Gamma^*$ is called a {\em homomorphism}
        if it respects two conditions:
        \begin{itemize}
        \item $h(\varepsilon) = \varepsilon$
        \item $h(xy) = h(x) h(y)$
        \end{itemize}
        \begin{footnotesize}
\begin{verbatim}
def homos(S,f):
    """String homomorphism wrt lambda f
       homos("abcd",hm) --> 'bcde'  where hm = lambda x: chr( (ord(x)+1) % 256 )
    """
    return "".join(map(f,S))
\end{verbatim}
        \end{footnotesize}


\item {\em Homomorphism on a language:}
	\index{String!homomorphism}
        Given a homomorphism from
        $\Sigma^*$ to range $\Gamma^*$, it can be applied to
        a language $L\subseteq \Sigma^{*}$ to produce
        a language $G\subseteq \Gamma^{*}$, and is defined 
        in the obvious manner:

        \( h(L) = \{h(x) \;\mid\; x\in L\} \)

        \begin{footnotesize}
\begin{verbatim}
def homol(L,f):
    """Language homomorphism wrt lambda f.
       homos("Hello there", rot13) --> 'Uryy|-\x81ur\x7fr'
       where rot13 = lambda x: chr( (ord(x)+13) % 256 )
    """
    return set(map(lambda S: homos(S,f), L))
\end{verbatim}
        \end{footnotesize}
        

\item {\em Intersection:}
	\index{Language!intersection}
	\( L_1 \cap L_2 = \{ x \;\mid\; x\in L_1 \wedge x\in L_2 \} \)
        \begin{footnotesize}
\begin{verbatim}
def lint(L1,L2):
    """Language intersection
    """
    return L1 & L2
\end{verbatim}
        \end{footnotesize}

\item {\em Language Subtraction:}
	\index{Language!setminus}
	\( L_1 \setminus L_2 = \{ x \;\mid\; x\in L_1 \wedge x\notin L_2 \}\)
        \begin{footnotesize}
\begin{verbatim}
def lminus(L1,L2):
    """Language subtraction. Can do it as L1.difference(L2) also. 
    """
    return L1 - L2
\end{verbatim}
        \end{footnotesize}

\item {\em Symmetric difference:}
	\index{Language!symmetric difference}
	\( (L_1\setminus L_2)\cup (L_2\setminus L_1) \)
        \begin{footnotesize}
\begin{verbatim}
def lsymdiff(L1,L2):
    """Language symmetric difference.
      lsymdiff(a,b) where a=set({'ab', 'bc'}) b=set({'11', 'ab', '22'}) --> set({'11', '22', 'bc'})
    """
    return L1 ^ L2
\end{verbatim}
        \end{footnotesize}

\end{list}

\subsection{Lexicographic Orders}

``Lexicon'' means ``dictionary'' and so ``lexicographic order''
 means ``order as in a dictionary.''
%
Is {\tt pig} before {\tt poblano} in a dictionary? Yes, but Why? Answer: Note that 
we see {\tt i} before {\tt o} and so quit bothering the comparison
of {\tt g} and {\tt b}.
%
This is the basic idea behind lexicographic ordering.

Mathematically speaking, two strings $s$ and $t$ are in lexicographic 
order $s \leq t$ if $s[i] == t[i]$  holds up to a point, and we
find $s[i+1] \leq t[i+1]$. 
%
We don't care what holds beyond $i+1$.

\noindent \verb|lexlt2| for two strings \verb|s| and \verb|t| can be defined as follows:
\begin{verbatim}
def lexlt2(s, t):
    if (s==""):
        return True
    if (t==""):
        return False
    if (s[0] < t[0]):
        return True
    return (s[0] == t[0]) & lexlt2((s[1::], t[1::]))
\end{verbatim}

\noindent We can also define \verb|lexlt(p)| for a pair of strings \verb|p| as follows (the
indexing becomes a bit trickier; first you index to get strings out of a pair; then index
the strings themselves):
\begin{verbatim}
def lexlt(p):
    if (p[0]==""):
        return True
    if (p[1]==""):
        return False
    if (p[0][0] < p[1][0]):
        return True
    return (p[0][0] == p[1][0]) & lexlt((p[0][1::], p[1][1::]))
\end{verbatim}



\vspace{1ex}
\noindent Drill problem:
Given the following languages

\begin{verbatim}
L1 = {"abacus", "bandana", "pig", "cat", "dodo", "zulu", "physics"}

L2 = {"dog", "zebra", "zzxyz", "pimento"}
\end{verbatim}
Define a function that lists the set of pairs from L1 and L2 which are 
in the lexicographically less-than order.

\subsection{Numeric Order}

One problem with lexicographic ordering is the following.
%
Suppose you are given $\Sigma=\{a,b\}$ and are asked to
list all the strings in $\Sigma^*$
in lexicographic order such that a string containing $b$
is listed in a finite amount of time, what will your reaction be?
%
The answer to this question motivates the notion of a numeric order.

According to
the numeric order, one lists all strings of a length-group in lexicographic
order before moving on to the next length group.
%
We process all strings  in length-group $0$ (how many strings are in that
group?), then move on to length-group $1$, and so on.

\vspace{1ex}

\noindent Drill Problem: Here is a function that generates the $N$th string
in numeric order. Your bonus reward is to de-obfuscate this function!
(It is not obfuscated although it may appear so; you just gotta know your
binary numbers.)

\begin{footnotesize}
\begin{verbatim}
from math import *

def nthnumeric(N):
    """Assume that Sigma is {a,b}. Produce the Nth string in numeric order, where N >= 0.
    Idea : Given N, get b = floor(log_2(N+1)) - need that many places; what to
    fill in the places is the binary code for N - (2^b - 1) with 0 as a and 1 as b.    
    """
    if(N==0):
        return ''
    else:
        width = floor(log(N+1, 2))
        tofill = int(N - pow(2, width) + 1)
        relevant_binstr = bin(tofill)[2::] # strip the 0b leading string
        len_to_makeup = width - len(relevant_binstr)
        return "a"*len_to_makeup + homos(relevant_binstr, lambda x: 'b' if x=='1' else 'a')
\end{verbatim}  
\end{footnotesize}


\subsection{Illustration of Homomorphisms}

We now present two illustrations of homomorphisms (you are asked to 
verify that they are homomorphisms).

\subsubsection{Changing labels so that `dot' can print it}

In printing DFA using \verb|dot|, I had to change state names 
to conform to a legal \verb|dot| syntax.
%
Here is the mapping I devised, where by ``dotsan'' I mean ``dot's sanity.''
%

\vspace{1ex}

\noindent Drill Problem: Is \verb|dot_san_str| a homomorphism? Argue
formally!


\begin{footnotesize}
\begin{verbatim}
def dotsan_map(x):
    """A homomorphism? Let the student answer!
    """
    if x in {  "{",   " ",   "'",   "}" }: 
        return ""
    elif x == ",":
        return "_"
    else:
        return x

def dot_san_str(S):
    """Make dot like strings which are in set of states notation.
    """
    return homos(S, dotsan_map)
\end{verbatim}
\end{footnotesize}

\subsubsection{Parsing without first Lexing}


In parsing a programming language such as C, the parser appeals
to a {\em tokenizer} that first recognizes the structure of
{\em tokens} (sometimes called {\em lexemes}) such as
integers, floating-point numbers, variable declarations,
keywords, and such.
%
The tokenizer pattern-matches according to
regular expressions, while the parser analyzes the structure of
the token stream using a context free grammar.
%
Suppose one wants to get rid of the tokenizer and write a context free grammar up to the level
of individual characters.
%
Such a grammar can be obtained by substituting
the character stream corresponding to each token
in place of each token in a {\em modular} fashion, according to a
homomorphism.
%
For example, if {\tt begin} were to be a keyword in the language, instead of treating it as a
token {\em keyword}, one would introduce additional productions of the form
%
\begin{quote}
\begin{verbatim}
begin -> b e g i n
\end{verbatim}
\end{quote}
%
Thanks to the {\em modular} nature of the substitutions,
it can be shown that
the resulting grammar would also be context-free.
%
There is a formal result to this effect that CFLs remain CFLs under
homomorphisms.

\subsection{Prefix-closure}

A language $L$ is said to be \index{Closure!prefix}
	prefix-closed if for every string $x\in L$,
every prefix of $x$ is also in $L$.
%--
If we are interested in {\em every} run of a machine, then
its language will be prefix-closed.
%--
This is because a physically realizable string processing machine must
encounter substrings of a string before it encounters the whole string.
%--
{\em Prefix closure} is an operation that `closes' a set by putting
in it all the prefixes of the strings in the set.
%--
For instance, given the set of strings $\{a,aab, abb\}$, its {\em prefix closure} is
$\{\varepsilon,a,aa,ab,aab,abb\}$.

\vspace{1ex}

\noindent Drill Problem: We have already asked you to write a
Python function to generate the prefix-closure of a language.
%
Please refer to that exercise, and $\ldots$ do it!

%...................................





\end{document}

%--


