Formally Defining Computation

Gregory M. Kapfhammer

September 1, 2025

Course objectives

• What can be computed?
  • In principle: undecidability and uncomputability
  • Efficiently in practice: complexity theory
  • Formal mathematical foundations for computation

• Why study computational theory?
  • Determine if problems are computable and tractable
  • Find suitable variants when problems are intractable
  • Compare efficiency of proposed solution methods
  • Apply techniques like reductions and automata theory

Why theory of computation matters

• Practical utility
  • Understand whether problems are computable and tractable
  • Find suitable variants or approximations when needed
  • Compare efficiency of different solution methods
  • Apply specific techniques with real applications

• Intellectual foundation
  • Beautiful and important theoretical framework
  • Deeper understanding of computer science discipline
  • Rigorous mathematical basis for computation
  • Connect abstract concepts to practical programming

Your proofgramming journey

Study Chapters 1 and 2 of “What Can be Computed” and grasp SISO programming

Experiment with the WCBC code examples and create your own SISO programs

• Focus areas for week two:
  • SISO program implementation and analysis
  • Decision problems and their computational properties
  • String processing and pattern recognition algorithms
  • Connection between programming and mathematical theory
  • Preparation for more advanced computational models

Computational problem classification

Category	Tractable	Intractable	Uncomputable
Description	Can be solved efficiently	Method exists but hopelessly time-consuming	Cannot be solved by any program
Theory	✓	✓	✗
Practice	✓	✗ (?)	✗
Example	Shortest route	Decryption	Finding all bugs

Understanding these categories helps proofgrammers determine the computational feasibility of problems and select appropriate solution strategies. Proofgrammers understand that not all problems are computable and that some problems are intractable! Be on alert!

Using Python for theoretical machines

• Introduce a restricted form of Python programs
• Give examples of these restricted programs
• Formally define the term “Python program”
• Introduce concept of a decision problem

Python basics for proofgrammers

• utils package: provided with WCBC online resources
• rf function: reads files from disk, used frequently
• def keyword: defines functions like procedures in other languages
• in operator: works on many data types, tests substring presence
• String quotes: use single or double quotes for strings
• Code blocks: defined by indentation, no curly braces needed

These Python fundamentals enable us to implement theoretical concepts as executable programs that demonstrate computational principles. We use Python programs to illuminate computational theory!

Introducing SISO programs

• Single Input, Single Output (SISO)
  • Takes exactly one string as input parameter
  • Returns exactly one string as output result
  • Foundation for formal computational models
  • Enables precise mathematical analysis

• Decision problems
  • Return “yes” or “no” to answer specific questions
  • Fundamental building blocks of computational theory
  • Enable formal proofs about computability
  • Connect to language recognition problems

SISO program characteristics

• Required
  • Input specification: single string parameter only
  • Output format: single string return value
• Desirable
  • Function definition: clear computational procedure
  • Deterministic behavior: same input always produces same output
  • Finite execution: program must terminate for valid inputs

SISO programs provide the mathematical foundation for analyzing what can be computed and how efficiently we can solve different types of problems. They are a powerful syntactic subset of Python! Go SISO!

Why is SISO suitable?

• Wait, real-world programs receive many kinds of input and output!
• But our theoretical results for SISO programs can be easily generalized to all other types of computer programs
  • All input and output to computers ultimately consists of binary
  • Binary strings can easily be converted to and from ASCII strings
• The restriction to Python is convenient for us, but not strictly needed
• Python programs are equivalent to other standard models of computation, including Turing machines and all other “reasonable” programming languages and realistic computers
• Unless otherwise specified, we use programs that are SISO!

Create the containsGAGA function

def containsGAGA(input_string: str) -> str:
    """Check if the input string contains the substring GAGA."""
    if "GAGA" in input_string:
        return "yes"
    else:
        return "no"

# test the function with sample inputs
test_input1 = "HELLO GAGA WORLD"
test_input2 = "programming theory"
test_input3 = "AGAGAGAGA patterns"

print(f"Testing '{test_input1}': {containsGAGA(test_input1)}")
print(f"Testing '{test_input2}': {containsGAGA(test_input2)}")
print(f"Testing '{test_input3}': {containsGAGA(test_input3)}")

Testing 'HELLO GAGA WORLD': yes
Testing 'programming theory': no
Testing 'AGAGAGAGA patterns': yes

Try the containsGAGA function

More sophisticated programs

Reading programs from files

Recap these two programs

• Recap for the containsGAGAnotHAHA function:
  • Complex conditions: combining multiple pattern checks
  • Boolean logic: using and and not operators effectively
  • Pattern exclusion: ensuring one pattern without another

• Recap for the rf and countVowels functions:
  • File operations: reading program source code as data
  • Program as data: treating code as input to other programs
  • Dynamic execution: foundation for universal computation

Programs are data! Start thinking like a proofgrammer about universal computation!

• Programs can read other programs as input
• Programs can execute other programs
• Programs can analyze other programs

Next steps in proofgramming

• Deepen theoretical understanding
  • Explore more complex computational models
  • Study formal language definitions and properties
  • Investigate decidability and undecidability results
  • Analyze computational complexity classes

• Apply programming skills
  • Implement automata and Turing machines
  • Create programs that analyze other programs
  • Build tools for complexity measurement
  • Develop proofs through computational examples

Ready to formally define what it means to compute? Let’s implement more SISO programs! Hooray!

Numerical inputs in SISO programs

• String-to-number conversion: int() and float() functions
• Number-to-string conversion: str() function for SISO compliance
• Data processing pattern: split, convert, compute, stringify

Try numerical conversion

ASCII character set in SISO programs

• ASCII standard: 128 characters including letters, digits, symbols
• Printable characters include symbols like !, @, #, $, %, ^, &, *, (, )
• Special characters: newline (\n) and space characters
• Multiline support: SISO programs can handle large text blocks
• Text processing: enables complex string manipulation operations

ASCII strings provide sufficient expressiveness for all computational input/output while maintaining formal simplicity for theoretical analysis. Note that ASCII strings are a subset of the Unicode standard supported by the Python programming language!

Multiline text processing

Examples of invalid SISO programs

• Not all valid Python programs are valid SISO programs!
• We restrict focus to SISO programs for our theoretical analysis
• Can we formally define what we mean by a program, input, and output?

Program \(P\) runs on Computer \(C\)

• A reference computer \(C\) executes SISO programs

• \(C\) has infinite memory and time unless stated otherwise

• We make no assumptions about characteristics of \(C\):

  • Speed of execution
  • Memory architecture
  • Instruction set
  • Input/output formats
  • Operating system

• \(C\) is a theoretical construct to support proofgramming!

Simple formal notation: \(P(I)\)

• \(P(I)\) definition
  • \(P\) is a SISO Python program
  • \(I\) is an input string
  • \(P(I)\) is the output when \(P\) executes on input \(I\)
  • The “first function” in \(P\) is the “main function”

• When \(P(I)\) is undefined
  • \(P\) doesn’t terminate (infinite loop or recursion)
  • \(P\) returns a non-ASCII string value
  • \(P\) throws an unhandled exception
  • \(P\) has syntax or runtime errors

Program termination analysis

# example of well-defined P(I)
def terminatingProgram(inString: str) -> str:
    """A program that always terminates correctly."""
    count = 0
    for char in inString:
        if char.isalpha():
            count += 1
    return str(count)

# example of potentially undefined P(I)
def potentiallyNonTerminating(inString: str) -> str:
    """This program might not terminate depending on input."""
    number = int(inString) if inString.isdigit() else 0
    # this could be an infinite loop if number <= 0
    while number > 0:
        number -= 1
        if number == 100:  # artificial condition
            number = 101
    return "done"

# test the well-defined program
test_string = "Hello123World"
result = terminatingProgram(test_string)
print(f"terminatingProgram('{test_string}') = '{result}'")

terminatingProgram('Hello123World') = '10'

Exploring non-termination

# The non-terminating example is shown for analysis only
print("potentiallyNonTerminating analysis:")
print("- With input '5': would terminate")
print("- With input '0': would terminate immediately") 
print("- With input '101': would loop infinitely")

potentiallyNonTerminating analysis:
- With input '5': would terminate
- With input '0': would terminate immediately
- With input '101': would loop infinitely

• Termination guarantee: crucial for \(P(I)\) to be “well-defined”
• Input validation: checking input format prevents runtime errors
• Computational analysis: predicting program behavior on inputs
• Halting detector: detects when a program will never terminate

Why potentiallyNonTerminating?

• The function contains a while number > 0: loop.
• Each time through the loop, number is decremented by 1.
• But if number ever becomes exactly 100, it is reset to 101.
• This creates an endless cycle (also called an infinite loop): when number reaches 100, it jumps back to 101, so it never reaches 0.
• For any input greater than or equal to 100, loop never terminates.
• Key point: The artificial reset prevents the loop variable from ever reaching 0, so the function does not halt for some inputs.
• Important Questions: Would it be possible to automatically detect this behavior? Why or why not? Can you justify your answer?

Decision programs: accept or reject

• Decision program definition
  • Always returns exactly “yes” or “no”
  • If \(P(I)\) = “yes”, then \(P\) accepts \(I\)
  • If \(P(I)\) = “no”, then \(P\) rejects \(I\)
  • Foundation for complexity and computability theory

• Examples of accept/reject
  • containsGAGA("GAGA") = “yes” → accepts “GAGA”
  • containsGAGA("hello") = “no” → rejects “hello”
  • Decision problems form the basis of language recognition

Implementing decision programs

Equivalent Python programs

• We also need a notion of equivalence between Python programs

• Two programs are equivalent if, for any given input, both programs produce the same output, including undefined behaviors.

• SISO Python programs \(P_1\) and \(P_2\) are equivalent if:

  • \(\forall I \in S\), \(P_1(I) = P_2(I)\)

• For multi-parameter SISO Python programs, we require:

  • \(\forall I_1, I_2, \ldots, I_n \in S\), \(P_1(I_1, I_2, \ldots, I_n) = P_2(I_1, I_2, \ldots, I_n)\)

• Importantly, this definition of equivalence includes undefined outputs! If \(P_1\) and \(P_2\) are equivalent, then \(P_1(I)\) is undefined if and only if \(P_2(I)\) is undefined, for all ways to be undefined.

Review course goals

• Computational Implementation:
  • Design and implement theoretical concepts as Python programs
  • Create frameworks for running proofs as Python programs
  • Test all aspects of implementations to ensure correctness
• Communicate theoretical insights through code and documentation
• Explain and evaluate insights during presentation sessions
• Participate in charette sessions to discuss and learn from each other
• Make sure you have completed these steps:
  • Review the course schedule
  • Study the course syllabus

Key takeaways

• Problem Classification: Tractable, intractable, and uncomputable
• SISO Programs: Single input/output model with restricted to strings
• Python Examples: Numerical conversions, text processing
• Decision Problems: SISO programs for which output is “yes” or “no”
• Formal Notation: \(P(I)\), undefined cases, termination, equivalence
• Insights: Programs as data, universal computation, SISO generality
• Next Steps: Advance to automata, Turing machines, undecidability
• Proofgramming Journey: Combine proofs and programming to explore computation and discern what can be computed
• Now, we are ready for the first Proofgramming Charette session!