The Effect of Coding on Computation Adina Lederhendler

The Effect of Coding on Computation Adina Lederhendler Topics in Biological Physics 23/12/08 Shannon: A universal Turing machine with two internal states. Automata Studies, 1956 Miller : The magical number seven, plus or minus two: some limits on our capacity for processing information , Psychological Review, 1956

Outline What is a code? Abstract computation Different codes for universal Turing machines Real-world computation The use of coding in working memory

What is a code? Code – Set of rules for translation of input into output. Input 1 Output 1 Input 2 Output 2 Input 3 Output 3 Input NI Output NO

What is a code? Examples of codes: ASCII code English alphabet Binary digits Genetic code Nucleotide sequences Amino acids

Turing machine Reading/ writing head Si Information encoded as symbols on tape. Configuration: Symbol + Internal state.

Turing machine Si Code Configuration Step symbol + state symbol + shift + state complexity of machine (size of code)

Universal Turing Machine Si Input tape and code of a Turing machine T UTM Same output as T. UTM not related to any specific computation.

Universal Turing machine Shannon, 1956 – what are the limitations on the code of a UTM? Minimal number of states for a UTM? Minimal number of symbols for a UTM? No 1 -state UTM No 1 -symbol UTM Each step depends only on the letter currently being read. Cannot carry enough information Cannot carry any information Shannon, Automata Studies, 1956

Universal Turing machine A Symbols States Universal Turing machine B States Universal Turing machine C States ? Symbols ? ? Symbols ? How do alternative codes affect the computation?

Two-state UTM Universal Turing machine B Symbols that represent input/output Symbols States Symbols that represent intermediate states during computation Information about states of A carried by symbols of B (symbol-state tradeoff)

Two-symbol UTM Universal Turing machine C States Symbols Information about symbols of A carried by states of C (symbol-state tradeoff)

Two-symbol UTM Example – Machine A: m = 5, n A tape: C tape:

Two-symbol UTM Machine A Machine C

Two-symbol UTM Use information about machine A

Two-symbol UTM

Two-symbol UTM

Two-symbol UTM Move to beginning of the next input

Two-symbol UTM Ready to begin next step.

Symbol-state product A B C Symbols States Product Shannon: What is the minimum symbol-state product required to construct a universal Turing machine?

Minimal Turing machines Minsky, 1962 7 -state, 4 -symbol UTM. Later efforts Find additional minimal (size) UTMs. Find more efficient minimal UTMs. Woods, Theoretical Computer Science, 2008

Real world systems Best code for specific task Real-world considerations such as: Efficiency: Accuracy: How much time, what resources can we devote to the computation? How much noise is present in the system? How accurate does the computation need to be? Biological systems Evolution/adaptation to specific task

Working memory Miller, 1956 Attempt to quantify working memory capacity How many items can we recall immediately after being presented with a sequence? AMOUNT OF INFORMATION PER ITEM Miller, The Psychological Review, 1956

Working memory Sequences of items Recoding into “chunks” and storage in short-term memory Adaptation through learning New codes = More kinds of “chunks” Retrieval

Working memory Baddeley & Hitch, 1974 O K B T P V What relation Some All of of. BCisis must there included inbe A B between A and C?

Summary • A code is a set of rules for carrying out a computation. • Different codes can be used to perform the same computation. • The choice of code may affect the efficiency and accuracy of a computation. • Biological systems can adapt codes to specific tasks.