Information systems theory Peter Fox Xinformatics ITEC CSCI

Information systems ‘theory’ Peter Fox Xinformatics – ITEC, CSCI, ERTH 4400/6400 Week 3, February 9, 2016 1

Contents • Review of last class – use cases? – Discussion of reading • Systems • Information theory • Principles covering a range of traditional foundation aspects • Next class(es) and assignments 2

Reading • http: //en. wikipedia. org/wiki/Use_case • http: //alistair. cockburn. us/index. php/Use_cas es, _ten_years_later • Or questions about last week’s material? 3

Systems • Regardless of the type of system, be it an irrigation system, a communications relay system, an information system, or whatever, all systems have three basic properties: – A system has a purpose - such as to distribute water to plant life, bouncing a communications signal around the country to consumers, or producing information for people to use in conducting business. – A system is a grouping of two or more components which are held together through some common and cohesive bond. The bond may be water as in the irrigation system, a microwave signal as used in communications, or, as we will see, data in an information system. – A system operates routinely and, as such, it is predictable in 4 terms of how it works and what it will produce.

Example 5

Thinking in systems • Consists of primarily three things (Meadows) – Elements – Interconnections – Function/ Purpose • Three attributes of – Resilience – Self Organization – Hierarchy 6

Example 7

Twelve Leverage Points 12. Constants, parameters, numbers (such as subsidies, taxes, standards) 11. The size of buffers and other stabilizing stocks, relative to their flows 10. Structure of material stocks and flows (such as transport network, population age structures) 9. Length of delays, relative to the rate of system changes 8. Strength of negative feedback loops, relative to the effect they are trying to correct against 7. Gain around driving positive feedback loops 6. Structure of information flow (who does and does not have access to what kinds of information) 5. Rules of the system (such as incentives, punishment, constraints) 4. Power to add, change, evolve, or self-organize system structure 3. Goal of the system 2. Mindset or paradigm that the system — its goals, structure, rules, delays, parameters — arises from 1. Power to transcend paradigms 8

Example of Leverage Points 12. Constants, parameters, numbers (such as subsidies, taxes, standards) 11. The size of buffers and other stabilizing stocks, relative to their flows 10. Structure of material stocks and flows (such as transport network, population age structures) 9. Length of delays, relative to the rate of system changes 8. Strength of negative feedback loops, relative to the effect they are trying to correct against 7. Gain around driving positive feedback loops 6. Structure of information flow (who does and does not have access to what kinds of information) 5. Rules of the system (such as incentives, punishment, constraints) 4. Power to add, change, evolve, or self-organize system structure 3. Goal of the system 2. Mindset or paradigm that the system arises from — its goals, structure, rules, delays, parameters 1. Power to transcend paradigms 9

Twelve Leverage Points Here? Example 10

First information system? • The first on-line, real-time, interactive, data base system was double-entry bookkeeping which was developed by the merchants of Venice starting in ~ 1200 A. D. Paciolo 1494 • Truth, author’s opinion or urban legend? 11

Details on first info sys. • “Every transaction gets entered twice in financial records. If one day you sold three gold coins' worth of pepper, you would write that the amount of cash you had went up by three gold coins. You would also write in that the amount of pepper you had went down by three gold coins' worth. • Before double-entry, people just kept diaries and counted their money at the end of the day. This innovation allowed merchants to see every aspect of their business in neat little rows. “ (NPR) 12

Data-Information-Knowledge Ecosystem Producers Consumers Experience Data Creation Gathering Information Presentation Organization Knowledge Integration Conversation Context 13

Presentation • Separation of content from presentation!! – The double-entry example (debit/ credit -/+) • Is developed based on a solid understanding of minimizing information uncertainty – Start with content (duh) – Look at any context and structural considerations – Account for cognitive and social factors to reduce uncertainty (e. g. for humans: color, …) 14

Organization • Organization of information presentation, e. g. layout on a web page, in a table, or figure, or report – Of the double-entry bookkeeping… • Also (again) content, context and structure • Think about how you organize your – – – Class notes Calendar and assignment schedule Your social life (just kidding) Assignments Do, or do not, connect with others’ ways of organizing! • A system? ? – Elements, Interconnections, Function/ Purpose 15

All take a deep breath 16

Equations! 17

Information theory • Entropy and randomness – Critical for information system design – More on why in a few slides 18

Huh? Entropy? • No, you are not in a physics class • Information is always a measure of the decrease of uncertainty at a receiver. 19

Entropy and Rates • R=H(x)-H(x|y) (Shannon; 1948) • R=Rate of transmission - measures the average ambiguity of the received signal 20 p(xi) is the probability mass function of outcome xi.

Entropy and Rates • R=H(x)-H(x|y) =Entropy – Conditional Entropy • Thus relies on conditional probability… "The entropy rate of a data source means the average number of bits per symbol needed to encode it. Shannon's experiments with human predictors show an information rate of between 0. 6 and 1. 3 bits per character, depending on the experimental setup; the PPM compression algorithm can achieve a compression ratio of 1. 5 bits per character in 21 English text. ” (wikipedia, reference in notes)

Shannon’s Theorem • For any channel, there exist ways of encoding input symbols such that we can simultaneously utilize the channel as closely as we wish to the capacity, and at the same time have an error rate as close to zero as we wish. • Think about this! – DNA? • encoding input symbols 22

Units • Thus, using different bases for the logarithm results in information measures which are just constant multiples of each other, corresponding with measurements in different units: – log 2 units are bits (from ’binary’) – log 3 units are trits(from ’trinary’) – loge units are nats (from ’natural logarithm’) (We’ll use ln(x) for loge(x)) – log 10 units are Hartleys, after an early worker in the field. 23

Units • Unless we want to emphasize the units, we need not bother to specifiy the base for the logarithm, and will write log(p). • Typically, we will think in terms of log 2(p), i. e. in bits 24

Example 1 • The maximum information the student gets from a grade will be: – Pass/Fail : 1 bit. – A, B, C, D, F : 2. 3 bits. – A, A-, B+, . . . , D-, F : 3. 6 bits. • Thus, using +/- grading gives the students about 1. 3 more bits of information per grade than without +/-, and about 2. 6 bits per grade more than pass/fail. 25

Example 2 • If a source provides us with a sequence chosen from 4 symbols (say A, C, G, T), then the maximum average information per symbol is 2 bits. P(x)=. 25 H = -(. 25 x -2 +. 25 x -2) = 2 • If the source provides blocks of 3 of these symbols, then the maximum average information is 6 bits per block (or, to use different units, 4. 159 nats per block). 26

Not a perfect story • Many authors criticize the use of the term entropy, and physics of information • Information conservation, diffusion, viscosity, advection, dissipation, instability, steady state, conversion … sort of all make some sense… • But entropy arises in thermodynamics not directly in other places 27

That’s not going to stop us! • However the idea is very relevant to – modeling (sometimes equations) – design (variables) – architecture (how they are put together) – as well as how we “condition the system” • We’ll revisit the components of information soon but first let’s take some examples 28

For information systems? • • 31045? (03) 1045 783 -1045 +161397831045 • What helps reduce entropy / uncertainty? • Notice: ‘signs’ as information representations 29

AIM=Information integrity • In later classes we cover cognitive and social factors in increasing the conditional entropy and thus reducing the uncertainty and thus increasing information content and value • We will also cover semiotics (signs) as a prelude to visualization as a presentation mechanism for information 30

Think of web pages – what is H here? 31

Not worst but poor 32

One more 33

Information gain/loss • In probabilistic terms, the entropy decreases by conditioning on the distribution. – The mutual information of two variables define how much information one variable contains about the other. – It is therefore defined as the decrease of the uncertainty of one variable by knowing the other. • What does this mean for an information system? E. g. a website or web service? 34

Physics of Information © 2005 Ev. REsearch LTD Ev. REsearch©

Context • Internal - Human context, tacit knowledge – Curiosity – User profiles – Analytics • External – Domain context – Skill/ education context – Organizational – Procedural/ process – Unknown/ random – what is an example of this? • Affects probabilities and thus entropy… 36

Structure • Structural representation of information content can bias presentation, e. g. – Modern image capture devices (digital camera) often convert 2 byte integer to float, or 4 byte integer, what are the implications • Affects probabilities and thus entropy… • Appropriate choice of information structure can significantly decrease uncertainty, e. g. returning land images in Geo. TIFF, which can encoding geographic location, instead of PNG 37

Content • Presentation – We’ve covered a fair bit of this so far – What other factors have you thought of? • Translation – Almost essential when transmitting between data structures, e. g. serialization over network protocols, sometime multiple levels; HTTP, TCP/IP – Shannon’s Theorem • Encoding – Lossless (Huffman, entropy based, 1952) – Lossy (mpeg, jpeg) 38

Geographic Information Systems • Geography! – – Spatial Provides context Provides structure Often predetermines content form • Wikipedia: “the term describes any information system that integrates, stores, edits, analyzes, shares, and displays geographic information” • A lesson for constraining uncertainty! 39

And now our enemy - noise • Most often refers to ‘data’ but does apply to information • Uncertainty, especially any that is introduced is a source of noise, or more accurately – bias in the use or interpretation of the information • Noise/ bias is context and structure dependent • Noise/ bias contamination is rampant in information systems • Quality control and verification is less developed for information sources, e. g. ‘people do not report problems’ 40

Mode of noise introduction From Shannon and Weaver (1949) Msg? Information Source Signal? Web Content, Structure Recvd? Msg? Web browser? Noise source HTML page, user 41

Summary • About systems • Information systems • The elements of theory so far – Entropy/ uncertainty • Content, context, structure • Presentation, organization, noise 42

Reading for this week • Is retrospective but … relates to a coming assignment – – – Information entropy Information Is Not Entropy, Information Is Not Uncertainty! More on entropy Context Information retrieval Abductive reasoning 43

What is next • Week 4 – Foundations; semiotics, library, cognitive and social science and class exercise - information modeling • Then … Assignment 2 44