Back of the Envelope Reasoning Praveen Paritosh Ken

Back of the Envelope Reasoning Praveen Paritosh Ken Forbus QR 04 Practice Talk Thursday, July 8, 2004

Outline of the talk • • • Examples Motivations for modeling Bot. E reasoning Hand-worked out examples Implementation of Bot. E-Solver Results and current status What next

Some examples • How many K-8 school teachers are in the USA? • How much money is spent on newspapers in USA per year? • How much time would be saved per year nationwide by increasing the speed limit from 55 to 65 mph? • What is the annual cost of healthcare in USA? • How much tea (weight) is there in China? • Last summer, the US Army bought Microsoft Windows/Office/Server software for 500, 000 computers. The deal included the software and six years of support. How much did the army pay for this?

Back of the Envelope Reasoning • Numeric answers • Specificity-Resources tradeoff • Only thing you can do in many domains – Environmental science, Biophysics, etc. – New, unfamiliar domains. • Useful – Engineers, Scientists, Policymakers, everyone who reads a newspaper • Problem solving + feel for numbers [contrast first principles reasoning]

Constraints guiding Common Sense QR 1. Incompleteness Domain theories incomplete in coverage. 2. Concreteness Knowledge of concrete, specific situations (made use of by analogical reasoning) in addition to first-principles reasoning. 3. Highly experiential: Experience improves - ability to reason through similar scenarios. - intuitions for quantity: what is reasonable, high, low in a domain. 4. Focused reasoning Tight reasoning, as opposed to maintaining ambiguity for completeness 5. Pervasively quantitative Real-world actions require that estimates manifest as exact values.

How many K-8 school teachers are in the USA? Number of teachers = number of students / students per teacher Number of students = population * fraction in the age range of K 8 students * fraction of kids who go to school Number of students = 300 mil * 9/75 * 1 Number of teachers = 40 mil / 25 = 1. 6 mil 1. 9 million, source: Statistical Abstracts, 2003.

How much money is spent on newspapers in USA per year? Total money spent = Money spent per buyer * number of buyers Annual expense per buyer = Units bought per year * cost per unit Annual expense per buyer = 365 * 0. 75 = 250 Number of buyers = 300 mil * ¼ = 75 mil Total money spent = 75 mil * 250 = 20 billion 26 billion, source: Statistical abstracts, 2003.

What is the total annual gasoline consumption by cars in US? Total consumption = Total miles driven/ miles per gallon Total miles driven = Number of cars in the US * Miles driven per car per year = Miles driven per day * 365 If we say that every household owns a car, since some don’t and some might have more than one, then Number of cars in the US = number of households = population / average size of American household. Now we have a model, and using the following numbers, Population ~ 300 million, Average size of household ~ 3, Daily miles driven ~ 20, Miles per gallon for a car ~ 20. We get an estimate of 36. 5 billion gallons. 35 billion gallons, Statistical Abstracts, 2003.

Formalizing Bo. TE reasoning Estimate parameter directly 1 Feel for numbers 2 Create estimation model Use known value if available Estimate based on similar situation Problem solving Find modeling strategy 3 Find values for parameters in model

Problem Solving • Representation – Of problems, strategies and domain knowledge. • Retrieval – Of relevant knowledge and strategies • Reasoning – Workspace: keeping track of progress made. – Agenda: figuring out what to do next.

Bot. E-Solver • Representation – Domain knowledge: 1. 2 million fact subset of Cycorp’s CYC knowledge base. – Strategies: Suggestions. • Retrieval – Pattern matching and backward chaining. • Reasoning – Built on top of FIRE reasoning engine – AND/OR tree as workspace. – Difficulty ordered agenda.

AND/OR Decomposition Total annual gasoline consumption Mileage. Strategy Suggestion/OR-node Total miles driven Goal/AND-node Miles per Gallon Per. Unit. Strategy Number of cars Household. Strategy. For. Counting. Units Number of cars per house Annual miles driven per car Per. Day. For. Year. Strategy Number of households Miles driven per day

Workings of the Solver Control Status Flags • SOLVED • FAILED • MOOT – MOOT-VIA-FAILURE – MOOT-VIA-SUCCESS Solution generation • Incremental • Two modes – Agenda processing – IN-PLAY processing Subgoals • Fully ordered

An example suggestion (def. Suggestion Household. Strategy. For. Counting. Units : trigger (units. Total ? obj ? place ? time ? total-units) : test (owned. By ? obj Family. Cohabitation. Unit) : subgoals ((number. Of. Households ? place ? time ? num-households) (units. Per. Household ? obj ? units-per-household)) : result-step (evaluate ? total-units (Times. Fn ? num-households ? units-per-household)))

Axioms for Household. Strategy. For. Counting. Units (ist-Asserted Suggestion. MT (suggestion. Result. Step Household. Strategy. For. Counting. Units (evaluate ? total-units (Times. Fn ? num-households ? units-per-household)))) (ist-Asserted Suggestion. MT (suggestion. Subgoals Household. Strategy. For. Counting. Units (The. List (number. Of. Households ? place ? time ? num-households) (units. Per. Household ? obj ? units-per-household)))) (ist-Asserted Suggestion. MT (suggestion. Goal. Form Household. Strategy. For. Counting. Units (units. Total ? obj ? place ? time ? total-units))) (ist-Asserted Suggestion. MT (implies (owned. By ? obj Family. Cohabitation. Unit) (suggest. For (units. Total ? obj ? place ? time ? total-units) Household. Strategy. For. Counting. Units))) (ist-Asserted Suggestion. MT (isa Household. Strategy. For. Counting. Units Suggestion)) (suggestion. Result. Step Household. Strategy. For. Counting. Units (evaluate ? total-units (Times. Fn ? num-households ? units-per-household))) (suggestion. Subgoals Household. Strategy. For. Counting. Units (The. List (number. Of. Households ? place ? time ? num-households) (units. Per. Household ? obj ? units-per-household))) (suggestion. Goal. Form Household. Strategy. For. Counting. Units (units. Total ? obj ? place ? time ? total-units)) (implies (owned. By ? obj Family. Cohabitation. Unit) (suggest. For (units. Total ? obj ? place ? time ? total-units) Household. Strategy. For. Counting. Units)) (isa Household. Strategy. For. Counting. Units Suggestion)

More examples

(annual. Sales Newspaper. Copy United. States. Of. America (Year. Fn 2003) ? money) (def. Suggestion Per. Buyer. Strategy (annual. Sales ? obj ? place ? time ? money) : subgoals ((annual. Expense. Per. Buyer ? obj ? place ? time ? money-per-buyer) (number. Of. Buyers ? obj ? place ? time ? number)) : result-step (evaluate ? money (Times. Fn ? money-per-buyer ? number))) (def. Suggestion Unitary. Strategy. For. Cost (annual. Expense. Per. Buyer ? obj ? place ? time ? money-per-buyer) : subgoals ((annual. Units. Bought. Per. Buyer ? obj ? place ? time ? units) (cost ? obj ? unit-cost)) : result-step (evaluate ? money-per-buyer (Times. Fn ? units ? unit-cost))) (def. Suggestion Fraction. Of. Total. Strategy (number. Of. Buyers ? obj ? place ? time ? number) : subgoals ((population. During ? place ? any-time ? total) (percent. Of. Buyers ? obj ? place ? percent)) : result-step (evaluate ? number (Quotient. Fn (Times. Fn ? total ? percent) 100)))

(cardinality K-8 School. Teacher ? numteachers) (def. Suggestion Students. Per. Teacher. Strategy (cardinality K-8 School. Teacher ? numteachers) : subgoals ((cardinality K-8 School. Student ? numstudents) (students. Per. Teacher K-8 School ? perteacher)) : result-step (evaluate ? numteachers (Times. Fun ? numstudents ? perteacher))) (def. Suggestion Uniform. Age. Distribution. Strategy (cardinality K-8 School. Student ? numstudents) : subgoals ((population. During United. States. Of. America (Year. Fn 1997) ? population) (minimum. Age K-8 School. Student ? min) (maximum. Age K-8 School. Student ? max) (life. Expectancy. For. Group. In. Region United. States. Of. America Person (Years. Duration ? life))) : result-step (evaluate ? numstudents (Times. Fn (Quotient. Fn (Difference. Fn ? max ? min) ? life) ? population))))

Current Status • Can successfully solve 8 problems • Strategy library contains 23 suggestions

Solved Problems - I Proble m Number Problem, and its predicate calculus representation Answer found by Bot. E-Solver, and comparison to a known answer if available #Axio ms 1 How many popcorn kernels would it take to fill in the 1890 Maple big classroom? (Count. Contained CS 381 Class. Room Popcorn ? number) (? number. 1. 343444 e+7) Bot. E-Solver: 13 million Correct answer: not available! 30 2 How much money is spent on newspapers in the US? (annual. Sales Newspaper. Copy United. States. Of. America (Year. Fn 2003) ? money) How many K-8 teachers are there in the US? (cardinality K-8 School. Teacher ? numteachers) (? money. 2. 1884363 e+10) Bot. E-Solver: 21 billion Correct answer: 26 billion 30 (? numteachers. 1056454) Bot. E-Solver: 1. 05 million Correct Answer: 1. 9 million 20 What is the annual cost of healthcare in the US? (annual. Sales Health. Care United. States. Of. America (Year. Fn 2003) ? money)) (? money. 799428834000) Bot. E-Solver: 0. 8 trillion Correct Answer: 1. 6 trillion 12 3 4

Solved Problems - II Proble m Number Problem, and its predicate calculus representation Answer found by Bot. ESolver, and comparison to a known answer if available # Axioms 5 How many cars are bought per year in the US? (units. Bought. Per. Year Automobile United. States. Of. America (Year. Fn 2003) ? num) (? num. 8920000) Bot. E-Solver: 8. 9 million Correct Answer: 8 million 30 6 What is the weight of garbage thrown away by American families each year? (annual. Production Garbage-Generic United. States. Of. America (Year. Fn 2003) (Pound-Unit. Of. Mass ? garbage-mass)) (? garbage-mass. 446000000) Bot. E-Solver: 446 million pounds Correct answer: not available! 10 7 How many hotdogs are sold in a baseball season in Wrigley Field? (units. Sold Hot. Dog. Sandwich Wrigley. Field Baseball. Season ? numdogs) What is the total amount of gasoline (? num-dogs. 1600000) Bot. E-Solver: 1. 6 million Correct answer: not available! 18 (? oil-consumption. 32558000000) Bot. E-Solver: 32. 6 billion gallons Correct Answer: 35 billion gallons 20 8 consumption by cars in the US? (annual. Automobile. Gas. Consumption United. States. Of. America (Year. Fn 2003) (Gallon-US ? oil-consumption)))

What next • Represent about 50 problems – Show stable re-usable representations, in strategies and theoretical knowledge added. – Knowledge level analysis of the kinds of knowledge involved in Bot. E. • Feel for numbers – Build symbolic representations for numbers • Large, expensive, upper class, etc. [CARVE, Paritosh, 2003] – Analogical estimator: makes guesses for a numeric parameter based on analogy.

For More The Bot. E website: www. cs. northwestern. edu/~paritosh/bote/ Library of problems, solutions, strategies.

Extra Stuff

Pseudocode

get-solution(original-goal) if in-play? (original-goal) get-next-solution(original-goal) else if the agenda is empty then quit process-agenda

get-next-solution(ao-node) if there are cached solutions at this node ; ; cur-bmarker points to the current solution increment cur-bmarker(ao-node) return cached solution if goal-node? (ao-node) if ao-node has a child suggestion that is in-play get-next-solution(in-play-suggestion(ao-node)) ; ; No in-play suggestion, if this goal has a younger ; ; sibling that can give us more new bindings, re-instantiate ; ; this node with those bindings. if younger-siblings(ao-node) bindings = get-next-solution(younger-sibling(ao-node)) if bindings found enqueue-on-agenda(re-instantiate-node(ao-node, bindings)) if suggestion-node? (ao-node) ; ; seek downward solution = get-next-solution(eldest-child, ao-node) add-bindings(solution, ao-node) increment cur-bmarker(ao-node) return solution

propagate-bindings(goal-node) if elder-sibling? (goal-node) increment cur-bmarker(goal-node) next-node = instantiate-node(elder-sibling(goal-node), current-bindings(goal-node)) add-to-tree(next-node) enqueue-on-agenda(next-node) return ; ; No elder sibling, so we can now see ; ; if the parent node got solved if parent(goal-node) exists bindings = get-next-solution(goal-node) while we have bindings for goal-node result-bindings = do-result-step(parent(goal-node)) if result-bindings found add-bindings(result-bindings, parent(goal-node) propagate-bindings(parent(goal-node))) bindings = get-next-solution(goal-node)

process-agenda(ao-node) if goal-node? (ao-node) solutions = ask(goal-node) ; ; primitive problem solving step if solutions found propagate-bindings(goal-node) else suggestions = gather-suggestions(goal-node) if suggestions found enqueue-on-agenda(suggestions) else ; ; no suggestions found, see if we can ; ; re-instiate this node if younger-siblings(goal-node) bindings = get-next-solution(younger-sibling(goal-n if bindings found enqueue-on-agenda(re-instantiate-node(ao-node, bi else update-failed (goal-node) if suggestion-node? (ao-node) ; ; since the subgoals are ordered, add the first one to agen enqueue-on-agenda(youngest-child(ao-node))

Examples

How much time would be saved by increasing the speed limit from 55 to 65 mph? Total time spent driving = time spent driving person * number of drivers Time spent driving person = Total distance driven person /Speed Total distance driven person = Distance per day * number of days driving Number of drivers: assume one driver per household = 1/3 * population = 100 mil Total distance driven person = 20 * 365 = 8000 Time spent person = 8000/55 = 150 hours Time spent if it was 65 mph = 8000/65 = 120 hours Time saved person = 30 hrs Total time saved = 3 billion hours = 300, 000 years.

The Microsoft Army – How much does software for 500, 000 computers cost? Total cost on the software = cost of desktop software + server software + upgrades Windows XP/Office CDW prices = 650 Cost of desktop software = 500, 000 * 650 = 325 mil Cost of upgrades = 500, 000 * 200 = 100 mil Assume 1 in 100 servers = 5000 servers Exchange server = 600 Windows 2003 server = 650 SQL Server = 1000 Cost of server software = 2250 * 5000 = 11. 25 mil Cost of upgrades = 400 * 5000 = 2 mil Total cost = 440 million Softmart, inc, PA got paid = 470 million

What is the annual cost of healthcare in USA? Lets say everyone was insured. Total cost of healthcare has to be less than insurance premiums, for the insurance companies to stay in business. Cost of healthcare = average insurance premium * population = 3, 000 * 300 million ~ 1 trillion [1. 6 trillion last year. How does this work? ]

How much tea is there in China? Total tea = amount of tea consumed per day * stockpile in time Tea consumed per day = tea consumed person per cup * number of cups * population 1 billion people 5 cups a day 5 grams a cup Lets say they stock an years supply 25 billion grams = 25 million kilos per day In an year 365*25 million ~ 10 billion kilos

CARVE, feel for numbers

Numbers in symbolic knowledge representation • Consider the Great black-bucked gull – Wingspan = Large – Wing-span = 0. 272 sq. m. • Numbers not handled right – Similarity: computing and making inferences – Retrieval – Generalization

CARVE: Symbolizing Quantity • Don’t do it – Sorites – Context/Utterer sensitivity • Dimensional partitions: Large and Small, based on distributional properties of the quantity. (isa Algeria (High. Value. Contextualized. Fn Area African. Countries)) • Structural partitions: Boiling point and Poverty line, denote changes of quality.

CARVE (isa Algeria (High. Value. Contextualized. Fn Area African. Countries). . . Dimensional partitioning for each quantity Ci Add these facts to original cases Quantity 1 S 2 S 1 L 1 S 3 Cj L 2 Structural clustering using SEQL Ci*

Analogical Estimator (Gross. Domestic. Product Brazil ? x) • The value is known. • Find an analogous case for which value is known. • Find anything in the KB which might help me make an estimate.

Thesis and Evaluation • More powerful and flexible back of the envelope reasoning can be done using these symbolic representations of quantity. • Evaluation: – Corpus of problems with/without representations generated by CARVE.

Numbers in News • The cost of Mars Pathfinder mission was 270 million. • 50 billion tax cut. • 62, 000 square feet.

Old slides

An example (def. Suggestion Volume. Formula. For. Sphere (volume. Of. Object ? obj ? vol) : test (shape. Of. Object ? obj Sphere. Shape) : subgoals ((radius ? obj ? radius)) : result-step (evaluate ? vol (Times. Fn 4. 187 ; ; 4/3*Pi (Exponent. Fn ? radius 3))))

SOLVE • Suggestions based problem solver • Represent problem solving progress in an and/or tree structure • Incremental solution generation, getnext-solution • Ordered subgoals