Sequential decision analytics and modeling ORF 411 Fall

Sequential decision analytics and modeling ORF 411 Fall, 2018 . Warren Powell Princeton University http: //www. castlelab. princeton. edu © 2017 Warren B. Powell, Princeton University

Week 11 - Monday Student decision problems Booking a flight © 2018 W. B. Powell Slide 2

n This lecture was run by choosing a student with a sequential decision problem. n The idea is to choose one student as the modeler who stands at the front of the classroom. They should write out the five elements of a sequential decision problem (see next slide). n A second student, who is familiar with the problem, serves as the domain experiment. He/she sits in the audience and answers questions. This lecture focused on booking flights, which is a problem that many students are familiar with (I found that the international students were particularly skilled at this). n The modeler then fills in the elements of the problem by asking the class.

Elements of a dynamic model n The modeling process » I conduct a conversation with a domain expert to fill in the elements of a problem: State variables Decision variables New information Transition function Objective function Performance metrics How the state variables evolve What we didn’t know when we made our decision What we control What we need to know (and only what we need)

n Note: » I recall doing this twice in the first year, but I suggest trying to do this as often as you can. » This exercise involves active student involvement in the modeling process, which is key.

Student decision problem Template © 2018 W. B. Powell Slide 6

Decision problem n Narrative n State variables © 2018 W. B. Powell

Decision problem n Decision variables n Exogenous information © 2018 W. B. Powell

Decision problem n Transition function n Objective function © 2018 W. B. Powell

Student decision problem Booking airline reservations © 2018 W. B. Powell Slide 10

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n Possible CFA … » Discuss logic from diabetes example. » We could estimate the value of reserving on each day before the departure date. You can learn within a simulator, or using actual experience. • Simplify – instead of day t, use weeks. © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

When to book a flight? n © 2018 W. B. Powell

Student decision problem Financial trading © 2018 W. B. Powell Slide 21

n Narrative © 2018 W. B. Powell

n State variables © 2018 W. B. Powell

n Decision variables © 2018 W. B. Powell

n Exogenous information © 2018 W. B. Powell

n Transition function © 2018 W. B. Powell

n Objective function © 2018 W. B. Powell

n Uncertainty model » The process is data-driven – no uncertainty model required. No simulator has been used, and there is no lookahead component. © 2018 W. B. Powell

n Designing policies © 2018 W. B. Powell