Sequential decision analytics and modeling ORF 411 Fall

Sequential decision analytics and modeling ORF 411 Fall, 2018 . Warren Powell Princeton University http: //www. castlelab. princeton. edu © 2017 Warren B. Powell, Princeton University

Week 1 - Monday Adaptive market planning © 2018 W. B. Powell Slide 2

Narrative © 2018 W. B. Powell Slide 3

Narrative © 2018 W. B. Powell

Narrative © 2018 W. B. Powell

Narrative © 2018 W. B. Powell

Narrative n © 2018 W. B. Powell

Stochastic gradient algorithms © 2018 W. B. Powell Slide 8

Adaptive market planning n Gradient-based stochastic search » Powerful algorithm, but it an important class of sequential decision problem © 2018 W. B. Powell

Adaptive market planning n Stepsize rules » Harmonic stepsize formula (deterministic » Kesten’s rule (stochastic) » Discuss: • Scaling • Handling bias (learning) vs. noise (smoothing) © 2018 W. B. Powell

Adaptive market planning n Harmonic stepsize rules © 2018 W. B. Powell

Basic model Kesten’s stepsize rule © 2018 W. B. Powell Slide 12

Basic model © 2018 W. B. Powell

Basic model © 2018 W. B. Powell

© 2018 W. B. Powell

Derivative-based stochastic search n © 2018 W. B. Powell

Derivative-based stochastic search n Mapping to box constraints © 2018 W. B. Powell

Basic model n Objective function © 2018 W. B. Powell

Basic model © 2018 W. B. Powell

Uncertainty modeling © 2018 W. B. Powell Slide 20

Modeling uncertainty n © 2018 W. B. Powell

© 2018 W. B. Powell

Designing policies © 2018 W. B. Powell Slide 23

Policies n Stepsize policies » Harmonic (deterministic) © 2018 W. B. Powell

Policies n Stepsize policies » Kesten (stochastic © 2018 W. B. Powell

Policies n Stepsize policies » Ada. Grad (stochastic) » Discuss variable scaling © 2018 W. B. Powell

Policy evaluation © 2018 W. B. Powell Slide 27

Policies n © 2018 W. B. Powell

© 2018 W. B. Powell

© 2018 W. B. Powell

© 2018 W. B. Powell

Extensions © 2018 W. B. Powell Slide 32

Extensions © 2018 W. B. Powell

Extensions © 2018 W. B. Powell

Extensions © 2018 W. B. Powell

Extensions © 2018 W. B. Powell