Regret in the OnLine Decision Problem Dean Foster
Regret in the On-Line Decision Problem Dean Foster & Rakesh Vohara Presented by: Tom Whipple 2/7/2006
Outline • Intuition • Notation • Definition of Regret – Internal Regret – No Regret – External Regret • Theorem 2 • How do we use Regret? – Stock market example
What is Regret? • Informal: I wish I had decided x instead of y. • Goal: Quantify this, accounting for: – “How bad? ” (I could have made $1, 000) – “How likely? ” (But it wasn’t very likely)
Notation • S denotes an arbitrary “scheme” for making decisions • T is the total number of time steps t. • the set of possible decisions. • Loss for decision i at time t. • stochastic vector s. t. is chosen at time t with probability
Definition: Regret • Total Regret from using scheme S is: Weighted by probability of making decision j at time t. Regret cannot be negative for any decision. Difference in loss from deciding j instead of i. If < 0, i is better than j at time t.
No (Internal) Regret • If the scheme S is asymptotically close to the number of time intervals T then S is said to have no internal regret. • “Internal” refers to the regret of decisions within S.
No External Regret • Scheme S has no external regret if • Uses Expected Loss • Note that • “External” refers to comparing scheme S to other schemes. • Intuition: The expected loss from S is no more than the expected loss from the best P.
Theorem 2 “Given any finite set of decision schemes F, there exists a (randomized) decision scheme S with no external regret w. r. t. F ” Intuition: There is some S, not in F that is at least as good (asymptotically) as the best scheme in F.
Application: Stock Selection • Theorem 2 shows we can approach best constant portfolio. – Constant refers to relative allocation (BCRP). – This is the same result we saw in previous paper. • In the context of this paper, is constant for
- Slides: 9