Experimental Finance Responsibilities of Coming of Age Shyam
Experimental Finance: Responsibilities of Coming of Age Shyam Sunder, Yale University Keynote Address, Society for Experimental Finance Tilburg University Tilburg, The Netherlands, June 27 -29, 2013
As a Discipline Grows Up • Since Chamberlin reported the results of his classroom economics experiment in 1948, the acceptability, recognition, role, and methods of this sub-discipline have evolved • Similar pattern has emerged in experimental finance since the early 1980 s • Unlike 1970 s and 80 s, when editors of economics, and then finance, journals routinely rejected experimental papers as a deviant curiosity, a recent issue of AER had more papers using experimental method than any other • Although its acceptability in finance lags economics, it is clear that the experimental method has grown beyond its “childhood” phase, is no longer “outside the tent” • Being inside the tent brings responsibilities of “adulthood” for a sub-discipline? 10/7/2020 Sunder: Experimental Finance 2
Responsibilities of a Mature Discipline • Identifying core concerns of the discipline on substantive, not just methodological grounds; • Going beyond show-and-tell to impress “adults”. No more: – Look no Hands, Ma! – I can do what you can do. • Contribution to core concerns of a discipline • Constructive interchange with sister methodologies of the discipline • Balance between advancement of method and substantive knowledge of real phenomena • Statistical methods permit rejection of null or failure to reject the null – No such thing as confirmation of hypothesis – The null is the current belief; not arbitrary 10/7/2020 Sunder: Experimental Finance choice 3
Seven Special Concerns • • • Robustness Time scale Risk: what is it? Institutions Properties of institutions, or individual behavior – Individual behavior: distinguish unobservable traits from observable actions (vs. Re-labeling actions as traits) • Is the experimenter a part of the game; subject expectations • Laboratory results as the final word, and the main source of research questions 10/7/2020 Sunder: Experimental Finance 4
Core concerns of the discipline: substantive, not just methodological • Disciplines get sterile when methods take the front seat, obscuring their classic or newly-identified substantive questions • While methodological development is necessary part of a healthy discipline, the dominant concern must still be with a better understanding of relevant aspects of the world we live in • What proportion of the effort of the discipline goes into research about questions about the world (external references), and questions about research itself and its methods (internal references)? • A simple test: try explaining our research question (and results) to our parents, (any non-expert, really) to assess if they appreciate what we contribute to human civilization with the resources we get to spend 10/7/2020 Sunder: Experimental Finance 5
Contribution to core concerns of a discipline� • Where do we look for questions to address? – On the street, news, and direct observation of the world – Questions arising in the classroom (by students as well as in our own minds) that we cannot answer to their or our satisfaction – Unresolved (perhaps abandoned) puzzles of the discipline – Incremental variations on recent publications – Proving your advisor or academic god-parent right – Because it may get published • Identifying core concerns of economics/finance that could not be addressed without experiments • What are the core concerns of economics/finance and sister disciplines such as psychology? • Do we need to distinguish among them? Does making distinction imply un/willingness to learn from others? 10/7/2020 Sunder: Experimental Finance 6
Constructive interchange with sister methodologies • Contribution of experimental method will also depend on how well we are able to take advantage of constructive interchange with sister methodologies of economics/finance • For example, theory and mathematical modeling; econometric modeling and estimation/testing with data from the field • What can be a constructive relationship between theory and experiments? 10/7/2020 Sunder: Experimental Finance 7
Robustness and Assumptions • We build models is to gain a better understanding of some real phenomena of interest • Models may use mathematics to understand the world, but are not mathematics (do not pursue math for its own sake) • Real phenomena are complex (perhaps infinitely detailed); rarely possible to understand/characterize them completely • Theory identifies one, or a few, critical variables from a large set to gain a satisfactory (not perfect) understanding of the phenomenon of substantive interest • Theories are neither wrong nor right; some are more helpful/robust than others in explaining/predicting/gaining insight • Compare theories on basis of their helpfulness in explaining/understanding the real phenomena of interest 10/7/2020 Sunder: Experimental Finance 8
Fractals: Infinite Detail 10/7/2020 Sunder: Experimental Finance 9
Nature of Theory • Essence of theory lies in its simplicity; we understand by simplifying • Simplification by abstraction from details of real phenomena • Assumptions are the way of discarding the great mass of detail to focus on one/few key factors; and adding a few for analytical convenience • Every model consists of : key assumptions and assumptions of convenience • Lack of correspondence between assumptions of convenience and reality is the essence of a theory, and not a defect of theory (no assumptions, no theory) • Key assumptions must hold; convenience assumptions need not 10/7/2020 Sunder: Experimental Finance 10
Empirical Test of a Theory • Theory is to real phenomena what a drawing or stick figure is to the human body, or a map is to earth’s surface • Correspondence is crude on purpose; it captures some, and only some, essential feature(s) which are relevant to the purpose on hand • Model identifies some tautologies which are necessarily true when assumptions hold (unless there are mathematical errors in derivation): If (x, y, z) P • What does it mean to empirically test a theory in the laboratory? 10/7/2020 Sunder: Experimental Finance 11
Single Theory Experiments • When only one interesting theory is available for the phenomenon of interest • “Test” is an assessment of robustness of theory to deviations from assumptions of convenience • If data are gathered from an environment that corresponds exactly to the assumptions of theory, we should expect no deviations (if we do observe deviations, either theory has error or the correspondence is missing) • Empirical test is a costly method of discovering errors of derivation 10/7/2020 Sunder: Experimental Finance 12
Creating Theoretical Model in Lab • Exact correspondence to theory in either the field or lab is rarely achievable • Even if we could, little can be learned from it except about any math errors of derivation or in lab/model correspondence; expensive way to discover errors • Error in derivation or lab environment or data collection • Little useful scientific inference is possible from perfect correspondence between theoretical model and its laboratory implementation 10/7/2020 Sunder: Experimental Finance 13
Scientific Value of A Single-theory Empirical Test • Assessment of how robustly the predictions of theory correspond to data as more of the convenience assumptions are relaxed in the environment of data source • See Figure 1: • At the origin, x-axis shows that data is gathered from a lab environment in which all assumptions of the model (core as well as convenience) hold; i. e. , zero distance between them • Under these conditions the correspondence between the data and the model prediction should be perfect. • If it is not found to be perfect, what could the reasons be? – Mathematical error in the model – Lack of correspondence between the model and the lab environment, i. e. , error in implementing the model in lab 10/7/2020 Sunder: Experimental Finance 14
Single Theory Experiment 10/7/2020 Sunder: Experimental Finance 15
What Can We Learn from Such a Lab Experiment? • Not much • If there is a math (derivation) error in the model, laboratory experiment is an expensive way for finding out such errors (try Mathematica instead!) • If the lab environment does not exactly replicate all (core and convenience) assumptions of the model, there is error of implementation. • Again, actually running the experiment is an expensive way of finding out such errors • In neither case, does the experiment enlighten us with an answer to the inquiry 10/7/2020 Sunder: Experimental Finance 16
What Can Experiments Do? • Systemcatically relax/drop the assumptions of convenience in the model • Remove these assumptions one by one, and move the design of the experiment to the right on the x-axis • For each design, assess the degree of correspondence between the model prediction and the data • See how rapidly the explanatory power of the model declines as the lab environment drop more assumptions of convenience • See A, B and C as three candidate models, for example 10/7/2020 Sunder: Experimental Finance 17
Single Theory Experiment 10/7/2020 Sunder: Experimental Finance 18
Single Theory Experiment 10/7/2020 Sunder: Experimental Finance 19
Single Theory Experiment 10/7/2020 Sunder: Experimental Finance 20
Robustness of Predictive Power to Assumptions of Convenience • Under all three cases, the model is literally true (when all its assumptions hold). • However, as the environment deviates from the strict assumptions, A’s predictive power declines more slowly than B’s • C’s predictive power declines very rapidly • C is valid literally, but its power to explain the world we live in is likely to be limited, compared to A and B • Which would you favor as a better model/theory? 10/7/2020 Sunder: Experimental Finance 23
How Do We Identify Key the Assumptions? • Distinguishing between model and theory • Model is a (“stick figure”) logical structure; a theory uses the model to suggest some statements about the real phenomena of interest • Think of the real phenomena that motivates the model and theory as the principal • Ask which assumptions of the model are intended to limit the real environments sought to be understood; those are the key assumptions • Number of states, preferences, probability distributions, etc. , tend to be assumptions of convenience, because they are not intended to limit application of the model (unless you are interested only in a 2 -state world, for example) 10/7/2020 Sunder: Experimental Finance 24
Design of Robustness Experiments • Conduct a series of experiments, all holding the key assumptions, and progressively relaxing the convenience assumptions (e. g. , the number of states) • Conduct a series of experiments progressively increasing the number of alternative choices available to subjects (increasing the number of possible outcomes) • If the predictive power of the model is relatively robust as shown by data when more alternatives are available, result is more robust; e. g. , Vernon Smith (1962) paper 10/7/2020 Sunder: Experimental Finance 25
Fig. 3: Single Theory Experiment Smith (1962) 10/7/2020 Sunder: Experimental Finance 26
Why Was the Smith (1962) Experiment So Powerful? • It dropped a whole basketful of assumptions which had been thought to be at the core, but turned out to be mere convenience in the basic supplydemand equilibrium model – No tatonnement – Hardly perfect competition – Only private erfect information – Profit motivated, but hardly optimizers • Showed the model to be far more robust than even the most ardent supporters had claimed (or even imagined) 10/7/2020 Sunder: Experimental Finance 27
But There Was More! • Gode and Sunder (JPE 1993): Combine – Double auction market institution (without randomization in Smith), and – Budget constrained random choice (without market institution in Becker 1962) – Found that allocative efficiency is largely a function of the double auction (ZI traders) 10/7/2020 Sunder: Experimental Finance 28
Market Behavior 1 10/7/2020 Sunder: Experimental Finance 29
Market Behavior 3 10/7/2020 Sunder: Experimental Finance 30
10/7/2020 Sunder: Experimental Finance 31
Fig. 4: Multi-Theory Experiment, Plott and Sunder (JPE 1982) 10/7/2020 Sunder: Experimental Finance 32
But How and Why? • While Plott and Sunder (1982) found that double auction markets could disseminate information from insiders to non-insiders with human traders • Human faculties are sufficient, but are they necessary? • Again, Jamal, Maier and Sunder (2012) replaced human traders in Plott and Sunder market design by minimally intelligent traders to find out if double auction markets without the entire portfolio of human faculties can still achieve this task 10/7/2020 Sunder: Experimental Finance 33
Design of Algorithmic Traders • Populate double auctions with simple algorithmic traders with two characteristics – Means-end heuristic to adjust their aspiration levels – Zero-intelligence bids and offers relative to the current aspiration levels 10/7/2020 Sunder: Experimental Finance 34
Means-End Heuristic • Use Newell and Simon (The Simulation of Human Thought, 1959) – Given a current state and a goal state, an action is chosen which will reduce the difference between the two. The action is performed on the current state to produce a new state, and the process is recursively applied to this new state and the goal state • Initial aspiration – For the uninformed: expected payoff – For the informed: pay off under the known state • Adjustment of aspiration levels with each observed price P(t) using parameter α ~U(0. 05, 0. 5) – ASP (t+1) = (1 -α) ASP (t) + α P(t) 10/7/2020 Sunder: Experimental Finance 35
Zero-Intelligence Bids and Offers • Bids: • Offers: ~ U(0, ASP) ~U(ASP, 1) • Double Auction Rule – At the start of each period – Set current bid = 0, and current ask = 1 – A higher bid replaces current bid, and a lower ask replaces current ask – As soon as a current bid equals or exceeds current ask, a transaction is recorded at the bid/ask submitted first 10/7/2020 Sunder: Experimental Finance 36
Results • Plott and Sunder (1982) results of markets with profit-motivated human traders shown in blue • Simulated results of same markets populated by minimally-intelligent algorithmic traders shown in red (cloud of 50 independent simulations and the median, α ~U(0. 05, 0. 5) drawn for each simulation) • RE equilibrium price in green line • Walrasian equilibrium price in broken line • Also shown, comparisons of mean squared deviation, trading volume, and allocative efficiency 10/7/2020 Sunder: Experimental Finance 37
10/7/2020 Sunder: Experimental Finance 39
10/7/2020 Sunder: Experimental Finance 40
10/7/2020 Sunder: Experimental Finance 41
10/7/2020 Sunder: Experimental Finance 42
10/7/2020 Sunder: Experimental Finance 43
10/7/2020 Sunder: Experimental Finance 44
10/7/2020 Sunder: Experimental Finance 45
10/7/2020 Sunder: Experimental Finance 46
10/7/2020 Sunder: Experimental Finance 47
10/7/2020 Sunder: Experimental Finance 48
10/7/2020 Sunder: Experimental Finance 49
10/7/2020 Sunder: Experimental Finance 50
10/7/2020 Sunder: Experimental Finance 51
10/7/2020 Sunder: Experimental Finance 52
10/7/2020 Sunder: Experimental Finance 53
10/7/2020 Sunder: Experimental Finance 54
1. Robustness Summary • Experimental method is a powerful device to subject economics/finance models and theories to robustness check to help distinguish those which hold a greater promise to give us a better understanding of the world we live and work in • Checking robustness of a model calls for observations in lab environment that is designed to deviate in assumptions of convenience from the model(s) under scrutiny 10/7/2020 Sunder: Experimental Finance 55
2. Time Scale • Most economic models include time dimension (usually denoted by symbol t • Few models specify what t represents in real terms— seconds, hours, days, years, or generations • Presumably, such theories are so general that they holds for all interpretations of the time interval in real units • Lab experiments could be a way of finding the appropriate interpretations of time in specific theories, in case they exist, and thus make a significant contribution of economic/finance theory 10/7/2020 Sunder: Experimental Finance 56
3. Risk: What Is It? • Dispersion or possibility of loss? • What is the empirical content of risk attitudes (expected utility) applied to curved functions of a increasingly rococo variety • Predictive content out of sample and context • Evidence on macro phenomena (medicine, sports, drugs, gambling, insurance, credit, equity, labor, monetary, real estate parts of economy • Is risk attitude a scientific concept, a modern day equivalent of the eighteenth century “phlogiston” in chemistry to explain combustion, or just a plug for whatever we do not know? • What has been the contribution of experimental finance to this key concept of finance theory and practice? 10/7/2020 Sunder: Experimental Finance 57
4. Institutions • Experimental methods have highlighted the importance of economic institutions, their properties, and their evolution over time • However, study of institutions in lab presents a special challenge • Most individual decisions involve choice of a point on a function, but institutions being functions themselves, examination of their evolution calls for choices from a set of functions • Choice on a function and of a function call for very different cognitive skills, experience, and time, and are difficult to study in the few hours of a typical session 10/7/2020 Sunder: Experimental Finance 58
5. Properties of Institutions (or Individual Behavior) • Experiments have been employed to identify the properties of institutions • Real life institutions have great deal of detail, and thus can be simplified for laboratory use in thousands of ways • When we try to use experiments to identify institutional properties, how do we choose which implementation of the institution in the lab is appropriate? • I have found no convincing answer to guide me in deciding on the lab implementation of a real life institution other than dropping assumptions of convenience mentioned above 10/7/2020 Sunder: Experimental Finance 59
6. Experimenter as a part of the game • What are the boundaries of the game we hypothesize the subjects to be playing? • What do we know about the expectations subjects bring to the lab? What, if any, control can we exercise on their expectations • Is experimenter inside that boundary or outside? • How do we keep ourselves outside the boundary? • Is it enough to tell them so? 10/7/2020 Sunder: Experimental Finance 60
7. Lab results as the final word? • When can we stop with the lab results, convinced that we have a good understanding of the phenomenon of interest? • When do we need to follow up the lab results with data from the field? • More engagement with sister research methods 10/7/2020 Sunder: Experimental Finance 61
Fundamental Principle of Research Designs (after Einstein) • Research design should be as simple as possible, but no simpler, to answer the question posed. 10/7/2020 Sunder: Experimental Finance 62
Research Questions • What question do you wish to answer with your research? • A question is one sentence with a question mark at the end (? ). • It should be a question whose answer you would like to know, but do not know • After asking your friends, if you are the only one who does not know, think again, unless you have reasons to disagree with them • What might the possible answers to the question? • How could one distinguish what is a better answer? • What is the best way to answer the question? Not necessarily an experiment 10/7/2020 Sunder: Experimental Finance 63
Finance Challenges for Experimental Method • Risk • Information and individual decision making • Interplay between financial decisions, reporting, and engineering • Financial institutions • Financial regulation and laws • Robustness check • Finance is related to, but is not economics 10/7/2020 Sunder: Experimental Finance 64
Shyam. sunder@yale. edu www. som. yale. edu/faculty/sunder/res earch Sunder: Experimental Finance
- Slides: 62