Is 80 Accuracy Good Enough presented by Charles

Is 80% Accuracy Good Enough presented by Charles E. Olson, Jr. Senior Image Analyst Michigan Tech Research Institute Ann Arbor, Michigan Prepared for the ASPRS Semi-Annual Conference, Denver, CO, November 2008

Accuracy can mean different things to different people. This paper addresses only accuracy of thematic data.

Accuracy can mean different things to different people. This paper addresses only accuracy of thematic data. The Interagency Steering Committee on Land Use Interpretation and Classification wrote: “The minimum level of interpretation accuracy in the identification of land use and land cover categories from remote sensor data should be at least 85 percent. ” (Anderson, et al. , 1971).

Accuracy means different things to different people. This paper addresses only accuracy of thematic data. The Interagency Steering Committee on Land Use Interpretation and Classification wrote: “The minimum level of interpretation accuracy in the identification of land use and land cover categories from remote sensor data should be at least 85 percent. ” (Anderson, et al. , 1971). At the time this proposal was made, this level of accuracy was routine and it was essentially a “lowest common denominator” approach.

Washtenaw County Mapping Project Photo Scale: 1: 24, 000 Compilation Scale: 1: 48, 000 Classification Scheme Based on U. S. Geological Survey Circular 671 Hierarchical to Level IV Nine Level I Classes Twenty-eight Level II Classes Fifty-nine Level III Classes More than one thousand Level IV Classes Accuracy 96% at Level II 92% at Level III

How did we do it? By using all of the Image Interpretation “tools” Existing maps - sources of existing data about the terrain. Stereoscopes - viewing the photos and seeing the terrain in 3 D. Scales - for measuring and determining feature size. Terrain sense - understanding the multiple facets of any terrain. - animals, especially the human animal - soils and rocks - vegetation - topography - water The EIIs - elements of image interpretation

Computer screens are today’s medium of choice Human interpreters do on their computer screens what we used to do with paper prints. But the interpretation process remains the same. Computer users can drape maps over their image data, can measure and determine sizes, can view the terrain in 3 -D, can use the EIIs (but seldom do), and seldom have the broad sense of terrain required of a good interpreter.

The Elements of Image Interpretation (Olson, 1960) Shape Shadow Site Size Pattern Association Tone Texture Resolution Of these nine EIIs, computer-based interpretation algorithms rely almost completely on just one: Tone

The Elements of Image Interpretation (Olson, 1960) Shape Shadow Site Size Pattern Association Tone Texture Resolution Of these nine EIIs, computer-based interpretation algorithms rely almost completely on just one: Tone When used with the deductive process that Bob Colwell (1954) called “… the convergence of evidence … “ useful results almost always follow.

Agricultural Fields near Ann Arbor, MI What happened to these Septembersmall 11 “ponds? ” and Are they significant? Map Date: 1954 Photo Date: 1952 University of Michigan photo

Agricultural Fields near Ann Arbor, MI September 11 Low oblique photos taken in 1983. University of Michigan photo

Changing Relationships Rapid growth of computer based interpretation systems has resulted in lowering minimum level of accuracy to 80%. Cost of computer hardware and software, and the personnel to keep it running, often leaves no funds to do it any other way. If the computer can’t do it, it can’t be done. Rapid rise in GIS usage. As long as it fits neatly into the data base, it’s OK. Computer processing is often faster than manual processing. Is that speed really necessary when it comes at the cost of reduced accuracy?

How Much Accuracy Do We Really Need? Is 80% actually good enough? The market place says YES. Clients are willing to buy it. What is the long-range cost? A 1% increase can be worth $10 million. Will providing poorer data to decision makers come back to haunt our profession?

The market says YES, because clients buy it.

The market says YES, because clients buy it. But we can do better. 95% accuracy used to be routine.

The market says YES, because clients buy it. But we can do better. 95% accuracy used to be routine. If we didn’t get 95% accuracy we didn’t get paid.

Thank you for listening. I will be happy to answer questions if time permits, or after the session is over. Prepared for the ASPRS Semi-Annual Conference, Denver, CO, November 2008