Digital Imagery Policies Standards and Guidelines InterAgency Digital

Digital Imagery Policies, Standards, and Guidelines Inter-Agency Digital Image Working Group October 2005 U. S. Department of the Interior U. S. Geological Survey

Quality Assurance Plan For Acquisition of Digital Imagery Inter-Agency Digital Image Working Group October 2005 U. S. Department of the Interior U. S. Geological Survey

Bottom-Line Goal n Ensure overall quality of image products and services to end-users

Today’s agenda n n Background The Proposed Quality Plan Issues to be resolved Next steps

Background n n ASPRS camera calibration report Interim solutions by USGS Formation of Inter-agency Digital Imagery Working Group (IADIWG) Focus Meeting in Denver

Proposed Four Step Quality Plan n n Manufacturers’ certification process Data Providers’ certification process Image specifications and selection of contractors Inspection and acceptance of deliverables

Manufacturers’ Certification Process n n n USGS will establish a Digital Sensor Certification Team to review and certify manufacturer’s sensor calibration process Initial certification based on review of documentation and on-site visit Certification letter and report issued to manufacturer

Manufacturers’ Certification Process n Core members of team will include: USGS management official F Expertise in radiometry, geometry, optics, system engineering, quality, standards F Specialized expertise, if necessary F

Manufacturers’ Certification Process n n Re-certifications due to calibration process changes and engineering modifications Request by letter and documentation On-site visit only when deemed necessary Re-issued certification letter and report as needed

Data Providers’ Certification Process n n n USGS will establish a Data Provider Certification Team to review and certify Data Provider’s sensor calibration and quality assurance process Initial certification based on review of documentation and on-site visit, if required Re-certification required every 3 years

Data Providers’ Certification Process n Short-term: Certify Data Provider’s calibration procedures and initial calibration results F Verify accuracy and image quality of specific products (i. e. , Product characterization) F

Data Providers’ Certification Process n Long-term: Development of certification guidelines and procedures F Use of guidelines is a requirement or factored into the evaluation of Data Providers by the contracting officer F

Contracting Process n The basic federal contracting process is: F F F Seek appropriate internal authorizations and establish availability of funding Develop written statement of work of technical requirements for products and services Develop evaluation criteria for selecting contractor The Contracting Officer (CO) determines the type of procurement mechanism or type of contract Publish Request for Proposal (RFP)

Contracting Process n RFPs include: F F F Complete statement of work of products and services to be provided by the contract (performance based) Contractor instructions for preparation of technical and pricing proposals Selection criteria and relative weighting factors Related documents to be provided with proposal Deadline for submittal of proposal

Contracting Process: Boiler Plate Guidelines n Process used in developing the draft guidelines: All existing specifications among the IADIWG agencies were gathered and compiled into a single document F Review and agreement to organization and common terms F

Contracting Process: Boiler Plate Guidelines n Goal: To establish a boiler plate document that could be used by all agencies as a starting point in image acquisition procurements F To use common terms and common language in specifying requirements F To seek industry review for correctness and understanding F

Contracting Process: Boiler Plate Guidelines n Future plans: Incorporate changes from user community F Provide document to other agencies via web-site and to State liaisons F Establish on-line application to “write” agency specification F

Contracting Process n Contractor selection process: F F F Contract Specialists determine if proposals submitted meet all minimum contract requirements and deadlines Contracting Officer establishes a technical evaluation panel with instructions to review and rate the technical proposals Evaluation panel provides proposal ratings and recommendations to the CO

Contracting Process n Example of initial evaluation factors: Project management capabilities statements F Past performance F Quality control system statements F Personnel qualifications F Aircraft and camera availability F Workload and incomplete contracts F

Contracting Process n BUT, selection factors during course of multi-year contract tend to be: Past performance F Quality of past deliverables F QA/QC of subcontractors F

Contracting Process n Contract negotiation and award process: CO clarifies any questions relating to the proposal to ensure a fair evaluation F CO conducts negotiations with contractors established within the competitive range F CO selects highest rated contractor F CO awards the contract F

Contracting Process n IADIWG recommends the following to evolve into requirements for all contracts: Factory calibration report for all sensors used F Manufacturers’ certification or equivalent F Data Provider’s latest calibration report F

Contracting Process n IADIWG recommends these factors become part of the evaluation process: F F F Data Provider’s sensor acceptance test plan and results Data Provider’s documentation of processes that ensure sensor is properly maintained and operated Records to indicate that procedures have been followed and that the sensor is in good calibration

Inspection and Acceptance of Deliverables n What is quality? I don’t know, but I know it when I see it! F Inspection and acceptance, and quality control requirements are generally spelled out in the contract specifications; however, this does not ensure quality F

Inspection and Acceptance of Deliverables n Quality: Does it meet contract specifications? F Does it meet user needs? F

Inspection and Acceptance of Deliverables n What’s easy? F Objective calls: t t Specific tolerances Formatting Media Naming convention and labels

Inspection and Acceptance of Deliverables n What’s hard? Unrealistic tolerances because of improper or impossible specifications F Meta data F Determining image quality (subjective calls) F

Inspection and Acceptance of Deliverables n What’s really impossible? 100% inspection F Agreement on image quality F Improving image quality when source is bad F

Inspection and Acceptance of Deliverables n What’s needed? F F F Early inspection of sample from contractor Delivery in small lots Practical specifications for such things as tone matching, color balance, and edge joins Better, less subjective, more automated means of determining image quality (e. g. , commercial NIIRS rating) Standardized quality assurance/quality control methods

Issues to be resolved n n Who pays and how much? Detailed plan for certification and recertification ISO certification vs. IADIWG certification? Guidelines for using local in-situ ranges for calibration

Next steps n n Invitation for comment to ASPRS and MAPPS membership Communicate plan to mapping community through education, publications, and USGS State Liaisons Final plan presented at Reno 2006 Begin manufacturers’ certification process

For more information: n http: //calval. cr. usgs. gov/

Contacts n Greg Stensaas, 605 -594 -2569, stensaas@usgs. gov n George Lee, 650 -329 -4255, gylee@usgs. gov