STRUCTUREFROMMOTION PIPELINES OR HOW TO RECREATE Daryl Van

STRUCTURE-FROM-MOTION PIPELINES; OR HOW TO RECREATE Daryl Van Dyke vandyke. geospatial@gmail. com

DISCLAIMER: � None of the materials shown here were developed on the tax-payer’s dime.

DISCLAIMER: � None of the materials shown here were developed on the tax-payer’s dime. � I’m personally taking my 15 -minute break during this talk, so it’s completely a recreational activity.

DISCLAIMER: � None of the materials shown here were developed on the tax-payer’s dime. � I’m personally taking my 15 -minute break during this talk, so it’s completely a recreational activity. � We’re only in it for the money lumpy gravy.

I HAVE HEARD OF UAV � I have heard that UAV – especially autonomous systems – could fundamentally change how we view imaging our world.

I HAVE HEARD OF UAV � I have heard that UAV – especially autonomous systems – could fundamentally change how we view imaging our world. � Between 1937 and 2000, ~ 66 wildlife employees perished due to aviation accidents doing wildlife surveys. D. Sasse, “Job-related mortality of wildlife workers in the United States, 1936 -2000”. WILDLIFE SOCIETY BULLETIN 31(4): 1015 -1020 · DECEMBER 2003

I HAVE HEARD OF UAV � I have heard that UAV – especially autonomous systems – could fundamentally change how we view imaging our world. � Between 1937 and 2000, ~ 66 wildlife employees perished due to aviation accidents doing wildlife surveys. D. Sasse, “Job-related mortality of wildlife workers in the United States, 1936 -2000”. WILDLIFE SOCIETY BULLETIN 31(4): 1015 -1020 · DECEMBER 2003 � How can this help me in my work?

MY OPTIONS: � As an analyst, one option is: � Cook up a project… � Convince the Chain-of-Command to go for it… � Start the COA process … � Engage my USGS partners, and try to get on the schedule… and probably find the funding for them to deploy… � And look to find support from my supervisor & chain of command to invest my time in learning this cutting-edge technology ….

MY OPTIONS: � As a lowly analyst, with no sway or influence on my program directions, I can… � Try to cook up a project… � Start the COA process … � Engage my USGS partners, and try to get on the schedule… � And look to find support from my supervisor & chain of command to invest my time in learning this cutting-edge technology ….

MY OPTIONS: � � � � � FAA Home ▸ Unmanned Aircraft Systems ▸ Key Initiatives ▸ Section 333 ▸ How to File a Petitioning for Exemption under Section 333 Due to the high volume of Section 333 petitions received, we are experiencing delays in processing petitions. We will do our best to process petitions being posted to the docket as soon as possible, and in the order they were received. We appreciate your patience as we work diligently to process your request. The below list contains step-by-step instructions for requesting FAA authorization to operate a UAS for civil (nongovernmental) purposes other than for recreation or hobby. Review the FAA's Guidelines for Submitting a Petition for Exemption. This website details the general exemption process for filing any petition for exemption, and it provides answers to frequently asked questions. Review the Section 333 Guidance (PDF) from the FAA's UAS Integration Office. This document describes what information petitioners should submit specifically to request FAA authorization to operate a UAS in the National Airspace System (NAS). Prepare your petition for filing. Before filing your petition, review our Section 333 Checklist and Guidance (PDF) to ensure your petition contains all the necessary information. When you have verified that all the necessary information is included, file your petition for exemption on the public docket. This docket has been created exclusively for the submission of Section 333 petitions. Submission instructions are available here (PDF). As of March 23, 2015, the FAA will automatically grant a "blanket" COA for flights at or below 200 feet to any UAS operator with a Section 333 exemption, provided the aircraft weighs less than 55 pounds, operations are conducted during daytime Visual Flight Rules (VFR) conditions within visual-line-of-sight (VLOS) of the pilot, and certain distances away from airports or heliports. Details are available here. AFTER receiving a grant of exemption, petitioners who want to fly outside these blanket parameters will be eligible to apply for a separate COA specific to the airspace required for their operation. Applications MUST be submitted through the UAS Civil COA Portal. DO NOT submit the COA application to the public docket. The FAA will ONLY accept and process complete COA applications MUST include: � � � An exemption number – corresponding to the Federal Register Docket ID for your petition for exemption A registration number – all aircraft must be registered with the FAA to be issued a COA NOTE: the COA application process is separate from the petition for exemption process. The COA process makes applicable FAA Air Traffic Control facilities aware of proposed UAS operations, and provides the FAA the ability to consider airspace issues unique to UAS operations. Both the COA application and the petition for exemption should be submitted under the same name/company name. Read the Section 333 FAQs for answers to common questions. Questions about the PETITION FOR EXEMPTION process should be directed to 333 exemptions@faa. gov. Questions about the COA APPLICATION process should be directed to 9 -AJV-115 -UASCOA@faa. gov.

MY OPTIONS: � � � � � FAA Home ▸ Unmanned Aircraft Systems ▸ Key Initiatives ▸ Section 333 ▸ How to File a Petitioning for Exemption under Section 333 Due to the high volume of Section 333 petitions received, we are experiencing delays in processing petitions. We will do our best to process petitions being posted to the docket as soon as possible, and in the order they were received. We appreciate your patience as we work diligently to process your request. The below list contains step-by-step instructions for requesting FAA authorization to operate a UAS for civil (nongovernmental) purposes other than for recreation or hobby. Review the FAA's Guidelines for Submitting a Petition for Exemption. This website details the general exemption process for filing any petition for exemption, and it provides answers to frequently asked questions. Review the Section 333 Guidance (PDF) from the FAA's UAS Integration Office. This document describes what information petitioners should submit specifically to request FAA authorization to operate a UAS in the National Airspace System (NAS). Prepare your petition for filing. Before filing your petition, review our Section 333 Checklist and Guidance (PDF) to ensure your petition contains all the necessary information. When you have verified that all the necessary information is included, file your petition for exemption on the public docket. This docket has been created exclusively for the submission of Section 333 petitions. Submission instructions are available here (PDF). As of March 23, 2015, the FAA will automatically grant a "blanket" COA for flights at or below 200 feet to any UAS operator with a Section 333 exemption, provided the aircraft weighs less than 55 pounds, operations are conducted during daytime Visual Flight Rules (VFR) conditions within visual-line-of-sight (VLOS) of the pilot, and certain distances away from airports or heliports. Details are available here. AFTER receiving a grant of exemption, petitioners who want to fly outside these blanket parameters will be eligible to apply for a separate COA specific to the airspace required for their operation. Applications MUST be submitted through the UAS Civil COA Portal. DO NOT submit the COA application to the public docket. The FAA will ONLY accept and process complete COA applications MUST include: � � � An exemption number – corresponding to the Federal Register Docket ID for your petition for exemption A registration number – all aircraft must be registered with the FAA to be issued a COA NOTE: the COA application process is separate from the petition for exemption process. The COA process makes applicable FAA Air Traffic Control facilities aware of proposed UAS operations, and provides the FAA the ability to consider airspace issues unique to UAS operations. Both the COA application and the petition for exemption should be submitted under the same name/company name. Read the Section 333 FAQs for answers to common questions. Questions about the PETITION FOR EXEMPTION process should be directed to 333 exemptions@faa. gov. Questions about the COA APPLICATION process should be directed to 9 -AJV-115 -UASCOA@faa. gov.

MY OPTIONS: � � � � � FAA Home ▸ Unmanned Aircraft Systems ▸ Key Initiatives ▸ Section 333 ▸ How to File a Petitioning for Exemption under Section 333 Due to the high volume of Section 333 petitions received, we are experiencing delays in processing petitions. We will do our best to process petitions being posted to the docket as soon as possible, and in the order they were received. We appreciate your patience as we work diligently to process your request. The below list contains step-by-step instructions for requesting FAA authorization to operate a UAS for civil (nongovernmental) purposes other than for recreation or hobby. Review the FAA's Guidelines for Submitting a Petition for Exemption. This website details the general exemption process for filing any petition for exemption, and it provides answers to frequently asked questions. Review the Section 333 Guidance (PDF) from the FAA's UAS Integration Office. This document describes what information petitioners should submit specifically to request FAA authorization to operate a UAS in the National Airspace System (NAS). Prepare your petition for filing. Before filing your petition, review our Section 333 Checklist and Guidance (PDF) to ensure your petition contains all the necessary information. When you have verified that all the necessary information is included, file your petition for exemption on the public docket. This docket has been created exclusively for the submission of Section 333 petitions. Submission instructions are available here (PDF). As of March 23, 2015, the FAA will automatically grant a "blanket" COA for flights at or below 200 feet to any UAS operator with a Section 333 exemption, provided the aircraft weighs less than 55 pounds, operations are conducted during daytime Visual Flight Rules (VFR) conditions within visual-line-of-sight (VLOS) of the pilot, and certain distances away from airports or heliports. Details are available here. AFTER receiving a grant of exemption, petitioners who want to fly outside these blanket parameters will be eligible to apply for a separate COA specific to the airspace required for their operation. Applications MUST be submitted through the UAS Civil COA Portal. DO NOT submit the COA application to the public docket. The FAA will ONLY accept and process complete COA applications MUST include: � � � An exemption number – corresponding to the Federal Register Docket ID for your petition for exemption A registration number – all aircraft must be registered with the FAA to be issued a COA NOTE: the COA application process is separate from the petition for exemption process. The COA process makes applicable FAA Air Traffic Control facilities aware of proposed UAS operations, and provides the FAA the ability to consider airspace issues unique to UAS operations. Both the COA application and the petition for exemption should be submitted under the same name/company name. Read the Section 333 FAQs for answers to common questions. Questions about the PETITION FOR EXEMPTION process should be directed to 333 exemptions@faa. gov. Questions about the COA APPLICATION process should be directed to 9 -AJV-115 -UASCOA@faa. gov.

OK – I’LL JUST RECREATE. � Buy my own platform and sensor � DJI Pro 3 – 4 K DJI sensor � Spend my nights and Saturdays on open-source development and processing � CERES, Eigen, Open. CV, Open. Sf. M, and Open. Drone. Map. � Build CUDA enabled versions on Linux (thanks Intel for that free Parallels C++ and MKL/IPP/TBB license) � Rewrite the core ODM work-flow to be OOP and try to convince them to take my pull requests. � Forget that – I’ll fork my python build into obj. Sf. M

OK – I’LL JUST RECREATE. � Buy my own platform and sensor � DJI Pro 3 – 4 K DJI sensor � Spend my nights and saturdays on open-source development and processing � CERES, Eigen, Open. CV, Open. Sf. M, and Open. Drone. Map. � Build CUDA enabled versions on Linux (thanks Intel for that free Parallels and MKL/IPP/TBB license) � Rewrite the core ODM work-flow to be OOP and try to convince them to take my pull requests. � Forget that – I’ll fork my python build into obj. Sf. M

OK – QUIT WHINING DARYL � What is Sf. M? � The inverse problem of the Panorama. Instead of a fixed camera, rotating projection plane. We have a fixed projection plane, and a translating camera.

WHAT IS SFM? The Inverse problem of the panorama.

WHAT IS SFM? � What is Sf. M? � The inverse problem of the Panorama. Instead of a fixed camera, rotating projection plane. We have a fixed projection plane, and a translating camera. � Auto-magic, linear-algebra driven photogrammetry � Leverages the burgeoning field of Computer Vision � Early work was in industrial applications (QC on

WHAT IS SFM? � Steps: � Key Point Detection � Key Point Matching � Key Point Filtering (removing bad matches) � Solve for camera positions (simultaneous solution or temporal gradient) � Solve for object vertices from camera positions and imagery qualities � Assemble vertex model � Process externally for surface models and texture maps.

VISUALSFM

I WANT IN, DARYL! � Data Collection: Use any set of overhead imagery (such as old airphoto sequences) Or try out some GPS-enabled camera shots (over a balcony, or building fronts)

I WANT IN, DARYL! � On Windows: �Visual. Sf. M (processing . Ply) �http: //ccwu. me/vsfm/ � Visualization, Node-editing, Meshing and Export � Mesh. Lab http: //www. 3 d-coform. eu/index. php/tools/meshlab

AND FOR UAV (W/ GPS EXIF) � On Linux � (Ubuntu 14. 04) Open. Drone. Map (https: //github. com/Open. Drone. Map ) � Mesh. Lab http: //www. 3 dcoform. eu/index. php/tools/meshlab

� Some Components of Open. Drone. Map are built for Windows w/ CUDA on my Github:

TAKE-AWAY � Sorry if this was a waste of your time… � I think this is crucial technology for my career, and the work of conserving our resources. � I sure could use your help – if you can code, fly, use a GPS enabled camera, etc. , please help.

FOR UNCLE SAM

THANKS !