CEOS ICF Baseband Data Archive Inter Change Format
CEOS ICF Baseband Data Archive Inter. Change Format Stéphane MBAYE GAEL Consultant Stephane. Mbaye@gael. fr Commitee on Earth Observation Satellites (CEOS) Working Group on Information Systems and Services (WGISS) Meeting Frascati, Italy Data SG – Archive Task Team - 9 th May 2002
Rationale Ø Document status Ø CEOS ICF - Changes Ø Presentation - “Data Request Broker” (DRB) Ø Demo – Derby application Data SG – Archive Task Team - 9 th May 2002
Document status Ø 21/04/2000 – Draft 0 - Creation of the document Ø 29/08/2000 – Draft 1 - Use of XML Schema instead of DTD Ø 09/01/2001 – Draft 2 - Includes outputs of Bangkok ATT meeting Ø 09/04/2001 – Draft 3 – Presented at Sioux Falls (USA) Ø 23/07/2001 – Draft 4 – Includes outputs of Sioux Falls ATT meeting Ø 23/08/2001 – Draft 5 – Vega France - Technical proof reading Ø 06/04/2002 – Draft 6 – ESA-ESRIN - Technical proof reading Ø 19/04/2002 – GAEL-CEOS-ICF-060. 2 – Issue 1, Revision 0 Data SG – Archive Task Team - 9 th May 2002
Partitioning – Analysis Ø For archives biggest than 2 Gigabytes Ø Continuous subsets of a single Baseband Interchange Unit Ø Free cutting policy Allows partitioning <<baseband. Unit>> CEOS ICF <<file>> Metadata File (XML) 1 <<uri. Reference>> 1. . * <<file>> Signal Data File (binary) Data SG – Archive Task Team - 9 th May 2002 Doc. § 3. 1
Partitioning – Declaration Ø New markup: <subset> Ø Moved attribute: “src” Ø Updated Naming convention ceos_icf subset signal_data 1 signal_data start : date. Time stop : date. Time <signal_data> <subset source="ERS-1 -1999 -03 -12 T 09: 38: 12. 354666 -1. dat"/> <subset source="ERS-1 -1999 -03 -12 T 09: 38: 12. 354666 -2. dat"/> <start>1999 -03 -12 T 09: 37: 12. 354666</start> <stop> 1999 -03 -12 T 09: 46: 52. 223553</stop> <block type=”composite”> … </block> </signal_data> Examples ERS-1 -19980225103347732 -19980225102345876 -1. dat ERS-1 -19980225103347732 -19980225102345876 -2. dat Data SG – Archive Task Team - 9 th May 2002 subset 1. . * block 1 block
CEOS ICF Namespace Ø New XML Namespace: http: //wgiss. ceos. org/ceos-icf-20020423 Ø Proposed XML Schema location: http: //wgiss. ceos. org/ceos-icf-20020423. xsd (TBC) Ø Use of xsd: target. Namespace in the CEOS ICF Schema Ø Recommend the use of xsi: schema. Location <ceos_icf xmlns=”http: //wgiss. ceos. org/ceos-icf-20020423“ xmlns: xsi=”http: //www. w 3. org/2001/ XMLSchema-instance” xsi: schema. Location=”http: //wgiss. ceos. org/ceos-icf-20020423. xsd”> … </ceos_icf> Doc. § 4. 1 Data SG – Archive Task Team - 9 th May 2002
Blocks Data SG – Archive Task Team - 9 th May 2002
Block interface <<interface>> block name : string type : string offset : positive. Integer length : positive. Integer occurrence : positive. Integer Data SG – Archive Task Team - 9 th May 2002
Composite block composite <block name=”Signal Data Record” type=”composite”> <occurrence>117235</occurrence> <block type=”telemetry_map”> <length>6120</length> <downlink_length>6120</downlink_offset> </block> Data SG – Archive Task Team - 9 th May 2002
Telemetry map telemetry_map downlink. Offset : positive. Integer downlink. Length : positive. Integer property [*] : string Data SG – Archive Task Team - 9 th May 2002
Telemetry map - example <block name=”Channel 1 status” type=”telemetry_map”> <-- Definition within the CEOS ICF Signal Data File --> <offset>2</offset> <length>2</length> <-- Corresponding subset within the telemetry --> <downlink_offset unit=”bit”>200</downlink_offset> <downlink_length unit=”byte”>25</downlink_length> <-- Applied transformations --> <property>pn_decoded</property> <property>compressed</property> <-- Just for example --> </block> Data SG – Archive Task Team - 9 th May 2002
Timestamp time_stamp facility Facility Data SG – Archive Task Team - 9 th May 2002
Timestamp - example <block type=”time_stamp”> <offset>2</offset> <facility> <organization>ESA</organization> <site>FUCINO</site> <country>ITALY</country> <system type="software"> <name>TMFR</name> <release>1. 4. 0</release> </system> </facility> </block> Data SG – Archive Task Team - 9 th May 2002
Quality flag quality_flag <block type=”quality_flag”> <length unit=”bit”>1</length> </block> Data SG – Archive Task Team - 9 th May 2002
ERS example <signal_data> <subset source=" ERS-2 -1999 -03 -12 T 09: 38: 12. 354666 -1999 -03 -12 T 09: 47: 52. 223553. 1. dat"/> <start>1999 -03 -12 T 09: 37: 12. 354666</start> <stop> 1999 -03 -12 T 09: 46: 52. 223553</stop> <block name="Signal Data Record" type="composite"> <occurence>6541</occurence> <block name="Frame 0" type="telemetry_map"> <length>253</length> <downlink_offset>3</downlink_offset> <downlink_length>253</downlink_length> </block> <block name="Frame 1 -28" type="telemetry_map"> <length>253</length> <occurence>28</occurence> <downlink_offset>259</downlink_offset> <downlink_length>253</downlink_length> </block> </signal_data> Data SG – Archive Task Team - 9 th May 2002
The “Data Request Broker” - DRB Data SG – Archive Task Team - 9 th May 2002
Derby Software DRB Data SG – Archive Task Team - 9 th May 2002 + extension
DRB architecture Applications XQuery scripts XQuery Engine Data Request Broker (DRB) Interface Facilities Schema File XML HDF SDF Implementations Data sources XML Schema Directories & Files XML docs HDF files CEOS ICF SPOT 5 - DIMAP Data SG – Archive Task Team - 9 th May 2002 Binary files ENVISAT-PDS Formats Landsat 5/7 CEOS, Fast SPOT 123 -4 etc.
Derby software (demo) Data SG – Archive Task Team - 9 th May 2002
Application 1 – Sharing knowledge description ESA CAO ESA + Industry Control Authority Office Version management Processor design in ground segments Validation, non regression ESA + Industry Format design iterative loop ENVISAT AATSR XML Schema ENVISAT MERIS XML Schema ENVISAT ASAR XML Schema ERS SAR XML Schema ESA Ground segments Product Quality Control Sciences community Other agencies Multi-disciplinary approach Archive interchange End users Data display, analysis Data SG – Archive Task Team - 9 th May 2002 ESA PDCC Long loop Quality Control
Application 2 – Towards a generic transcription 1 – Physical + logical transcription Data SG – Archive Task Team - 9 th May 2002 FRED to MDPS semantic translation CEOS ICF to MDPS semantic translation MDPS writer DOM CEOS ICF interchange Archive FRED reader FRED baseband Archive MDPS writer FRED to CEOS ICF semantic translation DOM FRED reader CEOS ICF format Generic « EO mission » model
Application 2 – Up to a generic transcription 2 - Logical transcription (DRB) CEOS ICF XML Schema or Generic « EO mission » model CEOS ICF interchange Archive DRB / SDF FRED baseband Archive Data SG – Archive Task Team - 9 th May 2002 FRED to MDPS semantic translation DRB / SDF CEOS ICF to MDPS semantic translation MDPS baseband Archive DRB / SDF FRED to CEOS ICF semantic translation DRB / XML MDPS XML Schema DRB / XML DRB / SDF FRED XML Schema
Application 2 – Up to a generic transcription 3 - Generic transcription CEOS ICF XML Schema or Generic « EO mission » model CEOS ICF interchange Archive DRB / SDF FRED baseband Archive Data SG – Archive Task Team - 9 th May 2002 Generic translation engine DRB / SDF Generic translation engine MDPS baseband Archive DRB / SDF Generic translation engine DRB / XML MDPS XML Schema DRB / XML DRB / SDF FRED XML Schema
Thank you ! Data SG – Archive Task Team - 9 th May 2002
- Slides: 24
