Manipulating Sequential Files in Platform Independent COOL Gen

Manipulating Sequential Files in Platform Independent COOL: Gen Applications with Report Composer David Woodburn Director – Product Development Canam Software Labs, Inc. – Toronto, Canada

Introduction Canam Software Labs, Inc. founded in 1994 Gold sponsor of this year’s conference Computer Associates Partner OEM Partner for Seagate Software COOL: Gen products and services – Report Composer (includes File Handler creation) – Data Composer – Test Composer Used by over 135 customers in 32 countries Over 2000 product licenses sold in 5 years

Today’s Presentation Objectives Discuss need for sequential file processing Define sequential file concepts and requirements Identify programming language related data definition issues Define checkpoint restart and how to do it Future direction for file handlers Demonstration of file handling capabilities in Report Composer product

Why Do We Need Sequential Files? System interfaces – Legacy systems – Non COOL: Gen systems – Inter-corporate files Export/Import to non-relational products – Excel, Word Simplistic I/O for batch processes outside the Entity Relationship Diagram Sometimes needed for large batch reporting processes as an intermediate step

COOL: Gen Sequential File Challenges COOL: Gen primarily deals with relational DBMS Almost no support for accessing sequential files – Windows platform C language provides limited support Data types vary by programming language – platform independence difficult Access must be provided by writing External Action Blocks

What are EAB's ? EAB's are subroutines written outside of COOL: Gen – Linkage Section (COBOL) or function arguments (C) are used to communicate with calling procedure Used to perform tasks that COOL: Gen cannot – Non-relational File I/O – Report creation – SQL extensions – Operating system calls Compiled and statically-linked to the calling procedure to form a single load module COOL: Gen generates empty EAB’s containing the implementation of the API

Typical Prad Logic Open File Error handling Primary Loop Write Data Error handling Close File Error handling Set PARM 1 = ‘OF’ Set PARM 2 = ‘ ’ USE EAB 1 which imports. . . which exports. . . IF PARM 1 NE Spaces… Set PARM 1 = ‘GR’ Set PARM 2 = ‘ ’ READ EACH. . . USE EAB 1 which imports. . . which exports. . . IF PARM 1 NE Spaces. . . Set PARM 1 = ‘CF ’ Set PARM 2 = ‘ ’ USE EAB 1 which imports. . . which exports. . . IF PARM 1 NE Spaces. . . EAB 1

Report Composer API COOL: Gen Action Diagram Open the File Write a Record Pass data for output The CANAMPARMS work attributes control EAB actions. API structure is critical Return Code Data from File reader Report Composer EAB Import: Export: CANAM-PARMS DATA (Output) Report File Writer – Both the calling CANAM-PARMS DATA (Input) File Reader and the called components must understand the same API

Changes that impact the API Perform Version Control on the EAB if any of the following change: – Generating to a different platform – C to Cobol, Windows to Unix, etc. – Attribute properties – Domain, length, name, decimals, TD properties, etc – Order of attributes in EAB views – EAB Packaging (Program ID) – Generation options – Missing Flags, High Performance View passing – New version of COOL: Gen is implemented – EAB structures can sometimes change

Sequential File Terminology Sequential File Handlers – File reader EAB – Brings data into the Prad via its export views – File writer EAB – Sends data out of the Prad via its import views – can write new file, or append to existing file Each Sequential File – has properties – COBOL, C or Java type data fields – Positional/Delimited format – Variable or Fixed Record Layout – File has one or more layouts – Each layout has fields

File Properties

Dialect-based Properties Dialect involves national languages and/or standards Impact file writers/readers – Formatting numbers Thousands separator, decimal indicator – Currency symbol – – Formatting dates – Spelling of month names in national languages

Physical vs. Symbolic Names Files are referenced by either a physical or symbolic name – Physical name – is a complete (existing) Drive: PathFilename – is used in non-MVS environments – must be coded into the EAB or passed in via API – may want the ability to be over-ridden at run time – Symbolic Name – is a DDNAME in MVS or an environment variable on other platforms – must exist outside EAB (JCL, Autoexec. bat, SET A=C: ) – may want the ability to be over-ridden at run time

File Style – Positional or Delimited Positional – Consist of records of fixed lengths or variable lengths – Fields always start and end in the same position – Consist of text and/or numeric data – Non-text fields may be in internal formats Example of Positional Record with Variable Record Layouts

File Style – Positional or Delimited – Consist of records of varying lengths, but only a single ‘layout’ of these fields – Cannot have variable record layouts – Fields are always in the same relative order – Each field is separated by a delimiter – Consist of text and/or numeric data – The value of each field is easy to see – e. g. identifier, account balance, first name, middle name, last name 1001, 123. 02, John, , Smith 2929, 1089. 23, Maria, Alicia, Gonzales

Record Format Each positional file is either – Fixed format – Variable format – Each format has a different record layout Delimited files have only a single record layout – Length in bytes varies due to nature of delimited files Positional files may have one or more record layouts The COMMON FIELDS layout is always present – it is the only layout for fixed record format files – it contains only the fields common to all layouts for variable record format files

Variable Record Format COMMON FIELDS layout contains fields that occur in same position in all record layouts – One of these fields is the key field – Its value determines the type of record Each unique record layout is defined – Each has a unique value for the key field – Key value assigned when writing, interrogated when reading – Move data to/from the appropriate attributes from EAB views depending on which record layout

Record Layouts Contain Fields Layout and Key Field indicator Name of Field Description (optional) Domain - Text or Numeric Field alignment Data Types – COBOL (PIC X, PIC 9, PIC S 9 COMP, PIC S 9 COMP-3) – C (char, short integer, long integer, double) Defined Length - logical size – (derived physical length) Starting Column or Relative Position Decimal Places

Field-Attribute Mapping Map attributes to fields to – Define the source of date for fields on file writers – Define which attributes to move data into from fields on file readers Edit patterns used on cross domain mappings to define formatting requirements

Programming Language Dependencies Numeric data types are stored differently – COBOL : PIC 999, COMP-3, etc. – C and Java : Int, Long, Double, etc. – C cannot natively read COBOL numeric data types and vice versa Files written by EAB’s in one language cannot be read by another – Workaround by using only text fields EAB’s may need to be written in more than one language to facilitate platform portability Test files cannot be ported between platforms Flushing of buffers may vary by platform

Checkpoint Restart Used in high volume batch update procedures Provides the ability to RESTART an update procedure at various commit points in the process Need to save all relevant data as at the time of the checkpoint – Last record read/written All files (both input and output) have to be synchronized to the same point in the update process in the event of a restart

Checkpoint Restart Special Functions to be Performed Place a special attribute RECORDNUMBER in your EAB’s Import and Export views – Numeric attribute, no decimals – Used with a File Reader or File Writer EAB keeps track of which record number is being read or written – Returned in export views after each record is read or written

Checkpoint Restart Re-positioning within a File Handler Uses the special attribute RECORDNUMBER in your EAB’s Import views – Numeric attribute, no decimals – Used with a File Reader or File Writer Setting RECORDNUMBER to an integer N>0 on the “OF” (open file) call to the EAB will read the first N records – If N = 0 then the file is read or written from the beginning – If N > 0 the first “GR” will read/write record number N+1

Checkpoint Restart Reminders Place RECORDNUMBER in the EAB Export Views to return the ordinal number of the processed record Keep track of the record number in the main Prad, along with other checkpoint data Checkpoint data written at checkpoint intervals determined by the needs of the application

Evolution of File Handling Needs COOL: Gen Java support implies the need for Java language file handlers Canam is developing this capability as needs evolve Role of XML still unclear – Perhaps a new file type in addition to positional and delimited

Demonstration Report Composer automates the production of file readers and writers in COBOL and C on all platforms supported by COOL: Gen Identical capabilities will generate Java programs in future release