YAML A New Language for Data INGY presents

YAML: A New Language for Data INGY presents: YAML@ YAPC

YAML: A New Language for Data &whois("Brian Ingerson"); • • • Libraries and Mainframes Assembler to Perl Term: : Scraper Inline YAML use Your. Brain. For. Once. Dude; Active. State Fat. Port Data. Built

YAML: A New Language for Data What is YAML? • Your Average Meat Loaf? • Yasser Arafat Making Lasers? • You'll Argue My Logic? • YAGNI Ass Mutha Load? • wh. Y Are Mongers Laughing? • Yet Another Module Legend? • Yet Another Markup Language • ? ? ? • YAML Ain’t Markup Language!

YAML: A New Language for Data What is YAML? • • YAML is a Data Serialization Language. A New Language for Data Modeled after Perl-like structures But not just intended for Perl • XML is a Markup Language • Sort of. . . • We use XML mostly for serialization

YAML: A New Language for Data YAML Design Goals YAML design goals: - YAML documents are very readable by humans. - YAML interacts well with scripting languages. - YAML uses host languages native data structures. - YAML has a consistent information model. - YAML enables stream-based processing. - YAML is expressive and extensible. - YAML is easy to implement. (cat | ysh)

YAML: A New Language for Data Ingy's Design Goals • • • To achieve Human/Computer Nirvana High Data/Markup (Signal/Noise) Ratio To be the world's best Data: : Dumper To be great for Configs, Caches, and Logs One Serialization to Rule Them All. . . and in the YAML bind them. Other Goals: • Unicode based • Speedy parsing • Alternative to XML (for some applications)

YAML: A New Language for Data Where YAML Came From • • • Inline. pm Data: : Denter Clark Evans Oren Ben-Kiki sml-dev yaml-core mailing list • • mini P 5 P 2000+ messages last year irc. debian. org #yaml Summits

YAML: A New Language for Data: : Denter and Inline md 5 : cc 9 b 569052 f 4 daa 5 b 343 b 1 dcb 94 dd 2 bc name : e_cc 9 b language : C date_compiled : Wed Jun 12 12: 48: 00 2002 inline_version : 0. 43 ILSM : % module : Inline: : C suffix : bundle type : compiled Config : % apiversion : 5. 005 archname : darwin ccflags : -g -pipe -fno-common osname : darwin osvers : 1. 4 so : dylib version : 5. 6. 0

YAML: A New Language for Data Influences --scripting languages: - Perl - Python -C - Java standards: - RFC 0822 (MAIL) - RFC 1866 (HTML) - RFC 2045 (MIME) - RFC 2396 (URI) protocols: - SAX - SOAP - XML

YAML: A New Language for Data YAML Syntax Basics • • • Mappings Sequences Streams and Documents Comments Scalars • • Simple Quoted Block Folded • Wiki • Escaping • Anchors & Aliases

YAML: A New Language for Data Mappings • A YAML mapping is like a Perl hash • Unordered Key/Value pairs • Separated by ': ' (space is mandatory) --name: Benjamin rank: Private serial number: 1234567890 12: 34 PM: My favorite time

YAML: A New Language for Data Sequences • A YAML sequence is like a Perl array • An ordered collection of data • YAML has a bullet like syntax '- ' --- red - white - blue - pinko

YAML: A New Language for Data A YAML Grocery List --Fruits: - Apples - Tomatoes Veggies: - Spinach - Broccoli Meats: - Burgers - Shrimp Household: - Candles - Incense - Toilet Duck

YAML: A New Language for Data The Matrix ---3 -5 -7 -0 -0 -7 -9 -1 -1

YAML: A New Language for Data Outline - Intro Part 1: - Up - Down - Side to Side - Part 2: - Here - There - Underwear - Part 3: - The Good - The Bad - The Ingy

YAML: A New Language for Data Comments • Comments/blank lines can go almost anywhere • Must not be ambiguous with content • Comments begin '# ' (almost like Perl) # comment before document --- #DIRECTIVE # comment foo: bar # inline comment phone: number #555 -1234 ### Comment fact: fiction --blue: bird # Comment

YAML: A New Language for Data Scalars • A YAML scalar is like a Perl scalar • YAML has many scalar forms • TMTOWTDI --simple: look ma, no quotes quoted: - 'Single quoted. Like Perl, no escapes' - "Double quotes. n. Like Perl, has escapes" -| A YAML block scalar. Much like Perl's here-document.

YAML: A New Language for Data Simple (unquoted) Scalar • Can use for almost all simple values • Some content limitations --simple key: simple value this value: can span multiple lines but the key cannot. it would need quotes stuff: - foo - 42 - 3. 14 - 192. 168. 2. 98 - m/^(. *)//

YAML: A New Language for Data Single Quoted Scalar • No escapes (like Perl) • Use when simple won't work --'key contains: colon': '$19. 99' or: ' value has leading/trailing whitespace ' 'key spans lines': 'double ticks for ''escaping'''

YAML: A New Language for Data Double Quoted Scalar • Allows escaping (like Perl) --The spec says: "The double quoted style variant adds escaping to the 'single quoted' style variant. " like this: "null->z newline->n bell->a smiley->u 263 a" self escape: "Brian "Ingy" Ingerson"

YAML: A New Language for Data Block Scalar • Blocks are like here-docs on vitamins • Newlines are hard • Termination is determined by indentation --what is this: | is it: a YAML mapping or just: a string chomp me: |sub foo { print "Love me do!"; }

YAML: A New Language for Data Embedding YAML in YAML • No Escaping of Markup Required • Simply use the Block Form • YAML Test Suite does this --- #YAML: 1. 0 old doc: | --- #YAML: 1. 0 tools: - XML - XSLT new doc: | --- #YAML: 1. 0 tools: - YAML - c. YATL

YAML: A New Language for Data Folded Scalar • Folded scalars work like HTML • Single newlines indicate a space character. • Empty lines mean a hard newline ---> Copyright © 2001 Brian Ingerson, Clark Evans & Oren Ben-Kiki, all rights reserved. This document may be freely copied provided that it is not modified. Next paragraph. - foo

YAML: A New Language for Data Wiki Content • Similar to POD or some Wikis • Indented text preserves line endings --The YAML Specification starts out by saying: > YAML(tm) (rhymes with "camel") is a straightforward machine parsable data serialization format designed for human readability and interaction with scripting languages such as Perl and Python. YAML documents are very readable by humans. YAML interacts well with scripting languages. YAML uses host languages' native data structures. Please join us, the mailing list is at Source. Forge.

YAML: A New Language for Data Nested Scalar Key • Sometimes you want fancy blocks for map keys • Know what I mean? --? >+ Even a key can: 1) Be Folded 2) Have Wiki : cool, eh?

YAML: A New Language for Data Anchors and Aliases • YAML can handle duplicate and recursive data • It uses an alias mechanism • Aliases do not span documents --Hey Jude: &chorus - na, na, - &4 na, na, - *4 - Hey Jude. - *chorus

YAML: A New Language for Data A Hard Example • Here is an example of preserving hard refs • Not sexy, but neither are hard refs. . . use YAML; $a=\\"foo"; $b=$$$$a; print Dump [$a, $b], [$b, $a];

YAML: A New Language for Data Using YAML. pm • Just like Data: : Dumper • perl -MData: : Dumper -e 'print Dumper %INC' • perl -MYAML -e 'print Dump %INC' • 10 character savings over D: : D • Basic API • Dump • Load • Other Functions: • Dump. File • Load. File • freeze(), thaw() # POE • Object Oriented # Not discussed

YAML: A New Language for Data YAML vs Data: : Dumper • YAML Pros: • • • Cleaner format Safe deserialization Hashes sorted by default Escapes unprintables Supports GLOB, CODE, Regexp and Filehandle Interacts with Ruby, Python, Java etc • Data: : Dumper Pros: • Familiar Perl Syntax • Fast C implementation • Mature

YAML: A New Language for Data Using YAML in Perl Code my $calendar = Load <<'. . . '; jan: [january, 31] feb: [february, 28] mar: [march, 31] apr: [april, 30] may: [may, 31] jun: [june, 30] jul: [july, 31] aug: [august, 31] sep: [september, 30] oct: [october, 31] nov: [november, 30] dec: [december, 31]. . . $july_days = $calendar->{jul}[1];

YAML: A New Language for Data Some Real YAML Applications • yaml 2 outline • phone 2 yaml • xyx • YAML. pm uses YAML! • Changes • Test suite • perl test-prd. pl

YAML: A New Language for Data A Quick Tour of the YAML Spec http: //www. yaml. org/spec • 1. Intro 2. Preview • Nice Examples 3. Information Models 4. Serialization Syntax • 160 productions 5. Transfer Methods

YAML: A New Language for Data Intermediate YAML Syntax • Whitespace. . . • . . . is the devil • • Headers and Directives Inline Collections Complex Keys Transfer Families Implicit Typing YAML Builtin Types Perl Builtin Types

YAML: A New Language for Data Whitespace • There is a special room in the hot place reserved for the person who invented tabs. • • • YAML has it harder than Python By default YAML disallows TABS for indenting But we have #TAB just in case Modern editors will have YAML support Keep computing tidy. Burn all tabs!

YAML: A New Language for Data Indentation Rules • Use same number of spaces for a given scope --two: - few four: - more blocks: -| # Beware the TAB sub spoon { ---- fork } - |2 QTY DESC PRICE 1) 3 Blind Mice Free 2) 5 Dollar Bill $10. 00

YAML: A New Language for Data Headers and Directives • • Headers start all documents Directives and top level qualifiers go there Two directives: #YAML & #TAB First document can be headerless: first document --- #YAML: 1. 0 #TAB: NONE --- > folded top level scalar --- &1 recurse: *1 --- simple header

YAML: A New Language for Data Inline Collections • So far YAML structure resembles Python • Time for a little TMTOWTDI!!! --seq: [ 14, 34, 55 ] map: {purple: rain, blue: skies} mixed: {sizes: [9, 11], shapes: [round]} span: {players: [who, what, I don't know], positions: [first, second, third]}

YAML: A New Language for Data The Matrix Revisited # Inline sequences make data more compact --- [3, 5, 7] - [0, 0, 7] - [9, 1, 1] # Above is identical to below --- [[3, 5, 7], [0, 0, 7], [9, 1, 1]] # A 3 D Matrix --- [[3, 5, 7], [0, 0, 7], [9, 1, 1]] - [[0, 0, 7], [9, 1, 1], [3, 5, 7]] - [[9, 1, 1], [3, 5, 7], [0, 0, 7]]

YAML: A New Language for Data Complex Keys • In Perl, all hash keys are strings • YAML doesn't have that restriction • YAML is Perl 6 enabled! --? - Kane - Kudra : engaged [Damian, Dominus]: engaging

YAML: A New Language for Data Transfer Families • In YAML, every node has a type • These types can be user defined • Every type has a unique URI same: - 42 - !int 42 - !yaml. org/int 42 - !http: //yaml. org/int 42 perl: - !perl/Foo: : Bar {} - !perl. yaml. org/Foo: : Bar {} - !http: //perl. yaml. org/Foo: : Bar {}

YAML: A New Language for Data More Transfer Families • You can define your own types • '^' is used for abbreviating • URI's may someday be real schema locations domain specific document: - !http: //foobar. org/foo [] - !foobar. org/^foo - !^bar xxx - !^baz yyy

YAML: A New Language for Data Implicit Typing • Simple (unquoted) Strings have implicit types • These types have predefined regexps --- 42 # integer - -3. 14 # floating point - 6. 02 e+23 # scientific notation - 0 x. CAFEBABE # hexadecimal int - 2001 -09 -11 # ISO 8601 time - '2001 -09 -11' # string -+ # boolean true - (false) # alternate boolean -~ # null (undef in Perl) - 123 Main St # string

YAML: A New Language for Data Roundtripping • • YAML processes should preserve types Especially within a language/application domain Should warn otherwise Python preserves Perl typeglobs (!perl/glob: ) Perl preserves !python/tuple Gets complicated with complex keys, etc YAML: : Node is the Perl roundtripping object A tied hash/array/scalar w/ YAML magic

YAML: A New Language for Data YAML Builtin Types • YAML has a good number of builtin types • Some are implicit, some aren't --- !str YAML, YAML! - !int 42 - !float 0. 707 - !time 2001 -12 -14 T 21: 59: 43. 10 -05: 00 - !bool 1 - !null 0 - !binary MWYNG 84 Bww. Ee. ECcgggo. BADs=

YAML: A New Language for Data Perl Builtin Types • Perl also has a number of official types • These are under YAML/Ingy's control • !http: //perl. yaml. org/type --- !perl/Foo: : Bar {} # hash-based class - !perl/@Foo: : Bar [] # array-based class - !perl/$Foo: : Bar '' # scalar-based class - !perl/glob: # typeglob - !perl/code: # code reference - !perl/ref: # hard reference - !perl/regexp: # regular expression - !perl/regexp: Foo: : Bar # blessed regexp

YAML: A New Language for Data Introducing the YAML Shell • • ysh - the YAML Shell Perl => YAML => Perl Term: : Readline support Logging Great way to learn Perl Distributed with YAML. pm

YAML: A New Language for Data Fun With GLOBs in ysh • • *foo *INC *ARGV *$ *_ *STDIN *YAML: : [*x, *y]

YAML: A New Language for Data Perl's Symbol Table (The Full Monty!) • ysh -uc -i 4 -L • *main: : • 10000+ lines • recursive (main: : ) • ISA, IO: , CODE: &6, ENV: , Config: (tied!), INC: , SIG: , Bits: , Offsets: , *|

YAML: A New Language for Data More ysh Fun • • • open $fh, $0; open $fh[0], $0; seek $fh, 100, 0; close $fh; scalar `perl -h` fork < Changes Roundtripping $x = $x;

YAML: A New Language for Data The World Wide YAML Shell • http: //wwysh. yaml. org • Regression Test Farm • Multiple Implementations

YAML: A New Language for Data Self Serializing Objects • YAML: : Node • Family URI • Ordered keys • Foo: : yaml_dump() • Foo: : yaml_load()

YAML: A New Language for Data Advanced YAML Concepts • • • YAML: : Node Information Models YAML Schemas YPATH & YATL YAML: : Node YAML: : Bless()

YAML: A New Language for Data YAML: : Node • Need more control? • YAML: : Node is the ticket • Tied wrapper abound a plain hash, array or scalar • Contains extra info: • YAML Family • Key order • YAML: : Bless($node) • $y->bless($foo); • Associates a ynode with a regular node • YAML: : Blessed($node) • May be a way to support complex keys in Perl

YAML: A New Language for Data Information Models • Syntax Model • parser/emitter • Serial Model • loader/dumper • Generic Model • application loader/dumpers • Native Model • user application (your script)

YAML: A New Language for Data YAML and Schemas • Key Order • Optional Fields • Validation (enums) • Structure

YAML: A New Language for Data YAML Schema --- #YAML: 1. 0 NAME: Address. Entry HASH: - NAME: Name HASH: - NAME: First - NAME: Last OPTIONAL: yes - NAME: Email. Addresses ARRAY: yes - NAME: Phone ARRAY: yes HASH: - NAME: Type OPTIONAL: yes - NAME: Number

YAML: A New Language for Data YAML Instance w/ Schema --- #YAML: 1. 0 Address. Entry: Name: First: Brian Email. Addresses: - ingy@CPAN. org - ingy@ttul. org Phone: - Type: Work Number: 604 -333 -4567 - Number: 843 -444 -5678

YAML: A New Language for Data libyaml - The core YAML engine • • • One engine for Perl, Python, Tcl, Ruby, PHP Maybe Java? ? YAML. pm will bind using Inline! Thank you Neil! Thank you Clark!

YAML: A New Language for Data YAML People • • • Clark Evans - Father of YAML - libyaml, Python Oren Ben-Kiki - The Master Mind - Java Brian Ingerson - Perl Watchdog - YAML. pm Neil Watkiss - Boy Wonder - libyaml Steve and Ryan - The XP factor - Python, Ruby Rolf Veen - The German Connection - Java Jason Diamond - The Lost Soul - C# Neil Kandalgoankar - brev - YAML. py. old Shane Caraveo - Mr 10 Rings - YAML. php. old Andrew Kurn - The Philosopher Preben Peppe Guldber - vim Master - yaml. syn Why - The Lucky Stiff - YAML. rb

YAML: A New Language for Data Resources: www. yaml. org • • • http: //www. yaml. org/spec yaml-core@lists. sourceforge. net http: //wiki. yaml. org perldoc YAML http: //wwysh. yaml. org

YAML: A New Language for Data THE END . . .