Wild Heart HTTPListener an open source Web Server

Wild. Heart. HTTPListener an open source Web Server for Dyalog APL v 11. 0 Stefano “Wild. Heart” Lanzavecchia

Agenda l APL should be a popular language for developing web pages, because: – APL has all you need to generating static and dynamic web content l It is not used as much as it deserves, because: – Even the best tools need good frameworks to level the learning curve a bit – No frameworks are readily available 2006 -10 -17 Dyalog User Group Conference 2

Goal l Explore the problem which needs to be solved and demonstrate a proposed solution. . . (obviously ) l So, what’s the problem? l The easy part is generating HTML l Hopefully nobody will disagree that APL is an EXCELLENT language for generating character vectors 2006 -10 -17 Dyalog User Group Conference 3

Some very simple HTML A ”web” 2006 -10 -17 Dyalog User Group Conference 4

The hard part If we have a form like this and the user hits the submit button… 2006 -10 -17 Dyalog User Group Conference 5

Before we look at the response… <tr> Some of you will want to know how the HTML looks for the form? <form action="RUN" method="post" name="Front. Page_Form 1"> <div> <table border="0" cellpadding="3"> <tr> <td>Purchase Amount</td> <td><input type="text" size="9" name="Loan. Amt"/></td> </tr> <td>Percent Down</td> <td><input type="text" size="6" maxlength="6" name="Percent. Down"/></td> </tr> <td>Maximum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="Len. Max"/></td> </tr> <td>Minimum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="Len. Min"/></td> </tr> … continued on the right … 2006 -10 -17 <td>Maximum Interest Rate (%)</td> <td><select name="Intr. Max" size="1"> <option>15. 0</option> <option>14. 5</option>. . etc down to… <option>1. 0</option> </select></td> </tr> <td>Minimum Interest Rate (%)</td> <td><select name="Intr. Min" size="1"> <option>15. 0</option> <option>14. 5</option> <option>14. 0</option> <option>13. 5</option> <option selected="selected">13. 0</option> <option>12. 5</option> <option>12. 0</option> …etc… <option>1. 5</option> <option>1. 0</option> </select></td> </tr> </table> <p><input type="submit" value="Calculate Repayments"/></p> </div> </form> Dyalog User Group Conference 6

When the user hits the button… POST /loan/RUN HTTP/1. 1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd. ms-excel, application/vnd. ms-powerpoint, application/msword, */* Referer: http: //localhost: 8080/loan. htm Accept-Language: it Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate User-Agent: Mozilla/4. 0 (compatible; MSIE 6. 0; Windows NT 5. 1; SV 1; . NET CLR 1. 1. 4322; . NET CLR 2. 0. 50727; Win. FX Run. Time 3. 0. 50727) Host: localhost: 8080 Content-Length: 74 Connection: Keep-Alive Cache-Control: no-cache Loan. Amt=10000&Percent. Down=50&Len. Max=10&Len. Min=15& Intr. Max=15. 0&Intr. Min=13. 0 2006 -10 -17 Dyalog User Group Conference 7

The hard part… l APL is not a bad tool for taking character vectors apart, either l But: Some knowledge about the HTTP protocol is required to decipher it l A rule for calling the code to deal with the request (in this case #. loan. RUN) is needed l Some way to decode the body of the request l This is what Web Server frameworks like Microsoft IIS coupled with ASP. NET do… 2006 -10 -17 Dyalog User Group Conference 8

What is a web application? “A collection of functions that take an HTTP Request as input and produce an HTTP response as output” 2006 -10 -17 Dyalog User Group Conference 9

Solutions l Dyalog APL integrates quite well with Microsoft IIS+ASP. NET, and can be called from other web servers, but: – These solutions require a lot of heavy machinery from other vendors – The learning curve is still quite high l Dyalog provide a sample SERVER workspace, but: – It leaves almost all the work of deciphering HTTP to you – It’s based on old standards 2006 -10 -17 Dyalog User Group Conference 10

Solutions The author has: l A keen (geek) interest in OO, web and other software development methodologies l Written numerous tools on top of the Dyalog TCPSocket object 2006 -10 -17 Dyalog User Group Conference 11

Enter Wild. Heart. HTTPListener l The advent of Object Orientation in Dyalog 11. 0 makes it attractive to start again l My goal is to create a framework which: – Requires a little knowledge of HTML and hardly any – – knowledge of HTTP Allows ”ordinary” APL developers to easily write Web applications, yet: Is built from reusable, replaceable components which can be extended by APL developers who learn a little more. . . Is portable across all platforms which support TCP Is run as an open source project, to be shared and extended by anyone who wishes to contribute 2006 -10 -17 Dyalog User Group Conference 12

To write or Not to write l Why NOT TO WRITE a web server in APL: – This is the third web server I develop: • I was paid to develop the first (and it was never used) • I was almost paid to the develop/steal the second (and it’s heavily used) • I was NOT paid to develop the third and I had to find the time to do it at night and during weekends l A wise man once said: “only a fool would do that” 2006 -10 -17 Dyalog User Group Conference 13

Not to write, seriously l HTTP is easy but tricky l Reliability l Security l Speed l So many web servers out there 2006 -10 -17 Dyalog User Group Conference 14

To write or not to write l Why write a web server in APL? (I wrote an editorial about it in Vector a few years ago) – Easily embeddable in an APL application – Easily understood by APLers – Can stay near the core of the application 2006 -10 -17 Dyalog User Group Conference 15

To write or not to write "One of the reasons to use Lisp in writing Web-based application is that you can use Lisp. When you're writing software that is only going to run on your own servers, you can use whatever language you want. “ Paul Graham 2006 -10 -17 Dyalog User Group Conference 16

To write! “Morten needs a demo. I need an HTTP/1. 1 web server, written in APL Italiana, the company I work for, also needs one, even if they haven’t realised this yet. ” Stefano Lanzavecchia 2006 -10 -17 Dyalog User Group Conference 17

Components of… Wild. Server’ 06 l HTTPListener l HTTPRequest l Static file server – Headers – Cookies – Request Variables l HTTPResponse – Cookies l HTML Utils l Event pipeline l Namespace as filesystem… l File Handlers: – aplx files – apls files – … – User Validation – Sessions 2006 -10 -17 Dyalog User Group Conference 18

Dyalog APL: an OO language l Web server: a good test bed for the OO extensions – Abstractions: divide et impera – Inheritance: this looks a bit like that – Keyed properties: syntax tricks l What is missing in the language? l Is the IDE good enough? 2006 -10 -17 Dyalog User Group Conference 19

HTTPListener fs←� NEW #. Wild. Heart. File. System('c: aplserve') listener←� NEW web. HTTPListener(8080 fs) listener. Start Then point your web browser at http: //localhost: 8080/ “It’s a kind of magic” 2006 -10 -17 Dyalog User Group Conference 20

The trip of an HTTP request l TCPSocket gets TCPRecv events l HTTPRequest objects collects them, and parses the request according to the HTTP specs l When the request is complete, it gets shipped to a dispatcher… 2006 -10 -17 Dyalog User Group Conference 21

The dispatcher: File. System l To me a File. System looks like a namespace: – Its folders are sub-namespaces – Idea: we can inject virtual folders: • A virtual folder can be a folder on the HD coming from a different sub-folder • A virtual folder can be… an APL namespace! Its functions would be files… 2006 -10 -17 Dyalog User Group Conference 22

APLNamespace fs←� NEW #. Wild. Heart. File. System('c: aplserve') fs['loan']←� NEW web. APLNamespace(#. loan'c: aplserveloan') listener←� NEW web. HTTPListener(8080 fs) listener. Start http: //localhost: 8080/loan/RUN executes #. loan. RUN but http: //localhost: 8080/loan/bullet. gif will transmit the file c: aplserveloanbullet. gif 2006 -10 -17 Dyalog User Group Conference 23

Zip. File. System fs←� NEW web. Zip. File. System('c: aplserve. zip? aplserve') fs['loan']←� NEW web. APLNamespace(#. loan'c: aplserveloan') listener←� NEW web. HTTPListener(8080 fs) listener. Start l Now the folders and files come from a zip file. l Neat, uh? l In fact this presentation… 2006 -10 -17 Dyalog User Group Conference 24

File. System l Basically one method: Process. Request ∇ Process. Request ctx; html; ns; raw; aplo : Access Public Instance Override ctx←#. Wild. Heart. HTTPContext. Current html←_NS� Map. File. Name ctx. URI r←ctx. _Response r. _Is. Body. Raw←raw← 82≠� DR html r. _Body←html : If ''≡⊃r. _Headers[⊂'Content-type']←u. Mime. Mappings['htm' 'bin'[1+raw]] : End. If ctx. Send. Response ∇ 2006 -10 -17 Dyalog User Group Conference 25

Tricks: thread local storage […] ctx←#. Wild. Heart. HTTPContext. Current […] l Thread context: wouldn’t it be nice to have an object local to a thread? 2006 -10 -17 Dyalog User Group Conference 26

HTTPHandler l A File. System delegates the handling of the requested resource to the appropriate handler: – Static. File. Handler by default – Aplx. Handler – Apls. Handler – … more to come fs. _File. Handlers[⊂'aplx']←#. Wild. Heart. Aplx. File. Handler 2006 -10 -17 Dyalog User Group Conference 27

HTTPHandler l One method: Process. Request ∇ Process. Request ctx; fname; tn; ext; etag : Access Public Instance Overridable fname←ctx. _File. System. Map. File. Name ctx. URI tn←fname � NTIE 0 ctx. _Response. _Is. Body. Raw← 1 ctx. _Response. _Body←� NREAD tn, 83, � NSIZE tn � NUNTIE tn ctx. Send. Response ∇ 2006 -10 -17 Dyalog User Group Conference 28

Aplx. Handler “This is only a test” <? @ master=". . /newindex. maplx" execute="" ? > <apl: content id="title"> <title>Rain Graphics: Climate</title> </apl: content> <apl: content id="cssstyle"> <link rel="stylesheet" type="text/css" href=". . /styles/spiffy. css"/> </apl: content> <apl: content id="main" execute="#. rain. Climate"> </apl: content> 2006 -10 -17 Dyalog User Group Conference 29

A master page <!DOCTYPE html PUBLIC "-//W 3 C//DTD XHTML 1. 0 Transitional//EN" "http: //www. w 3. org/TR/xhtml 1/DTD/xhtml 1 -transitional. dtd"> <html xmlns=“http: //www. w 3. org/1999/xhtml” xmlns: apl="http: //wildheart. com/2006/10/ " > <head> <apl: placeholder id="cssstyle"> <link rel="stylesheet" type="text/css" href="styles/spiffy. css"/> </apl: placeholder> <apl: placeholder id="title"> <title/> </apl: placeholder> <body> <div id="header"><h 1 class="banner">Wild. Heart. HTTPListener</h 1></div> <div id="subheader"> <apl: placeholder id="bcrumb" execute="#. Wild. Heart. Web. Demo s. Bread. Crumb" /> </div> <div id="toc"> <apl: placeholder id="apltoc"> <div class="toc"><h 2>APL Web Server</h 2> <ul> <li><a href="/intro. aplx">Introduction</a></li> <li><a href="/cv/Index. aplx">View Code</a></li> … </head> 2006 -10 -17 Dyalog User Group Conference 30

The event pipeline l As a request goes through the File. System hierarchy, events are raised: – On. Enter – On. Authenticate – On. Authorize –… 2006 -10 -17 Dyalog User Group Conference 31

Tricks: the observer pattern : Class Event : field _Observers ∇ make name : Access Public Instance : Implements Constructor _Observers←� � DF'Event: ', name ∇ ∇ Raise(sender arg); obj; fnname : Access Public Instance : For obj fnname : In _Observers (obj� fnname)(sender arg) : End. For ∇ 2006 -10 -17 ∇ Add(obj fnname) : Access Public Instance _Observers, ←⊂obj fnname ∇ ∇ Del(obj fnname) : Access Public Instance _Observers← _Observers~⊂obj fnname ∇ : End. Class Dyalog User Group Conference 32

The observed shows what it’s got l on. Authenticate←� NEW #. Wild. Heart. Event('Authenticate') auth←� NEW web. Web. Demos. Forms. Auth fs. on. Authenticate. Add auth'On. Authenticate' l on. Authenticate. Raise � THIS lurl 2006 -10 -17 Dyalog User Group Conference 33

Two simple authentication modules l Simple. Auth: based on HTTP challenge response: – Quick and dirty l Forms. Auth: based on cookies – More flexible – More work 2006 -10 -17 Dyalog User Group Conference 34

Sessions l HTTP is a stateless protocol l People want to have conversations with peers who don’t forget instantly what they just said l Have a cookie! 2006 -10 -17 Dyalog User Group Conference 35

Sessions l As easy as: fs. Enable. Session � NULL fs. on. Session. Start. Add #’Do. Something' fs. on. Session. End. Add #’Undo. Something' l In the application: : If � NULL≡ns←ctx. _Session[⊂'footer'] ns←� NS'' ns. count← 0 ctx. _Session[⊂'footer']←ns : End. If ns. count+← 1 2006 -10 -17 Dyalog User Group Conference 36

Extensibility l Some of the features (Zip. File. System, session module, authentication, aplx handler) were hacked together to prove that the server could support them l Even before its completion, the Wild. Server has: – Had a file handler written (by Nic & Morten) – Been used to run a course in writing Web Applications using Dyalog APL 2006 -10 -17 Dyalog User Group Conference 37

Open Source l Open Source means: “free as in free beer” (except: is there such a thing as free beer? ) l No strings attached: – You get the source code and whatever documentations exists with it at the time – You get the right to do whatever you want with it (even sell it! But only a bad boy would do that…) – You get from me all the support I can give you when I can if I can (alas, I am very busy and maybe one day I’ll get a life) 2006 -10 -17 Dyalog User Group Conference 38

Why Open Source? l Have you looked at Ruby on Rails? l Tons of contributors do a lot of work l Tons of clever contributors do a lot of good work! l OK, maybe that’s a good product. 2006 -10 -17 Dyalog User Group Conference 39

TODO list l It’s long, it’s scary, but… l APL Prototypes seem to acquire a life of their own l The “ducks” have started contributing 2006 -10 -17 Dyalog User Group Conference 40

Thanks l Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays 2006 -10 -17 Dyalog User Group Conference 41

Conclusion l Initial feedback from users (Nic, Morten and Mondays course participants) is promising l The Wild. Server is still very much ”work in progress”, and the author wants to keep control of it until about Christmas l However, you are all welcome to try it now and suggest or code enhancements 2006 -10 -17 Dyalog User Group Conference 42

Questions? l If not… now I need some sleep. 輝く空と君の声を 感じたい 2006 -10 -17 Dyalog User Group Conference 43

What is missing in the language l No public operators! C# has them! l No protected members! C# has them! l No object serialization! C# has it! l C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby” 2006 -10 -17 Dyalog User Group Conference 44