APIs for Data Handling and Communications Martin Kruli

APIs for Data Handling and Communications Martin Kruliš by Martin Kruliš (v 1. 0) 16. 5. 2019 1

Binary Data � Array. Buffer ◦ Represents generic binary data of fixed length var buf = new Array. Buffer(64); ◦ slice() – create a copy of (sub)buffer � Array. Buffer. View ◦ Creates a specific view for an array buffer �Multiple views can be created over one array ◦ Many subclasses: Int 8 Array, Uint 32 Array, … ◦ Getter/setter handles the array elements in the correct format (convert them to/from JS number) by Martin Kruliš (v 1. 0) 16. 5. 2019 2

Binary Data � Data. View ◦ Another way how to view data in array buffer ◦ Low level operations: get. Int 8(), set. Float 32(), … �Supports both little and big endian � Blob ◦ Represents file-like object of immutable raw data ◦ Constructor gets list (array) of array buffers, array buffer views, blobs, or strings ◦ Also holds data properties �MIME type by Martin Kruliš (v 1. 0) 16. 5. 2019 3

File API � File Interface ◦ Extension of the Blob interface �Attributes: name, size, type, last. Modified ◦ Can be constructed explicitly from binary data ◦ Real files can be accessed through input element <input id="file. Select" type="file". . . > �The input implements File. List interface (array of files) document. get. Element. By. Id("file. Select"). files[0]. . . �JS can access only those files the user have selected �Input element multiple attribute by Martin Kruliš (v 1. 0) 16. 5. 2019 4

File API � Reading Files ◦ File. Reader interface �read. As. . . (blob) – starts async. read �Read as Array. Buffer, Text, or Data. URL �ready. State (EMPTY, LOADING, DONE) �onload, onprogress, onerror, onloadend, … �result (either Array. Buffer or string), error �abort() by Martin Kruliš (v 1. 0) 16. 5. 2019 5

File API � Drag and Drop ◦ Files can be dragged into the browser mydiv. ondrop = function(e) { e. stop. Propagation(); e. prevent. Default(); var files = e. data. Transfer. files; . . . } � Saving Files ◦ Tricky, some nonstandard APIs exist (e. g. , save. As()) ◦ Data URL can be used in window. open() or <a> �Not a very neat way, but it works by Martin Kruliš (v 1. 0) 16. 5. 2019 6

File System API � File API: Directories and File System ◦ API for storing files at client side �Two types: temporal and persistent �Intended for situations when other types of data storage (e. g. , web storage) are inappropriate �Multimedia files, persistent uploads, … ◦ Support for directory structure �Both traversing and manipulation ◦ File reader and analogous file writer APIs ◦ Implemented in webkit only, specification rejected �Firefox OS implements its own Device Storage API by Martin Kruliš (v 1. 0) 16. 5. 2019 7

Web Storage (Revision) � Web Storage ◦ Simple storage for keeping persistent data ◦ Similar to cookies (storing key-value strings), but intended for client-side scripting only ◦ Two storage objects with the same API �Local storage – per web site, persistent �Session storage – per window, deleted on close ◦ Strict quotas �~ 5 MB per site in most browsers �Can be configured by Martin Kruliš (v 1. 0) 16. 5. 2019 8

Web SQL � Web SQL Database API ◦ Specialized SQL database for client-side scripts �Asynchronous API with JS data bindings �Transactional support �SQL dialect, which is supported by SQLite 3. 6. 19 ◦ Web browsers should apply some form of quotas �Prompt the user, when the quotas are to be exceeded ◦ Deprecated �Not implemented by Firefox and MSIE �Only SQLite implementation existed by Martin Kruliš (v 1. 0) 16. 5. 2019 9

Indexed DB � Web Database Indexed Storage ◦ Alternative to Web SQL (with different functionality) �Object-oriented database (not relational) �Simple queries and cursors are used instead of SQL ◦ Database stores key-value pairs �Values may be complex objects �Keys may be properties of value objects �Indexes may be built on any object properties ◦ Transactional model �All operations (objects, …) are tied to a transaction �Guarantees concurrent-safe access to data by Martin Kruliš (v 1. 0) 16. 5. 2019 10

Indexed DB � Application Interface ◦ Mostly asynchronous (like AJAX requests) �Most operations require callback �Callbacks are invoked when result becomes available �Request objects representing pending operations �Dom events onsuccess, onerror ◦ Database structure �Database is a set of object stores identified by name and version �Databases follow same-origin data isolation policy �Object store – a key-value set sorted by keys �Additional indexes may be applied on object store by Martin Kruliš (v 1. 0) 16. 5. 2019 11

Indexed DB � Opening Database Create/upgrade the structure var db = null; var request = indexed. DB. open("db_name"); request. onupgradeneeded = function() { db = request. result; db. create. Object. Store("store_name", {. . . }); }; request. onsuccess = function() { db = this. result; Open existing DB show. Data(); }; request. onerror = function() {. . . }; by Martin Kruliš (v 1. 0) 16. 5. 2019 12

Indexed DB � Data Manipulation var tx = db. transaction([stores], "readwrite"); var store = tx. object. Store(store. Name); tx. oncomplete = function(e) {. . . } store. put(data); When everything is done var range = IDBKey. Range. lower. Bound(0); store. open. Cursor(range). onsuccess = function(e) { var cursor = e. target. result; if (cursor) {. . . cursor. continue(); } A way to iterate over }; selection of data by Martin Kruliš (v 1. 0) 16. 5. 2019 13

Web Workers � Web Workers ◦ Worker represents a separate computing thread �The code of the worker is sandboxed �API for bidirectional communication with main thread �Workers may spawn sub-workers �Some APIs have synchronous counterparts for workers ◦ API �Worker(url) – constructing function �Worker. post. Message(msg) – send message to the peer �Worker. onmessage – receiving message event handler �Worker. terminate() – kill the worker thread Example 4 by Martin Kruliš (v 1. 0) 16. 5. 2019 14

Shared Workers � Shared Workers ◦ Workers that may be shared by multiple windows ◦ Ideal for stateless computations in background ◦ Connections to windows are represented by ports var sh. Worker = new Shared. Worker('worker. js'); sh. Worker. port. post. Message() sh. Worker. port. onmessage Main thread. . . onconnect = function(ev) { Inside worker var port = ev. ports[0]; port. onmessage = function() {. . . } } by Martin Kruliš (v 1. 0) 16. 5. 2019 15

Web Cryptography � Standard ◦ ◦ Cryptography Functions Encrypting/Decrypting data Signing data (asymmetric decryption) Hashing functions (integrity, security tokens, …) Arsenal of mainstream algorithms �RSA, AES, SHA, … ◦ API �Crypto – general interface (e. g. , random generator) �Crypto. subtle – Subtle. Crypto interface �encrypt(), decrypt(), verify(), digest(), … �Algorithm, Key. Algorithm by Martin Kruliš (v 1. 0) 16. 5. 2019 16

HTTP (Revision) � Hyper-Text Transfer Protocol ◦ Simple textual-based protocol ◦ Designed for data retrieval �Also allows data uploads and modification requests ◦ Completely stateless �Each request treated independently �Cookies extension allows limited session management ◦ Client initiated �Server cannot push data to the clients by Martin Kruliš (v 1. 0) 16. 5. 2019 17

AJAX (Revision) � Asynchronous Java. Script and XML ◦ A technique that combines three technologies �Java. Script �Asynchronous HTTP client API integrated in browser �XML or other semi-structured data format ◦ Script invokes HTTP transfer and provide callbacks �Both GET and POST requests are supported ◦ The callback is invoked asynchronously �At the conclusion of the HTTP transfer �It may process the returned data (e. g. , update the contents of the web page) by Martin Kruliš (v 1. 0) 16. 5. 2019 18

AJAX Old API (Revision) � XMLHttp. Request Object var http. Req = new XMLHttp. Request(); http. Req. open("GET", "index. php? ajax=1", true); http. Req. onreadystatechange = function() { if (http. Req. ready. State != 4) return; if (http. Req. status == 200) process. Response(http. Req. response. Text); else handle. Error(http. Req. status); } http. Req. send(); by Martin Kruliš (v 1. 0) 16. 5. 2019 19

XMLHttp. Request � More Advanced Features ◦ Managing headers �set. Request. Header(), get. Response. Header() ◦ Advanced response representation �Support for array buffers, blobs, and JSON data ◦ Easier way to send request content �Support for blobs �Special Form. Data API �Assembling a form-like request (including file uploads) ◦ Events �onload, onerror, onabort, ontimeout, onloadend, … by Martin Kruliš (v 1. 0) 16. 5. 2019 20

Fetch API � Fetch API ◦ Intended to replace old XMLHttp. Request fetch(input, init) �input – URL or Request object �init – additional configuration �Much more detailed, custom headers, various ways how to pass body, … �Returns promise object �On resolve provides HTTP Response object ◦ Abort. Controller �Allows aborting pending fetch() requests �Still experimental by Martin Kruliš (v 1. 0) 16. 5. 2019 21

Cross-Origin Requests � Same-Origin Security Policy ◦ Safety precaution implemented by browsers �AJAX requests must target the same domain ◦ Sometimes accessing other domains is required � Cross-Origin Requests ◦ Browser adds Origin header to the request ◦ If the request is allowed, the server adds header Access-Control-Allow-Origin in response ◦ Sometimes a preflight request is required by Martin Kruliš (v 1. 0) 16. 5. 2019 22

Cross-Origin Requests � Preflight Requests ◦ OPTION request goes first to get CORS parameters ◦ Full request is sent afterwards �If the CORS parameters permits by Martin Kruliš (v 1. 0) 16. 5. 2019 23

Bidirectional Communication � Comet Client starts asynchronous HTTP Request After timeout, an empty response is sent timeout Server postpones the response if there is nothing to report Client processes the event and issues another request … Client (Browser) Client immediately issues a new request Event notification is sent event Web Server Reportable event occurs by Martin Kruliš (v 1. 0) 16. 5. 2019 24

Server Messages � Server Messages ◦ Extension of the idea implemented by Comet ◦ Special protocol �HTTP response with text/event-stream content type �Continuous stream containing separate events data: transmitted data event: custom-event data: data for the custom event ◦ Simple client API var ev. Source = new Event. Source("msg. php"); evt. Source. onmessage = function(e) { e. data … } by Martin Kruliš (v 1. 0) 16. 5. 2019 25

Web. Socket Protocol � Extension of HTTP(S) Protocols ◦ Two way communication ◦ Persistent connections ◦ Layered over TCP or SSL/TLS connection � Protocol Properties ◦ Defined in detail in RFC 6455 ◦ Handshake is compatible with HTTP handshake ◦ Simple message-based communication �User can specify custom sub-protocols (i. e. , the contents and semantics of the messages) by Martin Kruliš (v 1. 0) 16. 5. 2019 26

Web. Socket Protocol � Handshake ◦ The connection is initiated by a HTTP request, which is a HTTP-upgrade request GET /chat HTTP/1. 1 The “upgrade” request Host: server. example. com Upgrade: websocket Security things Connection: Upgrade Sec-Web. Socket-Key: d. Ghl. IHNhb. XBs. ZSBub 25 j. ZQ== Origin: http: //example. com Sec-Web. Socket-Protocol: chat, superchat Sec-Web. Socket-Version: 13 Web. Socket specific information by Martin Kruliš (v 1. 0) 16. 5. 2019 27

Web. Socket Protocol � Handshake ◦ The server may respond by any HTTP-valid code �E. g. 404 if the URI is not valid or by 3 xx redirection �It may also include things like auth-tokens, cookies… ◦ Upgrade acceptation looks as follows HTTP/1. 1 101 Switching Protocols Upgrade: websocket Upgrade confirmation Connection: Upgrade Sec-Web. Socket-Accept: s 3 p. PLMBi. Txa. Q 9 k. YGz. . . Sec-Web. Socket-Protocol: chat Security verification Accepted subprotocol by Martin Kruliš (v 1. 0) 16. 5. 2019 28

Web. Socket Protocol 0 Format 4 Ctrl. bits Op. Code 8 16 Mask � Frame Length 32 Extended Length (if length > 125) Extended Length (if length == 127) Masking Key (if mask == 1) Masking Key (continuation) Payload Data by Martin Kruliš (v 1. 0) 16. 5. 2019 29

Web. Socket Protocol 0 Format 4 Ctrl. bits Op. Code 8 16 Mask � Frame Length 32 Extended Length (if length > 125) Extended Length. Opcode (if length(type == 127) of payload) x 0 – Continuation frame Control Bits Extended Length (if length == 127) Masking Key (if mask == 1) x 1 - Text Frame (UTF 8) 0. – Final fragment of x 2 – Binary Frame a message Masking Key (continuation) Payload Data x 3 -x 7 – Reserved (non control) 1. -3. Reserved (set to 0) x 8 – Connection Close x 9 Data – Ping Payload x. A – Pong x. B-x. F – Reserved (control) by Martin Kruliš (v 1. 0) 16. 5. 2019 30

Web. Socket Protocol 0 Format 4 Ctrl. bits Op. Code Whether payload is masked 8 16 Mask � Frame Length 32 Extended Length (if length > 125) Extended Length (if length == 127) Payload length (in bytes). If length == 126/127, extended bit length Masking 16/64 Key (continuation) is used. Extended Length (if length == 127) Masking Key (if mask == 1) Payload Data Randomly selected 32 bit key used for XOR masking of the payload. Payload Data by Martin Kruliš (v 1. 0) 16. 5. 2019 31

Web. Socket Protocol � Data Frames ◦ Messages may be fragmented �First frame defines the type, other frames are defined as continuation frames �Last frame has the appropriate control bit set ◦ Text messages must be in UTF 8, binary messages are interpreted only by the application � Control Frames ◦ Close frame initiates graceful connection shutdown ◦ Ping/pong frames verify the connection is still alive by Martin Kruliš (v 1. 0) 16. 5. 2019 32

Web. Socket Protocol � Security ◦ The connection may be established on a SSL/TLS channel instead of TCP �In the same way HTTP and HTTPS works ◦ The client sends an origin that refers to a domain from which the client script was downloaded ◦ The server must correctly transform given security key to verify it is really a Web. Socket server ◦ The client should use frame masking to defend against a specific cache-poisoning attack �When the attacker constructs frames that look like a HTTP request by Martin Kruliš (v 1. 0) 16. 5. 2019 33

Web. Socket API � Client Side (Java. Script) ◦ Encapsulated in window. Web. Socket object ◦ We can check the functionality by if (!"Web. Socket" in window) { // Do plan B; we do not have Web. Sockets } ◦ A new connection is initiated by constructing new Web. Socket object var ws = new Web. Socket('ws: //domain/', 'proto'); by Martin Kruliš (v 1. 0) 16. 5. 2019 34

Web. Socket API � Web. Socket ◦ Properties Object �ready. State – state of the connection (0 – connecting, 1 – connnected, 2 – closing, 3 – closed) �protocol – selected subprotocol �buffered. Ammount – number of bytes buffered by send but not yet transmitted over the network ◦ Methods �send(data) – send a data message (the type of the message is selected by the type of the data object) �close([code, [reason]]) – gracefully terminate the connection by Martin Kruliš (v 1. 0) 16. 5. 2019 35

Web. Socket API � Web. Socket ◦ Events Object �onopen() – when the connection is established �onmessage(e) – reports an incoming message (defragmented) in e. data field (string for text and Blob or Array. Buffer for binary messages) �onerror() – if connection was terminated by an error �onclose(e) – when the connection is terminated �e. was. Clean – whether the shutdown was graceful �e. code – termination code �e. reason – string with termination reason by Martin Kruliš (v 1. 0) 16. 5. 2019 36

Web. Socket API � Server Side ◦ Current HTTP servers (e. g. , Apache) do not natively support Web. Sockets �Similar problem as with Comet and Server Events �Applications often use two servers (HTTP and WS) ◦ Alternatives �Standalone application/script that implements HTTP and WS on its own �Specialized solutions like Node. js �Contains ready to use libraries for HTTP and WS which embrace event based programming Example 3 by Martin Kruliš (v 1. 0) 16. 5. 2019 37

Web RTC � Web Real-Time Communication ◦ API for direct p 2 p communication between browsers ◦ Originally designed for audiovisual data (videophone) Signaling channel (AJAX, WS, …) is required for establishing the connection RTC data are then passed directly or via TURN servers by Martin Kruliš (v 1. 0) 16. 5. 2019 38

Web RTC � Web RTC API ◦ Media. Stream �API that exposes camera or microphone as streams of multimedia data ◦ RTCPeer. Connection �Manages the established connection �Multiple channels may be attached ◦ RTCData. Channel �Special channel type for script-managed data transfers �Channel API is based on Web. Socket API by Martin Kruliš (v 1. 0) 16. 5. 2019 39

Discussion by Martin Kruliš (v 1. 0) 16. 5. 2019 40