Web Applications and Multimedia Martin Kruli by Martin
Web Applications and Multimedia Martin Kruliš by Martin Kruliš (v 1. 2) 10. 5. 2017 1
Images � Images and CSS 3 ◦ Wise application of CCS 3 can produce unexpected effects, including animations ◦ HTMLImage. Element DOM API var img = new Image(); img. src = "image. png"; elem. append. Child(img); ◦ Important properties and events �complete – is image loaded and ready �natural. Width, natural. Height – real proportions �onload – event fired when loading completes Example 1 by Martin Kruliš (v 1. 2) 10. 5. 2017 2
Images � Creating Images from Blob Data var xhr = new XMLHttp. Request(); xhr. open("GET", "picture. jpg", true); xhr. response. Type = "arraybuffer"; xhr. onload = function(e) { var ab. View = new Uint 8 Array(this. response); var blob = new Blob( [ ab. View ], { type: "image/jpeg" } ); var url. Creator = window. URL || window. webkit. URL; var image. Url = url. Creator. create. Object. URL(blob); var img = new Image(); img. src = image. Url; Create temporary URL, which parent. Elem. append. Child(img); is valid only within the page }; xhr. send(); by Martin Kruliš (v 1. 2) 10. 5. 2017 3
Animation Frames � Creating Animations ◦ Script can request a callback for next animation frame by window. request. Animation. Frame(anim) �Invocation returns request ID (similar to timers) �window. cancel. Animation. Frame(id) aborts request ◦ The callback itself can update the page just before the window is re-painted �It gets time argument, which gets current time in milliseconds from window. performance. now() �The callback is invoked only once, but it can register new animation frame request Example 2 by Martin Kruliš (v 1. 2) 10. 5. 2017 4
Video � HTML <video> Element ◦ Creates embedded screen for video playback <video controls> <source src="myvid. webm" type="video/webm"> <source src="myvid. ogg" type="video/ogg"> <source src="myvid. mp 4" type="video/mp 4"> The same video in Browser does not support HTML 5 video. different formats </video> �The controls attribute makes controls visible �If the controls are hidden, the element may be controlled by Javascript by Martin Kruliš (v 1. 2) 10. 5. 2017 5
Video � HTML <video> Element DOM API ◦ Important Methods and Properties �load(), ready. State �play(), paused, ended �duration, current. Time, seekable �volume, muted ◦ Events �loadstart, loadeddata, canplaythrough �pause, playing, progress, stalled �seeking, seeked �volumechange by Martin Kruliš (v 1. 2) 10. 5. 2017 6
SVG � Scalable Vector Graphics ◦ Quite complex vector graphics and animations ◦ HTML 5 allows direct embedding of SVG code <svg> <circle id="circle 1". . . > </svg> ◦ The SVG data are visible through DOM API document. get. Element. By. Id("circle 1")… Example 3 by Martin Kruliš (v 1. 2) 10. 5. 2017 7
Canvas � The canvas Element ◦ HTML 5 element for JS drawing <canvas id="mycanvas" width="640" height="480"> Your browser does not support HTML 5 canvas. </canvas> �Canvas can be sized by CSS, but width and height define the resolution of the viewport �Canvas is cleared when resized ◦ Drawing is performed through context var context = mycanvas. get. Context("2 d"); �Various types of context (for drawing, 3 D graphics, …) �Animations use timers or animation frame events by Martin Kruliš (v 1. 2) 10. 5. 2017 8
Canvas � 2 D Context Features ◦ Drawing rectangles �stroke. Rect(), fill. Rect(), clear. Rect() ◦ Drawing paths �begin. Path(), close. Path() �move. To(), line. To() �arc(), quadratic. Curve. To(), bezier. Curve. To() �stroke(), fill() ◦ Setting drawing styles �stroke. Style, fill. Style, global. Alpha �line. Width, line. Cap, line. Join by Martin Kruliš (v 1. 2) 10. 5. 2017 9
Canvas � 2 D Context Features ◦ Color gradients �create. Linear. Gradient(), create. Radial. Gradient() �gradient. add. Color. Stop() �Can be used in stroke. Style, fill. Style ◦ Patterns var img = new Image(); img. src = "fill-pattern. png"; ctx. stroke. Style = ctx. create. Pattern(img, "repeat"); ◦ Text rendering �fill. Text(), stroke. Text(), font by Martin Kruliš (v 1. 2) 10. 5. 2017 10
Canvas � 2 D Context Features ◦ Transformations �Global transformation matrix for all drawings �translate(), rotate(), scale() �transform(), set. Transform() ◦ Compositing �How is new drawing combined with existing drawings �global. Composite. Operation ◦ Clipping paths �Path terminated with clip(), instead of close. Path() �All subsequent drawing is clipped accordingly by Martin Kruliš (v 1. 2) 10. 5. 2017 11
Canvas � 2 D Context Features ◦ Context state stack �save(), restore() ◦ Retrieving/drawing image data �Image. Data object – raw RGB data in binary form �create. Image. Data() �get. Image. Data(), put. Image. Data() ◦ Saving canvas as image �canvas. to. Data. URL() ◦ Drawing images/video on canvas Example 4 �draw. Image(img. Elem, …) by Martin Kruliš (v 1. 2) 10. 5. 2017 12
2 D Graphics � General ◦ ◦ ◦ ◦ Libraries Pixi. JS Paper. js Fabric. js two. js Easel. js Graphics 2 D. js … � Specialized ◦ Graphs (Chart. js, plotly. js, D 3. js, Google charts, …) ◦ Diagrams (Joint. JS, GOJs, mx. Graph, Flowchart, …) by Martin Kruliš (v 1. 2) 10. 5. 2017 13
Pixi. JS � Pixi. JS ◦ ◦ Modern, promising 2 D graphics library Fast, hardware accelerated Open source Features �Basic shapes, sprites, text �Textures, blending �Transformations, grouping �Video �Animations, skeletons, meshes Example by Martin Kruliš (v 1. 2) 10. 5. 2017 14
DELETE 2 D Graphic Libraries � Kinetic. JS ◦ The programmer does not draw, but create objects that are automatically rendered �Similar to DOM elements �Supports geometric shapes, images, … �Modifying properties causes automatic repainting ◦ Complex object management �Layers, object groups, … ◦ Events �Extending event model to graphic objects �Drag and drop support by Martin Kruliš (v 1. 2) 10. 5. 2017 15
Web. GL � Web API for Open. GL ◦ A way to render 3 D graphic in web browser �Based on Open. GL ES 3. 0 specification ◦ canvas element is used as viewport �Using ‘webgl’ context, which provides bindings to Open. GL functions var canvas = document. get. Element. By. Id('glcanvas'); var gl = canvas. get. Context('webgl'); gl. viewport(0, 0, canvas. width, canvas. height); gl. clear. Color(0. 0, 1. 0); . . . by Martin Kruliš (v 1. 2) 10. 5. 2017 16
Web. GL � Web. GL Context ◦ Data buffers (vertices, colors, …) var buf = gl. create. Buffer(); gl. bind. Buffer(gl. ARRAY_BUFFER, buf); var v = new Float 32 Array([ 1. 0, -1. 0, . . . ]); gl. buffer. Data(gl. ARRAY_BUFFER, v, gl. STATIC_DRAW); ◦ Vertex and fragment shaders �May be stored along with HTML page in <script> elem. var shader = gl. create. Shader(gl. FRAGMENT_SHADER); gl. shader. Source(shader, shader. Source. Code); gl. compile. Shader(shader); . . . Example 5 by Martin Kruliš (v 1. 2) 10. 5. 2017 17
3 D Libraries � Three. js ◦ Lightweight 3 D graphic library �Primarily uses webgl, but can fallback to SW rendering ◦ Object based approach � 3 D entities are JS objects �Camera, lights, scene, … �Hides complexities of 3 D math ◦ Additional features �Object materials – different surface effects �Import/export 3 D data �… Example 6 by Martin Kruliš (v 1. 2) 10. 5. 2017 18
Audio � HTML <audio> Element ◦ Similar to <video> element ◦ Controlled from user panel or from Javascript � Web Audio API ◦ Playback from media files, oscillators, and binary data buffers ◦ Adding audio filters and effects ◦ Related to Web. RTC media streams �Recording from microphone �Sending audio to RTC channel by Martin Kruliš (v 1. 2) 10. 5. 2017 19
Web Audio API � Audio Context ◦ A context object must be created first var audio. Ctx = new (window. Audio. Context || window. webkit. Audio. Context)(); if (!audio. Ctx). . . // no audio ◦ Within the context a pipeline of audio nodes has to be assembled Inputs Effects Destination by Martin Kruliš (v 1. 2) 10. 5. 2017 20
Web Audio API � Audio Nodes ◦ Audio Sources �Oscillator. Node �Audio. Buffer, Audio. Buffer. Source. Node �Media. Element. Audio. Source. Node – e. g. , <audio> �Media. Stream. Audio. Source. Node – Web. RTC media stream ◦ Audio Destinations �Audio. Destination. Node �audio. Ctx. destination – default destination (speakers) �Media. Stream. Audio. Destination. Node – Web. RTC channel by Martin Kruliš (v 1. 2) 10. 5. 2017 21
Web Audio API � Audio Nodes ◦ Effects and Filters �Delay. Node �Gain. Node �… ◦ Other Nodes �Analysis. Node – analysis and visualization �Channel. Splitter. Node, Channel. Merger. Node �Audio. Listener – position the sound in space �Panner. Node – signal behavior in space Example 7 by Martin Kruliš (v 1. 2) 10. 5. 2017 22
Web Audio API � Audio Buffer Playback Example var audio. Ctx = new (window. Audio. Context || window. webkit. Audio. Context)(); Puts data from Array. Buffer var channels = 2, sample. Rate = 44100; to playback queue var next. Time = 0; function enqueue. Audio. Data(data. Array. Buffer) { var data = new Int 16 Array(data. Array. Buffer); var buf = audio. Ctx. create. Buffer(channels, data. length / channels, sample. Rate); Small constant to //. . . Copy samples from data to buf. . . prevent data jittering if (next. Time == 0) next. Time = audio. Ctx. current. Time + 0. 05; var source = audio. Ctx. create. Buffer. Source(); Buffer. Source wraps source. buffer = buf; the buffer for playback source. connect(audio. Ctx. destination); source. start(next. Time); next. Time is used to mark, next. Time += buf. duration; when the playback starts } by Martin Kruliš (v 1. 2) 10. 5. 2017 23
Discussion by Martin Kruliš (v 1. 2) 10. 5. 2017 24
- Slides: 24