Application Layer Chapter 7 DNS Domain Name System

  • Slides: 88
Download presentation
Application Layer Chapter 7 • • • DNS – Domain Name System Electronic Mail

Application Layer Chapter 7 • • • DNS – Domain Name System Electronic Mail The Web Streaming Audio and Video Content Delivery Revised: August 2011 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

The Application Layer Uses transport services to build distributed applications CN 5 E by

The Application Layer Uses transport services to build distributed applications CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011 Application Transport Network Link Physical

DNS – Domain Name System DNS is a world-wide distributed directory service. DNS delegates

DNS – Domain Name System DNS is a world-wide distributed directory service. DNS delegates the responsibility of assigning domain names and mapping those names to internet resources by designating authoritative name servers for each domain (zone). Net admins may delegate authority over subdomains within their allocated name space to other name servers. This provides a distributed and fault tolerant service that was designed to avoid a single large central database. Internet has two primary name-spaces: 1. domain name hierarchy (DNS addresses) 2. Internet Protocol Address spaces (IPv 4 and IPv 6) The DNS resolves high-level human readable names for computers to low-level IP addresses • DNS name space » • Domain Resource records » • Name servers » CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

The DNS Name Space (1) DNS namespace is hierarchical from the root down •

The DNS Name Space (1) DNS namespace is hierarchical from the root down • • • Different parts delegated to different organizations Can register under multiple top-level domains Absolute domain names always end in a period, relative domain names do not − Relative names have to be interpreted in a context to uniquely determine their meaning 250+ Top Level Domains: 1. Generic 2. Countries Generic Second Level The computer robot. cs. washington. edu Question: What is a “domain”? Domains run by Registrars CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011 appointed by ICANN

The DNS Name Space (2) The Generic top-level domains are controlled by ICANN who

The DNS Name Space (2) The Generic top-level domains are controlled by ICANN who appoints registrars to run them ICANN = Internet Corporation for Assigned Names and Numbers This one was controversial To create a new domain, permission is required of the domain in which it will be included. Naming follows organizational boundaries, not physical networks. CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Domain Resource Records (1) Every domain (all levels) has its own DNS database that

Domain Resource Records (1) Every domain (all levels) has its own DNS database that is comprised of Resource Records. Each Resource Record is a 5 tuple: Domain_Name, TTL, Class, Type, Value (page 616) The key resource records in the namespace are IP addresses (A/AAAA) and name servers (NS), but there are others too (e. g. , MX) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Domain Resource Records (2) Name server IP addresses of computers CNAME and PTR are

Domain Resource Records (2) Name server IP addresses of computers CNAME and PTR are aliases; CNAME allows the same IP address to have multiple names; PTR enables reverse lookups Mail gateways A portion of a possible DNS database for cs. vu. nl. Question: What is cs. vu. nl’s web server’s name and IP address? Question: What is a reverse lookup? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Name Servers (1) DNS name space divided into overlapping zones. Admins determine zone boundaries.

Name Servers (1) DNS name space divided into overlapping zones. Admins determine zone boundaries. Each zone has one or more name servers. Name servers contain data for portions of the name space called zones (circled). One zone CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Name Servers (2) Finding the IP address for a given hostname is called resolution

Name Servers (2) Finding the IP address for a given hostname is called resolution and is done with the DNS protocol. Resolution: • Computer requests local name server to resolve • Local name server asks the root name server • Root returns the name server for a lower zone • Continue down zones until a name server can answer DNS protocol: • Runs on UDP port 53, retransmits lost messages • Caches name server answers for better performance CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Name Servers (3) Example of a computer looking up the IP for a name

Name Servers (3) Example of a computer looking up the IP for a name CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Electronic Mail • • • Architecture and services » The user agent » Message

Electronic Mail • • • Architecture and services » The user agent » Message formats » Message transfer » Final delivery » CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architecture and Services (1) The key components and steps (numbered) to send email −

Architecture and Services (1) The key components and steps (numbered) to send email − User Agent = Email Reader; a program that supports composing, receiving, and replying to messages. − Uses Simple Mail Transfer Protocol (SMTP) over TCP (port 25) 1. Mail submission is method by which user agents send messages into the email system for delivery 2. Message transfer between MTAs 3. Final delivery to receiving User Agent Architecture of the email system Question: Where is the user’s mailbox located? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architecture and Services (2) Envelope Question: Who knows why this is funny for Tanenbaum?

Architecture and Services (2) Envelope Question: Who knows why this is funny for Tanenbaum? Message (= header and body) Paper mail Electronic mail CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

The User Agent What users see – interface elements of a typical user agent

The User Agent What users see – interface elements of a typical user agent − Email address format: user@dns-address CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Formats (1) Header fields related to message transport; headers are readable ASCII text

Message Formats (1) Header fields related to message transport; headers are readable ASCII text CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Formats (2) Other header fields useful for user agents CN 5 E by

Message Formats (2) Other header fields useful for user agents CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Formats (3) Originally only ASCII text could be sent by email. Multipurpose Internet

Message Formats (3) Originally only ASCII text could be sent by email. Multipurpose Internet Mail Extensions (MIME) was developed to send messages with richer content (non-Latin alphabets, Chinese, Japanese) as well as audio, images, or binary documents or programs. • MIME adds structures to the message body and also defines encoding rules for the transfer of non-ASCII messages. MIME header fields used to describe what content is in the body of the message Message Headers added by MIME CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Formats (4) Common MIME content types and subtypes CN 5 E by Tanenbaum

Message Formats (4) Common MIME content types and subtypes CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Formats (5) Putting it all together: a multipart message containing HTML and audio

Message Formats (5) Putting it all together: a multipart message containing HTML and audio alternatives. One part (HTML) Another (audio) Question: What does this email message do? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Transfer (1) Messages are transferred with SMTP (Simple Mail Transfer Protocol) • •

Message Transfer (1) Messages are transferred with SMTP (Simple Mail Transfer Protocol) • • Readable (4 byte long ASCII) text commands Submission from user agent to MTA on port 587 − If message cannot be delivered, an error report containing the first part of the undeliverable msg is returned to the sender • One MTA to the next MTA on port 25 usually using TCP − Recipient’s mailbox is within the destination MTA − The destination MTA is the @dns-address part of the user’s email address − user@dns-address is the user’s mailbox within the destination MTA • Other protocols for final delivery (IMAP, POP 3, webmail) − IMAP/POP 3/webmail get email messages from the mailbox on the user’s mail server to the user’s email user agent on the user’s machine. − Webmail systems use web protocols: E. g. , Gmail, Hotmail, Yahoo!mail CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Transfer (2) Sending a message: • From Alice to Bob • SMTP commands

Message Transfer (2) Sending a message: • From Alice to Bob • SMTP commands are marked [pink] C: = from client (sender) S: == from server (receiver) HELO – Hello message RCPT – ID email recipient(s) EHLO – Hello msg for clients wanting to use an extension (next slide) . . . (rest of message). . . Question: Does the Sender (client) or the Receiver (server) start the dialog? If the server is first, how does it know when to talk? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Message Transfer (3) Common SMTP extensions (not in simple example) CN 5 E by

Message Transfer (3) Common SMTP extensions (not in simple example) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Final Delivery (1) User agent uses protocol like IMAP for final delivery • Has

Final Delivery (1) User agent uses protocol like IMAP for final delivery • Has commands to manipulate folders / messages [right] Alternatively, a Web interface (with proprietary protocol) might be used CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

The World Wide Web (WWW) Standards body for the web is the W 3

The World Wide Web (WWW) Standards body for the web is the W 3 C (World Wide Web Consortium; see http: //www. w 3. org) Primary web protocol is HTTP, which is a simple text-based protocol that runs over TCP in the general case. • • • Architectural overview » Static Web pages » Dynamic pages and Web applications » HTTP – Hyper. Text Transfer Protocol » The mobile Web » Web search » CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (1) HTTP transfers pages from servers to browsers Question: Where is the

Architectural Overview (1) HTTP transfers pages from servers to browsers Question: Where is the User in this picture? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (2) Pages are named with URLs (Uniform Resource Locators) • Example: http:

Architectural Overview (2) Pages are named with URLs (Uniform Resource Locators) • Example: http: //www. phdcomics. com/comics. php Protocol Server Page on server Our focus Common URL protocols CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (3) Steps a client (browser) takes to follow a hyperlink: − −

Architectural Overview (3) Steps a client (browser) takes to follow a hyperlink: − − − Determine the protocol (HTTP) Ask DNS for the IP address of server Make a TCP connection to server (port 80 or 443) Send request for the page; server sends it back Fetch other URLs as needed to display the page Close idle TCP connections Steps a server takes to serve pages: − − − Accept a TCP connection from client Get page request and map it to a resource (e. g. , file name) Get the resource (e. g. , file from disk) Send contents of the resource to the client. Release idle TCP connections CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (4) -- Clients When a server returns a page, it also returns

Architectural Overview (4) -- Clients When a server returns a page, it also returns some additional info about that page. This info includes the MIME type of the page that instructs the client’s browser how to process that info. If the MIME type is not one of the built-in ones, then the browser consults its table of MIME types which associates MIME types with Third-Party-provided viewers. Two types of viewers: 1) plug in or 2) helper app. Content type is identified by MIME types • Browser takes the appropriate action to display • Plug-ins / helper apps extend browser for new types Installed as an extension to the browser (same process as the browser) Complete program running as a separate process Question: Where did we see MIME before? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (5) -- Servers Web servers are optimized to be able to quickly

Architectural Overview (5) -- Servers Web servers are optimized to be able to quickly service numerous requests To scale performance, Web servers can use: • Caching to optimize disk retrievals, multiple threads for parallel processing, and a front end to orchestrate CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (6) Server steps, revisited: 1. Resolve name of Web page requested −

Architectural Overview (6) Server steps, revisited: 1. Resolve name of Web page requested − Incoming request may not contain the actual name of file or program as a literal string 2. Perform access control on the Web page − Authenticate client to determine if (s)he can access material 3. Check the cache − Dynamic pages cannot be cached; only current cached content returned (if so, skip step 4) 4. Fetch requested page from disk or run program 5. Determine the rest of the response that accompany the contents of the page 6. Return the response to the client 7. Make an entry in the server log CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Architectural Overview (7) -- cookies Even though HTTP uses TCP, it does NOT have

Architectural Overview (7) -- cookies Even though HTTP uses TCP, it does NOT have any concept of a login session so each HTTP request has no memory nor context. Cookies provide a memory or context. Examples: pay-for-view web sites; e-commerce; customized web portals like Yahoo! Cookies can be abused to learn a user’s browsing habits. Cookies support stateful client/server interactions − Cookies are a small (4 KB or less) named string that the server associates with a browser − Browsers store the server-provided cookie for a time period within their browser directory on the client’s disk. Cookies persist across browser invocations to that server -- unless the user disables cookies. • • • Server sends cookies (state) with page response Client stores cookies across page fetches Client sends cookies back to server with requests Example of a browser directory within a client’s disk CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Static Web Pages (1) Static Web pages are simply files • Have the same

Static Web Pages (1) Static Web pages are simply files • Have the same contents for each viewing Can be visually rich and interactive nonetheless: • HTML that mixes text and images • Forms that gather user input • Style sheets that tailor presentation • Vector graphics, videos, and more (over). . . CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Static Web Pages (2) Progression of features through HTML 5. 0 CN 5 E

Static Web Pages (2) Progression of features through HTML 5. 0 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (1) Applications can now run inside the browser or

Dynamic Pages & Web Applications (1) Applications can now run inside the browser or the server with the web providing the user interface. Users do not need to install separate apps and user data can be accessed from different computers; this is a form of cloud computing Dynamic pages are generated by programs running at the server (with a database) and/or the client • E. g. , PHP, JSP, and CGI at server; Java. Script at client − − • Microsoft’s Active Server Pages. NET is a proprietary version of PHP and JSP CGI = Common Gateway Interface; JSP = Java. Server Pages vary each time like using an application Question: What does “a form of cloud computing” mean? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (2) Web page that gets form input and calls

Dynamic Pages & Web Applications (2) Web page that gets form input and calls a server program PHP server program that creates a custom Web page PHP calls Resulting Web page (for inputs “Barbara” and “ 32”) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (3) Client-side processing occurs inside the user’s browser. Technologies:

Dynamic Pages & Web Applications (3) Client-side processing occurs inside the user’s browser. Technologies: 1) Java. Script, 2) VBScript, 3) Java applets, and 4) Microsoft Active. X controls (see pages 677 -678) Java. Script program produces result page in the browser First page with form, gets input and calls program above CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (4) Client-side and Server-side scripting look the similar in

Dynamic Pages & Web Applications (4) Client-side and Server-side scripting look the similar in that they both embed code in HTML files, they are processed totally differently. The difference between server and client programs Server-side scripting with PHP After user clicks on submit button, the browser collects info into a long string and sends it to server as a request for PHP Page. The server loads the PHP file and executes the PHP script to produce a new HTML page that is sent to the browser to display. Client-side scripting with Java. Script When the user clicks on the submit button the browser interprets a Java. Script function contained on the page. All of the work is done locally within the browser. Question: when is it preferable to use server-side and when preferable to use client-side scripting? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (5) AJAX (Asynchronous Javascript and XML) is a set

Dynamic Pages & Web Applications (5) AJAX (Asynchronous Javascript and XML) is a set of technologies that work together to enable Web applications that are as responsive and powerful as traditional desktop applications. Web applications use a set of technologies that work together, e. g. AJAX: • • HTML: present information as pages. DOM: change parts of pages while they are viewed. − DOM (Document Object Model) – model of HTML page that is accessible to programs (e. g. , an API to change parts of a page) • XML: let programs exchange data with the server. − XML (e. Xtensible Markup Language) is a language for specifying structural content − W 3 C created XML to allow web content to be structured for automated processing • Asynchronous way to send and retrieve XML data. − SOAP (Simple Object access Protocol) – language/systemindependent way to do RPC between programs • Java. Script as a language to bind all this together. CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (6) The DOM (Document Object Model) tree represents Web

Dynamic Pages & Web Applications (6) The DOM (Document Object Model) tree represents Web pages as a structure that programs can alter CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (7) XML captures document structure HTML is concerned with

Dynamic Pages & Web Applications (7) XML captures document structure HTML is concerned with presentation. Example of XML: Note: XHTML is HTML defined in terms of XML CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Dynamic Pages & Web Applications (8) Web application developers have a library of technologies

Dynamic Pages & Web Applications (8) Web application developers have a library of technologies available to them to create web content: CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (1) HTTP is the primary protocol of the web. HTTP (Hyper. Text Transfer

HTTP (1) HTTP is the primary protocol of the web. HTTP (Hyper. Text Transfer Protocol) is a requestresponse protocol that runs on top of TCP • • • Fetches pages from server to client Server usually runs on port 80 Headers are given in readable ASCII Content is described with MIME types Protocol has support for pipelining requests Protocol has support for caching CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (2) HTTP 1. 0 – connection was set up and released for every

HTTP (2) HTTP 1. 0 – connection was set up and released for every response HTTP 1. 1 – Enables connection reuse HTTP uses persistent connections to improve performance One connection for each request HTTP 1. 0 Sequential requests on one connection HTTP 1. 1 Pipelined requests on one connection HTTP 1. 1 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (3) HTTP has several request methods. Requests server to Fetch a MIME-encoded page

HTTP (3) HTTP has several request methods. Requests server to Fetch a MIME-encoded page Used to send input data (forms) to a server program Both GET and POST are used for SOAP web services CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (4) Response codes tell the client how the request fared: CN 5 E

HTTP (4) Response codes tell the client how the request fared: CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (5) A request line (e. g. , GET, POST) may be optionally followed

HTTP (5) A request line (e. g. , GET, POST) may be optionally followed by additional lines with more information, called request headers. Responses also optionally may have response headers. Many headers carry key information: Function Example Headers Browser capabilities (client server) User-Agent, Accept-Charset, Accept. Encoding, Accept-Language Caching related (mixed directions) If-Modified-Since, If-None-Match, Date, Last. Modified, Expires, Cache-Control, ETag Browser context (client server) Cookie, Referer, Authorization, Host Content delivery (server client) Content-Encoding, Content-Length, Content-Type, Content-Language, Content-Range, Set-Cookie CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

HTTP (6) HTTP has built-in support to help clients identify when they can safely

HTTP (6) HTTP has built-in support to help clients identify when they can safely reuse pages HTTP caching checks to see if the browser has a known fresh copy, and if not if the server has updated the page • • − HTTP uses two strategies: 1) cache page validation (step 2), 2) conditional GET – asks server if the cached copy is still valid Uses a collection of headers for the checks Can include further levels of caching (e. g. , proxy caching) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

The Mobile Web Enhancement/extensions made to support web access for small handhelds and other

The Mobile Web Enhancement/extensions made to support web access for small handhelds and other mobile devices. Mobiles (phones, tablets) are challenging as clients: − − − Relatively small screens Limited input capabilities, lengthy input. Network bandwidth is limited Connectivity may be intermittent. Computing power is limited Strategies to handle them: • Content: servers provide mobile-friendly versions; transcoding can also be used • Protocols: no real need for specialized protocols; HTTP with header compression sufficient CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Web Search has proved hugely popular, in tandem with advertising that has proved hugely

Web Search has proved hugely popular, in tandem with advertising that has proved hugely profitable • A simple interface for users to navigate the Web Search engine requires: • Content from all sites, accessed by crawling. Follow links to new pages, but beware programs. − Each web search engine obtains its database of pages by doing web crawling – a systematic traversal of all pages and links − Deep web: Web content that search engines cannot catalog (e. g. , dynamic content), which is therefore “hidden” • Indexing, which benefits from known and discovered structure (such as XML) to increase relevance CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Audio and Video Real time and streamed multimedia traffic. • Real time multimedia

Streaming Audio and Video Real time and streamed multimedia traffic. • Real time multimedia is usually carried using RTP • Streamed multimedia may be carried by RTP or HTTP • Real Time Streaming Protocol – proprietary (Real. Networks) variant of RTCP • RTCP (real time control protocol) controls remote multimedia playout Audio and video have become key types of traffic, e. g. , voice over IP, and video streaming. • Digital audio » • Digital video » • Streaming stored media » • Streaming live media » • Real-time conferencing » CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Audio (1) ADC (Analog-to-Digital Converter) produces digital audio from a microphone • Telephone:

Digital Audio (1) ADC (Analog-to-Digital Converter) produces digital audio from a microphone • Telephone: 8000 8 -bit samples/second (64 Kbps); computer audio is usually better quality (e. g. , 16 bit) ADC Continuous audio (sine wave) Digital audio (sampled, 4 -bit quantized) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Audio (2) Digital audio is typically compressed before it is sent • Lossy

Digital Audio (2) Digital audio is typically compressed before it is sent • Lossy encoders (like AAC) exploit human perception • Large compression ratios (can be >10 X) Sensitivity of the ear varies with frequency A loud tone can mask nearby tones CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Video (1) Video is digitized as pixels (sampled, quantized) • TV quality: 640

Digital Video (1) Video is digitized as pixels (sampled, quantized) • TV quality: 640 x 480 pixels, 24 -bit color, 30 times/sec Video is sent compressed due to its large bandwidth • Lossy compression exploits human perception − E. g. , JPEG for still images, MPEG, H. 264 for video • • Large compression ratios (often 50 X for video) Video is normally > 1 Mbps, versus >10 kbps for speech and >100 kbps for music CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Video (2) JPEG lossy compression sequence for one image: Step 1 Step 2

Digital Video (2) JPEG lossy compression sequence for one image: Step 1 Step 2 Step 3 Step 5 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Video (3) Step 1: Pixels are mapped to luminance/chrominance (YCb. Cr) color space

Digital Video (3) Step 1: Pixels are mapped to luminance/chrominance (YCb. Cr) color space and chrominance is sub-sampled • The eye is less sensitive to chrominance Input 24 -bit RGB pixels 8 -bit luminance pixels CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011 8 -bit chrominances for every 4 pixels

Digital Video (4) Step 2: Each component block is transformed to spatial frequencies with

Digital Video (4) Step 2: Each component block is transformed to spatial frequencies with DCT (Discrete Cosine Transformation) • Captures the key image features One component block Transformed block CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Video (5) Step 3: DCT coefficients are quantized by dividing by thresholds; reduces

Digital Video (5) Step 3: DCT coefficients are quantized by dividing by thresholds; reduces bits in higher spatial frequencies • Top left element is differenced over blocks (Step 4) Input / Thresholds = CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011 Output

Digital Video (6) Step 5: The block is run-length encoded in a zig-zag order.

Digital Video (6) Step 5: The block is run-length encoded in a zig-zag order. Then it is Huffman coded before sending (Step 6) Order in which the block coefficients are sent CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Digital Video (7) MPEG compresses over a sequence of frames, further using motion tracking

Digital Video (7) MPEG compresses over a sequence of frames, further using motion tracking to remove temporal redundancy − I (Intra-coded) frames are self-contained − P (Predictive) frames use block motion predictions − B (Bidirectional) frames may base prediction on future frame Three consecutive frames with stationary and moving components CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (1) A simple method to stream stored media, e. g. ,

Streaming Stored Media (1) A simple method to stream stored media, e. g. , for video on demand, is to fetch the video as a file download • But has large startup delay, except for short files CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (2) Effective streaming starts the playout during transport • With RTSP

Streaming Stored Media (2) Effective streaming starts the playout during transport • With RTSP (Real-Time Streaming Protocol) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (3) Key problem: how to handle transmission errors Strategy Advantage Disadvantage

Streaming Stored Media (3) Key problem: how to handle transmission errors Strategy Advantage Disadvantage Use reliable transport (TCP) Repairs all errors Increases jitter significantly Add FEC (e. g. , parity) Repairs most errors Increases overhead, decoding complexity and jitter Interleave media Masks most errors Slightly increases decoding complexity and jitter CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (4) Parity packet can repair one lost packet in a group

Streaming Stored Media (4) Parity packet can repair one lost packet in a group of N • Decoding is delayed for N packets CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (5) Interleaving spreads nearby media samples over different transmissions to reduce

Streaming Stored Media (5) Interleaving spreads nearby media samples over different transmissions to reduce the impact of loss Packet stream Media samples Loss reduces temporal resolution; doesn’t leave a gap CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (6) Key problem: media may not arrive in time for playout

Streaming Stored Media (6) Key problem: media may not arrive in time for playout due to variable bandwidth and loss/retransmissions • Client buffers media to absorb jitter; we still need to pick an achievable media rate Safety margin, to avoid a stall Can pause server (or go ahead and save to disk) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Stored Media (7) RTSP commands • Sent from player to server to adjust

Streaming Stored Media (7) RTSP commands • Sent from player to server to adjust streaming CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Live Media (1) Streaming live media is similar to the stored case plus:

Streaming Live Media (1) Streaming live media is similar to the stored case plus: • Can’t stream faster than “live rate” to get ahead − Usually need larger buffer to absorb jitter • Often have many users viewing at the same time − UDP with multicast greatly improves efficiency. It is rarely available, so many TCP connections are used. − For very many users, content distribution is used [later] CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Live Media (2) With multicast streaming media, parity is effective • Clients can

Streaming Live Media (2) With multicast streaming media, parity is effective • Clients can each lose a different packet and recover CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Streaming Live Media (2) Production side of a student radio station. As before CN

Streaming Live Media (2) Production side of a student radio station. As before CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (1) Real-time conferencing has two or more connected live media streams, e.

Real-Time Conferencing (1) Real-time conferencing has two or more connected live media streams, e. g. , voice over IP, Skype video call Key issue over live streaming is low (interactive) latency • Want to reduce delay to near propagation • Benefits from network support, e. g. , Qo. S • Or, benefits from ample bandwidth (no congestion) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (2) H. 323 architecture for Internet telephony supports calls between Internet computers

Real-Time Conferencing (2) H. 323 architecture for Internet telephony supports calls between Internet computers and PSTN phones. Vo. IP call Internet/PSTN Internet Gatekeeper controls calls for LAN hosts CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (3) H. 323 protocol stack: • Call is digital audio/video over RTP/UDP/IP

Real-Time Conferencing (3) H. 323 protocol stack: • Call is digital audio/video over RTP/UDP/IP • Call setup is handled by other protocols (Q. 931 etc. ) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (4) Logical channels that make up an H. 323 call CN 5

Real-Time Conferencing (4) Logical channels that make up an H. 323 call CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (5) SIP (Session Initiation Protocol) is an alternative to H. 323 to

Real-Time Conferencing (5) SIP (Session Initiation Protocol) is an alternative to H. 323 to set up real-time calls • Simple, text-based protocol with URLs for addresses • Data is carried with RTP / RTCP as before CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (6) Setting up a call with the SIP three-way handshake • Proxy

Real-Time Conferencing (6) Setting up a call with the SIP three-way handshake • Proxy server lets a remote callee be connected • Call data flows directly between caller/callee CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Real-Time Conferencing (7) YES Comparison of H. 323 and SIP. CN 5 E by

Real-Time Conferencing (7) YES Comparison of H. 323 and SIP. CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011 Corrected by Instructor

Content Delivery of content, especially Web and video, to users is a major component

Content Delivery of content, especially Web and video, to users is a major component of Internet traffic • • Content and Internet traffic » Server farms and Web proxies » Content delivery networks » Peer-to-peer networks » CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Content and Internet Traffic Internet traffic: 1. Shifts seismically (email FTP Web P 2

Content and Internet Traffic Internet traffic: 1. Shifts seismically (email FTP Web P 2 P video) 2. Has many small/unpopular and few large/popular flows – mice and elephants Zipf popularity distribution, 1/k Shows up as a line on log-log plot CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Server Farms and Web Proxies (1) Server farms enable large-scale Web servers: • Front-end

Server Farms and Web Proxies (1) Server farms enable large-scale Web servers: • Front-end load-balances requests over servers • Servers access the same backend database CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Server Farms and Web Proxies (2) Proxy caches help organizations to scale the Web

Server Farms and Web Proxies (2) Proxy caches help organizations to scale the Web • Caches server content over clients for performance • Also implements organization policies (e. g. , access) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

CDNs – Content Delivery Networks (1) CDNs scale Web servers by having clients get

CDNs – Content Delivery Networks (1) CDNs scale Web servers by having clients get content from a nearby CDN node (cache) CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Content Delivery Networks (2) Directing clients to nearby CDN nodes with DNS: • Client

Content Delivery Networks (2) Directing clients to nearby CDN nodes with DNS: • Client query returns local CDN node as response • Local CDN node caches content for nearby clients and reduces load on the origin server CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Content Delivery Networks (3) Origin server rewrites pages to serve content via CDN Traditional

Content Delivery Networks (3) Origin server rewrites pages to serve content via CDN Traditional Web page on server Page that distributes content via CDN CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Peer-to-Peer Networks (1) P 2 P (Peer-to-Peer) is an alternative CDN architecture with no

Peer-to-Peer Networks (1) P 2 P (Peer-to-Peer) is an alternative CDN architecture with no dedicated infrastructure (i. e. , servers) • Clients serve content to each other as peers Challenges when servers are removed: 1. How do peers find each other? 2. How do peers support rapid content downloads? 3. How do peers encourage each other to upload? CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Peer-to-Peer Networks (2) Bit. Torrent lets peers download torrents − Peers find each other

Peer-to-Peer Networks (2) Bit. Torrent lets peers download torrents − Peers find each other via Tracker in torrent file − Peers swap chunks (parts of content) with partners, preferring those who send most quickly [2] − Many peers speed download; preference helps uploads [3] CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Peer-to-Peer Networks (3) Distributed Hash Tables (DHTs) are a fully distributed index that scales

Peer-to-Peer Networks (3) Distributed Hash Tables (DHTs) are a fully distributed index that scales to very many clients/entries • Need to follow O(log N) path for N entries • Can use as Tracker to find peers with no servers [1] • Look up torrent (identifier) in DHT to find IP of peers • Kademlia is used in Bit. Torrent CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

Peer-to-Peer Networks (3) A Chord ring of 32 identifiers. Finger tables [at right, and

Peer-to-Peer Networks (3) A Chord ring of 32 identifiers. Finger tables [at right, and as arcs] are used to navigate the ring. • Example: path to look up 16 from 1 is 1 12 15 Identifier values are stored at predecessor CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011

End Chapter 7 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall

End Chapter 7 CN 5 E by Tanenbaum & Wetherall, © Pearson Education-Prentice Hall and D. Wetherall, 2011