Computer Networks An Open Source Approach Chapter 6

Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer 1

Content n n n n n 6. 1 Issues 6. 2 Domain Name System 6. 3 E-Mail 6. 4 World Wide Web 6. 5 File Transfer Protocol 6. 6 Simple Network Management Protocol 6. 7 Voice over IP (Vo. IP) 6. 8 Streaming 6. 9 Peer-to-Peer Applications (P 2 P) 6. 10 Summary Chapter 6: Application Layer 2

6. 1 Issues n n How ports work? How servers start? Classification of servers Characteristics of application layer protocols Chapter 6: Application Layer 3

How Ports Work? n n n Well known ports: 0~1023 Registered ports: 1024~49151 Dynamic and/or private ports: 49152~65535 Chapter 6: Application Layer 4

How Servers Start? n n Is a server better to run as standalone or via (x)inetd? Standalone: q q q n requires explicit restarting for configuration changes to take effect crashed standalone server will probably stay unnoticed and the service will become unavailable better performance (x)inetd: q q reads configuration files each time a client attempts to access their information fresh copies will be spawned on demand Chapter 6: Application Layer 5

Classification of Servers n n Concurrent connectionless server Concurrent connection-oriented server Iterative connectionless server Iterative connection-oriented server Chapter 6: Application Layer 6

Underlying Transport Protocols Application layer protocol Underlying transport protocol electronic mail SMTP, POP 3, IMAP TCP remote terminal access Telnet TCP file transfer FTP TCP web HTTP TCP web caching ICP typically UDP name resolution DNS typically UDP network file system NFS typically UDP network management SNMP typically UDP routing protocol RIP typically UDP Internet telephony proprietary (e. g. Vocaltec) typically UDP streaming multimedia proprietary (e. g. Real. Networks) typically UDP P 2 P proprietary (e. g. Bit. Torrent, e. Donkey) UDP for queries and TCP for data transfer Chapter 6: Application Layer 7

Iterative Connectionless Server Chapter 6: Application Layer 8

Concurrent Connection-Oriented Server Chapter 6: Application Layer 9

Characteristics of Application Layer Protocols n Variable message formats and length q n Variable data types q q n Message formats and length of application layer protocols vary with different applications and requirements Messages can be transmitted in textual or non-textual formats. E. g. , a Web server replies textual Web pages and binary images Statefulness q q The server retains information of the session with the client E. g. , an FTP server remembers the client’s current working directory Chapter 6: Application Layer 10

6. 2 Domain Name System n n n Domain name space and domain delegation Zones and name servers Zone data files and resource records Resolvers and Name resolution Open source: BIND Chapter 6: Application Layer 11

Top Level Domains Domain com Description Commercial organizations, such as Intel (intel. com). org Non-profit organizations, such as WWW consortium (w 3. org). gov Government organizations, reserved for U. S government such as National Science Foundation (nsf. gov). edu Educational organizations, such as UCLA (ucla. edu). net Networking organizations, such as Internet Assigned Numbers Authority which maintains the DNS root servers (gtld-servers. net). int Organizations established by international treaties between governments. For example, International Telecommunication Union (itu. int). Mil Reserved exclusively for the United States Military. For example, Network Information Center, Department of Defense (nic. mil). Two-letter The two-letter country code top level domains (cc. TLDs) are based on the country code ISO 3166 -1 two-letter country codes. Examples are tw (Taiwan), uk (United Kingdom). arpa Mostly unused now, except for the in-addr. arpa domain, which is used to maintain a database for reverse DNS queries. Others Such as. biz (business), . name (for individuals), . info (similar with. com). Chapter 6: Application Layer 12

Domain Delegation n. Easier management n. Load distribution root domain tw org com edu uk co nthu nctu nsysu csie cis ee cis. nctu. edu. tw Chapter 6: Application Layer 13

Zones and Name Servers n Differences between a zone and a domain q q q n Zone data files in a name server q q n A zone contains some part of the domain name space A domain is a subtree in the domain name space A name server may be authoritative for multiple zones Contains Resource Records (RR) describing all the hosts within that zone Six major types of RR: “SOA”, “NS”, “A”, “PTR”, “CNAME” and “MX” Master and slave name servers q q Availability and load balance Data duplication by zone transfer n n Request from slave Notify by master Chapter 6: Application Layer 14

Master/Slave Name Servers 1. Redundancy for heavy load 2. Availability achieved slave df of A master (A) | slave (B) zone transfer clients zone A df of A and B zone transfer A large domain name space df of B clients zone B df: zone data file Chapter 6: Application Layer 15

Resource Records n SOA (Start Of Authority) q q q n NS (Name Server) q q n Specify a best name server to be authoritative (master) for a zone Ex: cis. nctu. edu. tw. 86400 IN SOA cisserv. cis. nctu. edu. tw. Options for refresh, retry, expire. . etc are supported Specify name servers for a domain Ex: cis. nctu. edu. tw. 86400 IN NS cisserv. cis. nctu. edu. tw. A (Address) q q Map names to addresses, Example (multi-homed): linux. cis. nctu. edu. tw. 86400 IN A 140. 113. 168. 127 linux. cis. nctu. edu. tw. 86400 IN A 140. 113. 207. 127 Chapter 6: Application Layer 16

Resource Records (cont) n CNAME (Canonical Name) q q n PTR (Pointer) q q n Create name-to-name alias Ex: www. cis. nctu. edu. tw. 86400 IN CNAME cache. cis. nctu. edu. tw. Point addresses to names (canonical name only) Ex: 10. 23. 113. 140. in-addr. arpa. 86400 IN PTR laser 0. cis. nctu. edu. tw. MX (Mail Exchanger) q q Provide mail-routing information The one of best (lowest) preference value is chosen cis. nctu. edu. tw. 86400 IN MX 0 mail. cis. nctu. edu. tw. 86400 IN MX 10 mail 1. cis. nctu. edu. tw. Chapter 6: Application Layer 17

Name Resolution n Query type q q n Resolution method q q n Forward query – name-to-address Reverse query – address-to-name Iterative (by referral, commonly adopted) Recursive (site-by-site recursion) Caching for speeding up the lookup process q q Expire according to TTL Tradeoff between performance and consistency Chapter 6: Application Layer 18

Name Resolution (Iterative) 1 query for “www. dti. gov. uk” client (resolver ) local name server www. dti. gov. uk. 5 M IN A 164. 36. 253. 20 www. dti. gov. uk. 5 M IN A 164. 36. 164. 20 2 3 4 root name server “uk” name server 5 6 ”gov. uk” name server 7 8 “dti. gov. uk” name server Chapter 6: Application Layer other candidate name servers 19

Message Format 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 bit ID Q R Opcode A A T C R D R A Reserved QDCOUNT ANCOUNT Rcode Header NSCOUNT ARCOUNT Question Answer Authority Additional Chapter 6: Application Layer 20

Execution Flowchart of Named Install a handler for signals Cache initialization Command line parsing chroot and daemonize 1. Task manager Create managers 2. Timer manager 3. Socket manager Initialize server configurations and data structures Wait for signals For possible restart or termination signals Chapter 6: Application Layer 21

Chapter 6: Application Layer 22

Basic Data Structure in Named viewlist view 2 zone_table zone 1 Red/Black Tree for zones … view 1 zone 3 … view. N NULL zone_table zone 3 zone 2 zone 4 zone 8 RR 1 zone. N RR 2 … Red/Black Tree of RRs for zone 4 RRN Chapter 6: Application Layer 23

Domain Information Gopher (DIG) ; <<>> Di. G 9. 2. 0 <<>> www. nctu. edu. tw ; ; global options: printcmd ; ; Got answer: ; ; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26027 ; ; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 3 ; ; QUESTION SECTION: ; www. nctu. edu. tw. IN A ; ; ANSWER SECTION: www. nctu. edu. tw. 259200 IN ; ; AUTHORITY SECTION: nctu. edu. tw. 259200 IN ; ; ADDITIONAL SECTION: ns. nctu. edu. tw. 259200 IN ns 2. nctu. edu. tw. 259200 IN ns 3. nctu. edu. tw. 259200 IN A 140. 113. 250. 5 NS NS NS ns. nctu. edu. tw. ns 2. nctu. edu. tw. ns 3. nctu. edu. tw. A A A 140. 113. 250. 135 140. 113. 6. 2 163. 28. 64. 11 Chapter 6: Application Layer 24

6. 3 E-Mail n n Introduction Message Formats Internet Mail Protocols Open Source: Qmail Chapter 6: Application Layer 25

Introduction n What is E-mail? q n A method of sending messages from one user to another via computer networks Internet mail addressing: q q format: user@host. network e. g. ydlin@cis. nctu. edu. tw Chapter 6: Application Layer 26

Introduction (cont. ) n Components of Internet mail system: q q Mail User Agent (MUA) Mail Transfer Agent (MTA) Mail Delivery Agent (MDA) Mail Retrieval Agent (MRA) Chapter 6: Application Layer 27

Message Formats n n Internet Message Format (RFC 822) Multipurpose Internet Mail Extensions (RFC 2045 -2049) Chapter 6: Application Layer 28

RFC 822 – Internet Message Format n A message consists of: q q An envelope which contains information needed to accomplish transport and delivery Contents which compose the object to be delivered to the recipient Chapter 6: Application Layer 29

RFC 822 – Internet Message Format (cont. ) n Common message header fields: Chapter 6: Application Layer 30

RFC 822 – Internet Message Format (cont. ) n An example of message header: From: "Dr. Ying-Dar Lin" <ydlin@cis. nctu. edu. tw> To: juvenia@cissun 53. cis. nctu. edu. tw Date: Mon, 30 Jul 2001 02: 42: 34 +0800 (CST) Subject: paper Return-Path: <ydlin@cis. nctu. edu. tw> Received: (from root@localhost) by mail. cis. nctu. edu. tw (8. 11. 4/8. 11. 4) id f 6 TIgct 33454 for gis 89813@cis. nctu. edu. tw; Mon, 30 Jul 2001 02: 42: 38 +0800 (CST) (envelope-from ydlin@cis. nctu. edu. tw) Received: from cissun 53. cis. nctu. edu. tw (ydlin@cissun 53. cis. nctu. edu. tw [140. 113. 23. 53]) by mail. cis. nctu. edu. tw (8. 11. 4/8. 11. 4 av) with ESMTP id f 6 TIga. I 33445 for <gis 89813@cis. nctu. edu. tw>; Mon, 30 Jul 2001 02: 42: 36 +0800 (CST) (envelope-from ydlin@cis. nctu. edu. tw) Received: (from ydlin@localhost) by cissun 53. cis. nctu. edu. tw (8. 9. 3/8. 9. 3) id CAA 16191 for juvenia; Mon, 30 Jul 2001 02: 42: 34 +0800 (CST) Message-Id: <200107291842. CAA 16191@cissun 53. cis. nctu. edu. tw> X-Virus-Scanned: by AMa. Vi. S perl X-UIDL: F 5%#!88!"!>RG"!l!G!! Chapter 6: Application Layer 31

MIME n What is MIME? q n MIME is a specification for enhancing the capabilities of conventional Internet message format. MIME enables email messages to have: q q textual header information and message bodies in character sets other than 7 -bit ASCII multiple objects within a single message binary or application-specific files multimedia files, such as images, audio, and video files Chapter 6: Application Layer 32

MIME (cont. ) n MIME message header: Chapter 6: Application Layer 33

MIME (cont. ) n The MIME content type set: Chapter 6: Application Layer 34

MIME (cont. ) n Possible values for the Content-Transfer-Encoding: field: q q q Quoted-Printable Base 64 7 bit 8 bit Binary X-Encoding Chapter 6: Application Layer 35

MIME (cont. ) n An example of MIME message: From: 'Ching-Ming Tien' <juvenia@cis. nctu. edu. tw> To: ydlin@cis. nctu. edu. tw Subject: Cover MIME-Version: 1. 0 Content-Type: image/jpg; name=cover. jpg' Content-Transfer-Encoding: base 64 Content-Description: The front cover of the book <. . . base 64 encoded jpg image of cover. . . > Chapter 6: Application Layer 36

Internet Mail Protocols n n n Simple Mail Transfer Protocol (RFC 2821) Post Office Protocol (RFC 1939) Internet Message Access Protocol (RFC 2060) Chapter 6: Application Layer 37

Simple Mail Transfer Protocol n What is SMTP? q A standard host-to-host mail transport protocol and traditionally operates over TCP on port 25 Chapter 6: Application Layer 38

Simple Mail Transfer Protocol (cont. ) n Important SMTP commands: n SMTP replies: Response Description 2 xx Command accepted and processed. 3 xx General flow control. 4 xx Critical system or transfer failure. 5 xx Errors with the SMTP command. Chapter 6: Application Layer 39

Simple Mail Transfer Protocol (cont. ) n An SMTP transaction scenario: Chapter 6: Application Layer 40

Post Office Protocol What is POP? n A protocol used to retrieve e-mail from a mail server. q POP 3 Session States: n Authorization: 1. n Must log in with password before entering transaction state. Transaction: 2. n Client can request actions of server, get mail for example. Update: 3. n Updates mail box to reflect actions taken in transaction state. Chapter 6: Application Layer 41

Post Office Protocol (cont. ) n Minimal POP 3 commands: Chapter 6: Application Layer 42

Post Office Protocol (cont. ) n POP 3 replies: q n +OK, -ERR Example POP session: Chapter 6: Application Layer 43

Internet Message Access Protocol n What is IMAP? q n A replacement for the POP 3 protocol Differences between IMAP 4 and POP 3: q q IMAP 4 allows messages being stored and manipulated on the mail system POP 3 only allows users to download their messages and store and manipulate messages on the client’s machines Chapter 6: Application Layer 44

Internet Message Access Protocol (cont. ) n Four states in the IMAP 4 server: q Non-authenticated state n q Authenticated state n n n q When a pre-authenticated connection starts, When acceptable authentication credentials have been provided After an error in selecting a mailbox Selected state n q When a connection is established between the server and client When a mailbox has been successfully selected Logout state n When the client asks to exit the server Chapter 6: Application Layer 45

Internet Message Access Protocol (cont. ) n IMAP 4 command summary: Chapter 6: Application Layer 46

Internet Message Access Protocol (cont. ) n An IMAP 4 transaction scenario: Chapter 6: Application Layer 47

Chapter 6: Application Layer 48

Open Source: qmail n Introduction to qmail: q q qmail is a secure, reliable, efficient, simple MTA designed for Unix-like operating systems qmail is a replacement for the sendmail qmail is the second most common SMTP server qmail has by far the fastest growth of any SMTP server Chapter 6: Application Layer 49

qmail System Structure n Core modules of qmail: Chapter 6: Application Layer 50

qmail Data Flow Chapter 6: Application Layer 51

qmail Control Files n n Control files are in /var/qmail/control Some control files of qmail: Chapter 6: Application Layer 52

qmail Queue Structure n n The qmail queue directory is in /var/qmail/quque Subdirectories in the qmail queue and contents : Chapter 6: Application Layer 53

How Messages Pass Through the qmail Queue Chapter 6: Application Layer 54

6. 4 World Wide Web n n n Introduction Web Naming and Addressing Web Data Formats Hypertext Transfer Protocol Open Source: Apache Chapter 6: Application Layer 55

Introduction n What is WWW? q n A system of Internet servers that support specially formatted documents. How does it work? Chapter 6: Application Layer 56

Web Naming and Addressing n n n Uniform Resource Identifier (RFC 2396) Uniform Resource Locator (RFC 1738) Uniform Resource Name (RFC 2141) Chapter 6: Application Layer 57

Uniform Resource Identifier n What is URI? q n URI syntax: q q n A compact string of characters for identifying an abstract or physical resource. Absolute URI: <scheme>: <scheme-specific-part> Generic URI: <scheme>: //<authority><path>? <query> URI examples: q q q q http: //speed. cis. nctu. edu. tw/~ydlin/index. html#Books http: //www. google. com/search? q=linux ftp: //ftp. cis. nctu. edu. tw/Documents/IETF/rfc 2300~2399/rfc 2396. txt mailto: ydlin@cis. nctu. edu. tw news: comp. os. linux telnet: //bbs. cis. nctu. edu. tw/. . /icons/logo. gif Chapter 6: Application Layer 58

Uniform Resource Locator n n What is URL? q A compact string representation of the location for a resource that is available via the Internet URL syntax: q <service>//<user>: <password>@<host>: <port>/<url-path> Chapter 6: Application Layer 59

Uniform Resource Locator (cont. ) n Some URL examples: q q http: //www. cis. nctu. edu. tw/chinese/ccg/title. Main. gif ftp: //john: secret@ftp. cis. nctu. edu. tw/projects/book. txt nntp: //news. cis. nctu. edu. tw/cis. course. computernetworks/5238 telnet: //mail. cis. nctu. edu. tw: 110/ Chapter 6: Application Layer 60

Uniform Resource Name n n What is URN? q A name that identifies a resource of unit of information independent of its location URN syntax: q q q n URN examples: q q q n <URN> : : = "urn: " <NID> ": " <NSS> NID: Namespace Identifier NSS: Namespace Specific String urn: path: /A/B/C/doc. html urn: ans: cis. nctu. edu. tw/ydlin/Resource urn: isbn: 0 -201 -56317 -7 URN resolutioin: q http: //www. isbn. com/0 -201 -56317 -7 Chapter 6: Application Layer 61

Web Data Formats n Evolution of Web Data Formats q q SGML HTML XHTML Chapter 6: Application Layer 62

Standard Generalized Markup Language n What is SGML? q n A system for organizing and tagging elements of a document. Characteristics of SGML q q q Descriptive Markup Types of Document Data Independence Chapter 6: Application Layer 63

Hyper. Text Makeup Language n What is HTML? q n A brief history of HTML q n The authoring language used to create documents on the World Wide Web. HTML 2. 0, 3. 2, 4. 0 HTML document structure q q q HTML version information Header Body Chapter 6: Application Layer 64

Extensible Markup Language n What is XML? q n n A pared-down version of SGML, designed especially for Web documents. Why XML? How to use XML? q q Traditional data processing Document-driven programming (DDP) Archiving Binding Chapter 6: Application Layer 65

Extensible Hyper. Text Markup Language n What is XHTML? q n n A hybrid between HTML and XML specifically designed for Net device displays. Why XHTML? Using XHTML with other W 3 C tag sets: q q q XHTML for structural markup of documents SMIL for multimedia Math. ML for mathematics SVG for scalable vector graphics XForms for smart web forms Chapter 6: Application Layer 66

Hypertext Transfer Protocol n n n What is HTTP? HTTP Conversation Client Request Server Response HTTP 1. 1 Chapter 6: Application Layer 67

What is HTTP? n n A comprehensive addressing scheme Client-Server Architecture The HTTP protocol is connectionless and stateless An extensible and open representation for data types Chapter 6: Application Layer 68

HTTP Conversation n n n Client request Request headers Request body Server status Response headers Requested data Disconnected Chapter 6: Application Layer 69

Client Request n HTTP request methods: Chapter 6: Application Layer 70

Client Request n Request example: Chapter 6: Application Layer 71

Server Response n Response example: n Server status codes: Chapter 6: Application Layer 72

Chapter 6: Application Layer 73

HTTP 1. 1 n HTTP 1. 1 features: q q q Persistent connections Chunked encoding byte ranges Caching Non-IP virtual hosting Chapter 6: Application Layer 74

Web Caching n n Web caching is a mechanism to expedite document downloading in WWW. To achieve maximum satisfaction from Web caching, some aspects need to be considered. q q q Candidates to be cached Content replacement Cache coherence Copyright Reserved 2010 Chapter 6: Application Layer 75

Transparent Proxy n n n A cache server can also act as a proxy server With port redirection, transparent proxy does not require manual configuration Two types of transparent proxy q q integrated within a gateway standalone server box Chapter Copyright 6: Application Reserved 2010 Layer 76

Open Source: Apache n Introduction to Apache: q q q Open-Source Web server originally based on NCSA server Available on over 160 varieties of Unix -- and Windows NT Over 58% of Internet Web servers run Apache or an Apache derivative Chapter 6: Application Layer 77

Chapter 6: Application Layer 78

Chapter 6: Application Layer 79

Chapter 6: Application Layer 80

Chapter 6: Application Layer 81

Chapter 6: Application Layer 82

Chapter 6: Application Layer 83

Ration of sendfile ( ) to ap_invoke_handler ( ) File size 1 kb 4 kb 16 kb 64 kb 256 kb 1024 kb # of calling sendfile ( ) 1 1 1 2 7 15 Total execution time (μs) of sendfile ( ) 37 37 42 78 215 527 38% 40% 53% 77% 87% Time ratio of sendfile ( ) 35% to ap_invoke_handler ( ) Chapter 6: Application Layer 84

Apache Server Life Cycle n n On Unix systems, Apache creates multiple processes to handle requests. The Windows and OS/2 ports are multithreaded. . Chapter 6: Application Layer 85

The Request Processing Cycle Chapter 6: Application Layer 86

6. 5 File Transfer Protocol n n File transfer service Operation model Inside the connections Open source: wu-ftpd Chapter 6: Application Layer 87

File Transfer Service n Goal q q q n n File sharing Data replication for backup Efficiency and reliability during transmission Use client-server model based on TCP/IP Authenticated anonymous accesses Relationship between FTP and Telnet Access an ftp server with browser q q ftp: //ftp. cis. nctu. edu. tw (anony) ftp: //gis 88559@cissol 2. cis. nctu. edu. tw (authen) Chapter 6: Application Layer 88

Some Application FTP Commands Command OPEN Description Connect to a remote host CAT View a file in a remote host GET Retrieve files in a remote host RENAME RM QUIT Change the name of a file in a remote host Delete a file in a remote host Terminate an FTP session Chapter 6: Application Layer 89

Operation Model PORT: Send the IP and port of the client to which the data is retrieved Client as both control host and receiver Chapter 6: Application Layer 90

Inside the Connections n Establishing control/data connections q Active Mode n n q Passive Mode n n n Control connection initiated by client Data connection initiated by server When client is behind a firewall Both control/data connections are initiated by client FTP Reply Example FTP session Error Recovery Chapter 6: Application Layer 91

Active/Passive Mode Active mode Request (Connect to me at port P of host H) Initiate a data connection Control client server firewall Data Passive mode Request (Ask server to listen) Reply (Listening on port P of host H) Initiate a data connection client firewall Chapter 6: Application Layer server 92

Some Proper FTP Commands Command Description Type USER Send the user name Access Control PASS Send the password Access Control PORT Send the IP and port of the client to which the data is retrieved Transfer Parameter PASV Tell the server to listen on a data port rather than initiate a data connection Transfer Parameter RETR Ask server to transfer a copy of the requested file to the client File service STOR Cause the server to accept and receive the data and store it as a file File service RNFR Specify the path of a source file to rename from File service RNTO Specify the path of a destination file to rename to File service ABOR Tell the server to abort the previous command the corresponding data transfer File service Chapter 6: Application Layer 93

FTP Reply Description Type 1 yz The requested action is being initiated; expect another reply before proceeding with a new command. 2 yz The requested action has been successfully completed. Positive Complete reply 3 yz The command has been accepted, but the requested action is being held, waiting for further information from another command. Positive Intermediate reply 4 yz The command is not accepted the and the requested action did not take place. The action can be requested again. Transient Negative Completion reply 5 yz Similar with 4 yz, except that the error condition is permanent so that the action cannot be requested again. Chapter 6: Application Layer Positive Preliminary reply Permanent Negative Completion reply 94

Error Recovery n The restart mechanism q q q Sender inserts ‘marker’ (used to identify the checkpoint) in the data stream Receiver marks the position of the marker and reply the latest marker position of both sender and receiver to user When error, user issues ‘restart’ with the position of the marker to the sender * User (control host) and receiver may/may not exist in the same machine Chapter 6: Application Layer 95

Example FTP Session STATUS: > Connecting to www. cis. nctu. edu. tw (ip = 140. 113. 166. 122) Socket connected. Waiting for welcome message. . . 220 www. cis. nctu. edu. tw FTP server (Version wu-2. 6. 0(1) Mon Feb 28 10: 36 EST 2000) ready. COMMAND: > USER www 331 Password required for www. COMMAND: > PASS **** 230 User www logged in. COMMAND: > TYPE I 200 Type set to I. COMMAND: > REST 100 350 Restarting at 100. Send STORE or RETRIEVE to initiate transfer. COMMAND: > REST 0 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer. COMMAND: > pwd 257 "/home/www" is current directory. COMMAND: > TYPE A 200 Type set to A. COMMAND: > PORT 140, 113, 189, 29, 10, 27 tell the server where to connect to 200 PORT command successful. COMMAND: > LIST retrieve directory listing 150 Opening ASCII mode data connection for /bin/ls. File status okay; about to open data connection ……. list of files…. COMMAND: > TYPE I 200 Type set to I. COMMAND: > PORT 140, 113, 189, 29, 10, 31 200 PORT command successful. COMMAND: > RETR test retrieve the file “test” 150 Opening BINARY mode data connection for test (5112 bytes). Chapter 6: Application Layer 96

Open Source: wu-ftpd n Introduction to wu-ftpd q q q n Features and configurations q q q n Originally developed at Washington University Most popular ftp daemon Maintained by WU-FTPD Development Group. Virtual ftp servers On-the-fly compression Important configuration files Inside wu-ftpd Chapter 6: Application Layer 97

Inside wu-ftpd start ftp server with some options read ACL files no (under (x)inetd) stand-alone? yes service initialization fork off (parent exits) no connection accepted? reverse DNS check listen to requests parse & execute commands yes fork a handler loop until termination signal is received Chapter 6: Application Layer 98

Virtual FTP Servers 1. Manage two or more FTP servers in one machine 2. Support guest groups for different virtual FTP servers clients FTP server ftp. site 2. com. tw Configuration file of each virtual server … ftp. site 3. com. tw Lookup the ftpaccess file ftp. site 1. com. tw # Virtual Server setup for ftp. site 1. com. tw virtual ftp. site 1. com. tw root /var/ftp/virtual/site 1 A rule segment virtual ftp. site 1. com. tw banner /var/ftp/virtual/site 1/banner. msg in ftpaccess virtual ftp. site 1. com. tw logfile /var/log/ftp/virtual/site 1/xferlog # Virtual Server setup for ftp. site 2. com. tw … Chapter 6: Application Layer 99

On-the-Fly File Compressions n n Server compress a file (or directory) right before retrieved by users Example User ynlin logged in. Logged in to wwwpc. cis. nctu. edu. tw. ncftp /home/ynlin > ls 1. tar. gz Desktop/ ucd-snmp-4. 2. 1/ ncftp /home/ynlin > get ucd-snmp-4. 2. 1. tar. gz: 7393280 bytes 552. 83 k. B/s ncftp /home/ynlin >lls -l drwxr-xr-x 24 gis 88559 gis 88 3584 Oct 8 12: 18. drwxr-xr-x 88 root gis 88 2048 Sep 10 17: 48. . -rw-r----- 1 gis 88559 gis 88 7393280 Oct 8 12: 18 ucd-snmp-4. 2. 1. tar. gz Chapter 6: Application Layer 100

Important Configuration Files File name Description ftpaccess Used to configure the operations of the ftp daemon. ftpconversions Specify the postfix of a retrieved file and its corresponding operations. ftphosts Used to deny/allow some hosts to login as certain accounts. ftpservers List the virtual servers and the corresponding directories containing their own configuration files. Chapter 6: Application Layer 101

6. 6 Simple Network Management Protocol n n n Background Architectural framework MIB SMI SNMP, SNMPv 2, SNMPv 3 Open source: NET-SNMP Chapter 6: Application Layer 102

Background n n n Control over network systems are demanded Small tools: ping, traceroute, netstate. . etc (base on ICMP) SNMP: q q n Remote control without being physically attached to managed entities Exchange management information between network devices Short history q q q SNMPv 1 (1989)– Management framework SNMPv 2 (1993)– Functionality enhancement SNMPv 3 (1998)– Security add-on Chapter 6: Application Layer 103

Architectural Framework n n n Management station Agent Managed device Managed object (specified in MIB) Management protocols management station trap or response SNMP poll or request agent & managed device in one machine master agent MIB Protocols simpler than SNMP subagent (managed device ) MIB Chapter 6: Application Layer MIB 104

Management Information Base (MIB) n n n Tree-like virtual information store Identified by object identifiers Extensible in “experimental” and “private” branches q n Register at IANA (http: //www. iana. org/) MIB-II: for network management of TCP/IP-based internets (RFC 1213) Chapter 6: Application Layer 105

MIB-II iso (1) …. org (3) …. dod…. (6) …. internet (1) snmpv 2 (6) security (5) private (4) experimental (3) mgmt (2) …. directory (1) mib-2 (1) snmp (11) transmission (10) egp (8) udp (7) tcp (6) icmp (5) ip (4) at (3) interface (2) system (1) OBJECT IDENTIFIER: 1. 3. 6. 1. 2. 1. 4 Chapter 6: Application Layer 106

Object Groups in Mib-2 Group system interface Description General information about the managed system Configuration information and statistics of each physical interface at Address translation between network address and physical address ip Information of implementation and operation of IP in a local system. For example, routing table, default TTL. icmp Information about the implementation and operation of ICMP. For example, number of messages ICMP sent and received. tcp Information about the implementation and operation of TCP. For example, the number of maximum and active connections in the system. udp Information about the implementation and operation of UDP. For example, the number of datagrams sent. egp Information about the implementation and operation of EGP. tranmission snmp Related information and statistics of different transmission schemes. Information about the accesses and errors of SNMP operations. Chapter 6: Application Layer 107

Structure of Management Information (SMI) n Define the structure of a particular MIB q n Place restrictions on the types of the objects allowed in the MIB Abstract Syntax Notation one (ASN. 1) q q q Exchange of information between application components in different systems Syntax – integer, octet string, object identifier Encode the object values (by Basic Encoding Rule) n n Define the way data is represented during transmission Three categories of data types q q q Simple: Primitive ASN. 1 data types Application-wide: Special data types used in particular Applications Simply constructed: table, row Chapter 6: Application Layer 108

TCP Connection Table (in MIB-II) -- the TCP Connection table -- The TCP connection table contains information about this -- entity's existing TCP connections. ACCESS read-write STATUS mandatory DESCRIPTION "The state of this TCP connection. . " tcp. Conn. Table OBJECT-TYPE : : = { tcp. Conn. Entry 1 } SYNTAX SEQUENCE OF Tcp. Conn. Entry tcp. Conn. Local. Address OBJECT-TYPE ACCESS not-accessible SYNTAX Ip. Address STATUS mandatory ACCESS read-only DESCRIPTION STATUS mandatory "A table containing TCP connection-specific information. " DESCRIPTION : : = { tcp 13 } "The local IP address for this TCP connection. In the case of a tcp. Conn. Entry OBJECT-TYPE connection in the listen state which is willing to accept connections for any IP SYNTAX Tcp. Conn. Entry interface associated with the node, the value 0. 0 is used. " ACCESS not-accessible : : = { tcp. Conn. Entry 2 } STATUS mandatory tcp. Conn. Local. Port OBJECT-TYPE DESCRIPTION SYNTAX INTEGER (0. . 65535) "Information about a particular current TCP connection. An ACCESS read-only object of this type is transient, in that it ceases to exist when (or soon after) STATUS mandatory the connection makes the transition to the CLOSED state. " DESCRIPTION INDEX { tcp. Conn. Local. Address, "The local port number for this TCP connection. " tcp. Conn. Local. Port, : : = { tcp. Conn. Entry 3 } tcp. Conn. Rem. Address, tcp. Conn. Rem. Address OBJECT-TYPE tcp. Conn. Rem. Port } SYNTAX Ip. Address : : = { tcp. Conn. Table 1 } ACCESS read-only Tcp. Conn. Entry : : = STATUS mandatory SEQUENCE { DESCRIPTION tcp. Conn. State INTEGER, "The remote IP address for this TCP connection. " tcp. Conn. Local. Address Ip. Address, : : = { tcp. Conn. Entry 4 } tcp. Conn. Local. Port INTEGER (0. . 65535), tcp. Conn. Rem. Port OBJECT-TYPE tcp. Conn. Rem. Address Ip. Address, SYNTAX INTEGER (0. . 65535) tcp. Conn. Rem. Port INTEGER (0. . 65535) ACCESS read-only } STATUS mandatory tcp. Conn. State OBJECT-TYPE DESCRIPTION SYNTAX INTEGER { "The remote port number for this TCP connection. " closed(1), listen(2), syn. Sent(3), syn. Received(4) : : = { tcp. Conn. Entry 5 } established(5), fin. Wait 1(6), fin. Wait 2(7), close. Wait(8), last. Ack(9), closing(10), time. Wait(11), delete. TCB(12) } Chapter 6: Application Layer 109

TCP Connection Table (tabular view) tcp. Conn. Table (1. 3. 6. 1. 2. 1. 6. 13) tcp. Conn. Entry = (x. 1) tcp. Conn. State tcp. Conn. Local. Address (x. 1. 1) (x. 1. 2) x. 1 Listen 0. 0 23 0. 0 0 Listen 0. 0 161 0. 0 0 x. 1 close. Wait 127. 0. 0. 1 161 127. 0. 0. 1 1029 140. 113. 88. 164 23 140. 113. 88. 174 3082 x. 1 established tcp. Conn. Local. Port tcp. Conn. Remote. Address tcp. Conn. Remote. Port (x. 1. 3) (x. 1. 4) (x. 1. 5) INDEX Chapter 6: Application Layer 110

Basic Operations in SNMP PDU Get. Request Get. Next. Request Set. Request Get. Response Trap Get. Bulk. Request Inform. Request Descriptions Version Retrieve the value of a leaf object V 1 Get the object lexicographically next to the one specified V 1 Set (update) a leaf object with a value V 1 Response for Get. Request (value) or Set. Request (ACK) V 1 Issued by agent to notify the management station of some significant event asynchronously V 1 Retrieve large blocks of data, such as multiple rows in a table. V 2 Allows one MS to send trap information to another MS and receive a response V 2 PDU: Basic data unit in SNMP operations MS : Management Station Variable-binding list: A list of variables and corresponding values in a PDU Chapter 6: Application Layer 111

TCP Connection Table tcp. Conn. Table (lexicographical view)(1. 3. 6. 1. 2. 1. 6. 13=x) Traverse the tree using Depth First Search tcp. Conn. Entry = (x. 1) … tcp. Conn. State tcp. Conn. Local. Address tcp. Conn. Local. Port (x. 1. 1) (x. 1. 2) (x. 1. 3) 0. 0 23 (x. 1. 1. 0. 0. 23. 0. 0. 0) (x. 1. 2. 0. 0. 23. 0. 0. 0) (x. 1. 3. 0. 0. 23. 0. 0. 0) Listen 0. 0 161 (x. 1. 1. 0. 0. 161. 0. 0. 0) (x. 1. 2. 0. 0. 161. 0. 0. 0) (x. 1. 3. 0. 0. 161. 0. 0. 0) Listen close. Wait 127. 0. 0. 1 161 (x. 1. 1. 127. 0. 0. 1. 161. 127. 0. 0. 1. 1029) (x. 1. 2. 127. 0. 0. 1. 161. 127. 0. 0. 1. 1029) (x. 1. 3. 127. 0. 0. 1. 161. 127. 0. 0. 1. 1029) established 140. 113. 88. 164 23 (x. 1. 1. 140. 113. 88. 164. 23. 140. 113. 88. 174. 3082) (x. 1. 2. 140. 113. 88. 164. 23. 140. 113. 88. 174. 3082) (x. 1. 3. 140. 113. 88. 164. 23. 140. 113. 88. 174. 3082) Chapter 6: Application Layer 112

User-based Security Model (USM, RFC 2574) n Four major threats q q n Modification of Information (between agent and MS) Masquerade (pretend to be an authorized user) Disclosure (eavesdropping) Message Stream Modification (reorder, delay, replay) Solutions q q q Timeliness module (for 1 st, 3 rd and 4 th threats) Authentication protocol—use MD 5 (for the 2 nd threat) Privacy protocol—Use DES (for 3 rd) Chapter 6: Application Layer 113

View-based Access Control Model (VACM RFC 2575) n Access control q n MIB view-- a collection of MIB objects Elements in the model q q q Group– Categorization of managers security. Level– Help distinguish the access rights of a group Context– A collection of managed object accessible by an snmp entity. MIB view– A subset of a particular context Access policy– Decide the final access rights of a group to a context Chapter 6: Application Layer 114

An SNMP Entity and its Component (RFC 2571) SNMP entity SNMP engine Dispatcher Message Processing Subsystem Security Subsystem Access Control Subsystem Application(s) Command Generator Notification Receiver Proxy Forwarder Command Responder Notification Originator Other Chapter 6: Application Layer 115

Comparisons Between Three SNMP Versions Version Description and improvement SNMPv 1 1. Define the SMI (RFC 1155) 2. A more concise MIB definition (RFC 1212) 3. SNMP framework and its related operations (RFC 1157) 4. Concept of security (authentication) is proposed SNMPv 2 1. Improved SMI (support 64 -bit counter, and other types of address than IP) 2. Inclusion of ‘Get. Bulk. Request’ to improve the efficiency for retrieving large blocks of data 3. ‘Inform. Request’ for communication between management stations SNMPv 3 1. Security and administration add-on’s (RFC 2571) 2. Multi-version SNMP message processing and dispatch capability (RFC 2572) 3. The five type of applications within an SNMP engine (RFC 2573) 4. User-based security model (RFC 2574) 5. View-based access control (RFC 2575) Chapter 6: Application Layer 116

Open Source: Net-SNMP n n n Introduction to Net-SNMP Some commands for query, set and trap Extensible architecture q q q n Ways of extending the MIB Subagent protocol– Agent. X (RFC 2741) Build and include a private MIB Inside Net-SNMP q Snmpd and snmptrapd Chapter 6: Application Layer 117

Introduction n n Started at CMU, moved to UCDavis (1995) and is now based at Source. Forge (2000~) What it provides: q q n n An extensible agent SNMP library for further development Tools to get or set information from SNMP agents Tools to generate and handle SNMP traps Support SNMP V 1, V 2 and V 3 Runs on many Unix-like systems and Windows Chapter 6: Application Layer 118

Some Commands for Query, Set and Trap Name Description and example PDU used SNMPGET Retrieve the value of a leaf object using get Get. Request SNMPSET Set (update) a leaf object with a value Set. Request SNMPBULKGET Get multiple objects at a time. Possibly under different subtrees Get. Bulk. Reques t SNMPWALK Explore all the objects under a subtree of the MIB Get. Next. Reques t SNMPTRAP Uses the TRAP Request to send information to a network manager. More than one object identifiers can be applied as arguments SNMPSTATUS Used to retrieve several important statistics from a network entity. Errors will also be reported, if any SNMPNETSTAT Displays the values of various networkrelated information retrieved from a remote system using the SNMP protocol Chapter 6: Application Layer Trap 119

Command Line Examples $ snmpget -v 3 -u ynlin -l auth. No. Priv -a MD 5 -A ynlinsnmp localhost system. sys. Contact. 0 = ynlin@cis. nctu. edu. tw $ snmpset -v 3 -u ynlin -l auth. No. Priv -a MD 5 -A ynlinsnmp localhost system. sys. Contact. 0 s gis 88559 system. sys. Contact. 0 = gis 88559 $ snmpget -v 3 -u ynlin -l auth. No. Priv -a MD 5 -A ynlinsnmp localhost system. sys. Contact. 0 = gis 88559 $ /usr/local/bin/snmpbulkwalk -v 3 -u ynlin -l auth. No. Priv –a MD 5 -A ynlinpasswd localhost system. sys. Descr. 0 = Linux ynlin 2. cis. nctu. edu. tw 2. 4. 14 #5 SMP 週四 11月 22 23: 6 system. sys. Object. ID. 0 = OID: enterprises. ucdavis. ucd. Snmp. Agent. linux system. sys. Up. Time. 0 = Timeticks: (30411450) 3 days, 12: 28: 34. 50 system. sys. Contact. 0 = gis 88559 system. sys. Name. 0 = ynlin 2. cis. nctu. edu. tw system. sys. Location. 0 = ynlin 2 system. sys. ORLast. Change. 0 = Timeticks: (0) 0: 00. 00 system. sys. ORTable. sys. OREntry. sys. ORID. 1 = OID: if. MIB system. sys. ORTable. sys. OREntry. sys. ORID. 2 = OID: . iso. org. dod. internet. snmp. V 2. snmp. B system. sys. ORTable. sys. OREntry. sys. ORID. 3 = OID: tcp. MIB system. sys. ORTable. sys. OREntry. sys. ORID. 4 = OID: ip system. sys. ORTable. sys. OREntry. sys. ORID. 5 = OID: udp. MIB. . Chapter 6: Application Layer 120

Snmpd and Snmptrapd n Snmpd q q n Process snmp packets and respond with the required information Can be either master agent or subagent Basic access control with a port/addr pair Load dynamical MIB module at start Snmptrapd q q Receive and log trap messages Configured to launch an external program when receiving a trap matching a particular object identifier Chapter 6: Application Layer 121

Processing flow inside Net. SNMP Chapter 6: Application Layer 122

Data Structure of a Session Chapter 6: Application Layer 123

6. 7 Vo. IP n n n Introduction Elements in a Vo. IP environment Underlying protocols q q n n H. 323 SIP Challenges for Vo. IP technology Open source: openphone Chapter 6: Application Layer 124

Introduction n Three ways of transferring voice q q q n Circuit switched Voice over frame relay Voice over IP Pros and cons of Vo. IP compared with PSTN q q Inexpensive cost Simplicity Less bandwidth consumption Extensibility Chapter 6: Application Layer 125

Vo. IP Environment (H. 323 adopted) Gateway Router MCU H. 323 gatekeeper H. 323 terminal Telephone Packet switched network Circuit switched network Chapter 6: Application Layer 126

H. 323 Protocol Stack Control Data A / V Control control (by GK) Audio Video G. 711 Q. 931 H. 245 T. 120 G. 723 H. 263 G. 729 RTCP (H. 225. 0 ) RTP TCP RAS UDP IP Recommendation RAS(H. 225. 0) Description Stands for Registration, Admission and Status. H. 245 Control protocol for capability negotiation and channel setup. Q. 931 Used for call setup and teardown between two terminals. T. 120 Data protocols for multimedia conferencing. (application sharing, whiteboarding) RTP/RTCP Used for real time traffic synchronization and transportation. Chapter 6: Application Layer 127

Setup Procedure of an H. 323 Call Registration and admission RAS Call setup Q. 931 Terminal capability negotiation , channel setup and master-slave detection Stable call established and proceeds H. 245 RTP/RTCP Close channel H. 245 Call teardown Q. 931 Disengagement Chapter 6: Application Layer RAS 128

Session Initialization Protocol (SIP) n SIP q Targeted to replace the H. 323 n n q q n Simplicity Compatibility Characterize a multimedia session With supplementary SDP/SAP protocols HTTP-like characteristics q q q Text-based protocol Message syntax and header fields identical to HTTP/1. 1 Client-server scheme Chapter 6: Application Layer 129

Elements in an SIP Environment Redirect server Location server Internet User Agent Local Client (UAC) proxy server Remote proxy server Chapter 6: Application Layer User Agent Server (UAS) 130

Protocol Stacks of SIP Control plane Data plane SIP Multimedia Traffic SAP/SDP RTP TCP RTCP UDP IP Chapter 6: Application Layer 131

Operators and Reply Codes in SIP Operators Description INVITE Invite a user to a call ACK Confirmation for the final response BYE Terminate a call between endpoints CANCEL Terminate the search for a user or request for a call OPTIONS Features supported for a call REGISTER Register current location of the client with location server INFO Use for mid-session signaling Reply Code Description 1 xx (Informational) Trying, ringing and queued 2 xx (Successful) The request was successful 3 xx (Redirection) Give information about the receiver’s new location 4 xx (Request Failures) Failure responses from a particular server 5 xx (Server Failures) Failure responses given when a server itself has erred 6 xx (Global Failures) Busy, decline, requests not acceptable Chapter 6: Application Layer 132

Example Operation in SIP 1. UAC INVITE UAS with the SIP URL 2. Destination IP address known => Send the request directly to the destination unknown => Redirected to the local proxy server (with location server). 3. In redirection mode, send back the callee’s new location 4. In proxy mode, forward the request to the destination. 5. The UAS answers 200 (OK), then the UAC ACKs 6. Session established Chapter 6: Application Layer 133

H. 323 V. S. SIP n n n Simplicity in operation and implementation Message encoding Capability exchange Supported data types Capability of handling a conference Chapter 6: Application Layer 134

Open Source: Asterisk n An integrated PBX (Private Branch e. Xchange) system Chapter 6: Application Layer 135

Block Diagram n Asterisk provides a framework to build a customized Vo. IP system. q Flexible to add/remove modules to establish the basic transport service Chapter 6: Application Layer 136

Data Structures n Two concepts, context and extension, are introduced Chapter 6: Application Layer 137

Call flow Chapter 6: Application Layer 138

Chapter 6: Application Layer 139

6. 8 Streaming n Introduction to streaming q q n Architecture and components for streaming q q q n Availability Short fetching time Avoid storage overhead Live broadcasting Compression scheme Protocol stack for streaming Qo. S and synchronization mechanism Open source: Darwin streaming server Chapter 6: Application Layer 140

Architecture and Components for Streaming Internet Server Client Transport Protocols Video/Audio raw data Transport Protocols Qo. S Control Video Files Qo. S Control Audio Files synchronization Video Decoder Chapter 6: Application Layer Audio Decoder 141

Compression Schemes n Temporal or spatial q q n Lossless or lossy q q n Temporal – describe changes between frames Spatial – describe similar patterns in a frame Recoverability when decompressed GIF (lossless) and JPG (lossy) Symmetrical or Asymmetrical q q The time for compression and decompression According to the load of the server Chapter 6: Application Layer 142

Protocol Stacks of Streaming n RTSP (Real Time Streaming Protocol) q q VCR-Style remote control functionalities Establish and control streams n n q n Description file before a session Methods in a session (SETUP, PLAY, TEARDOWN) Reuse of HTTP concepts (Ex: ASCII messages) HTTP q q Mostly by buffering Low performance Control plane HTTP RTSP RTCP Data plane RTP UDP IP Chapter 6: Application Layer 143

Qo. S and Synchronization Mechanism n Qo. S mechanism q q n Source-based rate control Receiver-based rate control Synchronization mechanism q Intra-stream synchronization n q Inter-stream synchronization n q Continuity of audio/video data frames inside a stream Consistency between cooperative streams Inter-object synchronization n Consistency between streams and other objects like text and still images Chapter 6: Application Layer 144

Open Source: Darwin Streaming Server n n n Open source version of Apple's Quick. Time Streaming Server (QTSS) Deliver streaming media with RTP and RTSP Provide both live and on-demand programs Supports many formats: H. 264/MPEG-4 AVC, MPEG-4 Part 2, 3 GP and MP 3 Provide Web-based administration, authentication, relay support, and integrated broadcaster administration, etc. Chapter 6: Application Layer 145

Block Diagram n Two parts: core server and modules Core server provides task scheduling q Module provides specific functions q Chapter 6: Application Layer 146

Data Structures Chapter 6: Application Layer 147

Call Flow: Task Handling Chapter 6: Application Layer 148

RTSP Handling State Transition Diagram Chapter 6: Application Layer 149

6. 9 Peer-to-Peer Applications (P 2 P) n Introduction n n P 2 P Architectures Performance Issues of P 2 P Applications Case Study: Bit. Torrent Open source: Vuze (Bit. Torrent Client) Chapter 6: Application Layer 150

Introduction to P 2 P n n A distributed network architecture in which participants act as both a client and a server Participants construct a virtual overlay network at the application layer on top of the underlying IP network Node: peers Edge: TCP/UDP connection Chapter 6: Application Layer 151

Introduction to P 2 P n Operations in P 2 P systems q q Joining the P 2 P overlay network, Resource discovery n q The most challenging problem for P 2 P applications Resource retrieval Chapter 6: Application Layer 152

P 2 P Architectures n Way of forming a P 2 P overlay network could be classified into three categories q q q Centralized Decentralized and unstructured Decentralized but structured Chapter 6: Application Layer 153

Centralized P 2 P n Operation overview Chapter 6: Application Layer 154

Centralized P 2 P n Utilizes a central directory server for locating objects in the P 2 P network q q q Peers join by registering to the directory server Peers inform the directory server of objects to be shared A peer sends query to the directory server to search an object The peer receives reply from the directory server The peer selects one or more peer in the reply to download the object directly from Example: Napster Chapter 6: Application Layer 155

Centralized P 2 P n Advantages q q q n Simple Easy to implement Support various kinds of search such as keyword, full-text, and metadata search Disadvantages q Not a true P 2 P system as it relies on a central directory server which becomes performance bottleneck, single point of failure, not scalable and vulnerable to Do. S attacks Chapter 6: Application Layer 156

Decentralized and Unstructured P 2 P n To get rid of server, this approach floods query messages to peers to search for shared objects q q Limited-scope flooding is adopted to reduce flooding messages A query hit message is returned along the reverse path back to the inquirer Example: Gnutella Chapter 6: Application Layer 157

Decentralized and Unstructured P 2 P n Join procedure q q A peer needs to know at least one of the peers already on the overlay network. The peer sends a join message to a peer already on the overlay. The existing peer then replies its identity as well as a list of its neighbors. It may also forward the join message to its neighbors. Upon receiving join reply messages, the newcomer knows more peers on the overlay. Chapter 6: Application Layer 158

Decentralized and Unstructured P 2 P n Hierarchical overlay with super peers q q Flooding is apparently not scalable Fast. Track adopts a hierarchical overlay A super peer acts as a local directory database which stores the indexes of objects shared by ordinary peers Two-level hierarchical overlay n n The lower level adopts the central server approach The upper level (super peers) adopts the decentralized and unstructured approach. Chapter 6: Application Layer 159

Decentralized and Unstructured P 2 P n Advantages q q q n Fully distributed Reliable, fault-tolerant No single point of failure Disadvantages Excessive query traffic make it not scalable q May fail to find content that is actually in the system q Super peer may become overloaded or been attacked q Chapter 6: Application Layer 160

Decentralized but Structured n n Combine the distributed directory service with an efficient query routing scheme Key ideas q q For distributed directory service, a hash function maps peers and objects into the same address space so that objects can be deterministically assigned to peers in a distributive manner. For efficient query routing, peers are organized into a structured overlay based on their positions in the address space. Chapter 6: Application Layer 161

Decentralized but Structured n Operations overview q q Each peer generates its own node ID by a predefined hash function. For each object being held and to be shared, the peer generates the object ID by the same or another hash function. For each object, the peer sendd a register message to the node that has the node ID same as the object’s ID. If a peer wants to query an object, it uses the hash function to generate the object ID and sends the query message to the node that hosts the object’s ID. Chapter 6: Application Layer 162

Decentralized but Structured n Message routing (use Chord as an example) q q q Key idea: have each peer maintain a specially designed routing table such that every peer could forward the arriving message to a neighboring peer with node ID that is further closer to the destination. Consider a 10 -node Chord overlay in a 6 -bit address space Chord views its address space as a onedimensional circular space such that peers in the space form a ring overlay. Chapter 6: Application Layer 163

Message Routing in Chord n n The routing table in Chord is called a finger table. For an m-bit address space, the finger table of a node with ID=x consists of at most m entries and the i-th entry points to the first node with ID following the ID of x+2 i-1 modulo 2 m, for 1≤i≤m. Chapter 6: Application Layer 164

Finger Table of Chord n Finger table of node N 8, where m =6. Chapter 6: Application Layer 165

Routing a Query Message n Routing a query message for object 54 from N 8 Chapter 6: Application Layer 166

Performance Issues of P 2 P Applications n n n n Free Riding Flash Crowd Topology Awareness NAT Traversal Churn Security Copyright Infringement Chapter Copyright 6: Application Reserved 2010 Layer 167

Free Riding n Scalability of P 2 P systems relies on the contribution from peers q q n A common solution is to implement some incentive mechanisms. q n If a peer only consumes but contributes little or no resources, it becomes a free rider of the system. 85% of peers share no files in Gnutella in 2005 For example, tit-for-tat in Bit. Torrent. Other solutions: reward-based and creditbased mechanisms Chapter 6: Application Layer 168

Flash Crowd n A sudden, unanticipated growth in the demand of a particular object q n Issues q q n e. g. , a new release of a DVD video or mp 3 file how to deal with a sudden large amount of query messages how long to find and download the object within a short time period Solutions q Cache, duplicating popular objects Chapter 6: Application Layer 169

Topology Awareness n A virtual link in a P 2 P overlay could be a long end-to-end connection across continents or a short one within a local area network q n How to avoid serious topology mismatch Solutions q Many route-proximity and neighbor-proximity enhancements for P 2 P overlay systems have been proposed based on RTT measurement, preference of routing domain or ISP, or geographical information. Chapter 6: Application Layer 170

NAT Traversal n Basic requirement for a P 2 P system is to provide peers with NAT traversal mechanisms q n If both peers are behind NAT devices, they cannot connect to each other without help from other peers or STUN servers Solutions q In most cases, NAT traversal is solved by relay peers or super peers that have public IP addresses Chapter 6: Application Layer 171

Churn n Churn refers to the phenomenon that peers dynamically join and leave the system at will. q q n high churn rate seriously affects the stability and scalability of a P 2 P system. e. g. , a high churn rate may cause a tremendous overlay maintenance overhead and dramatic routing performance degradation in DHT-based system Solutions q q Avoid a rigid structure or relation among peers Peers maintain a list of potential neighbors for quick and dynamic neighbor replacement Chapter 6: Application Layer 172

Security n Security issues q n P 2 P programs with back hole (Trojan Horse), spurious content, leaking of files not to be shared. Solutions to content pollution q protect the content with message digest such as MD 5 n q q In Bit. Torrent, the MD 5 digest of each piece of a shared file is stored in the metadata file peer reputation system object reputation system Chapter 6: Application Layer 173

Copyright Infringement n n Sharing copyrighted objects through P 2 P systems is a serious problem which hinders the promotion and usage of P 2 P systems. Not only P 2 P users are responsible for copyright infringement, so are the companies that host P 2 P applications q Especially in the case where P 2 P systems will not be able to exist without their serves, e. g. , Napster. Chapter 6: Application Layer 174

Case Study: Bit. Torrent n n Bit. Torrent (BT) is originally designed by Bram Cohen in 2001 Well thought-out protocol with several unique features q q q use tit-for-tat as an incentive mechanism to cope with free riders use out-of-band search to avoid copyright infringement issue use pull-based swarming for load balancing use hash check to prevent propagation of spurious pieces after a peer has successfully downloaded a file, it becomes a seeder to distribute the file Chapter Copyright 6: Application Reserved 2010 Layer 175

BT Terminologies n n n A file is cut into pieces of a fixed size. A piece is further divided into chunks, the basic data unit for a peer to request for content. The integrity of a piece is protected by a SHA-1 A peer becomes a seeder if it has successfully downloaded the file. A tracker for each file or group of files to be shared. The tracker tracks the downloading peers and seeds, and coordinates the file distribution among peers. Chapter 6: Application Layer 176

BT Operation Overview Chapter 6: Application Layer 177

Piece Selection n Random first piece selection q n Rarest first policy q n For the first few pieces, the client just randomly selects a piece to download. Selects the most scarce piece to download first End-game mode q To speed up the completion of a file download at the end, a peer with only a few pieces missing will send requests for all missing pieces to all the peers Chapter 6: Application Layer 178

Peer Selection n Choking/unchoking q q q n Optimistic unchoking q q n Choking refers to a temporal refusal to upload to a peer. At the beginning, all peers are chocked Tit-for-tat algorithm selects a fixed number of peers from which the peer downloaded most to unchoke new peer needs to move its first step when initially joined the system select one peer at random Anti-snubbing q If a peer is choked by all of its peers (snubbed), it is better to run optimistic unchoking more often to explore more peers that are willing to cooperate. Chapter 6: Application Layer 179

Open source: Vuze (Bit. Torrent Client) n n Many BT client programs are open-source Some popular client programs q q q Vuze u. Torrent Bit. Comet Chapter Copyright 6: Application Reserved 2010 Layer 180

Core Packages n Most of Vuze’s core packages are located under the. comaelitisazureuscore directory Chapter 6: Application Layer 181

Data Structure n Classes for the peer and piece objects Chapter 6: Application Layer 182

Algorithm Implementation n Main program for controlling the piece and peer selection is the PEPeer. Control. Impl class Chapter 6: Application Layer 183

Implementation of Peer and Piece Selection n Unchoking algorithms are implemented in Downloading. Unchocker. java and Seeding. Unchocker. java q q n tit-for-tat is implemented in calculate. Unchokes() Optimistic unchoking is implemented in Unchoker. Util. get. Next. Optimistic. Peer() get. Request. Candidate() defined in Piece. Picker. Impl. java is the core method for deciding which block to download Chapter 6: Application Layer 184

6. 10 Summary n General issues q n Major application protocols q n DNS, SMTP, POP 3, IMAP 4, HTTP, FTP, SNMP Real time, multimedia application protocols q n well-known ports, concurrent connection-oriented servers and iterative connectionless servers, statefulness/statelessness SIP, RTCP, RTSP P 2 P protocols q Napster, Gnutella, DHT, Chord, BT Chapter 6: Application Layer 185