Mail Services Presurvey How many serverclient programs or

Mail Services

Pre-survey: How many server/client programs (or services) are needed to send and receive email? A. B. C. D. E. 0 1 2 3 4 or more

Mail Services o Three major mail service protocols: n Simple Mail Transfer Protocol o n Post Office Protocol o n SMTP POP or POP 3 Internet Mail Access Protocol o IMAP or IMAP 4

Which one to use? o Outbound Mail n SMTP for outbound email o n o Port 25 or 2525 Server/repository Inbound Mail (clients) n POP 3 for inbound email o n Port 110 IMAP for inbound email o Port 143

What is SMTP? o Simple Mail Transfer Protocol (SMTP) is the standard protocol for sending emails across the Internet n n SMTP uses TCP port 25 or 2525 Sometimes you may have problems sending messages o o ISP may have closed port 25 To determine the SMTP server for a given domain name n MX (Mail e. Xchange) DNS record is used

IMAP or POP 3? o o Both protocols receive email Main differences between IMAP and POP 3: n POP 3 protocol assumes there is only one client (computer terminal) that ever connects to the mailbox o o n Connect Get/send mail Disconnect Read and compose email while offline IMAP protocol allows simultaneous access by multiple clients (computer terminals) o o Allows the same user on the clients IMAP is also suitable if the mailbox is used by multiple users

Material from: http: //en. wikipedia. org/wiki/Smtp SMTP

SMTP o Simple Mail Transfer Protocol n n The de facto standard for e-mail transmissions across the Internet Defined in RFC 821 (STD 10) o o amended by RFC 1123 (STD 3) chapter 5. Protocol used today is usually ESMTP n n Extended SMTP Defined in RFC 2821

SMTP o Relatively simple text-based protocol n One or more recipients of a message are specified o n o Most cases verified to exist The message text is transferred Client-server protocol n n The client transmits an email message to the server Either o o an end-user's email client, a. k. a. MUA (Mail User Agent), - or a relaying server's MTA (Mail Transfer Agents) n Which then, in turn, acts as an SMTP client

SMTP o o An email client knows the outgoing mail SMTP server from its configuration A relaying server typically determines which SMTP server to connect to by looking up the MX (Mail e. Xchange) DNS record for each recipient's domain name n o Conformant MTAs (not all) fall back to a simple A record in the case of no MX n o A Record: address record Some current mail transfer agents will also use SRV records n n n o The part of the email address to the right of the at (@) sign SRV Records: Se. RVice records More general form of MX These are not widely adopted Relaying servers can also be configured to use a smart host

SMTP o The SMTP client initiates a TCP connection n Server's port 25 o o Unless overridden by configuration Easy to test an SMTP server using telnet n see later example

SMTP o SMTP is a "push" protocol n Does not allow one to "pull" messages from a remote server on demand o o o It only sends to the next or end destination To “pull” (i. e. receive) a mail client must use POP 3 or IMAP Another SMTP server can trigger a delivery in SMTP n Using ETRN

Outgoing mail SMTP server o o Email client requires the name or IP address of an SMTP server as part of its configuration Server will deliver messages on behalf of the user Setting allows for various policies and network designs End users connected to the Internet can use the services of an e-mail provider that is not necessarily the same as their connection provider

Outgoing mail SMTP server o o Network topology, or the location of a client within a network or outside of a network, is no longer a limiting factor for email submission or delivery Modern SMTP servers typically use a client's credentials (authentication) rather than a client's location (IP address), to determine whether it is eligible to relay email

Outgoing mail SMTP server o Can use either TCP port 25 (SMTP) or port 587 (Submission) for relaying outbound mail to a mail server n n o Established by RFC 2476 Many servers support both Some servers still support 465 for legacy secure SMTP n Preferable to use encryption on standard ports according to RFC 2487

Outgoing mail SMTP server o Some servers are set to reject all relaying on port 25 n o Valid users authenticating on port 587 are allowed to relay mail to any valid address. Open Relay: n n Server that relays all email for all destinations for all clients connecting to port 25 Generally considered a bad practice worthy of blacklisting

Sample communication of a legal SMPT Session o After establishing a connection between the sender (the client) and the receiver (the server) n In the following conversation o o n everything sent by the client is prefaced with C: everything sent by the server is prefaced with S: On most computer systems, a connection can be tested using the telnet command on the client machine For example: o telnet www. example. com 25 n Opens a TCP connection from the sending machine to the MTA listening on port 25 on host www. example. com

Sample communication (after connect to server) S: C: S: C: C: S: C: S: 220 www. example. com ESMTP Postfix HELO mydomain. com 250 Hello mydomain. com MAIL FROM: <sender@mydomain. com> 250 Ok RCPT TO: <friend@example. com> 250 Ok DATA 354 End data with <CR><LF> Subject: test message From: sender@mydomain. com To: friend@example. com Hello, This is a test. Goodbye. . 250 Ok: queued as 12345 QUIT 221 Bye

Sample communications o Note: the data the client sends in the HELO and MAIL FROM commands can be retrieved in additional headers that the server adds to the message: n o Received and Return-Path respectively. Optional: (and not shown above) nearly all clients ask the server which SMTP extensions the server supports n n Uses the EHLO greeting to invoke Extended SMTP (ESMTP) These clients use HELO only if the server does not respond to EHLO

Sample communications o o Contemporary clients will use the ESMTP extension keyword SIZE to inquire of the server the maximum message size that will be accepted Older clients and servers n Try to transfer huge messages that will be rejected o o After wasting the network resources Including a lot of connect time to dialup ISPs n paid by the minute

Resume 3/15

Sample communications o Users can determine the maximum size accepted by ESMTP servers n n For the sending of giant files or sending with older clients The user telnets as above, but substitutes "EHLO mydomain. com" for the HELO command line: o o o o S: S: C: S: S: 220 -serverdomain. com ESMTP {postfix version and date} 220 NO UCE. {etc. , terms of service} EHLO mydomain. com 250 -serverdomain. com Hello mydomain. com [127. 0. 0. 1] 250 -SIZE 14680064 250 -PIPELINING 250 HELP Here, serverdomain. com declares that it: n Will accept a fixed maximum message size no larger than 14, 680, 064 octets o n n Depending on the server's actual resource usage, it may be currently unable to accept a message this large Can pipeline Can do help

Sample communications o o In the simplest case, an ESMTP server will declare a maximum SIZE with only the EHLO user interaction. If no number appears after the SIZE keyword, or if the current message limit must be exactly determined: n n User can further interact by simulating the ESMTP header of a message with an estimated size. See External Link RFC 1870

Material From: http: //en. wikipedia. org/wiki/Post_Office_Protocol POP

POP o Local e-mail clients use the Post Office Protocol version 3 (POP 3) n n o An application-layer Internet standard protocol Retrieves e-mail from a remote server over a TCP/IP connection Many subscribers to individual Internet service provider e-mail accounts access their e-mail with client software that uses POP 3

POP Overview o POP 3 has made earlier versions of the protocol obsolete n n POP (formally referred to POP 1 and POP 2) Now, the term POP almost always means POP 3 in the context of e-mail protocols

POP Overview o Design of POP 3 and its procedures supports endusers with intermittent connections n n n o (e. g. dial-up connection) Allows users to retrieve e-mail when connected View and manipulate the retrieved messages without needing to stay connected Although most clients have an option to leave mail on server, e-mail clients using POP 3 generally: n n n Connect Send all new messages composed on the client Retrieve all messages stored on the mail server o o n Save the messages on the user's PC as new unread messages Delete messages from the server Disconnect

POP Overview o Clients with a leave mail on server option generally use POP 3 UIDL command n n Unique IDentification Listing Most POP 3 commands identify specific messages by their ordinal number on the mail server Creates a problem for a client intending to leave messages on the server Message numbers may change from one connection to the server to another o For example if a mailbox contains five messages at last connect, and a different client then deletes message #3, the next connecting user will find the last two messages' numbers decremented by one

POP Overview o UIDL provides a mechanism to avoid these numbering issues n n Server assigns a string of characters as a permanent and unique ID for the message When a POP 3 -compatible e-mail client connects to the server o n Can use the UIDL command to get the current mapping from these message IDs to the ordinal message numbers Client can then use this mapping to determine which messages it has yet to download o o Saves time when downloading Note: IMAP has a similar mechanism, using a 32 -bit UID (Unique IDentifier) that is required to be strictly ascending.

POP Overview o Advantage of the numeric UID is with large mailboxes n o A client can request just the UIDs greater than its previously stored "highest UID" In POP, the client must fetch the entire UIDL map

POP Overview o Whether POP 3 or IMAP is used to retrieve messages: n n n E-mail clients typically use the SMTP_Submit profile of the SMTP protocol to send messages E-mail clients are commonly categorized as either POP or IMAP clients, but in both cases the clients also use SMTP There are extensions to POP 3 that allow some clients to transmit outbound mail via POP 3 o Known as "XTND XMIT" extensions. o The Qualcomm qpopper and Communi. Gate Pro servers and Eudora clients are examples of systems that optionally utilize the XTND XMIT methods of authenticated client-to-server e-mail transmission.

POP Overview o MIME serves as the standard for attachments and non-ASCII text in e-mail n Although neither POP 3 nor SMTP require MIMEformatted e-mail, essentially all Internet e-mail comes MIME-formatted o o POP clients must also understand use MIME IMAP, by design, assumes MIME-formatted e-mail

POP Overview o POP 3 originally supported only an unencrypted login mechanism n Plain text transmission of passwords in POP 3 still commonly occurs o POP 3 currently supports several authentication methods to provide varying levels of protection against illegitimate access to a user's e-mail o APOP: Authenticated POP n n o n Uses the MD 5 hash function Attempt to avoid replay attacks and disclosure of the shared secret Clients implementing APOP include Mozilla Thunderbird, Opera, Eudora, KMail and Novell Evolution POP 3 clients can also support SASL authentication methods via the AUTH extension. MIT Project Athena also produced a Kerberized version

POP Overview o POP 3 works over a TCP/IP connection n n Uses TCP on network port 110 E-mail clients can encrypt POP 3 traffic using TLS or SSL A TLS or SSL connection is negotiated using the STLS command Some clients and servers, like Google Gmail, instead use the deprecated alternate-port method, which uses TCP port 995

POP Example (APOP) S: <wait for connection on TCP port 110> C: <open connection> S: +OK POP 3 server ready <1896. 697170952@dbc. mtview. ca. us> C: APOP mrose c 4 c 9334 bac 560 ecc 979 e 58001 b 3 e 22 fb S: +OK mrose's maildrop has 2 messages (320 octets) C: STAT S: +OK 2 320 C: LIST S: +OK 2 messages (320 octets) S: 1 120 S: 2 200 S: . C: RETR 1 S: +OK 120 octets S: <the POP 3 server sends message 1> S: . C: DELE 1 S: +OK message 1 deleted C: RETR 2 S: +OK 200 octets S: <the POP 3 server sends message 2> S: . C: DELE 2 S: +OK message 2 deleted C: QUIT S: +OK dewey POP 3 server signing off (maildrop empty) C: <close connection> S: <wait for next connection>

POP Example (no auth) o POP 3 servers without the optional APOP command expect you to log in with the USER and PASS commands: C: USER mrose S: +OK User accepted C: PASS mrosepass S: +OK Pass accepted

POP 3 1. 2. 3. 4. Uses port 25 Leaves email on the server by default Allows the client to disconnect from the server, then work with the individual emails Is more flexible than IMAP

IMAP

IMAP o Internet Message Access Protocol n n Commonly known as IMAP or IMAP 4 An application layer Internet protocol o o n o Previously called Internet Mail Access Protocol, Interactive Mail Access Protocol , and Interim Mail Access Protocol Current version is IMAP version 4 revision 1 n o Operates on port 143 Allows a local client to access e-mail on a remote server IMAP 4 rev 1: defined by RFC 3501 IMAP 4 and POP 3 are the two most prevalent Internet standard protocols for e-mail retrieval n Virtually all modern e-mail clients and servers support both

IMAP o IMAP supports both connected and disconnected modes of operation n E-mail clients using IMAP generally leave messages on the server until the user explicitly deletes them o o o IMAP is often used in large networks n n o Allows multiple clients to access the same mailbox Clients in this case are different workstations For example, a college campus mail system Few Internet Service Providers (ISPs) support IMAP 4 offers access to the mail “store” n Client may store local copies of the messages o o Considered to be a temporary cache Server's store is authoritative

IMAP o Proprietary protocols: n Microsoft Outlook client o n IBM Notes client o n o When communicating with an Exchange server When communicating with a Domino server Both products also support SMTP, POP 3, and IMAP 4. Support for the Internet standard protocols n n n Allows other e-mail clients to access these servers Allows the clients to be used with other servers E. g. Qualcomm's Eudora or Mozilla Thunderbird

IMAP o IMAP allows users to access new messages instantly on their computers n o The mail is stored on the network With POP 3, users either n n download the e-mail to their computer access it via the web Both methods take longer than IMAP User must either download any new mail or "refresh" the page to see the new messages

POP – IMAP COMPARISONS

Overview o Most e-mail clients support either POP 3 or IMAP to retrieve messages n o Few Internet Service Providers (ISPs) support IMAP Fundamental differences between POP 3 and IMAP 4: n POP 3 offers access to a mail drop o o Mail starts on the server Downloaded to client when accessed n o If the client leaves some or all messages on the server n n Deleted from server The client's message store is considered authoritative IMAP 4 offers access to a mail store o o Mail stays on the server after accessed Client may store local copies of the messages n n These are considered to be a temporary cache The server's store is authoritative

IMAP Advantages over POP 3 o Connected and disconnected modes of operation n When using POP 3, clients typically connect to the e-mail server briefly o n When using IMAP 4, clients often stay connected as long as the user interface is active o n Only as long as it takes to download new messages Download message content on demand For users with many or large messages o IMAP 4 usage pattern can result in faster response times.

IMAP Advantages over POP 3 o Multiple clients simultaneously connected to the same mailbox n n The POP 3 protocol requires the currently connected client to be the only client connected to the mailbox. The IMAP protocol specifically allows simultaneous access by multiple clients o Provides mechanisms for clients to detect changes made to the mailbox by other, concurrently connected, clients

IMAP Advantages over POP 3 o Access to MIME message parts and partial fetch n Nearly all internet e-mail is transmitted in MIME format o Allows messages to have a tree structure where n n n The IMAP 4 protocol allows clients to o o n leaf nodes are any of a variety of single part content types non-leaf nodes are any of a variety of multipart types Separately retrieve any of the individual MIME parts Retrieve portions of either individual parts or the entire message Mechanisms allow clients to retrieve the text portion of a message without retrieving attached files or to stream content as it is being fetched

IMAP Advantages over POP 3 o Message state information n flags can keep track of message state o n Flags are stored on the server o n For example: whether or not the message has been read, replied to, or deleted Different clients accessing the same mailbox at different times can detect state changes made by other clients POP 3 provides no mechanism for clients to store such state information on the server o If a single user accesses a mailbox with two different POP 3 clients state information cannot be synchronized between the clients n n IMAP 4 protocol supports both pre-defined system flags and client defined keywords o n System flags indicate state information such as whether a message has been read Keywords allow messages to be given one or more tags whose meaning is up to the client o n E. g. whether a message has been accessed (not supported by all IMAP servers) Adding user created tags to messages is an operation supported by some Webmail services, such as Gmail o Generally not using IMAP

IMAP Advantages over POP 3 o Multiple mailboxes on the server n IMAP 4 clients can create, rename, and/or delete mailboxes on the server, and move messages between mailboxes o n o Multiple mailbox support also allows servers to provide access to shared and public folders Server-side searches n n o Usually presented to the user as folders IMAP 4 provides a mechanism for a client to ask the server to search for messages meeting a variety of criteria Avoids requiring clients to download every message in the mailbox in order to perform searches Built-in extension mechanism n n IMAP 4 defines an explicit mechanism by which it may be extended Many extensions to the base protocol have been proposed and are in common use o o IMAP 2 did not have an extension mechanism POP 3 now has one defined by RFC 2449

IMAP Disadvantages v. POP o IMAP remedies many of the shortcomings of POP n o inherently additional complexity Much of this complexity is compensated for by server-side workarounds n n n e. g. , multiple clients accessing the same mailbox at the same time Maildir Database backends

Disadvantages of IMAP o If the mail store and searching algorithms on the server are not carefully implemented: n Client can potentially consume large amounts of server resources when searching massive mailboxes

Disadvantages of IMAP o IMAP 4 clients need to explicitly request new email message content n o o Potentially causing additional delays on slow connections such as those commonly used by mobile devices. A private proposal, push IMAP, would extend IMAP to implement push e-mail by sending the entire message instead of just a notification However, push IMAP has not been generally accepted n Current IETF work has addressed the problem in other ways

Disadvantages of IMAP o Unlike some proprietary protocols which combine sending and retrieval operations, sending a message and saving a copy in a server-side folder with a base-level IMAP client requires transmitting the message content twice, once to SMTP for delivery and a second time to IMAP to store in a sent mail folder n n Remedied by a set of extensions defined by the IETF LEMONADE Working Group for mobile devices POP 3 servers don't support server-side folders so clients have no choice but to store sent items on the client Many IMAP clients can be configured to store sent mail in a clientside folder Courier Mail Server offers a non-standard method of sending using IMAP by copying an outgoing message to a dedicated outbox folder

IMAP 1. 2. 3. 4. Uses port 25 Allows a client to monitor email from multiple workstations at the same time By default, uses a lot of the client’s local disk storage to hold email Is the best email client service since it is a newer protocol

POP Overview o In contrast, the newer, more capable Internet Message Access Protocol (IMAP) supports both connected and disconnected modes of operation. n o E-mail clients using IMAP generally leave messages on the server until the user explicitly deletes them This and other facets of IMAP operation allow multiple clients to access the same mailbox

Sidebar o The mail on the server is kept in files in a directory until read and/or deleted n POP: until “read” o n E. g. until downloaded to client IMAP: until deleted

SMTP Authentication SMTP-AUTH

SMTP Authentication o SMTP-AUTH n Extension of the Simple Mail Transfer Protocol (SMTP) o n o o Includes an authentication step Client effectively logs in to the mail server during the process of sending mail Servers which support SMTP-AUTH can usually be configured to require clients to use this extension, ensuring the true identity of the sender is known. SMTP-AUTH is defined in RFC 2554

SMTP Authentication o SMTP-AUTH provides an access control mechanism n n Can be used to allow legitimate users to relay mail while denying relay service to unauthorized users, such as spammers Does not guarantee the authenticity of either the SMTP envelope sender or the "From: " header o Spoofing is possible even with SMTP-AUTH n (When one sender masquerades as someone else)

SMTP Authentication o SMTP-AUTH extension also allows one mail server to indicate to another that the sender has been authenticated when relaying mail n This requires the recipient server to trust the sending server o n This aspect of SMTP-AUTH is rarely used on the Internet The recipient of an e-mail message cannot tell whether the sender was authenticated o Use of SMTP-AUTH is only a partial solution to spam

SMTP Authentication o While SMTP-AUTH is generally a security improvement over unauthenticated SMTP, it can also introduce a weakness n If authenticated users are allowed to submit messages from IP addresses where unauthenticated users are not — that is, if authenticated users are allowed to relay mail — then an attacker who subverts one user's account is then able to use the authenticated server as an open mail relay o n n In such a configuration every user's password becomes a key to the mail system's security. Spammers have attacked SMTP-AUTH mail servers by brute forcing common usernames and passwords A good password policy can effectively prevent such an attack

Pop quiz: How many server/client protocols (or services) does an IT pro need to send and read email? A. B. C. D. 0 1 2 3

Acronyms

Acronym summary o o o o o ESMTP - Extended SMTP ETRN - Extended Turn IMAP - Internet Mail Access Protocol MTA - Mail Transfer Agents MUA - Mail User Agent MX - Mail e. Xchange POP - Post Office Protocol SMTP - Simple Mail Transfer Protocol UIDL - Unique IDentification Listing