Control Spam by the Use of Greylisting Torgny

Control Spam by the Use of Greylisting Torgny Hallenmark LDC - Computing Center Lund University, Sweden Torgny. Hallenmark@ldc. lu. se TERENA Networking Conference 2005 [] 2005 -06 -07

Lund University, Sweden • Located in the very south of Sweden • One of the biggest universities in Scandinavia • Has almost all faculties • 40 000 students • 6 000 employees • www. lu. se for more info []

Lund University LU – employees and students • Parallel mail servers used • Same software used mainly • Sun Java Enterprise System (JES) • Sun JES Messaging Server • Sun JES Directory Server (LDAP) • Sendmail used in mail gateway • Controlling systems: LUCAT and LADOK []

Mail systems LUCAT LADOK LDAP employees students mail server Cassandra [] Piraten

Mail systems New mail control structure • Central mail gateway for spam and virus detection • 3 -4 parallel Sun. Fire V 240 servers • In production since July 2004 • Spam detection: Spam. Assassin, Greylisting • Virus detection: Sophos, Clam. AV • Address verification: Only messages to valid addresses on our domains are accepted []

Mail systems Spam and virus detection Argus 1 -3 mail. lu. se mail server local mail server [] POP IMAP webmail

Mail systems Mail servers -- central / local • Central mail server with services for POP, IMAP and webmail • 75% of employees are using the central mail service • Local mail servers exist in some departments • Local mail servers can use the central mail gateway for spam and virus detection []

E-mail: Virus detection in mail • Software: Sophos and Clam. AV • Flagging in Subject: ***VIRUS*** • Virus infected attachments are removed, info text is inserted • Also possible to check for ”bad” file types []

E-mail: Spam detection in mail • Software: Spam. Assassin • Spam checks made, giving spam points • Also RBL blacklists may give spam points • Flagging in header: X-Spam-Flag • Flagging in Subject: ***SPAM*** • No messages are trashed centrally, only flagging is used (our policy) • User must set up filter rules in his mail program []

Spam control: Greylisting added • Set in production on 1 July 2004 • Immediate impact ! • Spam is no longer a problem !!! • 90 -95% of earlier spam is just gone ! • Spam messages are not received, means less messages to check (for both spam and virus) • No decision to take (if you would like to thrash messages) []

Spam control: Greylisting effect: Messages counted in a user mailbox []

Spam control: Greylisting effect: (Graph from Umeå University) []

Spam control: Greylisting Some user reactions: • This is fantastic! Now you can again use e-mail like in the old days! • The spam is gone! How did you do this? Lots of thanks! • Really magic! Earlier I got 200 spam messages a day, now I see at most two! Thank you for an excellent work! • It is almost sad with so few messages in my inbox … • There must be something wrong with the mail system, I hardly get any mail at all … []

Spam control: Greylisting technique • • Evan Harris: The Next Step in the Spam Control War http: //projects. puremagic. com/greylisting/ Using Internet SMTP standard (RFC 821) The trick is following SMTP status: 451 4. 7. 1 Temp. Fail – Please try again later • Status code 451 can be handled by ”real” SMTP mail servers (put message in queue, try to resend it later) • Spam spreading programs can not treat 451 status info … (not yet …) []

Spam control: Greylisting technique • For incoming messages following ”triplet” is checked: 1) IP address for sending SMTP server 2) Sender address (envelope sender) 3) Receiver address (envelope recipient) • • [] If this triplet has not been seen earlier: Send SMTP status 451, Temp. Fail If this triplet has been seen at least 5 minutes ago: Accept the incoming message

Spam control: Greylisting technique • A data base is needed (My. SQL): § § § [] Time that triplet was first seen Time that triplet blocking will expire Time that triplet record itself will expire Number of blocked delivery attempts Number of messages sucessfully passed Some other data

Spam control: Greylisting technique • Some configuration parameters: § Unknown triplet, initial delay (default: 1 hour, our value: 5 minutes) § Lifetime of new triplets that have not yet allowed a mail to pass (default: 5 hours, our value: 30 hours) § Lifetime of auto-whitelisted triplets that have allowed mail to pass (default: 36 days) []

Spam control: Greylisting – not always • Manual whitelisting possible: § Can be done for sending SMTP server, sender address, receiver address § Our own IP series are whitelisted (making the mail gateway accept outgoing messages from our local mail clients) § Some ”odd” SMTP servers with problems with Greylisting may be whitelisted. (But why not fix those servers in stead? ) []

Spam control: Greylisting – any problems? • Possible problems: § First delivery is always delayed (for an unknown triplet) § Some mail servers are really not following Internet SMTP standards, i e they don’t know how to handle SMTP status 451 § Some mail servers have enormous spool queues, making resend of messages something that might happen first in a very distant future … § Greylisting is no final solution to the spam problem. Spammers will learn and adapt. But don’t tell them … !? []

Spam control: Greylisting software § We use it together with Sendmail § But Greylisting can work with others: § Exim § Qmail § Qpsmtpd § Postfix § Squirrelmail § Mail proxies []

Spam control: Greylisting Software used at Lund University § § § Sendmail 8. 13 Greylisting (invoked via Sendmail Milter function) Mail. Scanner 4. 31 Spam. Assassin 2. 63 Some RBLs (used from Spam. Assassin) § Sophos anti-virus § Clam. AV anti-virus []

Spam control: Greylisting More Greylisting information http: //projects. puremagic. com/greylisting/ []

Lund University e-mail policy E-mail policy proposed § Outgoing e-mail will be accepted only from a few verified SMTP servers (very few servers running spam and virus programs) § All incoming e-mail must pass a central mail gateway performing spam and virus checking § Also internal e-mail (within the university) should pass spam and virus checking []

Lund University e-mail policy Why use local mail servers? § Goal: Reduce number of local mail servers § Since earlier: Only certified SMTP servers are accepted in the network (certified servers are listed in routers). Only certified servers can receive SMTP mail (via port 25). § Certification is mainly an open relay check. § Make central mail services better! No need for local mail servers. []