Stop Spam by using Spam Assassin and Procmail
Stop Spam by using Spam. Assassin and Procmail ! Mark Kushinsky – MDS Computer Solutions mark@mdspc. com
What is Spam? According to dictionary. com Spam is : Unsolicited e-mail, often of a commercial nature, sent indiscriminately to multiple mailing lists, individuals, or newsgroups; junk e-mail. A trademark used for a canned meat product consisting primarily of chopped pork pressed into a loaf. To crash a program by overrunning a fixed-size buffer with excessively large input data. Mark Kushinsky - mdspc. com - mark@mdspc. com 2
Why Bother to filter Spam ? Depending on who you ask Spam is about 30% to 60% of ALL email and growing! (January 2004 - 60%, December 2003 - 58%, November 2003 – 56%, brightmail. com) Users are forced to waste time clearing out inboxes, which cost everyone money. Legal “hostile work environment” ramifications due to the offensive nature of most spam. Mark Kushinsky - mdspc. com - mark@mdspc. com 3
What is Spam. Assassin ? (http: //www. spamassassin. org/full/2. 6 x/dist/README) Spam. Assassin is a mail filter which attempts to identify spam using text analysis and several Internet based real time blacklists. Using its rule base, it uses a wide range of tests on mail headers and body text to identify "spam", also known as unsolicited commercial email. Once identified, the mail can then be optionally tagged as spam for later filtering using the user's own mail user-agent application. Spam. Assassin typically differentiates successfully between spam and non-spam in between 95% and 99% of cases, depending on what kind of mail you get. Spam. Assassin also includes support for reporting spam messages automatically, and/or manually, to collaborative filtering databases such as Vipul's Razor(http: //razor. sourceforge. net/). Latest version 2. 63 now includes a daemonized version, which runs persistently. This allows an MTA to process large volumes of mail through Spam. Assassin without having to fork/exec a perl interpreter for each one. IOW low CPU utilization for high volume mail servers. Spam. Assassin lives at http: //spamassassin. org/ or in CPAN. Mark Kushinsky - mdspc. com - mark@mdspc. com 4
What do I need to run Spam. Assassin? Linux (DOH !). OK Windows also but don't ask me how ! Procmail Perl - 5. 6. x is recommended. File: : Spec, Pod: : Usage, HTML: : Parser, DB_File, Net: : DNS perl modules. See install documentation ! A local mail agent (fetchmail, qmail, sendmail, etc. ) root access to install for system wide usage. Normal access to run. Mark Kushinsky - mdspc. com - mark@mdspc. com 5
How do I install ? (http: //www. spamassassin. org/full/2. 6 x/dist/INSTALL) Installing or Upgrading Spam. Assassin: The easiest way to do this is using CPAN. pm : perl -MCPAN -e shell [as root] o conf prerequisites_policy ask install Mail: : Spam. Assassin quit Debian, you can apt-get it from unstable, thanks to Duncan Findlay. Alternatively download the tarfile, zipfile or Red Hat RPM from http: //spamassassin. org/ , and install that, like so: [unzip/untar the archive] cd Mail-Spam. Assassin-* perl Makefile. PL [option: add -DSPAMC_SSL to $CFLAGS to build an SSL-enabled spamc] make install [as root] Mark Kushinsky - mdspc. com - mark@mdspc. com 6
How do I install ? (Cont) Create a non privileged user “spamd” or similar Download this start up script, modify the start section and put it as “spamd” in /etc/init. d http: //www. peregrinehw. com/downloads/Spam. Assassin/spamassassin Start: “daemon spamd -d -a -u spamd” - (-d daemon, -a auto-whitelist, -u <user>) Don't forget to make it executable ! modify /etc/procmailrc to include : PMDIR=$HOME/procmail – sets local procmail directory LOGFILE=$PMDIR/log – sets up local logging – don't forget to rotate log files with cron VERBOSE=no LOG=" " MAILDIR=$HOME/msgs – sets location of local mail files INCLUDERC=$PMDIR/rc. spam – this tells procmail to run local procmail recipes, spamc will be called from here. We can make it run on ALL mail by calling it from within /etc/procmailrc. Mark Kushinsky - mdspc. com - mark@mdspc. com 7
How do I install ? (Cont) rc. spam – local procmail file that catches bad attachments and calls spamassassin. VERBOSE=yes – set verbose logging here. : 0 BH # contains a virus or other suspicious attachment ? *. */(Content-(Type|Description|Disposition): . *. (hta|vbs|exe|scr|pif|lnk|bat|ocx|cmd|zip)|. */name=. *. (hta|vbs|exe|scr|pif|lnk|bat|ocx|cmd|tst|zip)) { : 0 c: IN-attach : 0 h | /bin/mail -s "CHECK msgs-IN-attach" mark } : 0 fw: spamassassin. lock # here we call spamassassin * < 256000 # is it smaller that 256 K ? | spamc : 0: # does the Spam-Status header contain the word “Yes”. If so put it in special folder called IN-REALSPAM * ^X-Spam-Status: Yes IN-REALSPAM set up a. spamassassin folder and give spamd access rights so that it can manage the auto whitelist and Bayesian databases Mark Kushinsky - mdspc. com - mark@mdspc. com 8
How do I customize ? local. cf is the file that controls what Spam. Assassin does. It is usually located here : /etc/mail/spamassassin/local. cf http: //www. yrex. com/spamconfig. php - Spam. Assassin local. cf Configuration Generator Local. cf documentation lives here : perldoc Mail: : Spam. Assassin: : Conf typical local. cf below : rewrite_subject 1 subject_tag **SA** report_safe 2 required_hits 5. 0 ok_languages en he use_bayes 1 use_terse_report 0 score FORGED_HOTMAIL_RCVD 2 3. 5 blacklist_from bgates@microsoft. com whitelist_from mgates@microsoft. com Mark Kushinsky - mdspc. com - mark@mdspc. com 9
What does the Spam. Assassin report look like for a typical piece of spam? X-Spam-Flag: YES X-Spam-Checker-Version: Spam. Assassin 2. 63 (2004 -01 -11) on linux 1. bgaddict. com X-Spam-Status: Yes, hits=18. 4 required=5. 0 tests=BAYES_99, HTML_70_80, HTML_LINK_PUSH_HERE, HTML_MESSAGE, HTML_TITLE_UNTITLED, MIME_HEADER_CTYPE_ONLY, MIME_HTML_ONLY, RCVD_IN_DSBL, RCVD_IN_NJABL_PROXY, RCVD_IN_SORBS_SOCKS, SORTED_RECIPS, SUSPICIOUS_RECIPS autolearn=spam version=2. 63 X-Spam-Level: ********* Content preview: Untitled Document Order Rx Meds From Home Valium Xanax - Vicodin ES - Hydrocodone - Viagra Weight Loss, Sexual Health, Pain Relief [. . . ] Content analysis details: (18. 4 points, 5. 0 required) pts rule name description - 0. 1 HTML_70_80 5. 4 BAYES_99 BODY: Message is 70% to 80% HTML BODY: Bayesian spam probability is 99 to 100% [score: 1. 0000] 1. 0 HTML_LINK_PUSH_HERE 0. 3 MIME_HTML_ONLY 0. 1 HTML_MESSAGE BODY: Message only has text/html MIME parts BODY: HTML included in message 0. 4 HTML_TITLE_UNTITLED 3. 5 SUSPICIOUS_RECIPS 2. 7 SORTED_RECIPS BODY: HTML link text says "push here" or similar BODY: HTML title contains "Untitled" Similar addresses in recipient list Recipient list is sorted by address etc. . Mark Kushinsky - mdspc. com - mark@mdspc. com 10
Considerations Bayesian filtering does NOT kick in until Spam. Assassin has learned at least 200 spam messages. Use sa-learn to teach Spam. Assassin about “spam” and “ham” sa-learn --spam --no-rebuild --showdots --mbox ~/msgs/IN-REALSPAM sa-learn --ham --no-rebuild --mbox --showdots $MAIL sa-learn --rebuild sa-learn --dump magic By default local configuration files, called user_prefs, located in $HOME/. spamassassin are ignored unless specially allowed in local. cf. “allow_user_rules = 1” Mark Kushinsky - mdspc. com - mark@mdspc. com 11
Resource Page http: //spamassassin. org/ - Spam. Assassin lives here ! http: //news. gmane. org/gmane. mail. spamassassin. general Spam. Assassin list archive http: //wiki. spamassassin. org/ - Spam. Assassin FAQ in Wiki format http: //www. yrex. com/spamconfig. php - Spam. Assassin Configuration Generator http: //www. peregrinehw. com/downloads/Spam. Assassin/spamassassin Spam. Assassin start up script http: //www. spambouncer. org/ – Spam. Assassin alternative http: //www. ii. com/internet/robots/procmail/qs/ - Procmail quick start http: //pegasus. rutgers. edu/~elflord/unix/procmail. html - Procmail Tutorial comp. mail. sendmail -- and – others. Mark Kushinsky - mdspc. com - mark@mdspc. com 12
Questions ? If you got em, ask em ! Mark Kushinsky - mdspc. com - mark@mdspc. com 13
- Slides: 13