Filtering Mail with Mail Audit and Mail Spam

Filtering Mail with Mail: : Audit and Mail: : Spam. Assassin Creede Lambard penguinsinthenight. com 20 August 2002

General Outline:

General Outline: ● How UNIX handles mail

General Outline: ● ● How UNIX handles mail A simple understated diatribe against unsolicited commercial email

General Outline: ● ● ● How UNIX handles mail A simple understated diatribe against unsolicited commercial email Why mail filtering is a Good Thingtm

General Outline: ● ● How UNIX handles mail A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thingtm ● If you use Windows. . .

General Outline: ● ● How UNIX handles mail A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thingtm ● If you use Windows. . . ● Using Mail: : Audit

General Outline: ● ● How UNIX handles mail A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thingtm ● If you use Windows. . . ● Using Mail: : Audit ● Using Mail: : Spam. Assassin

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail

How Unix handles your mail. forward to another mail address: me@myotherisp. com

How Unix handles your mail Piping to another program: | vacation

Does this look familiar?

spam

spam ● Unsolicited commercial email

spam ● Unsolicited commercial email – Sent in bulk

spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service

spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient

spam ● ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient Not necessarily mail you don't want. . .

spam ● ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient Not necessarily mail you don't want. . . – Although for purposes of this presentation we'll treat them the same.

When Spamtm is acceptable

When Spamtm is acceptable

spam is a Bad Thingtm

spam is a Bad Thingtm ● It shifts the burden of costs to the recipient

spam is a Bad Thingtm ● It shifts the burden of costs to the recipient ● It clogs the Net

spam is a Bad Thingtm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time

spam is a Bad Thingtm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● Items/services advertised through spamming tend to be of questionable value

spam is a Bad Thingtm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● ● Items/services advertised through spamming tend to be of questionable value The vast majority of it is fraudulent

Dealing with spam

Dealing with spam ● Ignore it

Dealing with spam ● Ignore it. . . and hope it goes away

Dealing with spam

Dealing with spam ● Ignore it. . . not an option

Dealing with spam ● Ignore it. . . not an option ● Just hit Delete. . .

Dealing with spam ● ● Ignore it. . . not an option Just hit Delete. . . The damage is already done

Dealing with spam ● ● ● Ignore it. . . not an option Just hit Delete. . . The damage is already done Filter it as early as possible in its life cycle

Dealing with spam ● ● Ignore it. . . not an option Just hit Delete. . . The damage is already done ● Filter it as early as possible in its life cycle ● Filter it as it's trying to enter your machine

If you use Windows. . .

Mail filtering

Mail filtering | /home/you/mailfilter

Mail filtering apart from spam filtering

Mail filtering apart from spam filtering ● Separating mailing lists into their own folders

Mail filtering apart from spam filtering ● Separating mailing lists into their own folders ● News-to-mail gateways

procmail

procmail ● Advantages:

procmail ● Advantages: – Well-established

procmail ● Advantages: – Well-established – Lots of sample scripts

procmail ● ● Advantages: – Well-established – Lots of sample scripts Disadvantages:

procmail ● ● Advantages: – Well-established – Lots of sample scripts Disadvantages: – Arcane syntax

procmail ● ● Advantages: – Well-established – Lots of sample scripts Disadvantages: – Arcane syntax – Like learning a new language. . .

procmail ● ● Advantages: – Well-established – Lots of sample scripts Disadvantages: – Arcane syntax – Like learning a new language. . . – And it's not Perl!

Mail: : Audit

Mail: : Audit ● Written by Simon Cozens

Mail: : Audit ● Written by Simon Cozens procmail is nasty. It has a tortuous and complicated recipe format, and I don't like it. I wanted something flexible whereby I could filter my mail using Perl tests. - Simon Cozens, from the Mail: : Audit perldoc

Mail: : Audit ● ● Written by Simon Cozens Based on audit_mail and deliverlib by Tom Christiansen

Mail: : Audit ● ● ● Written by Simon Cozens Based on audit_mail and deliverlib by Tom Christiansen It's Perl!!!!!!!!

Mail: : Audit ● ● Written by Simon Cozens Based on audit_mail and deliverlib by Tom Christiansen ● It's Perl!!!!!!!! ● A module, not a standalone program

How Mail: : Audit Works

Parsing mail

Parsing mail ● Mail: : Internet object

Parsing mail ● Mail: : Internet object ● Parse by:

Parsing mail ● Mail: : Internet object ● Parse by: – From, To or CC lines

Parsing mail ● Mail: : Internet object ● Parse by: – From, To or CC lines – Subject

Parsing mail ● Mail: : Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers

Parsing mail ● Mail: : Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text

Parsing mail ● Mail: : Internet object ● Parse by: ● – From, To or CC lines – Subject – Absence, presence or content of headers – Body text Anything can be parsed

Parsing mail ● Mail: : Internet object ● Parse by: ● – From, To or CC lines – Subject – Absence, presence or content of headers – Body text Anything can be parsed – Using Mail: : Internet: : as_string

Installation

Installation ● Download and install Mail: : Audit from CPAN

Installation # perl -MCPAN -e shell cpan> install Mail: : Audit

Installation ● Download and install Mail: : Audit from CPAN ● Create. forward file

Installation | /home/creede/mailfilter

Installation ● Download and install Mail: : Audit from CPAN ● Create. forward file ● Create filter file

Installation #!/usr/bin/perl use Mail: : Audit; my $mail = new Mail: : Audit;

Installation #!/usr/bin/perl use Mail: : Audit; my $mail = new Mail: : Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject;

Installation #!/usr/bin/perl use Mail: : Audit; my $mail = new Mail: : Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body;

Installation #!/usr/bin/perl use Mail: : Audit; my $mail = new Mail: : Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body;

Installation #!/usr/bin/perl use Mail: : Audit; my $mail = new Mail: : Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body;

Installation ● Download and install Mail: : Audit from CPAN ● Create. forward file ● Create filter file ● Remember to chmod 0755!

Mail disposition ● $mail->accept – Accepts mail into default inbox

Mail disposition (continued) if ($mail->from =~ /mom@applepie. com/) { $mail->accept; }

Mail disposition (continued) ● $mail->accept(“/path/to/alternate/mailbox”) – Accepts mail into a non-default mailbox

Mail disposition (continued) my $maildir = “/home/me/mail”; if ($mail->subject =~ /spug/i) { $mail->accept(“$maildir/spug-list”); }

Mail disposition (continued) ● $mail->pipe(“/path/to/external/program”) – Pipes mail through the specified program

Mail disposition (continued) if ($mail->subject =~ /keplerian/i) { $mail->pipe(“/home/creede/parse_kepler”); }

Mail disposition (continued) ● $mail->resend(“someguy@otherisp. com”) – Sends the mail in its entirety to another address

Mail disposition (continued) if (is_419($message)) { $mail->{noexit} = 1; $mail->put_header('X-Loop', 'creede@penguinsinthenight. com'); $mail->put_header('To', "$to (forwarded -no monetary loss -- for your files)"); $mail->resend("uce@ftc. gov"); $mail->resend("419. fcd@usss. treas. gov"); $mail->{noexit} = 0; $mail->ignore;

Mail disposition (continued) ● $mail->reject($reason) – Rejects the mail, returning it to the sender with the (optional) reason specified

Mail disposition (continued) if (is_murky($mail)) { $mail->put_header('X-Loop', 'creede@penguinsinthenight. com'); $mail->reject("I don't like spam. "); }

Mail disposition (continued) ● $mail->ignore – Consigns the mail to the bit bucket

Mail disposition (continued) # kill off Korean spam if ($body =~ /ks. c/i) { $mail->ignore; }

Mail: : Spam. Assassin

Mail: : Spam. Assassin ● Header analysis

Mail: : Spam. Assassin ● Header analysis ● Text analysis

Mail: : Spam. Assassin ● Header analysis ● Text analysis ● Blacklists

Mail: : Spam. Assassin ● Header analysis ● Text analysis ● Blacklists ● Vipul's Razor

Mail: : Spam. Assassin – Installation ● Download and install Mail: : Spam. Assassin from CPAN

Mail: : Spam. Assassin – Installation # perl -MCPAN -e shell cpan> install Mail: : Spam. Assassin

Mail: : Spam. Assassin – Installation #!/usr/bin/perl use Mail: : Audit; use Mail: : Spam. Assassin; my $mail = new Mail: : Audit; my $spamtest = new Mail: : Spam. Assassin; my $status = $spamtest->check($mail); if ($status->is_spam()) { $mail>accept(“/home/you/spamtrap”); }

Mail: : Spam. Assassin – Configuration ● Load configuration from /etc/mail/spamassasin. conf or /home/you/. spamassassin/user_prefs

Mail: : Spam. Assassin – Configuration # Spam. Assassin user preference file # required_hits 4 # # default is 5 # whitelist_from mom@applepie. com blacklist_from scuzzball@spamspewer. com

Paul Graham's Plan for Spam

Paul Graham's Plan for Spam madam 0. 99 promotion 0. 99 republic 0. 99 shortest 0. 047225013 mandatory 0. 047225013 standardization 0. 07347802 2600 0. 0813768 sorry 0. 08221981

URLs for more information

URLs for more information ● Internet Mail http: //www. imc. org/rfcs. html

URLs for more information ● Internet Mail http: //www. imc. org/rfcs. html ● Mail: : Audit http: //simon-cozens. org/writings/mail-audit. html

URLs for more information ● Internet Mail http: //www. imc. org/rfcs. html ● Mail: : Audit http: //simon-cozens. org/writings/mail-audit. html ● Mail: : Spam. Assassin http: //www. spamassassin. org/ http: //www. deersoft. com (Outlook)

URLs for more information ● Internet Mail http: //www. imc. org/rfcs. html ● Mail: : Audit http: //simon-cozens. org/writings/mail-audit. html ● Mail: : Spam. Assassin http: //www. spamassassin. org/ http: //www. deersoft. com (Outlook) ● Paul Graham's Plan for Spam http: //www. paulgraham. com/spam. html

URLs for more information ● Internet Mail http: //www. imc. org/rfcs. html ● Mail: : Audit http: //simon-cozens. org/writings/mail-audit. html ● Mail: : Spam. Assassin http: //www. spamassassin. org/ http: //www. deersoft. com (Outlook) ● Paul Graham's Plan for Spam http: //www. paulgraham. com/spam. html ● And of course Google. com!

Questions?

Thank you! creede@penguinsinthenight. com http: //www. penguinsinthenight. com/spamtalk