SpamFiltering

From Request Tracker Wiki
Jump to navigation Jump to search

Spam Filtering

By moving all SPAM to a single queue, you can easily purge (shred) it periodically. You might also take the opportunity to purge any RT users who do not have a ticket associated with them.

Please find documented below 6 setups for processing spam (1a-1c, 2a-2c). In general, you want to filter your messages at the MTA, before they get RT if you can.

The list archives also contain a number of filtering in RT;#87056 useful tips

0. Pre-filtering & scoring messages

There are a number of ways of doing so, such as procmail or mailfilter which can do other somewhat-RT-specific pre-processing as well.

1. Pre-filtered mail

There are several options for processing messages once scored:

a. Pre-filtered mail + ::Filter::SpamAssassin (=> Spam Queue)

http://codepad.org/HQmnr0AG based on

http://documentroot.com/SpamAssassin.pm based on RT::Interface::Email::Filter::SpamAssassin from 3.8.1

This updated version of the filter provides several options for handling potential spam, such as rejecting all messages above a certain score, and filing others in a SPAM queue.

b. Pre-filtered mail + Scrip => Queue

In the example below a score of 5 or more is needed. Edit preparation code if you want to modify the level.

Create new Scrip using following information:

  • Condition: On Create
  • Action: User Defined
  • Custom action preparation code:
 # Match ***** level spam. You might want to set this higher/lower if needed
 my $match = '\*\*\*\*\*';
 my $inMessage = $self->TransactionObj->Attachments->First;
 return 0 if (!$inMessage);  # if no message attachment - assume web UI
 return 0 if (!$inMessage->GetHeader('Received'));  # exit if not email message
 
 my $spamLevel = $inMessage->GetHeader('X-Spam-Level');
 return ( $spamLevel !~ /$match/i ) ? 0 : 1;
  • Custom action cleanup code:
 my $newqueue = 'spam';
 my ($status, $msg) = $self->TicketObj->SetQueue($newqueue);
 return $status ? undef : 1;

c. Pre-filtered mail + ProcMail => Spam Queue

A procmail recipe (for each queue/(correspond|comment) pair) can be used to send spam to a spam queue in RT. Note that:

  • the flags bh need to be set so rt-mailgate sees both the body and header of the message.
  • the absence of a second colon so there is no lockfile
 PATH=/usr/local/bin:/bin/:/usr/bin
 LOGFILE=/var/log/procmail.log
 #rt-mailgate has already saved the message in the rt database
 #set DEFAULT to be somewhere real if you want procmail to save the message elsewhere
 DEFAULT=/dev/null
 
  #Messages >300000 characters proceed to recipient (unlikely to be spam)
 #:0fwbh
 :0wbh
 * > 300000
 | /usr/local/rt3/bin/rt-mailgate --queue default --action correspond --url http://localhost/
 
 #if the spam trigger is fired send to spam queue
 #:0fwbh
 :0wbh
 * ^X-Spam-Level: \*\*\*\*\*
 | /usr/local/rt3/bin/rt-mailgate --queue spam --action correspond --url http://localhost/rt/
 
 #if the spam trigger is not fired then send to expected destination
 #:0fwbh
 :0wbh
 | /usr/local/rt3/bin/rt-mailgate --queue default --action correspond --url http://localhost/rt/

2. Unfiltered messages

a. Unfiltered mail + ::Filter::SpamAssassin (=> Spam Queue)

RT::Interface::Email::Filter::SpamAssassin above (1a) can also pass unscored messages off to SpamAssassin for processing.

b. Unfiltered mail + ProcMail & SpamAssassin => Spam Queue

This procmail recipe is similar to 1c. above, except that it handles previously unscored messages, and accepts many of the rt-mailgate parameters as environment variables so that it may be easily used with multiple aliases.

Aliases: 

 rt: "|/usr/local/bin/procmail -m ACTION=correspond QUEUE=General /usr/local/etc/procmail/rt-deliver.rc"
 rt-comment: "|/usr/local/bin/procmail -m ACTION=comment QUEUE=General /usr/local/etc/procmail/rt-deliver.rc"

/usr/local/etc/procmail/rt-deliver.rc: 

 #Messages >300000 characters proceed to recipient (unlikely to be spam)
 :0w
 * > 300000
 | /usr/local/rt3/bin/rt-mailgate --queue $QUEUE --action correspond --url http://localhost/rt/
 
 #Is it spam?
 :0fw: spamassassin.lock
 * < 300000
 | /usr/local/bin/spamc
 
 #if the spam trigger is fired send to spam queue
 :0w
 * ^X-Spam-Status: Yes
 | /usr/local/rt3/bin/rt-mailgate --queue Spam --action correspond --url http://localhost/rt/
 
 #if the spam trigger is not fired then send to expected destination
 :0w
 | /usr/local/rt3/bin/rt-mailgate --queue $QUEUE --action $ACTION --url http://localhost/rt/

c. ProcMail + SpamAssassin + Scrip => Spam Queue

http://www.soundwave.net/~wmono/rt/

3. Mis-handled messages

a. False positives: Greylist

2c above outlines a general (non-procmail specific) setup (steps 1 through 6 for RT) to automatically greylist suspect messages. Legitimate requestors may reply to an autoresponse and have their misfiled message requeued.

b. False negatives: QuickSpamhandler

Note: QuickSpamHandler seems to no longer be available. The RT::Extension::ReportSpam is fairly similar though.

This extension provides a few tools that can be used for manual triage of SPAM in low-traffic installation, or handling the odd message that leaks through your automated filters:

  • Adds alternate forms of the "Unowned Tickets" and "My Tickets" global searches, which exclude messages in the spam queue.
  • Adds a link to the "Search - Unowned Tickets" query which permits you to instantly vanquish a ticket if the subject is sufficient to convince you the message is SPAM.
  • Adds a link to the Ticket tool bar that will instantly vanquish the displayed ticket.

Vanquished tickets are moved to a designated SPAM queue; the default is named "Possible SPAM".

Installation

  • Download the archive
  • Unpack it
  • ./configure
  • make install
  • Read and act on the output from the previous command.

Really, the only tricky bit is the database update. If I ever figure out a better way to do this I'll make sure to update the archive. Hopefully someone will find this useful. If you do, please drop me a line!

c. False negatives: Bookmarklet

An alternative to QuickSpamHandler above, this bookmarklet moves the displayed ticket to the spam queue (change 12 to the appropriate ID for your queue), and sets the status to rejected. 

javascript:self.location=self.location+'&Status=resolved&Queue=12'

Thus you can distinguish messages which you have already decided are spam, from those which are automatically filed into that queue.

Collect spam/ham for training with procmail

In /etc/postfix, I am maintaining a file named rt-aliases. It contains, for every queue: 

 rt:         "|/usr/bin/procmail -m /etc/procmailrcs/general"
 rt-comment: "|/usr/bin/procmail -m /etc/procmailrcs/general-comment"

in /etc/procmailrcs, the file "general" looks like: 

 PATH=/usr/local/bin:/bin:/usr/bin
 LOGFILE=/var/log/procmail.log
 INCLUDERC=/etc/procmailrcs/_spamfilter

The "_spamfilter" include does the actual collection: 

 # The lock file ensures that only 1 spamassassin invocation happens
 # at 1 time, to keep the load down.
 #
 :0fw: spamassassin.lock
 * < 256000
 | /usr/bin/spamassassin
 
 # Mails with a score of 15 or higher are almost certainly spam (with 0.05%
 # false positives according to rules/STATISTICS.txt). Let's put them in a
 # different mbox. (This one is optional.)
 :0:
 * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
 /local/mailcopy/almost-certainly-spam
 
 # Work around procmail bug: any output on stderr will cause the "F" in "From"
 # to be dropped.  This will re-add it.
 :0
 * ^^rom[ ]
 {
    LOG="*** Dropped F off From_ header! Fixing up. "
 
   :0 fhw
   | sed -e '1s/^/F/'
 }
 
 :0 c
 /local/mailcopy/unconfirmed-ham

Notes & Other Resources

Don't forget to run `newaliases` (or equivalent) after you alter your configuration if appropriate.

Retrieved from "https://rt-wiki.bestpractical.com/index.php?title=SpamFiltering&oldid=26286"