New Measures to Limit Spam

Recently, CIS has heard an increasing number of concerns from the WHOI community regarding spam, or unsolicited bulk email, being sent to WHOI mail accounts. In response we have implemented a number of mechanisms on the mail servers to reduce the impact of spam. The purpose of this message is to familiarize you with these mechanisms and how they can help you reduce spam in your email inbox.

All of these procedures are optional. We have selected them to optimally remove spam from the incoming mail with minimal impact on legitimate mail. If you have concerns that your legitimate mail is being incorrectly treated as spam, please report this to postmaster@whoi.edu promptly. Our systems are always being refined and improved to deliver better email service. In the extreme case we can "whitelist" your address, meaning that only the bare minimum of filtering will be done to your incoming email.

Spam reporting

In order to reduce spam, we need to know what spam you are getting. For this purpose, CIS has set up an email address to accept reports of spam received by WHOI mail users. If you receive spam, we ask that you forward it to us. Here's how:

1. Turn on full headers in your email reader. In Netscape Messenger, select the menu option View->Headers->All. If you don't use Netscape, take a look at this SpamCop page for instructions for over 20 different email clients.
2. If you can, forward the message as an attachment. Some mail clients will mangle your message if you forward it "inline". In Netscape, select the menu option Message->Forward As->Attachment. Note that forwarding as an attachment is not always a good idea for day-to-day use, but it helps us when dealing with spam because it shows us the message unaltered from the way you received it.
3. Forward to spam@whoi.edu. Please don't send spam to Helpdesk or to individual CIS technicians. The spam@whoi.edu address has been set up to deal especially with spam.

When we receive reports of spam you're receiving, we use them to tune the spam detection and filtering systems in the mail servers.

From time to time, when you report a spam message, the mail server may reject your message with an error such as Message content rejected. This occurs when someone else has already reported that particular spam, and we have already placed a filter on the mail server for it. This will happen particularly when the same spam is sent to several WHOI users.

Selective message filtering

Certain kinds of messages are obviously, trivially identifiable as spam -- for instance, many varieties of the "Nigeria 419" scam. Because there is precious little chance that any such message could be legitimate email, we feel safe in automatically rejecting these messages at the mail server. If someone sends you a message that the mail server rejects, you will simply not get the message. If the remote mail server works correctly, the sender will receive a notice of failed delivery (a "bounce message").

Naturally, it is of paramount importance that we avoid misidentifying non-spam messages as spam messages in this sort of filtering. For this reason we use this facility very selectively. It is the same mechanism we use to reject email viruses (such as Klez and Sircam).

SpamAssassin heuristic annotation

Updated, July 30 2002: We have scaled-back the marking done by the SpamAssassin system in response to comments that it was too intrusive. In order to make use of SpamAssassin, use message filters in your mail client to filter mail based on its headers.

SpamAssassin is a program which attempts to calculate the chance that a message is spam, by inspecting it for over 300 different weighted patterns. These patterns include common phrases used in spam, abusive HTML and JavaScript attacks, and invalid or mangled message headers. SpamAssassin is a "best guess" system and does not reject or delete messages; what it does is to add tags to the message headers, which you can use in your mail client program to filter out the spam.

Each pattern that SpamAssassin checks for has a weight associated with it -- a small number which indicates how likely it is that a message matching that pattern is spam. For instance, messages containing the expression "DISCOUNT VIAGRA" might get 2 points, whereas messages sent using software commonly used to send spam might get 0.5 points. Weights can also be negative, meaning that a message is likely not to be spam; mail sent using legitimate mailing-list software such as Mailman and Majordomo gets negative points. If the total number of points for a message meets or exceeds 5.0, the message is marked as spam.

Address based filtering and DNSBLs

Another system we use to reduce spam takes into account that a large proportion of spam comes from well-known sources. Because spamming is not illegal everywhere, and what laws there are are only shakily enforced, some marketing firms and ISPs feel free to host large numbers of spammers and send massive quantities of spam. Rarely if ever do these sites send anything but spam. Other sites operate insecure servers (open relays and proxies) -- often unintentionally, on systems not even intended to send mail. Spammers exploit these insecurities to flood spam out through these systems.

So why accept mail from such sites at all? A class of services known as blackhole lists exist to inform people of which systems are sources of spam. Most of these lists are implemented using the Domain Name Service, or DNS, and are known as DNSBLs. There are hundreds of these lists, with various policies as to how an IP address or site gets listed. We presently use a few of these lists, selected for their strong reputation for rejecting mail from sites whose business is spamming, and from the addresses of open proxies.

We also supplement the DNSBLs with a locally maintained list of IP addresses which have been the sources of spam reported by WHOI users. To make sure that neither the DNSBLs nor the local lists are unintentionally rejecting legitimate mail, we check the mail server logs daily. We are glad to say that in the months we have been using these facilities, this has not been a problem.

Filtering Mail in Netscape Messenger

SpamAssassin adds several headers to messages it determines resemble spam. By using Netscape Messenger's Message Filters facility, you can sort these messages into a separate folder from your inbox automatically. (We do not recommend deleting these messages outright, because SpamAssassin is not perfect.) The headers that SpamAssassin adds include the following:

1. X-Spam-Flag: Yes
2. X-Spam-Level: *****
   (The number of stars is equal to the number of points the message got.)

Here's how to sort spammy messages into a separate folder. The process is a little long, but you only have to do it once. Note: This only works in Netscape 4.7x, not Netscape 6.x as the ability to filter on custom headers was removed. It may work in some versions of Mozilla, though the menus are different.

1. Open Netscape Messenger.
2. Create the new folder. Right-click (or Control-click, on Macintosh) on the Inbox folder. A menu will pop up; select the option New Subfolder... Name the folder filtered-spam or some such.
3. Open Message Filters. From the Edit menu, select Message Filters... A window will pop up. If you already have some filters, they will be displayed.
4. Create a filter. Click the New... button, located on the right-hand side of the window. A new window will appear. Here's what to do in this window:
   • In the Filter Name text box, give the filter a name like "SpamAssassin filtering".
   • In the Filter Criteria section, pull down the menu that says "sender", and select Customize Headers. In the box that pops up, click New, and type X-Spam-Flag. Press Enter, then click OK.
   • On the right-hand side of the Filter Criteria, there is a text box. Type Yes in this box.
   • In the Filter Action section, pull down the menu that says "Inbox" and change it to the name of your new folder (such as "filtered-spam".)
   • Click OK.

As you receive new messages in your inbox, Netscape will automatically test them for SpamAssassin's X-Spam-Flag header. If the value of this header is Yes on on a given message, the message will be automatically shunted into the "filtered-spam" folder. Check this folder at least once a week to make sure nothing legitimate has been accidentally filed as spam -- and, if anything has, let CIS know at spam@whoi.edu.