A spam-Attack Detection and Prevention System

created for ACS Internet, Inc.

 

 

User’s Guide

 

 

 

 

 

presented by

Royce Williams

April 24th, 2001


Introduction                                                                                                               1

Installation                                                                                                                2

System Requirements                                                                                                                                                     2

Setup Requirements                                                                                                                                                        4

Customizing your Configuration file                                                                                                                           6

Customizing Your Allow and Ignore Files                                                                                                                10

Installing Relaydata.pm                                                                                                                                                12

Relay Reporting                                                                                                                                                             13

Sendmail Interaction                                                                                                                                                     14

Administration and Operation                                                                      15

Command-line Switches                                                                                                                                               16

-a: Display Information about All Hosts                                                                                                                     16

-c: Specifying a Different Configuration File                                                                                                           16

-d: Debugging                                                                                                                                                                 17

Debugging Levels                                                                                                                                                      17

Using A Local spamradar.conf file                                                                                                              18

-e: Exporting The Database                                                                                                                                          18

-f: Reading From a Different Mail Log File                                                                                                              19

-i: Importing Delimited Data                                                                                                                                        19

-m: Manually Search for an Arbitrary String                                                                                                          20

-o: Manually Testing A Relay                                                                                                                                      20

-O: Testing All Suspicious Relays                                                                                                                             21

-r: Generating Sendmail Reject Data                                                                                                                         21

-s: Adjusting the Number of Log Lines to Analyze                                                                                                 21

-t: Showing All Suspicious Activity in the Raw Mail Log                                                                                      22

-u: Adjusting the Username-Guessing Threshold                                                                                                   22

-v: Displaying Version Information                                                                                                                            22

-w: Watch and Act                                                                                                                                                          23

Troubleshooting                                                                                                  24

Deinstallation                                                                                                        25

References                                                                                                               26


 

Introduction

 

Welcome to spamradar!

 

You are probably an administrator for a moderate-sized mail server, interested in filling the gap between initial spam attack and ORBS/RBL listing.  Perhaps you are just interested in seeing some statistics about where your spam is coming from.  Either way, you’ve come to the right place.

 

Hopefully, you will find spamradar to be a useful tool in your fight against spam.  If you have any suggestions or questions, please send them to

 

            spamradar@tycho.org

 

The most recent version of spamradar can always be found at

 

http://www.tycho.org/spamradar/

 

 


Installation

System Requirements

 

An operating system that supports POSIX.  Most Unixlike operating systems (Unix, Linux, Mac OS X, BeOS) should be able to run spamradar.  Note, however, that spamradar has only been tested on Debian Linux and Sun Solaris 2.6/2.7/8.

 

Perl 5.004 or higher.  Perl can be downloaded from http://www.perl.com/.  There are distributions of Perl for many platforms.

 

BerkeleyDB 1.x - Most sendmail installations have BerkeleyDB 1.x support.  If you do not, you can download BerkeleyDB 1.x from http://www.sleepycat.com/.

 

DB_File - DB_File is a Perl module for writing to Berkeley DB 1.x databases. It is available from CPAN.  The easiest way to download and install DB_File is to use the Perl CPAN interface.  If you have used CPAN before on the system on which you are installing spamradar, you've probably already got DB_File.  If you haven't used CPAN on that system before, you may need to consult http://www.cpan.org/ for more information on configuring CPAN and downloading DB_File.  To get started, you can simply type

 

> perl –MCPAN –e shell

 

rlytest.pl – Chip Rosenthal’s rlytest.pl is what spamradar uses to perform its open-relay tests.  While rlytest.pl can be downloaded from http://www.unicom.com/sw/rlytest/, spamradar ships with a modified version that is designed specifically for spamradar to use.  If you do not plan to use spamradar’s relay-testing features, you do not need rlytest.pl.  Place rlytest.pl in the same directory as spamradar.

 

IMPORTANT NOTE: During testing of spamradar, it was noted that rlytest.pl returns OS errorlevel 0 if passed invalid parameters – the same errorlevel that is returned if a relay is tested and refuses to relay.  Since this is an undesirable side effect, a slightly modified version of rlytest.pl is included with spamradar that addresses this issue by o returning OS errorlevel 1 if rlytest.pl fails to initialize.  This version of rlytest also avoids some side effects that come from guessing your hostname and domain name by accepting them from the command line instead.  spamradar will not work with the unmodified version of rlytest.pl.

 


Setup Requirements

There are a number of things to consider before installing spamradar. 

 

spamradar needs a server on which to run.  If you have a separate loghost that receives mail logging information, this is the ideal place for spamradar to be installed. 

 

The standard area to place local executables on Unix systems is usually /usr/local/bin.  This is the recommended place for spamradar’s program file, spamradar.pl.  Permissions on spamradar’s program file should be adjusted as appropriate for your site.  If spamradar is not running as root/Administrator/superuser, you will need to make sure that all of spamradar’s files are accessible by the user you choose.

 

You will need to decide where to keep spamradar’s database files.  Since spamradar works closely with sendmail, placing spamradar’s database files in the same area that sendmail keeps its own database files – usually /etc/mail – is often a good idea.

 

spamradar has three configuration files that it will look for on startup – spamradar.conf, spamradar.allow, and spamradar.ignore.  Basic versions of these files are included in the distribution.  By default, spamradar will look for these files in /etc.  You will need to populate these files with data specific to your site for spamradar to work properly.  These files are explained later in more detail.

 

If you will be using spamradar to test other servers to see if they are open relays, it is probably a good idea to put a notice saying so in the message that mailservers see when they connect to you.  See the documentation for your mail server software for more information on how to do this.

 

You will need to set up two email addresses on your systems if you intend to test for open relays:

 

·        Mailbox for relaytest destination.  The most important part of relay testing is the end result – the evidence that the message was actually delivered.  When spamradar tests a relay, the destination mailbox must be available to receive that message, and the only messages that are sent to that mailbox should be relay tests.  It is recommended that the email address that you select be something relatively random, rather than “bob” or “relaytest” so as to make the account less likely to be abuse..

·        Mailbox for relaytest source.  This is the "From:"address of the relay test message.  If someone manually replies to the message, it will be sent to this address.  It is recommended this be the direct email address of an actual person, or (even better) a generic email address that is read regularly by an actual person.

 

Because the spamradar.conf file needs to store the password used to pop test results, make sure that this file is only readable by users authorized to do so.

 


Customizing your Configuration file

spamradar keeps a number of important configuration parameters in the spamradar.conf file.  Some parameters are in spamradar.conf as a matter of convenience – saving you the trouble of having to use the corresponding command-line switches every time the program executes.  Other parameters are required and must be set to appropriate values for your installation before spamradar will run.

 

In general, if a parameter is in CAPS, this value is not modified or overrideable while spamradar is running.  Parameters in lower case can be modified by command-line switches.

 

Parameter
Default Value
Description

ACCESSFILE

None – program will not run unless set

spamradar needs to be aware of any relays that you are manually rejecting in sendmail’s native access file (usually /etc/mail/access).

ALLOWFILE

/etc/spamradar.allow

The full path to the list of hosts, domains, and networks that are allowed to relay through your mailservers (or are your own mailservers) and that should never be blocked.

DBFILE

None – program will not run unless set

Location of the DB file that spamradar uses to store testing history and results.  The recommended location is the same area as your mail software’s other database files are kept.

debug

1

The level of debugging information that you want to see.  See the “Debugging” section later in this document for a list of possible values.

hard_unknown_limit

300

How many delivery attempts we’ll tolerate, even if a relay is listed in the spamradar.ignore file.

HOST

None – program will not run unless set

This value should be set to the fully-qualified domain name of the host on which spamradar is running.  If there are multiple network interfaces on the system, HOST should be set to the interface from which most tests will be performed.  You may need to examine the results of open-relay tests to determine the best value.

IGNOREFILE

/etc/spamradar.ignore

The full path to the list of hosts, domains, and networks that shouldn’t be tested even though they might pop up in spamradar’s reports for other reasons (high volume, old listservers, etc.)

lastlines

8000

How many lines to pull from the end of your mail log for analysis.  Larger values mean better accuracy and more spam detection.  Smaller values mean more speed of processing.

LOCAL_CONTACT

None – program will not run unless set

The email address of an actual person that can act on messages received about spamradar’s activities

LOCAL_TEST_DOMAIN

None – program will not run unless set

The primary administrative domain in which spamradar is operating.

log

/var/log/mail.log

The location of your mail log.

logstring

‘sendmail’

If the log that you are monitoring also contains non-mail-delivery information, any log records that do not contain this string will be ignored.

MAILER

/usr/lib/sendmail

The location of your mailer program.  This is used to send reports of open relays to ORBS or notify LOCAL_CONTACT of activities.

MAX_FIRSTCHAR_RATIO

2

Ratio of username guesses to unique first letters we'll tolerate.  For example, if someone tries to send email to 30 unknown users, and half of the usernames begin with "a" and the other half begin with "b", this number is ( 30 / 2 ) = 15.  The bigger the number, the more likely it is to be spam attack.

max_recipients

200

This is the maximum number of successful email deliveries that spamradar will tolerate before reporting and/or testing the server.  Be aware that this parameter is closely related to the “lastlines” parameter and they need to be evaluated together for best performance.

MAX_TEST_AGE

7

How long to wait before reevaluating the status of a relay (in days).  During this time period, attempts to manually test this relay will fail.

MAX_UNKNOWN_RATIO

.3

The tolerable percentage of deliveries that are bad.  If this threshold is exceeded, spamradar will be more likely to test the relay.  Valid values range from 0 (test everyone) to 1 (test no one).

max_username_guesses

2

This is the maximum number of username guesses that spamradar will tolerate before reporting and/or testing the server

mylogfile

/var/log/spamradar.log

This is the location of spamradar's log file.  It must be writeable by the user running spamradar.

POPHOST

‘pop’

The name of the pop server that has the account that will receive test results.  If not defined, spamradar will search any domains listed in your domain suffixes.

POP_PWD

None – program will not run unless set

The password to use to pop test results.  Note that the spamradar.conf file should be secured so that only authorized personnel can view this password.

PRG_EXEC_PATH

'/usr/bin:/usr/sbin'

This is the list of paths that are in the program execution path.  This needs to be set for security purposes.

REJECTDBFILE

None – program will not run unless set

Location of the Berkeley DB file that sendmail references to block relays that have tested as open.

RLYTEST

None – program will not run unless set

The full path to the location of the rlytest.pl binary that spamradar will call to test a relay.

show_all_hosts

0

Boolean flag to indicate whether or not we will display hosts that we would normally suppress in our output.

TAIL

/var/log/spamradar.log

This is the location of your external 'tail' program.  spamradar needs to use 'tail' to efficiently pull the last lines from the mail log.

TEST_RECIPIENT

None – program will not run unless set

This value should be set to the address of the mailbox you set up to be the recipient of relay tests.  For maximum security, choose a name that is difficult to guess.

TEST_SENDER

None – program will not run unless set

This value should be set to the address of the mailbox you set up to be the recipient of relay tests. This address will also receive error messages if the relay is rejected internally.

TEST_SERVICE_ADDR

None

The email address to which relays can be automatically submitted upon detection.  If this parameter is not set, open relays will not be submitted to such a service.  If spamradar is configured to only report confirmed open relays (its default behavior), you can submit those relays to relays@orbs.org by using that address for this parameter.  Also, note that only services that accept relay nominations without an actual attached spam will accept this type of submission.


Customizing Your Allow and Ignore Files

spamradar tests servers to determine whether or not they relay for the domain that you specify.  There are some servers (such as your own mail servers, for example) that you want to be able to relay by design.  Therefore, spamradar needs a list of relays that you do not want to test – the spamradar.allow file.

 

There are also mail servers that you probably don’t need to bother testing.  For example, AOL probably doesn’t have any open relays.  The spamradar.ignore file that ships with spamradar has a number of these domains already listed.  Do not accept this list as a given, however.  It is recommended that you use the reporting features of spamradar to learn what domains should be placed in both the spamradar.ignore and spamradar.ignore files.

 

Both files can contain IP addresses, classful networks, domain names, and full hostnames (Unfortunately, CIDR notation is not yet supported, but is planned).  For example, your spamradar.allow file might contain the following:

 

# spamradar.allow file for myisp.int

#

# all the servers in my mail farm are here

192.168.254

mydomain.com

mymailserver1.mydomain.com

mymailserver2.mydomain.com

10.0.0.4

 

The contents of your spamradar.ignore file might look something like this:

 

# spamradar.ignore file for myisp.int

#

# big mail-producing domains listed here

aol.com

bigfoot.com