In an attempt to be a little more friendly in terms of bandwidth to the strapped folk over at Malware Domains, I’ve retooled the script I wrote about in the post To Be Protecting Against the Malware.

I’ve added some lines to take advantage of remote zipped files (.zip), which will help them by reducing the number of bits we’re pulling from them.

I’ve added some lines to copy the downloaded malware zones file to other servers behind my firewall, which will help them by not making individual connections from each server to pull the files. I just set up a cron job on each internal “slave” server to bounce named every morning timed for after this process is complete.

Here’s the updated code. It is, as is my wont, rather verbose. It is considerably more verbose than other examples out there that take care of this same problem, but as I said, such is my wont.

The URLS array is filled with fake hosts right now b/c the zipped format is still in testing. When the folk at malwaredomains.com think it’s ready for public consumption, I’ll put the real hosts back in.

Also, it’s relatively untested, and I expect to be tweaking it. Use at your own risk.

On the “master” server, I’m using this…

#!/usr/local/bin/bash

# To know where script is running
HOSTNAME=$( hostname )

# To put file where named can see it
BLACKHOLEDIR=/var/named/etc/namedb/blackhole

# To name file so we know what named seeing
TMPZONEFILE=tmp.malwaredomains.zones
ZONEFILE=malwaredomains.zones
ZONEFILEBACKUP=malwaredomains.zones.bak

# To get updated file from remote server
URLGRABBER=/usr/local/bin/curl
USERAGENT="Malware Domain Grabber ( ${HOSTNAME}; unix; BASH )/0.1"

# To keep quiet while am getting file
URLGRABBEROPTS="-s -f"

# To know where file is hosted
URLS=( host1 host2 host3 host4 )
TIMESTAMPFILE=timestamp

# To know how to decompress the file
UNZIPCMD=/usr/local/bin/unzip
UNZIPOPTS="-o -qq"

# To copy files to other servers so that we are only
# pulling the files once, though we have multiple
# DNS servers in house

HOSTS=( host1 host2 host3 host4 )

# MOUNTCMD: The mount command
MOUNTCMD=/sbin/mount
UMOUNTCMD=/sbin/umount

# FSTYPE: The filesystem type of the mounted partition
FSTYPE=nfs

# MOUNTDIR: The directory that the dumps will be written to
MOUNTDIR=/mnt

# To control bind
NAMEDCMD="/usr/sbin/rndc reload"

#==============================================================

# Get start time so we can know how long this thing runs
START=$( date +%s )

# Make our working directory the location of the blackhole files
cd ${BLACKHOLEDIR}

# Copy the current timestamp file to ${TIMESTAMPFILE}.old so we can
# make a comparison between what we have and what's out there now.
if [ -f ${BLACKHOLEDIR}/${TIMESTAMPFILE} ]; then
	cp ${BLACKHOLEDIR}/${TIMESTAMPFILE} ${BLACKHOLEDIR}/${TIMESTAMPFILE}.old
fi

# Attempt to download the timestamp file and zone file from each mirror.
# Break out of the loop at the first successful download of a zone file,
# otherwise, try each one in turn

# Assume there are no updates available
NEW=0

for URL in "${URLS[@]}"; do
	echo "Attempting to download from ${URL}"
	echo " Checking timestamps..."
	${URLGRABBER} ${URLGRABBEROPTS} -A '${USERAGENT}' -o ${BLACKHOLEDIR}/${TIMESTAMPFILE}.zip ${URL}/${TIMESTAMPFILE}.zip

	if [ $? -ne 0 ]; then
	echo "  ... timestamp download from ${URL} failed! Code: $?"
	# Move on to next URL so we keep the timestamp/zonefile pair intact
	continue
	else
	if [ -f ${BLACKHOLEDIR}/${TIMESTAMPFILE} ]; then
		# Unzip the new timestamp file over the old old one
		${UNZIPCMD} ${UNZIPOPTS} ${BLACKHOLEDIR}/${TIMESTAMPFILE}.zip

		# Do a little cleanup
		rm -f ${BLACKHOLEDIR}/${TIMESTAMPFILE}.zip

		OLDTIMESTAMP=$( cat ${BLACKHOLEDIR}/${TIMESTAMPFILE}.old )
		NEWTIMESTAMP=$( cat ${BLACKHOLEDIR}/${TIMESTAMPFILE} )

		if [ ${OLDTIMESTAMP} -ge ${NEWTIMESTAMP} ]; then
			echo " ... no new updates."
			# No new updates on this server... but how well are the various mirrors
			# kept in sync?  Let's try the others. This is a tiny transfer, and it's
			# only once a day, so it's pretty cheap.
			continue
		fi
	else
		# Timestamp file does not exist. Create it.
		${UNZIPCMD} ${UNZIPOPTS} ${BLACKHOLEDIR}/${TIMESTAMPFILE}.zip
		rm ${BLACKHOLEDIR}/${TIMESTAMPFILE}.zip
	fi
	fi

	# Backup and copy file to final location for named to find
	# (via "include" directory in named.conf)
	echo "Backing up zone file"
	cp ${BLACKHOLEDIR}/${ZONEFILE} ${BLACKHOLEDIR}/${ZONEFILEBACKUP}

	echo "Retrieving new zone file from ${URL}..."
	${URLGRABBER} ${URLGRABBEROPTS} -o ${BLACKHOLEDIR}/${ZONEFILE}.zip ${URL}/${ZONEFILE}.zip

	if [ $? -ne 0 ]; then
		echo "  ... zonefile download from ${URL} failed!  Code: $?"
		# Oops.  Try the next server.  If this is the last, then ${NEW} is still
		# set to 0, and we'll be done. Better luck tomorrow...
		continue
	else
		# We have a new timestamp, and were able to download the zone file from
		# the same server we downloaded the timestamp from.  Set ${NEW} to 1 and
		# get out of the loop. No need to check further.

		echo "Unzipping new zone file..."
		if [ -f ${ZONEFILE}.zip ]; then
			${UNZIPCMD} ${UNZIPOPTS} ${BLACKHOLEDIR}/${ZONEFILE}.zip
			rm ${ZONEFILE}.zip
			# Rename the zone file temporarily to allow sed to work on it later, and
			# and in that process, rename it back to the name that named knows.
			mv ${ZONEFILE} ${TMPZONEFILE}
		else
			echo "No new zone file..."
			exit
		fi

		NEW=1
		break
	fi
done

# If ${NEW} hasn't been set, then we either error'd out of all servers, or there are no
# new files. Either way, we're done.
if [ ${NEW} == 0 ]; then
	exit 1
else
	# Disable name checking for only those domains with underscores,
	# so we don't have to turn off name checking globally.

	SEARCH='_'
	FIND='blockeddomain.hosts";};'
	REPLACE='blockeddomain.hosts"; check-names ignore;};'

	# Get a count of the zones from the last update
	OLDZONECOUNT=$( cat ${BLACKHOLEDIR}/${ZONEFILEBACKUP}|grep "^zone"|wc -l )

	echo "Disabling checking on domains with underscores"
	sed "/${SEARCH}/ s/${FIND}/${REPLACE}/g" ${BLACKHOLEDIR}/${TMPZONEFILE} > ${BLACKHOLEDIR}/${ZONEFILE}
	rm -f ${BLACKHOLEDIR}/${TMPZONEFILE}

	# Get a count of the zones from the current update
	NEWZONECOUNT=$( cat ${BLACKHOLEDIR}/${ZONEFILE}|grep "^zone"|wc -l )
	echo "${OLDZONECOUNT} Previous Zones"
	echo "${NEWZONECOUNT} Current Zones"

	echo "Reloading named"
	${NAMEDCMD}

	if [ $? -ne 0 ]; then
		echo "  ... failed! Restoring zone file"
		cp ${BLACKHOLEDIR}/${ZONEFILEBACKUP} ${BLACKHOLEDIR}/${ZONEFILE}

		echo "Reloading old zones in named"
		${NAMEDCMD}

		if [ $? -ne 0 ]; then
			echo "  ... failed again!! You'll want to see to that."
		fi
	fi

	echo "Copying files to other internal network servers..."

	for HOST in "${HOSTS[@]}"; do
	DUMPDEVICE=${HOST}:${BLACKHOLEDIR}
	MOUNTRESULTS=$( ${MOUNTCMD} | grep "${DUMPDEVICE} on ${MOUNTDIR}" )

	if [ "${MOUNTRESULTS}" == "" ]; then
		echo ""
		echo "Mounting ${DUMPDEVICE} on ${MOUNTDIR}"
		${MOUNTCMD} -t ${FSTYPE} ${DUMPDEVICE} ${MOUNTDIR}
		if [ $? = 1 ]; then
			echo " ... failed. Files will not be copied."
			continue
		else
			echo " ... succeeded"
		fi
	else
		echo "${HOSTNAME}:${DUMPDEVICE} already mounted on ${MOUNTDIR}"
	fi

	# Copy the files to ${MOUNTDIR} as a temporary file. On the remote server,
	# we'll manage bouncing named if necessary.
	echo ""
	echo "Copying ${BLACKHOLEDIR}/${ZONEFILE} to ${TMPZONEFILE}"
	cp ${BLACKHOLEDIR}/${ZONEFILE} ${MOUNTDIR}/${TMPZONEFILE}
	if [ $? = 1 ]; then
		echo "... Failed to copy ${ZONEFILE}! You might want to see to that."
	fi
	# Umount the backup filesystem
	echo ""
	echo "Unmounting ${MOUNTDIR}"
	${UMOUNTCMD} ${MOUNTDIR}
	if [ $? = 1 ]; then
		echo " ... failed. You might want to see to that."
	else
		echo " ... succeeded"
	fi
	done

	END=$( date +%s )
	RUNTIME=$(( ${END} - ${START} ))
	H=$(( ${RUNTIME}/3600 ))
	M=$(( ( ${RUNTIME}/60 ) % 60 ))
	S=$(( ${RUNTIME} % 60 ))

	echo "Malware zonefile download on ${HOSTNAME} complete in"
	echo "${H} hrs, ${M} mins and ${S} secs (${RUNTIME} secs)"

	exit
fi

On the “slave” servers, I’m using this…

#!/usr/local/bin/bash

# To put file where named can see it
BLACKHOLEDIR=/var/named/etc/namedb/blackhole
ZONEFILE=malwaredomains.zones
TMPZONEFILE=tmp.malwaredomains.zones

# To control bind
NAMEDCMD="/usr/sbin/rndc reload"

if [ -f ${BLACKHOLEDIR}/${TMPZONEFILE} ]; then
	echo "New zone file exists..."
	# Rename the zone file to back it up
	echo "Backing up current zone file."
	mv ${BLACKHOLEDIR}/${ZONEFILE} ${BLACKHOLEDIR}/${ZONEFILEBACKUP}
	# Rename the tmp file to the name the daemon can find
	echo "Replacing it with the new zone file and removing the temp file."
	mv ${BLACKHOLEDIR}/${TMPZONEFILE} ${BLACKHOLEDIR}/${ZONEFILE}

	# Reload named.
	${NAMEDCMD}

	if [ $? -ne 0 ]; then
		echo "    ... failed! Restoring zone file"
		cp ${BLACKHOLEDIR}/${ZONEFILEBACKUP} ${BLACKHOLEDIR}/${ZONEFILE}

		echo "Reloading old zones in named"
		${NAMEDCMD}

		if [ $? -ne 0 ]; then
				echo "    ... failed again!! You'll want to see to that."
		fi
	fi
else
	echo "No update.  Quitting..."
fi

Last night, my wife called me into the office with an alarming “It says it’s infected with malware!” Needless to say (and yet I’m going to say it anyway) I hurried into the room to see what the hullabaloo was all about.

Sure enough, there was a window exclaiming the existence of not one or two, but quite a few malware infections.

It fooled her, and damn if that stupid pop-up didn’t nearly fool me too! Truth be told, it did, if only for a second. Those malware serving fake malware pop-up warnings are clever.

It got me to thinking.

Then Osama bin Laden was shot in the head, and malware peddlers started leveraging our insatiable appetite for news about it (the sick bastards).

That got me thinking more.

It reminded me of the malware peddlers that took advantage of the quake in Japan recently. Now those are some seriously sick bastards.

Those events all in quick succession and all that thinking led me to this.

A little ditty that downloads the bind formatted zone file from MalwareDomains.com, moves it to where Named can see it, and reloads Named zone files if the download is complete. I’d verify the file if they provided an md5 of the zones file. But they don’t. Not that I could find, anyway.

I don’t even begin to hope to eliminate the risk of malware infected sites, but I think this is a positive step towards cutting off malware source domains which might, in turn, help against sites on legitimate domains that happen to be infected. As of today, May 3rd, 2011, there are nearly 10,000 domains in the latest file. That has to be nearly all of them.

Right?

I’ll try it out for a while and see what happens.

BTW, this only works if you’re running your own DNS. If not, you’re at the mercy of your ISP or whatever DNS you choose to use. There are plenty of options out there, and they’re not all horrible.

First, the script, which pulls down the latest malware domains zones file from malwaredomains.com, fixes some problems with underscores in the subdomains, copies the fixed zones file to the named chroot, and reloads the named configs.

#!/usr/local/bin/bash

# To know where script is running
HOSTNAME=$( hostname )

# To put file where named can see it
NAMEDDIR=/var/named/etc/namedb

# To name file so we know what named seeing
ZONEFILE=malwaredomains.zones

# To have a file for sed to work on
TMPZONEFILE=tmp.malwaredomains.zones

# To get updated file from remote server
URLGRABBER=/usr/local/bin/curl

# To keep quiet while am getting file
URLGRABBEROPTS="-s -S"

# To know where file is hosted
#URL=http://www.malwaredomains.com/files/spywaredomains.zones
URL=http://mirror1.malwaredomains.com/files/malwaredomains.zones

# To control bind
NAMEDCMD="/usr/sbin/rndc reload"

#==============================================================

# Get start time so we can know how long
START=$( date +%s )

# Get directory we're running from
SCRIPTDIR=$( dirname $0 )

cd ${SCRIPTDIR}
if [ $? -ne 0 ]; then
    echo "ERROR: Unable to cd to ${SCRIPTDIR}! AbOrTinG!!"
    exit 1
fi

# If we were executed like "./whatever.sh" - set SCRIPTDIR to the pwd
if [ "${SCRIPTDIR}" == "." ]; then
    SCRIPTDIR=$( pwd )
fi

echo "Script is running from ${SCRIPTDIR}"

# Download the zones file in bind format to a temporary location.
# We don't want to overwrite what we already have until we're sure
# the download worked

echo "Downloading file from ${URL}"
${URLGRABBER} ${URLGRABBEROPTS} -o ${SCRIPTDIR}/${ZONEFILE} ${URL}

# Check for errors.  If the file downloaded, then move on, but if not
# we don't want to reload named without the previously updated
# malware domain list

if [ $? -ne 0 ]; then
    echo "    ... download failed! Error: $?"
    exit 1
else
    # Disable name checking for only those domains with underscores,
    # so we don't have to turn off name checking globally.
    SEARCH='_'
    FIND=';};'
    REPLACE='; check-names ignore;};'

    echo "Disabling checking on domains with underscores"
    sed "/${SEARCH}/ s/${FIND}/${REPLACE}/g" ${SCRIPTDIR}/${TMPZONEFILE} > ${SCRIPTDIR}/${ZONEFILE}

    # Get a count of the zones from the last update
    OLDZONECOUNT=$( cat ${NAMEDDIR}/${ZONEFILE}|grep "^zone"|wc -l )

    # Copy file to final location for named to find
    #(via "include" directory in named.conf)
    echo "Copying file from ${SCRIPTDIR} to ${NAMEDDIR}"
    cp ${SCRIPTDIR}/${ZONEFILE} ${NAMEDDIR}

    if [ $? -ne 0 ]; then
        echo "    ... failed! AbOrTinG!!"
        exit 1
    fi

    echo "Reloading zones in named"
    ${NAMEDCMD}

    if [$? -ne 0]; then
        echo "    ... failed! You'll want to see to that."
    fi

    # Get a count of the zones from the current update
    NEWZONECOUNT=$( cat ${NAMEDDIR}/${ZONEFILE}|grep "^zone"|wc -l )
    echo "${OLDZONECOUNT} Previous Zones"
    echo "${NEWZONECOUNT} Current Zones"
fi

END=$( date +%s )
RUNTIME=$(( ${END} - ${START} ))
H=$(( ${RUNTIME}/3600 ))
M=$(( ( ${RUNTIME}/60 ) % 60 ))
S=$(( ${RUNTIME} % 60 ))

echo "Malware zonefile download on ${HOSTNAME} complete in"
echo "${H} hrs, ${M} mins and ${S} secs (${RUNTIME} secs)"
exit 0

Then, the cron job to update the list on a daily basis:

35 0 * * * /root/bin/malwaredomains/malwaredomains.sh 2>&1 | mail -E -s "Malware Domain Named Update" me@here.com

Then, the blackhole host file that all those zones in the malwaredomains.com download refer to. Careful with this one, and you’ll want to replace the domains with something a little more relevant:

$TTL    86400           ;one day
@ IN SOA ns0.example.net. hostmaster.example.net. (
        2011050100  ; serial number YYYYMMDDNN
        28800       ; refresh 8 hours
        7200        ; retry 2 hours
        864000      ; expire 10 days
        86400       ; min ttl 1 day
)
        NS      ns0.example.net.
        NS      ns1.example.net.
        A       127.0.0.1
*   IN  A    127.0.0.1

Finally, the line in the named.conf file (in my case, in the internal view) to call on the recently downloaded zones file:

include /etc/namedb/malwaredomains.zone

That should do it!

This is what I receive in my inbox after every update (daily for me):

Script is running from /root/bin/malwaredomains
Downloading file from http://mirror1.malwaredomains.com/files/malwaredomains.zones
Disabling checking on domains with underscores
Copying file from /root/bin/malwaredomains to /var/named/etc/namedb
Reloading zones in named server
reload successful
   10116 Previous Zones
   10116 Current Zones
Malware zonefile download on [hostname] complete in
0 hrs, 0 mins and 2 secs (2 secs)