This year has been an interesting year indeed for on-line technology. Being both a voracious consumer of technology and an obsessively dedicated I.T. professional gives me the opportunity to see both sides of the connectivity coin with sometimes blinding clarity.
In the early days of the Internet, it was all just one big happy wired family. People plugged machines into the 'net at research labs, corporations and universities, on ISDN and frame relay lines, even plugged in through dial-on-demand routers from their homes, and all anyone ever worried about was the occasional hacker rooting around for something interesting. People and organizations with good full-time connections would set up email (SMTP) and news (NNTP) servers on the Internet and leave them configured to be openly used by anyone, so that those who could not maintain their own servers didn't have to be deprived of the ability to send email and shoot the bull. The beginnings of the Internet was a sort of hippy geekdom - free love, free information and free email.
Then came the spammers.
As with pretty much everything else in North America or even remotely Western, anything of even the most miniscule intrinsic value can't exist for long without eventually being ruined through commercial exploitation.
Several years ago some insidiously clever bastards realized that rather than spending thousands of dollars in postage, materials, time and labor to send bulk postal mail to people, you can reach thousands or even millions of people via email by abusing someone else's SMTP server, for no more than the cost of an Internet connection and the email address list. It is so cheap and simple that you don't even have to bother trying to narrow down your target demographics - just email everyone on the planet! This unsolicited bulk email or "UCE", for Unsolicited Commercial Email, somehow became nick-named "spam".
This so-called spam spawned a market for a whole new world of software for sending the bulk email, for locating open SMTP servers and proxies to abuse, for harvesting email addresses from web pages and news servers, and for databases of email addresses. The spam monster also created a sneaky and frightening black market not just for your email addresses but also for compilations of related personal information.
If your email address ever appeared even once in a Usenet post or a chat forum, or was ever submitted to a web site or used for e-commerce, the genie was out of the bottle. Even the ISP's themselves sell and trade the very email address you signed up for. Within weeks you could be barraged with as many as one hundred junk emails a day, visually obliterating other correspondence and sometimes even preventing or delaying the arrival of legitimate correspondence.
Nightmare on Email Street.
I have several email accounts for business and for personal use and before I implemented filtering, I would typically receive a total of over one hundred spam emails a day. The most absurd aspect of the spam I receive is that there is little actual variance from day to day. I can't help but wonder, what makes these imbecilic spammers think that their relentless barrage of crap day after day will eventually make me want to buy something from them instead of vomit a burrito dinner, save it in a Zip-Loc(tm) bag, let it ferment in the hot summer sun for a week and then send it to them postage-due?
The unsolicited emails I receive attempt to lure me into all manners of pornography including pedophilia and bestiality, complete with explicit descriptions and lurid photos. They offer to increase my breast size, increase my penis size, enhance my sexual performance, find me sexual companions, restore my hair, whiten my teeth, help me lose weight and quit smoking, spy on my spouse, repair my credit and find me employment. They offer me life insurance, credit cards, mortgages, loans and stock tips. They peddle online gambling. They offer to host my web sites, buy my domain names, and host names on rogue domains. They try to sell me pet food, vitamins, herbal remedies, contraband drugs, illegal cable TV de-scramblers, printer supplies, cell phone accessories, phone cards, gray market and pirated software, those crappy X10 cameras, flimsy tool kits and all sorts of other cheaply made junk. They tempt me with lotteries, free vacations and airline tickets, seduce me into pyramid schemes and flimflams of every guise and of course and guarantee me I can "GET RICH QUICK!" Ironically (and infuriatingly), not only do I receive spam offering to sell me email address databases and spamming software, I even get spam offering to help me stop more spam!!!
These emails come with invalid, forged return addresses and from all sorts of phony names, no name at all or even my own name, and the subjects desperately and craftily try to get my attention. The bids to tempt you or fool you into opening the email vary in their ingenuity, using sender names like "Administrator" or using "Re:" in the subject line as if they were replying to an email you originated. Some of the attempts are downright laughable, like an email I received from "Spy Cam" with the subject "Re: Your Mortgage". More than a few spammers have the nerve to mark their emails with an "urgent" flag. The emails often fraudulently claim to be from "opt-in" mailing lists and include alleged opt-out mechanisms that either don't work at all or guarantee to immediately quadruple the volume of spam you receive. Some spam emails contains coded "bugs", where all you have to do is open it to certify your existence to the spammer and attract yet more spam.
The indiscriminate nature of this spam means that not only you but your young, impressionable children can be relentlessly subjected to advertisements for products and services that would blush even Al Goldstein and Larry Flint. Spam makes no distinctions as to age, sex, race, language, geography, prevailing political system, or any other demographic parameters.
More than just a nuisance.
The spam we receive originates through computer systems all over the world but mostly outside the United States, very often abusing the services of insufficiently protected and thus "open" mail relays, as well as abusing the capacity of the typically expensive data lines that connect those relays to the Internet. The name for this practice is "theft of service" but the spammers are difficult to catch and prosecute, especially over international borders. The transmission headers of the emails are usually forged and/or obfuscated and the path of the emails often hop through several servers, impeding efforts to trace it back to its source.
Spammers can get away with all of this because the economics are phenomenally favorable and what few laws there are have no teeth. Junk postal mail is annoying too, but the formidable costs make it naturally self-limiting. It can cost many thousands of dollars to mass-mail letters or even postcards to a limited and carefully chosen audience, but anyone with fifty dollars can buy a CD holding millions of email addresses and use it to disrupt the lives of millions of people in just a few days. While prohibitive costs make the potential mass mailer think very carefully about targeting his or her audience, the potential spammer can reach half the computer-owning human beings on earth with careless, wanton machine-gun sprays of email at a miniscule cost and with the casual effort of composing a single message. Just a handful of sheep willing to buy the spammer's wares can immediately offset his or her miniscule investment of money and time.
Spam emails amount to economic cyber-terrorism, or at least criminal mischief. Consider the lost productivity a spammer can cause with a mere US$50 investment. Fifty million people now have to delete the junk while reading email that actually matters. Even if it only takes one second for each reader to delete the junk email, society has lost 13,888 man-hours of productivity. At the current measly United States federal minimum wage that's a whopping $76,384 worth of damage from a $50 expenditure. And it gets even worse - that fifty bucks is a one-time expenditure for data that can be used and re-used for months. Used once every other weeks over the course of just six months, a single spammer has the power to inflict upon society damages approaching $1,000,000.00!
In addition to lost productivity, there is a huge direct cost to the recipients of spam, some of whom pay for their bandwidth byte by byte, pay for the storage consumed on the email servers, and pay toll rates and ISP charges for the time consumed on dial-up access. Business and ISPs are forced to maintain larger, more powerful and more expensive mail servers just to cope with the extra load of the spam or to help deflect it.
There are many other detrimental effects of spam, most of which amount effectively to Denial of Service attacks on individual mailboxes and on entire Internet domains. Spam causes recipients' email accounts to overflow and subsequent important emails to bounce. The forged reply addresses and headers cause excessive and futile complaint emails and bounce traffic to popular domains, wasting the valuable resources of those systems and the time of their highly paid system administrators. The costs that individuals and companies are forced to bear as a result of spam will only increase as more spammers peddle their goods, and as technology evolves and your email accounts become tied to expensive wireless and messaging services.
If all this weren't already bad enough, spammers have become increasingly reliant on using operating system vulnerabilities and virus -laden consumer software to plant remote control and relay or proxy software on unsuspecting people's computers, in order to use those computers to hide behind while sending spam email. Aside of the obvious security concerns such as fraud and identity theft, this slows down those victims' computers and hogs the capacity of their Internet connection, often resulting in many hours of frustration and troubleshooting.
When you add it all up you may plainly see that spammers are costing businesses and people around the world literally billions of dollars a year.
I don't like spam!
Apologies to Monty Python.
First of all, join CAUCE (the Coalition Against Unsolicited Commercial Email). It costs you nothing but a few moments of your time. Be an active, informed member of that and any other anti-spam organization you can find, and make sure you write to your politicians and write them often. The CAUCE web site and others even show you how. So far, only half the states in the USA have anti-spam laws, and only one of those states has laws with any kind of enforceable remedies.
Next, start filtering your email.
Luckily the world is Newtonian in nature, i.e. for every action there is a reaction. Just as spam spawned a market for mass-mailing tools, another market popped up in response with anti-spam tools. There are now tools for filtering and managing your email, protecting your email software and computer and your privacy, and there are RBLs and filter subscription services. RBLs are currently in my opinion the most interesting and efficient solution.
An RBL is a Real-time Black hole List - a dynamic list of IP addresses of email servers and networks that are known to support the activities of spammers. With an RBL, when email comes to your provider it gets checked to see if it came from a blacklisted IP address and if so, the email is rejected before it gets anywhere near your mailbox. RBLs can stop up to 70-90% of spam overall, and nearly 100% of the most repulsive varieties of spam. You will never even see the spam and you don't have to buy your own anti-spam software or constantly fiddle with email filters. For more information on RBLs check out MAPS (Mail Abuse Prevention System, LLC) and search the Internet using your favorite search engine for the term "DNSBL" (DNS -based black hole list).
Pressure your business, ISP and/or email provider to begin utilizing RBLs. If your employer balks over implementing filters, particularly for fears of legal repercussions about lost email or actions involving the RBL provider, don't back off. Remind them of their legal responsibility to stop objectionable and personally offensive materials. The word "harassment" may get the wheels turning.
If you maintain your own email server(s), look into utilizing an RBL of your choice and a tar pit as well, to help slow down the spammers for everyone else. There are many free and commercial RBLs with varying policies to suit nearly any personal preference. A tar pit is software that detects spam activity and causes the email server to respond very slowly to the spammer's software and impede their progress.
The mixed blessing of RBLs.
RBLs are not perfect a perfect solution - there is some risk that you may not receive a small amount of honest, legitimate email because it happened to originate from an IP address that was blacklisted (for whatever reason) at the time the email was sent.
The risk posed to the successful delivery of legitimate email is the single most significant factor in the effectiveness of RBLs.
Consider, if an ISP's customers can't reliably send email because of blacklisting, those customers pressure the ISP to take steps to stay off the RBL (by disabling the accounts of customers that were sending spam), or else the customers simply switch their business to another ISP. The ISP responds by enforcing rules to maintain an honest (responsible, spam free) customer base, or is committed to being forever listed in the RBLs and is hopefully forced into extinction as a result.
Businesses endure similar pressure. If a business sends spam (or permits its servers to be abused by spammers, intentionally or not) and ends up blacklisted, its ability to conduct business via email becomes hampered. This keeps most businesses honest and even helps honest business identify when the security of their servers has been compromised!
The end result amounts to a form of blackmail - "keep spammers off of your systems or you and your users will be unable to send email." The system works better when it is more widely implemented (read: when the threat is broader), and it works much more effectively and efficiently than any of the myriad signature-based filtering services or software.
RBLs have two big advantages for recipients compared to client-based and text filters. They can help the recipient email servers stop the spam before it ever reaches a user's mailbox, thus reducing the impact on Internet connection bandwidth. RBLs also help dissuade spammers from continuing to send more junk email, much like slamming the door in the vacuum cleaner salesman's face.
The only other solution that holds any promise is called a Bayesian filter. This is a system that uses an algorithm based on certain word frequencies to intelligently identify spam. Bayesian filters are still in relative infancy and while users report effectiveness nearing 100% and apparent infallibility, its usefulness and accuracy does depend greatly on your habits, and by nature it must be client based rather than server based.
Absolute power corrupts, absolutely.
Block lists - usually based on DNS mechanisms and thus called DNSBL's - are a centralized sort of approach to handling and blocking spam. A handful of people have taken upon themselves the task of watching the flow of junk email on the Internet, and adding the source IP addresses of those emails to a DNS server that they manage. Then recipient SMTP servers can choose whether or not to even accept a connection for email transmission, based on whether or not a DNS query against the DNSBL succeeds.
The different DNSBL's have varying parameters for how IP addresses get listed on their servers. Some merely record the source IP's of proven spam, some record the IP addresses of SMTP servers known to openly relay email (regardless of whether or not they actually have relayed spam yet), some record the IP addresses of servers that aren't "RFC compliant", etc. And some DNSBL's are actually compilations of some or all of the above.
After an eight-week evaluation of eight DNSBLs a few months ago, I chose to employ Osirusoft's DNSBL for my own email server. Of all the DNSBLs I used, Osirusoft's "combined list" seemed the most effective and during the test period caused only one questionable blockage, which was commercial bulk email anyway. I happily used this service for a few months, and then like the owner of a protective but genetically unpredictable pit bull, I got mauled.
Osirusoft is just one man, Joe Jared. Joe works alone, so if you somehow become caught in his trap and you need some assistance to make your way back out, you are entirely at his mercy and subject to the whims of his mood and schedule. If Joe is having a bad day, you're subject to having your email communications with him forwarded to a Usenet newsgroup - news.admin.net-abuse.email - for public flogging, or no response, or both no response AND public humiliation, justified or not.
Joe Jared is a busy and irritable man, so the likelihood of your correspondence being abused is actually quite substantial. Aside of running his own business and his involvement in a seemingly thankless public service chock-full of legal exposure, Joe is also a FidoNet node operator and a shareware author. This could partially explain his bitterness, and anyone who was involved in hobby telecommunications in the 80's and early 90's should be able to sympathize. Sadly, the bitterness appears to have robbed him of his creativity as well as his patience.
So when a client of mine accidentally landed themselves on Osirusoft's DNSBL and I fixed their email servers, I tried using Osirusoft's automated re-testing facilities several times over the course of two weeks. When after all that effort this client's emails were still being blocked, I found out for myself what a unique joy it could be to deal with Joe. It took seven sincere, humble and concise emails of varying persuasiveness over the course of several days before Joe finally noticed his system was broken and fixed it, tersely replied "Kindly find some other place to urinate", and CC'd the aforementioned Usenet newsgroup in his email reply to me, which has the side-effect of attracting yet more spam, thanks so much.
I'm not sure if Joe thinks that hanging his dirty laundry on Usenet is satisfyingly vengeful, or if it is simply cathartic. But the up-side to this is that Joe's CC'ing Usenet in his reply also gave me a chance to openly roast both him and the five or so self-important, self-righteous know-it-all's that had nothing better to do than reply to the post, allowing me to expose them all for exactly what they were. I even squeezed an apology out of one participant, but Mr. Jared remained quietly on the sidelines.
After all this, most of you probably think I dumped Osirusoft like an Israeli flag in a Palestinian Day Parade, but I am still using Joe's DNSBL to filter my incoming email. Although I feel like I'm sleeping with Satan, at least my bed is always warm.
Relentless attacks by spammers in August 2003 shut down the Osirusoft DNSBL for good. Joe just couldn't take the heat any more. Thankfully, dozens of other DNSBL providers have filled those shoes, and with a bit more grace. Shortly thereafter, Joe got on the wrong side of Steve Rombom. I know a little about the Jewish Defense League and about Steve and so all I can say about this is, HOO BOY, Joe. After two painful years of legal wrangling it seems that Joe has finally prevailed but, as in most lawsuits, only the attorneys truly win.
Today I'm utilizing five DNSBLs and one URIBL, the latter helping my email servers block email not by their point of origin, but by checking for the presence of potentially harmful link destinations in the message body. I maintain a similar configuration on a few of my client's sites. Before this, the spam problem got so bad at one client that their most important mailbox was getting as much as ten junk mails per minute. Every day these lists help repel thousands of undesirable emails and with literally no false positives. These lists have also helped a few clients identify when their own servers were being abused, and even helped spot when they purchased marketing data of dubious origin. Spammers will have to pry these resources from my cold, dead fingers.