The Sorry State of Spam and What You Can Do About It

Again, from the very helpful Web Worker Daily.

A year ago, the Messaging Anti-Abuse User Group put out a report stating that over 80% of email is spam. Judging from my unfiltered inbox alone, I’d say that’s very true. Many say that email is dead thanks to the overwhelming influx of junk, but web workers still rely on it too much to say eulogies.

We can maintain control over our inboxes. Regardless of the email client or server you use, here’s an overview of the tools we have to fight back against those that insist we need larger sexual organs, free software and pre-approved bank loans.

First, admit that you are powerless to stop spam, at least at the individual level. You can make sure that your email address never appears anywhere a spammer can get it, sure. But how realistic is that? Legislation? Maybe. For now, you just have to get through your inbox.

For the sake of this article, we’re not going to talk about Exchange or corporate email filtering. If you get your email through a corporate server, you should have an IT person to talk to. The rest of us are usually on our own.

Web-based email and spam filtering inboxes

Gmail’s built-in spam filtering is now very good and getting better all the time. You can’t do anything to train it for your specific inbox, but Google does learn by its mistakes improving the filtering for everyone. Many have been known to send their email through Gmail simply to take advantage of its intelligent filtering. Since you can’t train it directly, you do need to skim the Spam folder from time to time to make sure you’re not missing legitimate mail. Yahoo’s free email accounts include SpamGuard, similar in function to Gmail’s filter but the jury is still out on whether it’s as effective. Most say not. With the paid Yahoo email accounts, you get the option of training/configuring the filter.

Along the same lines, paid services like SpamCop provide a clean, spam-free email address. You just forward your “dirty” email to it and SpamCop only delivers the filtered results.

Server-side filtering

If you have direct access to the server that manages your email, you can install and configure tools that will catch the spam before it hits your inbox. Most of these tools use Bayesian filtering to analyze the message on many levels and determine whether or not it’s spam. The advantage here is that you can get your mail via whatever desktop, mobile or web-based client you choose and you never have to deal with add-on software locally. If you pay per byte downloaded, this can save you a great deal of money. This is also your best hope if you get your email on a handheld device. A list of many different options can be found here. For Apache servers, SpamAssassin is a top choice.

If your email is hosted on a shared server, your service provider may have server-side filtering available for your account. Dreamhost and Pair Networks use SpamAssassin, for example. You have very little control over these filters, setting its aggressiveness with a slider. Too aggressive and you’re trashing legitimate mail, too lenient and you’re still filtering spam in your inbox. They are typically all or nothing, with rudimentary black and whitelist support at best. Having tried the server-side filtering tools at many different shared web hosts, I have yet to find one that I am truly satisfied with.

Client-side filtering

If you use a desktop email application chances are that it came with a spam/junk filter that you have to spend time training to get it to work effectively often with mixed results. There are 3rd party add-ons for desktop clients that work far more effectively, with finer control over the options. Brute force blacklists are useless now. For Outlook, the open source SpamBayes is outstanding, but it takes some tweaking to work well.

If you want set-it-and-forget-it convenience, I would highly recommend Cloudmark Desktop at $40/year. It installs as a plug-in to Outlook or Outlook Express and relies on its community to effectively filter spam. It works on the premise that spam to you is spam to everyone else. Once a message is marked as spam by one user, that information is immediately updated in the network so by the time you received the same piece of junk, Cloudmark knows it’s spam and filters it away. The more you use the filter, the more your selections are trusted by the system. The only emails that Cloudmark may have difficulty with are opt-in newsletters, where community members “block” the email rather than delete. Overall, Cloudmark has been over 99% accurate in keeping my Outlook inbox free of crud.

On the Mac OS X side, the best 3rd party add-on by far is SpamSieve. It’s a powerful Bayesian filter that just works and is well worth the $30 fee. SpamSieve works in Apple Mail and most Mac OS X email clients including Entourage and MailSmith and now, at last, Mozilla Thunderbird. The application installs a plug-in into the email client and runs alongside it. With minimal training, it’s not unheard of for SpamSieve to be greater than 99.8% accurate all the time.

Challenge/response filtering

Have you ever sent email to someone with an Earthlink account and gotten one of those annoying messages back asking you to prove that you’re a human being? These spam fighting tools put the onus on the sender to prove to the recipient in an extra step that they are not a spammer. It certainly works, since no spammer will take the time to type in the captcha to get their email through. Unfortunately, you are relying on the sender to take this extra step which they may not always do. If a prospective client wants to talk to you about a job, do you really want to take the chance that they won’t jump through your verification hoop? If you must, SpamArrest is a popular choice. Others here.

The marvelous mayhem of Flickr and YouTube

In the olden days (a few years ago), posting photos to the web was a bit of a chore. And when you did post them, the images lived on your separate web space -- far from other photos posted by other folks around the world. Just those few years ago, posting videos was a distant dream, as bandwidth (the speed of your internet connection) and storage space was expensive.

Two web phenomena of the past few years have addressed both challenges, adding a whole new world of opportunity, connection, and social interaction in the process.

Both sites, Flickr and YouTube, are well beyond mere dumping grounds for photos and video content. They are social networking systems, designed to encourage users not only to post their content and browse the content of others, but to connect, comment, and cluster the content of others in a hundred different ways.

Flickr
Flickr is a photo-sharing site with both free and ''pro'' account options. An upgrade to the "pro" account gives you unlimited storage, uploads, and the like for about $2 per month. Best to try the free account until you know you need more. Essentially, Flickr is a bundle of software and web scripts that help you upload digital photos to your account (through a web browser, by e-mail, or even from your photo-ready mobile phone), and then add elements to those photos like captions, text, tags, and keywords. You can invite anyone to view your photos, or you can limit access to a select group.

The cool part comes once the photos have been posted. Flickr has features to help you share your photos with friends and colleagues, and to view the updated photo streams of your friends as well. If you open up your photos to the world, then a whole world of users can find your images, comment on them, subscribe to your photo stream, or add you as a "buddy."

Having such a large user base contributing and browsing images makes Flickr an addictive place to visit, with photo galleries of the ''most interesting'' images on the site ("interestingness" is determined by how many links, comments, tags, and visits an image gets each day), and connections with amateur and professional photographers.

As a socially active site, Flickr is also an obvious ground for marketing people, events, shows, and other entertainments, which makes it a natural place to advance your organization, your artist, or your events. Imagine creating a Flickr account for your performing arts venue or your artist agency, posting promotional photos as well as images from the most recent events and performances. Flickr offers easy ways to include your evolving photo stream in your own web site, on weblogs, or anywhere else on the web. And the effective use of image tags ensures that your events or your artists show up on major search engines like Google.

For an example, check out the Flickr account of the Kimmel Center for the Performing Arts.

YouTube
On the heels of Flickr came YouTube, a web site and system that does many of the same things, but with full-motion digital video. Users can post digital video to their free account, adding captions, descriptions, and keyword tags, and also view and connect the videos of others.

The synergy of this idea made YouTube the site to watch in 2006, as user-generated videos flowed in surrounding major news stories (London bombings, Hurricane Katrina), often with more speed and depth than network news could manage. The low barriers to entry also made YouTube a massive cultural phenomenon, with the rise of "video bloggers" (individuals posting video diaries of their lives, their opinions, and their creative expressions) and social networks built around favorite funny videos or long-lost film -- think America's Funniest Home Videos, on steroids.

For the arts presenter, manager, or artist, YouTube offers opportunities that were, until recently, too expensive and complex to consider. With a YouTube account, any arts presenter, manager, or independent artist can post videos for the world to see -- demo songs, performance excerpts, guided tours of a facility, video interviews with artists, and on and on. And as with Flickr, once content is posted, it can be dynamically included on your own organization's web site, promoted through e-mail, and linked to a larger world of arts lovers, business colleagues, or enthusiasts.

Already, Hollywood studios are posting movie trailers on YouTube, encouraging fans to spread the word about upcoming releases (remember Snakes on a Plane?...sorry if you do). Imagine a similar network of artist performance excerpts, interviews, and documentary footage, all connected to performing arts organization web sites at no cost to the agent, the artist, or the presenter.

A Caveat about Copyright
As Flickr and YouTube have grown, so has the challenge of protecting copyrighted creative work. Still images and moving images with sound often have complex ownership issues. And just because you have a digital copy of something, doesn't give you the right to post it for the wider world to see.

So, as you dip your toe into the content-sharing worlds of these two sites, be sure you have the rights or the express permission of the various owners of the content to post their work (the composer, the artist, the designer, the producer, the union, and anyone else who played a role in creating the image or video content).

Sharing is a wonderful thing. But you can only share what's yours.

[Ed note - for those visiting YouTube for the first time and wondering where to start browsing, Terry Teachout has compiled a wonderfully absorbing index of hundreds of amazing performances uploaded onto YouTube including Maria Callas, Louis Armstrong, Blossom Dearie, Edward Elgar, Leadbelly, Astor Piazzolla, Frank Zappa, and Stevie Wonder, among many many others. Just follow the link and scroll down nearly to the bottom and look on the right.]

International Collaborations Made Easy with Free Phone Calls and Online Storage

I discovered the power of online chat/telephony service and storage a couple of years ago when making a multi-media dance work called Efficiency – which included an original score by London-based artist – Scanner (aka Robin Rimbaud). To undertake this transatlantic collaboration (I live in Washington, DC), we quickly realized we needed a digital solution to bridge our distance. With iChat suddenly we could have free hour-long transatlantic conversations (which I once considered unimaginable given international calling rates)!

Another great online telephony service – Skype had just come out as well, which works well for both Macs and PCs; is easy and free to download; and can even be used to dial landline or mobile phones (for a per-minute fee). You can use a web cam (or other digital camera) to suddenly have video conferences.

Our extended conversations were supplemented by forwarding one another files through online storage and delivery services. I was able to record videos of dance rehearsals; upload them to my computer; and then, edit and compress the video using iMovie so that I could forward them to Robin via online storage. And, he could forward me mp3s to try out in rehearsals. He eventually scored the first draft to a video of spliced together video projections and the live dancing in the final work. Once, we finalized the score, he uploaded the highest quality file (“apple lossless” – so we could play it on a theater’s sound system). And, I downloaded it to burn to CD and play during the performances.

There are several online delivery services that are free – like DropLoad or YouSendIt. The beauty of them is that you can transfer a large file easily without having a website. You upload the file; the service creates a unique URL; and it sends the URL via email to your contact, who can then download the file by simply clicking on the URL through their web browser. Since then, I’ve discovered MoveDigital – a fee-based service – which provides the same concept but in a more versatile way. You have a permanent account where you can share and store files in a more consistent way. (MoveDigital also creates a unique URL for each file.) This provides a method of having permanent storage for very low cost providing high-bandwidth connectivity (The basic plan is about $10 for 1GB of storage for a year). I now use MoveDigital on my site to deliver video content – but it can also be used to simply share files.

The applications are endless. An agent or tour manager can meet with venues by teleconference and then quickly follow-up the conversation by providing images and videos of an artists’ work through online storage. Imagine being able to see a quick video of the set that you will need to load into your space in a few weeks or seeing the in-progress dance you’ve commissioned while speaking to the artist in Brazil. You may even be able to get in touch with artists and companies in hard-to-reach places (somewhere like Bangladesh – as the Grameen Bank proves). Maybe like me, you’ll wonder what you ever did without it.

Ed. note - Skype just announced an unlimited yearly calling plan from skype to any mobile or land line phone in the US (currently free in the US, and a per-minute charge to other countries). Starting in 2007, unlimited yearly calling in the US and Canada will cost $29.95, $14.95 if you purchase before January 31, 2007).

If you're interested in learning more about how to transfer large files, see JoRoan's post about Pando.