Improve Your RSS and Atom Feeds

Revision as of 23:28, 1 September 2009

So, you're finally on the syndication bandwagon. You're strollin' and trollin' web forums, making chit-chat about RSS and Atom while debating the finer features of your favorite news aggregators. It's quite the spectacle as heavyweights like Apple and Microsoft line up to join the party. It's quite the hip little scene, and you're right at the epicenter.

Wait, what's that? You don't know what I'm talking about? Here's a little primer about site syndication.

This tutorial, originally submitted by Webmonkey contributor Jason Cook, is a wiki. Portions of it may fall out of date. Please help Webmonkey keep it current by editing it. If you have extra advice, log in and add it.

Syndication Basics

RSS and Atom are formats for publishing syndication "feeds," which are simple lists of news headlines, blog postings or other online content. Today, large and growing audiences of web users are using RSS and Atom-enabled newsreaders to quickly scan and read dozens of their favorite sites at a glance. The syndication craze has caught on big time, as evidenced by the fact that almost everybody -- from The New York Times to the blogger next door -- has started publishing a syndication feed.

If this all sounds new, the best way to learn about site syndication is by reading Webmonkey's " RSS for Beginners". This introductory-level tutorial will get you up to speed on the site-syndication trend.

What We'll Cover

Let's assume that you've got a basic, bare-bones RSS feed built, and now you're ready to make it dance a little. This tutorial will cover some more advanced aspects of RSS and Atom syndication. The techniques described herein, like GZip compression, conditional get and feed validation, will ensure that your feed is as lean and clean as possible. We'll also run through some optional tags to improve the quality and usefulness of your feed. Plus, we'll explore some extraneous elements, like pinging and categorization, to get your feed in front of the right audience.

So, let's get started by talking about compression.

Lose Weight, Save Money with Compression!

Everybody likes it when webpages download quickly. That's why size optimization matters so much for commercial websites -- smaller sites (in terms of kilobyte-size) make for faster downloads, and happier users.

Speed is rarely an issue with RSS and Atom feeds. They're mostly free from fancy HTML layout and modem-clogging graphics, and usually download so quickly that users can't tell which feeds are fatter than others. In fact, serious RSS surfers usually prefer the feeds that carry a few extra kilobytes of text, in the form of unabridged articles or lengthy abstracts. Scanning a skimpy set of headlines in a news aggregator isn't as satisfying as getting the full story.

Size still matters, though, especially if you publish often enough that a relatively small set of subscribers can significantly increase your bandwidth usage. If you pay for your bandwidth like most of us, this may result in increased costs.

Thankfully, there's a trick from Webmonkey's Site Optimization Tutorial that works well in this case. It's called gzip compression. GZip shrinks the source code of your file before sending it out over the wires. So, while newsreaders will still be banging on your site, they'll be pulling smaller files. Most full-featured newsreaders accept GZipped files and understand how to decompress them.

Normally, GZip compression isn't something you put in your code. It's usually configured through the web server that's dishing out your pages. In fact, your sysadmin may already have GZip compression running somewhere on your site. The fastest and easiest way to check your pages (and your feeds) is by visiting the GZip tool at Leknor.com. You can enter in any URL, and the tool will report whether or not it's providing HTTP 1.1 with GZip compression turned on.

If your web server isn't serving pages with HTTP 1.1 and compression turned on, check with your sysadmin. It's possible they're compressing files with the .html extension, but forgot to add the .rss, .xml or .rdf extensions. It's also possible that they're just really behind the times, and still running an older HTTP 1.0 server.

If your sysadmin refuses to get with the program, take matters into your own hands and create dynamically GZip-compressed files using PHP. Thankfully, it's a teensy code addition, via the ob_gzhandler class. You can read some user experiences with this method at Textism and at The Learning Journal.

What kind of compressions levels can you expect? The average compressed feed is less than a third a size of a regular feed. Now those are results.

Conditional Get

Here's an even finer way of lowering your bandwidth costs: Don't dole out the same stale file more than once per user.

Again, the downside of auto-updating newsreaders is that they continually knock on the door, wanting to see what's new. But this doesn't mean you have to serve up a full copy of your feed each time they come calling. Instead, you can use a method called 'Conditional Get' to broadcast a curt '304' response code, which says "I done given you that file already, and ain't nothin' changed." (SFX:door slamming shut)

How does that work, exactly? Accompanying every file it transmits, your HTTP 1.1 server attaches some unseen (by you) headers. These include a 'Last-Modified' date, and a unique value called an Etag. Well-behaved newsreaders hang on to these, and on subsequent visits ask only for files containing a new ETag or a fresher 'Last-Modified' date.

As a feedmaster, your chore here is to ensure that your 'Last-Modified' dates and Etag values are set correctly. If you've confirmed that your server runs HTTP 1.1, and your publishing/blogging tool publishes only simple, static files, you're golden.

Things get tricky if your setup relies on dynamic files or otherwise regenerates your RSS/Atom file between content updates. As you'd expect, the problem here is that your 'Last Modified' date won't stay the same, and newsreaders will be fooled into thinking there's new material to be downloaded. The best resource for tackling this problem is Charles Miller's in-depth tutorial, "HTTP Conditional Get for RSS Hackers".

Everybody Needs A Little Validation

From the Common-Knowledge-Worth-Repeating Department: Validating feeds is more important than validating webpages, because browsers are intentionally built to be forgiving of sloppy, shoddy code. Can you imagine what surfing the web would be like if they weren't? RSS-parsing scripts and shareware newsreaders aren't necessarily so tolerant or robust, however. In most cases, well-formed XML is a requirement, not an option.

The FEED Validator is your one-stop validation shop for both Atom and RSS. Like a spell-checker, the validator tells you if and where your feed shows coding errors.

Perhaps you've validated already when you first set up your feed. Well, it won't hurt to do it again. It's a good idea to validate every time you alter your feed's structure.

Now, there's nothing wrong with a bare bones, factory stock, default-settings feed file. But one need only witness MTV's Pimp My Ride! to know that the fine things in life come only as after-market add-ons. With that in mind, let's explore some optional elements and add-ons to transform your dull, quotidian feed into a tricked-out, Full Custom Markup syndication lifestyle status symbol.

RSS 0.9x/2.0's <managingEditor> and <webMaster> elements:

Contact information is always helpful, and yet a minority of feeds even bother to include it. If you've avoided these tags because of spam worries, point to a freemail account of some sort, so you can still check your messages once in a blue moon.

You'll find implementation details for these elements in the RSS 2.0 spec.

Categorization and Metadata

RSS 0.9x was branded "Really Simple Syndication" for a reason: It's an easy-to-learn format with a simple, tightly focused set of features and tags. Because of this, there is not an abundance of metadata featured in the core set of tags.

If you're itching to include advanced metadata like geographical area, or subject/category information to your feed, workarounds exist. Third-party sites like Syndic8 will associate your RSS/Atom feed with a certain webpage's URL, and allow you to embed metadata within the <head> of that webpage. This will associate any given metadata with your feed.

The Syndic8 directory currently indexes these three additional metadata elements:

• Geographical Location (Latitude/Longitude, based on GeoTags)
• Geographical Names (Thesaurus of Geographic Names)
• Subject / Category (based on the Open Directory Project)

For step-by-step instructions on how to track down this information and associate extra metadata with your syndication feed, visit the metadata tutorial at Syndic8.com.

Pings and Custom Graphics

If you use a blogging tool such as Blogger, TypePad or MovableType, you'll notice the option to "ping" certain sites whenever you add new information to your blog. Pinging informs certain RSS-savvy portal sites and search engines (like Technorati) that your feed has been updated and ready to be added to their database. The advantages for you, of course, are increased visibility and traffic.

Traditional search engines, like Google, only glance at your site on a regular, fixed schedule. It can take up to a week for your site changes to register in their search index. In contrast, these RSS-savvy spiders will attempt to visit your site's feed whenever it is altered, thereby providing something closer to real-time search.

There's also an excellent site called Ping-O-Matic that will ping over a dozen services on your behalf, and at no cost. The single burden for you is that you are required to manually enter your URL on their home page. Gee, sure is tough, eh?

Custom Graphics and Favicons

The idea of branding your feed with a tiny graphic (in the venerable dimensions of 88x31 pixels) is a somewhat quaint holdover from RSS's earliest days. It still offers an opportunity to sneak some style into a text-centric medium. Serving up a graphic logo is primarily useful if other websites are integrating your feed's headlines. Displaying your <image> logo is an easy, standard method for others to provide attribution to your site.

The <image> tag exists in both the RSS 1.0 and RSS 0.9x/2.0 formats.

Also, you shouldn't rule out favicons. They're not just for bookmarks anymore. While never mentioned within your actual RSS or Atom code, many newsreaders make a point of displaying a domain's favicon within their interface. If you've been holding off making a favicon for your site, here's that impetus you were waiting for.

Content Always Wins

That just about wraps up our advice for juicing up your syndication feeds. However, there is one more very important piece of advice that I feel compelled to pass along.

The techniques we discussed in this article should help you nip and tuck your syndication files into very pretty RSS or Atom feeds, but only from a technical standpoint. Don't forget that in a text-centric medium like syndication, content is still king. File compression and favicons are no substitute for stories, opinions or ideas that deserve an audience.

Nothing trumps good writing! If you take the time to craft a great entry plus a thoughtful title and a clear description, that will impress users more than any 88x31 logo can.

So get writing. Or get coding. Or, better yet, both!