Now blogging at diego's weblog. See you over there!

The first part of this introductory guide was basically about publishing, but there is a second component to weblogs, perhaps as important, to cover, that of reading weblogs.

Note: For those that already know about weblogs, syndication, etc., I will greatly appreciate any feedback on this piece. This is a bit more technical that I would have liked, but there are some issues that, in my opinion, can't be ignored. If you have ideas on how this can be improved for end users (both technical and non-technical), pointers to other descriptions that they can go to and get a different take on this, please send them over. Thanks!

Moving on...

The need for syndication

Wait a minute (I hear you say) what do you mean reading? Don't we use the web browser for that? What would I need to know about reading weblogs?

Well, the answer is, technically, web browsing is just fine. But there is another component of weblog infrastructure that is quite important today, and that will probably become even more important as we have to deal with an ever-increasing number of information sources. This component is usually referred to as syndication. Syndication is also usually known as aggregation or news aggregation. The exact definition of the concept, or how and what it should be used for, is something that people could (and do) discuss ad infinitum, but without getting into specifics we can say that at least everyone agrees that certain things represent the idea of syndication or news aggregation quite well.

In the "hard copy" publishing world, syndication implies arrangements to republish something. Popular newspaper comic strips, for example, are usually syndicated, as are some news articles. While the meaning is similar in the web, it is primarily concerned with the technology, rather than with the contracts, or syndication agencies, etc.

Generally speaking, we could say that syndication is a process through which publishers make their content available in a form that software (as opposed to people) can read.

That is, if a site is supports syndication, and you are using appropriate software, you can subscribe to a certain site using that software. This allows updates on the site to be presented to you by the software, on your desktop (or web site that you use for that purpose) automatically.

This means that you can forget about checking certain websites for updates or news: the updates and news come to you.

Syndication is a dry, unassuming word for a powerful concept (as far as the web is concerned at least). It ties in together many ideas, and it is instrumental in sustaining the 'community' part of weblogs that I talked about in part one.

Why?

For an answer, let's go to an example. You have started your weblog, and you have been running it for a bit of time. You have found other weblogs that you enjoy reading, or that you find useful; you are also reading weblogs of friends and coworkers. Very quickly, you might be reading maybe ten or fifteen different weblogs. Additionally, you might also regularly check news sites, such as CNN or the New York Times. Suddenly, it's difficult to keep up. Bookmarks in the browser don't seem to help anymore, and you find yourself checking sites only every so often. Sometimes you miss a big piece of news that you'd liked to hear about sooner---or sometimes you find yourself wading through stuff simply because you haven't kept up. If you are a self-described 'news junkie' (as I am) you might already know about this problem, since keeping up with multiple news sources is also difficult. But with weblogs, the problem is greatly amplified: weblogs put the power of publishing on the hands of individuals, and as a result there are millions of weblogs. There are simply too many publishers. The problem of just 'keeping up' with what others are saying becomes unavoidable.

This is the problem that syndication solves. And the software that does the magic is usually called an aggregator.

Simply put, an aggregator is a piece of software designed to subscribe to sites through syndication, and automatically download updates. It does this regularly during the day, at intervals you can specify, or only when you are connected to the Internet. If the aggregator is running on your PC or other device, once you have the content you can read it in "offline" mode (unless the aggregator is web-based, which will require connectivity to the Internet at all times). For a more detailed take on what aggregators are, I recommend you read Dave's what is a news aggregator? piece. As usual in the weblog world, there is discussion about these definitions, see here, for example, for comment on Dave's piece.

A word of caution before we go on: for non-technical people, the issues surrounding syndication, aggregators and such can appear to get complicated if you start reading some of the links I provide here. There are acronyms and terms used here and there that you might see, such as "XML", "RSS", "RDF", "namespaces", and so on, that can be confusing. Let's skip that for the moment. I will (try) to go into them below (when necessary), in the section below, 'using your aggregator'.

Which aggregator?

Aggregators (or "news aggregators") come in different "shapes and sizes", and there are two main categories of aggregators on which everyone generally agrees on: 'webpage style' and 'email style' (also referred to as 'three-pane aggregators'). 'Webpage style' aggregators present new entries they have received as a webpage, in reverse chronological order (and so the end result looks very much like a weblog on the web does, but of pieces that are put together dynamically by the software). 'Email style' aggregators generally display new posts as messages (also in reverse chronological order) that you can click on and view on a separate area of the screen.

As in other cases, there are good arguments for preferring one over another, and in the end it comes down to personal choice. Reading different weblogs you might find people that are for one or for the other, and other people propose to do away with the whole thing and come up with something completely new. As with other things with weblogs: reading different opinions, and coming to your own conclusions is best. This is probably good in life in general :) but with
web-related things it becomes so easy to do that you generally end up doing it, sometimes without realizing  Look on search engines, other weblogs you like, leave comments, ask people you know, then try some of the software out. You'll find the one you prefer in no time (and, likely, as your usage changes and you have different needs, you might end up switching from one to another).

As it is the case with weblog software, all aggregators are invariably free to try, and many of them have to be purchased after a trial period (usually a month). Aggregators and weblog software are complementary, you could use both, but you could use one and not the other. It's quite possible that there are more people that use aggregators than people that have weblogs. (Certainly there are more people that read weblogs than people that write them).

If you have a weblog, chances are you also have a news aggregator already as well, because some weblog software includes news aggregation built in (as you'll see below). There are lots of news aggregators (and by lots I mean more than fifty, probably a lot more), and more on the way. Additionally, the underlying technology for syndication is simple enough that many software
developers implement and use their own aggregators.

This means that I can't possibly list all aggregators that exist here, and besides, there are other pages that do this already, such as this one, this one, this one, or this one. As it was the case with weblog directories, no listing of aggregator software is 100% complete (and probably can never be). However, I will mention a few aggregators that I know about and have tried myself, or have seen in action (and, in the case of clevercactus, that I developed :-)). (Lists in alphabetical order).

Some webpage-style aggregators

• Amphetadesk (multiplatform)
• LiveJournal (multiplatform -- server based, the aggregation feature is not included in the free version)
• Radio Userland (Windows and Macintosh)

Some email-style, or 'three-pane' aggregators

• clevercactus (multiplatform)
• FeedDemon (Windows)
• NetNewsWire (Macintosh)
• NewsGator (Windows - requires Microsoft Outlook)
• NewsMonster (Windows)
• NewzCrawler (multiplatform)
• Straw (Linux)

All have one or two distinguishing features that make them unique. In the end, which one works better for you is all about personal taste and work patterns. Check out the aggregator listings I mentioned above, and look for something that grabs your attention. Try them out, and see which one you like best. Note: in many cases, aggregators are ongoing projects. Some are open source and are updated often. My advise to save time in choosing an aggregator is to go for the simplest route possible at first, and then with time try out new things. For example, if you're a paying-LiveJournal or Radio user, it will probably be better to use the built-in aggregator at first, especially as you try to find your way around all of these new concepts. If you want to try other options, or you use different weblog software, I think that a quick glance at the webpage of a product is usually a good indication of how much knowledge you need to set up something: if you don't understand what the page says, it's probably not for you. It all depends on your knowledge and how much time you want to spend on it. But, by all means: if you have the inclination or the time or both, spend some time looking at the different options. You'll be surprised at some of the cool features that some aggregators have, even if they are sometimes 'experimental'.

Using your aggregator

Once you've installed an aggregator (or decided to use the built-in aggregator of your weblog software), it's time to subscribe to some feeds.

Feeds (or newsfeeds) are usually the name given to sources of information (used by aggregators) to obtain the content they display. Feeds are technically similar web pages, like those that are displayed in a web browser. Web pages, however, are written in HTML (HyperText Markup Language) which is designed to create pages readable by humans. Feeds, on the other hand,
used for syndication, are intended to be "read" (or rather, processed) by software, and so they have different type of information, are more structured and strict in the data they can contain. Feeds are written using a language called XML (eXtensible Markup Language) using a de-facto standard "dialect" of it called RSS.

Aggregators let you 'subscribe' to to these feeds in different ways. Most pages identify the feeds as 'Syndication', or 'RSS', or 'RSS+version number'. (See the next section if you'd like to know more about these differences). Many pages have an orange icon that says "XML" like this one: . Depending on the software you are using, subscription itself can be easier or more difficult. In all cases, the following set of steps will work to subscribe to a feed:

• Find the link on the page that says "Syndication", "Syndicate this site", "XML", "RSS", etc.
• Right-click (or press-hold in Macintosh) over that link. Your browser will show a menu of options, and one of them will be "Copy Link Location" or "Copy Shortcut". Select that option.
• Now go to your aggregator and find the option to Add or Subscribe to a new feed. Select it and when you are requested to type in the URL (link) of the feed, right-click (or press-hold in Macintosh) again on the field and select "Paste". This will make the URL be pasted on to the field. If right-click doesn't work, you can try with keyboard options: Ctrl+V or Shift+Insert on Windows, or Command+V on the Mac.

Now, just to be clear. These instructions are the most basic of all, and supported everywhere, and they are important so that you can use them "when all else fails". Many aggregators come built-in with a choice of feeds that you can subscribe to with only the click of a button. Many aggregators allow you to type in simply the URL for the page you are visiting (as opposed to that of the feed) and then discover the feed for you. Others also establish a "relationship" with your web browser so that when you click on the icon or link for a feed, they give you the option of automatically subscribing to that feed.

Okay, now for a bit of a detour. If you'd like to know a bit more about RSS and related technologies and have an interest in the technical background, or are technically proficient, please read the next section. Otherwise, skip to the following section.

Ready?

Okay, tell me more about RSS.

Before I start: this is a highly charged (and even emotional) issue in the weblog developer community. People have very different opinions, and this is just my take on the situation. By all means, go to different search engines and search for "history of RSS", "history of syndication", "RSS politics" and similar terms to find pointers to different sides of the argument.

Politics? Did I say "Politics"?

Yes. Yes I did.

Sometimes people mean different things when they say "RSS". Some people see it only as a way to syndicate web content. Others see it as a way to pull all sorts of information into clients. There are different opinions as to how it should be used, how it should do what it does, etc.

In the majority of cases RSS stands for "Really Simple Syndication" but you might come across other places where it is described as meaning "RDF Site Summary" (RDF, which stands for "Resource Description Framework" is yet another XML dialect, that is more flexible, but also more complex). I prefer to separate them clearly and call RSS-based feeds RSS feeds and RDF-based feeds RDF feeds, but I might be in the minority (So when I say "RSS" I mean Really Simple Syndication, not the RDF-based format). There is another syndication format being developed at the moment (also XML-based) called "Atom". Additionally, there are different versions of RSS: 0.91, 0.92 ... 2.0 (the current version of RSS is 2.0)... and RDF-based syndication is sometimes called RSS 1.0 (yes, this last one in particular is quite confusing). These are various formats for syndication. If we lived in a perfect world, we'd only have one format. But that is not the case.

I can imagine you're thinking: So, even if I know about technology, why do I care about all of this "XML mumbo-jumbo"?

Well, if you start your own weblog and begin to discover new weblogs and new feeds, and are curious about the technology, more likely than not you will read about this, about people passionately arguing about these things, mentions of RSS of this version and that, and so on. And so it's a subject that can't really be completely avoided. If you're interested in knowing more, not telling you about this would be like pretending that you can fly across the Atlantic and think that you'll never have to know about the fact that you are likely going to experience some kind of delay on departure.

But, as the Hitchhiker's Guide to the Galaxy says: Don't panic. :-)

More specifically, I'm mentioning this for two reasons:

1. Because you, as a user (that is nevertheless aware or interested in the technology behind this), are likely to encounter this in subtle forms. For example, you might go to one news site and see that they say they provide "RSS 0.91 Feeds". Or you might see the XML orange icon shown above. Or you might see they say just "RSS", or "RDF". You will quite possibly see mention of all of these names and acronyms when you're looking at aggregator software. So, in seeing all this, you might wonder: is one better than the other? Which one should I choose? To that I'd say: start by not worrying too much. RSS is the most common format by a mile, and that is good because it's the one you'll be most likely to encounter and so you won't have to think about this much. Additionally, all aggregators support the most used formats, and many of them support all the formats in existence. In general, you don't really have to even know which of these formats is actually being used. But once you get a grasp on things, it might be a good idea to read about the history behind these differences and make up your own mind about them. (If you are a software developer, and know about XML, good starting points are Dave's RSS 2.0 political FAQ and Mark's What is RSS? and History of the RSS Fork. Beware: these discussions tend to get quite technical very quickly).
2. Because as you're perusing weblogs or news sites, it's better to be aware of what these things mean to avoid getting confused. Once you have this bit of information, it becomes easy to look for the XML orange icon, or some link that says "RSS" or "RSS+version number" or RDF or whatever.

Again, if we lived in a perfect world only a few people would ever have to deal with this stuff. But this is relatively new technology, and we are still trying to figure out all of its uses, and in some cases, what it is exactly. The important thing is, in my opinion, that you as a technically proficient user don't feel as if things are beyond your grasp. My pet peeve is when people using say "I don't know what I did, it seems I broke something". But when (say) the washing machine stops working, we never say "I broke the washing machine" simply because we were using it. We say "the washing machine broke down". So why is that? I can think of many reasons: error messages in computers generally put the burden on the user, for a start. But regardless of that, what I would say is: if something seems complicated (like all of this "XML mumbo jumbo") it's not a problem with your knowledge of computers. It's our problem, a problem of the software developers. (If you get involved in the technology or the community in any way, then it will be your problem too :-)).

And so what? You ask. Well. Weblogs allow a new level of interaction. You can make a difference. Perhaps for the first time ever, users can actually influence and participate directly in the creation of the tools they use everyday through the tools they use every day. So if there is something that is difficult to use, something confusing, it's likely that you can find a weblog or reference for the software author(s). Post a comment. Write your own post about it. Get involved if you can, and by that I don't mean 'develop software'---simply giving opinions and ideas is a good start. People will listen, and the problem might even be fixed!

Now back from the technical depths of this section, and to simpler things.

So how do I find these 'feeds'? And how do I create them?

First of all, let's deal with feed creation. Just as your weblog software automatically generates the HTML page that is displayed in a browser (when you post an entry), most weblog software also generates the feeds for you, and places a link for the feed in your homepage. All of this is done automatically by default---if you are not sure of whether or how this is happening, check your weblog software's help page for "Syndication" or "RSS" and you should be on your way.

Finding feeds to subscribe to is not so difficult. If you're reading other weblogs or you find one of them that looks interesting and would like to keep up with what they are writing, just look for the link or icon that identifies their feed and subscribe to them "as you go". In some cases, the aggregator software will come pre-subscribed to some feeds, or will suggest new feeds to subscribe to. The methods used to find weblogs (mentioned in part one) apply to finding feeds as well, for example, both Technorati and Blogstreet (as well as other sites) allow you to find new weblogs, and hence new feeds to subscribe to. There are "Feed directories" like Syndic8, that allow you to find feeds of certain topics easily. Finally, there's Feedster which is a cool search engine that deals specifically with syndication. All the results in Feedster come from feeds, and so lets you not only look for information but also find new feeds that you'd be interested in looking at.

At the beginning you mentioned news sites. Are there 'feeds' for news sites too?

Yes. Many news sites and organizations today support news feeds. Examples: the BBC, Rolling
Stone Magazine, and News.com. Look in your favorite news site, or use the feeds recommended by your aggregator (if any), or use some of the directories mentioned in the previous section to find more.

Why did you say that syndication is 'instrumental' for weblog communities?

I think that weblogs are cool but syndication+weblogs is really cool. It's a case of 1+1 = 4. Because syndication allows you to subscribe to many sources, you can keep up to date with a lot more and so they allow to maintain people up to date easily on what others are doing in their particular community. Things like Feedster and Technorati reinforce the "loop" that feeds create. These loops are "loosely coupled," connected through links and with people notified of updates through feeds, both done in unobtrusive ways. The conversation moves across sites, as people find the time or have the interest to do it.

Posting from your aggregator

Since a big part of weblogs is the 'conversation' that is established between different sites, it would be great if you could just re-post a piece of something you've read, or comment on it, no? Many aggregators let you do just that. For example, since Radio is both weblog software and an aggregator, you can use them 'in tandem', to post comments on things you're reading about. NetNewswire, NewsGator, FeedDemon, clevercactus (as well as others) all allow you to post to weblog software as well as reading feeds. I won't go into the details of how to do this mainly because the configuration varies from software to software, but I just wanted to mention it as something that exists, and that you might find useful as you get more comfortable with weblogs and aggregation.

Final final words

Both part one and two are an overview of concepts that (as I said) are still relatively new. As a result, things are still evolving, and new applications are being created all the time. Sometimes the technology can appear to be daunting, but there's lots of people working on making it better, and easier to use. Once you are more 'embedded' in the world of weblogs, you will start finding new uses and applications---things that were simply not possible only a few years ago---to communicate, collaborate and express yourself.

See you in the blogsphere! :-)

Categories: soft.dev, technology
Posted by diego on November 2 2003 at 7:33 PM