Thursday, October 29, 2009

Getting Started With Yahoo Pipes: Merging RSS Feeds

If you don't already know what RSS is, you may have noticed the folllowing logo appear on different websites, and even within your browser, and never really been sure what it's actually for...

What it's for is wiring (or plumbing). What it's for is passing content from one web page or application to another. What it's for is never having to visit that web page again to keep up to date with new content that might appear on that web page or website. What it's for is letting you see content from that page or site in another application, such as feed reader like Google Reader, or a 'web desktop/dashboard' like Netvibes, or Google personal pages. What it's for is turning websites into 'not email', that you can subscribe to from a single application and then view updates from in a single location.

It's also for much more than that, but that's what we'll start with...

But that's not what this post is about... What this post is about is how you can use an online application called Yahoo Pipes to do all sorts of plumbing with RSS feeds.

To get started, you'll need a Yahoo account, then you can create your first pipe...

It's like Lego, but with bits of web content pulled into the pipe from one or more RSS feeds, with the content packaged up into bundles where each bundle contains:
- a title;
- some content (like the body of a blog post or news story), referred to as the description;
- a link (which is often to the original web page that contains the description).

So for example, if we look at the RSS from Cambridge University Library web page, we see links to a variety of RSS feeds.

We can pull one or more of these feeds into the pipes environment by creating a new Yahoo pipe and then using the Fetch Feed block from the Sources area of the left hand side bar:

(Highlighing a block by clicking on it lets you preview the output of that block.)

We can add combine the output of several feeds simply by adding the URL of each required feed to the Fetch Feed block:

The order of items in combined feed will be all the items from the last feed in the Fecth Feed block. followed by the items in the feed before it, and so on.

To order the items in the combination feed by date order, use the 'Sort' block:

The field you need to sort on is chosen from the drop down menu - PubDate is the element we want to sort on:

(Wire the blocks together by clicking on the 'output circle' at the bottom of a block and dragging the 'wire' that is produced onto the 'input circle' at the top of the next block.)

Finally, we need to connect the output block to the pipe to complete it.

If you run the Pipe, you will see its 'front page':

you can now subscribe to the output feed from this pipe (or use it in another pipe...), add it to your Yahoo or Google homepage, and so on.

There's a lot more you can do with Yahoo Pipes, but this is a good start: being able to aggregate (that is merge, or combine) content from several different sources into a single feed, and then order them accroding to time.

So how else might we use this simple 'aggregate and order' pattern?

How about combining table of contents feeds from different journals (you can find their URIs from TicTocs?

In this way, you can create a single RSS feed that keeps you up to date with the contents of several different journals you are interested in, and maybe also pulls in content from a recent/new books feed from your Library?

No comments:

Post a Comment