An introduction to Yahoo Pipes

by Ross McKillop on July 16, 2007

Yahoo describes Pipes as “an interactive data aggregator and manipulator that lets you mashup your favorite online data sources”. Alright, but what does that mean to the rest of us? Yahoo Pipes uses an easy-to-understand visual editor that allows you to (among other things) take multiple RSS feeds, combine and filter them, and then output all of that information to one RSS feed.

This tutorial will take you through the steps to filter the content of one RSS feed (Techmeme) so that content you may have already read isn’t included in your Techmeme feed.

  1. To create a pipe you’ll a need a Yahoo account, so if you don’t have one, you might as well sign up now. Once you’ve signed in, visit http://pipes.yahoo.com. The default/blank Pipes layout will be presented.
  2. yahoo pipes
    click to enlarge

  3. Get started by dragging Fetch Feed from the Sources list into the main module area. Enter the full URL to the RSS feed you want to customize. For this tutorial I’m using the Techmeme RSS feed, which is http://www.techmeme.com/index.xml
  4. yahoo pipes
    click to enlarge

  5. To confirm the feed was loaded correctly, drag a line from the small circle at the bottom of the Fetch Feed module to the Pipe Output module. Then, drag the Debugger window into focus and click the Refresh link if nothing shows up right away. The Debugger window should fill with a list of the articles in the feed you added.
  6. yahoo pipes
    click to enlarge

  7. Now we’ll filter out articles from the feed that we don’t want to include. From the Operations menu, drag the Filter module into the main workspace.
  8. yahoo pipes
    click to enlarge

  9. Now link the Fetch Feed module to the Filter module (you can leave the Pipe Output module unconnected for now).
  10. yahoo pipes
    click to enlarge

  11. In the Filter module, make sure that Block and Any are both selected. In the Rules area, select item.description. The item.description option is ‘created’ because Yahoo Pipes recognizes the “description” part of the Techmeme feed (see the bottom part of the screenshot below).

    For this feed, I’m going to block out certain URL’s from the Techmeme feed, because I already subscribe to those sites RSS feeds, and I don’t want to waste time scrolling through the Techmeme feed on things that I’ve already ready. So the first URL I entered was http://gigaom.com. That will block out any Techmeme articles that originate from GigaOM.com.

  12. yahoo pipes
    click to enlarge

  13. Click the + (plus) Rules button to create another rule, and repeat the steps above. Enter in the URLs to sites that you already read, and therefore don’t need to read again when they’re published in Techmeme. When you’re done, ‘drag’ the pipe from the Filter module to the Pipe Output module. Refresh the Debugger area and any articles that originate from a site that you’ve just added to your filter will be removed.
  14. yahoo pipes

  15. If you’d like, you can also add a Sort module (from the Operations menu) into the mix. That way you can sort the feed articles by the date published etc. Make sure the final pipe goes from Fetch Feed to Filter to Sort and finally to Pipe Output. When you’re done, click the Save button in the upper-right part of Yahoo Pipes.
  16. yahoo pipes
    click to enlarge

  17. Give your custom feed a name and click Save.
  18. yahoo pipes

  19. To preview your “new” feed, click Run Pipe from the top (center) of Yahoo Pipes. Your feed will be previewed in a new browser window/tab.
  20. yahoo pipes
    click to enlarge

  21. If you’re happy with the results, close the preview window/tab and click the Publish button (next to Save). Enter a description and some tags, then click Publish.
  22. yahoo pipes

  23. To subscribe to your feed, go back to the Preview page and use the handy Subscribe drop down menu. You can save your feed directly to a number of the popular online feed readers, or just use the Get as RSS item to display your RSS feed URL.
  24. yahoo pipes

  25. For Firefox users, the feed will be displayed in your browser. You can copy and paste the URL into any RSS reader you choose.
  26. yahoo pipes
    click to enlarge

  27. You can find more Yahoo Pipes resources/tutorials here (and feel free/encouraged to add more to that list).