Yahoo! Pipes: Deconstructing a Pipe

Yahoo! has just released a site for hosting and mixing RSS feeds; read Tim’s launch post for details. In my previous post Yahoo! Pipes: The Modules For Building Pipes I explained the tools available for building a pipe and mentioned the Apartment Near Something pipe. It allows you to input what you would like to be near (for example: “parks”), what city (for example: “Palo Alto, CA”), and how far (for example: 2 miles). It outputs GeoRSS of available apartments in a Palo Alto, CA that are near parks. In this post I am going to step through this pipe. If you want to follow along you can check the debugger in the editor to see the output at each step.

  1. You are prompted to fill in 3 fields: your location (palo alto), what you would like to be near (parks), and how far away they can be (2 miles).
  2. The Location User Input module takes the location field and passes it to the URL Builder. It constructs the following URL:
  3. The URL is passed to the Fetch Module.
  4. The resulting RSS feed is passed to the Location Extractor Module. The entries that have text relating to location get geotagged.
  5. The updated feed get passed to a Filter Module. Only locations that have been extracted with a confidence of 80 or higher can pass through.
  6. The filtered feed is passed to a For Each: Annotate Module; it has a nested Yahoo! Local Search Data Source Module. The Local module accepts “parks” as an input from the Text User Input Module.
  7. Then the each item of the feed is checked to see how close it is to area parks.
  8. The updated feed is passed through another Filter Module. This Filter Module takes input from a Number User Input Module to filter out rental properties that aren’t within 2 miles of a park.
  9. The Pipe is complete and can be consumed as GeoRSS

This was a relatively simple Pipe that was created by the Pipes team. It works well as an example because it pulls in most of the common features.
I want to thank Pasha Sadri, the principal software engineer of Pipes, who answered my questions and explained some concepts along the way.