Yahoo! Pipes: Some Tips
Yahoo! has released a new RSS remixer named Pipes. Right now it's got some performance issues, but once those are worked out you should go play with the service.
These are some facts and tips about Pipes that I learned while talking with the developers, playing with the app as they were making changes and from reading the documentation.
- To change the name of a Pipe, open it in edit mode and click on the tab. It will become a text field that you can modify and save.
- Pipes will output GeoRSS. They plan to release a map viewer soon. In the meantime you can consume GeoRSS over at Mapufacture.
- You can use Dapper (or Dappit) and OpenKapow to scrape sites. They both output RSS feeds that can be consumed by Pipes.
- They have a slick debugger that allows you to step through your modules one at a time and view the data at each step.
- The following browsers are suported: Firefox, Webkit, and IE6. Known Problems: Pipes is known not to work in Opera. Safari has minor issues. IE6 currently displays problems with the For Each module. IE7 has not been extensively tested.
- In addition to RSS, Pipes also output in JSON.
- The Pipes crawler respects the Robot Exclusion Standard. It has the User-agent:
Pipes-Crawler Yahoo Pipes.
This is post is the fourth in a series on the Pipes product. Here are the other posts on Radar:
- Pipes and Filters for the Internet
- Yahoo! Pipes: The Modules For Building Pipes
- Yahoo! Pipes: Deconstructing a Pipe
Update - Daniel Raffel of the Yahoo! TechDev Team, the creators of Pipes, wrote to me:
I just saw your latest post on radar about pipes and wanted to point out
that the correct user agent to block is "Yahoo Pipes" NOT "Pipes-Crawler".Even more details on that are listed here:
http://pipes.yahoo.com/docs/troubleshootingAnother cool feature I thought I'd mention is that you can search for feeds
in the feed finder in the top right corner, I've attached a pic. You can
drag the results to the canvas.Finally, we have some "easter eggs" for folks to discover but they are
hidden as usual, and, well, we can't spoil the fun.
tags: web 2.0
| comments: 37
| Sphere It
submit:
0 TrackBacks
TrackBack URL for this entry: http://blogs.oreilly.com/cgi-bin/mt/mt-t.cgi/5228
Comments: 37
Having played with it yesterday, I think it would be nice if feeds could come with authentication. I couldnt get my gmail atom feed into a pipe, so I couldnt get an RSS feed of my email translated into french.
with authentication, you could then make many more private atom/rss feeds that are then chained together in interesting ways. Admittedly, I dont know what this does for the pipes business model, but it could be cute nonetheless.
It does work in Opera, but there are some glitches (moving the boxes, connecting them, etc.)
Great product, anyway, in two minutes yesterday, I did a simple filtered Y! search with parameters, on two keywords.
You may have made a mistake for dapper :
it's http://www.dappit.com/ and not -www.dapper.com/ (fashion clothes) !
I was able to create a pipe in IE 7.0, but the description of the modules were not displaying properly.
I also tested with Firefox 2.0 and it was really displaying very poorly.
I really think the best way to use this new product is to have an entire community of users interacting with each other and creating content.
I have tried to kick start something of the sort. Guess we will just have to wait and see where it goes.
These were really useful tips. I like using feedity.com to scrape sites because it doesn't require you to log in. I created several video tutorials for Yahoo Pipes. With your permission, here is the link:
http://usefulvideo.blogspot.com/2007/02/yahoo-pipes-tutorials.html
I really have trouble understanding how the thing runs. I tried using a pipe yesterday and I just couldn't get the damn thing to do anything. I hope someone will give me a word how the thing works. I have not ever ever touched any UNIX environment at all. That said, I would greatly greatly appreciate if somebody would give me the time of day as to how this thing runs. I want "in" on the excitement. I hope someone tags me up and helps me. Thank you and hope everyone has a very good day.
Apatar Open Source can be used to compliment Yahoo Pipes. It can extract data from on-premise systems and feed mashups in situations where large amounts of data need to be extracted from databases like MySQL or Oracle, transformed and loaded into a target like Amazon or SalesForce.com, or into an XML feed.
I wanted to combine all favorite news feeds but Yahoo pipes not working in I.E. 7, but in I.E. 6, worked fine.
One of the main things that Yahoo Pipes has done is it’s made a difficult programming chore more accessible to users. So many programming languages make assembly language more accessible. But, there is still quite a learning curve for the newbie user :)
Imagine being able to take any feed, performing operations on them and outputting another feed.
Wow. This is the *perfect* tool for me to illustrate to my coworkers a number of concepts relating to blogging, rss, and the universal database of participation all of us have been building.
Sure you could all of what Pipes provided with some Python or Perl fairly quickly - if you were a scripter like myself.
But this opens it up to almost anyone. And that makes it huge.
On a personal note, this is precisely the kind of tool that will enable me to provide useful views into the conversation talking place in the Philly Future regional blog community.
I don't see an OPML input there, but I've just dipped my toes in this morning.
Great work Yahoo! :)
wow - great little application, makes life a little bit easier. Every little helps? hehe
hi Brady Forrest, Very nice Tips. I really reserching on yahoo pipes. Your information is very useful . I think this service is going to be a hit soon
hi Brady Forrest, Very nice Tips. I really reserching on yahoo pipes. Your information is very useful . I think this service is going to be a hit soon
I read something very same somwhere else in oreilly article can someone please tell me what is the difference between that article and this one over here? arent they basically the same... and could someone please put up detailed version of Pipes as many people "laymen" specially donno what exactly pipes are lol not to confuse with other pipe in household..
Reading the tips Brady shared with us, I considered necessary to share a resource containing short video tutorials on how to work with pipes.
usefulvideo.blogspot.com/2007/02/yahoo-pipes-tutorials.html
youtube.com/watch?v=2pVxqgt5jTs
Thank you Brady for useful tips about Yahoo's Pipes service. It helps me a lot. I think the Yahoo Pipes servie is a really great project!
Nice tips there, Brady. The other Oreilly article article on Yahoo Pipes explains the concept but you make it even clearer through these tips.
Apatar Open Source can be used to compliment Yahoo Pipes. It can extract data from on-premise systems and feed mashups in situations where large amounts of data need to be extracted from databases like MySQL or Oracle, transformed and loaded into a target like Amazon or SalesForce.com, or into an XML feed.
can you please tell me what are the performance issues with pipes. are the issues still there?
what is the difference between GeoRSS and simple RSS. is geo RSS a type of feed?
GeoRSS is an extension on RSS. Like MediaRSS. Check out georss.org for more info.
Nice tips Brady. Here is a video tutorial that gave me a good start to work on pipes : http://www.jumpcut.com/fullscreen?id=594F555C568011DC9D24000423CEF5B0&type=movie
Thank you for the tips, Brady. But I'm curious about Yahoo Pipes service working with Opera. Is it already possible for an Opera user to use it?
what is the difference between GeoRSS and simple RSS. is geo RSS a type of feed?
Thanks,
The Pipe is already a big hit in my country. Seems the concept spread like wildfire. Could this be the site that lit that fire?
Thanks for very interesting article. btw. I really enjoyed reading all of your posts. It’s interesting to read ideas, and observations from someone else’s point of view makes you think more. It’s really good written and I fully agree with You on main issue, btw. I must say that I really enjoyed reading all of Your posts. It’s interesting to read ideas, and observations from someone else’s point of view
Hi, I´m looking for a solution to mix different feeds by date using different date formats or even without date coded in the feed...
Does anyone know how to solve this?
Thank you Brady for useful tips about Yahoo's Pipes service. It helps me a lot. I think the Yahoo Pipes servie is a really great project! Again, very cool post!
Really great product from the yahoo stable but i've got issues with the filtering side. U're not able to prevent the same news item displaying twice or more when more feeds are merged to produce one pipe.
Hope they rectify this.
Post A Comment:
STAY CONNECTED
RECENT COMMENTS
- dmeanea on Yahoo! Pipes: Some Tips: @whatsonnews: use the "...
- whatsonnews on Yahoo! Pipes: Some Tips: Really great product fr...
- Motorhome on Yahoo! Pipes: Some Tips: Thank you Brady for use...
- Horatiu Vasile on Yahoo! Pipes: Some Tips: Hi, I´m looking for a ...
- Apotheke on Yahoo! Pipes: Some Tips: Thanks for very interes...
- The Muse on Yahoo! Pipes: Some Tips: The Pipe is already a b...
- Motorhomes on Yahoo! Pipes: Some Tips: what is the difference ...
- Nick on Yahoo! Pipes: Some Tips: Nice video tutorial Zeb...
- Tube on Yahoo! Pipes: Some Tips: Thank you for the tips,...
- zeb on Yahoo! Pipes: Some Tips: Nice tips Brady. Here i...
Joseph Perla [02.08.07 11:47 PM]
Brady,
I think by the Dapper link you mean to use the url dappit.com rather than dapper.com.
Cheers,
j