Yahoo Pipes are Awesome RSS Filters

Overwhelming is an understatement when I describe available news on the Internet. Just get on Google’s news section and search for something. If it is anything remotely general, mountains of pages of links will topple out of your monitor and cascade onto your head like a pile of bricks. You then grit your teeth and dig your way out.

___I happen to enjoy keeping up with news on China and Japan, but with so many sources, checking them individually is madness, and going to Google will drive you to use psychotic drugs. News about China and Japan in the local papers is scarce. There is always more space devoted to local issues, so world news gets pushed to the back of the queue, often not making the papers. By gathering news from around the world you gain different perspectives on news. I do like the objectivity of the British press. Both the Canadian and American press can warp stories one way or another.

___My search to filter rss feeds stopped at the door of Yahoo Pipes. After a brief video it was pretty easy to use and relatively intuitive. I threw together 26 rss feeds, filtered by keyword, eliminated duplicates, and sorted by descending order of date. My China Japan News feed was finished.

___My sources include: Google, NY Times, CNN, BBC, Danwei, Washington Post, Wall Street Journal, Times of India, Financial Times, International Herald Tribune, Asia Times, Asahi, Toronto Star, Globe and Mail, Japan Times, LA Times, The Straits Times, Channel News Asia (Singapore), The Star (Malaysia), Jakarta Post. Are there any other Asian flavours I should add?

___Some of my feeds are not cooperating and gave me errors:

warning error fetching (502 Bad Gateway)
warning error fetching (502 Bad Gateway)
warning error fetching (502 Bad Gateway)

___Oddly, some online newspapers do not provide rss, though they are mostly in Asia. These include:

  • People’s Daily: RSS feed does not work
  • China Daily, Yomiuri Shumbun: have no RSS feed. The Yomiuri Shimbun subscribe link is for a delivered paper copy subscription
  • Asahi Shimbun’s english section: gives an RSS in Japanese
  • Chicago Tribune: has no RSS. Maybe Obama will fix this

___I’ve fed this new feed into‘s news aggregator and it displays very nicely. I’ll play with it for a couple of days and see if the pipe breaks. Overall I’d give Yahoo Pipes a B+ for user interface and usability, and a B for bugs. Often the news preview and the save functions were not working, but you wait long enough and it fixes itself. The bottom line is that the functions of Yahoo Pipes work and work very well.

P.S. I just added The Straits Times, Channel News Asia (Singapore), The Star (Malaysia), Jakarta Post. I also notice that I must be logged into Yahoo mail or I cannot access Yahoo Pipes. Yahoo Pipes have their own login, but it won’t let me authenticate with my valid Yahoo account. Logged into Yahoo Mail I bypass Yahoo Pipes’ authentication.

2 thoughts on “Yahoo Pipes are Awesome RSS Filters

  1. David Ing

    I’ve been aware of the mashup/pipes technology for some time, and you now have me reflecting on why I haven’t used it yet.

    My response is that there are two ways to filter: technologically, or socially. Clearly, search terms are a way to filter.

    The other alternative, of social filtering, is to follow a small group of people whom I note read broadly but bookmark and blog selectively. The base interface is the same — feed technology — but instead of following primary sources, I’m following secondary sources. This is a function normally handled by newspapers, and now by individuals on the Internet in the age of peer production.

    Since I’m currently immersed in academic research on open source, I was making the joke that I’ve been using a vice-president at IBM as my research assistant. It’s most true, since Bob Sutor bookmarks the most significant daily headlines, and he does a much better job of building on a base of knowledge than newspapers who have to mostly assume their readers don’t have a history or background in the topic.

  2. dontai Post author

    While your social filtering can and does work well in North America, there are significant issues in the rest of the world. This is particularly so in China.

    It is not as easy to use another person to find and comment on important news in Asia. Firstly, technology standards are not the same. While rss is common here, it is not as standardized in Asia. Many news sites do not have rss, or they do not work. Technology and internet access is not as widespread. Computers and internet access are still prohibitively expensive for many locals.

    Secondly there is a significant language barrier between bloggers in North America and Asia. Their blogs will come out in Chinese, which is not as easy for me to read. I’m unsure if I could even read the rss from Chinese blogging software, though this would be the least of my issues. Asahi Shimbun’s English news rss comes to me in Japanese. How do our search terms, tags, and taxonomy translate to Chinese?

    Thirdly, and most problematically, China is a communist country, and a technologically savvy one at that. Local Chinese that blog may be tracked down by the police, arrested, disciplined with jail time, and blog entries or entire blogs deleted. They are charged with leaking state secrets, in essence, treason. There are documented cases in the news, before Beijing courts and human rights tribunals, and I believe them. It is simply too dangerous for Chinese to freely post on the Internet. Other “grassroots” social computing structures are also discouraged by the government. China controls a central firewall for the whole country and has the knowledge, resources and the political will to screen ALL content going in and coming out of their country.

    I hope that social network computing tools become popular in China, though there are significant barriers that we do not experience here in North America. The issues is not one of social networking software such as blogging, but more fundamentally of free speech and freedom. We take this for granted here in North America. I do not. While we use computing to foster knowledge and communication, technology can also be used to control and repress.

    While I would prefer to get news from independent Chinese bloggers, I’ve always known that embedded reporters from news agencies are the next best thing, and have been for the last 25 years. Even they get harassed, followed, equipment smashed, arrested and occasionally thrown in jail. Documented cases have occurred as recently as the Beijing Olympics, August 2008. This is after the Chinese government, as a condition for hosting the Olympics, agreed to allow reporters to do their jobs without censorship. They have a hard road to travel but they know the risks.

Leave a Reply

Your email address will not be published. Required fields are marked *