How to Build Your Own Lifestream with Yahoo Pipes and NO Server Side Logic
Step 6: Go Back and Fix/Prepare the Data
So, we are almost there this is basically the tips and tricks section. When I was building the interface for my Lifestream I noticed some issues that made me go back to the Yahoo Pipe to fix it.
Truncating
The first issue was to add a parameter to the pipe that would define the size of the output of the pipe. In some cases I only wanted one item outputted, in other cases 50, so adding a little parameter was of big importance. Doing this in Yahoo Pipes is pretty easy. By adding a “Number Input” module you can fetch a number, either direct from the user or as a parameter in the url. In my case, I added the module to accept a number for the parameter ‘l’ and then passed that number to the “Truncate Module” to truncate the output of the pipe. After this, passing a number to the pipe can be done easy by adding the parameter ‘l’ to the url:
http://pipes.yahoo.com/pipes/pipe.run?id=nLJtNct93BGnML4nl7okhQ&_render=json&_callback=drawLifeStream&l=50
Prevent Caching
Second thing I wanted to achieve was to get around a quirk of Yahoo Pipes that didn’t really fit my needs: caching. Yahoo Pipes caches stuff fanatically (about 24 hours) which is good for them and good for the response time, but not particularly nice for the refresh cycle of my lifestream. The workaround is pretty easy: just add a random cache parameter to the url and change the parameter whenever you wish to circumvent the caching.
In my case I only wanted to update every 30 minutes, so I got the current time in javascript, rounded that to the last 30 minutes, and got the MD5 hash. Then I passed this “cache token” to the Yahoo Pipe just like I did with the length parameter:
function loadJSON() {
var newScript = document.createElement('script');
newScript.type = 'text/javascript';
var time = new Date().getTime();
time = time - (time%1800000);
time = time += "";
var token = hex_md5(time);
newScript.src = 'http://pipes.yahoo.com/pipes/pipe.run?_id=nLJtNct93BGnML4nl7okhQ&_render=json&_callback=drawLifeStream&l=50&cache=abcdef0123456789';
document.getElementsByTagName("head")[0].appendChild(newScript);
}
As I wasn’t sure if this was enough, I also passed this token to all the separate Feed Fetcher modules in all the Lifestream Items to make sure that Yahoo Pipes would not cache any feed when I wanted it to.
Add identifiers
Last, I wanted to make it a bit easier to identify in my Javascript code which item belonged to which original source. This makes it easy to add the different colors and logos to all the different sources in my Lifestream. The dirty way to do this is to look at the source URL, but as some feeds get run through Feedburner this is hard, plus it makes things more complex on the Javascript side for no specific reason.
The other way is to add specific new values to every item in the feeds, identifying them as being a “Flickr” item or a “Twitter” item. Take a look at the LifeStreamItem module to see how I did this as it was not particularly standard procedure. In short: I make a copy of one item value and name it “source, after which I then change it’s value to whatever I want it to be.
