Skip to main content

PubSubHubBub and ColdFusion

I came across a publisher\subscriber protocol called PubSubHubBub. It is a server-to-server web-hook-based pubsub (publish/subscribe) protocol - an extension to Atom and RSS. Here the parties (servers) get an instant notification when a feed URL, that  they are interested in is updated.

Traditionally a subscriber would subscribe to a feed and poll for it at regular intervals, to see if there is an updated feed available. In this protocol rather than polling for a feed, the content is pushed out from the publisher. The theory here is that the subscriber can subscribe to a feed via a 'Hub', which then would inform the subscribers when the feed is updated.

How PubSubHubBub works?
  • A Publisher instead of sending an update to every subscriber, it includes a in its feed URL and sends an update to the Hub.
  • A Subscriber sends a subscription request to the Hub with the feed URL that it is interested in. The request also contains a callback URL to which the Hub should send an update.
  • To verify the subscription request, the Hub sends a GET request to subscribers' callback URL. The
    Subscriber then verifies itself by responding to the request.
  • When the Publisher posts new content, it notifies the Hub of the updates by sending a ping notification (POST request).
  • Hub on receiving a notification from the Publisher, fetches the new content and then POSTs an update to the Subscribers callback URL.
If the feed has multiple subscribers, then the Hub would send an update to each of these subscribers. The Subscription flow posted on the PubSubHubBub site is here:



Where is the Hub?

Anybody can run a hub, it is not owned by any company. The protocol is decentralized and free. A couple of implementations include pubsubhubbub.appspot.com and superfeedr.com

Publishing

Whenever a publisher adds new content (a new blog post), the subscribers are to be notified of the updated feed. A notification (POST request) is sent to the Hub by the Publisher. The Content-Type header must be set to application/x-www-form-urlencoded and the request should contain the parameters 'hub.mode' and 'hub.url' in the request body. The parameter hub.mode is set to 'publish' and the hub.url is set to topic URL that is updated.

The Hub then accepts the POST request and fetches the new content by sending a GET request to the topic URL. After fetching the content the Hub determines whether the feed has changed. The Hub would then send this information about the changes to each subscriber.

Subscribing

The Subscriber initiates its subscription by sending a POST request to the Hub URL. The Content-Type header must be set to application/x-www-form-urlencoded and it should contain the following parameters in the request body:
  • hub.mode - set to 'subscribe'
  • hub.callback - subscribers callback URL where the notifications should be delivered by the Hub.
  • hub.topic - the topic URL to which the subscriber wishes to subscribe
  • hub.verify - it can be either sync/async. 
    • sync when the verification request must occur before the subscription request's HTTP response is returned.
    • async when the verification request may occur at a later point after the subscription request has returned.

Verifying the subscription:

The Hub on receiving a subscription request verifies the same by sending a GET request to the subscriber's callback URL. The request contains the following query strings:
  • hub.mode - subscribe or unsubscribe depending on what was set in the original request.
  • hub.topic - the topic URL given in the subscription request.
  • hub.challenge - a random string.  
The subscriber's callback URL should respond to this request by echoing the hub.challenge string in response.

Code:

Registering the Publisher with the Hub:

The Publisher and Subscriber can be registered at pubsubhubbub.appspot.com and it is a one time activity. However, one can do the same programmatically:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="pub_result> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp> <cfif #pub_result.Responseheader.Status_Code# eq 204> Publisher registered successfully <cfelse> Publisher failed to register returned #pub_result.Responseheader.Status_Code# </cfif>

The Hub responds with a '204 No content' if the publishers feed is successfully registered.

Publishing content:

A form accepting the post title and description:
publisher_form.cfm:
<cfparam name="FORM.feedTitle" default="" > <cfparam name="FORM.feedDescription" default="" > <cfif FORM.feedTitle NEQ "" AND FORM.feedDescription NEQ ""> <cfset publisherObject = createObject("component","Publisher") > <cfset publisherObject.publish_update(FORM.feedTitle,FORM.feedDescription) > </cfif> <form action="publisher_form.cfm" method="post"> Add a new post here:
Title: <input name="feedTitle" type="text" style="width:300px">
Description:
<textarea name="feedDescription" rows="5" cols="50" ></textarea>
<input type="submit" value="Submit"> </form>

A ping notification is sent to the Hub, to notify of the new content. The function publish_update does that:
<cffunction name="publish_update" access="remote" returntype="String"> <cfargument name="feedTitle" required="true"/> <cfargument name="feedDescription" required="true"/> <cfscript> if(!isDefined("application.feedQuery")) { application.feedQuery = queryNew("title,description,pubdate"); } queryAddRow(application.feedQuery, 1); querySetCell(application.feedQuery, "title", '#arguments.feedTitle#'); querySetCell(application.feedQuery, "description", '#arguments.feedDescription#'); querySetCell(application.feedQuery, "pubdate", '#DATEFORMAT(now(), "mm/dd/yyyy")#'); </cfscript> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="new_content"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish"> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp>

The Hub then fetches the feed from the mentioned URL (hub.url). Here it is the getFeed method in Publisher.cfc:

<cffunction name="getFeed" access="remote"> <cfscript> /* Feed metadata structure */ feedMetaData = structNew(); feedMetaData.author = arrayNew(1); //arrayAppend(feedMetaData.author,{name:'Sagar Ganatra',email:'sagar@sagarganatra.com'}); feedMetaData.encoding = "UTF-8"; feedMetaData.link = arrayNew(1); //arrayAppend(feedMetaData.link,{href:'http://pubsubhubbub.appspot.com',rel:'rel'}); feedMetaData.title = structNew(); feedMetaData.title.type = "text"; feedMetaData.title.value = "Sagar's Kitchen"; feedMetaData.updated = "#now()#"; </cfscript> <cfset colMap = {publishedDate="pubdate", title="title", content="description"}> <cffeed action="create" query="#application.feedQuery#" properties="#feedMetaData#" columnmap="#colMap#" xmlvar="xmlResult"> <cfcontent type="text/xml" reset="true"> <cfoutput>#xmlResult#</cfoutput> </cffunction>

Registering a Subscriber with the Hub:

The Subscriber can subscribe to the feed by specifying the topic URL in the request body:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="sub_result"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="subscribe"> <cfhttpparam type="url" name="hub.verify" value="sync"> <cfhttpparam type="url" name="hub.topic" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> <cfhttpparam type="url" name="hub.callback" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/subscriber.cfc?method=receiveFeed> </cfhttp> <cfif #sub_result.Responseheader.Status_Code# eq 204> Subscriber registered successfully <cfelse> Subscriber failed to register. Returned <cfoutput>#sub_result.Responseheader.Status_Code#</cfoutput> </cfif>

Subscribing

As mentioned earlier, Hub checks the Subscribers' intent by sending a GET request to the callback URL. The Subscriber is then required to respond, by echoing back the 'hub.challenge' parameter present in the URL:
<cffunction name="receiveFeed" access="remote" returnformat="plain" returntype="string" output="false"> <cfheader name="Content-Type" value="text/plain"> <cfif structKeyExists(URL, "hub.challenge")> <cflog text="#url['hub.challenge']#"> <cfreturn #url['hub.challenge']> <cfelse> <cfset feeddata="getHTTPRequestData().content"> <cfset feeddata="feeddata" feedsource.xml="feedsource.xml" filewrite="filewrite" ram:="ram:" trim="trim"> <cffeed action="read" query="feedReceived" source="ram:///feedSource.xml"> <cfdump output="console" var="#feedReceived#"> <cfheader statuscode="200"> <cfreturn> </cfif> </cffunction>
As seen in the above code the feed is read in else part. One can store the read content in a database for later retrieval or send the same over a websocket to the clients so that it can be read live.

Who is using PubSubHubBub?

Google products - Google Alerts, FeedBurner, Blogger and Google Reader are using PubSubHubBub.

Reference:

Project Home - http://code.google.com/p/pubsubhubbub/
Protocol Basics - http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html
Labels:

Comments

  1. Richard DaviesJanuary 11, 2012 at 12:53 AM

    Thanks for this post. It's been helpful to me in learning how to implement the PubSubHubbub protocol in ColdFusion.

    FYI: I think there is an error in line 11 of your getFeed() method. The link to the hub server should have rel="hub" but your code uses rel="rel".

    ReplyDelete
  2. Sagar GanatraJanuary 11, 2012 at 2:49 PM

    Thanks Richard,

    I've updated the blog post now.

    ReplyDelete
  3. internet radioMay 23, 2012 at 9:09 PM

    Thanks a lot for the useful and detailed post, I am currently looking for a publisher for myself and consider sticking to this one. Seems to be a little complicated at first sight, but I am sure I will fiagure all the features out with such helpful tips.

     

    ReplyDelete

Post a Comment

Popular posts from this blog

File upload and Progress events with HTML5 XmlHttpRequest Level 2

The XmlHttpRequest Level 2 specification adds several enhancements to the XmlHttpRequest object. Last week I had blogged about cross-origin-requests and how it is different from Flash\Silverlight's approach .  With Level 2 specification one can upload the file to the server by passing the file object to the send method. In this post I'll try to explore uploading file using XmlHttpRequest 2 in conjunction with the progress events. I'll also provide a description on the new HTML5 tag -  progress which can be updated while the file is being uploaded to the server. And of course, some ColdFusion code that will show how the file is accepted and stored on the server directory.
18 comments
Read more

Server sent events with HTML5 and ColdFusion

There are several ways to interact with the server apart from the traditional request\response and refresh all protocol. They are polling, long polling, Ajax and Websockets ( pusherapp ). Of all these Ajax and Websockets have been very popular. There is another way to interact with the server such that the server can send notifications to the client using Server Sent Events (SSE) . SSE is a part of HTML5 spec:  http://dev.w3.org/html5/eventsource/
19 comments
Read more

Adding beforeRender and afterRender functions to a Backbone View

I was working on a Backbone application that updated the DOM when a response was received from the server. In a Backbone View, the initialize method would perform some operations and then call the render method to update the view. This worked fine, however there was scenario where in I wanted to perform some tasks before and after rendering the view. This can be considered as firing an event before and after the function had completed its execution. I found a very simple way to do this with Underscore's wrap method.
Post a Comment
Read more