Behind the scenes at #cpd23: automating (some of) the registration process

A couple of weeks ago, I asked for thoughts about ways of automating the process of adding new subscriptions to a single RSS feed drawn from multiple sources.  This was to try and streamline the process of managing registrations for 23 Things for Professional Development. I seem to have come up with something that works, albeit in a somewhat ungainly fashion, so I thought I'd write it up.
Here's a diagram of what's what.  Red lines and text are manual steps done by the cpd23 team, and the blue lines and text are the automated bits.  The yellow stars show the outputs that are intended for general use.  If you can't see the image for whatever reason don't worry - I explain all the steps below.

Icons by Mysitemyway Design Team via iconfinder.com

The aims of the system are

1. to generate a tagged list of participants, so that participants and others can find blogs that interest them, based on geographical region and professional sector
2. to create a feed of recent posts by participants to show on the cpd23 homepage
3. to create a combined feed of all cpd23 participants' posts, to which anyone who is interested can subscribe

This is achieved in the following way:

1. Participants register their blogs using a Google Docs form (embedded into this blog post).The data submitted into a Google Docs form is stored in a Google Docs spreadsheet, which can be accessed and edited.  
2. Cpd23 team members manually visit each blog that is registered, and (if the link submitted is valid and the blog is a personal cpd blog) they add it to Delicious, including tags for place, sector, year and an alphabetical tag for the first letter of the blog title.  (I use the handy Delicious bookmarklet for this.)
• Delicious provides scripts that power the 'recently signed-up' widget on the cpd23 blog, and the participants list on the participants page.  There is a limit to the number of links that Delicious will provide (100 at a time), which is why the alphabetic tags are added to the bookmarks: the participants list is powered by a script for each letter of the alphabet.
3. An ifttt task takes all new bookmarks added to Delicious with the tag '2012' and submits them to the Google Reader folder 2012. Where possible, Google Reader can automatically extract an RSS feed from the link given.
• Cpd23 team members periodically update the 'Participants posts' widget on the cpd23 blog, which runs using the Blogger blogroll widget.  It's easy to add new subscriptions in the 2012 folder from Google Reader.
4. Another ifttt task takes all new posts in the Google Reader 2012 folder, and bookmarks them in Diigo, adding the tag 'cpd23feed'. (It could bookmark them in Delicious instead, but I wanted to keep the Delicious account just for links to blogs, and not confuse matters by adding lots and lost of links to posts there).
• Diigo provides RSS feeds of users' tags, so an RSS feed of all new posts from cpd23 participants can be made available. The format of Diigo RSS feeds is: http://www.diigo.com/rss/user/[username]/[tag]
5. And that's it.

  by  roberthunt1987 

Benefits of this system:

• It minimizes the number of manual steps - we only need to get involved at two points in the process
• It enables us to offer the 'added value' service of a combined RSS feed, on top of the Blogger blogroll and the tagged Delicious participants list.

Disadvantages of this system:

• It relies on a lot of services running correctly
• Time lag: the ifttt tasks, and the feeds from Delicious and Diigo update periodically, so new blogs and posts aren't available everywhere immediately.  The lag is usually up to an hour.
• Ifttt only recognises new tags, so if Delicious recognises a 2011 participant blog such that we're just adding a new 2012 tag to an existing bookmark that blog has to be added to Reader by hand.
• Google Reader doesn't always correctly find the feed of a link submitted from Delicious.
• The resulting RSS feed, because it's a feed of bookmarks not posts, only shows links and not post text.  This really isn't ideal, but it's the best I could come up with.

#cpd23, Thing 4: current awareness

I was introduced to Twitter, to RSS feeds and to Google Reader last year during Cam23.  I'm a regular and committed user of them all.

Re-reading last year's posts, I see that I was particularly perceptive early on about the benefits of Twitter:

...if you have a wide circle of followers and followees, Twitter seems to good for spreading information and ideas quickly.  And it seems that, unlike with RSS feeds, it's information that you might not know you want to go looking for... 

The use of hashtags seems to make it possible to create a community of people interested in the same thing, and to widen knowledge about that thing and get a buzz going around it...

I wasn't, however, completely sold then, I wasn't sure if I'd stick at it.  That was a feeling that persisted for quite a while - I wondered if I'd go back to it after weeks away for holidays, Christmas, etc.  But I do go back.  It's important to remember that you can't keep up with everything, and just to let it waft by when you're too busy to pay attention.  But it's an invaluable way to meet people, share ideas, get answers and make things happen.  I'm feeling chuffed with myself this month for having introduced to each other some of the people who've gone on to organise #uklibchat.

'Yesterday News' by Zarko Drincic on Flickr

RSS is great for bringing the news to you - I use it for comics I follow, knitting blogs I follow, podcasts I like, institutional news, and a whole slew of library/librarians blogs.  The key, for me, is to remember that you don't have to read it all.  Some of the stuff I subscribe to is stuff I always look at when it comes in.  The rest, and this is by far and away the majority I either glance at and deicide to read later (or not), or is just there in case I want to fill some time with reading.  But mostly, I don't read a lot of it. Some of it I hear about via Twitter, some of it I just 'mark as read'.  There's no shame in not keeping up with everything - but RSS gives you a fighting chance if you're really keen to try!

A Google Reader feature I've recently started using is the 'shared items' feature. You can follow other people and see what they've chosen to share, and people who follow you can see what you share.  You can leave notes and comments on shared items, too.  It's a nice way to pick up on things outside feeds you follow.  To follow someone, the easiest way that I've found is to search for their Google profile (here's me - you can use the search box from the top of that page - and here are the things I've shared) and look for links to their shared items.  Otherwise use the shared items option in reader and search for people there.

Now, as for Pushnote, the word on Twitter and other blogs is that it's not great. And so I'm just not looking at it. Sorry.

As for other things useful to current awareness, I wonder sometimes if I ought to use Delicious as a stuff-discovery tool.  But you'd have to have very specific tags in mind for that to be very useful - event/theme hashtags, maybe?  It's not something I've really explored.

Thing 12: "Yet mark'd I where the bolt of Cupid fell"

It's a mystery to me why I haven't set up a Delicious account sooner, given that I can utterly see the point of it. While there are some links I only really use at work, and some I only use at home - which can therefore be cheerfully be bookmarked in the browser on home and work computers - there are some that I don't use everyday, that I won't remember without prompting, and that I might want to access from anywhere. Foremost in this list would be 'articles I saw somewhere-or-other but didn't have time to read immediately'. With these all neatly stored away in Delicious under 'toreadandinwardlydigest' they'll be available to me whenever and wherever I have a chance to read them. Very handy.

I also like the design of the site - clean, clear and simple. I like the ability to organise tags into bundles, the choice between viewing them as a list or as a cloud, and between them sorted aplhabetically or by frequency of use. The tools are handy, too, although I'm not sure I'll go all the way and embed a tag cloud or bookmark-roll on my own blog. What do you think?

I can definitely see the merit of using Delicious in a library or other information-management environment. To my mind, it's a great way of disseminating links and resources. Because it's a dynamic tool, users can pick out the links they're interested in by using and combining tags, rather than having to read all through a list put together by someone else: there should be a higher wheat-to-chaff ratio for each individual user than on a static page. From a librarian's perspective, it's also probably easier to maintain than a page of links written in a CMS or in straight html. One disadvantage of using Delicious is, however, that you will be referring users to an outside site if they want the full functionality of the service rather than just a 'latest links' widget embedded somewhere else. This means that they might not realise that the service is being provided and managed by the Library.

I haven't really investigated the 'community' aspect of Delicious yet, so I can't comment of the usefulness of being able to search other people's tags. As with so much, I expect that the value is dependent on the like-mindedness or otherwise of you and the other however-many-millions. It's probably fun to search, but I'd be surprised if it was an efficient search tool.

One final thought: where do I bookmark the Delicious homepage?

P.S. Magistra et Mater's thoughtful post on Delicious comes highly recommended if you're looking for another opinion.