Hack Warwickshire 2: The Revenge

Just a quick entry to say that we will be running the second Hack Warwickshire competition from Monday 23rd August until Friday 29th October.

This time there are loads more Open Data sets to play with and there will be one prize for the best entry from either a member of staff or a member of the public.

More info and rules on our Hack Warwickshire page…

Re-use of open data and applying the same idea within WCC

It has been a bit quiet around here while I have been concentrating on the Warwickshire Open Data intiative. This is now up and running and starting to show our strategic vision in action as well as paving the way for the new wave applications approach within the authority.

For the uninitiated, our open data site provides a one stop shop for public data held by Warwickshire county council. The key point is that the data is held in open, machine readable formats that can be re-used and re-purposed by any number of different web-sites and applications.

Currently the info is mostly static data but where possible we will be providing dynamic web services for developers and applications to take advantage of.

An early example of how this approach can provide real benefits in terms of scaleability and re-usability (not  a real word I am sure) can be seen through the web service that provides details of our recycling centres.

The data is provided via our Open Data catalogue at: http://opendata.warwickshire.gov.uk/datasets/recycling-centres

The raw data of which can be found at: http://ws.warwickshire.gov.uk/recycling_centres.xml

That same feed of data is feeding the wcc iphone application: http://itunes.apple.com/gb/app/warwickshire/id345904759?mt=8

…Which can also be used on the new iPad:

The data has also been used by this excellent entry to our Hack Warwickshire competition: http://www.pezholio.co.uk/warwickshire/

The demonstrable success of the open data approach has now led to a project to set up a similar catalogue and repository of internally facing data sources and web services.

The solution being piloted is the CKAN repository (also driving data.gov.uk)which we eventually hope will provide the foundation for both our internal and external open data sources – leading to the beginning of a universal API for the organisation.

Making Visionary Strategy a Practical Reality: The Open Data project

After a couple of years of persuasion, intimidation, bribery and tearful pleading we are now able to show how the strategic vision and  concepts that we have been pushing are now becoming a reality and are staring to provide benefits.

The decision to use existing and new projects to help build a strategic architecture has been endorsed by our management team and we are piloting the use of a design authority as a method of aligning our project work with the task of defining and building an architecture based on key concepts such as open standards, re-usable services, infrastructure as a service and business process management.

Along side this more formal work the open data project has shown that embracing new approaches can provide new benefits to the authority quickly and cheaply. Within three months, a two-man project team have designed, developed and deployed an application to provide WCC’s new open data initiative with a repository for data as well as a public facing interface to provide access to the information in question. Received wisdom is that such a project, resourced and carried out using more traditional methods would have taken up to nine months and incurred a far more considerable cost.

So how have our strategic concepts helped this project to deliver quickly and cheaply?

1. Service oriented infrastructure: The conscious decision to abstract traditionally embedded physical computing elements within our reference architecture opened up two key routes for the project to take. Firstly in terms of raw storage we could take advantage of Amazons EC2 cloud storage product, which will scale with demand and has a lead time for production deployment measured in hours rather than weeks. Secondly the developers chose to use the Ruby on Rails platform-as-a-service provider Heroku as the development and production environment for the open data application. This means we were up and running in hours, have incredibly simple development to production deployment processes, zero licensing costs and relatively inexpensive hosting costs (significantly cheaper than procuring building and supporting a physical and logical server environment).
2. Commitment to open standards and practices: Open standards are clearly a must for an open data project, but the non-proprietary nature of the development means that we will be able to provide it as an open source project for other public organisations to make use of.  This could contribute to the growing community supporting/promoting open data around the public sector as well as hopefully saving other people the time and trouble of starting such a project from scratch.
3. A focus on re-usable web services: The open data project was able to benefit from the previous work done to develop re-usable web services as a basis for our iphone application. Using single sources of the truth to drive multiple channels or applications will demonstrably improve the efficiency and accuracy of our ICT architecture. As an early example the same web service from http://ws.warwickshire.gov.uk/libraries is driving the iphone application, is available via our open data application at http://opendata.warwickshire.gov.uk/datasets/libraries and is already being used by an external developer on a mapping project at: http://www.meanboyfriend.com/overdue_ideas/2010/04/putting-warwickshire-libraries-on-the-map/. What we now want to see are the services made available by the open data project being re-used as widely as possible by both internal systems and external web sites and applications.

The challenge is to translate this way of working to a broader and more business critical set of projects. We have started the work to define strategic fit and take a new approach with a diverse number of projects such as HR-Electronic Records, Disabled Facilities Grants, Children and Young People’s Plan and our Network Comms Database.

As these projects progress we will be publishing our findings and experiences here.

Open Data: Building the tools to help you Hack Warwickshire.

In the last month or so we have shamelessly used the iPhone project as a way of kicking off the slightly less sexy, but ultimately much more fulfilling work on open data.

Although we originally wanted something available by the start of March, work has taken slightly longer than anticipated. However things are looking really good and hopefully we will soon be able to join the likes of Lichfield and London in the open data revolution. I can also reveal that we will be running some sort of exciting contest – more sketchy details below.

The main outcome I want us to achieve is establishing a web presence to provide access to the data as well as a framework of standards and processes for getting the data up there, making sure it is in the right formats and ensuring that it is properly maintained.

To this end Terry Rich-Whitehead (twitter: @whitehtj) and Steve Woodward (twitter: @equaliser) have been working on several fronts to make my naïve utopian open data daydreams a reality:

• Defining the formats and standards for data. There is some detail on the types of stuff that the chaps have been working on at http://abigbang.wikidot.com/opendata-project-plan-of-action. As well as defining the authority’s technical approach to open data we also want to start to follow the same standards for our internal systems and data stores – helping to improve efficiency and contribute to our nascent service oriented approach to applicatiuon development.
• Establishing a database to manage the sets of data. This includes a standard XML schema for the metadata that we will associate with each data set – working this way will help in enforcing good practice for data governance and version control. Additionally we will be able to automatically publish an RSS feed when new data sets are added or existing stuff is amended.
• Building a web presence to hold the actual data sets and act as a front end for all users. This will be set up as an adjunct to the Warwickshire web site but will be hosted in the cloud in order to contribute further to the development of our infrastructure strategy. It is highly likely that we will use Ruby on Rails as the platform.

The other thing we have to sort out is how to engage with the audience and ensure that we are providing a useful service. To this end Kate Sahota (twitter: @808kate) is working on setting up a user community (which will hopefully mirror the sort of thing seen at: http://www.datato.org/app/) as well as running the aforementioned “Hack Warwickshire” competition, open to all, where exciting prizes will be up for grabs for the people who can develop the best applications using the data and services that we make available. More details very soon indeed.