Open Government Data: possibilities and problems

View a .PDF version of this report here

“Government 2.0 involves direct citizen engagement in conversations about government services and public policy through open access to public sector information and new Internet based technologies. It also encapsulates a way of working that is underpinned by collaboration, openness and engagement”[1]

Back ground and context

The Political Issues Analysis System (PIAS) project (view original report .pdf)—in which this work is a sub-set—sought to investigate how citizens in Melbourne, Australia used the Internet to seek political information about key political issues. It also sought to understand how citizens contacted and interacted with their elected representative in relation to these issues. Through workshops, case studies, and the development and testing of prototype software, the research uncovered some notable trends in terms of user engagement with important aspects of the formal political process online.

The PIAS project principally focussed upon citizen information use through investigating interaction with party web-sites and the policy documents that they made available. However, the participants in our study largely found 1), the sites difficult to use 2), the information hard to navigate and compare with other policies and 3), the written policies unreliable and unclear. One of our key recommendations from the study emphasized that polices published by political parties should be made available in a ‘machine readable’ form so that they can be automatically aggregated into other systems to enable citizens to compare the policy positions of the parties. Also, strict metadata publishing standards and frameworks should be used so that the information aggregated is of a high-standard allowing it be re-utilised effectively.

This work compliments the PIAS project through listing some of the key projects and services that available that utilise government data. It also explores in more detail the limited availability of what could be termed ‘democratic data’. For the purposes here, “democratic data” is described as: 1) Hansard: making the working of government available in new ways, 2) Transparency: newer forms of transparency through ‘data’, and 3) Policy: enhance and extend the policy making process through online open consultation.

Why Open Access to government data?

Much of the impetus behind the drive for Open Access to government data stems from a push for greater transparency to the functions of government. However, in the case of Victoria, for instance, much of the data being released within the Gov 2.0 agenda tends to be of an administrative nature and of little democratic potential. Whist the Parliament of Victoria does make an enormous amount of useful material available to the public through its website; it is not made available in a technically sophisticated, machine readable way, to take full advantage of the potential of the Internet. Bills are only available in .pdf or word format and the most important document about the workings of government, Hansard, is also only available as .pdf (although it is possible to do a full-text search of Hansard from 1991 onwards). If these important documents were available in a machine readable form, they could be utilised by application developers in innovative ways.

The Open Access movement is a push to make data both machine readable and interoperable so that it may be linked together and leveraged for all sorts of purposes. This may be for new business opportunities, medical research, or new areas of social research. However, doing this is no easy task as multiple data sources require linking and matching across diverse and complex systems (and ‘cleansing’). The first step in this process is to expose data in a standardised way so that it may be located and machine-read. The Victorian public sector has a policy framework specifically designed to achieve these tasks titled the Victorian Public Sector Action Plan. Two key points are:

  1. Participation: Engaging communities and citizen through using Government 2.0 initiatives to put citizens at the centre and provide opportunities for co-design, co-production and co-delivery.
  2. Transparency: Opening up government through making government more open and transparent through the release of public sector data and information[2]

Making data available in this way can only help to “deepen democratic processes” and promote a strong and healthy democracy (however this is often an aspiration rather than an actuality).[3] Accordingly, there is a promising international trend to promote a two-way dialogue between political representatives and the public through combining ‘’democratic data’’ with citizen produced data through popular social media platforms.[4] Rather than building a completely new platform (as has been the case with a number of somewhat underutilised government initiatives), some projects take advantage of largely existing and heavily used social network platforms and provide tools and services to augment their existing capacity (usually to inform and communicate government policy processes) The large EU funded WeGov project[5] and other projects in the US and Europe are welcome movements in this direction. [6]

Continue reading “Open Government Data: possibilities and problems”

What they are saying’: Political Issue Analysis System (PIAS): Political Issue analysis in an age of the ‘data deluge’

(This new seeding project has just been accepted for funding from the Institute for Broadband Enabled Society (IBES) at the University of Melbourne. Led by VeRSI and myself, it is a short project with results available towards the end of the year or early next year).

Summary of Proposal

The Internet is recognised as a vital component of our political information systems.  Although extensively used by governments and civil society groups, its effects upon political processes; particularly deliberative political processes, currently remains relatively unknown.  Emerging research suggests that the Internet’s capacity to easily produce information has also led to data overload, undermining its deliberative potential.  With the advent of the National Broadband Network the ‘data deluge’ promises to intensify increasing the need for political information—in its various guises—to be delivered in much more meaningful ways.[1] This is especially important for younger audiences who are increasingly abandoning broadcast media in favour of online political information[2].

This project is an iterative study and design of an online ‘Political Issues Analysis System’ (PIAS) to assist users’ research and analyse political issues. It will deliver information about important political topics (ie. environmental issues, socio-economic issues, immigration, government policy etc.) using important data sources within a coherent ‘deliberative’ framework.  It will evaluate the needs of users to comprehend political issues through the application of a number of semantic indexing and data matching tools and design a prototype system.  It will do this in part through five public workshops using the University of Melbourne’s Usability Lab; each workshop focussing on a particular issue utilising particular tools and methods.[3] It will in tandem uncover recommendations to assist in the design of a unique software tool that fosters user-driven processes to effectively filter and visualise online political information obtained from government data-sets (partly within the ‘Government 2.0’ policy framework), the media, NGOs, historical data, and other user-generated online sources; (blogs, video etc).

The outputs of the research will be a working prototype as well as a report documenting the research outcomes with a series of recommendations for further research. This project may lead to the first major study of online deliberative processes within Australia; competitive within the ARC’s Linkage or Discovery scheme. The work will be of benefit to governments, community groups and other major producers of political sites and the users of such sites. The project is within IBES’s Social Infrastructures and Community theme and in particular, adheres to IBES’s and VeRSI’s shared aspirations ‘to make existing and available data more accessible’. In summary the broad aims of the project are:

  • To explore the evolving applications of online political information tools in an Australian and International context (especially in the analysis of broadband-enabled video and audio)
  • To examine deliberative processes with a number of stakeholder groups using semantic indexing methods and various communication tools at the University’s IDEA Lab.
  • To build, test and provide further recommendations for a ‘Political Issues Analysis System’ (PIAS)

Through these processes we address the following research questions:

  • How can we better understand online deliberation in the international and Australian context and what tools need to be developed to assist this?
  • How can we better design deliberative ‘ideas’ using data and online analysis tools that will involve people in a meaningful and inclusive way in consequential goal-orientated political processes?

Approach and Outcomes:

The combination of theoretical groundwork, empirical study, and the design and implementation of the PIAS, will make an important contribution to the emerging body of research on the nature of political information on the Internet and in particular, the use of government data within it. Of chief significance is that the research will make explicit and open up to critical analysis the dichotomy between the availability of government and other data sources and effective online deliberative design. By consciously foregrounding information abundance as a condition of the present ‘information revolution’—through a unique fusion of political theory with semantic analysis and clustering tools—new perspectives will emerge and fresh research areas in design will open up.

The approach, then, is both innovative and unique because it combines the theoretical sophistication of Politics and Media Studies with the technical proficiency of Humanities Computing, eDemocracy, and Information Systems to expose important issues of online political information to critique in ways that were previously unavailable. [4] The work will open up theoretical and technological pathways towards a more genuinely identifiable (and sustainable) online political engagement and democratic structuring.

Technology and potential collaborators:

Potential collaborators for this work include the UK’s They have developed some of the UK’s most well-know sites including and its local derivative,  The open source solutions, API, raw data and results will be collaboratively developed and shared with mysociety and OpenAustralia to complete the PIAS. Likewise, solutions developed through the ‘inquiry into Improving Access to Victorian Public Sector Information and Data’ as well as the Federal ‘e-Government Strategy’ will be investigated and may provide potential collaborators. In essence the PIAS is a ‘parsing’ project; to parse structured government and other data sets to extract and deliver meaningful political information to a general audience. It will explore ways to crawl, cluster and analyse unstructured data contained in blogs and other ‘unofficial’ sources including video and audio (perhaps using XPROC processing).

The broad samples obtained through the PIAS iterative design workshops and subsequent prototype will provide a unique model to analyse web-based dialogue, agenda setting, and responses to official government positions on important political topics. This work may be up-scaled at a later date to include other collaborators; particularly the Pollsters who may be eager to invest in such a system.

[1]One of the first major agencies to coin the term the ‘Data Deluge’ was the UK’s JISC (Joint Information Systems Committee):  Briefing Paper, Data Deluge: Preparing for the Explosion in Data, 1 November, 2004  <> (Accessed 14 May, 2010).

[2] See: Clare Kurmond, Readership Decline Continues for Papers, Sydney Morning Herald, Sydney, 14 Mat, 2010

<> (Accessed 14 May, 2010).

[3]Interaction Design Evaluation Analysis (IDEA), Department of Information Systems, University of Melbourne,

<> (Accessed 14 May 2010).

[4] Carson, L ‘Avoiding ghettos of like-minded people: Random selection and organisational collaboration’ in S. Schuman, (ed) Creating a Culture of Collaboration, ed. Jossey Bass/Wiley.pp.418-423.

Podcast/Press Release: ‘HE in a Web 2.0 World’ report


JISC recently released a report on ‘Higher Education in a Web 2.0 World’. The aim of the report is to critically assess recent Web-based developments commonly termed ‘Web 2.0’ and assess them in relation to education and pedagogical practice. The report is available on-line and in hard-copy; plus some of the key findings are discussed in a podcast with David Melville, one of the report’s authors.

Some of the key findings of that report are that students may not be developing the critical skills to evaluate information and that ‘Web 2.0’ may be promoting shallowness. And although Melville discusses Web 2.0 as a solution to all sorts of social ills from those associated with multiculturalism and globalism to a ‘collaborative’ deficit in education, I do worry that the report itself is not critical enough as many technologies are produced within commercial and other contexts that may not have the unique interests of education in mind.

The report and podcast is available on the JISC website; discussions in this forum are most welcome.

Private Sheriffs in Cyberspace: Jonathan Zittrain OII Event: London, 19th May 2009

On Tuesday evening I attended an Oxford Internet Institute sponsored lecture by Jonathan Zittrain, Professor of Harvard Law School, Co-Founder and Faculty Director, Berkman Centre for Internet & Society (at the salubrious legal offices of Wragge and Co). Zittrain talked about regulation on-line by major Internet players such as Facebook and Apple and asserted that many of the regulating methods employed by them were outside of the rule of law. His contention was that many ‘Web 2’ companies have immense and increasing social and economic power within the fabric of our lives and are regulating their sites in a rather ad hoc and random way in terms of banning application developers, individuals, and groups that do not adhere to their governance structures. He used a number of examples to support his thesis, plus introduced a simple graph to illustrate emergent styles of governance:


Hierarchy >poligarchy


As an example of a ‘bottom-up’ governance structure Zittrain cited Wikipedia which includes a deliberative system to manage thorny editorial decisions. As a top-down system of governance he cited Facebook; although Facebook is beginning to include the community in decisions relating to its structure and functionality. He used the term ‘social governance’ to describe this bottom-up governance approach and suggested ways in which this approach may be designed into a system (through flagging certain tasks that help tap into the ‘reservoir of good will’ of the community). A well-designed system should have mechanisms to ask users for their input.

Although I tend to agree with many of the arguments of Zittrain, I feel there is a tendency to overstate the importance of sites such as Facebook and Youtube to the broader public. Sure they are popular, but this isn’t the British Library, the University of California, or the Library of Congress we are talking about! They are just large and fashionable web sites; a small part of the fabric of our complex lives. And commercial companies will perhaps always act in their own interests; either commercially or ideologically.

I suppose what is needed is some sort of bill of rights/responsibilities that is general to the operation of the Web within a certain geographical region balanced with the specific values of the site in question. There is nothing wrong with sites asserting behaviour norms upon users; but then again governance structures should be transparent and open; not outside of acceptable norms of the broader public sphere. A site should never assert policies that are deemed racist nor discriminatory (perhaps this is Zittrain’s anxiety when he claimed than many sites operate outside of ‘the rule of law’). The relationship between the community and the platform should always be fair and equitable; especially in large user-based sites such as Facebook. In my mind, governance structures, whether online or off, should always be open and transparent.

One of the respondents to the talk, Ian Brown, a Fellow at the Oxford Internet Institute (and author of the recent report Database state) asserted that the relationship between Citizen and State and Cyberspace needed to be reconsidered. He also claimed (from his experience) that that the issues raised by Zittrain are not well-known in the UK;  especially in senior government levels. As an historian (and not a legal expert), my  scepticism relates to the actual significance of the entire debate.  I suppose that the significance of the debates depends on the importance the public places on systems such as Facebook and their governance structures. I may agree with Eric Hobsbawn that Terrorism is more a perceived threat in the UK that an actual threat (to the state), but then again the public is led to believe otherwise so it now painfully significant.  So if the debates about governance are perceived to be important by the public; then they will become important. So we may have a ‘Facebook Parliament’ in the making deliberating about the rise of rudeness on Facebook . They should start with the Tube system!

Soap Box Project

This project led by Dr Sally Young at the University of Melbourne will be of interest to those who wish to understand the history of political advertising in Australia.

Politicians and members of the public would once stand atop a soapbox in order to shout their message across to an audience. Now they use a wide range of media including TV ads, social networking websites and all manner of radio and television appearances. This website will harness a range of materials ─ including photographs, texts of speeches, transcripts of debates and political ads ─ to allow visitors to see (and assess) how Australian political actors communicate.

Election campaigns are usually focused on the short-term – the hectic 3 to 6 weeks of the formal election campaign. This website instead allows you to see elections as a continuum; to look back over time to see what the parties and their leaders have said (and promised) in the past. The website includes material dating back over a hundred years so that visitors can recall recent campaigns or compare current events with historical ones (link).