Welcome to Australia's biggest political and financial transparency project!

Help Me Turn The Tables On Government Surveillance
Commonwealth grants will help fill the blanks in understanding who gets what, but my work needs your support
Rosie Williams
14th Oct 2019

Following general [focus group] discussion about ABS, its role and how it is perceived, the concept of the ‘evolution of the ABS’ was introduced by the moderator. This concept was explained as a transition from ‘snapshots in time’ featuring single data sets to ‘movies’ whereby data sets are linked either across time or across different data collections. (Colmar Brunton, 2014)

In a country where the government is hard at work joining administrative data from different agencies together to create a 'movie' of our lives, very little is being invested in the flip-side of this equation: tranpsarency and accountability of government and the reasons behind the decisions of the powerful. Apparently Australia's open data policy is about putting the poor and vulnerable under a microscope, not improving and increasing the data the public is most interested in- who and what is influencing decision-makers.

Today I launch a funding drive to add yet another project to my large and colourful pallette. In this article I'm going to overview the recent changes I've made across the entire site.

AusGov.info is a massive transparency project which is made up of multiple projects allowing the search and analysis of financial data published by the Commonwealth government. Theses projects allow search and analysis across multiple datasets to highlight potential links between say, political donations, tax deductions and government tenders. This type of research puts the focus back where it needs to be: on the people making the decisions and the people benefiting from them.

Each of these projects champions a particular dataset- allowing search and totalling across a comprehensive range of variables. There are actually more projects than appear in the main menu (pinned to the top of each page) which now only features the main ones used by the public.

Project Topics

Who gets grants?

Find out which organisations receive the most government grants and the Ministers responsible

Search by:

  • Organisation
  • Agency
  • Program
  • Category
  • Electorate
  • Minister

Who Meets Who?

Find out who's meeting who with NSW Ministerial diaries data

Search by:

  • Date
  • Organisation Name
  • Agenda topic
  • Ministry

What pollies own, owe & are given

Search the pecuniary interests register for potential conflicts of interest and which politicians own property near you

Search by:

  • MP
  • Keyword
  • Category
  • Party
  • Property Location

Who gets what?

See tax paid by top-earning corps, relevent gifts, grants, shares & tenders

Search by:

  • Revenue
  • Tax Paid
  • Donors
  • Party
  • All datasets

Who wins contracts?

Find out who gets the biggest tenders & which countries supply the govt

Search by:

  • Biggest tenders
  • Agency
  • Supplier
  • Description
  • Title
  • Country

Which charities rule the roost?

Find out which charities receive the most in donations and where they operate

Search by:

  • Organisation
  • Assets
  • Revenue
  • Donations
  • Activity
  • Electorate
  • Operating Country

It is easy to get overwhelmed with the amount of information on the site but all the projects work the same way. There are two main buttons in each page. The main menu is pinned to the top of each page and is used to switch between the projects.

The button in the middle of every page reveals the search options customised for each project. These vary from project to project depending on the characteristics of the dataset it features. The Commonwealth tenders project only contains tenders data (integrated with mapping data) because it is such a huge and complex dataset.

Other projects champion a certain datasets but also provide results from other projects. It is only by integrating data that patterns and relationships can be spotted. The example of this integration which is easiest to use and most powerful is the advanced search which provides matches from NSW Ministerial Meetings, pecuniary interests, AEC political donations, tax transparency data, Commonwealth tenders, ACNC charities data and DGR data. Check out the example search for records matching Woolworths- now with Commonwealth grants!

Today I also launch a funding drive to add Commonwealth grants. While Commonwealth tenders data across all agencies has been collated into a single dataset for download for years, it was a long road to have the same level of accountability implemented with Commonwealth grants. Grants.gov.au (a site similar to AusTenders) finally launched in January 2018.

You can get a good taste of what the Commonwealth grants data looks like in my article article on religious charities.

Shining A Spotlight On The Finances Of Religious Charities
Religious charities receive millions in Commonwealth grants, but not all of them are required to disclose their finances to the regulators.
Rosie Williams
14th Oct 2019

One big issue with the grants prior to Grants.gov.au is that (unlike with Commonwealth tenders), no ABN was included with any of the published data. Administrative datasets collected in the course of the government carrying out its functions (as opposed to those collected solely for research) have quality issues that make them very challenging to work with.

A main issue is that organisation names are spelled every which way that a human being can find to spell it. This is a major problem with Commonwealth tenders where organisations also have multiple ABNs because it hampers the capacity to find or total spending for each specific organisation. Finding ways to resolve these issues is technically difficult and often under-estimated. ABN was finally introduced to Commonwealth grants reporting with the launch of Grants.gov.au. (despite being specified in RMG 412 but ignored for years).

I have had to face a chicken and egg challenge with such a large and complex project in that people want to see the evidence of what comes at the end of a very long process of integrating multiple administrative datasets in the form of investigative journalism before they feel it is worth it to invest in this work or give credit where it is due.

In a context where the complexity of this work is too-often underestimated I struggle to solve all these problems at the same time in a time-frame that suits the 24 hour media cycle where articles are produced in hours, not the days, weeks or months that in-depth research requires.

Due to the scope and complexity of this task, I've only begun to demonstrate the value of the work in recent months in articles on the not-for-profit sector, particularly with the work on tranpsarency of religious charities. Please take a look at the about page and become a regular donor or make a one off payment. Support option are available at the foot of every page of the site.

In preparation for this funding drive, I did a wholesale re-design of every project with the aim of improving usability. This basically required me to unpick the most complex projects and stitch them back together in a way which reflected a new logic that I could apply to every project.

Usability is a huge challenge with this kind of project as the more data there is (all of which is requested by the public at some point) the more complex it is to interact with it. In a way, having so much information becomes confusing. At the same time, all the projects are here for a reason so I'm constantly trying to reconcile these conflicting ends.

The fact that this about as far from 'another Wordpress site' as you can get seems lost on a lot of people. I am not downloading code written by large groups of people, I am writing that code myself, because financial transparency through integrated data is not a content-management system and code to create it on this scale does not exist elsewhere on the web.

Each and every line of the original code and articles now constitutes over 150 MB (without the images). I believe hand-coding and maintaining a project of this size from scratch alone would be a considerable challenge for any developer. There are over 80 tables in the database totaling over 800 MB in data. While the government makes this data available, administrative data can't just be used as it is, it needs to be cleaned and often integrated with other datasets. This kind of functionality is unique to every dataset and on top of designing and creating the site, cleaning and updating the data, I have to do the investigative journalism on top of it!

I do like a challenge though and all I ask is that people help me to get from month to month so that I can continue what I have invested so much of my life in. I began crowdfunding in 2015 and there are donors who have been with me ever since. Thank you to all the people who have ever supported my work in whatever way you have done so. I am grateful for that support, and in turn, I hope that you are proud to know that you have helped make all this happen.

Copyright licence applies only to text of article (not videos, images and other included third-party content). If you republish my work, a donation is appreciated. Click here for details.

Article Topics

[8] Privacy
[5] Charities
[3] Interests
[3] Security
[2] Whistleblowing
[2] Thinktanks
[2] Budget
[2] Open Government Partnership
[2] About
[1] Welfare
[1] Grants
[1] Racism
[1] Housing
[1] Immigration

Articles matching about

Help Me Turn The Tables On Government Surveillance
Commonwealth grants will help fill the blanks in understanding who gets what, but my work needs your support
Rosie Williams
14th Oct 2019
Meet The Maker
There is a lot of information at this site, but where did it come from and who is responsible for it?
Rosie Williams
6th Oct 2019