With the #Watergate scandal in full flight, it is a useful time to launch my newest section of the site where the public can search Commonwealth tenders.
While the government has their own searchable version of this data, my version is designed for the purposes of financial transparency as opposed to meeting the needs of tender suppliers.
The government's search does not provide any analysis whereas my site is able to list number of contracts awarded/received by different criteria providing overall summaries:
I have to admit, the peculiarities of this huge dataset did present some challenges which I've had to work around to make the site searches as effective as possible.
Chief among these issues is the multiple spellings including typos that applies to most company names in the system and the multiple ABNs that can apply to each of those companies.
Commonwealth tenders are one of the oldest open datasets with public data going back to 2000. This also makes it quite a large and cumbersome set of data for a server to have to search and perform algorithms on.
With over 441,000 individual ABN's in the system, it is quite a web to untangle!
To deal with performance issues, my searchable version spans the decade from 2009 to the current/last month. I expect most controversies are only going to be of interest if they occur in the past decade.
With so much data to search, I've expanded from my standard one search box per project to having two so that users can search not only by company name but also by the contract description.
Contract description search works in two ways. The code looks for an exact match for the input and if there is an exact match then those contracts are displayed. Users will see which agencies award contracts matching the form input exactly, who those contracts are awarded to and where those business addresses are located.
Users can drill down into the specifics of each contract including the tender method (open, pre-qualified, limited), agency, contract value and amendments.
There are 393,433 Limited tenders, 262,738 Open tenders and 74,600 Pre-qualified tenders in the system (2009- present). You can find out more about government procurement here.
If there is no exact match the search falls back to a Boolean search which matches any words in the input. This has both pros and cons but I have opted to be more inclusive and generate a list of partial matches for the user to click on to drill down rather than risk missing matches through a more exact match. Keep in mind these looser searches can take quite a few seconds to run on the 700k+ row dataset.
If you follow the embedded links above you will see that the searches bring up the #Watergate contracts awarded to Eastern Australia Agriculture if you drill down on the top title 'purchase of water entitlement'. Perhaps ANAO will find the list useful in their audit!
You can see that the contract values for these purchases appear to be the top values paid by the Department of Agriculture. If you are doing research you need to be careful however that you do not miss alternative descriptions that also come up further down the list:
If you want to search political donations and pecuniary interests to see if tender recipients have donated, it is available here.
If you're more interested in the PNG/Paladin contracts you might want to drill down by Agency and click on the Department of Home Affairs, although you can also pop search terms like Manus into the description. Enjoy!
If you would like to see this work stay online, be updated regularly with new data and/or improved, please become a donor via Patreon or send directly to my account [Rose Williams Bank: St George BSB: 112-908 Account number: 485 746 256]
This is my only income so every donation counts!