Mashup Australia competition entry from NeoGopher

Our submission has two parts:

Part one: 16 MashupAustralia lists in NeoGopher.

NeoGopher is a universa data browser developed in Melbourne, Australia and is designed to allow browsing of an HTTP web based API. NeoGopher is the intersection of linked data and the web. We have created 16 NeoGopher lists, each list presenting a data set from data.australia.gov.au. To see our MashupAustralia lists, download NeoGopher

Part two: 26 HTTP web based APIs.

We have converted CSV and XLS data from data.australia.gov.au into 26 HTTP web based XML APIs. These APIs are accessible by the public and support XQuery access including searching, sorting, paging and return compressed XML.

Screencast

Download screencast:   · Low quality (24MB)   · High quality (43MB)

API Endpoints

Dataset Description API Endpoint
NSW Crime data

New South Wales Crimes by offence type, month and Local Government Area.

data.neogopher.com/au/gov/nsw/crimeData.xml
ACT – Barbecue (BBQ) Locations

Location and attribute detail of Parks Conservation and Lands (PCL) barbecues – located within major urban open space locations across the Australian Capital Territory.

data.neogopher.com/au/gov/act/publicBBQs.xml
Australian Atlas of Mineral Resources, Mines and Processing Centres

The Atlas delivers authoritative minerals and mining information to individual Australians and provides a virtual-showcase of the industry for global audiences.

data.neogopher.com/au/gov/mineralProcessingPlants.xml data.neogopher.com/au/gov/operatingMines.xml data.neogopher.com/au/gov/ports.xml
Australian Broadband Guarantee program data

Paid subsidised broadband connections by postcode and year

data.neogopher.com/au/gov/broadbandSubsidies.xml
Austrics Timetable set

Darwin, Casuarina, Northern Suburbs, Palmerston Timetable sets

data.neogopher.com/au/gov/nt/busRoutes.xml
Byteback™ locations (VIC)

Locations for Byteback™ – a free computer recycling service that lets Victorian residents and small businesses dispose of old, unwanted and non-functioning computers safely. This service is currently being trialled in Victoria only.

data.neogopher.com/au/gov/vic/bytebackLocations.xml
Commonwealth Agencies

Data of records series held by the NAA including descriptions of Commonwealth agencies responsible for creating the records series and consisting of:
Agency.xml = the administrative history of the Commonwealth Government – It contains information about Agencies. An Agency creates a series of records based on its functions as agency functions change, the custodians of the [...]

data.neogopher.com/au/gov/agencies.xml data.neogopher.com/au/gov/items.xml data.neogopher.com/au/gov/series.xml
Crime incident type and frequency, by capital city and nationally

Crime incident type and frequency, by capital city and nationally

data.neogopher.com/au/gov/crimeIncidents.xml
Detox Your Home Locations (VIC)

Victorian locations where unwanted household chemicals can be disposed of safely. Locations comprise permanent drop-off points and a schedule for a mobile service.

data.neogopher.com/au/gov/vic/detoxYourHomeLocations.xml
Economic Stimulus Package Project Data (community infrastructure & road & rail projects)

Nation Building – Economic Stimulus Plan projects for the Regional and Local Community Infrastructure Program and for Rail and Road projects from the Department of Infrastructure, Transport, Regional Development and Local Government

data.neogopher.com/au/gov/economicStimulusPackages.xml
Fire Brigade Locations (VIC)

Fire Brigade Locations

data.neogopher.com/au/gov/vic/fireBrigades.xml
Location of Centrelink Offices

Location of Centrelink Offices

data.neogopher.com/au/gov/centrelinkOffices.xml
Location of Medicare offices

Location of Medicare offices

data.neogopher.com/au/gov/medicareOffices.xml
Mildenhall photographs of early Canberra

The NAA series registration number for the Mildenhall data/photos is CRS A 3560.

data.neogopher.com/au/gov/act/mildenhallphotos.xml
Northern Territory Place Names

Northern Territory place names register extract, containing locations of aboriginal communities, ambulance stations, roadhouses, urban parks, post offices, museums, libraries, fire stations, colleges and schools.

data.neogopher.com/au/gov/nt/places.xml
PLGR – Playgrounds (ACT)

Location and attribute detail of Parks Conservation and Lands (PCL) playgrounds – located within major urban open space locations across the Australian Capital Territory.

data.neogopher.com/au/gov/act/publicPlayGrounds.xml
Police Station Locations (VIC)

Police Station Locations

data.neogopher.com/au/gov/vic/policeStations.xml
Powerhouse Museum Collection

Search 73,567 objects collected from 1880 to the present day from steam engines to fine glassware, postage stamps to robot dogs. This interactive database contains thousands of zoomable images and research into the Museum’s collection, much of it made public for the first time.
The Museum is continually adding to its collection and a selection of [...]

data.neogopher.com/au/gov/powerhouseMuseum.xml
Public Internet Locations (VIC)

Public internet locations

data.neogopher.com/au/gov/vic/publicInternetLocations.xml
ResourceSmart Retailers (VIC)

The ResourceSmart Retail Program provides information available through over 300 partnering stores to help consumers compare the environmental performance and running costs of the products they’re looking to buy. This dataset provides address and contact information for all program partner stores.

data.neogopher.com/au/gov/vic/resourceSmartRetailers.xml
School Locations (VIC)

School Locations

data.neogopher.com/au/gov/vic/schools.xml
Solar Report (VIC)

Using data from the Bureau of Meteorology and figures from the Climate Group, the Solar Report shows the percentage cost and greenhouse gas emissions that households in Victorian regions would save on their hot water bill if they had a solar water heater. Data is also expressed in the equivalent number of cars taken off [...]

data.neogopher.com/au/gov/vic/solarReport.xml
South Australian Boat Ramp Locator data.neogopher.com/au/gov/sa/boatramps.xml
South Australian Recreational Boating Hazards Map data.neogopher.com/au/gov/sa/boathazards.xml
TOLT – Public Toilets (ACT)

Location and attribute detail of Parks Conservation and Lands (PCL) public toilets – located within major urban open space locations across the Australian Capital Territory.

data.neogopher.com/au/gov/act/publicToilets.xml
Water consumption in Melbourne

Water consumption in Melbourne

data.neogopher.com/au/gov/vic/waterConsumption.xml

API Summary

API consists of an endpoint URL, an XQuery (_query) parameter, and paging (_start and _howmany) parameters.
eg: http://data.neogopher.com/au/gov/medicareOffices.xml?_query=[XQuery]&_start=1&_howmany=10

Endpoint URL

The endpoint URLs are given in the right-most column of the above table.
Each endpoint may be accessed via HTTP on port 80.

XQuery

The _query parameter defines which information to extract from the data source.

In most cases, a good starting query would be:

  for $result in /*/* return $result

However, depending on the XML source data format, it usually makes sense to make the

XQuery's XPath more specialised for better performace. For example:

  for $result in /data/sheet/row return $result

would be appropriate for many endpoints.

Tip: You can begin with the more generic XQuery to see that data format first, then specialise from there.

Tip: Each endpoint listed in the above table is hyper-linked to an example URL containing a good starting XQuery for that data source.

Sorting can be achieved by adding an appropriate "order by" clause to the XQuery. For example:

  for $result in /data/sheet/row order by $result/cell[@name='Postcode']/@value ascending return $result

will sort on the 'Postcode' field, in ascending order.

See http://www.w3.org/TR/xquery/ for more information on XQuery

Paging

Results can be split into pages by using the _start and _howmany parameters. The _start parameter specifies which result index to being at (results indexes begin at 1), and the _howmany parameter specifies the maximum number of results to return.

Putting it all together

So a complete HTTP request might look like:

  http://data.neogopher.com/au/gov/medicareOffices.xml?_query=for%20$row%20in%20/data/sheet/row%20return%20$row*_start=1&_howmany=10