What is this?

Geomancer is a project with a simple goal: to create a prototype tool that will help journalists easily mash up data based on shared geography.

It’s such a common task for data-savvy journalists that it’s easy to underestimate the difficulty. But on deadline, it can become daunting. It requires locating the right data set to provide context, getting it into shape and into place to join it with the data in hand and then merging the two data sets — all before the reporter can even get started with the analysis.

We aim to take some of that complexity out of the way by providing an intuitive interface to discover available data sets for a given geography and supplement the data in hand with the relevant values. We’ll know it’s working when it appears too easy.

Read more on the Geomancer blog >

Data policy

Uploading your data to our server is necessary for Geomancer to work. We do not, however, store any of it permenantly. All processing is done in memory, so when you end your session, your uploaded data will automatically be deleted.

Please note: Do not upload any sensitive data to this site! If you have sensitive data that you want to use with Geomancer, look in to setting up your own instance.

API (for the nerds)

If you want to access Geomancer programatically, we have an API for you! The Geomancer API can be used to list the geographic types and data sources we support and geomance posted spreadsheets.

A few things to know:

Endpoints

GET /api/data-sources/

list all available data sources

This returns all available data sources and tables in Geomancer.

Query Parameters

All query parameters are optional.

Parameter Default value Description
geo_type none Filter the data sources by a specific geography type. See /api/geo-types/ for the full list of options.

Response

Parameter Description
machine_name Name of data source
description Data source description
info_url Link to more information on data source
base_url API url root
data_types
Parameter Description
count Number of tables in data source
source_name Name of data source
description Description of table
geo_types Geographic types supported
source_url API endpoint for this table
table_id API unique identifier of table
human_name Friendly name of table
columns List of columns that will be added from this table

Example

http://geomancer.io/api/data-sources/
Returns a list of data sources and tables in Geomancer

GET /api/geo-types/

list all supported geographic types

This returns all supported geographic types and related data sources in Geomancer.

Query Parameters

All query parameters are optional.

Parameter Default value Description
geo_type none Geography type. Used to fetch a single geographic type and related data sources.

Response

Parameter Description
machine_name Name of geographic type
human_name Friendly name of geographic type
formatting_notes Instructions on expected format for this geographic type
formatting_example Example of properly formatted geographic type
tables
Parameter Description
count Number of tables in data source
human_name Friendly name of table
table_id API unique identifier of table
source_name Name of data source
source_url API endpoint for this table

Example

http://geomancer.io/api/geo-types/
Returns a list of all supported geographic types and related data sources in Geomancer

POST /api/geomance/

run geomancer on a posted file

Once you've figured out what geographic types and data sources you want to use, you can geomance a spreadsheet via an API call by posting a file and a JSON configuration as a multipart POST.

Once a task is started, geomancer will start processing the request and give you back a link to your file with the additional columns by polling the /api/geomance-results/<session_key>/ endpoint.

Post Parameters

Parameter Default value Description
file none Must be in CSV, XLS or XLSX format. See more on spreadsheet formatting
form none Field definitions in a string encoded JSON blob like so:

{
  10: {
    'type': 'city_state',
    'append_columns': ['total_population', 'median_age']
  }
}

The key is the zero-indexed position of the columns within the posted spreadsheet.

The value is a dict containing the geographic type and the columns to append. The values in that list can be fetched from /api/data-sources/ and /api/geo-type/.

You can also match on two columns. To do so, separate the column indexes and geotypes with a semicolon like so:

{
  10;2: {
    'type': 'city;state',
    'append_columns': ['total_population', 'median_age']
  }
}

In this example, column 10 contains the city and column 2 contains the state.

Response

Responds with a key that can be used to poll for results.

Parameter Description
session_key Unique identifier of a task. Poll the /api/geomance-results/<session_key>/ endpoint to get your result and see when the task finishes
GET /api/geomance-results/<session_key>/

query the status of a geomancer task

Query Parameters

Parameter Default value Description
session_key none Unique identifier of a task. Returned after posting to the /api/geomance/ endpoint

Response

Responds with a key that can be used to poll for results.

Parameter Description
ready Returns true if the task is completed, false otherwise.
result Link to download your completed file. Only available if ready is true
status Success or failure of task. Only available if ready is true

We're open source!

All of the code for this project is open source. You can find, view and fork it on GitHub.

Technologies used:

We built this as an open & extensible platform, so that others can adapt this tool to their data needs. If you know how to program in Python, you can contribute new data to Geomancer (otherwise, post your request to our Google Group).

See documentation for contributing data >

Team and credits

Geomancer is a project of Associated Press and was implemented by DataMade, a civic technology company. We are:

Contact us

Have a question for us or having trouble using Geomancer? Join our Google Group.

Notice a bug or want to contribute to this project? Get involved on GitHub.