Manage your Data with ODK Aggregate

Open Data Kit (ODK) is a suite of tools that allows data collection using mobile devices and data submission to an online server, even without an Internet connection or mobile carrier service at the time of data collection. Once you've collected data in the field with ODK Collect, you can upload and manage your data using ODK Aggregate. ODK Aggregate is the intermediary server storage platform that accepts the data and can send it on to external applications, if desired. ODK Aggregate also allows you to download your datasets in aggregated formats, such as one .csv file. ODK Aggregate allows you to use Google's AppEngine hosting platform for managing your remotely collected data online.

Created by developers at the University of Washington's Computer Science and Engineering department and members of Change, Open Data Kit is an open-source project available to all. Please visit their ODK project page for more information, recent updates, more tutorials, and to contribute to the project.

 

Prerequisites

  • No programming skills needed!
  • We strongly recommend that you take the tutorial "Mobile Data Collection with ODK Collect" before proceeding.
  • You will need a Google Account to view your data in ODK Aggregate. Don't have a Google Account? Get one here. (It's free!)

In this exercise, we will use the same sample.xml form we used in the previous tutorial, Mobile Data Collection Using ODK Collect.

Let's Get Started!

  1. Go to the training instance of ODK Aggregate at http://odk-tutorial.appspot.com.
  2. This instance of ODK Aggregate is available to anyone with a Google Account. Log in with your Google Account.
  3. You will see a list of forms that have been uploaded to the instance of ODK Aggregate for http://odk-tutorial.appspot.com.


Upload a Form to your ODK Aggregate Instance

You can upload forms that have been created with ODK Build or using xforms to your ODK Aggregate instance. Once you've uploaded a form, anyone with a Google Account who knows the domain name of your AppEngine instance can view your form and its submissions, and download your form to their mobile device to upload submissions to your form. (You can set access preferences in your AppEngine settings. Read more in "Getting Started with Mobile Data Collection Using ODK".)

For this tutorial, we have already uploaded a form to http://odk-tutorial.appspot.com. If you won't be creating new forms or uploading forms, you can skip this section.

  1. Click the "Form Management" tab at the top of the page.
  2. Click the "+Add New Form" button.
  3. You will be able to select the form to upload from your computer. Choose the .xml form you wish to upload to your ODK Aggregate instance. If your form has media files associated with it, for example, images inside the form's questions, you can also upload media files that your form references in the XML here.


  4. Click "Upload." If your XML form is in the correct format, it will be uploaded and you'll see it appear on the "List of Forms" page.


Delete a Form from your ODK Aggregate instance

You can host up to 500 MB of data storage on your AppEngine instance. If you would prefer to have control over which forms are uploaded to your instance, you can learn how to manage user permissions to your AppEngine instance. You can also delete forms you don't want to have appear.

  1. Click "Delete a Form".
  2. A dialogue box will appear showing you a list of forms. Check the box next to the form you wish to delete, and hit "Delete Forms."

Upload a Submission Manually using a USB Cable

ODK Aggregate allows you to import your data submissions individually if you don't have an Internet or mobile data connection to send your forms to your ODK Aggregate instance directly from your mobile device. This is useful if you want to transfer files from your mobile device to a computer or external hard drive and free up some space on your mobile device. You can use a USB cable to connect your phone to your computer and upload the files to ODK Aggregate when you next have access to the Internet.

  1. Connect your mobile device to your computer with a compatible cable. You can use a mini USB to USB cable or whatever proprietary port that is used by your phone. Your device probably came with a cable that allows you to connect to a computer. Be sure to mount your device to your computer so that you can transfer files over the USB cable. You do not need an Internet connection for this step.
  2. When you have access to the Internet again, go to your ODK Aggregate URL (i.e. http://odk-tutorial.appspot.com) and click "Form Management" and then "Submission Admin".

  3. Find the files on your computer, along with any corresponding data files, such as photos in .jpg format, that you would like to link to your data submission.
  4. Hit "Upload Submission." Your data submission will now appear within the list of submissions corresponding to the appropriate form.

Viewing Your Form Submissions

  1. Click on the Submissions tab and select the form "Sample" from the drop-down menu, or the name of the form whose submissions you wish to view. Click on the "View Submissions" button next to the "Sample" form.
  2. You'll see all submissions, created by yourself or anyone else who has sent completed submissions to http://odk-tutorial.appspot.com. Submissions with photos have a link on the far right of the row to view each individual photo submission.

Other Form Management Tasks

  • Published Data: You can also view where your datasets have been published apart from the ODK Aggregate instance. This tab tells you, for example, if your dataset has been published to a Google Fusion Table, and includes a link to that Fusion Table.
  • Submission Admin: For each form, you can decide either to purge submission data from your ODK Aggregate instance or you can upload submissions manually. You may want to purge submission data if you're storing your submissions elsewhere, such as in an Excel file, and you want to re-export only the new submissions that come in for the latest field data collection effort, without including all the prior data that was collected.

Exporting Data from ODK Aggregate as a CSV File

ODK Aggregate makes it easy to export your data as a .csv file. You might be interested in this functionality if you want to import your data to other servers or applications such as Google Spreadsheets or Microsoft Excel.

  1. Click "Download CSV".

  2. A .csv (comma separated values) file will be downloaded to your computer.

Setting up User Access Lists and Admin Settings

ODK Aggregate allows you to control who has access to your data in versions 1.0.n and newer. You can decide who can submit data, who can view data, who can manage forms and who can be a site administrator.

  1. The first time you set up access permissions to your ODK Aggregate instance, you must log in with the Google Account that you used to set up the AppEngine instance. Select "Sign in with Google".
  2. Now you should see another tab next to "Submissions" and "Form Management", called "Site Admin". Open the "Site Admin" tab.
  3. Add new users, either by assigning them usernames (for sign-in via ODK Aggregate, not via a Google Account) or by entering their Google Account. Then press the "Add" button.

  4. Once you've added users to the account, you can set their access permissions in the "Edit Users" section. The capabilities are explained above the Edit Users section.
  5. You can also allow Anonymous access, so that anyone with a Google account can view your ODK Aggregate instance. This is the default setting, which you can also remove. If you allow anonymous retrieval of images, audio and video, this means that if you have published your data to a Google Earth KML file and hosted that KML file on the web, you want to allow anyone who downloads that KML file to view the content inside the balloons. If you've ticked this box, viewers will be able to see the images hosted on your ODK Aggregate instance.

Congratulations! You have learnt how to use ODK Aggregate to manage your ODK submissions.

 

Discussion & Feedback

Have questions about this tutorial? Want to give us some feedback? Visit the Google Earth Outreach Discussion Group to discuss it with others.

What's Next?