Cost Data Import - Migrating from Daily Uploads to Uploads

The document describes how to migrate from dailyUploads to Uploads for cost data import.

Introduction

The Daily Upload resource has been deprecated as of June 18, 2014. It will continue to work for a minimum of 6 months but we encourage you to migrate to the Upload resource as soon as possible.

Daily Upload vs Upload

The main differences when importing cost data using DailyUploads vs the new Upload resource are:

• The need to include a date as part of the uploaded cost data file.
• The ability to upload cost data for multiple days in a single CSV file.
• The ability to make modifications to data without any time limits.

What's Changing

Custom Data Source / Data Set Creation

During the deprecation period, two types of Custom Data Sources / Data Sets can be created. One that works with the deprecated DailyUpload resource and one that works with the Upload resource. The list below describes where in the web interface admin page, under the property section, you can create a Custom Data Source / Data Set for the resource you want to use.

• dailyUpload - Go to: Custom Definitions -> Custom Data Sources -> New Custom Data Source.
• Uploads - Go to: Data Import -> +New Data Set.

Auto Migration

At the end of the six month deprecation period, all Custom Data Sources / Data Sets that have not been migrated will be automatically migrated. It is important to note that after auto migration:

• You will only be able to upload cost data using the Upload resource.
• Attempts to upload to a Custom Data Source / Data Set using the dailyUpload resource will fail and return an error.

Migration Overview

The process to begin using the Upload resource for cost data with an existing Custom Data Source starts by migrating the custom data source from DailyUpload to Upload. The main steps for migration are:

1. Initiate Custom Data Source Migration.
   • This is a one time step required to migrate an existing custom data source so that you can begin using the Upload resource to import cost data. It is only possible to initiate the migration of a custom data source via the API.
2. Upload Cost Data Using Uploads.
   • Prepare cost data CSV file and include ga:date column (The rest of the cost data schema is unchanged).
   • Upload CSV file using uploadData method of Upload resource.

Before you Begin

Review the following documents:

• The Google Analytics Help Center Data Import article.
• The API Reference for the Uploads Resource
• The Data Import - Developer Guide.

Initiate Custom Data Source Migration

The steps required to migrate a Custom Data Source to start using the upload resource are:

1. Migrate the Custom Data Source
2. Verify the migration.

1. Migrate the data.

Execute the migrateDataImport method. See the reference documents for examples.

2. Verify the migration has completed

Verify that the migration is complete by making a Management Uploads List request. You should see a "COMPLETED" status for all uploaded items.

{
 "kind": "analytics#uploads",
 "totalResults": 1,
 "startIndex": 1,
 "itemsPerPage": 1000,
 "items": [
  {
   "id": "XXXXX...XXXX",
   "kind": "analytics#upload",
   "accountId": "XXXXXXX",
   "customDataSourceId": "XXXXX...XXXX",
   "status": "COMPLETED"
  }
 ]
}
          

Upload Cost Data Using Uploads

Once a custom data source has been migrated it can be used with the Upload resource. The steps are:

1. Prepare the CSV file.
2. Upload the CSV file.

1. Prepare the CSV file.

From this point on ensure that your data is correctly formatted with the ga:date column and upload new data using the Upload resource. Note that data for multiple dates can be uploaded in the same CSV file.

ga:medium,ga:source,ga:adClicks,ga:adcost
cpc,email,13,24.35
...
        

ga:date,ga:medium,ga:source,ga:adClicks,ga:adcost
20140210,cpc,email,13,24.35
20140211,cpc,email,11,22.55
...
        

2. Upload the CSV file.

When using the Upload resource you can upload directly through the web interface or you can execute the uploadData method.

media = MediaFileUpload('data_20140210.csv',
    mimetype='application/octet-stream',
    resumable=False)

analytics.management().dailyUploads().upload(
    accountId='XXXXXXXX',
    webPropertyId='UA-XXXXXXXX-1',
    customDataSourceId='XXXXXXXXX',
    date='2014-02-10',
    appendNumber=1,
    reset=False,
    type='cost',
    media_body=media).execute()
        

media = MediaFileUpload('data.csv',
    mimetype='application/octet-stream',
    resumable=False)

analytics.management.uploads.uploadData(
    accountId='XXXXXXXX',
    webPropertyId='UA-XXXXXXXX-1',
    customDataSourceId='XXXXXXXXXXXXX',
    media_body=media).execute()
        

Custom Data Sources

During the deprecation period, Custom Data Sources can have one of two upload types; The deprecated analytics#dailyUploads which requires that you use the dailyUpload resource to upload cost data and analytics#uploads which requires you to use the upload resource to upload cost data.

Before you upload cost data for a custom data source you should check which upload type is required to make sure you are using the correct upload resource and/or to identify whether a custom data source should be migrated.

Check the upload type of a Custom Data Source

To determine if a custom data source has been migrated execute the Custom Data Sources list method and check the value of the uploadType property for the custom data source. For example, the possible responses during the deprecation period are:

{
  "id": "XXXXXXXXXXXXXX",
  "kind": "analytics#customDataSource",
  "name": "My first data set",
  ...
  "uploadType": "analytics#dailyUploads"
  ...
}
      

{
  "id": "XXXXXXXXXXXXXX",
  "kind": "analytics#customDataSource",
  "name": "My second data set",
  ...
  "uploadType": "analytics#uploads"
  ...
}
      

Error Messages

Through the course of migrating to the new resource you may come across the following error messages if you call the wrong resource.

analytics.management().uploads().uploadData(...)
...
400 : Cost data upload is not supported.
        

analytics.management().dailyUploads().upload(...)
...
400 : Please use the new upload API for the newer version data source
        

Frequently Asked Questions

What happens to your upload history after you migrate?

Since the cost data schema when using the Upload resource is identical to the schema of using the Daily Upload resource, except for the addition of the ga:date column, your data will remain fully intact. There is a difference in how you make modifications for previously uploaded cost data. If you appended cost data multiple times using the Daily Upload resource and then migrated, you will not be able to selectively delete a single legacy dailyUpload but will have to delete the whole day.

Below is an example of a Custom Data Source that had multiple changes for the same date. After Migration, these changes are now represented as a single Upload.

{
  "kind": "analytics#dailyUploads",
  ...
  "items": [
  {
    "kind": "analytics#dailyUpload",
    ...
    "accountId": "XXXXXX",
    "webPropertyId": "UA-XXXXXX-1",
    "customDataSourceId": "XXXX...XXXX",
    "date": "2012-09-12",
    "appendCount": 2,
    "createdTime": "2014-02-28T18:43:16.324Z",
    "modifiedTime": "2014-02-28T19:10:35.259Z",
    "parentLink": {
       "type": "analytics#customDataSource",
       "recentChanges": [
         {
           "change": "APPEND",
           "time": "2014-02-28T19:10:34.108Z"
         },
         {
           "change": "APPEND",
           "time": "2014-02-28T18:43:16.318Z"
         }
       ]
    }
  ]
}
        

{
  "kind": "analytics#uploads",
  "totalResults": 1,
  "startIndex": 1,
  "itemsPerPage": 1000,
  "items": [
    {
      "id": "XXXX….XXXX",
      "kind": "analytics#upload",
      "accountId": "XXXXXX",
      "customDataSourceId": "XXXX….XXXX",
      "status": "COMPLETED"
    }
  ]
}
        

What is the difference between Custom Data Sources and Data Sets?

Most of the developer documentation refer to Custom Data Sources and Data Sets interchangeably. However in the Google Analytics web interface the term Custom Data Source is only used to refer to the deprecated dailyUploads custom data sources used only for cost data. The term Data Set is used in the Google Analytics web interface to refer to any data import custom data source including the new uploads custom data sources.