Collections

Collections provide a way to store data in your workspace that you can use in automations with Liquid.

What’s a Collection?

A collection is a new type of data in Customer.io that’s separate from people or events in your account. They represent any set of “things” that exist in your business. For instance, a Collection could be a list of upcoming events, promotions/coupons, or available classes.

You should store this data in Customer.io when it’s helpful to be able to sort through the Collection to identify a specific set of data that you want to include in a message, tailored to the data you know about each person. Let’s take the example of an online learning service. By storing all the courses offered in a Collection, you could retrieve the specific set of classes that are relevant for each individual person and list them in an automation email.

Collections are flexible. They’re simply lists of JSON objects that don’t require specific schemas or indexes. Your collections need to stay within the collection limits.

Collection limits

You can create as many collections as you need in a workspace. However, your collections must stay within the following limits:

• Collection size: A collection can be up to 10 MB.
• Field length: Each top-level field in a row can hold up to 10,000 characters. For a string, Customer.io counts the characters in the text. For any other value, like a nested object or an array, Customer.io counts the characters in its JSON.
• Account storage: The total size of all collections in your account can’t exceed 5 GB. If creating or updating a collection would take your account over that limit, Customer.io rejects the change. You can prune or delete collections to save space.

Create a Collection

You can upload your collection as JSON, a CSV file, or by sharing a Google Sheet with us. If you upload a CSV file or share a Google Sheet, we expect your file to be a simple table with headers—each column is an attribute of your collection items. To share a Google Sheet with us, you must grant Customer.io access to specific files in your Google Drive.

For example, if your collection contains a list of courses, your CSV might look like this:

name,level,description,duration,instructor
Introduction to Python,beginner,Learn the basics of Python programming,4 weeks,Jane Smith
Data Analysis with R,intermediate,Analyze datasets using R,6 weeks,John Doe
Advanced Machine Learning,advanced,Deep dive into ML algorithms,8 weeks,Alice Johnson

Each row represents an item in your collection, and each column header becomes an attribute that you can reference in your messages.

Your file needs to fit within the collection limits.

To add a collection:

1. Under Configure data, click More > Collections and click Add collection.
2. Give your collection a name and description.
3. Select your collection file: JSON, CSV, or Google Sheet.
4. Click Add collection.

Use Collection data in your messages

Now that you’ve added a Collection, the next step is to use them in your automations. The power of Collections is that you can query the Collection as part of an automation workflow, meaning that you can retrieve only the Collection items relevant to that particular person and automation.

Add a Query Collection action

When a person reaches this action, the automation retrieves collection data and stores info based on your parameter. Then you can use this data in subsequent messages.

Build a query to retrieve relevant info

Select the query collection action, give it a name, and click Add Query to get started.

Let’s build a query that retrieves course recommendations based on a person’s skill level.

Choose a collection to query

1. Select the Collection you want to query.
2. Click Add query condition to filter for the courses you want to send a message about.

For example, you could filter for all courses where the collection attribute level is equal to the person’s attribute skill_level.

Sort the results

Under 2. Filter collection query result, pick an attribute in Sort results by. A new step sorts in descending order. To switch between ascending and descending order, click the sort direction button next to the attribute. Sorting decides which rows a query returns when you set a limit or choose Limit to one result.

If you choose Do not sort, leave Randomize results off, and set a limit or choose Limit to one result, the query can return any rows that match, not necessarily the first ones in your collection. The query preview follows the same rule. If it matters which rows the query returns, sort the results.

In ascending order, Customer.io sorts values by type, and then by value within each type:

1. Numbers, by value. 9 comes before 10, even if your CSV stores them as text.
2. Version numbers, like 1.2.0, in version order.
3. Text, alphabetically and ignoring case.

Descending order reverses that, so text comes first and numbers come last. Rows with a missing or empty value for the attribute always come last, in either direction. Rows with equal values keep the order they have in your collection.

If an attribute mixes numbers and words, like 12, n/a, and 7, ascending order puts every number before any word. To get the words first, sort descending or clean up the attribute’s values.

Decide how to store the results

Under Store the query results, decide whether you want to use this information outside of the automation or not:

• Choose Customer attribute to store data on the person, accessible outside of the automation.
• Choose Journey attribute to store data temporarily during the journey; this data expires when the person exits the automation.

Keep the following limits in mind if you’re setting journey attributes:

• Overall limit: 100 journey attributes per journey. You can create or change up to 100 journey attributes during a single journey. Updates beyond this limit fail, and the person moves forward without the update.
• Attribute name limit: 128 bytes
• Attribute value limit: 100 KiB

Then choose or create an attribute, like course_recommendations. We store data on this attribute as JSON.

Preview the results

Choose a person from the Sample data panel to preview the query results. Go to the Preview tab to see what the Collection rows would return for the sample user.

Reference collection data with liquid

Now that you’ve set up your query, you can use the collection data in a message!

Reference profile attributes

For collection data stored as Customer Attributes, reference them using the customer object:

{% for item in customer.course_recommendations %}
  {{ item.name }}
{% endfor %}

If the data is an array, you can use liquid filters, such as first, last, and map, to get specific items or attributes.

{% assign first_course = customer.course_recommendations | first %}

If you limited your query to a single result, then the attribute value is an object, not an array. You can instead directly access the name and course description:

{{ customer.promo_course.name }}
{{ customer.promo_course.description }}

Reference journey attributes

For collection data stored as Journey Attributes, reference them using the journey object:

{% for item in journey.course_recommendations %}
  {{ item.name }}
{% endfor %}

If the data is an array, you can use liquid filters, such as first, last, and map, to get specific items or attributes.

{% assign first_course = journey.course_recommendations | first %}

If you limited your query to a single result, then the attribute value is an object, not an array. You can instead directly access the name and course description:

{{ journey.promo_course.name }}
{{ journey.promo_course.description }}

Collections API

The endpoints for Collections are available via our App API:

• US region: https://api.customer.io/v1/
• EU region: https://api-eu.customer.io/v1/

Creating a new Collection:

POST:/v1/api/collections

Create a new collection by providing both a name and either an array of JSON objects as data, or a url to download CSV or JSON data from.

{
  "name": "collection name",
  "data": [ { row }, { row } ],
}

If you provide a URL, the data can either be JSON (array or newline delimited) or a CSV file. Ensure that you provide a Content-Type definition so that we know how to parse the data. If you don’t provide a Content-Type, we use the file extension. If you don’t include a content type or file extension, we assume the URL contains JSON.

You cannot share a Google Sheet with us using this API.

Once created, a Collection has the following JSON definition:

"bytes": bytes_of_data_in_collection,
"created_at": 1589471110,
"updated_at": 1589471132
"id": 1,
"name": collection_name
"rows": number_of_rows_in_collection,
"schema": ["name1", "name2", "name3" ],

The schema will have a list of all attributes used by any items in the Collection, but remember that we don’t enforce that all items have this set of attributes.

Retrieve info about a Collection you’ve already created

GET:/v1/api/collections/:id

Get the details of a collection with the given id. This does not include the content.

Retrieve the content of a Collection you’ve already created

GET:/v1/api/collections/:id/content

Returns the entire content of the collection with the given id.

Retrieve a list of all your Collections

GET:/v1/api/collections

Returns the details (but not content) for all of your Collections:

{"collections": ["collection1", "collection2"]}

Update the content of a Collection

PUT:/v1/api/collections/:id/content

Provide the content of a Collection as JSON-encoded text or CSV-encoded data. This will replace all the content for the Collection. For example:

[
  {
    "name":"Steak",
    "price":20,
    "quantity":10,
    "can_deliver":false
  }
]

Update the name and content of a Collection

PUT:/v1/api/collections/:id

Update an existing collection. If name or data or url aren’t provided, the associated data is not updated. This will replace all content for the Collection.

{
"name": "collection name",
"data": [
  {"name":"Hot Dog","price":2,"quantity":1,"can_deliver":true},
  {"name":"Pizza","price":10,"quantity":2,"can_deliver":true},
  {"name":"Steak","price":20,"quantity":10,"can_deliver":false} ]
}
OR
{
  "name": "collection name",
  "url": "https://example.com/sheet.csv"
}

Delete a Collection

DELETE:/v1/api/collections/:id

Deletes the collection with the given id.

Change rows by key

You can add, update, or delete individual rows without uploading the whole collection. First, give the collection a key field: the attribute whose value identifies each row, like sku or id.

• Set key_field when you create the collection, or later with update a collection.
• Every row needs a unique value in the key field. The value must be a non-empty string or number of up to 1,024 bytes.

To change rows, send them to POST:/v1/collections/:id/writes with a mode:

{
  "mode": "upsert",
  "data": [
    { "sku": "A-100", "price": 19 },
    { "sku": "A-101", "price": 24 }
  ]
}

A keyed write can change up to 500 rows, and Customer.io applies it all or nothing. If any row in the request is invalid, Customer.io rejects the whole write and tells you which rows failed.

Customer.io applies writes to a collection one at a time, in the order it receives them. The writes endpoint responds right away with a write id, so large writes don’t time out. Use GET:/v1/collections/:id/writes/:write_id to check the state of the write:

• queued: the write is waiting to run.
• applied: Customer.io made the change.
• replaced: a newer replace write made this one unnecessary, so Customer.io skipped it.
• failed: Customer.io didn’t make the change. The response includes a code and, for keyed writes, the rows that failed.

Customer.io keeps the result of each write for 24 hours. The other update endpoints still wait for the result before they respond.

To read rows by key, use GET:/v1/collections/:id/content?keys=A-100,A-101. You can read up to 1,000 keys at a time. To page through a collection, use GET:/v1/collections/:id/content?limit=1000 and pass the next_cursor from each response as cursor in the next request. See the API reference for every field and error code.