Comment on page
Exporting resources
How to bulk export resources and their relationships
Commerce Layer lets you export resources in JSON (default) or CSV formats. To do that, you need to create a new export resource, specify the
resource_type
you want to export, and the format
of the exported file. Optionally, you can also specify the relationships you want to include, apply some filters to narrow the exported data, and decide to skip some redundant attributes.To export a CSV list you need to set the
format
attribute to csv
. Otherwise, if the format is not specified, your data will be exported using the default format (JSON).The process is asynchronous and you can poll the
status
attribute to check the export progress. As soon as an export is created you can check the number of items that are going to be exported by inspecting the records_count
attribute. If you try to export a resource with records_count = 0
, an error is returned.Resource relationships are exported by default with their IDs, appending the
_id
suffix to the name of the related resource (e.g. "price_list_id": "xYZkjABcde"
). Some relationships can also be exported as a nested object if specified as valid included associations.External storage service URLs expire in 5 minutes. You need to uncompress (
gunzip
) the file in order to read the data back. If the exported data URL is expired you can just fetch the completed export to get a new working one.There is no limit on the total number of resources you can export, but the single batches are subject to some soft limits: the
records_count
must be a maximum of 10000 records, otherwise the export will be rejected at the time of creation.The maximum number of concurrent exports (i.e. exports whose status is
pending
or in_progess
) allowed per organization is 10.If you absolutely need export any of the supported resources in one go, overriding the API limits above, you can leverage the power of our CLI export plugin (see example). Alternatively, you can also perform a search query using our Metrics API (at the moment limited to orders, carts, and returns).
At moment, exports are available for the following resources (more to come):
- Addresses
- Bundles
- Coupons
- Customer addresses
- Customer subscriptions
- Customers
- Gift cards
- Line items
- Line item options
- Orders
- Payment methods
- Price tiers
- Prices
- Shipments
- Shipping categories
- Shipping methods
- SKU list items
- SKU lists
- SKU options
- SKUs
- Stock items
- Tags
- Tax categories
- Transactions (Authorizations, Captures, Voids, Refunds)
With very few exceptions (e.g. shipments, transactions, customer payment sources etc.), exported outputs (both in JSON or CSV format) can be used as
inputs
for the imports API. If you're using the export + import features to duplicate your dataset records (e.g. to migrate your test data to your live environment) you may need to skip some redundant attributes. Specifically, if you want to export resources from an organization and import them into a different one, make sure to set the export's dry_data
attribute to true
in order to avoid ID conflicts.It's possible to include one or more relationships of the exported resource using the
includes
attribute. Relationships will be exported as an object if singular (has_one
or belongs_to
), or as an array of objects if multiple (has_many
) — check Commerce Layer API data model for more information on how resources relate to each other.If you use the CSV format to export your data the output will have the relationship attributes flattened together with the parent resource ones. As a result, you'll find the resource attributes repeated on multiple lines, with relationship ones appended at the end. When exporting tags together with a tagged resource the cells belonging to the
tags.id
and tags.name
columns contain a string with the list of IDs/names, comma separated. In general, using CSV is fine for simple exports (resources with a few attributes and no relationship included), otherwise we strongly recommend using JSON.When including associations, multiple levels of relationships are supported. You just need to append the more specific using the "dot"
.
notation (e.g. line item options for orders, or price tiers for SKUs).CSV
format
supports only one level of relationship (e.g. if you try to include line_items.line_item_options
when exporting orders, only the line items exported).Only specific
includes
are allowed for each resource:- Addresses —
tags
- Bundles —
sku_list
,sku_list_items
,tags
- Customer addresses —
customer
,address
- Customer subscriptions —
customer
- Customers —
customer_subscriptions
,tags
- Coupons —
tags
- Gift cards —
tags
- Line items —
tags
- Line item options —
tags
- Orders —
customer
,shipping_address
,billing_address
,payment_method
,line_items.line_item_options
,shipments.shipping_method
,authorizations
,captures
,voids
,refunds
,transactions
,tags
authorizations, captures, voids, and refunds
can also be associated with the exported orders singularly (see an example for refunds) or as a unique list of transactions.
- Payment methods —
order
- Prices —
sku
,price_tiers
- Shipments —
order
,shipping_category
,shipping_method
- Shipping categories —
skus
- Shipping methods —
shipments
- SKUs —
shipping_category
,prices.price_tiers
,stock_items
,tags
,tax_categories
- SKU list —
sku_list_items
,bundles
- SKU list items —
sku
- SKU options —
tags
- Stock items —
sku
- Tax categories —
sku
- Transactions —
order
When exporting resources, you can fine-tune the data to be exported by applying some filters (both to the resources and their relationships) using the
filters
attribute:...
"filters": {
"{{predicate}}": {{value}},
...
}
To compose the filter predicate you just need to follow the same syntax you use when filtering a collection of resources —
{{attributes}}_{{matcher}}
. You must specify filtering rules as a valid JSON object. List values for the *_in
matcher need to be expressed as arrays (as in this example).You might want to compact exported data by removing some redundant attributes from the final JSON or CSV output. To do that, set the
dry_data
specific boolean attribute to true
— the following attributes and values will be skipped:- the main resource IDs.
- the timestamp attributes (
created_at
andupdated_at
). - any empty or
null
attribute value (working when exporting in the JSON format only). - all the formatted amounts other than cents (e.g. for prices, orders, etc.).
Request
Response
The following request creates an export of a list of addresses in
JSON
format, skipping the redundant attributes:curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "addresses",
"filters": {
"country_code_eq": "IT",
"city_in": ["Rome", "Milan", "Prato"]
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "addresses",
"format": "json",
"status": "pending",
"includes": [],
"filters": {"country_code_eq": "IT", "city_in": ["Rome", "Milan", "Prato"]},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 8765,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of bundles and their associated SKU lists and SKU list items in CSV format, filtered by SKU list name:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "bundles",
"format": "csv",
"includes": ["sku_list", "sku_list_items"],
"filters": {
"sku_list_name_in": ["six pack", "12 pack"]
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "bundles",
"format": "csv",
"status": "in_progress",
"includes": ["sku_list", "sku_list_items"],
"filters": {"sku_list_name_in": ["six pack", "12 pack"]},
"dry_data": null,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 43,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of coupons in JSON format, skipping the redundant attributes:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "coupons",
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "coupons",
"format": "json",
"status": "in_progress",
"includes": [],
"filters": {},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 9876,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of customer addresses and their associated addresses and customers in JSON format (skipping the redundant attributes), filtered by customer's email:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "customer_addresses",
"format": "json",
"includes": ["address", "customer"],
"filters": {
"customer_email_end": ".com"
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "customer_addresses",
"format": "json",
"status": "in_progress",
"includes": ["address", "customer"],
"filters": {"customer_email_end": ".com"},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 254,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of customer subscriptions and their associated customers in CSV format, filtered by email:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "customer_subscriptions",
"format": "csv",
"includes": ["customer"],
"filters": {
"customer_email_matches": "%gmail.com"
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "customer_subscriptions",
"format": "csv",
"status": "in_progress",
"includes": ["customer"],
"filters": {"customer_email_matches": "%gmail.com"},
"dry_data": null,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 345,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of customers in JSON format (skipping the redundant attributes), filtered by email:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "customers",
"format": "json",
"includes": ["customer_subscriptions"],
"filters": {
"email_end": ".com"
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "customers",
"format": "json",
"status": "in_progress",
"includes": ["customer_subscriptions"],
"filters": {"email_end": ".com"},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 254,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of gift cards in CSV format, filtered by the date and time at which they were created:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "gift_cards",
"format": "csv",
"filters": {
"created_at_gteq": "2018-01-01T12:00:00.000Z"
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "gift_cards",
"format": "csv",
"status": "in_progress",
"includes": [],
"filters": {"created_at_gteq": "2018-01-01T12:00:00.000Z"},
"dry_data": null,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 349,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of orders dry data with the associated customer, addresses, line items, line item options, payment method, and refunds in JSON format, filtered by status, country code, and the date and time at which they were placed:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "orders",
"includes": [
"customer",
"shipping_address",
"billing_address",
"line_items.line_item_options",
"payment_method",
"refunds"
],
"filters": {
"status_in": ["placed", "approved"],
"placed_at_gteq": "2018-01-01T12:00:00.000Z",
"country_code_eq": "IT"
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "orders",
"format": "json",
"status": "in_progress",
"includes": ["customer", "shipping_address", "billing_address", "line_items.line_item_options", "payment_method", "refunds"],
"filters": {"status_in": ["placed", "approved"], "placed_at_gteq": "2018-01-01T12:00:00.000Z", "country_code_eq": "IT"},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 6499,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of prices with associated price tiers in CSV format (skipping the redundant attributes), filtered by the associated price list's currency code:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "prices",
"format": "csv",
"includes": ["price_tiers"],
"filters": {
"price_list_currency_code_eq": "USD"
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "prices",
"format": "csv",
"status": "in_progress",
"includes": ["price_tiers"],
"filters": {"price_list_currency_code_eq": "USD"},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 3456,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of SKU lists with associated SKU list items in JSON format, filtered SKU list type (manual):
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "sku_lists",
"includes": ["sku_list_items"],
"filters": {
"manual_true": true
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "sku_lists",
"format": "json",
"status": "in_progress",
"includes": ["sku_list_items"],
"filters": {"manual_true": true},
"dry_data": null,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 3456,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export with a list of SKUs, with the associated prices, price tiers, stock items, and tax categories in JSON format (skipping the redundant attributes), filtered by SKU name:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "skus",
"includes": ["prices.price_tiers", "stock_items", "tax_categories"],
"filters": {
"name_cont": "FW"
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "skus",
"format": "json",
"status": "in_progress",
"includes": [ "prices.price_tiers", "stock_items", "tax_categories" ],
"filters": { "name_cont": "FW" },
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 2345,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of a list of stock items and the associated SKUs in CSV format (skipping the redundant attributes), filtered by stock item quantity:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "stock_items",
"format": "csv",
"includes": ["sku"],
"filters": {
"quantity_gteq": 1
},
"dry_data": true
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "PmjlkIJzRA",
"type": "exports",
"links": {...},
"attributes": {
"resource_type": "stock_items",
"format": "csv",
"status": "in_progress",
"includes": ["sku"],
"filters": {"quantity_gteq": 1},
"dry_data": true,
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"records_count": 4567,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"meta": {...}
}
}
Request
Response
The following request creates an export of all the tags you created for your organization in CSV format:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "exports",
"attributes": {
"resource_type": "tags",
"format": "csv"
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created export object:{
"data": {
"id": "WoDOghDQlz",
"type": "exports",
"links": { ... },
"attributes": {
"resource_type": "tags",
"format": "csv",
"status": "in_progress",
"includes": [],
"filters": {},
"dry_data": null,
"started_at": "2018-01-01T12:00:00.000Z",
"completed_at": null,
"interrupted_at": null,
"records_count": 123,
"attachment_url": null,
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": { ... },
"meta": { ... }
}
}
Request
Response
The following request creates an export of all the SKU associated with the tag identified by the "geJmexflJQ" ID or by the "XEqZPxfPam" ID in JSON format:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/exports' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \