Cleaning up resources

How to bulk remove resources and their relationships
Commerce Layer lets you remove resources and their associated relationships in batches. To do that, you need to create a new cleanup resource and specify the resource_type you want to remove. Optionally, you can also apply some filters to narrow the affected data.
The process is asynchronous and you can poll the status attribute to check the clean-up progress. Once the operation has been completed you can check the number of removed items by inspecting the processed_count attribute. In case of validation errors, the errors_count and errors_log attributes are populated.
Since it is fired in the background as soon as the cleanup resource is created, the removal process is not reversible: all of the resources matching the filters criteria and their associated relationships will be removed in bulk. We suggest that you create a backup of the resources you're going to clean up exporting them first (making sure to use the same filters and include all of the affected relationships).
Cleanup limits
Maximum cleanup size
Cleanups are subject to some soft limits: the records_count must be a maximum of 10000 records, otherwise the cleanup will be rejected at the time of creation.
Maximum error percentage
During the clean-up process, the errors count is also monitored: cleanups whose errors_count attribute value overflows the maximum percentage of 10% of the total records_count will be interrupted.
Concurrent cleanups
The maximum number of concurrent cleanups (i.e. cleanups whose status is pending or in_progess) allowed per organization is 10.
Supported resources
At moment, you can clean up the following resources and associated relationships (more to come):
Bundles
Gift cards
Prices > price tiers
Promotions > promotion rules
SKU lists > SKU list items, bundles
SKU options > line item options
SKUs > in-stock subscriptions, prices, SKU list items, stock items, tax categories
Stock items
Please find some examples of how to clean up them here below.
Please note that associated relationships will be automatically removed along with the parent resource. The clean-up process works in cascade (e.g. removing a promotion will also remove its associated promotion rules and all the related coupons).
Constraints
In order to preserve data integrity the cleanups are subject to some relationship constraints. If such constraints are violated an error is added to the errors_log and the errors_count is incremented (resulting in an interruption if the 10% threshold is exceeded):
SKU lists — must not be linked to any promotion or promotion rules (i.e. any associated promotion must be removed first)
SKUs — must not be linked to any SKU list item within a bundle (i.e. any associated bundle must be removed first)
Filtering the data to be removed
When cleaning up resources, you can fine-tune the data to be removed by applying some filters (both to the resources and their relationships) using the filters attribute:
...
  "filters": {
    "{{predicate}}": {{value}},
    ...
  }
To compose the filter predicate you just need to follow the same syntax you use when filtering a collection of resources — {{attributes}}_{{matcher}}. You must specify filtering rules as a valid JSON object. List values for the *_in matcher need to be expressed as arrays.
Filtering data
Examples
Cleaning up  a batch of SKUs
Request
Response
The following request creates a cleanup to remove the SKUs matching the specified code, along with their relationships:
curl -g -X POST \
  'https://yourdomain.commercelayer.io/api/cleanups' \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Bearer your-access-token' \
  -H 'Content-Type: application/vnd.api+json' \
  -d '{
	"data": {
	  "type": "cleanups",
	  "attributes": {
	    "resource_type": "skus",
            "filters": {
	      "code_start": "SHIRT"
	    }
	  }
	}
      }'
On success, the API responds with a 201 Created status code, returning the created cleanup object:
{
  "data": {
    "id": "WmjloIJzAS",
    "type": "cleanups",
    "links": {...},
    "attributes": {
      "resource_type": "skus",
      "status": "pending",
      "started_at": null,
      "completed_at": null,
      "interrupted_at": null,
      "filters": {"code_start": "SHIRT"},
      "records_count": 100,
      "errors_count": 0,
      "processed_count": 0,
      "errors_log": {...},
      "created_at": "2018-01-01T12:00:00.000Z",
      "updated_at": "2018-01-01T12:00:00.000Z",
      "reference": null,
      "reference_origin": null,
      "metadata": {}
    },
    "relationships": {
      "events": { ... }
    },
    "meta": {...}
  }
}
Cleaning up a batch of promotions
Request
Response
The following request creates a cleanup to remove all the promotions that expired after a specified date, along with their relationships:
curl -g -X POST \
  'https://yourdomain.commercelayer.io/api/cleanups' \
  -H 'Accept: application/vnd.api+json' \
  -H 'Authorization: Bearer your-access-token' \
  -H 'Content-Type: application/vnd.api+json' \
  -d '{
	"data": {
	  "type": "cleanups",
	  "attributes": {
	    "resource_type": "promotions",
	    "filters": {
	      "expired_at_gteq": "2018-01-01T12:00:00.000Z"
	    }
	  }
	}
      }'
On success, the API responds with a 201 Created status code, returning the created import object:
{
  "data": {
    "id": "HGfdRsErUk",
    "type": "cleanups",
    "links": {...},
    "attributes": {
      "resource_type": "promotions",
      "status": "pending",
      "started_at": null,
      "completed_at": null,
      "interrupted_at": null,
      "filters": {"expired_at_gteq": "2018-01-01T12:00:00.000Z"},
      "records_count": 55,
      "errors_count": 0,
      "processed_count": 0,
      "errors_log": {...},
      "created_at": "2018-01-01T12:00:00.000Z",
      "updated_at": "2018-01-01T12:00:00.000Z",
      "reference": null,
      "reference_origin": null,
      "metadata": {}
    },
    "relationships": {
      "events": { ... }
    },
    "meta": {...}
  }
}
Checking the cleanup status
You can inspect the status of a specific cleanup by fetching it by ID and looking at the status attribute.
When you create a cleanup, it tries immediately to start the removal process, entering the in_progress status. In case the async queue is saturated, the cleanup remains pending  until it gets a chance to be processed.
Request
Response
The following request fetches a single cleanup, the one identified by the ID "WmjloIJzAS":
curl -g -X GET \ 
  'https://yourdomain.commercelayer.io/api/cleanups/WmjloIJzAS' \
  -H 'Content-Type: application/vnd.api+json' \
  -H 'Authorization: Bearer your-access-token'
On success, the API responds with a 200 OK status code, returning the single cleanup object:
{
  "data": {
    "id": "WmjloIJzAS",
    "type": "cleanups",
    "links": {...},
    "attributes": {
      "resource_type": "skus",
      "status": "completed",
      "started_at": "2018-01-01T12:00:00.000Z",
      "completed_at": "2018-01-01T12:01:00.000Z",
      "interrupted_at": null,
      "filters": {},
      "records_count": 100,
      "errors_count": 1,
      "processed_count": 99,
      "errors_log": {...},
      "created_at": "2018-01-01T12:00:00.000Z",
      "updated_at": "2018-01-01T12:00:00.000Z",
      "reference": null,
      "reference_origin": null,
      "metadata": {}
    },
    "relationships": {
      "events": { ... }
    },
    "meta": {...}
  }
}
You can also leverage Commerce Layer real-time webhooks mechanism, listen to cleanups.create, cleanups.start, cleanups.complete, cleanups.interrupt, or cleanups.destroy and react properly.
Real-time webhooks
Handling cleanup errors
When a cleanup is completed or interrupted, constraints errors are reported through the errors_count and errors_log attributes:
...
​
"errors_count": 5,
"processed_count": 95,
"errors_log": {
  "HjKhdjGfDD": "Cannot delete record because of dependent bundles",
  ...
},
​
...
Exporting resources
External resources
Last modified 6mo ago