Cleaning up resources
How to bulk remove resources and their relationships
Last updated
How to bulk remove resources and their relationships
Last updated
Commerce Layer lets you remove resources and their associated relationships in batches. To do that, you need to create a new resource and specify the resource_type you want to remove. Optionally, you can also apply some filters to narrow the affected data.
The process is asynchronous and you can poll the status attribute to check the clean-up progress. Once the operation has been completed you can check the number of removed items by inspecting the processed_count attribute. In case of validation errors, the errors_count and errors_log attributes are populated.
Since it is fired in the background as soon as the cleanup resource is created, the removal process is not reversible: all of the resources matching the filters criteria and their associated relationships will be removed in bulk. We suggest that you create a backup of the resources you're going to clean up exporting them first (making sure to use the same filters and include all of the affected relationships).
There is no limit on the total number of resources you can clean up, but the single batches are subject to some soft limits: the records_count must be a maximum of 10000 records, otherwise the cleanup will be rejected at the time of creation.
During the clean-up process, the errors count is also monitored: cleanups whose errors_count attribute value overflows the maximum percentage of 10% of the total records_count will be interrupted.
The maximum number of concurrent cleanups (i.e. cleanups whose status is pending or in_progess) allowed per organization is 10.
If you absolutely need to clean up any of the supported resources in one go, overriding the maximum cleanup size and concurrent cleanup API limits above, you can leverage the power of our CLI cleanups plugin (see example).
At moment, you can clean up the following resources and associated relationships (more to come):
Bundles
Gift cards
Prices > price tiers
Promotions > promotion rules
SKU lists > SKU list items, bundles
SKU options > line item options
SKUs > in-stock subscriptions, prices, SKU list items, stock items, tax categories
Stock items
Please find some examples of how to clean up them here below.
Please note that associated relationships will be automatically removed along with the parent resource. The clean-up process works in cascade (e.g. removing a promotion will also remove its associated promotion rules and all the related coupons).
In order to preserve data integrity the cleanups are subject to some relationship constraints. If such constraints are violated an error is added to the errors_log and the errors_count is incremented (resulting in an interruption if the 10% threshold is exceeded):
SKU lists — must not be linked to any promotion or promotion rules (i.e. any associated promotion must be removed first)
SKUs — must not be linked to any SKU list item within a bundle (i.e. any associated bundle must be removed first)
When cleaning up resources, you can fine-tune the data to be removed by applying some filters (both to the resources and their relationships) using the filters attribute:
To compose the filter predicate you just need to follow the same syntax you use when filtering a collection of resources — {{attributes}}_{{matcher}}. You must specify filtering rules as a valid JSON object. List values for the *_in matcher need to be expressed as arrays.
When specifying filters as a JSON attribute, the syntax of the *_jcont predicate changes. You must specify the value of the predicate as a plain string:
{ "metadata_jcont": \"{"key": "value"}\" }
The following request creates a cleanup to remove the SKUs matching the specified code, along with their relationships:
The following request creates a cleanup to remove all the promotions that expired after a specified date, along with their relationships:
The following command cleans up all the SKUs whose code contains a specific subset of characters:
You can inspect the status of a specific cleanup by fetching it by ID and looking at the status attribute.
When you create a cleanup, it tries immediately to start the removal process, entering the in_progress status. In case the async queue is saturated, the cleanup remains pending until it gets a chance to be processed.
The following request fetches a single cleanup, the one identified by the ID "WmjloIJzAS":
You can also leverage Commerce Layer real-time webhooks mechanism, listen to cleanups.create, cleanups.start, cleanups.complete, cleanups.interrupt, or cleanups.destroy and react properly.
When a cleanup is completed or interrupted, constraints errors are reported through the errors_count and errors_log attributes: