Cleaning up resources
How to bulk remove resources and their relationships
The process is asynchronous and you can poll the
status attribute to check the clean-up progress. Once the operation has been completed you can check the number of removed items by inspecting the processed_count attribute. In case of validation errors, the errors_count and errors_log attributes are populated.Since it is fired in the background as soon as the cleanup resource is created, the removal process is not reversible: all of the resources matching the filters criteria and their associated relationships will be removed in bulk. We suggest that you create a backup of the resources you're going to clean up exporting them first (making sure to use the same filters and include all of the affected relationships).
There is no limit on the total number of resources you can clean up, but the single batches are subject to some soft limits: the
records_count must be a maximum of 10000 records, otherwise the cleanup will be rejected at the time of creation.During the clean-up process, the errors count is also monitored: cleanups whose
errors_count attribute value overflows the maximum percentage of 10% of the total records_count will be interrupted.The maximum number of concurrent cleanups (i.e. cleanups whose status is
pending or in_progess) allowed per organization is 10.If you absolutely need to clean up any of the supported resources in one go, overriding the maximum cleanup size and concurrent cleanup API limits above, you can leverage the power of our CLI cleanups plugin (see example).
At moment, you can clean up the following resources and associated relationships (more to come):
- Bundles
- Gift cards
- Prices > price tiers
- Promotions > promotion rules
- SKU lists > SKU list items, bundles
- SKU options > line item options
- SKUs > in-stock subscriptions, prices, SKU list items, stock items, tax categories
- Stock items
Please note that associated relationships will be automatically removed along with the parent resource. The clean-up process works in cascade (e.g. removing a promotion will also remove its associated promotion rules and all the related coupons).
In order to preserve data integrity the cleanups are subject to some relationship constraints. If such constraints are violated an error is added to the
errors_log and the errors_count is incremented (resulting in an interruption if the 10% threshold is exceeded):- SKU lists — must not be linked to any promotion or promotion rules (i.e. any associated promotion must be removed first)
- SKUs — must not be linked to any SKU list item within a bundle (i.e. any associated bundle must be removed first)
When cleaning up resources, you can fine-tune the data to be removed by applying some filters (both to the resources and their relationships) using the
filters attribute:...
"filters": {
"{{predicate}}": {{value}},
...
}
To compose the filter predicate you just need to follow the same syntax you use when filtering a collection of resources —
{{attributes}}_{{matcher}}. You must specify filtering rules as a valid JSON object. List values for the *_in matcher need to be expressed as arrays.Request
Response
The following request creates a cleanup to remove the SKUs matching the specified code, along with their relationships:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/cleanups' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "cleanups",
"attributes": {
"resource_type": "skus",
"filters": {
"code_start": "SHIRT"
}
}
}
}'
On success, the API responds with a
201 Created status code, returning the created cleanup object:{
"data": {
"id": "WmjloIJzAS",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "skus",
"status": "pending",
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"filters": {"code_start": "SHIRT"},
"records_count": 100,
"errors_count": 0,
"processed_count": 0,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
Request
Response
The following request creates a cleanup to remove all the promotions that expired after a specified date, along with their relationships:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/cleanups' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "cleanups",
"attributes": {
"resource_type": "promotions",
"filters": {
"expired_at_gteq": "2018-01-01T12:00:00.000Z"
}
}
}
}'
On success, the API responds with a
201 Created status code, returning the created import object:{
"data": {
"id": "HGfdRsErUk",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "promotions",
"status": "pending",
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"filters": {"expired_at_gteq": "2018-01-01T12:00:00.000Z"},
"records_count": 55,
"errors_count": 0,
"processed_count": 0,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
CLI command
Success message
The following command cleans up all the SKUs whose code contains a specific subset of characters:
commercelayer cleanups:create -t skus -w code_cont=FLASHSALE
On success, the CLI prompts this message on your terminal, separating the single cleanups and showing some basic information about the completed processes:
| ID | Cleanup progress | % | Status | TBP↓ | Prc. | Err. |
---------------------------------------------------------------------------------------
| DRlYbHDopv | █████████████████████████ | 100% | completed | 0 | 10000 | 0 |
| MReZgHXLpw | █████████████████████████ | 100% | completed | 0 | 5000 | 0 |
Cleanup of 15000 skus completed.
You can inspect the status of a specific cleanup by fetching it by ID and looking at the
status attribute.When you create a cleanup, it tries immediately to start the removal process, entering the
in_progress status. In case the async queue is saturated, the cleanup remains pending until it gets a chance to be processed.Request
Response
The following request fetches a single cleanup, the one identified by the ID "WmjloIJzAS":
curl -g -X GET \
'https://yourdomain.commercelayer.io/api/cleanups/WmjloIJzAS' \
-H 'Content-Type: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token'
On success, the API responds with a
200 OK status code, returning the single cleanup object:{
"data": {
"id": "WmjloIJzAS",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "skus",
"status": "completed",
"started_at": "2018-01-01T12:00:00.000Z",
"completed_at": "2018-01-01T12:01:00.000Z",
"interrupted_at": null,
"filters": {},
"records_count": 100,
"errors_count": 1,
"processed_count": 99,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
You can also leverage Commerce Layer real-time webhooks mechanism, listen to
cleanups.create, cleanups.start, cleanups.complete, cleanups.interrupt, or cleanups.destroy and react properly.When a cleanup is completed or interrupted, constraints errors are reported through the
errors_count and errors_log attributes:...
"errors_count": 5,
"processed_count": 95,
"errors_log": {
"HjKhdjGfDD": "Cannot delete record because of dependent bundles",
...
},
...
Last modified 2mo ago