Cleaning up resources
How to bulk remove resources and their relationships
The process is asynchronous and you can poll the
status
attribute to check the clean-up progress. Once the operation has been completed you can check the number of removed items by inspecting the processed_count
attribute. In case of validation errors, the errors_count
and errors_log
attributes are populated.Since it is fired in the background as soon as the cleanup resource is created, the removal process is not reversible: all of the resources matching the filters criteria and their associated relationships will be removed in bulk. We suggest that you create a backup of the resources you're going to clean up exporting them first (making sure to use the same filters and include all of the affected relationships).
There is no limit on the total number of resources you can clean up, but the single batches are subject to some soft limits: the
records_count
must be a maximum of 10000 records, otherwise the cleanup will be rejected at the time of creation.During the clean-up process, the errors count is also monitored: cleanups whose
errors_count
attribute value overflows the maximum percentage of 10% of the total records_count
will be interrupted.The maximum number of concurrent cleanups (i.e. cleanups whose status is
pending
or in_progess
) allowed per organization is 10.If you absolutely need to clean up any of the supported resources in one go, overriding the maximum cleanup size and concurrent cleanup API limits above, you can leverage the power of our CLI cleanups plugin (see example).
At moment, you can clean up the following resources and associated relationships (more to come):
- Bundles
- Gift cards
- Prices > price tiers
- Promotions > promotion rules
- SKU lists > SKU list items, bundles
- SKU options > line item options
- SKUs > in-stock subscriptions, prices, SKU list items, stock items, tax categories
- Stock items
Please note that associated relationships will be automatically removed along with the parent resource. The clean-up process works in cascade (e.g. removing a promotion will also remove its associated promotion rules and all the related coupons).
In order to preserve data integrity the cleanups are subject to some relationship constraints. If such constraints are violated an error is added to the
errors_log
and the errors_count
is incremented (resulting in an interruption if the 10% threshold is exceeded):- SKU lists — must not be linked to any promotion or promotion rules (i.e. any associated promotion must be removed first)
- SKUs — must not be linked to any SKU list item within a bundle (i.e. any associated bundle must be removed first)
When cleaning up resources, you can fine-tune the data to be removed by applying some filters (both to the resources and their relationships) using the
filters
attribute:...
"filters": {
"{{predicate}}": {{value}},
...
}
To compose the filter predicate you just need to follow the same syntax you use when filtering a collection of resources —
{{attributes}}_{{matcher}}
. You must specify filtering rules as a valid JSON object. List values for the *_in
matcher need to be expressed as arrays.Request
Response
The following request creates a cleanup to remove the SKUs matching the specified code, along with their relationships:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/cleanups' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "cleanups",
"attributes": {
"resource_type": "skus",
"filters": {
"code_start": "SHIRT"
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created cleanup object:{
"data": {
"id": "WmjloIJzAS",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "skus",
"status": "pending",
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"filters": {"code_start": "SHIRT"},
"records_count": 100,
"errors_count": 0,
"processed_count": 0,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
Request
Response
The following request creates a cleanup to remove all the promotions that expired after a specified date, along with their relationships:
curl -g -X POST \
'https://yourdomain.commercelayer.io/api/cleanups' \
-H 'Accept: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token' \
-H 'Content-Type: application/vnd.api+json' \
-d '{
"data": {
"type": "cleanups",
"attributes": {
"resource_type": "promotions",
"filters": {
"expired_at_gteq": "2018-01-01T12:00:00.000Z"
}
}
}
}'
On success, the API responds with a
201 Created
status code, returning the created import object:{
"data": {
"id": "HGfdRsErUk",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "promotions",
"status": "pending",
"started_at": null,
"completed_at": null,
"interrupted_at": null,
"filters": {"expired_at_gteq": "2018-01-01T12:00:00.000Z"},
"records_count": 55,
"errors_count": 0,
"processed_count": 0,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
CLI command
Success message
The following command cleans up all the SKUs whose code contains a specific subset of characters:
commercelayer cleanups:create -t skus -w code_cont=FLASHSALE
On success, the CLI prompts this message on your terminal, separating the single cleanups and showing some basic information about the completed processes:
| ID | Cleanup progress | % | Status | TBP↓ | Prc. | Err. |
---------------------------------------------------------------------------------------
| DRlYbHDopv | █████████████████████████ | 100% | completed | 0 | 10000 | 0 |
| MReZgHXLpw | █████████████████████████ | 100% | completed | 0 | 5000 | 0 |
Cleanup of 15000 skus completed.
You can inspect the status of a specific cleanup by fetching it by ID and looking at the
status
attribute.When you create a cleanup, it tries immediately to start the removal process, entering the
in_progress
status. In case the async queue is saturated, the cleanup remains pending
until it gets a chance to be processed.Request
Response
The following request fetches a single cleanup, the one identified by the ID "WmjloIJzAS":
curl -g -X GET \
'https://yourdomain.commercelayer.io/api/cleanups/WmjloIJzAS' \
-H 'Content-Type: application/vnd.api+json' \
-H 'Authorization: Bearer your-access-token'
On success, the API responds with a
200 OK
status code, returning the single cleanup object:{
"data": {
"id": "WmjloIJzAS",
"type": "cleanups",
"links": {...},
"attributes": {
"resource_type": "skus",
"status": "completed",
"started_at": "2018-01-01T12:00:00.000Z",
"completed_at": "2018-01-01T12:01:00.000Z",
"interrupted_at": null,
"filters": {},
"records_count": 100,
"errors_count": 1,
"processed_count": 99,
"errors_log": {...},
"created_at": "2018-01-01T12:00:00.000Z",
"updated_at": "2018-01-01T12:00:00.000Z",
"reference": null,
"reference_origin": null,
"metadata": {}
},
"relationships": {
"events": { ... }
},
"meta": {...}
}
}
You can also leverage Commerce Layer real-time webhooks mechanism, listen to
cleanups.create
, cleanups.start
, cleanups.complete
, cleanups.interrupt
, or cleanups.destroy
and react properly.When a cleanup is completed or interrupted, constraints errors are reported through the
errors_count
and errors_log
attributes:...
"errors_count": 5,
"processed_count": 95,
"errors_log": {
"HjKhdjGfDD": "Cannot delete record because of dependent bundles",
...
},
...
Last modified 4mo ago