Object Deletion

Overview

This page summarizes how a DELETE operation affects objects depending on the configuration of the bucket that contains the object.

Any combination of the following factors may impact how DELETE operations function:

Buckit uses a policy based access control system for access management. The user or service account must provide the correct policy action and conditions to allow a DELETE for the bucket and object.

Unversioned Objects

When performing a DELETE operation on an object in a bucket that does not have versioning enabled, the operation is straightforward. After verifying the user or service account has permission to perform the DELETE operation, Buckit permanently removes the object.

The user or service account requesting the delete action the action must have the s3:DeleteObject action permission for the bucket and object.

Versioned Objects

DELETE operations work differently when an object is versioned.

The user or service account must have the s3:DeleteObjectVersion action permission for the bucket and object.

Delete operations on the current version

A DELETE operation on a versioned object that does not specify a version UUID results in the creation of a DeleteMarker placed as the head of the object.

In this scenario, Buckit does not actually remove the object or any of its versions from the disk. All existing versions of the object remain available to access by specifying the version’s UUID. When a DeleteMarker is the head for the object, Buckit does not serve the object for GET requests that do not specify a version ID. Instead, Buckit returns a 404-like response.

You can find the UUID of object versions with bm ls --versions.

To remove the current version of the object from the drive, find the UUID of the version, and then use bm rm --version-id=UUID ... to delete the current version. In this scenario, the immediately preceding version of the object then becomes the current version of the object served for GET requests of the object with no UUID specified.

Warning

Specifying a version-id in a DELETE operation is irreversible. Buckit removes the specified version from the drive and cannot retrieve it.

Delete operations on a prior version

To delete prior versions of an object, specify the version’s UUID. You can retrieve the version UUID with bm ls --versions. When the DELETE request specifies a version-id and the user has the correct permissions to delete the object version`, Buckit permanently removes the specified version from the drive.

Warning

Specifying a version-id in a DELETE operation is irreversible. Buckit removes the specified version from the drive and cannot retrieve it.

Delete all versions

Use bm rm --versions to delete all versions of an object. This is irreversible.

Lifecycle Management Expiration

You can define one or more lifecycle management expiration rule(s) to expire objects after a certain version number count or a certain period of time. When more versions exist than the rule specifies, or when a version is older than specified, Buckit permanently removes the object version from the drive.

These rules rely on the scanner to process the rule on the bucket. The scanner operates as a lower priority continuous process where READ and WRITE actions are preferred. Because of this, object versions that meet the requirements for expiration may not immediately be removed from Buckit.

See the scanner page for more details on how the scanner works and configuration options.

DeleteMarkers are their own objects. Lifecycle rules can remove DeleteMarkers that are the only remaining versions of their objects.

With JSON, lifecycle rules can remove all versions of a deleted object after a specified number of days.

Retained Objects

Buckit protects objects subject to a locking rule from being overwritten or deleted. These rules require that objects be retained until either the rule expires or is removed.

DELETE operations on locked objects without a specified version result in the creation of a DeleteMarker for the object. However, the object versions themselves are retained as required by the lock.

DELETE operations that specify an object version are subject to the retention rules. Buckit protects object versions subject to a lock from being overwritten or deleted until the lock expires or is removed.

Replicated Objects

Replication duplicates objects from one location to another. Buckit supports replication at the bucket level or the cluster (“site”) level.

Delete operations may or may not replicate, depending on the type of replication and how the replication is configured.

Site Replication

For clusters with multi-site replication enabled, Buckit replicates all delete operations performed on any cluster to each of the other clusters in the peer group.

Delete behavior on any single peer follows the same processes as any Buckit deployment.

Bucket Replication

With bucket replication, Buckit supports replicating delete operations between a source bucket and a configured remote bucket. Buckit synchronizes deleting specific object versions and new delete markers. Delete operation replication uses the same replication process as all other replication operations.

Buckit requires explicitly enabling versioned deletes and delete marker replication. Use the bm replicate add --replicate field to specify either delete and delete-marker or both to enable versioned deletes and delete marker replication, respectively. To enable both, specify both strings using a comma separator: delete,delete-marker.

For delete marker replication, Buckit begins the replication process after a delete operation creates the delete marker. Buckit uses the X-Minio-Replication-DeleteMarker-Status metadata field for tracking delete marker replication status. In active-active replication configurations, Buckit may produce duplicate delete markers if both clusters concurrently create a delete marker for an object or if one or both clusters were down before the replication event synchronized.

For replicating the deletion of a specific object version, Buckit marks the object version as PENDING until replication completes. Once the remote target deletes that object version, Buckit deletes the object version on the source. While this process ensures near-synchronized version deletion, it may result in listing operations returning the object version after the initial delete operation. Buckit uses the X-Minio-Replication-Delete-Status for tracking delete version replication status.

Buckit only replicates explicit client-driven delete operations. Buckit does not replicate objects deleted by lifecycle management expiration rules. For active-active configurations, set the same expiration rules on all of of the replication buckets to ensure consistent application of object expiration.

Buckit Trims Empty Object Prefixes on Source and Remote Bucket

If a delete operation removes the last object in a bucket prefix, Buckit recursively removes each empty part of the prefix up to the bucket root. Buckit only applies the recursive removal to prefixes created implicitly as part of object write operations. Buckit does not recursively remove prefixes created using an explicit directory creation command, such as bm mb.

If a replication rule enables replication delete operations, the replication process also applies the implicit prefix trimming behavior on the destination Buckit cluster.

For example, consider a bucket photos with the following object prefixes:

photos/2021/january/myphoto.jpg // 2021/january/ created implicitly based on the object name
photos/2021/february/myotherphoto.jpg // 2021/february/ created implicitly based on the object name
photos/NYE21/NewYears.jpg // NYE21/ explicitly created in the bucket

photos/NYE21 is the only prefix explicitly created using bm mb. All other prefixes were implicitly created as part of writing the object located at that prefix.

A command removes myphoto.jpg. Buckit automatically trims the empty /january/ prefix.
A command then removes the myotherphoto.jpg. Buckit automatically trims the /february/ prefix and the now-empty /2021 prefix.
A command removes the NewYears.jpg object. Buckit leaves the /NYE21/ prefix remains in place since it was explicitly created.