bm replicate add

Parameters

ALIAS

Required

The alias of the Buckit deployment and full path to the bucket or bucket prefix on which to create the replication rule. For example:

bm replicate add --remote-bucket https://user:secret@mybuckit.cloudprovider.tld:9001/bucket play/mybucket

--remote-bucket

Required

Changed in version mc: RELEASE.2024-03-03T00-13-08Z

The --remote-bucket supports specifying an existing alias.

Specify the credentials, destination deployment, and bucket of the remote location. Value may be an IP address, URL, or alias/bucket.

For example, a URL based target might look like the following:

https://user:secret@mybuckit.cloudprovider.tld:9001/bucket

An alias based target might look like the following:

--remote-bucket minio-target/my-bucket

--bandwidth

Optional

Limit bandwidth rates to no more than the specified rate in KiB/s, MiB/s, or GiB/s. Valid units include:

• B for bytes

• K for kilobytes

• G for gigabytes

• T for terabytes

• Ki for kibibytes

• Gi for gibibytes

• Ti for tebibytes

For example, to limit bandwidth rates to no more than 1 GiB/s, use the following:

--limit-upload 1Gi

If not specified, Buckit does not limit the bandwidth rate.

--disable

Optional

Creates the replication rule in the “disabled” state. Buckit does not begin replicating objects using the rule until it is enabled using bm replicate update.

Objects created while replication is disabled are not immediately eligible for replication after enabling the rule. You must explicitly enable replication of existing objects by including "existing-objects" to the list of replication features specified to bm replicate update --replicate. See Replication of Existing Objects for more information.

--disable-proxy

Optional

When defining active-active replication between buckets, do not proxy.

By default, Buckit proxies.

--healthcheck-seconds

Optional

The length of time in seconds between checks on the health of the remote bucket.

If not specified, Buckit uses an interval of 60 seconds.

--id

Optional

Specify a unique ID for the replication rule. Buckit automatically generates an ID if one is not specified.

--limit-download

Optional

Limit download rates to no more than a specified rate in KiB/s, MiB/s, or GiB/s. Valid units include:

• B for bytes

• K for kilobytes

• G for gigabytes

• T for terabytes

• Ki for kibibytes

• Gi for gibibytes

• Ti for tebibytes

For example, to limit download rates to no more than 1 GiB/s, use the following:

--limit-download 1G

If not specified, Buckit uses an unlimited download rate.

--limit-upload

Optional

Limit upload rates to no more than the specified rate in KiB/s, MiB/s, or GiB/s. Valid units include:

• B for bytes

• K for kilobytes

• G for gigabytes

• T for terabytes

• Ki for kibibytes

• Gi for gibibytes

• Ti for tebibytes

For example, to limit upload rates to no more than 1 GiB/s, use the following:

--limit-upload 1G

If not specified, Buckit uses an unlimited upload rate.

--path

Optional

Enable path-style lookup support for the remote bucket.

Valid values include:

• on - use a path lookup to find the remote bucket

• off - use a resource locator style (such as a domain or IP address) lookup to find the remote bucket

• auto - ask Buckit to identify the correct type of lookup to use to find the remote bucket

When not defined, Buckit uses the auto value.

--priority

Optional

Specify the integer priority of the replication rule. The value must be unique among all other rules on the source bucket. Higher values imply a higher priority than all other rules.

The default value is 0.

--region

Optional

The region of the destination bucket to replicate contents to.

--replicate

Optional

Specify a comma-separated list of the following values to enable extended replication features.

• delete - Directs Buckit to replicate DELETE operations to the destination bucket.

• delete-marker - Directs Buckit to replicate delete markers to the destination bucket.

• existing-objects - Directs Buckit to replicate objects created before replication was enabled or while replication was suspended.

• metadata-sync - Directs Buckit to replicate metadata for each object. For active-active replication situations only.

  Omitting this value directs Buckit to stop replicating metadata-only changes back to the source.

If not specified, Buckit syncs all options.

--storage-class

Optional

Specify the Buckit storage class to apply to replicated objects.

--sync

Optional

Enable synchronous replication for this remote target.

By default, Buckit uses asynchronous replication.

--tags

Optional

Specify one or more ampersand & separated key-value pair tags which Buckit uses for filtering objects to replicate. For example:

bm replicate add --tags "TAG1=VALUE&TAG2=VALUE&TAG3=VALUE" ALIAS

Buckit applies the replication rule to any object whose tag set contains the specified replication tags.

Global Flags

This command supports any of the global flags.

Configure Bucket Replication

The following bm replicate add command creates a replication configuration that synchronizes all new objects, existing objects, delete operations, and delete markers to the remote target:

bm replicate add mybuckit/mybucket \
   --remote-bucket https://user:secret@minio.mysite.tld/remotebucket \
   --replicate "delete,delete-marker,existing-objects"

• Replace mybuckit/mybucket with the ALIAS and full bucket path for which to create the replication configuration.

• Replace the --remote-bucket value with the URL or path of the remote target. If using a file path format location, use the --path on option.

• The --replicate flag directs Buckit to replicate all delete operations, delete markers, and existing objects to the remote. See Replication of Delete Operations and Replication of Existing Objects for more information on replication behavior.

Configure Bucket Replication for Historical Data Record

The following bm replicate add command creates a new bucket replication configuration that synchronizes all new and existing objects to the remote target:

bm replicate add mybuckit/mybucket \
   --remote-bucket https://user:secret@minio.mysite.tld/remotebucket \
   --replicate "existing-objects"

• Replace mybuckit/mybucket with the ALIAS and full bucket path for which to create the replication configuration.

• Replace the --remote-bucket value with the location of the remote target. If using a file path format location, use the --path on option.

• The --replicate flag directs Buckit to replicate all existing objects to the remote. See Replication of Existing Objects for more information on replication behavior.

The resulting remote copy represents a historical record of objects on the remote, where delete operations on the source have no effect on the remote copy.

Server-Side Replication Requires Buckit Source and Destination

Buckit server-side replication only works between Buckit deployments. Both the source and destination deployments must run Buckit.

To configure replication between arbitrary S3-compatible services, use bm mirror.

Enable Versioning on Source and Destination Buckets

Buckit relies on the immutability protections provided by versioning to synchronize objects between the source and replication target.

Use the bm version enable command to enable versioning on both the source and destination bucket before starting this procedure:

bm version enable ALIAS/PATH

• Replace ALIAS with the alias of the Buckit deployment.

• Replace PATH with the bucket on which to enable versioning.

Required Permissions

Buckit strongly recommends creating users specifically for supporting bucket replication operations. See bm admin user and bm admin policy for more complete documentation on adding users and policies to a Buckit deployment.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "admin:SetBucketTarget",
                "admin:GetBucketTarget",
                "admin:ListBatchJobs",
                "admin:DescribeBatchJob",
                "admin:StartBatchJob",
                "admin:CancelBatchJob"
            ],
            "Effect": "Allow",
            "Sid": "EnableRemoteBucketConfiguration"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:GetObjectRetention",
                "s3:GetObjectLegalHold",
                "s3:PutReplicationConfiguration"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicationRuleConfiguration"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:GetBucketObjectLockConfiguration",
                "s3:GetEncryptionConfiguration"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicationOnBucket"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetReplicationConfiguration",
                "s3:ReplicateTags",
                "s3:AbortMultipartUpload",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetObjectVersionTagging",
                "s3:PutObject",
                "s3:PutObjectRetention",
                "s3:PutBucketObjectLockConfiguration",
                "s3:PutObjectLegalHold",
                "s3:DeleteObject",
                "s3:ReplicateObject",
                "s3:ReplicateDelete"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Sid": "EnableReplicatingDataIntoBucket"
        }
    ]
}

Replication of Existing Objects

Buckit supports automatically replicating existing objects in a bucket. Buckit existing object replication implements functionality similar to AWS Replicating existing objects between S3 buckets without the overhead of contacting technical support.

• To enable replication of existing objects when creating a new replication rule, include "existing-objects" to the list of replication features specified to bm replicate add --replicate.

• To enable replication of existing objects for an existing replication rule, add "existing-objects" to the list of existing replication features using bm replicate add --replicate. You must specify all desired replication features when editing the replication rule.

See Replication of Existing Objects for more complete documentation on this behavior.

Synchronization of Metadata Changes

Buckit supports two-way active-active replication configurations, where Buckit synchronizes new and modified objects between a bucket on two Buckit deployments. Buckit by default synchronizes metadata-only changes to a replicated object back to the “source” deployment. Prior to the this update, Buckit did not support synchronizing metadata-only changes to a replicated object.

With metadata synchronization enabled, Buckit resets the object replication status to indicate replication eligibility. Specifically, when an application performs a metadata-only update to an object with the REPLICA status, Buckit marks the object as PENDING and eligible for replication.

To disable metadata synchronization, use the bm replicate update --replicate command and omit replica-metadata-sync from the replication feature list.

Replication of Delete Operations

Buckit supports replicating delete operations onto the target bucket. Specifically, Buckit can replicate both Delete Markers and the deletion of specific versioned objects:

• For delete operations on an object, Buckit replication also creates the delete marker on the target bucket.

• For delete operations on versions of an object, Buckit replication also deletes those versions on the target bucket.

Buckit does not replicate objects deleted due to lifecycle management expiration rules. Buckit only replicates explicit client-driven delete operations.

Buckit requires explicitly enabling replication of delete operations using the bm replicate add --replicate flag. This procedure includes the required flags for enabling replication of delete operations and delete markers. See Replication of Delete Operations for more complete documentation on this behavior.

Replication of Encrypted Objects

Buckit supports replicating objects encrypted with automatic Server-Side Encryption (SSE-S3). Both the source and destination buckets must have automatic SSE-S3 enabled for Buckit to replicate an encrypted object.

As part of the replication process, Buckit decrypts the object on the source bucket and transmits the unencrypted object. The destination Buckit deployment then re-encrypts the object using the destination bucket SSE-S3 configuration. Buckit strongly recommends enabling TLS on both source and destination deployments to ensure the safety of objects during transmission.

Buckit does not support replicating client-side encrypted objects (SSE-C).

S3 Compatibility

The bm commandline tool is built for compatibility with the AWS S3 API and is tested with Buckit and AWS S3 for expected functionality and behavior.

Buckit provides no guarantees for other S3-compatible services, as their S3 API implementation is unknown and therefore unsupported. While bm commands may work as documented, any such usage is at your own risk.