Core Administration Concepts

What Is Object Storage?

An object is binary data, sometimes referred to as a Binary Large OBject (BLOB). Blobs can be images, audio files, spreadsheets, or even binary executable code. Object Storage platforms like Buckit provide dedicated tools and capabilities for storing, retrieving, and searching for blobs.

Buckit Object Storage uses buckets to organize objects. A bucket is similar to a folder or directory in a filesystem, where each bucket can hold an arbitrary number of objects. Buckit buckets provide the same functionality as AWS S3 buckets.

For example, consider an application that hosts a web blog. The application needs to store a variety of blobs, including rich multimedia like videos and images.

Buckit supports multiple levels of nested directories through the feature of prefixing to support even the most dynamic object storage workloads.

How does Buckit determine access to objects?

Buckit requires the client perform both authentication and authorization for each new operation. Identity and access management (IAM) is therefore a critical component of a Buckit configuration.

Authentication verifies the identity of a connecting client. Buckit requires clients to authenticate using AWS Signature Version 4 protocol with support for the deprecated Signature Version 2 protocol. Specifically, clients must present a valid access key and secret key to access any S3 or Buckit administrative API, such as PUT, GET, and DELETE operations.

Buckit then checks that authenticated users or clients have authorization to perform actions or use resources on the deployment. Buckit uses Policy-Based Access Control (PBAC), where each policy describes one or more rules that outline the permissions of a user or group of users. Buckit supports S3-specific actions and conditions when creating policies.

By default, Buckit denies access to actions or resources not explicitly referenced in a user’s assigned or inherited policies.

Buckit provides an access management feature as part of the software. Alternatively, you can configure Buckit to authenticate with one of several external IAM providers using either Active Directory/LDAP or OpenID/OIDC.

How does Buckit secure data?

Buckit supports methods that encode objects while on drive (encryption-at-rest) and during transition from one location to another (encryption-in-transit, or “in flight”). When enabled, Buckit utilizes server-side encryption to write objects in an encrypted state. To retrieve and read an encrypted object, the user must have appropriate access privileges and also provide the object’s decryption key.

Buckit supports Transport Layer Security (TLS) versions 1.2 and 1.3 encrypting objects. TLS replaces the previously used Secure Socket Layer (SSL) method that has since been deprecated. The TLS standard, maintained by the Internet Engineering Task Force (IETF), provides the standards used by internet communications to support encryption, authentication, and data integrity.

The process of authenticating a user and verifying access to objects is known as the TLS Handshake. Once authenticated, TLS provides the cipher to encrypt and then decrypt the transfer of information from the server to the requesting client.

Buckit supports several methods of Server-Side Encryption.

Can I organize objects in a folder structure within buckets?

Buckit utilizes a prefix method for each object that mimics a folder structure from traditional file systems. Prefixing involves prepending the name of an object with a fixed string.

With prefixes, you do not manually create folders and subfolders. Instead, Buckit looks for the / character in the prefix of an object’s name. Each / indicates a new folder or subfolder.

Using the object’s name and prefix, Buckit automatically generates a series of folders and subfolders for stored objects. When you use the same prefix string on multiple objects, Buckit identifies those as similar or grouped objects.

For example, an object named /articles/john.doe/2022-01-02-Buckit-Object-Storage.md winds up in the articles bucket in a folder labeled john.doe.

A Buckit object store might resemble the following structure, with three buckets. Buckit automatically generates two folders in the articles bucket based on the prefixes for those objects.

/ #root
/images/
   2022-01-02-Buckit-Diagram.png
   2022-01-03-Buckit-Advanced-Deployment.png
   Buckit-Logo.png
/videos/
   2022-01-04-Buckit-Interview.mp4
/articles/
   /john.doe/
      2022-01-02-Buckit-Object-Storage.md
      2022-01-02-Buckit-Object-Storage-comments.json
   /jane.doe/
      2022-01-03-Buckit-Advanced-Deployment.png
      2022-01-02-Buckit-Advanced-Deployment-comments.json
      2022-01-04-Buckit-Interview.md

Buckit itself does not limit the number of objects that any specific prefix can contain. However, hardware and network conditions may show performance impacts with large prefixes.

• Deployments with modest or budget-focused hardware should architect their workloads to target 10,000 objects per prefix as a baseline. Increase this target based on benchmarking and monitoring of real world workloads up to what the hardware can meaningfully handle.

• Deployments with high-performance or enterprise-grade hardware can typically handle prefixes with millions of objects or more.

For a deeper discussion on the benefits of limiting prefix contents, see the article on optimizing S3 performance.

How can I backup and restore objects on Buckit?

Buckit provides two types of replication to copy an object, its versions, and its metadata from one location to another. You can configure replication at either the bucket level or at the site level.

• Bucket level replication can function as either one-way, active-passive replication (such as for archival purposes) or as two-way, active-active replication to keep two buckets in sync with each other.

• Site level replication functions as two-way, active-active replication to keep multiple data locations (such as different geographic data centers) in sync with one another.

Besides replication, Buckit provides a mirroring service. bm mirror copies only the actual object to any other S3 compatible data store, including other Buckit stores. However, versions and metadata do not back up with the bm mirror command.

What tools does Buckit provide to manage objects based on speed and frequency of access?

Tiering rules allow frequently accessed objects to store on hot or warm storage, which is typically more expensive but provides better performance.

Less frequently accessed objects can move to cold storage. Cold storage often exchanges slower performance for a cheaper price.

How does Buckit protect objects from accidental overwrite or deletion?