File Transfer Protocol (FTP/SFTP)

Supported Protocols

When enabled, Buckit supports FTP access over the following protocols:

• SSH File Transfer Protocol (SFTP)

  SFTP is defined by the Internet Engineering Task Force (IETF) as an extension of SSH 2.0. SFTP allows file transfer over SSH for use with Transport Layer Security (TLS) and virtual private network (VPN) applications.

  Your FTP client must support SFTP.

• File Transfer Protocol over SSL/TLS (FTPS)

  FTPS allows for encrypted FTP communication with TLS certificates over the standard FTP communication channel. FTPS should not be confused with SFTP, as FTPS does not communicate over a Secure Shell (SSH).

  Your FTP client must support FTPS.

• File Transfer Protocol (FTP)

  Unencrypted file transfer.

  Buckit does not recommend using unencrypted FTP for file transfer.

Supported Commands

When enabled, Buckit supports the following SFTP operations:

• get

• put

• ls

• mkdir

• rmdir

• delete

Buckit does not support either append or rename operations.

Considerations

Prerequisites

• Buckit RELEASE.2023-04-20T17-56-55Z or later.

• Enable an FTP or SFTP port for the server.

• A port to use for the FTP commands and a range of ports to allow the FTP server to request to use for the data transfer.

Procedure

1. Start Buckit with an FTP and/or SFTP port enabled.

   FTPS

   The following example starts Buckit with FTPS enabled.

   buckit server http://server{1...4}/disk{1...4} \
   --ftp="address=:8021"                         \
   --ftp="passive-port-range=30000-40000"        \
   --ftp="tls-private-key=path/to/private.key"   \
   --ftp="tls-public-cert=path/to/public.crt"    \
   ...
   

   Note

   Omit tls-private-key and tls-public-cert to use the Buckit default TLS keys for FTPS. For more information, see the TLS on Buckit documentation.

   SFTP/FTP

   buckit server http://server{1...4}/disk{1...4}        \
   --ftp="address=:8021"                                \
   --ftp="passive-port-range=30000-40000"               \
   --sftp="address=:8022"                               \
   --sftp="ssh-private-key=/home/buckituser/.ssh/id_rsa" \
   ...
   

   See the buckit server --ftp and buckit server --sftp for details on using these flags to start the Buckit service. To connect to the an FTP port with TLS (FTPS), pass the tls-private-key and tls-public-cert keys and values, as well, unless using the Buckit default TLS keys.

   The output of the command should return a response that resembles the following:

   Buckit FTP Server listening on :8021
   Buckit SFTP Server listening on :8022
   

2. Use your preferred FTP client to connect to the Buckit deployment. You must connect as a user whose policies allow access to the desired buckets and objects.

   The specifics of connecting to the Buckit deployment depend on your FTP client. Refer to the documentation for your client.

   To connect over TLS or through SSH, you must use a client that supports the desired protocol.

3. Connect to Buckit

   SFTP/FTP

   The following example connects to an SFTP server as a Buckit user named buckituser and lists the contents of a bucket named runner.

   > sftp -P 8022 buckituser@buckit.example.net
   buckituser@buckit.example.net's password:
   Connected to buckit.example.net.
   sftp> ls runner/
   chunkdocs  testdir
   

   FTPS

   The following example uses the Linux FTP CLI client to connect to the Buckit server using a Buckit user named buckituser and list contents in a bucket named runner.

   > ftp buckit.example.net -P 8021
   Connected to buckit.example.net.
   220 Welcome to Buckit FTP Server
   Name (buckit.example.net:user): buckituser
   331 User name ok, password required
   Password:
   230 Password ok, continue
   Remote system type is UNIX.
   Using binary mode to transfer files.
   ftp> ls runner/
   229 Entering Extended Passive Mode (|||39155|)
   150 Opening ASCII mode data connection for file list
   drwxrwxrwx 1 nobody nobody            0 Jan  1 00:00 chunkdocs/
   drwxrwxrwx 1 nobody nobody            0 Jan  1 00:00 testdir/
   ...
   

4. Download an Object

   SFTP/FTP

   This example lists items in a bucket, then downloads the contents of the bucket.

   > sftp -P 8022 buckituser@buckit.example.net
   buckituser@buckit.example.net's password:
   Connected to buckit.example.net.
   sftp> ls runner/
   chunkdocs  testdir
   sftp> get runner/chunkdocs/metadata metadata
   Fetching /runner/chunkdocs/metadata to metadata
   metadata                               100%  226    16.6KB/s   00:00
   sftp>
   

   FTPS

   This example lists items in a bucket, then downloads the contents of the bucket.

   > ftp buckit.example.net -P 8021
   Connected to buckit.example.net.
   220 Welcome to Buckit FTP Server
   Name (buckit.example.net:user): buckituser
   331 User name ok, password required
   Password:
   230 Password ok, continue
   Remote system type is UNIX.
   Using binary mode to transfer files.ftp> ls runner/chunkdocs/metadata
   229 Entering Extended Passive Mode (|||44269|)
   150 Opening ASCII mode data connection for file list
   -rwxrwxrwx 1 nobody nobody           45 Apr  1 06:13 chunkdocs/metadata
   226 Closing data connection, sent 75 bytes
   ftp> get
   (remote-file) runner/chunkdocs/metadata
   (local-file) test
   local: test remote: runner/chunkdocs/metadata
   229 Entering Extended Passive Mode (|||37785|)
   150 Data transfer starting 45 bytes
      45        3.58 KiB/s
   226 Closing data connection, sent 45 bytes
   45 bytes received in 00:00 (3.55 KiB/s)
   ...
   

Connect to Buckit Using SFTP with a Certificate Key File

Buckit supports mutual TLS (mTLS) certificate-based authentication on SFTP, where both the server and the client verify the authenticity of each other.

This type of authentication requires the following:

1. Public key file for the trusted certificate authority

2. Public key file for the Buckit Server minted and signed by the trusted certificate authority

3. Public key file for the user minted and signed by the trusted certificate authority for the client connecting by SFTP and located in the user’s .ssh folder (or equivalent for the operating system)

The keys must include a principals list of the user(s) that can authenticate with the key:

ssh-keygen -s ~/.ssh/ca_user_key -I buckituser -n buckituser -V +1h -z 1 buckituser.pub

• -s specifies the path to the certificate authority public key to use for generating this key. The specified public key must have a principals list that includes this user.

• -I specifies the key identity for the public key.

• -n creates the user principals list for which this key is valid. You must include the user for which this key is valid, and the user must match the username in Buckit.

• -V limits the duration for which the generated key is valid. In this example, the key is valid for one hour. Adjust the duration for your requirements.

• -z adds a serial number to the key to distinguish this generated public key from other keys signed by the same certificate authority public key.

Buckit requires specifying the Certificate Authority used to sign the certificates for SFTP access. Start or restart the Buckit Server and specify the path to the trusted certificate authority’s public key using an --sftp="trusted-user-ca-key=PATH" flag:

buckit server {path-to-server} --sftp="trusted-user-ca-key=/path/to/.ssh/ca_user_key.pub" {...other flags}

When connecting to the Buckit Server with SFTP, the client verifies the Buckit Server’s certificate. The client then passes its own certificate to the Buckit Server. The Buckit Server verifies the key created above by comparing its value to the the known public key from the certificate authority provided at server startup.

Once the Buckit Server verifies the client’s certificate, the user can connect to the Buckit server over SFTP:

sftp -P <SFTP port> <server IP>

> sftp -P 8022 my-ldap-user=ldap@[buckit.example.net]:/bucket

> sftp -P 8022 my-ldap-user=svc@[buckit.example.net]:/bucket