S3 Compatible Remote Server

Files.com's support for S3 Compatible Remote Servers allows you to integrate with files on a remote server that speaks the S3 protocol in several different ways.

This allows you to integrate other private cloud services not directly integrated by Files.com. These include things like IBM Cloud, Oracle Cloud, Minio, and many other on-premise solutions that speak the S3 protocol.

For connections to S3 itself, Wasabi, Backblaze, please use our native integration to those services.

Files.com's Remote Server Mount feature gives you the ability connect a specific folder on Files.com to the remote server in a real time manner.

That folder then becomes a client, or window, accessing the files stored in your remote server or cloud.

Once you configure a Mount, any operation you perform on or inside that folder will act directly on the remote in real time. Whether you are dropping a file into that folder, deleting a file, creating a subfolder, or performing any other file/folder operations your Files.com user has permissions for, those operations will "pass through" to the remote in real time.

This powerful feature enables a wide variety of use cases such as accessing files on a counterparty (client or vendor)'s cloud without provisioning individual access to individual users, reducing storage costs by leveraging on-premise or bulk storage solutions, enabling applications to access 3rd party clouds via Files.com API, FTP, SFTP, or Files.com Apps and many more.

Alternatively, Files.com's Remote Server Sync feature give you the ability to push or pull files to or from remote servers. This means that the files will exist in both places at the end of the sync process.

A remote sync can be a "push", where files from your Files.com site are transferred to the remote server, a "pull" where files are transferred from the remote server to your Files.com site, or a two-way "sync" where files that are new or changed in either location are pushed and pulled to maintain a synchronized state between the folder on your Files.com site and that on the remote server.

Type "Remote Servers" in the search box at the top of the page and then click on the matching result. Click the Add new remote server button. Click to select S3 Compatible.

Provide an Internal name which is a friendly name for this Remote Server.

The Bucket name is the container where files are stored. The exact specification of the bucket name will depend on the exact server or software you are using that is S3 Compatible.

The Endpoint is the URL to your S3 Bucket (publicly accessible URL) that Files.com will connect to for your remote server.

The Region setting specifies the cloud region of the bucket. This is an optional setting and should only be used if your bucket provides specific region information to be used for the connection. If in doubt, leave this setting blank when setting up the connection to your bucket.

Files.com uses an Access Key and Secret Key to authenticate requests, following the protocol established by AWS for S3. The process to generate the Access Key and Secret Key, as well as the exact permissions required will differ depending on the exact server or software you are using that is S3 Compatible.

In general, Files.com requires full access to the specified Bucket.

If you have a Custom Domain installed on your site, that means Files.com has provisioned two dedicated IP addresses for your site and it will use them by default for outbound connections to the remote server. Provide these 2 IP addresses to your counterparties and ask them to whitelist them in any applicable firewall.

If you do not have a Custom Domain installed on your site, you do not have Dedicated IP Addresses provisioned for your site and Files.com will use its entire pool of IP addresses for connecting outbound to the remote server. If your counterparties maintain an IP Address whitelist, you will need to have them whitelist all of the IPs on this list.

Customers often ask for Dedicated IP addresses as a way to avoid having to ask their counterparty to whitelist a huge list of IP addresses.

We are able to offer that for Remote Server connection purposes via somewhat of a backdoor method, which is adding a Custom Domain to your site. Having a custom domain provides a justification for the dedicated IP address.

Files.com automatically provisions a pair of dedicated IP addresses for every site that has a custom domain enabled. We do that because FTP, unlike HTTP, requires that every custom domain be hosted on a dedicated IP address in order to have a custom SSL Certificate that matches the domain.

This means that if you have users who restrict outbound access via a firewall, they will only need to whitelist your two dedicated IP addresses. rather than having to whitelist our entire published list of IP addresses (see above).

Dedicated IPs, once provisioned, are used for both inbound connections to your site via your custom domain, as well as outbound connections from Files.com to certain applicable Remote Servers that are used for Remote Server Sync and Remote Server Mount.

By default, Files.com will use your dedicated IP addresses for outbound connections to FTP, SFTP, WebDAV, and S3 Compatible remote servers. However, you can disable the use of your dedicated IP in these circumstances if you need to. (You might do that if your counterparty has already whitelisted the main Files.com IP range, for example.)

Unlike FTP and SFTP remote servers, Files.com does not allow configuring a maximum number of connections to S3 Compatible remotes. This is because most services that offer S3-like services support a high number of parallel requests via HTTP. If you have a need for reducing the number of connections made to your remote S3 Compatible server, we would love to know more about your use case.

Once your Remote Server is added, now you need to integrate it to Files.com as either a Remote Server Mount or Remote Server Sync.

Remote Server Mounts are created by mounting them onto an empty folder in Files.com. This folder should ideally not be the Root of your site, although that is supported if you need it.

From the Files icon on the left, navigate to the location where you want the mounted folder to be and create a new folder. Navigate into the newly created folder and click the Folder Settings button on the top right.

Select Remote Server Mount from the list and click Add new remote server mount button. Select the remote server.

Choose the Remote folder, which is the portion of the remote file system that will be mounted into this folder on Files.com. You can either by leave the default "/" (i.e., the remote server's root directory) or click on Choose a different folder link and navigate to the remote folder you want to this folder to connect to.

Click the Save button. The folder will reload and immediately list the remote folders/files from the selected remote path.

If you instead prefer to do a Sync with the remote, follow these directions.

From Files, navigate into the folder where you would like to add the remote server sync and click Folder settings > Sync to/from remote server.

Click the Add new remote server sync button to reveal the form.

Select the server you would like to transfer to or from by clicking on the Remote server menu.

Next choose your Sync direction. You have three choices:

1. Push to the remote server: This option uploads files and folders from your designated folder in your Files.com site to the remote server.
2. Pull from the remote server: This option downloads files from the remote server and saves them in your designated folder in your Files.com site.
3. Two-way sync: this option checks for new files, deleted files, and changed modification dates on both servers and then pushes and pulls as needed to keep the folders synchronized on both servers.

You have the option to delete files on the source server after a push or pull. Use the After copying menu to select whether you would like files that are successfully transferred to be deleted from or kept on the source server.

Enter the remote path to or from which you would like files and folders transferred, starting after the folder/directory your remote user lands in upon authentication.

For example: if the remote server has a folder structure folderA/folderB/folderC, and the user credentials that you have configured your sync server to log in with automatically land that user inside folderA, then to properly configure your sync folder behavior to transfer files to or from folderC, you would enter the path as folderB/folderC.

Be aware of case sensitivity differences when copying, moving, or syncing files and folders between S3 compatible storage and other storage locations. S3 is a case sensitive system whereas other systems may not be. This can cause files to be overwritten, and folders to have their contents merged, if their case insensitive names are a match.

Certain remotes that use OAuth for authentication may require regular rotation of your credentials. When this is needed, you will see an alert in the top left of the web interface. You can click the link in that alert to re-authenticate and re-establish the connection to the remote.

S3 object storage is not a hierarchical file system and does not use directories (folders) to organize files. Files and data are stored in objects but are presented in the illusion of a hierarchical file system.

This becomes most apparent when creating, syncing, or uploading an empty folder to S3 compatible storage.

S3 compatible storage may represent an empty folder as a zero-byte object of the same name. The zero-byte object will be presented as a zero-byte file.

S3 will manage these zero-byte objects, and their corresponding empty folders, itself. However these zero-byte objects may present themselves to other programs, applications, and services that use the S3 bucket. They should be considered a "normal" side effect of using S3 object storage.

Files.com follows the same conventions used by other software to emulate folders on these non-hierarchical file systems. We aim to interoperate using as many reasonable conventions, standards, and best practices, as possible.