Amazon S3

Files.com's integration with Amazon S3 allows you to integrate with files on a Amazon S3 bucket in several different ways.

Files.com's Remote Server Mount feature gives you the ability connect a specific folder on Files.com to the remote server in a real time manner.

That folder then becomes a client, or window, accessing the files stored in your remote server or cloud.

Once you configure a Mount, any operation you perform on or inside that folder will act directly on the remote in real time. Whether you are dropping a file into that folder, deleting a file, creating a subfolder, or performing any other file/folder operations your Files.com user has permissions for, those operations will "pass through" to the remote in real time.

This powerful feature enables a wide variety of use cases such as accessing files on a counterparty (client or vendor)'s cloud without provisioning individual access to individual users, reducing storage costs by leveraging on-premise or bulk storage solutions, enabling applications to access 3rd party clouds via Files.com API, FTP, SFTP, or Files.com Apps and many more

Alternatively, Files.com's Remote Server Sync feature give you the ability to push or pull files to or from remote servers. This means that the files will exist in both places at the end of the sync process.

A remote sync can be a "push", where files from your Files.com site are transferred to the remote server, a "pull" where files are transferred from the remote server to your Files.com site, or a two-way "sync" where files that are new or changed in either location are pushed and pulled to maintain a synchronized state between the folder on your Files.com site and that on the remote server.

Files.com supports connecting to S3 buckets in many regions, even regions where Files.com itself doesn't have an AWS presence. This includes Govcloud. We are happy to expand our list of supported regions; please contact us if you have a need to access a region you don't see listed.

Files.com requires full access to the Bucket being used, so we recommend creating a bucket for the exclusive use of Files.com.

The AWS Region Code of your S3 bucket name can be found by using the get-bucket-location command of the AWS CLI tool. You can cross-reference the region code and region name using Amazon's online documentation.

Your Amazon S3 bucket name can be found in the Amazon AWS Console, within the Amazon S3 section, under the Buckets list.

Files.com uses an Access Key and Secret Key to authenticate requests, following the protocol established by AWS for S3. You will use AWS IAM to create an IAM user for Files.com.

After generating the IAM User, copy its Access Key and Secret Key to Files.com.

You will also need to apply a policy to the IAM user to grant it full permissions to the bucket being used.

Below is an example S3 user policy (replace <bucketname> with your bucket name):

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action": "s3:*",
         "Resource": [
           "arn:aws:s3:::<bucketname>",
           "arn:aws:s3:::<bucketname>/*"
         ]
      }
   ]
}

Access to the S3 bucket is determined by the policies and permissions in Amazon S3.

If your S3 bucket is read-only, or you wish Files.com to be restricted to read-only permissions, then configure the S3 user policy (replace <bucketname> with your bucket name) as below:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
           "s3:GetObject",
           "s3:ListBucket",
           "s3:GetBucketLocation"
         ],
         "Resource": [
           "arn:aws:s3:::<bucketname>",
           "arn:aws:s3:::<bucketname>/path/to/subfolder/*"
         ]
      }
   ]
}

If your S3 user policy does not have access to the entire bucket, or is restricted to specific subfolders only, then verification of access to the bucket may fail when testing credentials.

Use the advanced setting for Can these credentials access the root of this remote or are they restricted to a subfolder? to specify a subfolder that can be used to verify access when testing your credentials.

Files.com's integration with Amazon S3 previously failed to support S3 buckets that use S3's "Object Lock" features, which includes S3's Legal Hold feature. As of March 2022, this feature is now supported.

Type "Remote servers" in the search box at the top of every page, and then click on the matching result. Click the Add new remote server button and select Amazon S3.

Once your Remote Server is added, you can integrate it to Files.com as either a Remote Server Mount or Remote Server Sync.

Remote Server Mounts are created by mounting them onto an empty folder in Files.com. This folder should ideally not be the Root of your site, although that is supported if you need it.

From the Files icon on the left, navigate to the location where you want the mounted folder to be and create a new folder. Navigate into the newly created folder and click the Folder Settings button on the top right.

Select Remote Server Mount from the list and click Add new remote server mount button. Select the remote server.

Choose the Remote folder, which is the portion of the remote file system that will be mounted into this folder on Files.com. You can either by leave the default "/" (i.e., the remote server's root directory) or click on Choose a different folder link and navigate to the remote folder you want to this folder to connect to.

Click the Save button. The folder will reload and immediately list the remote folders/files from the selected remote path.

If you instead prefer to do a Sync with the remote, follow these directions.

From Files, navigate into the folder where you would like to add the remote server sync and click Folder settings > Sync to/from remote server.

Click the Add new remote server sync button to reveal the form.

Select the server you would like to transfer to or from by clicking on the Remote server menu.

Next choose your Sync direction. You have three choices:

1. Push to the remote server: This option uploads files and folders from your designated folder in your Files.com site to the remote server.
2. Pull from the remote server: This option downloads files from the remote server and saves them in your designated folder in your Files.com site.
3. Two-way sync: this option checks for new files, deleted files, and changed modification dates on both servers and then pushes and pulls as needed to keep the folders synchronized on both servers.

You have the option to delete files on the source server after a push or pull. Use the After copying menu to select whether you would like files that are successfully transferred to be deleted from or kept on the source server.

Enter the remote path to or from which you would like files and folders transferred, starting after the folder/directory your remote user lands in upon authentication.

For example: if the remote server has a folder structure "folderA/folderB/folderC", and the user credentials that you have configured your sync server to log in with automatically land that user inside "folderA", then to properly configure your sync folder behavior to transfer files to or from "folderC", you would enter the path as "folderB/folderC".

Certain remotes that use OAuth for authentication may require regular rotation of your credentials. When this is needed, you will see an alert in the top left of the web interface. You can click the link in that alert to re-authenticate and re-establish the connection to the remote.

Amazon S3 object storage is not a hierarchical file system and does not use directories (folders) to organize files. Files and data are stored in objects but are presented in the illusion of a hierarchical file system.

This becomes most apparent when creating, syncing, or uploading an empty folder to Amazon S3 storage.

Amazon S3 storage will represent an empty folder as a zero-byte object of the same name. The zero-byte object will be presented as a zero-byte file.

Amazon S3 will manage these zero-byte objects, and their corresponding empty folders, itself. However these zero-byte objects may present themselves to other programs, applications, and services that use the S3 bucket. They should be considered a "normal" side effect of using S3 object storage.

Files.com follows the same conventions used by other software to emulate folders on these non-hierarchical file systems. We aim to interoperate using as many reasonable conventions, standards, and best practices, as possible.

Generally speaking, most issues are caused by incorrectly entering a configuration item. Make sure that there are no typos and that you are specifying the correct bucket name, region and keys.

If you specify a region that does not match where the bucket or keys exist then you may see an error such as:

The bucket you are attempting to access must be addressed using the specified endpoint.

Make sure that you choose the correct AWS region that corresponds to the location of the bucket and keys.

Be aware of case sensitivity differences when copying, moving, or syncing files and folders between Amazon S3 and other storage locations. Amazon S3 is a case sensitive system whereas other systems are not. This can cause files to be overwritten, and folders to have their contents merged, if their case insensitive names are a match.