Cloud file storage is a method for storing data in the cloud that provides servers and applications access to data through shared file systems. This compatibility makes cloud file storage ideal for workloads that rely on shared file systems and provides simple integration without code changes.
A file system in the cloud is a hierarchical storage system that provides shared access to file data. Users can create, delete, modify, read, and write files and can organize them logically in directory trees for intuitive access.
Cloud file sharing can be defined as a service that provides simultaneous access for multiple users to a common set of file data in the cloud. Security for file sharing in the cloud is managed with user and group permissions enabling administrators to tightly control access to the shared file data.
Storing file data in the cloud delivers advantages in three key areas:
1. Scalability - Although not every cloud file storage solution can scale, leveraging all the capabilities of the cloud, the most advanced solutions provide the ability to start with the capacity and performance you need today and grow your capacity as needed. No more over provisioning to try and anticipate future needs.
2. Interoperability - Many existing applications require integration with shared file services that follow existing file system semantics. Cloud file storage solutions offer a distinct advantage as there is no new code to write to have secure, shared file access.
3. Budget and Resources - Operating file services on-premises requires budget for hardware, ongoing maintenance, power, cooling, and physical space. Cloud file storage enables organizations to redeploy technical resources to other projects that bring more value to the business.
Use cases such as large content repositories, development environments, media stores, and user home directories are ideal workloads for cloud file storage.
The flexibility to support and easily integrate with existing applications and the ease to deploy, manage, and maintain a file storage solution in the cloud provides the ability to support a broad spectrum of applications and verticals.
The need for shared file storage for web serving applications can be a challenge when integrating backend applications. Typically, there are multiple web servers delivering a web site’s content, with each web server needing access to the same set of files. Since cloud file storage solutions adhere to the expected file system semantics, file naming conventions, and permissions that web developers are accustomed to, cloud file storage can easily integrate their web applications.
Content Management Systems (CMS) require a common namespace and access to a file system hierarchy. Similar to web serving use cases, CMS environments typically have multiple servers that need access to the same set of files to serve up content. Since cloud file storage solutions adhere to the expected file system semantics, file naming conventions, and permissions that developers are accustomed to, storage of documents and other files can be easily integrated into existing CMS workflows.
Big data requires storage that can handle very large amounts of data while scaling to keep up with growth and providing the performance necessary to deliver data to analytics tools. Many analytics workloads interact with data via a file interface, rely on file semantics such as file locks, and require the ability to write to portions of a file. Since cloud file storage supports the needed file systems semantics and can scale capacity as well performance, it is ideal for delivering a file sharing solution that is easy to integrate into existing big data workflows.
Digital Media & Entertainment workflows are constantly changing. Many leverage a hybrid cloud deployment and need standardized access using network file protocols, such as NFS. These workflows require flexible, consistent, and secure access to data from off-the-shelf, custom-built, and partner solutions. Since cloud file storage adheres to existing file system semantics, storage of rich media content for processing and collaboration can be easily integrated for content production, digital supply chains, media streaming, broadcast playout, analytics and archive.
The use of home directories for storing files only accessible by specific users and groups is useful for many cloud workflows. Businesses that are looking to take advantage of the scalability and cost benefits of the cloud are extending access to home directories for many of their users. Since cloud file storage solutions adhere to required file system semantics and standard permissions models, customers can easily lift-and-shift applications to the cloud that need this capability.
Backing up data using existing mechanisms, software, and semantics can create an isolated recovery scenario with little locational flexibility for recovery. Many businesses want to take advantage of the flexibility of storing database backups in the cloud either for temporary protection during updates or for development and test. Since cloud file storage solutions present a standard filesystem that can be easily mounted from database servers, they can be an ideal platform to create portable database backups using native application tools or enterprise backup applications.
Development environments can be challenged to share data in a safe and secure way as they are collaborating to develop their latest innovations. With the need to easily share code and other files in an organized way, utilizing shared cloud file storage provides an organized and secure repository that is easily accessible within their cloud development environments. Cloud file storage delivers a scalable and highly available solution ideal for collaborating.
Docker containers are ideal for building microservices because they’re quick to provision, easily portable, and provide process isolation. Containers that need access to the original data each time it starts may require a shared file system that it can connect to regardless of which instance they’re running on. Cloud file storage can provide persistent shared access to data that all containers in a cluster can use.
A good file storage solution in the cloud must deliver the right performance and capacity for today and scale seamlessly as business needs change.
The benefits of cloud file storage are clear, but it is important to also note that not all cloud file storage solutions are created equal. There are many solutions that exist that range from a single node file server on a compute instance using block storage as the underpinnings with no scalability or few redundancies to protect the data; to a do-it-yourself clustered solution that requires time consuming setup, management and maintenance; to a fully managed solution, such as Amazon EFS, Amazon FSx for Windows File Server, or Amazon FSx for Lustre, that needs little to no setup or maintenance and performance to meet the needs of the most demanding application workloads.
Cloud file storage can be delivered in one of two ways: through fully managed solutions with minimal setup and little to no maintenance or through do-it-yourself solutions with separate compute, storage, software, and licensing, which also requires staffed expertise to configure and maintain. Amazon Elastic File System (Amazon EFS), FSx for Windows File Server, and FSx for Lustre are examples of a fully managed solution that provides simple, scalable file storage for use with Amazon EC2 instances in the AWS Cloud and on-premises.
There are three types of cloud storage: Object, File, and Block. Each offers their own unique advantages.
1. Object Storage - Applications developed in the cloud often take advantage of object storage's vast scalability and metadata characteristics. Object storage solutions like Amazon Simple Storage Service (Amazon S3) are ideal for building modern applications from scratch that require scale and flexibility, and can also be used to import existing data stores for analytics, backup, or archive.
2. File Storage - Many applications need to access shared files and require a file system. This type of storage is often supported with a Network Attached Storage (NAS) server. File storage solutions like Amazon Elastic File System (EFS), Amazon FSx for Windows File Server, and Amazon FSx for Lustre are ideal for use cases like large content repositories, development environments, media stores, user home directories, and Amazon FSx for Lustre is ideal for high-performance computing and machine learning workloads.
3. Block Storage - Other enterprise applications like databases or ERP systems often require dedicated, low latency storage for each host. This is analogous to direct-attached storage (DAS) or a Storage Area Network (SAN). Block-based cloud storage solutions like Amazon Elastic Block Store (EBS) are provisioned with each virtual server and offer the ultra-low latency required for high performance workloads.
Although object storage solutions enable storage of files as objects, accessing with existing applications requires new code and the use of API’s and direct knowledge of naming semantics. File storage solutions that support existing file system semantics and permissions models have a distinct advantage in that they do not require new code to be written to integrate with applications that are easily configured to work with shared file storage.
Block storage can be used as the underlying storage component of a self-managed file storage solution. However, the one-to-one relationship required between the host and volume makes it difficult to have the scalability, availability, and affordability of a fully managed file storage solution and would require additional budget and management resources to support. Using a fully managed cloud file storage solution removes complexities, reduces costs, and simplifies management.
There’s a vast amount of file-based data in the world and AWS provides fully managed file system services that help you easily address the diverse needs of your file-based applications and workloads.
Business application storage
Organizations require their mission critical business applications to be highly available, and a great many of these applications use shared file storage. Migrating these applications to the cloud provides the scalability, high availability and durability, security, and reduced costs, while increasing agility.
AWS offers two file system services optimized for your business applications.
- Amazon EFS provides a cloud-native fully managed file system that provides scalable, elastic file storage for a broad range of Linux based applications.
- Amazon FSx for Windows File Server provides a fully managed native Windows file system with the features and performance optimized for Windows-based business applications.
Amazon EFS and Amazon FSx for Windows File Server enable customers to migrate their Linux and Windows-based applications to AWS using fully managed file systems with the features, compatibility, performance, and security that these applications rely on.
Compute-intensive applications, like high performance computing, machine learning, and media processing, often require massive throughput and low latencies from a file system. These workloads often execute for a short period of time using input data that is stored in a low-cost data lake.
- AWS offers Amazon FSx for Lustre for these compute-intensive applications. Amazon FSx for Lustre allows customers to easily process their data with a file system that’s optimized for the performance and cost of short-lived, compute-intensive processing jobs, with input and output stored on Amazon S3.