Cloud-native applications need cloud-native storage systems

Storage challenges Cloud-native applications need cloud-native storage systems

07.12.2021By Sascha Uhl *

The way companies develop and operate software applications has been fundamentally changed by cloud-native applications. But this change has not only influenced the app development itself, but also has a decisive impact on the associated infrastructure.

Related companies

Cloud-native applications pose special challenges for storage systems.

New technologies such as containerization, serverless computing and microservices are among the core competencies of cloud-native environments. Companies planning to support such modern, cloud-native environments, including the associated technologies, need to be aware that traditional storage infrastructure is simply no longer enough.

Instead, companies must also use cloud-native storage platforms for cloud-native applications. But what does “cloud-native” even mean? The term is so broad that it even includes applications that are not really cloud-native.

Cloud-native – mobile, scalable and dynamic

The Cloud Native Computing Foundation (CNCF) describes cloud-native applications as “scalable applications in modern, dynamic environments, such as public, private and hybrid clouds”. In practice, this means: applications that are distributed over several locations, easily scalable, containerized and managed using Kubernetes.

Cloud-native storage systems must be mobile, scalable, and support the same technologies that are commonly supported by cloud-native environments. In other words, these systems must be able to generate data for “stateful“ Manage apps properly. At the same time, the usual challenges of cloud-native environments in data storage must be able to be overcome. These include, for example, the challenges associated with managing data for Kubernetes-based applications.

To be as effective as possible, cloud-native storage must be convincing in the following six categories: scalability, performance, consistency, durability, deployability and availability.

Scalability

Scalability is a crucial feature, both for cloud-native applications and environments. Therefore, it goes without saying that cloud-native storage must also be easily scalable. The overall scalability of a storage platform can be estimated by the following four criteria: client scalability (how well or easily the number of clients or users accessing the storage system can be increased), capacity scalability (how easily the storage capacity can be increased in a single provision of storage systems), throughput scalability (the ability to achieve more throughput or process larger amounts of data per second) and cluster scalability (the ability to additional components). A cloud-native storage must be highly scalable in all of these four areas.

Performance

The performance of cloud-native storage platforms has a significant impact on cloud-native environments, as they must be able to quickly complete reads or writes and perform a significant number of storage operations per second. In addition, they must deliver high data throughput, which can be stored or retrieved in MB/s or GB/s.

Consistency

Another essential factor for cloud-native storage systems is consistency. Platforms are considered consistent when reads after writing, updating or deleting data output the latest version without delays, the faster the update occurs, the more consistent the system is. If the data is only available after a delay, then the system is classified as only conditionally consistent. In this case, the delay can be considered as RPO (Recovery Point Objective), that is, the starting time for data recovery, since it represents the maximum extent of data loss in the event of a component failure.

Durability

Cloud-native storage solutions must have a long service life and protect against data loss. However, longevity means more than just a reliable system – truly durable platforms must be able to guarantee that data is stored securely for longer periods of time. These systems should support data protection at several levels (including a sufficient number of backups) and include several levels of redundancy (for example, local redundancy, redundancy across regions, redundancy across different availability zones of the public cloud and redundancy to remote locations). In addition, the systems must be able to detect damaged data and automatically restore or reconstruct it. Storage media that lose performance over their lifetime, as well as wear, for example due to failing components, are common causes of damaged data.

In addition, the specific storage media that make up a cloud-native storage platform (for example, SSDs, spinning disks and tapes) should also be physically resistant to protect the data from external hazards such as natural disasters.

Deployability

Cloud-native applications are extremely mobile and can be easily distributed across multiple locations. To ensure this flexibility, it is important to be able to deploy storage systems as needed. This requires a software-defined, scalable approach that allows companies to immediately increase storage capacity without having to add new devices. A storage architecture with a single namespace is suitable for this purpose. Since these systems connect all nodes in a global peer-to-peer data structure, new nodes (and thus more capacity) can be added at any location if necessary, all using the existing infrastructure.

Availability

Last but not least, cloud-native storage must be highly available, i.e. provide immediate access to the data in the event of a failure, regardless of where the error occurs (for example, in the transmission system, storage medium, controller, and so on). In order to be considered highly available, the software must be able to detect, repair and restore failed components, store redundant copies on a separate device and also perform a failover to redundant devices.

Not just in DevOps environments

In conclusion, it can be said that cloud-native applications are no longer used exclusively in DevOps environments, but are increasingly also used in production. This applies to both “stateless” as well “stateful” Cloud-native apps. As more and more cloud-native applications support real–world use cases – especially stateful cloud–native applications – companies need a storage infrastructure that is specifically designed for these environments.

Effective cloud-native storage must be convincing in all six of the above areas.

Of course, the listed criteria are not the only ones that need to be observed. Security, manageability, observability, and API-based automation are also important factors. But for companies considering cloud-native storage options, the above checklist of scalability, performance, consistency, durability, deployability, and availability provides a good starting point.