Storage types#

  • File Storage: Data stored in a hierarchical system, like traditional file systems.
    • Example: Dropbox, Microsoft OneDrive
  • Block Storage: Data divided into blocks, used for databases and high-performance applications.
    • Example: Amazon EBS (Elastic Block Store)
  • Object Storage: Manages data as objects, ideal for unstructured data like images and videos.
    • Example: IBM Cloud Object Storage, Google Cloud Storage

Cloud Storage Tiers#

Cloud storage tiers are designed to meet different requirements for data access frequency, storage duration, and cost efficiency. Here are the common tiers offered by cloud storage providers:

1. Standard or Hot Tier#

  • Description: Designed for data that is accessed frequently. Offers the highest performance in terms of access speed.
  • Use Case: Active database storage, streaming content, and any other data requiring immediate access.
  • Example: Amazon S3 Standard, Google Cloud Storage Standard

2. Cool Tier#

  • Description: For data that is accessed less frequently but still requires relatively quick access. Lower cost than the hot tier but may have higher access fees.
  • Use Case: Short-term backups, older project files not needed daily but that still might be needed unexpectedly.
  • Example: Azure Cool Blob Storage, Google Cloud Storage Nearline

3. Cold Tier#

  • Description: Best for data that is rarely accessed and stored for longer periods. Offers the lowest storage cost but higher access costs and longer access times.
  • Use Case: Long-term data archiving, legal records, and historical data not regularly accessed.
  • Example: Amazon S3 Glacier, Google Cloud Storage Coldline

4. Archive Tier#

  • Description: The most cost-effective option for data that is almost never accessed but must be retained for regulatory or compliance reasons. Access times can be several hours or longer.
  • Use Case: Archival of data required for compliance, such as financial records and medical records, which need to be stored for many years.
  • Example: Amazon S3 Glacier Deep Archive, Azure Blob Storage Archive

Each tier is priced differently based on the expected frequency of access and the storage duration, allowing users to optimize their storage costs according to their needs.

Cloud Storage Protocols and Interfaces#

When discussing cloud storage, it’s essential to understand the protocols and interfaces that facilitate communication between systems, dictating how data is stored, accessed, and transferred across networks. This overview presents these technologies chronologically, from the oldest to the newest, highlighting their significance in the evolution of storage solutions.

IDE (Integrated Drive Electronics)#

  • What It Is: An early standard for connecting storage devices to computers, utilizing a parallel interface for data transfer.
  • Why It Matters: Marked the beginning of direct integration of storage devices into computers, foundational in the evolution of storage connectivity.

SCSI (Small Computer System Interface)#

  • What It Is: A set of standards for connecting and transferring data between computers and peripheral devices.
  • Why It Matters: Introduced versatile connection options for various devices, evolving into modern interfaces like SAS but remaining foundational for storage protocols.

NFS (Network File System)#

  • What It Is: A protocol mainly used in Unix/Linux systems for accessing files over a network as if they were on the user’s own computer.
  • Why It Matters: Enabled seamless file sharing across different operating systems, becoming versatile for various storage scenarios.

CIFS/SMB (Common Internet File System/Server Message Block)#

  • What It Is: A protocol for network file sharing, allowing computers to access and transfer files over a network.
  • Why It Matters: Facilitated file sharing in Windows and mixed OS environments, with SMB continuing to be widely used despite CIFS being less common in modern contexts.

Fiber Channel#

  • What It Is: A high-speed network technology primarily used for connecting computer data storage, utilizing optical fiber.
  • Why It Matters: Provided high data transfer rates and reliability for SANs, suitable for mission-critical systems.

iSCSI (Internet Small Computer System Interface)#

  • What It Is: An IP-based storage networking standard for carrying SCSI commands over IP networks.
  • Why It Matters: Enabled the connection of storage arrays over long distances, ideal for disaster recovery and cloud storage.

SATA (Serial ATA)#

  • What It Is: A common interface for connecting storage devices like hard drives and SSDs to computers.
  • Why It Matters: Became widely used in consumer and some enterprise applications for its cost-effectiveness and improved performance over IDE.

SAS (Serial Attached SCSI)#

  • What It Is: An interface offering higher speeds and reliability than SATA, used in enterprise environments.
  • Why It Matters: Supports high-performance applications and is backwards compatible with SATA, offering storage solution flexibility.

FCoE (Fibre Channel over Ethernet)#

  • What It Is: A convergence of Fibre Channel and Ethernet technologies.
  • Why It Matters: Reduces the complexity and costs of Fibre Channel SANs by leveraging Ethernet infrastructure.

InfiniBand#

  • What It Is: A high-speed networking technology for high-performance computing and data centers.
  • Why It Matters: Offers high data transfer rates and low latency, suitable for demanding environments and data-intensive applications.

NVMe over Fabrics (NVMe-oF)#

  • What It Is: An extension of NVMe enabling high-speed storage access over network fabrics like Ethernet, Fiber Channel, and InfiniBand.
  • Why It Matters: Provides faster, more efficient access to storage over networks, enhancing performance for cloud and enterprise environments.

This chronological overview illustrates the technological advancements in storage protocols and interfaces, from the foundational IDE and SCSI to the modern, high-speed capabilities of NVMe over Fabrics. Each step represents an evolution in how data storage is integrated, accessed, and utilized across different platforms and networks, underpinning the functionality and performance of today’s cloud storage solutions.

RAID Configurations#

RAID (Redundant Array of Independent Disks) technology enhances storage arrays’ performance and provides fault tolerance and redundancy. Here’s a breakdown of the most popular RAID levels:

RAID 0: Striping#

  • How It Works: Data is divided and written across multiple disks, improving performance because operations can be carried out on multiple disks simultaneously.
  • Pros: Maximizes performance; uses the full capacity of all disks.
  • Cons: No fault tolerance—if one disk fails, all data in the array is lost.
  • Minimum Disks: 2

RAID 1: Mirroring#

  • How It Works: Data is duplicated across two disks, providing redundancy.
  • Pros: If one disk fails, the other can be used to recover data, ensuring high data availability.
  • Cons: Only half of the total disk capacity is usable because data is duplicated.
  • Minimum Disks: 2

RAID 5: Striping with Parity#

  • How It Works: Data and parity information are distributed across all disks. The parity information allows the reconstruction of data in case of a disk failure.
  • Pros: Good balance between performance, storage efficiency, and fault tolerance. Can survive the failure of one disk.
  • Cons: Slightly reduced storage capacity (equivalent to the capacity of one disk is used for parity); write operations can be slower due to the need to calculate parity.
  • Minimum Disks: 3

RAID 6: Striping with Double Parity#

  • How It Works: Similar to RAID 5 but uses two sets of parity data, providing additional fault tolerance.
  • Pros: Can survive the failure of two disks, making it more fault-tolerant than RAID 5.
  • Cons: Higher storage capacity loss (equivalent to the capacity of two disks is used for parity) and slower write performance due to double parity calculation.
  • Minimum Disks: 4

RAID 10 (1+0): Mirroring and Striping#

  • How It Works: Combines the features of RAID 1 and RAID 0 by mirroring a set of striped disks.
  • Pros: High performance and fault tolerance; can survive multiple disk failures as long as no two failed disks are from the same mirrored pair.
  • Cons: High cost due to reduced usable capacity—only 50% of total disk capacity is usable.
  • Minimum Disks: 4 (even number)

Each RAID level offers a different balance of performance, storage efficiency, and fault tolerance, catering to various storage needs and scenarios.

Another Storage Features#

Understanding storage features is crucial for optimizing cloud storage and meeting business requirements. Here’s a breakdown of key storage features:

Compression#

  • What It Is: Reduces the size of your data to save storage space.
  • Benefits: Saves costs by reducing the amount of storage needed.

Deduplication#

  • What It Is: Eliminates redundant data blocks to optimize storage usage.
  • Benefits: Further reduces storage requirements and costs by ensuring only unique data is stored.

Thin Provisioning vs. Thick Provisioning#

  • Thin Provisioning: Allocates storage space dynamically as data is added, optimizing storage utilization.
  • Thick Provisioning: Allocates all the designated storage space upfront, ensuring immediate availability but potentially wasting space.

User Quotas#

  • What It Is: Limits the amount of storage an individual user or department can use.
  • Benefits: Helps manage and allocate storage resources effectively across an organization.

Hyperconverged Infrastructure (HCI)#

  • What It Is: Combines compute, storage, and networking into a single system to simplify management and increase efficiency.
  • Benefits: Simplifies deployment and management of storage and other resources, making it easier to scale and maintain.

Software-Defined Storage (SDS)#

  • What It Is: Storage management and provisioning are controlled by software rather than traditional hardware.
  • Benefits: Offers flexibility, scalability, and efficiency by abstracting storage management from the hardware.

Replication#

  • Types:
    • Local Replication: Copies data within the same system or network for redundancy.
    • Remote Replication: Copies data to a remote location for disaster recovery purposes.
  • Synchronous vs. Asynchronous:
    • Synchronous Replication: Ensures the remote copy is always identical to the source, suitable for critical data.
    • Asynchronous Replication: Copies data with a delay, suitable for less critical data where a slight lag is acceptable.

Each of these features plays a role in optimizing storage solutions, balancing costs, performance, and availability to meet specific business needs.

Backward Cloud Availability Cloud Compute Forward