ZoneTier-LLM: A New Hierarchical Flash Storage Management Scheme for Edge LLM Inference

ZoneTier-LLM is a two-tier zoned flash storage management prototype based on ConZone+ designed specifically for edge LLM inference. It addresses the challenges of limited resources on edge devices and the unique I/O characteristics of LLM inference (sequential read-only for weights, random read/write for KV cache) through strategies like media-aware data placement, heat-driven migration, and hybrid I/O scheduling, achieving storage optimization, improving inference performance, reducing hardware costs, and extending device lifespan.

As LLMs penetrate edge devices, the limited memory, computing power, and storage bandwidth of edge devices make efficient management of model weights and KV cache a key challenge. Traditional storage solutions assume sufficient DRAM/high-speed SSDs, which is not valid in edge scenarios. Additionally, the conflicting I/O characteristics of LLM inference—model weights are large-capacity sequential read-only access, while KV cache is dynamic random read/write access—place higher demands on storage systems.

Core Concepts

ZoneTier-LLM leverages the characteristics of zoned flash memory (e.g., ZNS SSDs) and uses intelligent data tiering and scheduling strategies to maximize inference performance under limited resources. Zoned flash divides storage into independent zones with sequential-only writes, simplifying management but requiring optimized data placement.

Key Technologies

1. Media-aware Data Placement: Dynamically adjust data storage tiers based on access characteristics and lifecycle; place active KV cache in high-speed zones and migrate cold data to low-speed zones.
2. Heat-driven Migration: Track data access heat via heat maps; promote hot data to faster tiers and migrate cold data to cost-effective tiers, adapting to the hot/cold changes of KV cache during LLM long-sequence generation.
3. Hybrid I/O Scheduling: Tiered scheduling of weight access (large-block sequential reads) and KV cache access (small-grained random reads/writes); reserve sequential bandwidth and optimize latency to ensure inference real-time performance is not affected by migration.

ZoneTier-LLM is built on the ConZone+ storage management layer, which provides:

• Zone abstraction: Encapsulates zoned flash details into a unified data zone interface
• Lifecycle management: Tracks the creation, activity, and recycling of data zones
• Concurrency control: Manages concurrent access to multiple zones to avoid conflicts
• Metadata management: Maintains metadata such as zone mapping and heat statistics

ZoneTier-LLM adds an LLM-aware optimization layer on top of this, transforming general zoned flash management into a dedicated solution for LLM inference.

1. Reduce Hardware Costs: Intelligent tiering allows low-cost storage media (e.g., QLC flash) to be used for most weights, with only active KV cache using TLC/DRAM.
2. Extend Device Lifespan: Sequential writes in zoned flash reduce write amplification, and heat migration balances wear, making it suitable for edge devices running long-term.
3. Improve Response Speed: Hot data is located in fast storage tiers, reducing inference latency in resource-constrained environments and adapting to real-time interactive applications (e.g., voice assistants, real-time translation).

Limitations

The current prototype mainly focuses on data placement and migration strategies, and lacks sufficient support for complex scenarios such as multi-model concurrency and dynamic model switching.

Future Directions

1. Introduce machine learning to predict access patterns and optimize data placement decisions
2. Support isolation and scheduling of multiple LLM instances on shared edge devices
3. Integrate zoned flash with heterogeneous storage media such as traditional block devices and persistent memory

ZoneTier-LLM is a valuable exploration of edge AI infrastructure, showing that LLM optimization requires not only model architecture and algorithm innovation but also underlying storage system optimization to significantly improve performance. As edge computing becomes increasingly important today, such scenario-specific deep optimizations will be more valuable. For edge AI deployment engineers, drawing on the idea of hierarchical storage management can help design more cost-effective system solutions.