Dell and NVIDIA advance KV Cache with BlueField-4 CMS

Dell Technologies announced a collaboration with NVIDIA to advance Key-Value (KV) Cache offloading using the NVIDIA BlueField-4 DPU and Dell’s AI storage portfolio.

• Main announcement/action: Dell and NVIDIA are promoting a Context Memory Storage Platform (CMS) that leverages NVIDIA BlueField-4 to offload KV Cache from GPU HBM, aiming to reduce recomputation and latency and to optimize GPU utilization. Key components include Dell PowerScale, Dell ObjectScale, and Project Lightning (private preview). The article cites claimed performance gains of 19x improvement in TTFT (Time to First Token) and up to 5.3x improvement in queries per second for Dell’s KV Cache offload solutions (linked references provided).
• Background and implementation details: Dell describes a software stack integrating LMCache and NVIDIA NIXL to enable KV Cache offload over RDMA, supporting NFS-over-RDMA and S3-over-RDMA for low-latency access and NVMe-over-Fabrics for Project Lightning. Author: Rajesh Rajaraman (technology strategy and architecture lead, 25+ years experience; prior roles at NetApp, Cohesity, DEC).