NASiC: A New 3D NAND In-Memory Computing Architecture for Edge Inference of MoE Large Models

NASiC is a 3D NAND in-memory computing architecture specifically designed for Mixture-of-Experts (MoE) models. By integrating CAM content addressing and CIM computing, it completes expert selection and computation in a single cycle, solving the memory wall problem in edge deployment of MoE models. This architecture delivers a 4-114.8x performance improvement and a 3.9-70x energy efficiency boost, providing an innovative solution for edge large model inference.

Mixture-of-Experts (MoE) is a mainstream paradigm for Large Language Models (LLMs). It achieves parameter scale expansion while controlling inference costs through a sparse activation mechanism. However, edge deployment of MoE faces the memory wall challenge: the complete parameter set needs to reside in memory, and traditional HBM+ off-chip data transfer solutions bring high energy consumption and latency bottlenecks, limiting their application in resource-constrained devices.

3D NAND provides a unique hardware foundation for in-memory computing: high storage capacity (hundreds of GB/TB level), low data movement overhead, and multi-bit storage capability. However, applying it to MoE presents challenges: dynamic sparse activation leads to reduced computational parallelism, low multi-bit storage utilization, and additional overhead for expert selection. Existing solutions struggle to unleash the potential of MoE.

The core innovation of NASiC is the deep integration of CAM and CIM: using the 3D NAND string structure to support CAM (expert selection) and CIM (multiply-accumulate computation) modes, completing selection and computation in a single cycle. Circuit optimizations include block-level parallel computing (processing multiple experts simultaneously), in-situ signed multi-bit expansion (no additional conversion overhead), and dynamic voltage regulation (optimizing energy consumption on demand).

Performance evaluation shows that compared to advanced in-memory computing designs, NASiC achieves: 4-114.8x performance improvement (integration eliminates control overhead, block parallelism leverages density, CAM reduces routing latency); 3.9-70x energy efficiency improvement (reduces data movement, dynamic voltage regulation, efficient multi-bit utilization); and controllable inference accuracy loss compared to the floating-point baseline.

Significance and prospects of NASiC: Promote edge deployment of MoE models with tens of billions of parameters; provide an algorithm-architecture-circuit co-design paradigm; expand the application direction of 3D NAND in AI. Challenges include process compatibility, programming interface development, reliability assurance, and multi-task support.

NASiC represents an important breakthrough in in-memory computing design for MoE edge inference, integrating CAM and CIM to achieve order-of-magnitude improvements in performance and energy efficiency. We look forward to this architecture moving from paper to product, enabling widespread application of edge large models and pointing the way for the future development of in-memory computing.