PiscesL1: A Multimodal MoE Large Model That Runs on a Single RTX 4090 GPU

PiscesL1 is the first version of the PiscesLx series developed by the Dunimd team. It uses the Yv architecture and supports understanding of 6 modalities: text, images, audio, video, documents, and agents. Through the Mixture of Experts (MoE) architecture and hardware optimizations, it achieves local operation on a single RTX 4090 GPU. Additionally, the model is open-source, providing researchers and developers with a low-cost opportunity to explore multimodal AI and agent technologies.

In the field of large language models, there is often a conflict between multimodal capabilities and hardware accessibility: top-tier multimodal models require expensive multi-card clusters to run, while models that can run on consumer-grade hardware often compromise on capabilities. The PiscesL1 project aims to balance these two goals by providing a high-performance multimodal MoE model that can run on consumer-grade GPUs.

• Mixture of Experts (MoE) Architecture: Improves parameter efficiency through sparse activation, specialized experts, and dynamic routing;
• Multimodal Fusion: Supports processing of 6 modalities via a unified representation space, cross-modal attention, and modality-specific encoders;
• Agent Capability Integration: Supports tool usage, task planning, autonomous execution, and environmental awareness.

• Quantization techniques (4-bit/8-bit) to reduce memory usage;
• Efficient attention mechanisms like FlashAttention to lower memory overhead;
• Dynamic expert loading to activate only the required parameters;
• Gradient checkpointing and activation recomputation to trade computation for memory.

• Content Understanding and Generation: Multimedia analysis, intelligent document processing, cross-modal retrieval;
• Intelligent Assistants and Automation: Personal AI assistants, customer service automation, content moderation;
• Research and Development: Multimodal AI research, agent exploration, intelligent educational tutoring systems.

• Performance trade-offs: Performance on some tasks may be inferior to large-scale cloud models;
• Limited context length;
• Uneven processing quality across different modalities;
• Safety issues need to be emphasized for local agent systems.

Feature	PiscesL1	GPT-4V	Gemini Pro	Qwen-VL
Local Operation	✅	❌	❌	✅
Single RTX 4090	✅	N/A	N/A	Partial
Open-source Weights	✅	❌	❌	✅
Multimodal	6 types	3 types	4 types	3 types
MoE Architecture	✅	Unknown	Unknown	❌
Agent Capability	✅	Limited	Limited	Limited

PiscesL1 is an important milestone in the localization of multimodal AI. Through the Yv architecture and optimizations, it enables high-end consumer-grade GPUs to run multimodal MoE models. Future outlook: Larger-scale models, more perceptual modalities, stronger agent capabilities, better efficiency, and a more complete tool ecosystem.