Complete Solution for NVIDIA Nemotron Inference Challenge: Practical LoRA Fine-tuning of 30B MoE Model

This article introduces a complete pipeline project for the Kaggle competition, demonstrating how to perform LoRA fine-tuning on the NVIDIA Nemotron-3-Nano-30B-A3B-BF16 large model in a resource-constrained environment to solve complex logical reasoning puzzles. The project covers the entire workflow from data exploration, chain-of-thought generation, LoRA training, evaluation to packaging and submission.

NVIDIA hosted the Nemotron Model Inference Challenge on the Kaggle platform, requiring participants to train a LoRA adapter (with rank no more than 32) based on the Nemotron-3-Nano-30B-A3B-BF16 model to achieve the highest accuracy on logical reasoning puzzles. This is a typical resource-constrained scenario— the 30B-parameter MoE (Mixture of Experts) model still requires about 15GB of VRAM even with 4-bit quantization, posing a challenge to single-GPU environments.

This open-source project provides a complete competition pipeline, covering the entire lifecycle of modern large model fine-tuning from data preparation to final submission. The project uses a modular design, splitting the workflow into five stages: Exploratory Data Analysis (EDA), data preparation, LoRA Supervised Fine-tuning (SFT), evaluation, and packaging & submission.

Nemotron-3-Nano-30B-A3B-BF16 is a 30-billion-parameter MoE model using BF16 precision. The project uses 4-bit quantization combined with LoRA (Low-Rank Adaptation) technology to limit trainable parameters to the adapter layers, significantly reducing VRAM requirements. The default configuration uses LoRA with rank 16 and can run on a dual T4 GPU environment.

The second stage of the project focuses on chain-of-thought generation. By calling the Anthropic API (or other configured APIs), detailed reasoning steps arere generated for training data. This "slow thinking" data is crucial for improving the model's performance on logical puzzles. The generated CoT data is formatted and converted into the JSONL format required for SFT.

To address weak performance on specific puzzle types, the project supports synthetic data generation. Users can generate additional training samples for specific categories— this data-driven improvement strategy is particularly effective in competition scenarios. Synthetic data is mixed with real data to enhance the model's generalization ability.

The training script supports a GRPO (Generalized Reward Policy Optimization) reinforcement learning stage following the SFT baseline training. This two-stage strategy first allows the model to master basic formats and reasoning patterns, then optimizes specific reward signals through reinforcement learning— an effective method to improve competition performance.

The project natively supports three runtime environments: Kaggle, Anaconda Cloud, and local. The Kaggle notebook is optimized for dual T4 GPU environments and addresses common dependency conflict issues, such as compatibility between mamba_ssm and torch versions, torchvision matching, etc.