Kaggle SVG Competition Solution: SVG Generation Pipeline Fine-Tuned with Qwen2.5-Coder

This project is an entry for the Kaggle SVG Competition, showing how to fine-tune Alibaba Cloud's Qwen2.5-Coder-1.5B model to generate valid complex SVG graphics from natural language descriptions. It includes a complete workflow of data processing, model fine-tuning, inference generation, and post-processing validation, providing a practical reference for domain specialization of code generation models.

With the improvement of large language models' capabilities in code generation tasks, the demand for domain specialization (e.g., SVG generation) has increased. SVG generation requires understanding natural language plus mastering SVG syntax and graphics concepts. This project selects Qwen2.5-Coder-1.5B (a lightweight code model optimized with pre-trained code data). The tech stack includes: base model Qwen2.5-Coder-1.5B, fine-tuning frameworks Hugging Face Transformers + PEFT, data processing tools, inference engine, and quality validation tools.

Data Preparation: Combine public SVG datasets + synthetic data; generate multi-level descriptions for each sample (high-level visual concepts, mid-level element layout, low-level attribute parameters); data cleaning includes syntax validation, rendering validation, and complexity control.

Fine-Tuning Strategy: Adopt LoRA (Parameter-Efficient Fine-Tuning) with rank 16-32, target attention layer projection matrices, and scaling coefficients; training strategy uses cosine annealing learning rate (1e-4 ~5e-5), gradient accumulation, and early stopping mechanism; loss function weights SVG tags and key attributes.

Inference Workflow: Input preprocessing → model autoregressive SVG generation (using temperature sampling 0.7-0.8, Top-p sampling, repetition penalty) → post-processing validation.

Validation and Repair: Syntax check (XML parsing), completeness check (root element existence), rendering test; automatically repair or regenerate failed results.

Technical Highlights: SVG tokenization optimization (improves numerical representation efficiency), progressive generation (step-by-step refinement of complex graphics), multimodal feedback (visual evaluation to improve generation), controllable generation interface (style/color/complexity control).

Application Scenarios: Rapid prototyping, icon generation, data visualization assistance, educational tools, accessibility design.

Limitations: The quality of complex graphic generation needs improvement, style consistency is insufficient, semantic accuracy needs optimization, and computational efficiency needs to be enhanced.

Improvement Directions: Introduce diffusion models to optimize graphics, combine retrieval-augmented generation, and develop interactive editing interfaces.

Domain Insights: Domain data quality and diversity are key; validation-driven training can leverage SVG verifiability; lightweight models can be comparable to large models after fine-tuning.

Conclusion: This project achieves reliable conversion from natural language to SVG, providing tools for SVG generation and also offering methodological references for domain-specialized code generation.