Hrothgar: Implementation of Multimodal Few-Shot Font Generation Based on Global-Aware Autoregressive Model

Hrothgar is an independent implementation project of the GAR-Font paper initiated by Simon Cozens, supporting multimodal few-shot font generation. Through the GTok tokenizer, AR generator, and multimodal adapter, it generates complete high-quality fonts from a small number of reference glyphs. This project aims to verify the reproducibility of the paper's method, provide open-source tools for the font generation community, and has both academic and engineering value.

Font generation is a classic challenge in the intersection of computer vision and graphics. Traditional design requires extensive manual work to draw each glyph. Few-shot font generation technology learns styles from a small number of reference glyphs to generate missing characters, which is of great significance for scenarios like low-resource language font development and historical font digitization. GAR-Font is a research result of the global-aware autoregressive model published in 2025, and Hrothgar, as its independent implementation, aims to verify the method's feasibility and provide open-source tools.

Hrothgar implements the three-stage architecture of GAR-Font:

G-Tok Tokenizer

Hybrid CNN-ViT architecture: The CNN encoder (modified based on LlamaGen) processes local features, the 6-layer ViT encoder extracts global features, the 6-layer causal ViT decoder reconstructs the image, and a 2048-entry codebook (dimension 8) generates 64 tokens from a 64×64 image.

AR Generator

The core is a 24-layer Transformer decoder (314M parameters), including a content encoder (28.56M parameter CNN), a style encoder (2.78M parameter lightweight CNN), and a 3-layer cross-attention aggregator (0.79M parameters) to fuse content and style.

Multimodal Adapter

Supports text guidance: Freeze the Flan-T5 encoder to encode text, use a 6-layer cross-attention adapter (4.74M parameters) to align text and visual features, a projection layer (0.52M parameters) to map the feature space, and L2 alignment loss to ensure consistency.

Global-Aware Generation

Unlike traditional local patch methods, it uses global-aware autoregressive modeling. When generating each token, it accesses the complete context, improving glyph coherence and style consistency.

Multimodal Condition Injection

Supports three conditional inputs: content condition (target character structure skeleton), style condition (reference glyph visual style), and text condition (natural language description), flexibly adapting to various scenarios.

Neural Font Adaptation (NFA)

Uses LoRA technology to add low-rank adaptation layers to the Transformer decoder. Fine-tune with 128 reference glyphs for 10 epochs at a learning rate of 2e-5 (AdamW optimizer).

Style Enhancement (SE)

Reinforcement learning via the GRPO algorithm: OCR reward ensures readability, style reward ensures consistency with references, and each group of 4 samples is trained for 10 epochs.

Hrothgar is suitable for:

• Low-resource language font development: Designers only need to design a subset of commonly used characters; the system automatically generates the remaining characters to reduce costs;
• Historical font digitization: Extract a small number of reference glyphs from ancient books/steles to generate complete digital fonts, aiding cultural heritage protection;
• Font style transfer: Transfer the style of an existing font to a new character set to quickly create multilingual font families;
• Font variant generation: Generate variants like bold and italic based on the base font, maintaining design consistency.

Inference of Implementation Difficulties

Some details in the paper are not publicly available; the team made reasonable inferences:

Component	Inference Strategy
CNN architecture details	Based on the open-source LlamaGen tokenizer
ViT hidden dimension	Inferred from parameter count (approx. 384 dimensions)
Transformer configuration	314M/24 layers ≈13.1M per layer, matching GPT-2 Medium scale
Loss weights	Use VQ-GAN standard values as the starting point

Evaluation Metrics

Uses the paper's multi-dimensional system: RMSE (pixel reconstruction error), SSIM (structural similarity), LPIPS (perceptual similarity), FID (distribution similarity), content accuracy (character recognition rate), style accuracy (style classification rate).

The value of Hrothgar:

1. Reproducibility verification: Verify the feasibility of the GAR-Font method, providing a reference implementation for subsequent research;
2. Open-source contribution: Provide usable tools for the font generation community;
3. Method improvement: Independent implementation may discover optimization spaces not covered in the paper;
4. Application落地: Lower the technical threshold for use, promoting practical applications. It is expected to become an important open-source tool in the font generation field in the future, driving the popularization and development of AI-assisted font design.