Differentiable Image Stylization Engine Based on CNN and CDF: From Principles to Practice

This project proposes an image stylization engine that combines global CDF analysis with a CNN-driven differentiable renderer, supporting end-to-end learning of multiple photographic styles such as Fujifilm classic film, cyberpunk, and tilt-shift effect. The engine balances global tone statistics (captured by CDF) and local spatial structure (encoded by CNN), enabling powerful and interpretable image editing capabilities. Maintained by kyleyhw, the source code is available on GitHub (link: https://github.com/kyleyhw/image_editing) and was released on May 26, 2026.

Digital image stylization is a classic problem in computer vision. However, traditional rule-based methods struggle to capture complex style details, while pure neural network methods (e.g., style transfer) lack interpretability and controllability. The core insight of this project is that image style includes global tone statistical features (capturable via CDF) and local spatial structure information (encodable via CNN). Thus, a hybrid architecture is designed to address the above issues.

System Architecture: Modular design, including a feature extractor (differentiable CDF module + ResNet-18 spatial encoder), transformation heads (three types: Fujifilm film dedicated/general renderer/tilt-shift composite), and a differentiable renderer. Core Innovations: 1. Differentiable CDF: Gaussian soft binning is used to achieve differentiable CDF calculation; 2. Identity initialization: The renderer maintains an identity mapping when parameters are zero, ensuring stable optimization; 3. Composite loss: Pixel-level L1 + perceptual loss (VGG-16 multi-layer features) + CDF matching loss; 4. General renderer primitives: Tone curve, color matrix, grain, vignetting; 5. Tilt-shift effect parameterization: Three scalar parameters (center position c_y, bandwidth w, blur intensity σ_s) are used to implement spatially variable blur.

Training Data: The MIT-Adobe FiveK dataset (professional retouching pairs) and Picsum random images are used; the style generator produces training data for Fujifilm classic film (blue-shifted white balance, soft highlights, etc.), cyberpunk (cyan-orange S-curve, etc.), and tilt-shift (horizontal focus band). Engineering Highlights: The toolchain includes uv (package management), pre-commit (code quality), and Streamlit (interactive UI); the project structure is clear (data_generation, models, docs, etc.); a complete verification report (qualitative results, quantitative analysis, etc.) is provided.

Current Capabilities: Three styles have been verified: Fujifilm classic film (warm retro), cyberpunk (high-contrast cyan-orange), and tilt-shift (miniature model effect). Expansion Path: Adding new styles only requires implementing the StyleGenerator class; supports expansion of spatially variable effects; can be fine-tuned for high resolution; inference speed supports real-time preview, making it suitable for mobile/Web integration.

Current Limitations: The tilt-shift blur amount is half of the generator's (to optimize artifacts); spatial effects are limited to horizontal focus bands; high-resolution training requires more resources. Future Directions: Explore U-Net decoders to support per-pixel parameter maps; introduce adversarial training to improve quality; support video stylization (temporal consistency); implement user-controllable interactive editing.

This project is an excellent practice of fusing deep learning and traditional image processing, building a powerful and interpretable engine through differentiable CDF, CNN encoding, and end-to-end rendering. Its rigorous derivation, complete implementation, and detailed documentation provide valuable references for researchers (learning differentiable rendering) and developers (stylization solutions). The project emphasizes reproducibility (locked dependencies, verification report), reflecting modern machine learning engineering best practices.