CAW-Conv: Forward Convolutional Neural Network Based on Learnable Channel-Class Assignment

Core Idea

CAW-Conv is a biologically inspired forward-only convolutional learning method that replaces backpropagation with local forward objectives. It uses learnable channel-class assignment, entropy regularization, and orthogonal regularization to train deep residual networks (ResNet-17), outperforming previous forward learning methods on multiple benchmarks.

Source Details

• Authors: Mohammadnavid Ghader, Saeed Reza Kheradpisheh, Bahar Farahani, Mahmood Fazlali
• Paper Link: https://arxiv.org/abs/2606.09928
• GitHub Repo: https://github.com/mngh-cs/CAW-Conv
• Release Time: 2026-06-11

Backpropagation's Biological Controversy

Deep learning's core training mechanism (backpropagation) requires global gradient information to flow backward through the network, which contradicts the brain's local learning mechanism (neurons use only local signals).

Rise of Forward-Forward (FF) Algorithm

FF is a bio-inspired alternative that uses local forward learning objectives—each layer optimizes independently without error backpropagation. This offers potential gains in computational efficiency and memory usage.

Challenges for FF in CNNs

Traditional FF convolutional methods use static channel grouping, which fails to adapt to dynamic category-specific feature needs.

1. Learnable Channel-Class Assignment

Dynamic learning of each convolution channel's contribution to different categories, enabling adaptive feature specialization and efficient network capacity use.

2. Entropy Regularization

Prevents channel assignment concentration (avoiding a few channels being shared by all categories) to ensure uniform channel utilization.

3. Orthogonal Regularization

Encourages complementary feature representations between channels, reducing redundancy and improving discriminative power.

4. Loss-Aware Layer Contribution

Evaluates each layer's impact on final predictions to adjust learning strategies for specific categories.

5. Fully Local Layer Optimization

Each layer optimizes independently using local information, saving memory (no intermediate activation storage) and enabling parallel training.

6. Deep Residual Forward CNN Training

Successfully trains ResNet-17 (a deep network), overcoming previous FF limitations in deep model training.

Standard Datasets

Method	Architecture	CIFAR-10	MNIST	Fashion-MNIST
FF	MLP	59.00	98.69	-
SymBa	MLP	59.09	98.58	-
CaFo	CNN	67.43	98.80	-
CwComp	CNN	78.11	99.42	92.31
DeeperForward	CNN	86.22	99.63	93.13
CAW-Conv	ResNet-17	89.37	99.74	94.55

Challenging Datasets

Method	CIFAR-100	Tiny-ImageNet
DeeperForward	53.09	41.36
DeeperForward (CH×3)	60.28	-
CAW-Conv	63.52	49.87
CAW-Conv (CH×3)	69.74	-

Key Observations

• CAW-Conv outperforms all previous forward learning methods on standard datasets.
• It shows strong scalability: increasing channel count by 3x boosts CIFAR-100 accuracy to 69.74%.

Channel Weight Learning Mechanism

Each convolution layer maintains a learnable weight matrix of shape [channel count, category count] to determine channel contributions to categories during forward propagation.

Local Loss Function

Layers use a combination of:

• Classification loss (accuracy of current layer's feature predictions)
• Entropy loss (uniform channel assignment)
• Orthogonal loss (complementary feature representations)

Residual Connection Handling

Ensures compatibility between residual and main path features while preserving local learning properties.

Bio-Inspired Learning Progress

Demonstrates how local learning can achieve performance comparable to backpropagation, advancing understanding of brain-like AI.

Computational Efficiency

No need to store intermediate activations for backpropagation, reducing memory usage and enabling faster training.

Hardware Friendliness

Local learning allows parallel layer training, making it suitable for specialized AI chip design.

Interpretability

Explicit channel-class correspondence helps analyze which features the network uses for category recognition.

Current Limitations

• Accuracy gap compared to state-of-the-art backpropagation methods.
• Longer training iterations required for convergence.
• High sensitivity to hyperparameter tuning (e.g., regularization coefficients).

Future Research

1. Extend to larger models (ResNet-50/101).
2. Apply to Transformer/Vision Transformer architectures.
3. Design hybrid forward-backpropagation training strategies.
4. Conduct theoretical analysis of channel-class assignment effectiveness.
5. Validate on resource-constrained edge devices.

Reproduction Steps

1. Clone the GitHub repo and install dependencies.
2. Prepare datasets (CIFAR-10/100, MNIST, Fashion-MNIST, Tiny-ImageNet).
3. Run training scripts with adjusted hyperparameters (learning rate, regularization coefficients).
4. Use evaluation scripts to test model performance.

Tips for Custom Use

• Adjust network architecture to fit input size.
• Modify channel count and layer depth based on task complexity.
• Tune entropy and orthogonal regularization weights carefully.