Kolmogorov-Arnold Networks: A New Interpretable Neural Network Architecture Based on TensorFlow

This article introduces the emerging neural network architecture KAN, which replaces the fixed node activation functions of traditional MLPs with learnable edge activation functions. This project provides a clear TensorFlow-based implementation, focusing on interpretability and educational value to help understand the internal mechanism of this architecture.

Multilayer Perceptrons (MLPs) are fundamental building blocks of deep learning, but their fixed node activation functions limit expressive power and are difficult to interpret. Proposed in 2024, KAN is inspired by the Kolmogorov-Arnold Representation Theorem, which states that any multivariate continuous function can be expressed as a finite combination of univariate continuous functions.

Edge Activation vs. Node Activation

In traditional MLPs, nodes perform weighted summation followed by fixed nonlinear activation, while edges only transmit linear signals. In KAN, nodes only perform simple summation, and edges contain learnable activation functions (often parameterized by B-splines), with each edge able to learn different patterns.

Advantages of B-spline Parameterization

1. Local Support: Control points only affect local regions, facilitating fine adjustment
2. Smoothness: High-order B-splines have good smoothness properties
3. Interpretability: Learned activation functions can be directly visualized and analyzed

The implementation of this project focuses on educational value, with the core component being the KANLinear layer:

• Key Parameters: Input/output feature dimensions, grid size, spline order, regularization coefficient, etc.
• Core Methods: Initialize B-spline grid, compute combined output of linear and spline transformations, etc. B-spline basis functions are calculated using the Cox-de Boor recursive formula, supporting boundary extension to handle cases where input exceeds the range. The KAN class stacks multiple KANLinear layers to form a complete network.

Main Advantages

1. Better fitting accuracy than MLP with the same number of parameters
2. Strong interpretability (activation functions can be visualized)
3. Suitable for modeling low-dimensional complex functions
4. Can learn both combinatorial structures and univariate functions simultaneously

Current Limitations

1. Slower training speed than MLP
2. Lack of large-scale training and GPU acceleration optimization
3. Performance on high-dimensional data needs more verification

Scientific Computing and Physical Modeling

Symbolic regression, physical law learning, partial differential equation solving

Medical and High-Risk Applications

Medical diagnosis (meeting regulatory requirements), financial risk control, autonomous driving

Few-Shot Learning

Structured representation may make more effective use of limited data

Recommended Learning Path:

1. Understand the design principles of KAN and its differences from MLP
2. Study the implementation details of the KANLinear layer
3. Try different grid sizes and spline orders
4. Visualize the learned activation functions
5. Compare performance and interpretability with MLP

KAN represents an important exploration direction in neural network architectures, improving interpretability through edge activation functions while maintaining expressive power. Although there are challenges in training efficiency, it has great potential in fields such as scientific computing and medical AI in the future. The TensorFlow implementation of this project provides a good starting point for learning and research, with clear code and detailed documentation.