In the cross-disciplinary field of systems and AI, there are many exciting cutting-edge topics. For learners interested in this direction, here are some worth paying attention to:
Machine Learning Systems (ML Systems) is a rapidly developing field. Research focuses on how to efficiently train and deploy large-scale machine learning models, involving distributed training, model parallelism, pipeline optimization, memory management, compilation optimization, etc. The underlying optimization of frameworks like TensorFlow and PyTorch, as well as programming for dedicated AI accelerators (such as TPU and GPU), are important topics in this field.
Neural network compilers are the bridge between AI algorithms and hardware. Research focuses on how to automatically compile high-level neural network descriptions into efficient hardware code, involving graph optimization, operator fusion, memory scheduling, code generation, etc. Projects like Apache TVM and MLIR represent the cutting edge of this direction.
Edge AI focuses on running AI models on resource-constrained devices. This involves model compression (pruning, quantization, knowledge distillation), efficient inference engines, and dedicated hardware design. With the popularity of the Internet of Things and mobile devices, the importance of edge AI is increasingly prominent.
AI-driven system optimization uses machine learning to improve traditional systems. For example, using reinforcement learning to optimize database query plans, using neural networks to predict system load, using generative models to synthesize test data, etc. This AI for Systems approach is opening up new research directions.