Edge-STGNN: An Automated Spatio-Temporal Graph Neural Network Search Framework for Edge Devices

This article introduces Edge-STGNN, an automated spatio-temporal graph neural network search framework for edge devices, specifically designed for skeleton-based human action recognition tasks. The framework balances model accuracy and computational efficiency using Neural Architecture Search (NAS) technology, addressing the problem that existing STGNNs are difficult to run in real time on edge devices due to resource constraints. Its core innovations include a targeted spatio-temporal graph search space, hardware-aware multi-objective optimization, and an efficient search strategy.

Skeleton-based human action recognition is an important direction in computer vision, applied in scenarios such as intelligent surveillance and human-computer interaction. Skeleton data has advantages like light insensitivity, low computational cost, and privacy-friendliness. However, edge devices have limited resources (computation, memory, battery), and existing STGNNs are designed to be complex for accuracy, making them hard to run in real time; traditional NAS has high computational costs and ignores the characteristics of spatio-temporal graph networks.

1. Compact and flexible spatio-temporal graph search space: Covers multiple variants of spatio-temporal graph convolution operations, balancing search efficiency and architectural possibilities; 2. Hardware-aware multi-objective optimization: Considers both accuracy and computational efficiency, with a built-in edge device latency predictor to quickly evaluate candidate architectures; 3. Efficient one-shot search strategy: Based on weight-sharing supernetwork training, enabling subnetwork performance evaluation without retraining.

Processes human skeleton sequence data (extracted by key point detectors), supporting joint coordinates, bone vectors, and combined inputs; The architecture is stacked by spatio-temporal graph convolution layers, each containing spatial graph convolution (aggregating joint relationships) and temporal convolution (capturing dynamics), with hyperparameters determined automatically; Edge optimizations include depthwise separable convolution, lightweight activation functions, and compact feature dimensions.

Evaluated on datasets such as NTU RGB+D and Kinetics-Skeleton, it achieves an excellent balance between accuracy and efficiency: higher accuracy under the same computational budget, or significantly reduced overhead with similar accuracy; Edge deployment demonstrates real-time performance. Ablation experiments verify the effectiveness of the search space and multi-objective optimization, and hardware-aware search can find architectures more suitable for edge devices.

It is of great significance to edge AI: proving the feasibility of NAS in spatio-temporal graph networks and providing an automated design tool; The hardware-aware paradigm can be extended to other edge tasks. In practical applications, it helps deploy high-performance action recognition systems on smartphones, smart cameras, etc., serving fields like intelligent security and sports analysis, allowing users to enjoy real-time services while protecting privacy.

The code has been open-sourced on GitHub, including search algorithms, pre-trained models, evaluation scripts, and documentation, facilitating reproduction and research. Open source promotes academic exchanges in the fields of edge AI and action recognition, allowing researchers to explore new strategies, expand tasks, or conduct practical deployments based on it.

Edge-STGNN is an important advancement of NAS in the field of spatio-temporal graph networks, achieving efficient and accurate recognition on edge devices through targeted design. In the future, with the development of edge computing and NAS, we expect more automated tools to lower deployment barriers and promote the popularization and innovation of intelligent applications.