TSFMx: An Open-Source Framework Infusing Multimodal Capabilities into Time Series Foundation Models

TSFMx is an innovative open-source framework developed by himura467. It extends the capabilities of time series foundation models like TimesFM and Chronos by integrating multimodal exogenous features such as text. It addresses the problem that existing foundation models rely solely on numerical sequences and cannot effectively utilize auxiliary information like news texts and policy announcements, opening up new possibilities in the field of time series forecasting.

Traditional time series forecasting assumes that historical data contains all necessary information, but in real-world scenarios (e.g., stock prices, energy demand, retail sales), text information (such as financial reports, policy changes, and promotion descriptions) is often more critical. Existing methods require training specialized models from scratch or complex modifications to pre-trained models, which have high engineering costs and are difficult to reuse.

The core architecture of TSFMx includes: 1. Text encoder layer (supports multiple types; connects to pre-trained language models for English scenarios); 2. Multimodal fusion mechanism (dynamically integrates text and time series features via attention-based weighting, not simple concatenation); 3. Model adaptation layer (lightweight adaptation for TimesFM/Chronos without modifying the core architecture, plug-and-play).

TSFMx provides an example workflow for the Time-MMD dataset (nine domains, numerical + text): automatic preprocessing (6:2:2 split), cache-accelerated training, and W&B hyperparameter search. Comparative experiments show that the multimodal extension outperforms pure time series models, following the practical paradigm of "enhancement rather than replacement".

TSFMx uses the MIT open-source license, with code hosted on GitHub. Dependency management supports pip installation and uv execution to ensure environment consistency. The documentation covers the complete workflow, with clear YAML configurations, and acknowledges the Time-MMD dataset team, reflecting a good open-source culture.

Current limitations: mainly supports English text; other languages need improvement; fusion mechanism is relatively simple. Future directions: expand multilingual support, integrate complex fusion technologies like cross-modal Transformers, and explore model interpretability (e.g., key parts of text that influence predictions).

TSFMx provides a practical tool for the time series forecasting community, balancing technical advancement and engineering practicality. For researchers, it is an experimental platform; for industry, it lowers the threshold for multimodal fusion; for the open-source community, it is a good example. Multimodal time series modeling will become an important direction, and TSFMx demonstrates future possibilities.