Project
Distributed Neural Network Inference over Low-Power Wireless Networks: Design and Real-World Evaluation
Modern machine learning methods, such as transformer models and Bayesian optimization, are becoming increasingly important in the context of cyber-physical systems (CPS). A key component of CPS is distributed computation, wherein multiple agents collaborate toward a common goal. In this work, we investigated two distinct approaches: distributed transformer inference on wireless, ultra-low-power devices and federated Bayesian optimization.
In the first project, we aimed to enable small, ultra-low-power devices (e.g., microcontrollers) communicating via a low-power wireless mesh network to execute transformer models significantly larger than a single device can handle natively. In addition to a detailed hardware evaluation, we conducted an in-depth analysis of specific system properties through high-performance computing (HPC) simulations, which required large-scale training studies.
In the second project, our goal was to enable privacy-preserving Bayesian optimization. We examined a setting with multiple agents featuring coupled objective functions, meaning the functions are distinct yet mathematically related across agents. We primarily aimed to allow these agents to leverage this coupling to accelerate their local optimization processes without compromising data privacy. To evaluate our approach, we conducted a large-scale simulation study, comparing our method against existing baselines across various datasets.
Project Details
Project term
May 2, 2025–April 30, 2026
Affiliations
RWTH Aachen University
Institute
Institute for Data Science in Mechanical Engineering
Principal Investigator
Methods
Our distributed transformer framework employs targeted partitioning methods that account for the underlying hardware and communication constraints. We combine these with dedicated pruning strategies to reduce memory and bandwidth requirements. Additionally, we apply targeted dropout during training to ensure inference remains robust against the message loss inherent to wireless networks.
Our federated Bayesian optimization method builds on recent advances in classic Bayesian optimization, demonstrating that probabilistic neural networks can achieve performance similar to the Gaussian processes traditionally used as surrogate models. We capitalize on this by using classical personalized federated learning techniques to train coupled neural network surrogates across agents. This approach enables agents to effectively pool their knowledge without compromising data privacy.
Results
Using our distributed inference method, we successfully executed transformer models up to 14 times larger than a single device’s capacity across a network of 16 ultra-low-power devices. Our simulation results results show that our approach effectively reduces bandwidth and memory requirements without sacrificing accuracy, while remaining robust against probabilistic message loss.
Furthermore, our federated Bayesian optimization framework enables effective knowledge sharing among agents, which significantly accelerates the optimization process without compromising accuracy. The proposed method also significantly outperforms existing baselines across a variety of benchmark problems. Beyond these core findings, we showed that agents joining the network late can warm-start their local optimization by leveraging the knowledge already acquired by the existing agents.
Discussion
Our results demonstrate that large transformer models can be successfully executed across networks of ultra-low-power devices. These findings open up new avenues for future research, allowing our method to be deployed in real-world applications such as smart healthcare, environmental monitoring, and smart grids.
Furthermore, the proposed federated Bayesian optimization framework enables privacy-preserving, distributed black-box optimization. This approach is highly applicable to fields like healthcare, smart homes, manufacturing, and autonomous vehicles. While the framework significantly accelerates the optimization process, a current limitation is the requirement for agent synchronization, meaning the slowest agent creates a bottleneck for the entire system. In future work, we plan to investigate an asynchronous extension of our method to address this.
Additional Project Information
DFG classification: 409 Computer Science
Software: DNNI, Federated BO, Foundational Model Training
Cluster: CLAIX
Publications
Bernd Frauenknecht, Lukas Kesper, Daniel Mayfrank, Henrik Hose, Sebastian Trimpe,
Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics,
Peer-reviewed publication, 2026
Paul Brunzema, Sebastian Trimpe,
BayeSQP: Bayesian Optimization through Sequential Quadratic Programming,
Peer-reviewed publication, 2026
Alexander Gräfe, Christoph Scherer, Wolfgang Hönig, Sebastian Trimpe,
How to Model Your Crazyflie Brushless,
Peer-reviewed publication, 2026
Alexander Gräfe, Fabian Mager, Marco Zimmerling, Sebastian Trimpe,
RockNet: Distributed Learning on Ultra-Low-Power Devices,
ACM Transactions on Cyber-Physical Systems Vol. 10 1-28, 2026
Alexander Gräfe, Ding Huo, Johannes Berger, Marco Zimmerling, Sebastian
Trimpe,
Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices
Peer-reviewed publication, 2026
Alexander Gräfe, Sebastian Trimpe,
MPX: Mixed Precision Training for JAX,
White Paper, 2026
Thesis:
Ding Huo,
Distributed Inference of Transformer Networks in Low-Power Sensor Networks,
Master Thesis, 2025
Lukas Kesper,
Reinforcement Learning with Uncertainty-Aware Model Predictive Safety Certification,
Master Thesis, 2025