QuantFlow pairs Mamba state-space layers with federated learning for privacy-preserving time-series forecasting
A new arXiv preprint combines bidirectional Mamba state-space layers, quantile regression, and federated learning for privacy-preserving time-series prediction across crypto, traffic, and weather datasets.
QuantFlow, a probabilistic forecasting framework published on arXiv this week, replaces Transformer attention with bidirectional Mamba state-space decoders to handle long, high-dimensional time-series while preserving privacy. The architecture embeds each variable over the complete observation window, processes it in forward and reverse directions through Mamba layers, then projects the result to five conditional quantiles. This inverted sequence embedding paired with bidirectional state-space modeling sidesteps the quadratic attention cost that limits Transformer context length, addressing a persistent tension in foundation models: centralized Transformers deliver strong transfer across forecasting tasks but struggle with privacy constraints that prohibit raw-data pooling.
The framework incorporates TSMixup, a Dirichlet-weighted interpolation technique that expands temporal diversity by blending sequences while preserving their internal structure. Experiments span six benchmark families—cryptocurrency, traffic, electricity, Electricity Transformer Temperature (ETTm1), influenza, and weather—with QuantFlow recording mean squared errors of 0.2834 on ETTm1 and 0.2218 on Weather. The quantile-regression head outputs five conditional quantiles per forecast step, giving practitioners uncertainty bands alongside point predictions. A 20-client non-IID federated deployment demonstrates privacy-preserving capability: after three communication rounds, each client trains locally on its partition and shares only gradient updates without centralizing raw records, yet the federated ensemble retains useful accuracy. The preprint acknowledges two limitations: irregular epidemiological signals (influenza data with variable observation intervals) degrade performance, and long-horizon generalization remains weak when forecast windows extend beyond training horizons, both attributed to Mamba's fixed-length hidden state.
