Temporal semantic cache cuts industrial agent latency by 30.6× on repeat queries

AssetOpsBench (AOB), a new industrial agent benchmark from IBM Research, exposes a critical latency problem in plan-execute pipelines that coordinate sensor data, work orders, and forecasting tools. A preprint published May 21 shows that existing LLM caching techniques—KV-cache reuse and embedding-based semantic caching—break down when output validity depends on time, asset identity, or live sensor parameters. The authors propose a temporal semantic cache and a set of MCP workflow optimizations that together cut median end-to-end latency by 40 percent and achieve up to 30.6× speedup on cache hits.

Industrial asset operations workflows are latency-sensitive because a single user query may trigger tool discovery, LLM planning, MCP tool execution, and final summarization across multiple domain-specific agents. The paper introduces two complementary optimization layers: a temporal semantic cache that respects parameter drift over time, and MCP workflow optimizations combining disk-backed tool-discovery caching with dependency-aware parallel step execution. The MCP workflow layer alone delivered a 1.67× speedup. On cache hits, the temporal cache achieved a median 30.6× speedup—orders of magnitude faster than re-executing the full pipeline.

Parameter drift and stale answers

Pure semantic caching treats queries with similar embeddings as equivalent, which works for chatbot serving but fails for parameter-rich industrial queries. A question about turbine vibration at 10 a.m. is not semantically equivalent to the same question at 2 p.m. if sensor readings have changed. The paper documents this failure mode in detail, showing that embedding-based caches return stale answers when asset state or time parameters shift. The temporal cache solves this by keying on both semantic similarity and parameter metadata—asset ID, timestamp, sensor context—so that cache hits respect the validity window of the underlying data.

The AssetOpsBench dataset and evaluation code are available on HuggingFace. The authors note that the results provide a critical analysis of how caching choices interact with evaluation correctness in MCP-backed agent benchmarks, a consideration that matters as industrial AI moves from prototypes to production deployments where stale answers can trigger costly maintenance decisions.