Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference¶

🕒 Published (v1): 2026-07-03 02:28 UTC · Source: Arxiv · link

Why this paper was selected

Symbolic inference auto-repairs platform-orchestrated workflows; addresses harness reliability gap

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

FlowFixer is a diagnosis-driven automated repair framework for platform-orchestrated agentic workflows (Dify, Coze, n8n). It converts execution trajectories into symbolic traces, infers formal behavioral specifications via an LLM-based generator, localizes failure-responsible nodes, and produces targeted repair patches. On real-world workflow failures, it achieves 71.3% repair success rate, outperforming baselines by 11.9–27.6%.

Problem¶

Agentic workflows exhibit non-deterministic failures arising from probabilistic LLM outputs, implicit inter-node dependencies, and heterogeneous tool interactions. Existing repair/optimization methods operate on trajectory-level feedback without explicit root-cause attribution, producing coarse-grained changes weakly coupled to the actual failure source. Traditional automated program repair is inapplicable because workflow failures originate in prompts, node configurations, and control dependencies rather than in source code.

Method¶

FlowFixer operates in two tightly coupled stages.

Stage 1 — Failure Diagnosis: - Symbolic Modeling: Normalize execution trajectories across platforms into a unified symbolic trace \(\{node_1, \ldots, node_n\}\), each node carrying id, type, I/O, status, and configuration. - Symbolic Inference: For each node, an LLM-based generator produces formal behavioral specifications in a lightweight assertion DSL (BNF-defined). Specifications cover three dimensions: existence (required fields/outputs present), temporal (execution-order constraints), and causal (semantic relationships, variable consistency, task requirements). - Failure Attribution: Evaluate each node's specifications against actual outputs; compute a suspicious score combining assertion-violation rate and structural propagation position (earlier nodes weighted higher to account for downstream symptom propagation). - Root Cause Analysis: Map the failure-responsible node to one of 16 taxonomy categories (grouped into node capability, node orchestration, node execution) using symbolic context and assertion evidence.

Stage 2 — Workflow Repair: - Patch Generation: Root-cause-aware repair knowledge maps each root cause to a repair strategy (e.g., poor prompt design → prompt constraint enforcement). Patches are sequences of atomic edits: \(\text{edit} = op(\text{target}, \text{content}, \text{position})\) with \(op \in \{\text{insert, remove, replace, append, swap}\}\). - Pre-execution Assessment: Four-dimensional lightweight filter before costly dynamic execution: (a) structural correctness (valid syntax/connections), (b) semantic correctness (LLM infers expected config from masked workflow and compares to actual), (c) behavioral consistency (patch addresses diagnosed root cause), (d) offset rationality (modification magnitude within threshold). - Experience Pool: Online repair feedback (trajectories, verdicts, reflections) accumulates to improve future diagnosis and repair iterations.

Key Contributions¶

Symbolic modeling + inference framework that makes agentic workflow failures formally analyzable across three constraint dimensions (existence, temporal, causal).
Unified node behavioral specification instantiated as executable DSL assertions providing structured evidence for failure diagnosis.
Multi-dimension pre-execution assessment to filter infeasible patches before dynamic verification, reducing overhead.
Empirical evaluation on three real-world platforms (Dify, Coze, n8n) demonstrating effectiveness over state-of-the-art baselines.

Results¶

Repair success rate: 71.3%, outperforming all baselines by 11.9% to 27.6%.
Failure attribution accuracy: improved by 4.8% to 33.1% over baseline methods.
Root cause analysis accuracy: improved by 15.3% to 38.8% over baseline methods.
Ablation studies confirm contributions of symbolic modeling, root cause taxonomy, repair knowledge, execution information, and experience accumulation individually.
Evaluated on workflow failures from Dify, Coze, and n8n platforms.

Limitations¶

Specifications are inferred by an LLM-based generator; quality depends on the generator model's ability to correctly formalize expected node behavior, which may be unreliable for highly complex or novel node types.
The paper is truncated before full experimental details (RQ results, dataset statistics, baseline descriptions) are presented.
The root cause taxonomy (16 types from prior literature) may not cover failure modes unique to newer or proprietary platforms.
Pre-execution assessment's semantic correctness check re-uses an LLM to infer expected configurations, introducing potential circularity if the same model produced the faulty specification.
Experience pool utility depends on repeated failures of similar types; cold-start scenarios with novel failure patterns receive no benefit.

Relevance to Harnesses / Meta-Harnesses¶

FlowFixer is directly relevant as a meta-level repair harness that wraps and monitors lower-level agentic workflow execution: it intercepts failing trajectories, symbolically analyzes them, and issues corrective patches—exactly the diagnostic+remediation loop that a meta-harness must implement. Its symbolic trace abstraction, which normalizes heterogeneous platform outputs into a uniform representation, is a concrete blueprint for platform-agnostic observability in multi-platform harness designs. The diagnosis↔repair feedback loop with an experience pool mirrors the self-improving harness pattern where repeated executions refine the orchestration policy. Researchers building harnesses that must handle runtime failures across LLM-based pipelines can directly adopt FlowFixer's three-dimensional specification schema and pre-execution assessment structure.