RefineAct Accepted at ASE 2026

June 19, 2026

Links: Read the paper

Keeping AI Agents Within the User’s Intent

Our paper “RefineAct: Automatic Runtime Verification of LLM Agent Actions” has been accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026) in Munich. Fraol Batole led the work, with Foutse Khomh of Polytechnique Montréal and Hridesh Rajan.

Language-model agents that call external tools can do real damage when they drift from what a user actually asked, such as deleting an important file or exposing private data. Ordinary software has tests and contracts that say what correct behavior looks like, but an agent acting on a natural-language instruction has no such specification to check against. RefineAct builds one while the task runs. It turns the user’s instruction into formal predicates, plans the steps needed to satisfy them, and then inspects each action the agent proposes before it executes. Safe actions pass through, risky ones prompt the user to confirm, and actions that violate the user’s intent are blocked with corrective feedback. Across 144 tasks in the ToolEmu benchmark, RefineAct lowered the share of harmful runs from 77 percent to 39 percent and nearly doubled task quality.

The work extends our lab’s effort to make AI-enabled systems dependable, building on earlier results in repairing errors inside large language models and verifying fairness in neural networks.