The rise of AI-driven browser automation, robotic process automation, and web crawlers has introduced a new class of operational challenges: intermittent failures that are hard to reproduce, diagnose, and remediate at scale. Traditional logging and monitoring tools often miss the nuanced interplay between DOM changes, network latency, and heuristic decision‑making that leads to a test or bot falling over. Teams waste hours sifting through raw traces, screenshots, and console outputs, trying to infer root cause from fragmented evidence. This reactive stance not only slows delivery pipelines but also erodes confidence in automation initiatives, especially when compliance or security reviews demand auditable failure records. Agent Failure Doctor steps into this gap with a deliberately local‑first approach, keeping sensitive execution data on the developer’s machine while still providing rich, structured diagnostics that can be shared safely when needed.
At its heart, the project enforces two core principles: install only the essential diagnosis and benchmark core locally, and layer optional capabilities only where they are actually executed. This modularity reduces attack surface and ensures that teams in highly regulated environments can adopt the tool without pulling in unnecessary dependencies. The bootstrap command, failure-doctor agent-bootstrap –target all –project ., scaffolds a minimal workspace that pulls in the core diagnostic engine while leaving room for plugins such as OCR evidence extraction or visual runtime analysis. By separating core from extensions, the project respects plugin safety rules that prevent accidental execution of unverified code in production‑like contexts, a crucial guardrail for any tool that touches browser automation pipelines.
Getting started is deliberately lightweight. After installing the package from PyPI, developers run the bootstrap command to initialize a .failure-doctor directory in their project root. This directory houses run folders, configuration files, and the local knowledge base that powers the diagnostic engine. The design encourages a “install‑once, use‑many” mindset: the core remains static across projects, while each automation run generates its own isolated evidence bundle under .failure-doctor/runs/
The tool’s evolution reflects a steady accumulation of features that address real‑world pain points. Version 4.0 introduced local, evidence‑bound reasoning, meaning the diagnostic engine now ties conclusions directly to observable artifacts such as DOM snapshots, network traces, and console logs rather than relying on heuristic guesswork. Earlier releases laid the groundwork: v3.9 added a local‑only failure knowledge base that encodes recurring failure patterns observed across Playwright, Selenium, Puppeteer, and other frameworks; v3.8 brought local CI gates and starter templates, enabling teams to embed failure checks directly into their pipelines; v3.7 shipped a local‑only report console that renders diagnostic summaries without requiring a web server or external dashboard. Each iteration has reinforced the commitment to keeping data local while expanding the analytical depth available to engineers.
The command‑line interface is organized into logical groups that mirror a typical failure‑resolution workflow. The diagnosis group probes collected evidence to produce a structured conclusion; repair planning translates that conclusion into actionable remediation steps; verification confirms that proposed fixes actually resolve the observed symptom; collection gathers raw logs, traces, and screenshots from various adapters; OCR evidence extracts text from image‑based outputs; visual runtime records page‑state changes over time; patch proposal generates dry‑run code changes; fleet batch aggregates diagnostics across many runs for trend analysis; and safe sharing prepares sanitized bundles for external collaboration. This modular command structure lets teams pick and choose the stages that fit their existing DevOps practices, whether they run a full gate on every pull request or only invoke deep diagnostics on a nightly basis.
Quality assurance is enforced through the rigorous P98 master gate, a multi‑stage validation pipeline that must be cleared before a release is deemed stable. The gate begins with the knowledge base, verifying that its coverage matrix adequately represents known failure modes across supported frameworks. It then proceeds through trace analysis, cross‑framework correlation, training data validation, composite scenario testing, handoff simulations, batch processing checks, sanitization routines, and finally the auto‑collector logic. Only after successfully navigating each of these stages does the change reach the master gate, where the safety compliance pillar is evaluated. This pillar ensures that no diagnostic step inadvertently exposes secrets, that all generated reports respect the safe_to_share flag, and that the tool’s behavior remains deterministic under varied inputs—a critical requirement for trustworthy automation tooling.
Upon publishing to PyPI, the canonical installation command simplifies to pip install agent-failure-doctor, which pulls the latest stable release (currently 6.3.1). For users who need bleeding‑edge features, the project maintains advanced commands introduced in v3.2: failure-doctor collect gathers raw execution artifacts into a failure pack, while failure-doctor watch runs a persistent observer that triggers diagnostics automatically whenever a test suite fails. These commands enable tight feedback loops: a developer can watch a test run, capture the failure instantaneously, and immediately run a diagnosis without leaving the terminal. Such immediacy reduces mean‑time‑to‑resolution (MTTR) and supports a shift‑left mindset where defects are addressed as close to their point of origin as possible.
Integration with frontend agents and CI systems is facilitated through a set of focused commands. failure-doctor collect-playwright ingests Playwright test results, including traces and videos, and normalizes them into the internal failure‑pack format. failure-doctor pack-logs takes an unstructured folder of logs from Selenium, Puppeteer, Cypress, Scrapy, requests, or httpx and applies framework‑specific parsers to extract timestamps, error messages, and contextual data. failure-doctor adapt serves as a bridge for custom adapters, allowing teams to plug in proprietary automation frameworks by implementing a simple evidence‑extraction interface. Because all adapters write to the same .failure-doctor/runs/
When a failure occurs, the tool automatically creates a local run folder under .failure-doctor/runs/
The diagnostic engine follows an explicit evidence priority hierarchy to avoid overconfident conclusions when data is sparse. It first looks for direct artifacts such as DOM snapshots, network request/response pairs, and console error stacks. If those are present, it proceeds to cross‑framework traces, visual diffs, and OCR‑extracted text. Only when sufficient corroborating evidence exists does it emit a definitive conclusion; otherwise, it downgrades the verdict to insufficient_evidence rather than guessing. This conservative stance prevents false positives that could lead developers down incorrect remediation paths, thereby preserving trust in the tool’s output. Proxy or network failures, strict‑mode locator clashes, and low‑evidence screenshot‑only runs are all handled through this hierarchy, ensuring that each failure mode receives an appropriate depth of analysis.
A completed diagnosis yields a report structured into five digestible sections: conclusion, evidence, why (root cause rationale), next action (suggested remediation steps), and a Codex fix prompt that can be fed directly into AI‑assisted coding tools like Codex, Claude Code, or Cursor. Teams can then transform the report into task packs that these AI agents can execute, turning a static diagnostic into an actionable code change proposal. For those who prefer manual review, the propose-patch command generates a dry‑run patch that shows exactly what files would be altered, without touching the repository, running tests, or opening a pull request. This proposal‑only mode lets architects evaluate the impact of a fix before committing, aligning with modern trunk‑based development practices where every change is scrutinized.
Beyond individual runs, the tool supports fleet‑level analytics through commands that aggregate diagnostics across many executions. By feeding a directory of failure packs into the batch processor, organizations obtain a summary view that highlights prevalent failure patterns, framework‑specific hotspots, and temporal trends. This insight is invaluable for prioritizing engineering effort: if a particular selector pattern repeatedly causes flaky tests across multiple teams, a centralized investment in more robust locator strategies or wrapper libraries can yield outsized returns. The validation file v2.5 validation/ai_handoff_validation.json exemplifies how the project tracks the fidelity of AI‑generated remediation suggestions, closing the loop between human expertise and machine‑assisted fixes.
Looking at the broader market, the emergence of Agent Failure Doctor signals a maturing recognition that observability for AI‑driven automation must be both powerful and privacy‑preserving. As enterprises scale browser‑based bots for customer service, data extraction, and workflow orchestration, the cost of undiagnosed failures climbs rapidly—each minute of downtime can translate into lost revenue, damaged brand reputation, or regulatory penalties. A local‑first diagnostic tool that can be run behind firewalls, without sacrificing depth, addresses a critical gap left by cloud‑only APM solutions that often struggle with the ephemeral, DOM‑centric nature of browser automation. Furthermore, the shift toward AI‑augmented development (Copilot, Codex, etc.) creates a natural synergy: failure diagnostics feed directly into AI code‑generation pipelines, accelerating the remediation cycle.
For teams considering adoption, the practical advice is straightforward: start small, iterate, and institutionalize the habit of local diagnostics. First, install the core package and run the bootstrap command in a representative project to familiarize yourselves with the folder structure and basic commands. Next, enable the Playwright adapter if you use that framework, and run failure-doctor collect-playwright on a known failing test to see the end‑to‑end flow from raw trace to actionable report. Once comfortable, integrate the watch command into your CI pipeline so that every failure triggers an automatic diagnosis, and configure the sanitize step as a gate before any artifact leaves your internal network. Over time, leverage the fleet‑level summary to identify systemic issues and advocate for targeted improvements in your automation framework, selector strategies, or error‑handling patterns. By treating failure diagnosis as a first‑class citizen rather than an afterthought, you’ll not only reduce MTTR but also build more resilient, observable automation systems that scale confidently with your business needs.