Skip to content

ApplyPatch reports "Target already exists" for deleted files — stale creation cache not invalidated by shell rm #938

@harshav167

Description

@harshav167

Summary

When a file is created via ApplyPatch, then deleted via shell rm (through the Execute tool), subsequent ApplyPatch create attempts for the same path intermittently fail with "Target already exists" — even though the file is confirmed absent on disk. An immediate Read on the same path returns ENOENT, proving the file doesn't exist.

This primarily impacts OpenAI/Codex model sessions where ApplyPatch is the only available file creation tool (no Edit/Create/Write fallback).

Error Output

Apply Patch lsp_diag_smoke.go
 ↳ Target already exists - choose a different name

Read 1 file
 ↳ .../.factory/droid-lsp/lsp_diag_smoke.go (error)
 1 file failed

ApplyPatch says the file exists. Immediate Read says it doesn't. These two results contradict each other.

Environment

  • Droid version: 0.99.0 (also present in 0.98.0 — not a regression, latent bug)
  • OS: macOS 26.4 (Darwin 25.5.0), arm64
  • Terminal: Ghostty
  • Model: GPT-5.4 via BYOK (provider: "openai")

Reproduction

Step 1:  rm file (confirm absent via `test -e` → "absent")
Step 2:  ApplyPatch create file → SUCCESS, file created
Step 3:  rm file (confirm removed)
Step 4:  ApplyPatch create same file → "Target already exists"  ← BUG
Step 5:  Read same file → ENOENT (file doesn't exist)

Frequency: ~30% of create-after-delete cycles in the same session. Tested across 10 rounds:

Round rm confirmed absent? ApplyPatch create Read after Result
1 yes SUCCESS OK Clean
2 yes "already exists" ENOENT BUG
3 yes SUCCESS OK Clean
4 yes "already exists" ENOENT BUG
5 yes SUCCESS OK Clean
6 yes SUCCESS OK Clean
7 yes SUCCESS OK Clean
8 yes SUCCESS OK Clean
9 yes SUCCESS OK Clean
10 yes "already exists" ENOENT BUG

The bug reproduces within the same session — starting a new session clears the issue entirely, confirming it's session-scoped state.

Investigation

Through runtime analysis and binary investigation:

The cause: session-scoped creation tracker never invalidated on deletion

ApplyPatch's existence check doesn't rely solely on a direct filesystem stat. Based on observed behavior, there is a session-scoped in-memory tracker that records which files ApplyPatch has created. When checking whether a file exists before creating it, the check falls back to this tracker if the filesystem read returns "file not found." If the path was ever created by ApplyPatch during the session, the tracker still reports it as existing — even though the file has been deleted from disk.

The observed flow:

  1. ApplyPatch creates a file → path recorded in the session-scoped creation tracker
  2. Shell rm (via Execute tool) deletes the file from disk
  3. The Execute tool does not notify ApplyPatch's creation tracker about the deletion
  4. Next ApplyPatch create: filesystem check returns "not found" → falls back to creation tracker → tracker says "exists" → ApplyPatch rejects the create
  5. Immediate Read: goes directly to filesystem → ENOENT (correctly reports the file doesn't exist)

The tracker appears to only grow (paths are added on creation) with no removal mechanism. Files deleted through any path other than ApplyPatch itself (shell rm, manual deletion, another tool) leave ghost entries that permanently block re-creation for the rest of the session.

Why it's intermittent

The model sometimes issues duplicate parallel tool calls (a known pattern with OpenAI reasoning models). When duplicates race, one succeeds and the other hits the stale tracker. The ~30% failure rate correlates with how often the model emits parallel create calls, not with actual filesystem state.

Why it matters for OpenAI models specifically

Droid exposes different tool sets per provider:

  • Claude/Anthropic models: Get Edit, Create, Write, and ApplyPatch — if ApplyPatch fails, the model can fall back to Create or Write
  • OpenAI/Codex models: Get only ApplyPatch as the file writing tool — when the tracker is stale, the model has no fallback and cannot write files at all

The model then resorts to shell heredocs or Python file writes through Execute, which violates the intended tool boundaries.

Impact

  • Severity: Medium — blocks file creation in affected sessions, forces workarounds
  • User impact: OpenAI model users hit this repeatedly in tool-call-heavy sessions (missions, repeated test cycles, iterative file operations)
  • Workaround: Start a new session (clears the session-scoped tracker), or use a different filename

Suggested Fix

The creation tracker needs to be invalidated when files are deleted. Two approaches:

Option A — Invalidate on shell delete: When the Execute tool runs a command containing rm that succeeds, remove matching paths from the creation tracker.

Option B — Verify on disk before trusting the tracker: When the filesystem read returns "file not found" and the creation tracker says "exists," do an additional synchronous filesystem existence check before trusting the tracker. If the file isn't on disk, treat it as not existing regardless of what the tracker says.

Option B is simpler and handles all deletion paths (shell rm, manual deletion, other tools, external processes) without needing to parse shell commands.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions