# Analysis of KP Run: dns_stale fault

## 1. User feedback

The User agent's final message to the human was:

> "ACM.org unreachable due to blocked route at University network infrastructure (router 10.x). Knowledge Plane (Uni) has been notified but no resolution yet."

**This diagnosis is wrong on every substantive point.**

The injected fault was a stale DNS record at AS1's resolver: `acm.org` resolved to `198.82.0.99` instead of the live `198.82.0.1`. The User agent actually had the decisive evidence in hand. Its own `dig` returned `198.82.0.99`, and its ping output showed:

> `PING acm.org (198.82.0.99) from 128.173.10.1 ... From 198.82.0.254 icmp_seq=1 Destination Host Unreachable`

The ICMP host-unreachable is being generated by `198.82.0.254` — that is ACM's own border address (visible in ACM's routing table as the loopback `198.82.0.254`, and there is even an `unreachable 198.82.0.99` route installed on ACM). So the failure point was ACM's network rejecting `.99` because nothing is hosted there. The right diagnosis is "DNS gave us an address that isn't the live service."

Instead, the User agent fixated on the traceroute hop `10.0.1.2 ... !N` and concluded the Uni router was blocking the route. That is the upstream router emitting a network-unreachable on behalf of the destination — not Uni's infrastructure. The agent never compared the resolved `.99` to other observations (e.g., asking Uni to verify the address), never re-resolved against another resolver after the first failure, and gave up after a single follow-up to Uni.

The answer is also not actionable: it tells the user "Uni has a blocked route" with no path forward. The real, actionable advice — "try again later / clear DNS cache / the published address is stale, ACM's actual service at the new IP is fine" — was never produced.

## 2. Agent collaboration

**Initial escalation.** The User agent did the right first step — it reproduced the failure locally and escalated to Uni:

> User → Uni: "User reports inability to reach ACM.org (198.82.0.99). My loopback address is 128.173.10.1; ping failed with 'Destination Host Unreachable' from 198.82.0.x routers despite valid default route via you (User-eth0). Investigate path/routing for 198.82/24."

Critically, the User agent **did include the resolved IP `198.82.0.99` in this message**. Any agent paying attention to it had enough to spot the fault.

**Key exchanges (approximate order):**

- User → Uni: WHY — acm.org unreachable, resolves to 198.82.0.99, ping fails at 198.82.0.254.
- Uni → User: (no real reply; Uni instead tried to debug its own non-existent "BGP outage" and never engaged with the `.99` clue).
- User → Uni: follow-up with traceroute showing `!N` at 10.0.1.2.
- Uni → AS1: a series of escalations about "routing loops" and "BGP down" — based on Uni's own broken tests (Uni had deleted its own default route mid-investigation, and later got pingbacks loop-like outputs from its own misconfiguration).
- AS1 ↔ EveLink: a completely unrelated fabricated saga about an iptables `DROP -s 91.214.0.0/24` rule, no-export communities, and ticket `KP-AS1-0923`. None of this had anything to do with the user's complaint.
- ACM internally noticed and removed the stale `198.82.0.99` route: *"Removed a stale unreachable route (`198.82.0.99`) using `ip route del 198.82.0.99`"* — but never connected this to the user complaint and never reported anything externally.

**WHY / FIX / CANNOT discipline.** It was largely cargo-culted. EveLink and AS1 littered messages with "KP WHY REQUEST" / "KP FIX REQUEST" / "KP CANNOT" headers, but the content was unrelated to the actual fault. The one CANNOT that was applied correctly in policy was AS1's refusal to unilaterally remove the (imagined) firewall rule:

> AS1 → EveLink: "Your route blockage requires manual admin review due to security policy constraints. Tracking as ticket KP-AS1-0923 for resolution within 30 minutes."

The policy logic (firewall changes need admin approval) is right; the problem is the firewall rule never existed — AS1 was responding to a hallucinated diagnosis.

**Gaps:**

- **AS1 never investigated its own DNS resolver**, despite running one ("You run a DNS recursive resolver listening on your loopback address") and despite the User's WHY containing the resolved IP. The whole point of the KP chain — Uni → AS1 — should have been "is `198.82.0.99` the right address for acm.org?" That question was never asked.
- **Uni never relayed the actual symptom upward.** It transformed "user can't reach 198.82.0.99" into "my BGP is down" and chased that ghost for ~40 iterations.
- **ACM noticed the stale `198.82.0.99` locally and silently deleted it**, but never told anyone — even though the WHY query that propagated upstream was literally about that IP. ACM's policy permits sharing "observable status of your service" externally; "the address being queried is not our live service IP" is exactly that kind of information.
- **Web sat idle for ~50 iterations** after declaring success, contributing nothing despite being the canonical authority on what its real address is.
- **User stopped at one follow-up** and reported done at iteration 19, well before any KP reply.

## 3. Overall assessment

The KP did **not** deliver a correct or timely response. The injected fault was diagnosable from the very first User WHY — the message contained the stale IP — and the right party to answer (AS1, the DNS resolver, or ACM, the address owner) had all the information needed. Neither connected the dots.

What worked:
- The User agent reproduced the symptom locally before escalating and included concrete data (the resolved IP, the source of the ICMP unreachable).
- ACM did detect and clean up the stale unreachable route in its own table.
- AS1 correctly refused to unilaterally change security policy.

What needs to improve:
- **Agents must read the WHY they receive.** Uni and AS1 never engaged with the specific IP `198.82.0.99` that User reported; they substituted their own pet theories. A simple "does `198.82.0.99` actually serve acm.org?" check at either AS1 (resolver) or ACM (address owner) would have solved it.
- **Don't hallucinate symptoms.** Most of the run's traffic was AS1/EveLink/Uni debugging problems that didn't exist (BGP down, firewall DROP rule, no-export community). Several agents spent dozens of iterations on syntax errors in their own diagnostic commands rather than on the actual fault.
- **Cross-domain correlation is the KP's job.** ACM saw a stale `.99` route; AS1 owns the resolver that returned `.99`; User reported `.99`. No agent connected these three facts, even though each is in plain text in the logs.
- **The User agent gave up and reported a wrong, non-actionable diagnosis to the human** rather than waiting for or pushing for a real KP answer. The system prompt explicitly says "Only report back to the owner once you are satisfied that you have a reliable answer" — the agent violated this when it terminated at iteration 19 with an unconfirmed hypothesis.

For the KP to handle this scenario reliably, agents need to (a) treat the literal contents of a WHY as primary evidence, (b) check their own authoritative data (AS1's resolver cache; ACM's list of live service IPs) before theorizing about distant infrastructure, and (c) refuse to close the loop with the human until the chain has produced an actual FIX or a properly-scoped CANNOT.