As a nabledge user, I want search quality held to the same standard as RBKC so that answers are accurate and hallucination-free

[kiyotis]

Situation

RBKC generates knowledge files under a ゼロトレランス quality standard. The search layer that delivers those files to users has not been held to the same standard. Additionally, the nabledge-test benchmark is heavily weighted toward questions that already contain Nablarch-specific keywords, making it easy to score well without genuinely good retrieval — the benchmark does not reflect real-world question diversity.

Pain

• When a knowledge file contains relevant information but it is not surfaced in the answer, users are misled by omission.
• When the answer includes content not present in any knowledge file, users are misled by hallucination.

Benefit

• Users can make decisions based on accurate, knowledge-grounded information without second-guessing what was omitted or invented.
• Maintainers can validate search improvements against a general-purpose benchmark that reflects real question diversity, giving them confidence that improvements are genuine.

Success Criteria

• Benchmark scenarios revised to cover general-purpose questions not dependent on Nablarch-keyword presence — scores reflect real retrieval capability, not keyword matching
• Answer accuracy (no omission of relevant KB content) and hallucination prevention (no content outside KB) both improve over the current search flow, as measured by the revised benchmark