Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3093,6 +3093,14 @@ Downstream users can add lightweight language support without rebuilding
the extracted JSON to a fixture. The source and expectation files are capped
at 4 MiB each.

Query-side `--lang` resolution uses this same workspace-aware extension and
extractor registry rather than a separate built-in list. Registered language
IDs, aliases, and extension-like spellings resolve to the registry's canonical
ID. Unknown values fail with `E010_USAGE_ERROR` and bounded edit-distance
suggestions; `--allow-unknown-lang` is the explicit escape hatch for an
unregistered plugin ID and preserves its trimmed spelling through the database
filter.

Minimal examples:

```yaml
Expand Down Expand Up @@ -5693,6 +5701,13 @@ cleared range を証明するテストが必要です。Bounded accumulation pat
symbol extraction だけを実行し、`--expect-symbols <json>` で fixture JSON と比較できます。
source と expectation file はそれぞれ 4 MiB に制限されます。

query 側の `--lang` 解決は、別の組み込み一覧ではなく、この workspace-aware な
extension / extractor registry を共有します。登録済み language ID、alias、拡張子形式の
表記は registry の canonical ID に解決されます。未知の値は上限付き edit-distance 候補を
伴う `E010_USAGE_ERROR` になり、未登録 plugin ID には明示的な escape hatch
`--allow-unknown-lang` を使います。この場合、前後空白を除いた表記を database filter まで
保持します。

最小例:

```yaml
Expand Down
15 changes: 15 additions & 0 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1251,6 +1251,14 @@ cdidx search "--open-reports" --path README.md --count # quoted literal that st
cdidx search --query "--path" --path README.md # search for an option-looking literal
```

`--lang` validates built-in language names, recognized aliases, extension-like
spellings such as `.cs`, and language IDs registered for the indexed workspace.
A misspelling such as `cshrap` is a usage error (`E010_USAGE_ERROR`) and reports
up to three nearby canonical IDs instead of silently returning zero results.
Use `--allow-unknown-lang` only when querying an unregistered plugin language
ID; that escape hatch trims surrounding whitespace but otherwise preserves the
ID's case and punctuation exactly.

Search normalizes literal FTS queries to Unicode NFC before matching. If every
literal token exceeds SQLite FTS5 unicode61's 1000-character token cap,
zero-result JSON includes `query_degraded_reason` and `tokens_dropped`. Index
Expand Down Expand Up @@ -4391,6 +4399,13 @@ cdidx search "--open-reports" --path README.md --count # `--` で始まる引
cdidx search --query "--path" --path README.md # オプションに見えるリテラルを検索
```

`--lang` は組み込み言語名、認識済み alias、`.cs` のような拡張子形式の表記、
および indexed workspace に登録された language ID を検証します。`cshrap` のような
入力ミスは 0 件として黙って成功せず、usage error(`E010_USAGE_ERROR`)として近い
canonical ID を最大3件報告します。未登録 plugin の language ID を検索する場合だけ
`--allow-unknown-lang` を使ってください。この escape hatch は前後の空白のみを除去し、
ID の大小文字と句読点はそれ以外そのまま保持します。

literal FTS クエリは照合前に Unicode NFC へ正規化されます。すべての literal
token が SQLite FTS5 unicode61 の 1000 文字 token 上限を超える場合、0 件
JSON には `query_degraded_reason` と `tokens_dropped` が含まれます。index
Expand Down
37 changes: 37 additions & 0 deletions changelog.d/unreleased/4842.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
category: fixed
issues:
- 4842
affected:
- src/CodeIndex/Database/DbReader.cs
- src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs
- src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.Search.cs
- src/CodeIndex/Cli/QueryCommandRunner.Search.cs
- src/CodeIndex/Cli/QueryCommandRunner.Definition.cs
- src/CodeIndex/Cli/QueryCommandRunner.Graph.cs
- src/CodeIndex/Cli/QueryCommandRunner.Discovery.cs
- src/CodeIndex/Cli/QueryCommandRunner.Find.cs
- src/CodeIndex/Cli/QueryCommandRunner.Map.cs
- src/CodeIndex/Cli/QueryCommandRunner.Inspect.cs
- src/CodeIndex/Cli/QueryCommandRunner.Dependencies.cs
- src/CodeIndex/Cli/QueryCommandRunner.Unused.cs
- src/CodeIndex/Cli/QueryCommandRunner.Hotspots.cs
- src/CodeIndex/Cli/QueryCommandRunner.cs
- src/CodeIndex/Cli/CliFlagSchema.cs
- src/CodeIndex/Cli/ConsoleUi.Help.cs
- tests/CodeIndex.Tests/QueryCommandRunnerTests.cs
- tests/CodeIndex.Tests/QueryCommandRunnerSymbolTests.cs
- tests/CodeIndex.Tests/QueryCommandRunnerReferencesTests.cs
- tests/CodeIndex.Tests/QueryCommandRunnerInspectTests.cs
- tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs
- USER_GUIDE.md
- DEVELOPER_GUIDE.md
---

## English

- **Misspelled query language filters now fail with bounded suggestions (#4842)** — query commands validate `--lang` against the same workspace-aware language and plugin registry used for indexing, accept aliases and extension-like spellings, and return `E010_USAGE_ERROR` in human or JSON output instead of a silent empty result. `--allow-unknown-lang` explicitly permits an unregistered plugin ID while preserving its case and punctuation.

## 日本語

- **query の言語フィルタ入力ミスを上限付き候補とともに拒否するようになりました (#4842)** — query command は indexing と同じ workspace-aware な language / plugin registry に対して `--lang` を検証し、alias と拡張子形式の表記を受理します。入力ミスは空結果として黙って成功せず、人間向け/JSON 出力で `E010_USAGE_ERROR` を返します。`--allow-unknown-lang` を明示すると、未登録 plugin ID の大小文字と句読点を保持したまま検索できます。
3 changes: 2 additions & 1 deletion src/CodeIndex/Cli/CliFlagSchema.cs
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,8 @@ private static IReadOnlyList<CliFlag> BuildAll()
new() { Name = "--max-results", ValuePlaceholder = "<n>", Description = "Search alias for --limit", PrimaryCommands = Set("search") },
new() { Name = "--top", ValuePlaceholder = "<n>", Description = "Max results", PrimaryCommands = Set(LimitCapableCommands) },
new() { Name = "--offset", ValuePlaceholder = "<n>", Description = "Suggestions: skip this many filtered rows before output", PrimaryCommands = Set("suggestions") },
new() { Name = "--lang", ValuePlaceholder = "<lang>", Description = "Filter by language", PrimaryCommands = Set(LangCapableCommands), AlsoAcceptedBy = Set("suggestions") },
new() { Name = "--lang", ValuePlaceholder = "<lang>", Description = "Filter by a registered language, alias, or extension-like spelling", PrimaryCommands = Set(LangCapableCommands), AlsoAcceptedBy = Set("suggestions") },
new() { Name = "--allow-unknown-lang", Description = "Allow an unregistered plugin language ID and preserve its exact spelling", PrimaryCommands = Set(LangCapableCommands) },
new() { Name = "--language", ValuePlaceholder = "<lang>", Description = "Suggestions: filter by language; languages: look up one language by canonical name or recognized language spelling", PrimaryCommands = Set("suggestions", "languages") },
new() { Name = "--extension", ValuePlaceholder = "<ext>", Description = "Languages: look up language support by extension or recognized filename pattern", PrimaryCommands = Set(LanguagesFilterCommands) },
new() { Name = "--alias", ValuePlaceholder = "<alias>", Description = "Languages: look up language support by display alias", PrimaryCommands = Set(LanguagesFilterCommands) },
Expand Down
1 change: 1 addition & 0 deletions src/CodeIndex/Cli/ConsoleUi.Help.cs
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ private static void PrintFlagReference(Action<string> WriteHelpLine)
Console.WriteLine(" --limit <n>, --top <n>, --max-results <n>");
Console.WriteLine(" Max results to return (default: 20)");
Console.WriteLine(" --lang <lang> Filter by language (aliases: bat, cmd, cshtml, razor, ts, tsx, cts, mts)");
Console.WriteLine(" --allow-unknown-lang Allow an unregistered plugin language ID and preserve its exact spelling");
Console.WriteLine(" --path <glob> Restrict matches to glob-style path patterns (* and ?)");
WriteHelpLine($" --query <query> Pass a query literal, useful when the query starts with '-' (`search`/`find` max {QueryLimits.MaxQueryLength} chars)");
WriteHelpLine(" --named-query <name>=<query> search only: add a named ad hoc batch query; repeat to run related searches with grouped compact results");
Expand Down
4 changes: 4 additions & 0 deletions src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.Search.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,15 @@ private bool TryParseSearchOption(string normalizedArg, string currentArg, strin
// DB の `files.lang` はすべて lowercase なので、SQL filter と WriteLangHint が一致する。
// さらに `py` のような短縮エイリアスを正規名へ畳み込み、Python 利用時の慣用入力で
// 意図せず 0 件になる事故を避ける。
rawLang = langValue;
lang = NormalizeLangFilterValue(langValue);
}
else
AddParseError(langError!);
break;
case "--allow-unknown-lang":
allowUnknownLang = true;
break;
case "--query":
if (!allowNamedQuery)
{
Expand Down
71 changes: 71 additions & 0 deletions src/CodeIndex/Cli/QueryCommandRunner.ArgumentParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ private sealed partial class QueryArgumentParser
private readonly string? defaultLimitError;
private int? totalLimit;
private string? lang;
private string? rawLang;
private bool allowUnknownLang;
private bool languageValidationError;
private string? kind;
private string? unusedBucket;
private string? minUnusedConfidence;
Expand Down Expand Up @@ -218,6 +221,7 @@ internal QueryCommandOptions Parse(string[] args)
var resolvedDbPath = dbResolution.DbPath;

ResolveProjectFilters(resolvedDbPath);
ResolveLanguageFilter(resolvedDbPath);
ValidateParsedOptions();
ApplySearchSourceOptionDefaults();
ValidateEnvironmentDefaults();
Expand Down Expand Up @@ -267,6 +271,71 @@ private void ValidateParsedOptions()
AddParseError("Error: --duplicate-confidence and --duplicate-threshold cannot be combined; use the preset or the explicit score threshold.");
}

private void ResolveLanguageFilter(string resolvedDbPath)
{
if (rawLang == null)
{
if (allowUnknownLang)
{
languageValidationError = true;
AddParseError($"Error [{CommandErrorCodes.UsageError}]: --allow-unknown-lang requires --lang <lang>.");
}
return;
}

var input = rawLang.Trim();
var primaryRoot = s_batchReader != null && dbPathExplicit
? ResolveProjectRootForDbPath(resolvedDbPath, dbPathExplicit).Root
: ResolveProjectFilterRoot(resolvedDbPath, dbPathExplicit).Root;
var queryRoots = workspaceDbPaths
.Select(path => ResolveProjectRootForDbPath(DbPathResolver.NormalizeDbPath(path), dbPathExplicit: true).Root)
.Prepend(primaryRoot)
.Distinct(StringComparer.OrdinalIgnoreCase);
var aliases = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
foreach (var queryRoot in queryRoots)
{
foreach (var (alias, registeredCanonical) in DbReader.GetQueryLanguageAliases(queryRoot))
aliases.TryAdd(alias, registeredCanonical);
}
var lookupKey = DbReader.NormalizeQueryLanguageLookupKey(input);
if (lookupKey.Length == 0)
{
languageValidationError = true;
AddParseError(
$"Error [{CommandErrorCodes.UsageError}]: --lang must contain at least one letter or digit; got '{ConsoleUi.FormatBoundedValue(input)}'.");
return;
}
if (aliases.TryGetValue(lookupKey, out var canonical))
{
lang = canonical;
return;
}

if (allowUnknownLang)
{
// An unregistered plugin ID is an explicit escape hatch. Keep its spelling
// intact (apart from surrounding whitespace) so punctuation and case still
// match the exact value stored in files.lang.
// 未登録 plugin ID は明示的な escape hatch として扱い、前後の空白以外は
// 変更しない。句読点と大小文字を files.lang の保存値へ正確に一致させる。
lang = input;
return;
}

var suggestions = ConsoleUi.FindClosestMatches(lookupKey, aliases.Keys)
.Select(candidate => aliases[candidate])
.Distinct(StringComparer.Ordinal)
.Take(3)
.ToArray();
var suggestionText = suggestions.Length == 0
? string.Empty
: $" Did you mean {string.Join(", ", suggestions.Select(value => $"'{ConsoleUi.FormatBoundedValue(value)}'"))}?";
languageValidationError = true;
AddParseError(
$"Error [{CommandErrorCodes.UsageError}]: unknown language identifier '{ConsoleUi.FormatBoundedValue(input)}'.{suggestionText} " +
"Use --allow-unknown-lang only for an unregistered plugin language ID.");
}

private void ApplySearchSourceOptionDefaults()
{
if (parseErrors != null
Expand Down Expand Up @@ -316,6 +385,8 @@ private QueryCommandOptions BuildOptions(DbPathResolution dbResolution, string r
TotalLimit = totalLimit,
LimitExplicit = limitExplicit,
Lang = lang,
AllowUnknownLang = allowUnknownLang,
LanguageValidationError = languageValidationError,
Kind = kind,
UnusedBucket = unusedBucket,
MinUnusedConfidence = minUnusedConfidence,
Expand Down
8 changes: 6 additions & 2 deletions src/CodeIndex/Cli/QueryCommandRunner.Definition.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,11 @@ public static int RunDefinition(string[] cmdArgs, JsonSerializerOptions jsonOpti
allowNamedQuery: true,
validateDefaultSnippetLines: false,
validateDefaultMaxLineWidth: false);
using var exactLanguageScope = DbReader.BeginExactQueryLanguageScope(
options.Lang);
if (TryWriteUnsupportedOptionError("definition", cmdArgs, CliFlagSchema.GetAcceptedFlagNamesForCommand("definition"), options.Query))
return CommandExitCodes.UsageError;
if (TryWriteParseError(options, "definition"))
if (TryWriteParseError(options, "definition", options.LanguageValidationError ? jsonOptions : null))
return CommandExitCodes.UsageError;
if (TryWriteInvalidKindFilterError(options, "definition", KnownSymbolKindFilters))
return CommandExitCodes.InvalidArgument;
Expand Down Expand Up @@ -297,9 +299,11 @@ public static int RunGoto(string[] cmdArgs, JsonSerializerOptions jsonOptions)
var all = cmdArgs.Any(arg => arg == "--all");
var filteredArgs = cmdArgs.Where(arg => arg != "--all").ToArray();
var options = ParseArgs(filteredArgs, jsonDefault: true, allowNamedQuery: true);
using var exactLanguageScope = DbReader.BeginExactQueryLanguageScope(
options.Lang);
if (TryWriteUnsupportedOptionError("goto", cmdArgs, CliFlagSchema.GetAcceptedFlagNamesForCommand("goto"), options.Query))
return CommandExitCodes.UsageError;
if (TryWriteParseError(options, "goto"))
if (TryWriteParseError(options, "goto", options.LanguageValidationError ? jsonOptions : null))
return CommandExitCodes.UsageError;
if (TryWriteInvalidKindFilterError(options, "goto", KnownSymbolKindFilters))
return CommandExitCodes.InvalidArgument;
Expand Down
8 changes: 6 additions & 2 deletions src/CodeIndex/Cli/QueryCommandRunner.Dependencies.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,11 @@ public static int RunImpact(string[] cmdArgs, JsonSerializerOptions jsonOptions)
return CommandExitCodes.UsageError;
}
var options = ParseArgs(cmdArgs, jsonDefault: false, allowNamedQuery: true);
using var exactLanguageScope = DbReader.BeginExactQueryLanguageScope(
options.Lang);
if (TryWriteUnsupportedOptionError("impact", cmdArgs, CliFlagSchema.GetAcceptedFlagNamesForCommand("impact"), options.Query))
return CommandExitCodes.UsageError;
if (TryWriteParseError(options, "impact"))
if (TryWriteParseError(options, "impact", options.LanguageValidationError ? jsonOptions : null))
return CommandExitCodes.UsageError;
if (TryWriteSnippetLinesZeroUnsupportedError(options, "impact"))
return CommandExitCodes.UsageError;
Expand Down Expand Up @@ -464,9 +466,11 @@ public static int RunDeps(string[] cmdArgs, JsonSerializerOptions jsonOptions, C
jsonDefault: false,
validateDefaultSnippetLines: false,
validateDefaultMaxLineWidth: false);
using var exactLanguageScope = DbReader.BeginExactQueryLanguageScope(
options.Lang);
if (TryWriteUnsupportedOptionError("deps", cmdArgs, CliFlagSchema.GetAcceptedFlagNamesForCommand("deps")))
return CommandExitCodes.UsageError;
if (TryWriteParseError(options, "deps"))
if (TryWriteParseError(options, "deps", options.LanguageValidationError ? jsonOptions : null))
return CommandExitCodes.UsageError;
if (TryWriteUnexpectedPositionals("deps", options))
return CommandExitCodes.UsageError;
Expand Down
Loading
Loading