Skip to content

Reset Copilot utility model cache and force one retry when CAPI omits chat fallback #316720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
benvillalobos wants to merge 6 commits into from
Closed

Reset Copilot utility model cache and force one retry when CAPI omits chat fallback #316720

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3693d4a
Reset utility model cache and force retry when fallback flag is missing
benvillalobos May 15, 2026
77ba9ea
Gate utility-model forced retry with a flag and add observability
benvillalobos May 16, 2026
94dfd62
Tighten utility-model retry: scope flag reset to logout, compress com…
benvillalobos May 16, 2026
5cad5ed
Reset utility-model retry budget on each successful fetch
benvillalobos May 16, 2026
bf7088f
Clear completions family map and gate forced retry on env active
benvillalobos May 18, 2026
f6e3227
Coalesce concurrent forced-retry callers via TaskSingler factory gate
benvillalobos May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions extensions/copilot/src/platform/endpoint/node/modelMetadataFetcher.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ export class ModelMetadataFetcher extends Disposable implements IModelMetadataFe
private _completionsFamilyMap: Map<string, IModelAPIResponse[]> = new Map();
private _copilotUtilityModel: IModelAPIResponse | undefined;
private _lastFetchTime: number = 0;
private _hasForcedUtilityModelRetry: boolean = false;
private readonly _taskSingler = new TaskSingler<IModelAPIResponse | undefined | void>();
private _lastFetchError: any;

Expand All @@ -106,6 +107,8 @@ export class ModelMetadataFetcher extends Disposable implements IModelMetadataFe
// Only clear the family map if the copilot token is undefined, as this means the user has logged out and we should clear the models, otherwise we want to keep the old models around until we get a new list
if (this._authService.copilotToken === undefined) {
this._familyMap.clear();
this._copilotUtilityModel = undefined;
this._hasForcedUtilityModelRetry = false;
}

this._completionsFamilyMap.clear();
Expand Down Expand Up @@ -172,6 +175,18 @@ export class ModelMetadataFetcher extends Disposable implements IModelMetadataFe

public async getCopilotUtilityModel(): Promise<IChatModelInformation> {
await this._taskSingler.getOrCreate(ModelMetadataFetcher.ALL_MODEL_KEY, this._fetchModels.bind(this));
if (!this._copilotUtilityModel && this._familyMap.size > 0 && this._envService.isActive) {
// One-shot retry per auth epoch: gated inside the factory so concurrent callers
// coalesce on the same forced fetch via TaskSingler instead of falling through.
await this._taskSingler.getOrCreate(ModelMetadataFetcher.ALL_MODEL_KEY, async () => {
if (this._hasForcedUtilityModelRetry) {
return;
}
this._hasForcedUtilityModelRetry = true;
this._logService.warn('Utility model unset after initial fetch; forcing one refresh');
await this._fetchModels(true);
});
}
Comment thread
benvillalobos marked this conversation as resolved.
Comment on lines 176 to +189
const resolvedModel = this._copilotUtilityModel;
if (!resolvedModel || !isChatModelInformation(resolvedModel)) {
throw new Error(await this._getErrorMessage('Unable to resolve Copilot utility chat model (server did not mark a chat fallback model)'));
Expand Down Expand Up @@ -272,6 +287,8 @@ export class ModelMetadataFetcher extends Disposable implements IModelMetadataFe
}

this._familyMap.clear();
Comment thread
benvillalobos marked this conversation as resolved.
this._completionsFamilyMap.clear();
this._copilotUtilityModel = undefined;

const data: IModelAPIResponse[] = (await response.json()).data;
this._requestLogger.logModelListCall(requestId, requestMetadata, data);
Comment thread
benvillalobos marked this conversation as resolved.
Expand All @@ -290,11 +307,15 @@ export class ModelMetadataFetcher extends Disposable implements IModelMetadataFe
familyMap.get(family)?.push(model);
}
this._lastFetchError = undefined;
if (this._copilotUtilityModel) {
this._hasForcedUtilityModelRetry = false;
}
this._onDidModelRefresh.fire();
} catch (e) {
this._logService.error(e, `Failed to fetch models (${requestId})`);
this._lastFetchError = e;
this._lastFetchTime = 0;
this._hasForcedUtilityModelRetry = false;
}
}

Expand Down
Loading