Add Copilot preference for a chat's custom instructions loading (from all projects or from referenced projects only)

[travkin79]

This PR fixes issue #62 according to the comment #62 (comment)

The PR

• introduces a new setting in the Copilot preferences with currently two options for loading custom instructions in Copilot chats:
  • Load instructions from all projects in the Eclipse workspace (new default, similar to previous behavior)
  • Load only instructions from projects derived from files / folders selected in the chat window (from the resources' parent projects)
• if activated, derives projects only from given resources and passes the corresponding workspace folders to the language server (CopilotLanguageServerConnection), i.e.
  • if no files / folders are selected => no custom instructions are loaded at all
  • if a set of files / folders is selected in the chat window, only these resources' parent projects are checked for custom instructions. Custom instructions from other projects are ignored.
• adds some unit tests

Preference page preview (new group highlighted):

[travkin79]

Hi @iloveeclipse,
I'm not sure, if it's OK if the custom copilot instructions are only loaded when starting a new chat. The set of loaded custom instructions doesn't change in an open chat window when the set of files in the context changes. I think, as a user, I would expect the set of custom instructions to change with the file selection in the chat. WDYT?

[Copilot]

Pull request overview

This PR adds a new chat preference to control how project-level Copilot custom instructions are scoped when starting a chat, so Eclipse can avoid loading unrelated projects’ instruction files and reduce the prompt bloat behind issue #62.

Changes:

• Added a persisted CustomInstructionsChatLoadScope preference with UI labels/defaults for chat instruction loading.
• Derived workspace folders from selected/current resources in ChatView and threaded that list into conversation creation.
• Added enum/helper tests and ResourceUtils coverage for workspace-folder derivation.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/utils/ResourceUtils.java	Adds helper to derive distinct accessible workspace folders from selected resources.
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/preferences/messages.properties	Adds user-facing labels for the new chat instruction scope preference.
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/preferences/Messages.java	Exposes new NLS message keys.
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/preferences/CustomInstructionPreferencePage.java	Adds the new combo box to the preferences UI and persists the selection.
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/preferences/CopilotPreferenceInitializer.java	Sets the default chat instruction scope preference.
com.microsoft.copilot.eclipse.ui/src/com/microsoft/copilot/eclipse/ui/chat/ChatView.java	Derives scoped workspace folders before creating a new conversation.
com.microsoft.copilot.eclipse.ui.test/src/com/microsoft/copilot/eclipse/ui/utils/ResourceUtilsTest.java	Adds tests for workspace-folder derivation behavior.
com.microsoft.copilot.eclipse.ui.test/META-INF/MANIFEST.MF	Adds JUnit params dependency for new parameterized tests.
com.microsoft.copilot.eclipse.core/src/com/microsoft/copilot/eclipse/core/lsp/CopilotLanguageServerConnection.java	Adds a createConversation overload that accepts scoped workspace folders.
com.microsoft.copilot.eclipse.core/src/com/microsoft/copilot/eclipse/core/Constants.java	Defines the new preference key and enum values/default.
com.microsoft.copilot.eclipse.core.test/src/com/microsoft/copilot/eclipse/core/CustomInstructionsChatLoadScopeTest.java	Adds enum serialization/deserialization tests.

[iloveeclipse]

I think, as a user, I would expect the set of custom instructions to change with the file selection in the chat. WDYT?

Correct.

[travkin79]

Hi @iloveeclipse,

I think, as a user, I would expect the set of custom instructions to change with the file selection in the chat. WDYT?

Correct.

I've adapted the code to retrieve custom instructions from selected files on each conversation turn, i.e. each time a new prompt is sent to the LS.

[jdneo]

I have no more comments.

@iloveeclipse could you help validate with this patch, your project setup will work as expected?

[iloveeclipse]

could you help validate with this patch, your project setup will work as expected?

I hope to get some time next week for that.

[jdneo]

@travkin79 Let's wait for the confirmation, thanks.

[travkin79]

could you help validate with this patch, your project setup will work as expected?

I hope to get some time next week for that.

FYI @iloveeclipse: the PR #176 might help you setting up the Eclipse projects when you're going to test this PR.

[iloveeclipse]

the PR #176 might help you setting up the Eclipse projects when you're going to test this PR.

Thanks, but it would be also helpful if you could rebase your PR on top of master, I see there 3 merge conflicts now.

[jdneo]

Sorry we have so many changes recently

[iloveeclipse]

Sorry we have so many changes recently

Not your fault, glad to see progress :-).

Interestingly, with 0.17 + this patch I don't see Copilot crashing anymore on every chat interaction with all instruction files enabled, just sometimes, with the original setup and both preference enabled for all projects / only in context.

I guess context window calculations is changed? What I see, and I believe it is related, that the Copilot reports following token consumption for both cases (same command, same model, new chat started)

I've asked

Which custom instructions files were used?

"Projects from context":
[LS:GitHub Copilot Language Server][Info] [roundMetricsTracker] Turn token usage: 7413 prompt, 38 completion, 7397 cached (cache rate: 99.8%)

"All projects":
[LS:GitHub Copilot Language Server][Info] [roundMetricsTracker] Turn token usage: 138937 prompt, 1795 completion, 138936 cached (cache rate: 100.0%).

One error I saw (with "all projects" enabled, one in 5 times) was:

[LS:GitHub Copilot Language Server][Error]	[default] unhandledRejection: Error: Tests: either GITHUB_PAT, GITHUB_OAUTH_TOKEN, or GITHUB_OAUTH_TOKEN+VSCODE_COPILOT_CHAT_TOKEN must be set unless running from an IS_SCENARIO_AUTOMATION environment. Run "npm run get_token" to get credentials.
    at ddt.tokenProvider (/snapshot/cls-source/dist/main.js:1450:8372)
    at Object.get accessToken [as accessToken] (/snapshot/cls-source/dist/main.js:1319:1047)
    at ddt._handleAuthChangeEvent (/snapshot/cls-source/dist/main.js:788:11061)
    at processTicksAndRejections (node:internal/process/task_queues:103:5)

Not sure how "138937" vs "7413" turn tokens are counted, we have currently 51 projects with 51 identical instruction files, so I would have expected 51x higher consumption, but we see "only" 18x increase. "Completion tokens" counts 38 vs 1795 are closer to the 51x projects.

Anyway, it looks like the change works as expected.