-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Call schema on inputSchema validation failure #1160
Copy link
Copy link
Open
Labels
P3Nice to haves, rare edge casesNice to haves, rare edge casesenhancementRequest for a new feature that's not currently supportedRequest for a new feature that's not currently supportedhelp wantedContributions especially welcome if you have a good knowledge of the codebase and languageContributions especially welcome if you have a good knowledge of the codebase and languageneeds decisionIssue is actionable, needs maintainer decision on whether to implementIssue is actionable, needs maintainer decision on whether to implementv2Ideas, requests and plans for v2 of the SDK which will incorporate major changes and fixesIdeas, requests and plans for v2 of the SDK which will incorporate major changes and fixes
Metadata
Metadata
Assignees
Labels
P3Nice to haves, rare edge casesNice to haves, rare edge casesenhancementRequest for a new feature that's not currently supportedRequest for a new feature that's not currently supportedhelp wantedContributions especially welcome if you have a good knowledge of the codebase and languageContributions especially welcome if you have a good knowledge of the codebase and languageneeds decisionIssue is actionable, needs maintainer decision on whether to implementIssue is actionable, needs maintainer decision on whether to implementv2Ideas, requests and plans for v2 of the SDK which will incorporate major changes and fixesIdeas, requests and plans for v2 of the SDK which will incorporate major changes and fixes
Is your feature request related to a problem? Please describe.
Currently, it's pretty hard to tie in custom logic to input schema validation. If a tool is invoked with the wrong input schema, I would like to be able to add logging logic to track this. This would be helpful for observability of my tool-calls and to better tune the tool names / params / descriptions to help with LLM usage.
Describe the solution you'd like
It would be ideal if we can specify a callback to call on input schema validation failures.
Describe alternatives you've considered
One option is to not pass in inputSchema and just completely handle input schema validation in the
handler. This seems a bit unideal when the SDK already gives a more officialinputSchemaarg.Another option is by intercepting the response downstream and "detecting" it's an inputSchema validation error. This also seems a bit hacky to do and at the wrong layer.