Skip to content

feat(mistral-ai): update model YAMLs [bot]#1740

Open
models-bot[bot] wants to merge 1 commit into
mainfrom
bot/update-mistral-ai-20260710-180058
Open

feat(mistral-ai): update model YAMLs [bot]#1740
models-bot[bot] wants to merge 1 commit into
mainfrom
bot/update-mistral-ai-20260710-180058

Conversation

@models-bot

@models-bot models-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Auto-generated by poc-agent for provider mistral-ai.


Note

Low Risk
Single provider model YAML metadata update with no application code changes.

Overview
Updates ministral-14b-latest catalog metadata to reflect prompt caching: adds cache_read_input_token_cost: 2e-8 under costs and lists prompt_caching in features. No runtime or routing logic changes—pricing and capability flags only.

Reviewed by Cursor Bugbot for commit 29309c9. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

/test-models

@harshiv-26

Copy link
Copy Markdown
Collaborator

Gateway test results

  • Total: 10
  • Passed: 8
  • Failed: 0
  • Validation failed: 2
  • Errored: 0
  • Skipped: 0
  • Success rate: 80.0%
Provider Model Scenarios
mistral-ai ministral-14b-latest success: parallel-tool-call, params:stream, tool-call:stream, tool-call, params, parallel-tool-call:stream, json-output:stream, json-output

validation_failure: structured-output, structured-output:stream
Failures (2)

mistral-ai/ministral-14b-latest — structured-output (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmpi8rg115r/snippet.py", line 43, in <module>
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-mistral-ai/ministral-14b-latest",
    messages=[
        {"role": "system", "content": "Extract the event information as JSON."},
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=False,
)
import json as _json

_content = response.choices[0].message.content
print(_content)

if not _content:
    raise Exception("VALIDATION FAILED: structured-output - response content is empty")

_parsed = _json.loads(_content)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output - unexpected keys present: {set(_parsed.keys())}"
    )

print("VALIDATION: structured-output SUCCESS")
Output
{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "date": {
      "type": "string"
    },
    "participants": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}

mistral-ai/ministral-14b-latest — structured-output:stream (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp313voxgp/snippet.py", line 54, in <module>
    raise Exception(
Exception: VALIDATION FAILED: structured-output stream - unexpected keys present: {'date', 'participants', 'required', 'name', 'title', 'properties', 'additionalProperties', 'type'}
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-mistral-ai/ministral-14b-latest",
    messages=[
        {"role": "system", "content": "Extract the event information as JSON."},
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=True,
)
import json as _json

_accumulated = ""
for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta
        if delta.content is not None:
            _accumulated += delta.content
            print(delta.content, end="", flush=True)

if not _accumulated:
    raise Exception("VALIDATION FAILED: structured-output stream - no content received")

_parsed = _json.loads(_accumulated)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output stream - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output stream - unexpected keys present: {set(_parsed.keys())}"
    )

print("\nVALIDATION: structured-output stream SUCCESS")
Output
{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {},
  "required": [],
  "additionalProperties": true,
  "name": "Science Fair",
  "date": "Friday",
  "participants": ["Alice", "Bob"]
}
Successes (8)

mistral-ai/ministral-14b-latest — parallel-tool-call (success)

Output
Number of parallel tool calls: 2
Function: get_weather
Arguments: {"location": "London"}
Function: get_weather
Arguments: {"location": "Paris"}
VALIDA
... (truncated, 33 chars omitted)

mistral-ai/ministral-14b-latest — params:stream (success)

Output
The capital of France is **Paris**! 🇫🇷

Would you like to know more about it? 😊

mistral-ai/ministral-14b-latest — tool-call:stream (success)

Output
{"location": "London"}
VALIDATION: tool-call stream SUCCESS

mistral-ai/ministral-14b-latest — tool-call (success)

Output
Function: get_weather
Arguments: {"location": "London"}
VALIDATION: tool-call SUCCESS

mistral-ai/ministral-14b-latest — params (success)

Output
The capital of France is **Paris**! 🇫🇷

Would you like to know more about it? 😊

mistral-ai/ministral-14b-latest — parallel-tool-call:stream (success)

Output
{"location": "London"}{"location": "Paris"}
Number of parallel tool calls: 2
VALIDATION: parallel-tool-call stream SUCCESS

mistral-ai/ministral-14b-latest — json-output:stream (success)

Output
{
  "colors": [
    {
      "name": "Deep Sky Blue",
      "hex_code": "#00BFFF"
    },
    {
      "name": "Forest Green",
      "hex_code": "#228B22
... (truncated, 120 chars omitted)

mistral-ai/ministral-14b-latest — json-output (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex_code": "#FF0000"
    },
    {
      "name": "Blue",
      "hex_code": "#0000FF"
    },
    {
   
... (truncated, 92 chars omitted)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant