Menu rel

README.md

@@ -1,88 +1,192 @@
 # OpenVINO™ Model Server
 
-Model Server hosts models and makes them accessible to software components over standard network protocols: a client sends a request to the model server, which performs model inference and sends a response back to the client. Model Server offers many advantages for efficient model deployment:
-- Remote inference enables using lightweight clients with only the necessary functions to perform API calls to edge or cloud deployments.
-- Applications are independent of the model framework, hardware device, and infrastructure.
-- Client applications in any programming language that supports REST or gRPC calls can be used to run inference remotely on the model server.
-- Clients require fewer updates since client libraries change very rarely.
-- Model topology and weights are not exposed directly to client applications, making it easier to control access to the model.
-- Ideal architecture for microservices-based applications and deployments in cloud environments – including Kubernetes and OpenShift clusters.
-- Efficient resource utilization with horizontal and vertical inference scaling.
+**High-performance model serving for Generative AI and classic deep learning — powered by [OpenVINO](https://github.com/openvinotoolkit/openvino) and optimized for Intel hardware.**
 
-![OVMS diagram](docs/ovms_diagram.png)
-
-OpenVINO™ Model Server (OVMS) is a high-performance system for serving models. Implemented in C++ for scalability and optimized for deployment on Intel architectures. It uses the [generative API](https://docs.openvino.ai/2026/model-server/ovms_docs_clients_genai.html) like OpenAI and Cohere, [KServe](https://docs.openvino.ai/2026/model-server/ovms_docs_clients_kfs.html) and [TensorFlow Serving](https://docs.openvino.ai/2026/model-server/ovms_docs_clients_tfs.html) and while applying OpenVINO for inference execution. Inference service is provided via gRPC or REST API, making deploying new algorithms and AI experiments easy.
-
-![OVMS picture](docs/ovms_high_level.png)
-
-The models used by the server can be stored locally, hosted remotely by object storage services or pulled from HuggingFace Hub. For more details, refer to [Preparing Model Repository](https://docs.openvino.ai/2026/model-server/ovms_docs_models_repository.html) and [Deployment](https://docs.openvino.ai/2026/model-server/ovms_docs_deploying_server.html) documentation.
-Model server works inside Docker containers, Bare Metal and in Kubernetes environment.
-
-Start using OpenVINO Model Server with a fast-forward serving example from the [QuickStart guide](https://docs.openvino.ai/2026/model-server/ovms_docs_quick_start_guide.html) or [LLM QuickStart guide](https://docs.openvino.ai/2026/model-server/ovms_docs_llm_quickstart.html).
+[![Apache License](https://img.shields.io/badge/license-Apache%202.0-green.svg)](https://github.com/openvinotoolkit/model_server/blob/main/LICENSE)
+[![Docker Pulls](https://img.shields.io/docker/pulls/openvino/model_server.svg)](https://hub.docker.com/r/openvino/model_server)
+[![GitHub Release](https://img.shields.io/github/v/release/openvinotoolkit/model_server)](https://github.com/openvinotoolkit/model_server/releases)
+[![Platform](https://img.shields.io/badge/platform-Linux%20%7C%20Windows-blue)](https://docs.openvino.ai/2026/model-server/ovms_docs_deploying_server.html)
 
-Read [release notes](https://github.com/openvinotoolkit/model_server/releases) to find out what’s new.
-
-### Key features:
-- **[NEW]** [Speech Generation and Speech Recognition with OpenAI API](https://docs.openvino.ai/2026/model-server/ovms_demos_audio.html)
-- **[NEW]** [Support for AI agents](https://docs.openvino.ai/2026/model-server/ovms_demos_continuous_batching_agent.html)
-- **[NEW]** [Image generation compatible with OpenAI API](https://docs.openvino.ai/2026/model-server/ovms_demos_image_generation.html)
-- Native Windows support. Check updated [deployment guide](https://docs.openvino.ai/2026/model-server/ovms_docs_deploying_server_baremetal.html)
-- [Text Embeddings compatible with OpenAI API](https://docs.openvino.ai/2026/model-server/ovms_demos_embeddings.html)
-- [Reranking compatible with Cohere API](https://docs.openvino.ai/2026/model-server/ovms_demos_rerank.html)
-- [Efficient Text Generation via OpenAI API](https://docs.openvino.ai/2026/model-server/ovms_demos_continuous_batching.html)
-- [Python code execution](docs/python_support/reference.md)
-- [gRPC streaming](docs/streaming_endpoints.md)
-- [MediaPipe graphs serving](docs/mediapipe.md)
-- Model management - including [model versioning](docs/model_version_policy.md) and [model updates in runtime](docs/online_config_changes.md)
-- [Dynamic model inputs](docs/shape_batch_size_and_layout.md)
-- [Directed Acyclic Graph Scheduler](docs/dag_scheduler.md) along with [custom nodes in DAG pipelines](docs/custom_node_development.md)
-- [Metrics](docs/metrics.md) - metrics compatible with Prometheus standard
-- Support for multiple frameworks, such as TensorFlow, PaddlePaddle and ONNX
-- Support for [AI accelerators](./docs/accelerators.md)
+---
 
-Check full list of [features](./docs/features.md)
+## What is OVMS?
 
-**Note:** OVMS has been tested on RedHat, Ubuntu and Windows. 
-Public docker images are stored in:
-- [Dockerhub](https://hub.docker.com/r/openvino/model_server)
-- [RedHat Ecosystem Catalog](https://catalog.redhat.com/software/containers/intel/openvino-model-server/607833052937385fc98515de)
-Binary packages for Linux and Windows are on [Github](https://github.com/openvinotoolkit/model_server/releases)
+OpenVINO Model Server (OVMS) is a production-grade, C++ inference server that exposes ML models over standard network APIs. It serves both **Generative AI** workloads (LLMs, VLMs, image generation, audio) and **classic deep learning** models (object detection, classification, OCR, and more).
 
-## Run OpenVINO Model Server
+- **OpenAI-compatible API** for text generation, embeddings, image generation, and audio
+- **KServe** APIs for classic model inference
+- **Runs anywhere** — Docker, bare metal, Kubernetes/OpenShift, Windows
+- **Intel-optimized** — CPU, GPU, NPU acceleration via OpenVINO
 
-A demonstration on how to use OpenVINO Model Server can be found in our [quick-start guide for vision use case](docs/ovms_quickstart.md) and [LLM text generation](docs/llm/quickstart.md).
+![OVMS diagram](docs/ovms_diagram.png)
 
-Check also other instructions:
+---
 
-[Preparing model repository](https://docs.openvino.ai/2026/model-server/ovms_docs_models_repository.html)
+## Quick Start
+
+### Serve an LLM with OpenAI-compatible API
+
+**On Linux (Docker):**
+```bash
+# Model is downloaded automatically from HuggingFace
+docker run --rm -p 8000:8000 \
+  openvino/model_server:latest \
+  --source_model OpenVINO/Qwen3-4B-int4-ov \
+  --model_repository_path /tmp/models \
+  --rest_port 8000
+```
+> For GPU acceleration, use the `latest-gpu` image tag and pass `--device /dev/dri --group-add $(stat -c '%g' /dev/dri/render* | head -n1)` to expose the Intel GPU device.
+
+**On Windows (binary package):**
+```bat
+mkdir c:\models
+ovms.exe --source_model OpenVINO/Qwen3-4B-int4-ov --model_repository_path c:\models --rest_port 8000
+```
+
+**Query the model:**
+```console
+pip install openai
+```
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
+stream = client.chat.completions.create(
+    model="OpenVINO/Qwen3-4B-int4-ov",
+    messages=[{"role": "user", "content": "What are the 3 main tourist attractions in Paris?"}],
+    stream=True,
+    extra_body={"chat_template_kwargs": {"enable_thinking": False}}
+)
+for chunk in stream:
+    if chunk.choices[0].delta.content:
+        print(chunk.choices[0].delta.content, end="", flush=True)
+```
+> [LLM QuickStart](https://docs.openvino.ai/2026/model-server/ovms_docs_llm_quickstart.html)
+
+
+
+### Serve a Classic Model with KServe API
+
+**Download the model:**
+```console
+curl -L https://huggingface.co/OpenVINO/resnet50-int8-ov/resolve/main/resnet50.bin -O
+curl -L https://huggingface.co/OpenVINO/resnet50-int8-ov/resolve/main/resnet50.xml -O
+```
+
+**On Linux (Docker):**
+```bash
+docker run --rm -d -u $(id -u) -v ${PWD}:/models -p 9000:9000 \
+  openvino/model_server:latest \
+  --model_name resnet --model_path /models/resnet50.xml \
+  --mean "[123.675,116.28,103.53]" --scale "[58.395,57.12,57.375]" --layout "NHWC:NCHW" \
+  --port 9000
+```
+> For GPU acceleration, use the `latest-gpu` image tag and pass `--device /dev/dri --group-add $(stat -c '%g' /dev/dri/render* | head -n1)` to expose the Intel GPU device.
+
+**Windows (binary package):**
+```bat
+ovms --model_name resnet --model_path resnet50.xml --mean "[123.675,116.28,103.53]" --scale "[58.395,57.12,57.375]" --layout "NHWC:NCHW" --port 9000
+```
+
+Run inference with a sample client
+```console
+pip install numpy tritonclient[grpc]
+curl -o image.jpeg https://github.com/openvinotoolkit/model_server/blob/main/demos/common/static/images/bee.jpeg?raw=true
+```
+```python
+import numpy as np
+import tritonclient.grpc as grpcclient
+with open("image.jpeg", "rb") as f:
+    image_bytes = f.read()
+client = grpcclient.InferenceServerClient(url="localhost:9000")
+inputs = [grpcclient.InferInput("image", [1], "BYTES")]
+inputs[0].set_data_from_numpy(np.array([image_bytes], dtype=object))
+outputs = [grpcclient.InferRequestedOutput("output")]
+result = client.infer(model_name="resnet", inputs=inputs, outputs=outputs)
+output = result.as_numpy("output")  # (1, 1000) FP32
+print("Top-1 class index:", int(np.argmax(output[0])))
+```
+
+> [Vision model QuickStart](https://docs.openvino.ai/2026/model-server/ovms_docs_quick_start_guide.html)
 
-[Deployment](https://docs.openvino.ai/2026/model-server/ovms_docs_deploying_server.html)
+---
 
-[Writing client code](https://docs.openvino.ai/2026/model-server/ovms_docs_server_app.html)
+## Features
+
+### Generative AI
+- [LLM text generation](https://docs.openvino.ai/2026/model-server/ovms_demos_continuous_batching.html) — continuous batching, streaming, structured output, speculative decoding
+- [VLM (Vision Language Models)](https://docs.openvino.ai/2026/model-server/ovms_demos_continuous_batching_vlm.html)
+- [AI Agents with MCP servers](https://docs.openvino.ai/2026/model-server/ovms_demos_continuous_batching_agent.html)
+- [Text embeddings](https://docs.openvino.ai/2026/model-server/ovms_demos_embeddings.html) — OpenAI-compatible `/v1/embeddings`
+- [Reranking](https://docs.openvino.ai/2026/model-server/ovms_demos_rerank.html) — Cohere-compatible API
+- [Image generation](https://docs.openvino.ai/2026/model-server/ovms_demos_image_generation.html) — OpenAI-compatible `/v1/images/generations`
+- [Speech recognition and TTS](https://docs.openvino.ai/2026/model-server/ovms_demos_audio.html) — OpenAI-compatible audio API
+- [GGUF model support](https://docs.openvino.ai/2026/model-server/ovms_demos_gguf.html)
+
+### Classic Models & Pipelines
+- TensorFlow, ONNX, PaddlePaddle, OpenVINO IR model formats
+- [MediaPipe graphs](https://docs.openvino.ai/2026/model-server/ovms_docs_mediapipe.html)
+- [Python execution nodes](https://docs.openvino.ai/2026/model-server/ovms_docs_python_support_reference.html)
+- [Dynamic input shapes](https://docs.openvino.ai/2026/model-server/ovms_docs_shape_batch_size_and_layout.html)
+
+### Deployment & Integration
+- [Docker](docs/deploying_server_docker.md), [bare metal (Linux & Windows)](docs/deploying_server_baremetal.md), [Kubernetes / OpenShift](docs/deploying_server_kubernetes.md)
+- [Model repository](https://docs.openvino.ai/2026/model-server/ovms_docs_models_repository.html): local storage, S3, GCS, Azure Blob, HuggingFace Hub
+- [Model versioning](https://docs.openvino.ai/2026/model-server/ovms_docs_model_version_policy.html) and [hot-reload](https://docs.openvino.ai/2026/model-server/ovms_docs_online_config_changes.html)
+- [Prometheus-compatible metrics](https://docs.openvino.ai/2026/model-server/ovms_docs_metrics.html)
+- [gRPC streaming](https://docs.openvino.ai/2026/model-server/ovms_docs_streaming_endpoints.html)
+- [C API](https://docs.openvino.ai/2026/model-server/ovms_docs_c_api.html) for embedding OVMS in native applications
+
+### Hardware Acceleration
+- CPU (x86, including Xeon), Intel integrated and discrete GPU, NPU
+- See [supported accelerators](https://docs.openvino.ai/2026/model-server/ovms_docs_accelerators.html)
+
+[→ Full feature list](https://docs.openvino.ai/2026/model-server/ovms_docs_features.html)
 
-[Demos](https://docs.openvino.ai/2026/model-server/ovms_docs_demos.html)
+---
 
+## Documentation
 
+| Topic | Link |
+|---|---|
+| Deployment | [Deploying the server](https://docs.openvino.ai/2026/model-server/ovms_docs_deploying_server.html) |
+| Model repository | [Preparing models](https://docs.openvino.ai/2026/model-server/ovms_docs_models_repository.html) |
+| Client libraries | [Writing client code](https://docs.openvino.ai/2026/model-server/ovms_docs_server_app.html) |
+| Demos & examples | [Demos](https://docs.openvino.ai/2026/model-server/ovms_docs_demos.html) |
+| Release notes | [GitHub Releases](https://github.com/openvinotoolkit/model_server/releases) |
 
-## References
+---
 
-* [OpenVINO™](https://software.intel.com/en-us/openvino-toolkit)
+## Get the Server
 
-* [ADVANCING GENAI WITH CPU OPTIMIZATION](https://cdrdv2-public.intel.com/864404/vFINAL_Intel%20SLM%20Whitepaper.pdf)
+**Docker images** (recommended):
+```bash
+docker pull openvino/model_server:latest        # Intel CPU
+docker pull openvino/model_server:latest-gpu    # Intel CPU,GPU,NPU
+```
 
-* [Manage deep learning models with OpenVINO Model Server](https://developers.redhat.com/articles/2024/07/03/manage-deep-learning-models-openvino-model-server#)
+- [Docker Hub](https://hub.docker.com/r/openvino/model_server)
+- [Red Hat Ecosystem Catalog](https://catalog.redhat.com/software/containers/intel/openvino-model-server/607833052937385fc98515de)
 
-* [RAG building blocks made easy and affordable with OpenVINO Model Server](https://medium.com/openvino-toolkit/rag-building-blocks-made-easy-and-affordable-with-openvino-model-server-e7b03da5012b)
+**Binary packages** (Linux & Windows): [GitHub Releases](https://github.com/openvinotoolkit/model_server/releases)
 
-* [Simple deployment with KServe API](https://blog.openvino.ai/blog-posts/kserve-api)
+---
 
-* [Benchmarking results](https://docs.openvino.ai/2026/about-openvino/performance-benchmarks.html)
+## Contributing
 
+Contributions are welcome! Please open an issue or pull request on GitHub.  
+See [security policy](security.md) for responsible disclosure.
 
-## Contact
+---
 
-If you have a question, a feature request, or a bug report, feel free to submit a Github issue.
+## References
 
+- [OpenVINO Toolkit](https://software.intel.com/en-us/openvino-toolkit)
+- [Performance benchmarks](https://docs.openvino.ai/2026/about-openvino/performance-benchmarks.html)
+- [GenAI with CPU optimization — Intel whitepaper](https://cdrdv2-public.intel.com/864404/vFINAL_Intel%20SLM%20Whitepaper.pdf)
+- [RAG with OpenVINO Model Server — blog post](https://medium.com/openvino-toolkit/rag-building-blocks-made-easy-and-affordable-with-openvino-model-server-e7b03da5012b)
+- [AIPC turned into a mighty assistant](https://medium.com/openvino-toolkit/ai-pc-turned-into-a-mighty-ai-assistant-with-local-models-and-openvino-model-server-1f41913252c9)
 
 ---
+
 \* Other names and brands may be claimed as the property of others.