Embedding Service
Added in version 3.2.0
The Embedding Service is an ML inference sidecar for the ESS Search Service. It generates 384-dimensional vector embeddings from text, enabling semantic search over Pod content. The service is called by both the Search Service (to embed search queries) and the Ingest Service (to embed content chunks during indexing).
Model
The Embedding Service uses the BAAI/bge-small-en-v1.5 model from the SentenceTransformers ecosystem. The model is baked into the container image at build time; the service does not download models at runtime.
The BGE model distinguishes between query embeddings and document embeddings. When embedding a search query, the model prepends a query-specific prefix to improve retrieval quality. This distinction is controlled by the isQuery parameter in the API.
Model
BAAI/bge-small-en-v1.5
Embedding dimensions
384
Language
English
Max input length
10,000 characters
API Endpoint
The Embedding Service exposes a single endpoint on the main API port (default 8443, mTLS in production):
POST /api/embed
Generates a vector embedding for a text string.
Input
Endpoint
https://enterprise-embedding/api/embed
Method
POST
Content-Type
application/json
Payload
Embed request object
Internal Service
The Embedding Service is an internal ESS service. It is called by the Search and Ingest services over mTLS and is not directly accessible to external clients.
Request Body
text
String
Yes
The text to embed. 1–10,000 characters.
isQuery
Boolean
No
If true, applies the BGE query prefix for improved search retrieval. Default: false.
Example Request
Output
embedding
Array of Number
A 384-dimensional vector of floating-point values.
Example Response
The actual response contains a 384-dimensional vector. The example above is truncated for readability.
Error Responses
422
Validation error. The text field is missing, empty, or exceeds the maximum length.
Health Endpoints
The Embedding Service provides health and metrics endpoints on the management port (default 9000, HTTPS without client certificate verification):
GET /health/live
Liveness probe. Returns {"status": "up"}.
GET /health/ready
Readiness probe. Runs a test inference and returns model information. Returns 503 if the model is not loaded. The readiness result is cached for 30 seconds.
GET /metrics
Prometheus metrics.
Metrics
The Embedding Service exports the following Prometheus metrics:
http_request_duration_seconds
Histogram
Request duration by method, path, and status.
http_requests_total
Counter
Total request count by method, path, and status.
embedding_inference_duration_seconds
Histogram
Model inference duration (custom buckets from 5 ms to 2.5 s).
embedding_model_ready
Gauge
Whether the model is loaded and ready (1 = ready, 0 = not ready).
http_requests_in_flight
Gauge
Current number of in-flight requests.
Degradation Behavior
When the Embedding Service is unavailable, the Search Service degrades to keyword-only mode and the Ingest Service queues affected messages to the dead-letter queue. See Search Service: Search Modes for details.
Startup Time
The Embedding Service has a slow startup due to model loading (up to 2.5 minutes). Kubernetes probes are configured accordingly with an initial delay and extended failure threshold.
Configuration
All configuration is via environment variables.
PORT
Default: 8443
The main API listening port.
MANAGEMENT_PORT
Default: 9000
The health and metrics port.
LOG_LEVEL
Default: info
The log level. Values: debug, info, warning, error, critical.
GUNICORN_WORKERS
Default: 2
The number of Gunicorn worker processes. Each worker loads a copy of the model.
TORCH_NUM_THREADS
Default: 2
The number of PyTorch intra-op threads per worker.
TLS_CERTFILE
Path to the server TLS certificate file. Must be set together with TLS_KEYFILE and TLS_CA_CERTFILE, or all must be unset.
TLS_KEYFILE
Path to the server TLS private key file.
TLS_CA_CERTFILE
Path to the CA certificate for client certificate verification. When TLS is enabled, the Embedding Service enforces mutual TLS (mTLS) on the main API port. The management port uses server-side TLS only (no client certificate required).
Additional Information
Search Service — Overview and architecture.
Ingest Service — How content is indexed using embeddings.
OCR Service — Text extraction from PDFs and images.
Last updated