> For the complete documentation index, see [llms.txt](https://docs.inrupt.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.inrupt.com/ess/services/service-search/embedding-service.md).

# Embedding Service

{% hint style="success" %}
Added in version 3.2.0
{% endhint %}

The Embedding Service is an ML inference sidecar for the ESS [Search Service](/ess/services/service-search.md). It generates 384-dimensional vector embeddings from text, enabling semantic search over Pod content. The service is called by both the [Search Service](/ess/services/service-search.md) (to embed search queries) and the [Ingest Service](/ess/services/service-search/ingest-service.md) (to embed content chunks during indexing).

## Model

The Embedding Service uses the **BAAI/bge-small-en-v1.5** model from the SentenceTransformers ecosystem. The model is baked into the container image at build time; the service does not download models at runtime.

The BGE model distinguishes between query embeddings and document embeddings. When embedding a search query, the model prepends a query-specific prefix to improve retrieval quality. This distinction is controlled by the **`isQuery`** parameter in the API.

| Property             | Value                  |
| -------------------- | ---------------------- |
| Model                | BAAI/bge-small-en-v1.5 |
| Embedding dimensions | 384                    |
| Language             | English                |
| Max input length     | 10,000 characters      |

## API Endpoint

The Embedding Service exposes a single endpoint on the main API port (default 8443, mTLS in production):

### POST /api/embed

Generates a vector embedding for a text string.

#### Input

| Field        | Value                                        |
| ------------ | -------------------------------------------- |
| Endpoint     | **`https://enterprise-embedding/api/embed`** |
| Method       | **`POST`**                                   |
| Content-Type | **`application/json`**                       |
| Payload      | Embed request object                         |

{% hint style="info" %}
**Internal Service**

The Embedding Service is an internal ESS service. It is called by the Search and Ingest services over mTLS and is not directly accessible to external clients.
{% endhint %}

#### Request Body

| Field         | Type    | Required | Description                                                                                      |
| ------------- | ------- | -------- | ------------------------------------------------------------------------------------------------ |
| **`text`**    | String  | Yes      | The text to embed. 1–10,000 characters.                                                          |
| **`isQuery`** | Boolean | No       | If **`true`**, applies the BGE query prefix for improved search retrieval. Default: **`false`**. |

#### Example Request

```http
POST /api/embed HTTP/1.1
Host: enterprise-embedding
Content-Type: application/json

{
  "text": "quarterly revenue report",
  "isQuery": true
}
```

#### Output

| Field           | Type            | Description                                        |
| --------------- | --------------- | -------------------------------------------------- |
| **`embedding`** | Array of Number | A 384-dimensional vector of floating-point values. |

#### Example Response

```http
HTTP/1.1 200 OK
Content-Type: application/json

{
  "embedding": [0.0234, -0.1567, 0.0891, 0.0412, -0.0633]
}
```

{% hint style="info" %}
The actual response contains a 384-dimensional vector. The example above is truncated for readability.
{% endhint %}

#### Error Responses

| Status    | Description                                                                              |
| --------- | ---------------------------------------------------------------------------------------- |
| **`422`** | Validation error. The **`text`** field is missing, empty, or exceeds the maximum length. |

## Health Endpoints

The Embedding Service provides health and metrics endpoints on the management port (default 9000, HTTPS without client certificate verification):

| Endpoint                | Description                                                                                                                                                        |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **`GET /health/live`**  | Liveness probe. Returns **`{"status": "up"}`**.                                                                                                                    |
| **`GET /health/ready`** | Readiness probe. Runs a test inference and returns model information. Returns **`503`** if the model is not loaded. The readiness result is cached for 30 seconds. |
| **`GET /metrics`**      | Prometheus metrics.                                                                                                                                                |

### Metrics

The Embedding Service exports the following Prometheus metrics:

| Metric                                     | Type      | Description                                                       |
| ------------------------------------------ | --------- | ----------------------------------------------------------------- |
| **`http_request_duration_seconds`**        | Histogram | Request duration by method, path, and status.                     |
| **`http_requests_total`**                  | Counter   | Total request count by method, path, and status.                  |
| **`embedding_inference_duration_seconds`** | Histogram | Model inference duration (custom buckets from 5 ms to 2.5 s).     |
| **`embedding_model_ready`**                | Gauge     | Whether the model is loaded and ready (1 = ready, 0 = not ready). |
| **`http_requests_in_flight`**              | Gauge     | Current number of in-flight requests.                             |

## Degradation Behavior

When the Embedding Service is unavailable, the Search Service degrades to keyword-only mode and the Ingest Service queues affected messages to the dead-letter queue. See [Search Service: Search Modes](/ess/services/service-search.md#search-modes) for details.

{% hint style="info" %}
**Startup Time**

The Embedding Service has a slow startup due to model loading (up to 2.5 minutes). Kubernetes probes are configured accordingly with an initial delay and extended failure threshold.
{% endhint %}

## Configuration

All configuration is via environment variables.

#### PORT

Default: **`8443`**

The main API listening port.

#### MANAGEMENT\_PORT

Default: **`9000`**

The health and metrics port.

#### LOG\_LEVEL

Default: **`info`**

The log level. Values: **`debug`**, **`info`**, **`warning`**, **`error`**, **`critical`**.

#### GUNICORN\_WORKERS

Default: **`2`**

The number of Gunicorn worker processes. Each worker loads a copy of the model.

#### TORCH\_NUM\_THREADS

Default: **`2`**

The number of PyTorch intra-op threads per worker.

#### TLS\_CERTFILE

Path to the server TLS certificate file. Must be set together with [**`TLS_KEYFILE`**](#tls_keyfile) and [**`TLS_CA_CERTFILE`**](#tls_ca_certfile), or all must be unset.

#### TLS\_KEYFILE

Path to the server TLS private key file.

#### TLS\_CA\_CERTFILE

Path to the CA certificate for client certificate verification. When TLS is enabled, the Embedding Service enforces mutual TLS (mTLS) on the main API port. The management port uses server-side TLS only (no client certificate required).

## Additional Information

* [Search Service](/ess/services/service-search.md) — Overview and architecture.
* [Ingest Service](/ess/services/service-search/ingest-service.md) — How content is indexed using embeddings.
* [OCR Service](/ess/services/service-search/ocr-service.md) — Text extraction from PDFs and images.