Qwen3 Reranker 4B is an Apache-2.0 multilingual text reranking model for retrieval, RAG, code retrieval, and cross-lingual search.
Observed working set on a supported GPU.
After the last request a backend stays warm on its GPU for about 15 minutes, then frees the GPU. The next request triggers a fresh cold start.
POST to the native route with the shown model field; the playground below generates a full payload.
/v1/inference/qwen-qwen3-reranker-4bPOSTqwen-qwen3-reranker-4bmodel_version: vllm-0-21-0-cuda13-rerankgpu_type: B200region: us-central1Terminal-ready smoke test for this selected target.
set -euo pipefail
# Forge API smoke test
# Forge selected target: route=/v1/models/qwen-qwen3-reranker-4b/inference-routes model=qwen-qwen3-reranker-4b version=vllm-0-21-0-cuda13-rerank gpu=B200 region=us-central1
FORGE_API_BASE=${FORGE_API_BASE:-'https://YOUR_FORGE_HOST'}
case "${FORGE_API_KEY:-}" in
""|replace-with-your-forge-api-key)
echo 'Set FORGE_API_KEY to a real Forge API key before running this snippet; browser SSO sessions are not sent to copied curl or SDK clients.' >&2
exit 1
;;
esac
forge_api_url() {
endpoint="$1"
base="${FORGE_API_BASE%/}"
case "$base:$endpoint" in
*/v1:/v1|*/v1:/v1/*|*/v1:/v1\?*) printf '%s%s\n' "$base" "${endpoint#/v1}" ;;
*) printf '%s%s\n' "$base" "$endpoint" ;;
esac
}
curl -sS --fail-with-body "$(forge_api_url '/v1/models/qwen-qwen3-reranker-4b/inference-routes?model_version=vllm-0-21-0-cuda13-rerank&gpu_type=B200®ion=us-central1')" \
--max-time "${FORGE_REQUEST_TIMEOUT_SECONDS:-600}" \
-H "Authorization: Bearer ${FORGE_API_KEY}" | \
python3 -m json.toolNative route check · Best for model-specific payloads; run the route check first, then copy the schema-specific request from the playground.
Copied snippets include gpu_type and region, so the first request targets this verified GPU and region. Remove those fields to let Forge choose another compatible target.
8 free GPUs · Live capacity for B200 in us-central1.
https://YOUR_FORGE_HOST/v1/inference/qwen-qwen3-reranker-4bClient auth: Set FORGE_API_KEY to a real Forge API key before running copied curl, fetch, or SDK snippets. Browser SSO only authenticates this web session.
Open AccountAuthorization: Bearer $FORGE_API_KEYexport FORGE_API_BASE='https://YOUR_FORGE_HOST'
export FORGE_API_KEY="${FORGE_API_KEY:-replace-with-your-forge-api-key}"
export FORGE_REQUEST_TIMEOUT_SECONDS="${FORGE_REQUEST_TIMEOUT_SECONDS:-600}"
export FORGE_API_ROUTE='/v1/inference/qwen-qwen3-reranker-4b'
export MODEL_OR_FAMILY_SLUG='qwen-qwen3-reranker-4b'
export FORGE_MODEL_VERSION='vllm-0-21-0-cuda13-rerank'
export FORGE_GPU_TYPE='B200'
export FORGE_REGION='us-central1'Copy these values into a local .env file when moving the selected target into an app or SDK client.
# Forge selected target: route=/v1/inference/qwen-qwen3-reranker-4b model=qwen-qwen3-reranker-4b version=vllm-0-21-0-cuda13-rerank gpu=B200 region=us-central1
FORGE_API_BASE="https://YOUR_FORGE_HOST"
FORGE_API_ROUTE="/v1/inference/qwen-qwen3-reranker-4b"
FORGE_API_KEY="replace-with-your-forge-api-key"
FORGE_REQUEST_TIMEOUT_SECONDS="600"
MODEL_OR_FAMILY_SLUG="qwen-qwen3-reranker-4b"
FORGE_MODEL_VERSION="vllm-0-21-0-cuda13-rerank"
FORGE_GPU_TYPE="B200"
FORGE_REGION="us-central1"Add these rules before replacing the placeholder API key so local Forge secrets stay out of commits while .env.example can remain tracked.
# Forge local API secrets
.env
.env.*
!.env.examplehttps://YOUR_FORGE_HOST/v1/models/qwen-qwen3-reranker-4b/run-estimate?model_version=vllm-0-21-0-cuda13-rerank&gpu_type=B200®ion=us-central1set -euo pipefail
# Forge selected target: route=/v1/inference/qwen-qwen3-reranker-4b model=qwen-qwen3-reranker-4b version=vllm-0-21-0-cuda13-rerank gpu=B200 region=us-central1
FORGE_API_BASE=${FORGE_API_BASE:-'https://YOUR_FORGE_HOST'}
export MODEL_OR_FAMILY_SLUG=${MODEL_OR_FAMILY_SLUG:-'qwen-qwen3-reranker-4b'}
export FORGE_MODEL_VERSION=${FORGE_MODEL_VERSION:-'vllm-0-21-0-cuda13-rerank'}
export FORGE_GPU_TYPE=${FORGE_GPU_TYPE:-'B200'}
export FORGE_REGION=${FORGE_REGION:-'us-central1'}
case "${FORGE_API_KEY:-}" in
""|replace-with-your-forge-api-key)
echo 'Set FORGE_API_KEY to a real Forge API key before running this snippet; browser SSO sessions are not sent to copied curl or SDK clients.' >&2
exit 1
;;
esac
forge_api_url() {
endpoint="$1"
base="${FORGE_API_BASE%/}"
case "$base:$endpoint" in
*/v1:/v1|*/v1:/v1/*|*/v1:/v1\?*) printf '%s%s\n' "$base" "${endpoint#/v1}" ;;
*) printf '%s%s\n' "$base" "$endpoint" ;;
esac
}
reliability_path="$(python3 -c 'import os
from urllib.parse import quote, urlencode
model = os.environ.get("MODEL_OR_FAMILY_SLUG", "").strip()
if not model:
raise SystemExit("Set MODEL_OR_FAMILY_SLUG from search or route finder output before checking reliability.")
params = {}
model_version = os.environ.get("FORGE_MODEL_VERSION", "").strip()
if model_version:
params["model_version"] = model_version
gpu_type = os.environ.get("FORGE_GPU_TYPE", "").strip()
if gpu_type:
params["gpu_type"] = gpu_type
region = os.environ.get("FORGE_REGION", "").strip()
if region:
params["region"] = region
path = "/v1/models/" + quote(model, safe="") + "/reliability"
if params:
path += "?" + urlencode(params)
print(path)')"
curl -sS --fail-with-body "$(forge_api_url "$reliability_path")" \
--max-time "${FORGE_REQUEST_TIMEOUT_SECONDS:-600}" \
-H "Authorization: Bearer ${FORGE_API_KEY}" | \
python3 -c 'import json, shlex, sys
payload = json.load(sys.stdin)
print(
f"{payload.get('\''slug'\'')} reliability={payload.get('\''reliability_status'\'')} "
f"supported={payload.get('\''supported_rows'\'', 0)}/{payload.get('\''total_rows'\'', 0)}"
)
filters = payload.get("filters") or {}
if filters:
print("filters: " + ", ".join(f"{key}={value}" for key, value in filters.items()))
def describe_target(target):
details = []
request_ms = target.get("request_ms_p50") or target.get("request_ms")
if request_ms is not None:
details.append(f"p50={request_ms}ms")
warm_cost = target.get("estimated_warm_request_cost_usd")
if warm_cost is not None:
details.append(f"warm_cost_usd={warm_cost}")
elif target.get("cost_per_gpu_hour_usd") is not None:
details.append(f"gpu_hour_usd={target['\''cost_per_gpu_hour_usd'\'']}")
success_rate = target.get("observed_success_rate")
if isinstance(success_rate, (int, float)):
details.append(f"success={success_rate:.0%}")
return ", ".join(details) or target.get("status") or "supported"
exports = {}
for label, key in (
("fastest supported", "fastest_supported_target"),
("lowest-cost supported", "lowest_cost_supported_target"),
):
target = payload.get(key) or {}
gpu_type = target.get("gpu_type")
if not gpu_type:
continue
identity = (str(gpu_type), str(target.get("region") or ""))
exports.setdefault(identity, {"labels": [], "target": target})["labels"].append(label)
if not exports:
print("No supported GPU/region target returned.", file=sys.stderr)
print(json.dumps({
"status_counts": payload.get("status_counts", {}),
"failure_reason_counts": payload.get("failure_reason_counts", {}),
}, indent=2))
raise SystemExit(1)
for (gpu_type, region), entry in exports.items():
assignments = [f"FORGE_GPU_TYPE={shlex.quote(gpu_type)}"]
if region:
assignments.append(f"FORGE_REGION={shlex.quote(region)}")
labels = " + ".join(entry["labels"])
details = describe_target(entry["target"])
print(f"export {'\'' '\''.join(assignments)} # {labels}: {details}")'https://YOUR_FORGE_HOST/v1/model-families/qwen-qwen3-reranker-4b/status?version=vllm-0-21-0-cuda13-rerankset -euo pipefail
# Forge selected target: route=/v1/inference/qwen-qwen3-reranker-4b model=qwen-qwen3-reranker-4b version=vllm-0-21-0-cuda13-rerank gpu=B200 region=us-central1
FORGE_API_BASE=${FORGE_API_BASE:-'https://YOUR_FORGE_HOST'}
export MODEL_OR_FAMILY_SLUG=${MODEL_OR_FAMILY_SLUG:-'qwen-qwen3-reranker-4b'}
export FORGE_MODEL_VERSION=${FORGE_MODEL_VERSION:-'vllm-0-21-0-cuda13-rerank'}
export FORGE_GPU_TYPE=${FORGE_GPU_TYPE:-'B200'}
export FORGE_REGION=${FORGE_REGION:-'us-central1'}
export FORGE_KEEP_WARM=${FORGE_KEEP_WARM:-false}
case "${FORGE_API_KEY:-}" in
""|replace-with-your-forge-api-key)
echo 'Set FORGE_API_KEY to a real Forge API key before running this snippet; browser SSO sessions are not sent to copied curl or SDK clients.' >&2
exit 1
;;
esac
forge_api_url() {
endpoint="$1"
base="${FORGE_API_BASE%/}"
case "$base:$endpoint" in
*/v1:/v1|*/v1:/v1/*|*/v1:/v1\?*) printf '%s%s\n' "$base" "${endpoint#/v1}" ;;
*) printf '%s%s\n' "$base" "$endpoint" ;;
esac
}
runtime_start_path="$(python3 -c 'import os
from urllib.parse import quote
model = os.environ.get("MODEL_OR_FAMILY_SLUG", "").strip()
if not model:
raise SystemExit("Set MODEL_OR_FAMILY_SLUG from the model picker output")
print("/v1/model-families/" + quote(model, safe="") + "/start")')"
python3 -c 'import json, os
def env_value(name):
value = os.environ.get(name, "").strip()
return value or None
payload = {}
version = env_value("FORGE_MODEL_VERSION")
if version:
payload["version"] = version
gpu_type = env_value("FORGE_GPU_TYPE")
if gpu_type:
payload["gpu_type"] = gpu_type
region = env_value("FORGE_REGION")
if region:
payload["region"] = region
keep_warm = env_value("FORGE_KEEP_WARM")
payload["run_until_stopped"] = (keep_warm or "").lower() in {"1", "true", "yes", "on"}
print(json.dumps(payload))' | \
curl -sS --fail-with-body "$(forge_api_url "$runtime_start_path")" \
--max-time "${FORGE_REQUEST_TIMEOUT_SECONDS:-600}" \
-X POST \
-H "Authorization: Bearer ${FORGE_API_KEY}" \
-H "Content-Type: application/json" \
-d @- | \
python3 -c 'import json, sys
payload = json.load(sys.stdin)
slug = payload.get("slug") or "runtime"
gpu_type = payload.get("gpu_type") or "scheduler-selected GPU"
region = payload.get("region") or "scheduler-selected region"
startup_ms = payload.get("startup_ms")
state = "cold-started" if payload.get("was_cold_start") else "already warm"
suffix = f"; startup_ms={startup_ms}" if startup_ms is not None else ""
print(f"{slug} {state} on {gpu_type} in {region}{suffix}; keep_warm={payload.get('\''keep_warm'\'')}")'Pick a verified target for repeatable runs. Failed or pending details appear on the status hover.
5 verified targets
Lowest warm model time among verified targets: 72 ms p50 warm model time across 20 samples.
Lowest estimated GPU price among verified targets: $1.80/GPU-hr; 170 ms p50 warm model time across 20 samples.
| GPU | Region | Status | VRAM | Cold start | Model time | Relative | Tokens/s | Est. $/GPU-hr | Target |
|---|---|---|---|---|---|---|---|---|---|
| B200 | us-central1 | works | 125.2 GB | 6m 23s | 168 msp50 warm model time p95 169 ms · 10 samples | 43% · -57% | 512 | $7.15 | Use in playground |
| B300fastest | uk-south1 | works | 187.9 GB | 1m 15s | 72 msp50 warm model time p95 85 ms · p99 86 ms · 20 samples | 100% | 1075 | $7.85 | Use in playground |
| H100 | — | not probed | — | — | — | — | — | — | — |
| H200 | eu-north2 | works | 97.9 GB | 6m 21s | 110 msp50 warm model time p95 113 ms · p99 115 ms · 20 samples | 65% · -35% | 768 | $4.50 | Use in playground |
| L40S | eu-north1 | works | 30.9 GB | 2m 5s | 112 msp50 warm model time p95 120 ms · 10 samples | 64% · -36% | 768 | $1.82 | Use in playground |
| RTX6000 | us-central1 | works | 66.5 GB | 57.6s | 170 msp50 warm model time 20 samples | 42% · -58% | 506 | $1.80 | Use in playground |
Model time uses the p50 warm model-reported execution time when available, then falls back to the latest probe time; p95/p99 and sample count appear when there is enough probe history. Cold start excludes the first (uncached) run. VRAM is the peak GPU memory seen during the probe. Relative compares each row's model time to the highlighted baseline (fastest row by default; hover any row to re-root). The fastest chip marks only verified supported GPU-region rows. Estimated on-demand GPU price (Nebius pay-as-you-go); shown for performance/price comparison. Configured minimum GPU memory: 48 GB.
/v1/inference/qwen-qwen3-reranker-4bModelqwen-qwen3-reranker-4bVersionvllm-0-21-0-cuda13-rerankGPUautomaticRegionautomaticSnippet target: qwen-qwen3-reranker-4b version vllm-0-21-0-cuda13-rerank using scheduler-selected GPU/region.
Client auth: Set FORGE_API_KEY to a real Forge API key before running copied curl, fetch, or SDK snippets. Browser SSO only authenticates this web session.
Open AccountFORGE_API_BASE=${FORGE_API_BASE:-'https://YOUR_FORGE_HOST'}
case "${FORGE_API_KEY:-}" in
""|replace-with-your-forge-api-key)
echo 'Set FORGE_API_KEY to a real Forge API key before running this snippet; browser SSO sessions are not sent to copied curl or SDK clients.' >&2
exit 1
;;
esac
forge_api_url() {
endpoint="$1"
base="${FORGE_API_BASE%/}"
case "$base:$endpoint" in
*/v1:/v1|*/v1:/v1/*|*/v1:/v1\?*) printf '%s%s\n' "$base" "${endpoint#/v1}" ;;
*) printf '%s%s\n' "$base" "$endpoint" ;;
esac
}
forge_print_response() {
response_file="$1"
if [ ! -s "$response_file" ]; then
printf '%s\n' '(empty response)'
return 0
fi
if command -v python3 >/dev/null 2>&1; then
python3 -m json.tool "$response_file" 2>/dev/null || cat "$response_file"
else
cat "$response_file"
fi
}
response_file="$(mktemp)"
if curl -sS --fail-with-body "$(forge_api_url '/v1/inference/qwen-qwen3-reranker-4b')" \
--max-time "${FORGE_REQUEST_TIMEOUT_SECONDS:-600}" \
-H "Authorization: Bearer ${FORGE_API_KEY}" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-qwen3-reranker-4b",
"query": "How do shared model caches reduce inference cold starts?",
"top_n": 3,
"documents": [
"A shared cache keeps downloaded weights and runtime artifacts on durable storage so later pods can reuse them instead of hydrating from scratch.",
"Cold starts are usually dominated by frontend CSS parsing and browser layout.",
"Rerankers compare a query with candidate passages and assign relevance scores for retrieval pipelines."
],
"model_version": "vllm-0-21-0-cuda13-rerank"
}' \
-o "$response_file"; then
forge_print_response "$response_file"
status=$?
rm -f "$response_file"
(exit "$status")
else
status=$?
cat "$response_file" >&2
rm -f "$response_file"
(exit "$status")
fiCopy setup before the request when moving this snippet into a fresh shell. The default 600 second timeout is intentional for GPU cold starts and can be overridden with FORGE_REQUEST_TIMEOUT_SECONDS.
export FORGE_API_BASE='https://YOUR_FORGE_HOST'
export FORGE_API_KEY="${FORGE_API_KEY:-replace-with-your-forge-api-key}"
export FORGE_REQUEST_TIMEOUT_SECONDS="${FORGE_REQUEST_TIMEOUT_SECONDS:-600}"Copy these values into a local .env file when moving the selected target into an app or SDK client.
# Forge selected target: route=/v1/inference/qwen-qwen3-reranker-4b model=qwen-qwen3-reranker-4b version=vllm-0-21-0-cuda13-rerank
FORGE_API_BASE="https://YOUR_FORGE_HOST"
FORGE_API_ROUTE="/v1/inference/qwen-qwen3-reranker-4b"
FORGE_API_KEY="replace-with-your-forge-api-key"
FORGE_REQUEST_TIMEOUT_SECONDS="600"
MODEL_OR_FAMILY_SLUG="qwen-qwen3-reranker-4b"
FORGE_MODEL_VERSION="vllm-0-21-0-cuda13-rerank"Run a request to see output here.
Run a dedicated, autoscaling endpoint in your own Nebius account. The endpoint runs under your account and billing — Forge just pre-fills the configuration for you.
Opens the Nebius Console with the image pre-filled for Qwen3 Reranker 4B (Forge version vLLM 0.21.0 CUDA 13 Rerank).
Prefer to create the endpoint from the CLI, or self-manage the container image? Use the commands below.
The image is hosted on cr.eu-north1.nebius.cloud; you may need registry credentials in the Console form. The CLI below includes placeholders.
The links use Forge’s eu-north1 Nebius Container Registry mirror. If your project can’t pull that private mirror, add pull credentials or a registry secret.
# Runs in YOUR Nebius account (you own + pay for the endpoint).
# platform/preset must exist in your project — list them with:
# nebius compute platform list
export ENDPOINT_NAME="qwen-qwen3-reranker-4b-vllm-cuda13-private"
export AUTH_TOKEN=$(openssl rand -hex 32)
export SUBNET_ID=$(nebius vpc subnet list --format jsonpath='{.items[0].metadata.id}')
export REGISTRY_USERNAME="YOUR_REGISTRY_USERNAME"
export REGISTRY_PASSWORD="YOUR_REGISTRY_PASSWORD"
# Note: the --image above points at Forge's regional Nebius CR mirror.
# Serverless AI can pull Container Registry images without credentials
# only when the image is public or in the same project. For a private
# mirror in another project, provide pull credentials or a MysteryBox
# registry secret with REGISTRY_USERNAME and REGISTRY_PASSWORD.
nebius ai endpoint create \
--name "$ENDPOINT_NAME" \
--image "cr.eu-north1.nebius.cloud/e00h91c5sa606xfwpj/models/vllm-vllm-openai:v0.22.0@sha256:0fec7ec5f3e6bc168e54899935fb0557da908a4832a1dbc88e2debcf2f889416" \
--registry-username "$REGISTRY_USERNAME" \
--registry-password "$REGISTRY_PASSWORD" \
--container-port 8000 \
--auth token \
--token "$AUTH_TOKEN" \
--subnet-id "$SUBNET_ID"
export ENDPOINT_ID=$(nebius ai endpoint get-by-name --name "$ENDPOINT_NAME" --format jsonpath='{.metadata.id}')
nebius ai endpoint get "$ENDPOINT_ID"Need a throughput- and cost-optimized build tuned for specific Nebius GPUs? Nebius Token Factory is coming soon — contact your Nebius account team for early access.