Hosting DAB on OpenShift

Hosting DAB on OpenShift#

This guide runs a Data API instance as a Data API Builder (DAB) container on OpenShift while Querona stays the data source and the source of the DAB configuration. The container fetches its configuration from Querona when it starts, keeps its own copy, and a scheduled check restarts the deployment when Querona reports a changed configuration. Everything below also applies to plain Kubernetes; only the Route and the oc commands are OpenShift-specific.

For the endpoint contract that the manifests rely on, see Configuration endpoint for a separately hosted DAB.

How it fits together#

        flowchart LR
    Q[Querona<br/>configuration endpoint] -->|GET configuration<br/>Authorization: ApiKey| I[Init container<br/>writes dab-config files]
    I --> D[DAB container<br/>serves REST / GraphQL / MCP]
    D -->|TDS, login from the Secret| Q
    C[CronJob<br/>If-None-Match check] -->|304 unchanged / 200 changed| Q
    C -->|hash changed: rollout| D

  • The init container asks Querona for the configuration and writes the files into a volume shared with the DAB container. It runs on every pod start, so a restarted pod always starts from the current configuration.

  • The DAB container is the unmodified Microsoft image. It reads the files from the volume and connects to Querona over TDS with the login you inject through the Secret.

  • The CronJob asks Querona whether the configuration changed, using the hash the deployment currently runs with. When it did, the job writes the new hash into the deployment’s pod template, which is what triggers a rolling restart.

Prerequisites#

In Querona:

  1. A Data API instance (Administer ‣ Data API Instances). Its authentication option can be Anonymous or JWT; both work with an external host. Whether the instance is also started locally does not matter.

  2. The instance’s API key, copied from the instance details. It is the only Querona credential the container needs, and it identifies the instance - the configuration endpoint takes no instance name.

  3. A login the container connects with over TDS. The configuration was generated from the permissions of the instance’s user (or role), so the login you inject must be that user, or a member of that role, for the published endpoints to work. Querona does not put this login into the configuration; you provide it through the Secret below.

  4. Network reachability from the cluster to the Querona TDS port (default 4433) and, for JWT instances, to the Querona API URL (DAB validates tokens against it).

  5. TLS trust. The generated configuration connects with Encrypt=Strict or Encrypt=Mandatory and TrustServerCertificate=False, so the container must trust the certificate Querona presents on TDS (and on the API URL). Configure a certificate on the node (the SSL thumbprint instance parameter): a node without one presents a generated fallback certificate that no container can trust. If the certificate is not issued by a public authority, mount your CA bundle and point SSL_CERT_FILE at it - both .NET and Python honour that variable on Linux.

  6. For a JWT instance, HTTPS on the Querona API. DAB validates tokens against the API URL, and while HTTPS is off that URL is http://localhost, unreachable from a pod; the configuration endpoint refuses to render such an instance and says so. Anonymous instances have no such requirement.

On the cluster: a project you can deploy to, and - for the refresh job - permission to patch the deployment (RoleBinding below).

1. Secret#

Three values, all yours: the instance API key, and the login and password the container connects with over TDS.

apiVersion: v1
kind: Secret
metadata:
  name: dab-sales-api
type: Opaque
stringData:
  QUERONA_API_KEY: "<api key copied from the instance>"
  QUERONA_DAB_LOGIN: "sales_reader"
  QUERONA_DAB_PASSWORD: "<password>"

The generated connection-string carries the placeholders @env('QUERONA_DAB_LOGIN') and @env('QUERONA_DAB_PASSWORD'); DAB substitutes them from the container environment when it loads the configuration. Keep the variable names exactly as shown. Each placeholder sits inside double quotes, so the password may contain semicolons, equals signs and spaces; it must not contain a double quote.

Each container below takes only the values it uses: the init container and the refresh job the API key, the DAB container the login and password. The API key never reaches the pod that serves traffic.

2. The sync script#

One script serves both the init container and the refresh job. It needs only Python 3 - no extra packages - so it runs on the stock ubi9/python-311 image.

#!/usr/bin/env python3
"""Fetch a Data API instance configuration from Querona and keep a local copy of it.

Exit codes: 0 = unchanged, 3 = changed (files written), 1 = error (existing files untouched).
"""
import hashlib
import json
import os
import sys
import urllib.error
import urllib.request

url = os.environ["QUERONA_URL"].rstrip("/") + "/api/1.0/dataapiinstances/configuration"
api_key = os.environ["QUERONA_API_KEY"]
out_dir = sys.argv[1] if len(sys.argv) > 1 else "."
hash_file = os.path.join(out_dir, ".dab-config-hash")

known_hash = os.environ.get("KNOWN_HASH") or (open(hash_file).read().strip() if os.path.exists(hash_file) else "")

request = urllib.request.Request(url, headers={"Authorization": "ApiKey " + api_key})
if known_hash:
    request.add_header("If-None-Match", '"' + known_hash + '"')

try:
    with urllib.request.urlopen(request, timeout=60) as response:
        body = json.load(response)
except urllib.error.HTTPError as error:
    if error.code == 304:
        print("unchanged " + known_hash)
        sys.exit(0)
    print("Querona answered HTTP %d" % error.code, file=sys.stderr)
    sys.exit(1)

if not body.get("success"):
    for problem in body.get("errors") or []:
        print(problem.get("message"), file=sys.stderr)
    sys.exit(1)

data = body["data"]
if data["hash"] == known_hash:
    # The cache period had expired, so the answer was generated again - but nothing changed.
    print("unchanged " + known_hash)
    sys.exit(0)

os.makedirs(out_dir, exist_ok=True)
for entry in data["files"]:
    name = entry["name"]
    if os.path.basename(name) != name:
        print("refusing path-like file name: " + name, file=sys.stderr)
        sys.exit(1)
    content = entry["content"].encode("utf-8")
    if hashlib.sha256(content).hexdigest().upper() != entry["sha256"].upper():
        print("checksum mismatch for " + name, file=sys.stderr)
        sys.exit(1)
    temporary = os.path.join(out_dir, name + ".tmp")
    with open(temporary, "wb") as handle:
        handle.write(content)
    os.replace(temporary, os.path.join(out_dir, name))

with open(hash_file, "w") as handle:
    handle.write(data["hash"])
print("changed " + data["hash"])
sys.exit(3)

Ship it to the cluster as a ConfigMap so both consumers mount the same file:

oc create configmap dab-config-sync --from-file=dab-config-sync.py

What the script does, in order: sends the stored hash as If-None-Match; treats 304 as “nothing to do”; on 200 checks the envelope’s success flag and compares the returned hash with the stored one - a 200 with the same hash only means the cache period had expired - then verifies every file’s sha256, refuses any file name that is not a plain name, writes each file atomically, and finally stores the new hash. An error leaves the previous files in place, so a pod that cannot reach Querona at the moment of a restart still starts from its last good configuration - provided the volume survived, which it does not for an emptyDir; use a small PersistentVolumeClaim instead if that matters to you.

3. Deployment, Service and Route#

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dab-sales-api
spec:
  replicas: 2
  selector:
    matchLabels:
      app: dab-sales-api
  template:
    metadata:
      labels:
        app: dab-sales-api
      annotations:
        querona.com/config-hash: ""          # maintained by the CronJob; a change rolls the pods
    spec:
      volumes:
        - name: config
          emptyDir: {}
        - name: sync
          configMap:
            name: dab-config-sync
            defaultMode: 0555
      initContainers:
        - name: fetch-config
          image: registry.access.redhat.com/ubi9/python-311
          # The script exits 3 after writing new files, which is a successful start here. On an error it
          # exits 1: start from the files already on the volume when there are any, otherwise fail.
          command:
            - /bin/sh
            - -c
            - |
              /scripts/dab-config-sync.py /config
              status=$?
              if [ "$status" -eq 0 ] || [ "$status" -eq 3 ]; then exit 0; fi
              if [ -f /config/dab-config.json ]; then echo "starting from the files on the volume" >&2; exit 0; fi
              exit "$status"
          env:
            - name: QUERONA_URL
              value: "https://querona.example.com:9000"
            - name: QUERONA_API_KEY
              valueFrom:
                secretKeyRef: { name: dab-sales-api, key: QUERONA_API_KEY }
          volumeMounts:
            - { name: config, mountPath: /config }
            - { name: sync, mountPath: /scripts }
      containers:
        - name: dab
          image: mcr.microsoft.com/azure-databases/data-api-builder:2.0.9
          # DAB reads dab-config.json and its data-source-files, and writes its log, relative to the
          # working directory. The image's /App is read-only for the arbitrary user OpenShift runs pods
          # as, so DAB runs from the writable volume instead of from /App.
          workingDir: /config
          command: ["dotnet", "/App/Azure.DataApiBuilder.Service.dll"]
          env:
            - name: QUERONA_DAB_LOGIN
              valueFrom:
                secretKeyRef: { name: dab-sales-api, key: QUERONA_DAB_LOGIN }
            - name: QUERONA_DAB_PASSWORD
              valueFrom:
                secretKeyRef: { name: dab-sales-api, key: QUERONA_DAB_PASSWORD }
          ports:
            - containerPort: 5000
          volumeMounts:
            - { name: config, mountPath: /config }
          readinessProbe:
            httpGet: { path: /health, port: 5000 }
            initialDelaySeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: dab-sales-api
spec:
  selector:
    app: dab-sales-api
  ports:
    - port: 80
      targetPort: 5000
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: dab-sales-api
spec:
  to:
    kind: Service
    name: dab-sales-api
  port:
    targetPort: 5000
  tls:
    termination: edge

Notes on the manifest:

  • Pin the image tag to the DAB version Querona embeds (2.0.9); the generated configuration is produced for that version.

  • The generated configuration contains no appsettings.json and no Kestrel binding: the container listens on plain HTTP port 5000 and TLS terminates at the Route. Override the port with ASPNETCORE_HTTP_PORTS if needed.

  • The init container fails the pod start when it cannot fetch a configuration and nothing is on the volume yet. That is intended: a DAB without configuration would fail anyway, with a less useful message.

  • With two replicas, a rollout replaces pods one at a time, so a configuration change costs no downtime.

4. Refresh on configuration change#

The job below asks Querona whether the configuration behind the hash the deployment runs with is still current. 304, or 200 carrying the same hash (the cache period had expired), means nothing to do. 200 with another hash means it changed: the job stores the new hash in the pod template, and that change is what rolls the deployment; the new pods fetch the new files on start.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: dab-refresher
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: dab-refresher
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    resourceNames: ["dab-sales-api"]
    verbs: ["get", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dab-refresher
subjects:
  - kind: ServiceAccount
    name: dab-refresher
roleRef:
  kind: Role
  name: dab-refresher
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: dab-sales-api-refresh
spec:
  schedule: "*/5 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: dab-refresher
          restartPolicy: Never
          containers:
            - name: check
              image: registry.redhat.io/openshift4/ose-cli
              env:
                - name: QUERONA_URL
                  value: "https://querona.example.com:9000"
                - name: QUERONA_API_KEY
                  valueFrom:
                    secretKeyRef: { name: dab-sales-api, key: QUERONA_API_KEY }
              command: ["/bin/sh", "-c"]
              args:
                - |
                  set -e
                  current=$(oc get deployment dab-sales-api -o jsonpath='{.spec.template.metadata.annotations.querona\.com/config-hash}')
                  code=$(curl -s -o /dev/null -D /tmp/headers -w '%{http_code}' \
                    -H "Authorization: ApiKey $QUERONA_API_KEY" \
                    -H "If-None-Match: \"$current\"" \
                    "$QUERONA_URL/api/1.0/dataapiinstances/configuration")
                  case "$code" in
                    304) echo "unchanged $current" ;;
                    200) new=$(sed -n 's/^[Ee][Tt][Aa][Gg]: *"\{0,1\}\([^"[:space:]]*\).*/\1/p' /tmp/headers)
                         if [ -z "$new" ]; then echo "no ETag in the answer" >&2; exit 1; fi
                         if [ "$new" = "$current" ]; then echo "unchanged $current"; exit 0; fi
                         oc patch deployment dab-sales-api --type merge \
                           -p "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"querona.com/config-hash\":\"$new\"}}}}}"
                         echo "rolled to $new" ;;
                    *)   echo "Querona answered HTTP $code" >&2; exit 1 ;;
                  esac

The check costs Querona nothing inside the configuration cache window and one configuration generation per instance after it (see the TTL in Configuration endpoint for a separately hosted DAB), so a five-minute schedule is a reasonable default. A change becomes visible in the API after: cache TTL + schedule interval + rollout time.

The annotation starts empty, so the first run after you apply the manifests rolls the deployment once, although its pods already run the current files. To avoid that restart, set querona.com/config-hash to the hash the init container printed (changed <hash>) before the first run.

A 200 with success: false in the body is also answered with a 200 status but carries no ETag; the CronJob above then stops with “no ETag in the answer” and leaves the deployment alone. If you prefer to see the reason in the job log, run the Python script with KNOWN_HASH=$current instead of the curl call and act on its exit code (0 unchanged, 3 changed, 1 error with the message).

In a Querona cluster the cached answers are per node: saving an instance clears the cache of the node that handled the save, and another node keeps its answer until the cache period runs out. Size the period and the schedule together with that in mind.

Each node also writes its own TDS network name into the files (and, for a JWT instance, its own API URL), so two nodes answer the same hash only when they name themselves the same way. When the job can reach more than one node, set the same TDS network name or IP override instance parameter on every node - the name the pods reach Querona by - otherwise the job sees a different hash on each node and rolls the deployment on every run.

5. Verify#

oc rollout status deployment/dab-sales-api
curl -s https://$(oc get route dab-sales-api -o jsonpath='{.spec.host}')/api/openapi | head -c 400

For a JWT instance, obtain a token with the instance API key as described in Managing Data API and pass it as Authorization: Bearer to the route.

Troubleshooting#

Symptom

Cause and fix

Init container exits with success: false and a message about the API key

The key is not the one on the instance, or the Authorization header does not use the ApiKey scheme. Copy the key again from the instance details.

success: false with “No tables or views were found for principal …”

The instance’s user or role no longer sees any object matched by the inclusion rules. Fix the grants or the rules in Querona; the container keeps its last good files until then.

success: false saying a table or view “must have a primary key”

DAB addresses every row by its key, so an object without one cannot be published. Give the object a primary key, or exclude it with the instance’s inclusion rules.

The CronJob rolls the deployment on every run

The job reaches more than one Querona node and each writes its own network name into the files. Set the same TDS network name or IP override instance parameter on every node.

DAB starts and logs a login failure against Querona

QUERONA_DAB_LOGIN / QUERONA_DAB_PASSWORD are wrong, or the login is not the instance’s user (or in its role) and therefore cannot see the published objects.

DAB logs a TLS error such as “The remote certificate is invalid”

The container does not trust the certificate Querona presents on TDS. Configure a certificate on the node if it still presents the generated fallback one, mount the CA bundle and set SSL_CERT_FILE; do not weaken the generated TrustServerCertificate=False.

success: false saying the instance authenticates with JWT and the API must run over HTTPS

DAB validates tokens against the Querona API URL, which is http://localhost while HTTPS is off and unreachable from a pod. Enable HTTPS on the Querona API, or switch the instance to anonymous authentication, and fetch again.

IDX10214: Audience validation failed on a JWT instance

The token was issued for another instance: its aud claim is the instance identifier the configuration expects. Request the token with this instance’s API key.

The CronJob never rolls the deployment although the schema changed

The change is not part of the DAB configuration (for example a new column), or it is not yet visible because of the configuration cache TTL. A column change is picked up when DAB restarts; a new table, view, key or grant changes the hash after the TTL.

See also