Running Iceberg on Apache Polaris: one compose file, two storage backends

• Architecture: who holds what
• Step 1: start the stack
• Step 2: create the catalogs and grants
• Step 3: connect Spark
• Step 4: use the table
• The same code on both backends
• Troubleshooting
• Frequently asked questions
• Common misconceptions
• Production notes
• Where this leaves you
• References
• Trademarks

TL;DR

• Polaris is an Iceberg REST catalog plus an authorization model. Spark talks to it with type=rest and learns where the data lives from the catalog.
• Access needs a four-link chain: principal, principal role, catalog role, privilege. Nothing is implicit, not even for the principal that created the catalog.
• The same SQL runs against a local directory and MinIO with one word changed, which is the point of putting a catalog in front of storage.
• With an S3-compatible server, endpoint and endpointInternal differ: Polaris and your engine reach the same bucket by different hostnames.
• The default metastore is in-memory, so keep provisioning in a script you can re-run.

A catalog is the part of a lakehouse people postpone. Iceberg works fine against a filesystem path, right up to the day two engines need a consistent view of the same table and something has to arbitrate. That something is a catalog, and Apache Polaris speaks Iceberg’s REST protocol natively while adding the access control a shared catalog needs.

This is the whole setup in four steps, run against Polaris 1.7.0 with Spark 3.5.9 and again with Spark 4.1.3. Every output below is real. The errors I hit are collected in one section at the end, so the steps stay short.

Architecture: who holds what

flowchart LR
  S["<b>Spark</b><br/>SparkCatalog, type=rest"]
  P["<b>Polaris</b><br/>Iceberg REST API<br/>+ authorization"]
  M["<b>metastore</b><br/>catalogs, roles, grants,<br/>table pointers"]
  W["<b>storage</b><br/>Parquet + Iceberg metadata"]
  S -->|"1  OAuth2 token"| P
  S -->|"2  load table"| P
  P --> M
  P -->|"3  metadata location"| S
  S -->|"4  read and write files"| W
  P -->|"writes the first metadata file"| W

Three things follow, and between them they explain every error in this post:

• Polaris holds pointers, not data. It records which metadata file is current. Spark reads and writes the Parquet directly.
• Both processes touch storage. Polaris writes the first metadata file, Spark writes everything after it, and both need access to the same location.
• Authorization happens at the catalog. Spark presents a token and Polaris decides what it may do.

Step 1: start the stack

Three services: MinIO, a one-shot job that creates the bucket, and Polaris. Save this as docker-compose.yml:

# Apache Polaris with two storage backends: a local directory and MinIO.
# Bring it up with:  docker compose up -d
services:
  minio:
    image: minio/minio:latest
    container_name: minio
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    ports:
      - "9000:9000"   # S3 API
      - "9001:9001"   # web console
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 3s
      retries: 20

  # Creates the bucket, then exits. `depends_on` waits for the healthcheck.
  minio-init:
    image: minio/mc:latest
    container_name: minio-init
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      mc alias set local http://minio:9000 minioadmin minioadmin &&
      mc mb --ignore-existing local/warehouse &&
      mc ls local &&
      echo 'bucket ready'"

  polaris:
    image: apache/polaris:latest
    container_name: polaris
    depends_on:
      minio-init:
        condition: service_completed_successfully
    ports:
      - "8181:8181"
      - "8182:8182"
    volumes:
      # The host path and the container path must be IDENTICAL. Polaris writes
      # the first metadata file and Spark writes everything after it, and each
      # resolves file:///tmp/polaris-warehouse in its own filesystem. A relative
      # mount like ./warehouse silently splits one table across two directories.
      - /tmp/polaris-warehouse:/tmp/polaris-warehouse
    environment:
      POLARIS_BOOTSTRAP_CREDENTIALS: "POLARIS,root,s3cr3t"
      quarkus.otel.sdk.disabled: "true"
      # Accepts the risk of FILE storage. Test only, see the post.
      polaris.readiness.ignore-severe-issues: "true"
      # Credentials Polaris uses to reach MinIO
      AWS_ACCESS_KEY_ID: minioadmin
      AWS_SECRET_ACCESS_KEY: minioadmin
      AWS_REGION: us-east-1
      JAVA_OPTS_APPEND: >-
        -Dpolaris.features."ALLOW_INSECURE_STORAGE_TYPES"=true
        -Dpolaris.features."SUPPORTED_CATALOG_STORAGE_TYPES"=["FILE","S3"]
    healthcheck:
      test: ["CMD-SHELL", "exec 3<>/dev/tcp/localhost/8181"]
      interval: 5s
      timeout: 3s
      retries: 30

mkdir -p /tmp/polaris-warehouse && chmod 777 /tmp/polaris-warehouse
docker compose up -d

Verify before moving on:

docker compose ps --format '{{.Name}}\t{{.Status}}'

minio     Up 39 seconds (healthy)
polaris   Up 32 seconds (healthy)

MinIO’s console is at http://localhost:9001 with minioadmin / minioadmin.

Two details in that file earn their place. The warehouse mount uses the same absolute path on both sides, because Polaris and a host Spark each resolve file:///tmp/polaris-warehouse in their own filesystem, and a relative mount silently splits one table across two directories. And chmod 777 is a laptop shortcut, because the Polaris process runs as uid 10000.

Step 2: create the catalogs and grants

Two catalogs, one on the local directory and one on MinIO, plus the roles that make them usable. The metastore is in-memory, so keep this in a file you can re-run after every restart:

#!/usr/bin/env bash
# Provision both catalogs, their roles and grants. Re-runnable.
set -euo pipefail
BASE=${BASE:-http://localhost:8181}

TOKEN=$(curl -s -X POST "$BASE/api/catalog/v1/oauth/tokens" \
  -d 'grant_type=client_credentials' -d 'client_id=root' -d 'client_secret=s3cr3t' \
  -d 'scope=PRINCIPAL_ROLE:ALL' | python3 -c 'import sys,json; print(json.load(sys.stdin)["access_token"])')
[ ${#TOKEN} -gt 100 ] || { echo "token fetch failed"; exit 1; }
echo "token length: ${#TOKEN}"

API="$BASE/api/management/v1"
post() { curl -s -o /dev/null -w "%{http_code}" -X "$1" "$API/$2" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -d "$3"; }

# the principal role first, so the later assignments have something to bind to
echo "principal role          : $(post POST principal-roles '{"principalRole":{"name":"engineer"}}')"

echo "local_catalog           : $(post POST catalogs '{"catalog":{"name":"local_catalog","type":"INTERNAL","readOnly":false,"properties":{"default-base-location":"file:///tmp/polaris-warehouse"},"storageConfigInfo":{"storageType":"FILE","allowedLocations":["file:///tmp/polaris-warehouse"]}}}')"

echo "s3_catalog              : $(post POST catalogs '{"catalog":{"name":"s3_catalog","type":"INTERNAL","readOnly":false,"properties":{"default-base-location":"s3://warehouse/polaris"},"storageConfigInfo":{"storageType":"S3","allowedLocations":["s3://warehouse/polaris"],"roleArn":"arn:aws:iam::000000000000:role/minio-unused","region":"us-east-1","endpoint":"http://localhost:9000","endpointInternal":"http://minio:9000","pathStyleAccess":true,"stsUnavailable":true}}}')"

for CAT in local_catalog s3_catalog; do
  echo "$CAT role         : $(post POST "catalogs/$CAT/catalog-roles" '{"catalogRole":{"name":"admin_role"}}')"
  echo "$CAT grant        : $(post PUT "catalogs/$CAT/catalog-roles/admin_role/grants" '{"grant":{"type":"catalog","privilege":"CATALOG_MANAGE_CONTENT"}}')"
  echo "$CAT assign       : $(post PUT "principal-roles/engineer/catalog-roles/$CAT" '{"catalogRole":{"name":"admin_role"}}')"
done
echo "root -> engineer        : $(post PUT principals/root/principal-roles '{"principalRole":{"name":"engineer"}}')"
curl -s -H "Authorization: Bearer $TOKEN" "$API/catalogs" \
  | python3 -c 'import sys,json; print("catalogs                :", [c["name"] for c in json.load(sys.stdin)["catalogs"]])'

bash provision.sh

token length: 630
principal role          : 201
local_catalog           : 201
s3_catalog              : 201
local_catalog role         : 201
local_catalog grant        : 201
local_catalog assign       : 201
s3_catalog role         : 201
s3_catalog grant        : 201
s3_catalog assign       : 201
root -> engineer        : 201
catalogs                : ['local_catalog', 's3_catalog']

Every line is 201. Two parts of that script are worth understanding rather than copying.

The grant chain is four links, and nothing is implicit. Creating a catalog gives nobody access to it, including the principal that created it:

flowchart LR
  A["<b>principal</b><br/>root"] -->|"is assigned"| B["<b>principal role</b><br/>engineer"]
  B -->|"is granted"| C["<b>catalog role</b><br/>admin_role"]
  C -->|"holds privilege"| D["<b>CATALOG_MANAGE_CONTENT</b><br/>on the catalog"]

Principal roles describe people and jobs; catalog roles describe what may be done to one catalog. One engineer role can hold different catalog roles in different catalogs, so a team can own its own data and read someone else’s. The principal role is created first, or the assignments have nothing to bind to.

The S3 catalog needs four fields that exist for S3-compatible servers. endpoint is what Polaris hands to clients and endpointInternal is what the server itself uses; they differ because Polaris reaches MinIO at minio:9000 while your host Spark reaches the same bucket at localhost:9000. pathStyleAccess is required because MinIO is not virtual-hosted. And stsUnavailable tells Polaris not to attempt credential vending, which has a real consequence: Polaris vends nothing, so the engine supplies its own credentials. Fine on a laptop, not what you want in production.

Step 3: connect Spark

Keep the connection settings in a file too:

# Source this, do not run it:  source polaris-env.sh
# No spaces around '=' . `export VAR = value` is a syntax error in bash and zsh.

export POLARIS_URI='http://localhost:8181/api/catalog'
export POLARIS_SCOPE='PRINCIPAL_ROLE:ALL'
export CLIENT_ID='root'
export CLIENT_SECRET='s3cr3t'
export POLARIS_CREDENTIAL="${CLIENT_ID}:${CLIENT_SECRET}"

# Versions, so the two jars can never drift apart.
# Spark 3.5.x: '3.5' and '2.12'. Spark 4.1.x: '4.1' and '2.13'.
export SPARK_VERSION='3.5'
export SCALA_VERSION='2.12'
export ICEBERG_VERSION='1.11.0'
export ICEBERG_PACKAGES="org.apache.iceberg:iceberg-spark-runtime-${SPARK_VERSION}_${SCALA_VERSION}:${ICEBERG_VERSION},org.apache.iceberg:iceberg-aws-bundle:${ICEBERG_VERSION}"

# MinIO, for the S3 catalog. Polaris has stsUnavailable set, so it vends no
# credentials and the engine supplies its own.
export AWS_ACCESS_KEY_ID='minioadmin'
export AWS_SECRET_ACCESS_KEY='minioadmin'
export AWS_REGION='us-east-1'
export MINIO_ENDPOINT='http://localhost:9000'

# A management-API token. Valid for one hour; re-source this file when it expires.
export POLARIS_TOKEN=$(curl -s -X POST "${POLARIS_URI}/v1/oauth/tokens" \
  -d 'grant_type=client_credentials' \
  -d "client_id=${CLIENT_ID}" -d "client_secret=${CLIENT_SECRET}" \
  -d "scope=${POLARIS_SCOPE}" \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["access_token"])')

if [ ${#POLARIS_TOKEN} -lt 100 ]; then
  echo "POLARIS_TOKEN looks wrong (length ${#POLARIS_TOKEN}). Is the stack up?" >&2
else
  echo "POLARIS_TOKEN length ${#POLARIS_TOKEN}, packages ${ICEBERG_PACKAGES}"
fi

No spaces around the =. export POLARIS_SCOPE = 'x' is not an assignment: bash reports export: '=': not a valid identifier and leaves the variable unset, and zsh fails outright with bad assignment.

source polaris-env.sh
# POLARIS_TOKEN length 630

Then launch the shell with both catalogs attached:

pyspark \
  --packages "${ICEBERG_PACKAGES}" \
  --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
  \
  --conf spark.sql.catalog.local_cat=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.local_cat.type=rest \
  --conf spark.sql.catalog.local_cat.uri="${POLARIS_URI}" \
  --conf spark.sql.catalog.local_cat.warehouse=local_catalog \
  --conf spark.sql.catalog.local_cat.credential="${POLARIS_CREDENTIAL}" \
  --conf spark.sql.catalog.local_cat.scope="${POLARIS_SCOPE}" \
  --conf spark.sql.catalog.local_cat.rest.auth.type=oauth2 \
  --conf spark.sql.catalog.local_cat.oauth2-server-uri="${POLARIS_URI}/v1/oauth/tokens" \
  \
  --conf spark.sql.catalog.s3_cat=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.s3_cat.type=rest \
  --conf spark.sql.catalog.s3_cat.uri="${POLARIS_URI}" \
  --conf spark.sql.catalog.s3_cat.warehouse=s3_catalog \
  --conf spark.sql.catalog.s3_cat.credential="${POLARIS_CREDENTIAL}" \
  --conf spark.sql.catalog.s3_cat.scope="${POLARIS_SCOPE}" \
  --conf spark.sql.catalog.s3_cat.rest.auth.type=oauth2 \
  --conf spark.sql.catalog.s3_cat.oauth2-server-uri="${POLARIS_URI}/v1/oauth/tokens" \
  --conf spark.sql.catalog.s3_cat.io-impl=org.apache.iceberg.aws.s3.S3FileIO \
  --conf spark.sql.catalog.s3_cat.s3.endpoint="${MINIO_ENDPOINT}" \
  --conf spark.sql.catalog.s3_cat.s3.path-style-access=true \
  --conf spark.sql.catalog.s3_cat.client.region="${AWS_REGION}"

Three names are easy to conflate, and mixing them is the most common mistake:

The alias in your --conf flags and the alias in your SQL must be the same word. The same arguments work unchanged with spark-submit.

Step 4: use the table

Ordinary Iceberg from here. Nothing below is Polaris-specific:

spark.sql("CREATE NAMESPACE IF NOT EXISTS local_cat.sales")
spark.sql("""
  CREATE OR REPLACE TABLE local_cat.sales.orders (
    order_id BIGINT, city STRING, amount DECIMAL(10,2), ordered_at TIMESTAMP
  ) USING iceberg PARTITIONED BY (days(ordered_at))""")
spark.sql("""
  INSERT INTO local_cat.sales.orders VALUES
    (1, 'pune',      1200.50, TIMESTAMP '2026-09-01 10:15:00'),
    (2, 'hyderabad',  845.00, TIMESTAMP '2026-09-01 11:00:00'),
    (3, 'pune',       310.25, TIMESTAMP '2026-09-02 09:30:00')""")
spark.sql("""
  SELECT city, sum(amount) AS total
  FROM local_cat.sales.orders GROUP BY city ORDER BY city""").show()

+---------+-------+
|     city|  total|
+---------+-------+
|hyderabad| 845.00|
|     pune|1510.75|
+---------+-------+

Metadata tables, schema evolution, MERGE and time travel behave exactly as they do against any other Iceberg catalog:

spark.sql("""
  SELECT partition, record_count
  FROM local_cat.sales.orders.partitions ORDER BY partition""").show()
spark.sql("ALTER TABLE local_cat.sales.orders ADD COLUMN channel STRING")
spark.sql("""
  MERGE INTO local_cat.sales.orders t
  USING (SELECT 2 AS order_id, 'web' AS channel) s
  ON t.order_id = s.order_id
  WHEN MATCHED THEN UPDATE SET t.channel = s.channel""")

+------------+------------+
|   partition|record_count|
+------------+------------+
|{2026-09-01}|           2|
|{2026-09-02}|           1|
+------------+------------+

The same code on both backends

Change local_cat to s3_cat and nothing else:

[local_cat] rows=3 snapshots=2 time_travel_rows=3
[local_cat] location=file:/tmp/polaris-warehouse/sales/orders/data/ordered_at_day=2026-09-01/00000-8-....parquet

[s3_cat]    rows=3 snapshots=2 time_travel_rows=3
[s3_cat]    location=s3://warehouse/polaris/sales/orders/data/ordered_at_day=2026-09-01/00000-26-....parquet

Confirm it on disk and in the bucket:

find /tmp/polaris-warehouse -type f | grep -v crc | sort

docker compose exec -T minio sh -c \
  'mc alias set local http://localhost:9000 minioadmin minioadmin >/dev/null && mc ls -r local/warehouse/'

In both, Polaris’s 00000-....metadata.json sits beside Spark’s Parquet. That is the check that storage is configured correctly: two processes addressing one location by different names.

Troubleshooting

Every error I hit, in one place. None of their messages name the real cause.

Four of those are worth a sentence more.

The silent 401. A 401 from the management API has an empty body, so curl -s prints nothing and it looks like a dead server. The token lives in one shell, expires after an hour, and a failed fetch leaves the variable empty rather than failing loudly. Add -w '\nHTTP %{http_code}\n' to every call and nothing is ever invisible.

The 503 on the first metadata file. This reads like a Spark fault and is not: Polaris writes that file itself, server side. If it cannot, the commit fails before Spark writes anything.

The unconfigured catalog. Spark falls back to the session catalog and complains about the shape of your identifier. spark_catalog appearing in a message about your own catalog name is the tell. On Spark 3.5 a CREATE NAMESPACE hits a missing message template and reports [INTERNAL_ERROR]; nothing is internally wrong, the real error is “that catalog does not exist”.

spark.sql.extensions in an existing session. Catalog properties are runtime SQL configs and do apply to an already-built session. Extensions are installed when the session is created, so setting that one afterwards has no effect even though reading it back shows your value.

Frequently asked questions

Which Iceberg runtime works with my Spark? iceberg-spark-runtime-4.1_2.13:1.11.0 for Spark 4.1.x and iceberg-spark-runtime-3.5_2.12:1.11.0 for 3.5.x. The number after runtime- is the Spark minor line, not the Iceberg version, and each minor line gets its own build. Iceberg 1.11.0 is compiled for Java 17, so on Spark 3.5 it needs a Java 17 JVM: with the default apache/spark:3.5.9-python3 image, which is Java 11, the first SQL statement fails with UnsupportedClassVersionError ... class file version 61.0. Use the 3.5.9-java17-python3 tag. Add iceberg-aws-bundle too if you use the S3 catalog.

Do I pass the bearer token to Spark? No. Spark gets credential as client-id:client-secret and runs its own OAuth2 exchange. POLARIS_TOKEN is only for the management API.

Why does a management API call print nothing at all? A 401 has an empty body, so curl -s shows nothing and it looks like a dead server. Check echo ${#POLARIS_TOKEN}: tokens last one hour, live in a single shell, and a failed fetch leaves the variable empty.

Can two engines share one Polaris catalog? That is the reason it exists. Any Iceberg REST client can attach and the catalog arbitrates commits, so both see the same table state instead of two filesystem views that drift.

Is this setup safe to run in production? No. The local-file backend, ignore-severe-issues, the in-memory metastore and stsUnavailable are all laptop shortcuts. For a real deployment use object storage, a PostgreSQL metastore, and leave credential vending on.

Common misconceptions

“The warehouse config is a path.” It is the catalog name. The path comes back from Polaris in the /v1/config response, which is what lets storage move without a client change.

“Creating a catalog gives its creator access.” It does not. The grant chain is four links and the failure names none of them.

“Polaris stores my data.” It stores pointers and permissions. Spark reads and writes Parquet directly, which is why storage must be reachable from the engine and not only from Polaris.

“A REST catalog means I can skip the Iceberg jar.” The REST protocol is spoken by the Iceberg client library. Without iceberg-spark-runtime there is no SparkCatalog to configure.

Production notes

• Do not use FILE storage. It exists for tests, Polaris tries to stop you, and the flag you need is called ignore-severe-issues for a reason.
• Configure a real metastore. The default loses every catalog and grant on restart.
• Keep provisioning in a re-runnable script. You will rebuild more often than you expect.
• Prefer credential vending over shared keys. With real S3 or GCS, Polaris hands each engine scoped, short-lived credentials. That is the main reason to run it rather than a filesystem catalog, and stsUnavailable switches it off.
• Pin the Iceberg runtime to your Spark line. iceberg-spark-runtime-4.1_2.13 for Spark 4.1.x, iceberg-spark-runtime-3.5_2.12 for 3.5.x. A mismatch surfaces as a missing class, not a version error.

Where this leaves you

The work splits in two. The Iceberg half is what you already know: same table format, same metadata files, same snapshots and time travel, untouched. The Polaris half is about identity, which is what the catalog adds: who is asking, which role they hold, and what that role may do here.

If you take one habit from this, make it the grant chain. Almost every access failure in Polaris is one of those four links missing, and checking them in order is faster than reading an error that will not tell you which one.

References

• Apache Polaris documentation, for the management API and the production configuration guide
• Iceberg Spark configuration for the catalog properties used above
• Iceberg REST catalog spec, the protocol Polaris implements
• Apache Iceberg architecture for what a commit actually swaps
• Apache Iceberg on Spark: the complete cheat sheet for the table surface once the catalog is wired up

Trademarks

Apache Polaris, Apache Iceberg, Apache Spark, Apache Parquet, Apache Avro and Apache are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.