One Hudi write, two table formats: committing Iceberg metadata inside the Hudi commit

• The core problem and architecture: one commit, two formats
  • Why a sync job leaves a gap
  • The hook Hudi added
  • What happens inside one commit
  • What the adapter replaces, and what it keeps
  • What it needs
• Under the hood: what a commit leaves on disk
  • One folder, two metadata trees
  • Each Hudi instant becomes one Iceberg snapshot
  • Why five metadata files for three commits
  • The Iceberg schema is Hudi’s, with field ids assigned
  • Why the metadata table has to be off
  • Rollback, clean and archive go through the same hook
• Hands-on demo: write once, read as Hudi and as Iceberg
  • Prerequisites and the stack
  • Step 1: build the XTable jars
  • Step 2: the script
  • Step 3: run it
  • Step 4: the output
  • Step 5: clean up
• Real-world scenarios and production failures
  • The write that succeeds and then disappears
  • The metadata table left on
  • Merge-on-read
  • Classpath failures that look like data problems
  • The limits that come with the format
• Optimisation, tuning knobs and best practices
  • Choosing between the ways to give a Hudi table an Iceberg face
  • What the choice costs
  • Best practices
• Summary and key takeaways
• References
• Trademarks

TL;DR

• hoodie.table.format is a hook in Hudi core, and native is the only format Hudi itself implements. The Iceberg implementation lives in Apache XTable, which Hudi finds at runtime through Java’s ServiceLoader.
• With that adapter on the classpath and hoodie.table.format=ICEBERG, every Hudi commit also commits an Iceberg snapshot in the same table folder. There is no separate sync job and no window in which the two formats disagree.
• Two settings are mandatory: the table is copy-on-write, and hoodie.metadata.enable=false on every write.
• Without the adapter jars, the write succeeds as plain native Hudi, the property is still saved, and no Iceberg metadata is produced. Nothing fails.
• The adapter is not on Maven Central yet, so the jars come from a source build. Pin the commit you build.
• Hudi stays the only writer. Iceberg engines read the table; they must not commit to it.

A Hudi table that Iceberg engines also need to read usually grows a second job. A sync step runs after the Hudi commit, reads what the commit changed, writes the matching Iceberg metadata, and exits. That step has its own failure modes: it can lag, it can fail after the Hudi commit has already succeeded, and between the two commits an Iceberg reader sees an older table than a Hudi reader does. Teams then add monitoring for the gap, which is a sign that the gap is part of the design.

Hudi 1.1 added a different way to close it. A table format in Hudi is the layer that records commits: it decides what a completed instant is, how the timeline is read, and what happens on clean, rollback and archive. The HoodieTableFormat interface makes that layer pluggable, so another format can be written in the same commit as Hudi’s own instead of after it. Apache XTable provides the Iceberg implementation. When the implementation is present, a Hudi write, clean, rollback or archive also commits an Iceberg snapshot, and a failed Iceberg commit fails the Hudi operation rather than leaving the two to be reconciled later.

This post runs it. Three Hudi writes (an insert, an upsert and a delete) produced three Iceberg snapshots in the table’s metadata/ folder, and a Hudi reader and an Iceberg reader returned the same rows after each one. It also runs the failures, because this feature has an unusual one: leave the adapter off the classpath and the write still succeeds.

Who this is for. Data engineers and platform leads who run Hudi as the write path and need Iceberg-only engines to read the same tables, without a second copy of the data and without a translation job to babysit. You should know what a Hudi commit is. Two terms are used throughout. The timeline is Hudi’s ordered log of actions on a table, each an instant that moves from requested to inflight to completed. A file slice is one version of a file group: a base file plus any log files written against it.

What was run. Every output below came from Hudi 1.2.1 on Spark 3.5.9, in the datalake-playground Spark image, against an XTable build from source. The parts of this feature that are specific to a version say so where they appear.

The rest of the post follows the data. First the problem and the architecture of the hook, then what lands on disk when a commit goes through it, then a runnable demo, then the failures and the limits, and finally what each choice costs.

The core problem and architecture: one commit, two formats

Why a sync job leaves a gap

The established way to give a Hudi table an Iceberg face is a sync step. Hudi’s writer can call a sync tool after the commit, and Apache XTable ships one, XTableSyncTool, which is switched on with three writer options:

hoodie.datasource.meta.sync.enable = true
hoodie.meta.sync.client.tool.class = org.apache.xtable.hudi.sync.XTableSyncTool
hoodie.xtable.formats.to.sync      = ICEBERG

That works, and it is worth knowing its shape, because the shape is the problem. The sync is a second transaction. The Hudi instant is already complete when it starts, so a sync that fails leaves a committed Hudi write with no Iceberg snapshot behind it. When it was run against a Hudi version it was not built for, the data files were on disk and the write still raised HoodieMetaSyncException: Could not sync using the meta sync class org.apache.xtable.hudi.sync.XTableSyncTool. The two formats had diverged, and the only record of it was an exception in a job log.

The hook Hudi added

Hudi 1.1 moved the question into the commit path. Table format became an extension point, described in RFC-93, and the interface is small enough to read in one sitting: HoodieTableFormat declares callbacks for commit, clean, archive, rollback, completedRollback, restore and savepoint, and two factories, one for the timeline and one for table metadata. The table property that selects an implementation is hoodie.table.format, and the lookup is a few lines in HoodieTableConfig:

 hoodie.table.format = ?
        |
        |-- "native" (the default) ----------> NativeTableFormat   (built in)
        |
        '-- anything else
               |
               v
        ServiceLoader.load(HoodieTableFormat)
               |
               |-- an implementation whose getName() matches --> use it
               |
               '-- none matches ------------------------------> NativeTableFormat
                                                  (no error, no warning)

Read the bottom branch twice. Hudi core implements only the native format. Any other value is resolved through Java’s ServiceLoader, which scans the classpath for jars that register an implementation under META-INF/services/org.apache.hudi.common.HoodieTableFormat. If no jar answers to the name, Hudi does not fail; it writes a native table. The Iceberg implementation is not in Hudi at all. It is the IcebergTableFormat class in the xtable-iceberg-pluggable-tf module of Apache XTable, and it registers itself under the name ICEBERG.

What happens inside one commit

With the adapter present, a write goes through these steps:

  df.write.format("hudi") ...
        |
        v
  1. Hudi writes the data files and the instant moves
     requested -> inflight -> completed       .hoodie/timeline/<t>.commit
        |
        v
  2. Hudi calls IcebergTableFormat.commit(...)
        |
        v
  3. The adapter converts the commit into table changes and asks XTable
     to write an Iceberg snapshot:
        manifest(s)  ->  manifest list (snap-*.avro)  ->  v<N>.metadata.json
        and version-hint.text
        |
        |-- success -> the write returns; both formats now show the commit
        |
        '-- failure -> UpdateException; the Hudi operation fails
                       (the instant exists, but has no snapshot)

Two details in the source matter more than the flow itself.

Iceberg decides what is committed. The adapter’s timeline factory rebuilds Hudi’s active timeline from the Iceberg table. In the words of IcebergActiveTimeline, an instant counts as completed only when a snapshot in the current snapshot’s ancestry records it. A .commit file with no snapshot behind it is a pending write, and the next writer rolls it back.

A failed hook fails the write. The comment on failUnlessSynced in IcebergTableFormat says why: the Hudi instant is already complete when the hook runs, so a failure that was not surfaced would leave an instant without a snapshot, and the next writer would roll back a write the user believes succeeded. Raising the error makes the user retry instead.

What the adapter replaces, and what it keeps

Neither option copies data. Iceberg’s manifests point at the Parquet files Hudi wrote, in place.

What it needs

• Hudi 1.2.x. The adapter loads Hudi classes that 1.1.1 does not have. The playground’s Spark 3.5.9 image ships Hudi 1.1.1, and with the adapter jars on top of it the write fails with ClassNotFoundException: org.apache.hudi.common.schema.HoodieSchema. The Hudi 1.2.1 Spark bundle fixes that.
• Four XTable jars and an Iceberg runtime. xtable-iceberg-pluggable-tf, xtable-core, xtable-api and xtable-hudi-support-utils, plus the Iceberg Spark runtime. The Iceberg runtime is needed on the writer, not only for reads. Without it the write dies with ClassNotFoundException: org.apache.iceberg.exceptions.AlreadyExistsException.
• A source build. At the time of writing the module is not on Maven Central. The jars below were built from XTable commit 0e97d0a, whose version is 0.5.0-SNAPSHOT. A 0.5.0-incubating-rc1 candidate is on the Apache dev distribution site. Pin whatever you build.

Under the hood: what a commit leaves on disk

One folder, two metadata trees

Run the demo in the next section and the table folder looks like this:

orders/
|-- eu/  <uuid>-0_0-..._<instant>.parquet     <- data files written by Hudi
|-- us/  <uuid>-0_1-..._<instant>.parquet        (Iceberg points at these)
|
|-- .hoodie/                                  <- Hudi's metadata
|   |-- hoodie.properties                        hoodie.table.format=ICEBERG
|   '-- timeline/
|       |-- 20261109055454773.commit.requested
|       |-- 20261109055454773.inflight
|       |-- 20261109055454773_20261109055500898.commit
|       '-- ... one triple per commit
|
'-- metadata/                                 <- Iceberg's metadata
    |-- v1.metadata.json ... v5.metadata.json
    |-- version-hint.text                        contains: 5
    |-- snap-<snapshot-id>-1-<uuid>.avro         one manifest list per commit
    '-- <uuid>-m0.avro, <uuid>-m1.avro           manifests

There is no .hoodie/metadata directory, because the Hudi metadata table is off. That is a requirement, and the reason is in the next subsection.

An Iceberg reader that opens the folder by path reads version-hint.text, which holds the number of the current v<N>.metadata.json, and follows it to a snapshot, a manifest list, manifests and then the Parquet files. A Hudi reader ignores metadata/ and reads the timeline. Both end at the same files.

Each Hudi instant becomes one Iceberg snapshot

The demo writes an insert, an upsert and a delete. These are the three Hudi instants and the three Iceberg snapshots they produced:

The snapshot records which instant it mirrors in its summary, under the key XTABLE_METADATA. For the third snapshot of an earlier run, the value decodes to this:

{
  "sourceTableFormat": "HUDI",
  "sourceIdentifier": "20261107065224958",
  "latestTableOperationIdentifier": "{\"action\":\"commit\",\"ts\":\"20261107065224958\",
      \"state\":\"COMPLETED\",\"requestedTime\":\"20261107065224958\",
      \"completionTime\":\"20261107065226457\"}"
}

That string is the join key between the two formats. It also explains the operations. This is a copy-on-write table, so an update or delete rewrites a whole base file: Iceberg sees one file added and one file deleted, which is overwrite, with added-data-files and deleted-data-files both at 1 in the summary. Only the first write, which adds files and removes none, is append.

Why five metadata files for three commits

The table folder holds v1 to v5 for three commits. The Iceberg table is created in two steps before the first snapshot lands:

So a table with N commits has N + 2 metadata files. A reader never opens the older ones; version-hint.text points at the newest. The table property write.metadata.delete-after-commit.enabled is true, so Iceberg removes metadata files once they fall outside its retained-versions limit. All five were still present in this run.

The Iceberg schema is Hudi’s, with field ids assigned

Parquet files written by Hudi carry no Iceberg field ids. The adapter assigns them in column order and records the names so that Iceberg can match columns by name:

The five Hudi meta columns appear as ordinary Iceberg columns. An Iceberg query that runs SELECT * returns them, and you can use _hoodie_commit_time in Iceberg to ask which Hudi commit last wrote a row. The mapping from names to ids is stored in the table property schema.name-mapping.default, and the partition spec is identity(region), which is the only kind of partitioning the adapter supports.

Why the metadata table has to be off

Hudi’s metadata table normally answers file listings and column statistics. For an Iceberg-format table the metadata factory is IcebergBackedTableMetadata, and its documentation says it deliberately lists the file system rather than reading Iceberg manifests. An enabled metadata table would fail with Unsupported operation: getColumnsStats. The adapter therefore refuses the combination up front, which is the error you will see in the failures section. The cost is on the Hudi side: every file listing Hudi does on this table is a real storage listing. The source calls the design a placeholder, because Iceberg’s manifests already carry the file lists and column bounds this class should eventually serve.

Rollback, clean and archive go through the same hook

commit is not the only callback. clean and archive each convert their instant into Iceberg changes (read from the source, not run here), and a rollback commits a snapshot too. The earlier failure run shows it: after a rollback the Iceberg snapshot list reads append for the first commit, overwrite with the action rollback for the rollback, and overwrite with the action commit for the next write. A restore is refused outright, which is covered in the limits below.

Hands-on demo: write once, read as Hudi and as Iceberg

Prerequisites and the stack

Every command below ran on a laptop. The demo needs Docker and nothing else: the XTable build runs inside a Maven container, so no local Java or Maven is installed. The datalake-playground gives you the Spark image and the same stack the other posts use; see the setup post.

git clone https://github.com/rangareddy/datalake-playground.git
cd datalake-playground
SPARK_VERSION=3.5.9 sh run_datalake.sh start
sh run_datalake.sh status

The adapter does not run on the Hudi 1.1.1 that stack ships, so this demo does not use the running spark-master. It starts a throwaway container from the same Spark 3.5.9 image, which keeps the shared stack untouched, and loads Hudi 1.2.1 and the XTable jars with --jars. The image is linux/amd64, so Apple Silicon runs it under emulation.

Step 1: build the XTable jars

mkdir work && cd work
git clone https://github.com/apache/incubator-xtable.git
cd incubator-xtable
git checkout 0e97d0a
docker run --rm -v "$PWD":/src -v "$HOME/.m2":/root/.m2 -w /src \
  maven:3.9-eclipse-temurin-17 \
  mvn -B -q -pl xtable-hudi-support/xtable-iceberg-pluggable-tf -am \
      package -DskipTests -Dspotless.check.skip=true -Dspotless.apply.skip=true
cd ..

Collect the four jars and the Hudi 1.2.1 bundle into one folder:

mkdir jars
X=incubator-xtable
cp $X/xtable-hudi-support/xtable-iceberg-pluggable-tf/target/xtable-iceberg-pluggable-tf_2.12-0.5.0-SNAPSHOT.jar \
   $X/xtable-core/target/xtable-core_2.12-0.5.0-SNAPSHOT.jar \
   $X/xtable-api/target/xtable-api-0.5.0-SNAPSHOT.jar \
   $X/xtable-hudi-support/xtable-hudi-support-utils/target/xtable-hudi-support-utils-0.5.0-SNAPSHOT.jar \
   jars/
curl -sSfO --output-dir jars \
  https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.5-bundle_2.12/1.2.1/hudi-spark3.5-bundle_2.12-1.2.1.jar
ls jars

Step 2: the script

Save this as hudi_iceberg_demo.py. It writes three batches to a Hudi table with the two settings that matter, compares the Hudi and Iceberg reads after each one, and then prints each Iceberg snapshot with the Hudi instant it records.

import json
import sys

from pyspark.sql import SparkSession

BASE = sys.argv[1]  # for example file:///data/orders

spark = (
    SparkSession.builder.appName("hudi-writes-iceberg")
    .config("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
    .config("spark.sql.extensions", "org.apache.spark.sql.hudi.HoodieSparkSessionExtension")
    .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.hudi.catalog.HoodieCatalog")
    .config("spark.kryo.registrator", "org.apache.spark.HoodieSparkKryoRegistrar")
    .config("spark.sql.shuffle.partitions", "2")
    .getOrCreate()
)
spark.sparkContext.setLogLevel("ERROR")

COLUMNS = ["order_id", "region", "amount", "updated_at"]
HUDI_OPTIONS = {
    "hoodie.table.name": "orders",
    "hoodie.datasource.write.recordkey.field": "order_id",
    "hoodie.datasource.write.precombine.field": "updated_at",
    "hoodie.datasource.write.partitionpath.field": "region",
    "hoodie.datasource.write.table.type": "COPY_ON_WRITE",
    # The two settings that matter for this post.
    "hoodie.table.format": "ICEBERG",
    "hoodie.metadata.enable": "false",
}


def write(rows, operation, mode):
    df = spark.createDataFrame(rows, COLUMNS)
    (df.write.format("hudi").options(**HUDI_OPTIONS)
       .option("hoodie.datasource.write.operation", operation)
       .mode(mode).save(BASE))


def read(fmt):
    df = spark.read.format(fmt).load(BASE)
    return sorted(tuple(r) for r in df.select("order_id", "region", "amount").collect())


def compare(label):
    hudi, iceberg = read("hudi"), read("iceberg")
    print(f"== {label}")
    print("hudi   :", hudi)
    print("iceberg:", iceberg)
    print("same   :", hudi == iceberg)


write([(1, "eu", 10.0, "2026-11-01"), (2, "us", 20.0, "2026-11-01"),
       (3, "eu", 30.0, "2026-11-01")], "upsert", "overwrite")
compare("after insert")

write([(2, "us", 25.0, "2026-11-02"), (4, "us", 40.0, "2026-11-02")], "upsert", "append")
compare("after upsert")

write([(3, "eu", 30.0, "2026-11-03")], "delete", "append")
compare("after delete")

print("== iceberg snapshots and the Hudi instant each one records")
snapshots = spark.read.format("iceberg").load(BASE + "#snapshots").orderBy("committed_at")
for s in snapshots.collect():
    instant = json.loads(s.summary["XTABLE_METADATA"])["sourceIdentifier"]
    print(s.snapshot_id, s.operation, "hudi instant", instant)

first = snapshots.first().snapshot_id
as_of_first = (spark.read.format("iceberg").option("snapshot-id", first).load(BASE)
               .select("order_id", "region", "amount"))
print("== iceberg time travel to the first snapshot")
print(sorted(tuple(r) for r in as_of_first.collect()))

Step 3: run it

mkdir -p data
docker run -d --name hudi-iceberg --user root --entrypoint bash \
  -v "$PWD/jars":/jars:ro -v "$PWD":/work -v "$PWD/data":/data \
  rangareddy1988/ranga-spark:1.0.0 -c 'sleep infinity'

docker exec hudi-iceberg bash -c '
  JARS=$(ls /jars/*.jar | tr "\n" ","),$(ls $ICEBERG_HOME/iceberg-spark-runtime*.jar)
  $SPARK_HOME/bin/spark-submit --master "local[2]" \
    --conf spark.eventLog.enabled=false --conf spark.ui.enabled=false \
    --conf spark.sql.catalogImplementation=in-memory \
    --jars $JARS /work/hudi_iceberg_demo.py file:///data/orders'

Three flags need a word. spark.eventLog.enabled=false avoids an event-log directory the image preconfigures but does not create. The Iceberg runtime comes from $ICEBERG_HOME, which the playground image sets, so there is nothing to download for it. spark.sql.catalogImplementation=in-memory is needed because this container is not on the stack’s network: without it the image’s defaults point Hudi’s reader at a Hive Metastore it cannot reach, and the first read fails with Unable to instantiate SessionHiveMetaStoreClient.

Step 4: the output

== after insert
hudi   : [(1, 'eu', 10.0), (2, 'us', 20.0), (3, 'eu', 30.0)]
iceberg: [(1, 'eu', 10.0), (2, 'us', 20.0), (3, 'eu', 30.0)]
same   : True
== after upsert
hudi   : [(1, 'eu', 10.0), (2, 'us', 25.0), (3, 'eu', 30.0), (4, 'us', 40.0)]
iceberg: [(1, 'eu', 10.0), (2, 'us', 25.0), (3, 'eu', 30.0), (4, 'us', 40.0)]
same   : True
== after delete
hudi   : [(1, 'eu', 10.0), (2, 'us', 25.0), (4, 'us', 40.0)]
iceberg: [(1, 'eu', 10.0), (2, 'us', 25.0), (4, 'us', 40.0)]
same   : True
== iceberg snapshots and the Hudi instant each one records
1579142869345373296 append hudi instant 20261109055454773
6206417245994243896 overwrite hudi instant 20261109055505444
6278283655811247997 overwrite hudi instant 20261109055509672
== iceberg time travel to the first snapshot
[(1, 'eu', 10.0), (2, 'us', 20.0), (3, 'eu', 30.0)]

The Hudi and Iceberg readers agree after every write, there is one snapshot per Hudi instant, and the Iceberg time travel returns the table as it was after the insert. Your snapshot ids and instants will differ; the shape will not. Look at the folder to see both metadata trees:

ls data/orders data/orders/.hoodie/timeline data/orders/metadata | grep -v crc
grep table.format data/orders/.hoodie/hoodie.properties

The last command prints hoodie.table.format=ICEBERG. That is the property every later writer reads from the table, so only the first write needs to set it.

Step 5: clean up

docker rm -f hudi-iceberg
rm -rf data

The container only ever wrote into data/, so there is nothing to remove from the shared stack.

Real-world scenarios and production failures

Most failures of this feature are quiet, because the fallback to a native table raises nothing. Each one below was reproduced on the demo setup with the exact output shown.

The write that succeeds and then disappears

Three writers touch one table. The first and third have the adapter jars. The second does not, because it is a different job with a different classpath.

Writer 2 raised nothing and its data is in the Hudi table, but the Iceberg side never heard about it: the property hoodie.table.format=ICEBERG was set, no jar answered to it, and Hudi fell back to native. Then writer 3 started. It found a completed Hudi instant with no Iceberg snapshot, treated it as pending, and rolled it back. After writer 3, order 2 is gone from both formats. The timeline shows the rollback:

20261109055001528_20261109055007254.commit        writer 1
20261109055044928.rollback.inflight               writer 3 rolls back writer 2
20261109055044928.rollback.requested
20261109055044928_20261109055046624.rollback
20261109055047824_20261109055053164.commit        writer 3

Writer 2’s .commit file is no longer there, and the Iceberg snapshot list is append (commit), overwrite (rollback), overwrite (commit). A Hudi writer without the adapter is therefore not a harmless degraded mode. It acknowledges data that the next correct writer will delete.

What to do. Put the jars on every job that touches the table, including the ones nobody thinks of as writers: clean, clustering and compaction run as separate jobs in many deployments. Then check the invariant instead of trusting it. Every completed Hudi commit must be recorded by an Iceberg snapshot. This script compares the two and needs only the Iceberg runtime, not the adapter:

import json
import os
import re
import sys

from pyspark.sql import SparkSession

TABLE = sys.argv[1]  # a local table folder, for example /data/orders

spark = SparkSession.builder.config("spark.sql.catalogImplementation", "in-memory").getOrCreate()
spark.sparkContext.setLogLevel("ERROR")

# Hudi instants that the timeline calls completed.
completed = set()
for name in os.listdir(f"{TABLE}/.hoodie/timeline"):
    m = re.match(r"(\d+)_\d+\.commit$", name)
    if m:
        completed.add(m.group(1))

# Hudi instants that an Iceberg snapshot records.
snapshots = spark.read.format("iceberg").load(f"file://{TABLE}#snapshots").collect()
recorded = {json.loads(s.summary["XTABLE_METADATA"])["sourceIdentifier"] for s in snapshots}

pending = sorted(completed - recorded)
print(f"completed hudi commits: {len(completed)}, iceberg snapshots: {len(snapshots)}")
print("completed in hudi but missing from iceberg:", pending or "none")

Run against the table above after writer 3, it printed 2 completed commits, 3 snapshots (the rollback adds one) and none. Run against the next table, it finds the problem. For an object store, replace the os.listdir with a listing of .hoodie/timeline/ through your storage client.

The metadata table left on

hoodie.metadata.enable defaults to true in Hudi. A writer that forgets to set it to false on a table with the Iceberg format fails with:

UnsupportedOperationException: The Iceberg table format does not support the
Hudi metadata table, but table orders has one; set hoodie.metadata.enable=false
on every writer of the table and remove the metadata table

The error message is accurate and the damage is instructive. The failure happens in the hook, after Hudi has completed its own instant, so the timeline holds a completed .commit for the failed write and no snapshot for it. What each reader sees right after that failed write depends on whether it has the adapter:

A Hudi reader without the jars reads the native timeline and shows the write the adapter considers uncommitted. The next write, with the metadata table off, rolled the failed commit back, so the row that reader saw is gone. The rule from the previous failure extends to readers: any Hudi job that reads this table needs the jars as well.

Merge-on-read

A merge-on-read table is refused at the first write, before any files appear:

UnsupportedOperationException: The Iceberg table format only supports
COPY_ON_WRITE tables, but table orders is MERGE_ON_READ

No metadata/ folder is created. If you need the merge-on-read write profile, this feature is not available to that table today.

Classpath failures that look like data problems

The limits that come with the format

These come from the adapter’s source and the XTable documentation for the feature, and they were not all run here. Only the first two were reproduced above.

• Copy-on-write tables only. Reproduced above.
• The metadata table stays off on every write. Reproduced above.
• Identity partitioning only. The adapter maps each Hudi partition field to an Iceberg identity partition. Timestamp-based and custom key generators with typed partition fields are refused.
• No restore. A Hudi restore is refused before it changes the table.
• One writer at a time. Compare this with the locking options in Hudi concurrency control, which describes what Hudi alone does with two writers.
• Iceberg format version 2. The metadata.json files in the demo say format-version: 2.
• Hudi is the only writer. Other engines read the table as Iceberg. A commit made by an Iceberg writer is invisible to Hudi, which does not see snapshots it did not make.

Optimisation, tuning knobs and best practices

Choosing between the ways to give a Hudi table an Iceberg face

The sync tool is the right answer when you need a released artifact today and can accept lag. It is also strict about versions. XTable 0.4.0 worked against Hudi 0.14.1 on Spark 3.4.4, with Iceberg 1.9.2: an insert and an upsert both read identically in both formats. The same jar against Hudi 0.15.0 and against Hudi 1.1.1 failed with the same error:

NoSuchMethodError: HoodieTableMetaClient$Builder.setConf(org.apache.hadoop.conf.Configuration)

That method exists in the Hudi 0.14 line the jar was built against and not in 0.15.0 or 1.1.1, so a jar built against 0.14 does not load on later lines. The sync tool also needs the jars in a particular order. The XTable bundle carries its own unshaded copy of org.apache.iceberg.parquet, and when it comes before the Iceberg runtime on the classpath it shadows the runtime’s shaded copy, so reads fail with NoClassDefFoundError: org/apache/parquet/conf/ParquetConfiguration. Listing the Iceberg runtime first fixed it. The pluggable adapter was not subject to either problem in the demo, because it was built against the Hudi line it runs on.

What the choice costs

Listings replace the metadata table. With the metadata table off, Hudi lists storage for every file listing it needs. On a local disk that is invisible. On an object store it is a request per partition per write, and the cost scales with partition count. Keep partition counts bounded, and expect list latency to show up in write time before it shows up on a bill. The index you pick also changes: the demo ran on Hudi’s default index, and an index that lives in the metadata table, such as the record-level index, is off the table by implication of the requirement.

Snapshots accumulate. Every Hudi commit adds an Iceberg snapshot, and so does every rollback. Hudi’s cleaner and archiver get callbacks and convert their instants into Iceberg changes, but how Iceberg’s own snapshot expiry interacts with that was not tested here. The XTable documentation says to write the table only through Hudi, so do not schedule Iceberg maintenance procedures against it without testing them on a copy.

Storage does not double. The demo’s data is one set of Parquet files. The Iceberg metadata for three commits was a few kilobytes of JSON and Avro: each metadata.json was under 7 KB, and each manifest or manifest list about 4 to 8 KB. The cost of the feature is in operations, not bytes.

The Iceberg runtime version is yours to choose. The XTable jars were built against Iceberg 1.9.2. The snapshots in the demo carry iceberg-version: Apache Iceberg 1.11.0, the runtime on the classpath at write time, and the Iceberg 1.11.0 reader read them back. You do not have to match the build version, but pin whichever you run, on writers and readers alike.

Best practices

1. Pin the build. Record the XTable commit and Hudi bundle version beside the job, as this post does, because the artifact is a source build.
2. Put the same jars on every job. Writers, table services and Hudi readers. Treat a job without them as a defect, not a configuration choice.
3. Set hoodie.metadata.enable=false explicitly on every write, including SQL sessions, rather than relying on it being off in hoodie.properties.
4. Run the consistency check on a schedule, and alert on a non-empty result. It is the only signal that the silent fallback happened.
5. Make Hudi the only writer and give Iceberg readers read-only credentials.
6. Test on a copy first if you operate Iceberg maintenance or a catalog on top of the folder.

Summary and key takeaways

The mental model is short. A table format is the layer that records commits. Hudi made that layer replaceable, and XTable supplies an Iceberg implementation, so one Hudi write commits twice in the same operation: once to Hudi’s timeline and once to an Iceberg snapshot that names the Hudi instant it mirrors. The data files are shared. Iceberg’s snapshot ancestry decides which Hudi commits count.

• The parameters are small and the prerequisites are not. The table property and hoodie.metadata.enable=false are two lines. The prerequisites are Hudi 1.2.x, four XTable jars built from source, the Iceberg runtime on the writer and a copy-on-write table.
• Failure is silent by default. A writer without the jars writes a native table without a warning, and the next correct writer rolls its commit back.
• The jars are a table-wide contract. Every writer, table service and Hudi reader needs them.
• Verify the invariant. Every completed Hudi commit needs an Iceberg snapshot that records it. The check in this post tests exactly that.
• Choose the sync tool when you need a released artifact now. Choose the pluggable format when removing the lag matters more than building the jars.

The feature is young. Its metadata serving is explicitly a placeholder, restore is refused, and the artifact is a release candidate away from Maven Central. That is a reason to pin, test on a copy and automate the check, not a reason to avoid a design that removes a whole class of two-commit divergence.

References

• RFC-93, pluggable table formats for the design behind hoodie.table.format
• HoodieTableFormat.java, the interface an implementation provides
• HoodieTableConfig.java for hoodie.table.format and the ServiceLoader lookup with its fallback
• IcebergTableFormat.java, the adapter, including the copy-on-write and metadata-table checks
• IcebergActiveTimeline.java for how the timeline is rebuilt from snapshot ancestry
• IcebergBackedTableMetadata.java for why the metadata table must be off
• Apache XTable documentation for the sync tool and the standalone job
• Hudi catalog sync documentation for XTable for the writer-side sync route
• One table, three formats for how translation compares with writing natively
• Life of a write query in Iceberg for the manifest, manifest list and metadata file this adapter writes
• Time travel and rollback in Iceberg for the snapshot-id reads used in the demo

Trademarks

Apache Hudi, Apache Iceberg, Apache XTable, Apache Spark, Apache Parquet, Apache Avro and Apache are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.