Module cow_rewrite 

Copy-on-write rewrite primitives.

This module plans candidate data files, reads their visible rows, applies a caller-provided batch rewriter, and writes replacement data files. It returns old and new file sets that can be committed by an overwrite-style transaction action.

The primitive does not parse SQL and does not commit metadata by itself. Rewriters must emit batches compatible with the schema rows were read in (the planned snapshot’s schema) and must preserve each source file’s partition values; this primitive does not repartition rewritten rows.

Replacement files deliberately mirror their source file’s layout instead of re-optimizing it — a deliberate divergence from Java’s RewriteDataFiles and Spark DML: they are written in the planned snapshot’s schema rather than promoted to the table’s current schema, they keep the source file’s partition spec rather than being repartitioned into the table’s default spec, and they stay unsorted since no sort order is applied. An opt-in promote-to-current-schema rewrite is left for follow-up work.

The result carries data files only. A commit adapter consuming these file lists may remove position delete files and deletion vectors only when they exclusively reference removed data files. Equality delete files must be retained while they can still apply to other live data files by sequence number.

struct KeepAll;

impl CowBatchRewriter for KeepAll {
    fn rewrite_batch(&self, batch: RecordBatch) -> Result<CowBatchRewrite> {
        Ok(CowBatchRewrite {
            output: Some(batch),
            changed: false,
        })
    }
}

let result = CowRewriteBuilder::new(table)
    .with_rewriter(Arc::new(KeepAll))
    .rewrite()
    .await?;

assert!(!result.has_changes());

Structs

CowBatchRewrite

Result of rewriting a single record batch.

CowRewriteBuilder

Builder for orchestrating copy-on-write data file rewrites.

CowRewriteFile

A data file selected for COW rewrite.

CowRewriteResult

Result of a copy-on-write rewrite operation.

CowRewriteStats

Counters produced by a copy-on-write rewrite.

Traits

CowBatchRewriter

Rewrites record batches for copy-on-write operations.