Skip to main content

compute_unified_partition_type

Function compute_unified_partition_type 

Source
pub fn compute_unified_partition_type<'a>(
    partition_specs: impl Iterator<Item = &'a PartitionSpec>,
    schema: &Schema,
) -> Result<StructType>
Expand description

Computes the unified partition type across all partition specs in the table.

This is equivalent to Java’s Partitioning.partitionType(table). The result is a StructType containing all partition fields ever used across all specs, enabling correct representation of the _partition metadata column when partition evolution has occurred.

Matches Java’s buildPartitionProjectionType behavior:

  • Specs are sorted by spec_id in descending order (newer specs first), so newer field names take precedence when deduplicating by field_id.
  • Unknown transforms cause an error.
  • Fields whose source column was dropped from the schema are skipped.
  • Two specs defining the same field_id must be compatible (same source, compatible transforms); V1 tables do not guarantee field ids are unique across specs.
  • When a newer spec marks a field as Void (dropped) but an older spec has it with a real transform, the older spec’s type is preserved while the newer spec’s name is kept.
  • Fields are deduplicated by field_id; each unique field_id appears exactly once.
  • Output fields are sorted by field_id ascending.

§Arguments

  • partition_specs - Iterator over all partition specs in the table
  • schema - The current table schema (needed to determine result types of transforms)