pub fn strip_metadata_from_schema(schema: &Schema) -> Result<Schema>Expand description
Strips all metadata from an Arrow schema and its nested fields.
This function recursively removes metadata from all fields at every level of the schema, including nested struct, list, and map fields. This is useful for schema comparison where metadata differences should be ignored.
Some other differences are normalized away too, so that schemas that differ only in
them compare equal once stripped. The fields inside lists and maps are renamed as
schema_to_arrow_schema names them, and dictionaries are unwrapped:
- A list’s element field is renamed to
LIST_FIELD_NAME, soitem(the arrow-rs default) becomeselement. - A map’s entries field is renamed to
DEFAULT_MAP_FIELD_NAMEand made non-nullable. - A map’s key and value fields are renamed to
MAP_KEY_FIELD_NAMEandMAP_VALUE_FIELD_NAME, sokeysandvalues(the arrow-rsMapBuilderdefault) becomekeyandvalue, and the key is made non-nullable. - A dictionary-encoded field becomes a field of the dictionary’s value type.
The names of the schema’s fields and of struct fields are kept. Extension type
metadata (ARROW:extension:*) is dropped with the rest, so a variant field becomes a
plain struct of its metadata and value fields.
§Arguments
schema- The Arrow schema to strip metadata from
§Returns
A new Arrow schema with all metadata removed, or an error if the schema structure is invalid.
§Example
use std::collections::HashMap;
use arrow_schema::{DataType, Field, Schema as ArrowSchema};
use iceberg::arrow::strip_metadata_from_schema;
let mut metadata = HashMap::new();
metadata.insert("key".to_string(), "value".to_string());
let field = Field::new("col1", DataType::Int32, false).with_metadata(metadata);
let schema = ArrowSchema::new(vec![field]);
let stripped = strip_metadata_from_schema(&schema).unwrap();
assert!(stripped.field(0).metadata().is_empty());