Skip to main content

read_manifest_list

Function read_manifest_list 

Source
pub fn read_manifest_list<B: AsRef<[u8]>>(
    bytes: B,
) -> Result<ManifestListBatchReader<B>>
Expand description

Reads the entries of a manifest list as batches with one row per manifest.

The schema is the spec’s manifest_file struct for the latest format version, with field IDs in the Arrow field metadata, so manifest lists of every format version read into the same schema. Fields are matched to the manifest list’s Avro schema by field ID. The format version comes from the format-version key that writers put in the file’s metadata, and a manifest list without one is v1. A v1 manifest list reads with the spec’s v2+ defaults. Partition summary bounds keep the binary single-value serialization they were written with.

If the snapshot has an encryption_key_id, the manifest list is encrypted and bytes must be decrypted first. Table::manifest_list_reader returns a reader that decrypts.

§Errors

Returns an error if the manifest list’s header can’t be read or names an unsupported format version. Encrypted bytes fail at the header. Errors in the entries come from the reader, which yields no batches after one.

§Example

use iceberg::arrow::read_manifest_list;

let bytes = std::fs::read(concat!(
    env!("CARGO_MANIFEST_DIR"),
    "/testdata/manifests_lists/manifest-list-v2-2.avro"
))?;
let mut rows = 0;
for batch in read_manifest_list(&bytes)? {
    rows += batch?.num_rows();
}
assert_eq!(rows, 1);