Struct FileIO 

pub struct FileIO { /* private fields */ }

FileIO implementation, used to manipulate files in underlying storage.

FileIO wraps a dyn Storage with lazy initialization via StorageFactory. The storage is created on first use and cached for subsequent operations.

Note

All paths passed to FileIO must be absolute paths starting with the scheme string appropriate for the storage backend being used.

This crate provides native support for local filesystem (file://) and memory (memory://) storage. For extensive storage backend support (S3, GCS, OSS, Azure, etc.), use the iceberg-storage-opendal crate.

Example

use iceberg::io::{FileIO, FileIOBuilder};
use iceberg::io::{LocalFsStorageFactory, MemoryStorageFactory};
use std::sync::Arc;

// Create FileIO with memory storage for testing
let file_io = FileIO::new_with_memory();

// Create FileIO with local filesystem storage
let file_io = FileIO::new_with_fs();

// Create FileIO with custom factory
let file_io = FileIOBuilder::new(Arc::new(LocalFsStorageFactory))
    .with_prop("key", "value")
    .build();

Implementations

impl FileIO

pub fn new_with_memory() -> Self

Create a new FileIO backed by in-memory storage.

This is useful for testing scenarios where persistent storage is not needed.

pub fn new_with_fs() -> Self

Create a new FileIO backed by local filesystem storage.

This is useful for local development and testing with real files.

pub fn config(&self) -> &StorageConfig

Get the storage configuration.

pub async fn delete(&self, path: impl AsRef<str>) -> Result<()>

Deletes file.

Arguments

• path: It should be absolute path starting with scheme string used to construct FileIO.

pub async fn delete_prefix(&self, path: impl AsRef<str>) -> Result<()>

Remove the path and all nested dirs and files recursively.

Arguments

• path: It should be absolute path starting with scheme string used to construct FileIO.

Behavior

• If the path is a file or not exist, this function will be no-op.
• If the path is a empty directory, this function will remove the directory itself.
• If the path is a non-empty directory, this function will remove the directory and all nested files and directories.

pub async fn delete_stream( &self, paths: impl Stream<Item = String> + Send + 'static, ) -> Result<()>

Delete multiple files from a stream of paths.

Arguments

• paths: A stream of absolute paths starting with the scheme string used to construct FileIO.

pub async fn exists(&self, path: impl AsRef<str>) -> Result<bool>

Check file exists.

Arguments

• path: It should be absolute path starting with scheme string used to construct FileIO.

pub fn new_input(&self, path: impl AsRef<str>) -> Result<InputFile>

Creates input file.

Arguments

• path: It should be absolute path starting with scheme string used to construct FileIO.

pub fn new_output(&self, path: impl AsRef<str>) -> Result<OutputFile>

Creates output file.

Arguments

• path: It should be absolute path starting with scheme string used to construct FileIO.

Trait Implementations

impl Clone for FileIO

fn clone(&self) -> FileIO

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileIO

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.