Trait PartitioningWriter

pub trait PartitioningWriter<I = RecordBatch, O = Vec<DataFile>>: Send + 'static {
    // Required methods
    fn write<'life0, 'async_trait>(
        &'life0 mut self,
        partition_key: PartitionKey,
        input: I,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn close<'async_trait>(
        self,
    ) -> Pin<Box<dyn Future<Output = Result<O>> + Send + 'async_trait>>
       where Self: 'async_trait;
}

A writer that can write data to partitioned tables.

This trait provides methods for writing data with partition keys and closing the writer to retrieve the output.

Required Methods

fn write<'life0, 'async_trait>( &'life0 mut self, partition_key: PartitionKey, input: I, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Write data with a partition key.

Parameters

• partition_key - Partition key to determine which partition to write to
• input - The input data to write

Returns

Ok(()) on success, or an error if the write operation fails.

fn close<'async_trait>( self, ) -> Pin<Box<dyn Future<Output = Result<O>> + Send + 'async_trait>>

where Self: 'async_trait,

Close the writer and return the output.

Returns

The accumulated output from all write operations.

Implementors

impl<B, I, O> PartitioningWriter<I, O> for ClusteredWriter<B, I, O>

where B: IcebergWriterBuilder<I, O>, I: Send + 'static, O: IntoIterator + FromIterator<<O as IntoIterator>::Item> + Send + 'static, <O as IntoIterator>::Item: Send + Clone,

impl<B, I, O> PartitioningWriter<I, O> for FanoutWriter<B, I, O>

where B: IcebergWriterBuilder<I, O>, I: Send + 'static, O: IntoIterator + FromIterator<<O as IntoIterator>::Item> + Send + 'static, <O as IntoIterator>::Item: Send + Clone,