Struct RollingFileWriter

pub struct RollingFileWriter<B: FileWriterBuilder, L: LocationGenerator, F: FileNameGenerator> { /* private fields */ }

A writer that automatically rolls over to a new file when the data size exceeds a target threshold.

This writer wraps another file writer that tracks the amount of data written. When the data size exceeds the target size, it closes the current file and starts writing to a new one.

Implementations

impl<B, L, F> RollingFileWriter<B, L, F>

where B: FileWriterBuilder, L: LocationGenerator, F: FileNameGenerator,

pub async fn write( &mut self, partition_key: &Option<PartitionKey>, input: &RecordBatch, ) -> Result<()>

Writes a record batch to the current file, rolling over to a new file if necessary.

Parameters

• partition_key - Optional partition key for the data
• input - The record batch to write

Returns

A Result indicating success or failure

Errors

Returns an error if the writer is not initialized or if writing fails

pub async fn close(self) -> Result<Vec<DataFileBuilder>>

Closes the writer and returns all data file builders.

Returns

A Result containing a vector of DataFileBuilder instances representing all files that were written, including any that were created due to rollover

Trait Implementations

impl<B: FileWriterBuilder, L: LocationGenerator, F: FileNameGenerator> CurrentFileStatus for RollingFileWriter<B, L, F>

fn current_file_path(&self) -> String

Get the current file path.

fn current_row_num(&self) -> usize

Get the current file row number.

fn current_written_size(&self) -> usize

Get the current file written size.

impl<B, L, F> Debug for RollingFileWriter<B, L, F>

where B: FileWriterBuilder, L: LocationGenerator, F: FileNameGenerator,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.