pub struct ArrowReaderOptions { /* private fields */ }Expand description
Configures Arrow reads independently of file planning and local I/O resources.
Serializable alongside planned file tasks so readers can be constructed later. In self-describing map formats such as JSON, field names are part of the serialized format: missing fields use defaults and unknown fields are ignored. Compatibility across different crate versions is not otherwise guaranteed.
let options = ArrowReaderOptions::builder()
.with_batch_size(1024)
.with_parquet_options(
ParquetReadOptions::builder()
.with_metadata_size_hint(512 * 1024)
.build(),
)
.build();
let stream = table.scan().build()?.to_arrow(options).await?;Implementations§
Source§impl ArrowReaderOptions
impl ArrowReaderOptions
Sourcepub fn builder() -> ArrowReaderOptionsBuilder<((), (), (), (), (), ())>
pub fn builder() -> ArrowReaderOptionsBuilder<((), (), (), (), (), ())>
Create a builder for building ArrowReaderOptions.
On the builder, call .batch_size(...)(optional), .data_file_concurrency_limit(...)(optional), .row_group_filtering_enabled(...)(optional), .row_selection_enabled(...)(optional), .bloom_filter_enabled(...)(optional), .parquet_options(...)(optional) to set the values of the fields.
Finally, call .build() to create the instance of ArrowReaderOptions.
Source§impl ArrowReaderOptions
impl ArrowReaderOptions
Sourcepub fn batch_size(&self) -> Option<usize>
pub fn batch_size(&self) -> Option<usize>
Returns the configured batch size.
Sourcepub fn data_file_concurrency_limit(&self) -> Option<usize>
pub fn data_file_concurrency_limit(&self) -> Option<usize>
Returns the configured data file concurrency limit.
When unset, reader construction uses execution-host parallelism. Zero is treated as one.
Sourcepub fn row_group_filtering_enabled(&self) -> bool
pub fn row_group_filtering_enabled(&self) -> bool
Returns whether row-group statistics filtering is enabled.
Enabled by default: statistics can skip groups that cannot match the predicate, generally improving performance with little overhead.
Sourcepub fn row_selection_enabled(&self) -> bool
pub fn row_selection_enabled(&self) -> bool
Returns whether page-index row selection is enabled.
Disabled by default: parsing page indexes can cost more than the reads it avoids. Evaluate it against the table’s sorting, partitioning, row-group size, and page size before enabling it.
Sourcepub fn bloom_filter_enabled(&self) -> bool
pub fn bloom_filter_enabled(&self) -> bool
Returns whether bloom-filter pruning is enabled.
Disabled by default: bloom filters require extra reads per relevant column and row group. They can eliminate groups for equality and IN predicates.
Sourcepub fn parquet_options(&self) -> &ParquetReadOptions
pub fn parquet_options(&self) -> &ParquetReadOptions
Returns the configured parquet options.
Trait Implementations§
Source§impl Clone for ArrowReaderOptions
impl Clone for ArrowReaderOptions
Source§fn clone(&self) -> ArrowReaderOptions
fn clone(&self) -> ArrowReaderOptions
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for ArrowReaderOptions
impl Debug for ArrowReaderOptions
Source§impl Default for ArrowReaderOptions
impl Default for ArrowReaderOptions
Source§impl<'de> Deserialize<'de> for ArrowReaderOptionswhere
ArrowReaderOptions: Default,
impl<'de> Deserialize<'de> for ArrowReaderOptionswhere
ArrowReaderOptions: Default,
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl PartialEq for ArrowReaderOptions
impl PartialEq for ArrowReaderOptions
Source§impl Serialize for ArrowReaderOptions
impl Serialize for ArrowReaderOptions
impl StructuralPartialEq for ArrowReaderOptions
Auto Trait Implementations§
impl Freeze for ArrowReaderOptions
impl RefUnwindSafe for ArrowReaderOptions
impl Send for ArrowReaderOptions
impl Sync for ArrowReaderOptions
impl Unpin for ArrowReaderOptions
impl UnsafeUnpin for ArrowReaderOptions
impl UnwindSafe for ArrowReaderOptions
Blanket Implementations§
§impl<U> As for U
impl<U> As for U
§fn as_<T>(self) -> Twhere
T: CastFrom<U>,
fn as_<T>(self) -> Twhere
T: CastFrom<U>,
self to type T. The semantics of numeric casting with the as operator are followed, so <T as As>::as_::<U> can be used in the same way as T as U for numeric conversions. Read moreSource§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Downcast for Twhere
T: AsAny + ?Sized,
impl<T> Downcast for Twhere
T: AsAny + ?Sized,
§fn downcast_ref<T>(&self) -> Option<&T>where
T: AsAny,
fn downcast_ref<T>(&self) -> Option<&T>where
T: AsAny,
Any.§fn downcast_mut<T>(&mut self) -> Option<&mut T>where
T: AsAny,
fn downcast_mut<T>(&mut self) -> Option<&mut T>where
T: AsAny,
Any.§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more