Struct iceberg_datafusion::IcebergTableProviderFactory

pub struct IcebergTableProviderFactory {}

A factory that implements DataFusion’s TableProviderFactory to create IcebergTableProvider instances.

Example

The following example demonstrates how to create an Iceberg external table using SQL in a DataFusion session with IcebergTableProviderFactory:

use std::sync::Arc;

use datafusion::execution::session_state::SessionStateBuilder;
use datafusion::prelude::*;
use datafusion::sql::TableReference;
use iceberg_datafusion::IcebergTableProviderFactory;

#[tokio::main]
async fn main() {
    // Create a new session context
    let mut state = SessionStateBuilder::new().with_default_features().build();

    // Register the IcebergTableProviderFactory in the session
    state.table_factories_mut().insert(
        "ICEBERG".to_string(),
        Arc::new(IcebergTableProviderFactory::new()),
    );

    let ctx = SessionContext::new_with_state(state);

    // Define the table reference and the location of the Iceberg metadata file
    let table_ref = TableReference::bare("my_iceberg_table");
    // /path/to/iceberg/metadata
    let metadata_file_path = format!(
        "{}/testdata/table_metadata/{}",
        env!("CARGO_MANIFEST_DIR"),
        "TableMetadataV2.json"
    );

    // SQL command to create the Iceberg external table
    let sql = format!(
        "CREATE EXTERNAL TABLE {} STORED AS ICEBERG LOCATION '{}'",
        table_ref, metadata_file_path
    );

    // Execute the SQL to create the external table
    ctx.sql(&sql).await.expect("Failed to create table");

    // Verify the table was created by retrieving the table provider
    let table_provider = ctx
        .table_provider(table_ref)
        .await
        .expect("Table not found");

    println!("Iceberg external table created successfully.");
}

Note

This factory is designed to work with the DataFusion query engine, specifically for handling Iceberg tables in external table commands. Currently, this implementation supports only reading Iceberg tables, with the creation of new tables not yet available.

Errors

An error will be returned if any unsupported feature, such as partition columns, order expressions, constraints, or column defaults, is detected in the table creation command.

Implementations

impl IcebergTableProviderFactory

pub fn new() -> Self

Trait Implementations

impl Debug for IcebergTableProviderFactory

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for IcebergTableProviderFactory

fn default() -> IcebergTableProviderFactory

Returns the “default value” for a type.

impl TableProviderFactory for IcebergTableProviderFactory

fn create<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _state: &'life1 dyn Session, cmd: &'life2 CreateExternalTable, ) -> Pin<Box<dyn Future<Output = DFResult<Arc<dyn TableProvider>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Create a TableProvider with the given url