Expand description
OpenDAL-based storage implementation for Apache Iceberg.
This crate provides OpenDalStorage and OpenDalStorageFactory,
which implement the [Storage] and
[StorageFactory] traits from the iceberg crate
using OpenDAL as the backend.
Structs§
- AwsCredential
- AWS credentials: access key ID, secret access key, and optional session token. Credential that holds the access_key and secret_key.
- Custom
AwsCredential Loader - Custom AWS credential loader.
- Hdfs
Native Storage - State of
OpenDalStorage::HdfsNative: the parsed configuration and the per-NameNode operator cache. Only the storage factories build it. - Open
DalClient Config - Backend-independent client settings shared by every
OpenDalStoragevariant. - Open
DalResolving Storage - A resolving storage that auto-detects the scheme from a path and delegates
to the appropriate
OpenDalStoragevariant. - Open
DalResolving Storage Factory - A resolving storage factory that creates
OpenDalResolvingStorageinstances.
Enums§
- Open
DalStorage - OpenDAL-based storage implementation.
- Open
DalStorage Factory - OpenDAL-based storage factory.
Constants§
- OPENDAL_
IO_ TIMEOUT_ MS - Timeout in milliseconds for each IO call on a reader, writer, lister or deleter, applied per
retry attempt via OpenDAL’s
TimeoutLayer::with_io_timeout. Defaults to 10 seconds. - OPENDAL_
RETRY_ MAX_ WAIT_ MS - Maximum wait in milliseconds between two retries on any backend. Unset keeps OpenDAL’s default.
For S3,
S3_RETRY_MAX_WAIT_MStakes precedence. - OPENDAL_
RETRY_ MIN_ WAIT_ MS - Minimum wait in milliseconds before retrying a failed operation on any backend. Unset keeps
OpenDAL’s default. For S3,
S3_RETRY_MIN_WAIT_MStakes precedence. - OPENDAL_
RETRY_ NUM_ RETRIES - Number of times to retry a failed operation on any backend,
0to disable retrying. Unset keeps OpenDAL’s default. For S3,S3_RETRY_NUM_RETRIEStakes precedence.
Traits§
- Provide
Credential - Trait for types that can asynchronously supply
AwsCredentialto aCustomAwsCredentialLoader. ProvideCredential is the trait used by signer to load the credential from the environment. ` Service may require different credential to sign the request, for example, AWS require access key and secret key, while Google Cloud Storage require token.