Skip to content
New: Star-import, signal unraveling, Python 3.12 support, and more

#
duckdb module

Module with DuckDBData.

## DuckDBData class

    DuckDBData(
        wrapper,
        data,
        single_key=True,
        classes=None,
        level_name=None,
        fetch_kwargs=None,
        returned_kwargs=None,
        last_index=None,
        delisted=None,
        tz_localize=None,
        tz_convert=None,
        missing_index=None,
        missing_columns=None,
        **kwargs
    )

Data class for fetching data using DuckDB.

See DuckDBData.pull() and DuckDBData.fetch_key() for arguments.

Usage

- Set up the connection settings globally (optional):

    >>> from vectorbtpro import *

    >>> vbt.DuckDBData.set_custom_settings(connection="database.duckdb")

- Pull tables:

    >>> data = vbt.DuckDBData.pull(["TABLE1", "TABLE2"])

- Rename tables:

    >>> data = vbt.DuckDBData.pull(
    ...     ["SYMBOL1", "SYMBOL2"],
    ...     table=vbt.key_dict({
    ...         "SYMBOL1": "TABLE1",
    ...         "SYMBOL2": "TABLE2"
    ...     })
    ... )

- Pull queries:

    >>> data = vbt.DuckDBData.pull(
    ...     ["SYMBOL1", "SYMBOL2"],
    ...     query=vbt.key_dict({
    ...         "SYMBOL1": "SELECT * FROM TABLE1",
    ...         "SYMBOL2": "SELECT * FROM TABLE2"
    ...     })
    ... )

- Pull Parquet files:

    >>> data = vbt.DuckDBData.pull(
    ...     ["SYMBOL1", "SYMBOL2"],
    ...     read_path=vbt.key_dict({
    ...         "SYMBOL1": "s1.parquet",
    ...         "SYMBOL2": "s2.parquet"
    ...     })
    ... )

Superclasses

- Analyzable
- AttrResolverMixin
- BaseDataMixin
- Cacheable
- Chainable
- Comparable
- Configured
- CustomData
- DBData
- Data
- DataWithFeatures
- ExtPandasIndexer
- HasSettings
- IndexingBase
- LocalData
- OHLCDataMixin
- PandasIndexer
- Pickleable
- PlotsBuilderMixin
- Prettified
- StatsBuilderMixin
- Wrapping

Inherited members

- AttrResolverMixin.deep_getattr()
- AttrResolverMixin.post_resolve_attr()
- AttrResolverMixin.pre_resolve_attr()
- AttrResolverMixin.resolve_attr()
- AttrResolverMixin.resolve_shortcut_attr()
- BaseDataMixin.assert_has_feature()
- BaseDataMixin.assert_has_symbol()
- BaseDataMixin.get_feature()
- BaseDataMixin.get_feature_idx()
- BaseDataMixin.get_symbol()
- BaseDataMixin.get_symbol_idx()
- BaseDataMixin.has_feature()
- BaseDataMixin.has_multiple_keys()
- BaseDataMixin.has_symbol()
- BaseDataMixin.prepare_key()
- BaseDataMixin.select_features()
- BaseDataMixin.select_symbols()
- Cacheable.get_ca_setup()
- Chainable.pipe()
- Configured.copy()
- Configured.equals()
- Configured.get_writeable_attrs()
- Configured.prettify()
- Configured.resolve_merge_kwargs()
- Configured.update_config()
- CustomData.get_custom_setting()
- CustomData.get_custom_settings()
- CustomData.has_custom_setting()
- CustomData.has_custom_settings()
- CustomData.key_match()
- CustomData.resolve_custom_setting()
- CustomData.set_custom_settings()
- DBData.classes
- DBData.close
- DBData.cls_dir
- DBData.column_only_select
- DBData.column_type
- DBData.columns
- DBData.config
- DBData.daily_log_returns
- DBData.daily_returns
- DBData.data
- DBData.delisted
- DBData.dict_type
- DBData.drawdowns
- DBData.feature_classes
- DBData.feature_config
- DBData.feature_oriented
- DBData.feature_wrapper
- DBData.features
- DBData.fetch_kwargs
- DBData.freq
- DBData.group_select
- DBData.has_ohlc
- DBData.has_ohlcv
- DBData.high
- DBData.hlc3
- DBData.iloc
- DBData.index
- DBData.indexing_kwargs
- DBData.key_wrapper
- DBData.keys
- DBData.last_index
- DBData.level_name
- DBData.loc
- DBData.log_returns
- DBData.low
- DBData.missing_columns
- DBData.missing_index
- DBData.ndim
- DBData.ohlc
- DBData.ohlc4
- DBData.ohlcv
- DBData.open
- DBData.plots_defaults
- DBData.range_only_select
- DBData.rec_state
- DBData.returned_kwargs
- DBData.returns
- DBData.returns_acc
- DBData.self_aliases
- DBData.shape
- DBData.shape_2d
- DBData.single_feature
- DBData.single_key
- DBData.single_symbol
- DBData.stats_defaults
- DBData.symbol_classes
- DBData.symbol_oriented
- DBData.symbol_wrapper
- DBData.symbols
- DBData.trade_count
- DBData.tz_convert
- DBData.tz_localize
- DBData.volume
- DBData.vwap
- DBData.wrapper
- DBData.xloc
- Data.add()
- Data.add_column()
- Data.add_feature()
- Data.add_key()
- Data.add_symbol()
- Data.align_columns()
- Data.align_data()
- Data.align_index()
- Data.build_feature_config_doc()
- Data.check_dict_type()
- Data.column_stack()
- Data.concat()
- Data.dropna()
- Data.fetch()
- Data.fix_data_dict_type()
- Data.fix_dict_types_in_kwargs()
- Data.from_csv()
- Data.from_data()
- Data.from_data_str()
- Data.from_duckdb()
- Data.from_feather()
- Data.from_hdf()
- Data.from_parquet()
- Data.from_sql()
- Data.get()
- Data.get_base_setting()
- Data.get_base_settings()
- Data.get_feature_wrapper()
- Data.get_key_wrapper()
- Data.get_keys()
- Data.get_symbol_wrapper()
- Data.has_base_setting()
- Data.has_base_settings()
- Data.has_key_dict()
- Data.indexing_func()
- Data.invert()
- Data.invert_data()
- Data.merge()
- Data.modify_state()
- Data.override_feature_config_doc()
- Data.plot()
- Data.prepare_dt()
- Data.prepare_dt_column()
- Data.prepare_dt_index()
- Data.prepare_tzaware_index()
- Data.realign()
- Data.remove()
- Data.remove_columns()
- Data.remove_features()
- Data.remove_keys()
- Data.remove_symbols()
- Data.rename()
- Data.rename_columns()
- Data.rename_features()
- Data.rename_in_dict()
- Data.rename_keys()
- Data.rename_symbols()
- Data.replace()
- Data.resample()
- Data.resolve_base_setting()
- Data.resolve_columns()
- Data.resolve_features()
- Data.resolve_key_arg()
- Data.resolve_keys()
- Data.resolve_keys_meta()
- Data.resolve_symbols()
- Data.row_stack()
- Data.run()
- Data.select()
- Data.select_classes()
- Data.select_columns()
- Data.select_delisted()
- Data.select_feature_from_dict()
- Data.select_feature_idxs()
- Data.select_feature_kwargs()
- Data.select_fetch_kwargs()
- Data.select_from_dict()
- Data.select_key_from_dict()
- Data.select_key_kwargs()
- Data.select_keys()
- Data.select_last_index()
- Data.select_returned_kwargs()
- Data.select_run_func_args()
- Data.select_run_func_kwargs()
- Data.select_symbol_from_dict()
- Data.select_symbol_idxs()
- Data.select_symbol_kwargs()
- Data.set_base_settings()
- Data.sql()
- Data.switch_class()
- Data.to_csv()
- Data.to_duckdb()
- Data.to_feather()
- Data.to_feature_oriented()
- Data.to_hdf()
- Data.to_parquet()
- Data.to_sql()
- Data.to_symbol_oriented()
- Data.transform()
- Data.try_fetch_feature()
- Data.try_fetch_symbol()
- Data.try_run()
- Data.try_update_feature()
- Data.try_update_symbol()
- Data.update()
- Data.update_classes()
- Data.update_fetch_kwargs()
- Data.update_returned_kwargs()
- Data.use_feature_config_of()
- HasSettings.get_path_setting()
- HasSettings.get_path_settings()
- HasSettings.get_setting()
- HasSettings.get_settings()
- HasSettings.has_path_setting()
- HasSettings.has_path_settings()
- HasSettings.has_setting()
- HasSettings.has_settings()
- HasSettings.reset_settings()
- HasSettings.resolve_setting()
- HasSettings.resolve_settings_paths()
- HasSettings.set_settings()
- IndexingBase.indexing_setter_func()
- OHLCDataMixin.get_daily_log_returns()
- OHLCDataMixin.get_daily_returns()
- OHLCDataMixin.get_drawdowns()
- OHLCDataMixin.get_log_returns()
- OHLCDataMixin.get_returns()
- OHLCDataMixin.get_returns_acc()
- PandasIndexer.xs()
- Pickleable.decode_config()
- Pickleable.decode_config_node()
- Pickleable.dumps()
- Pickleable.encode_config()
- Pickleable.encode_config_node()
- Pickleable.file_exists()
- Pickleable.getsize()
- Pickleable.load()
- Pickleable.loads()
- Pickleable.resolve_file_path()
- Pickleable.save()
- PlotsBuilderMixin.build_subplots_doc()
- PlotsBuilderMixin.override_subplots_doc()
- PlotsBuilderMixin.plots()
- StatsBuilderMixin.build_metrics_doc()
- StatsBuilderMixin.override_metrics_doc()
- StatsBuilderMixin.stats()
- Wrapping.regroup()
- Wrapping.resolve_column_stack_kwargs()
- Wrapping.resolve_row_stack_kwargs()
- Wrapping.resolve_self()
- Wrapping.resolve_stack_kwargs()
- Wrapping.select_col()
- Wrapping.select_col_from_obj()
- Wrapping.split()

### fetch_feature class method

    DuckDBData.fetch_feature(
        feature,
        **kwargs
    )

Fetch the table of a feature.

Uses DuckDBData.fetch_key().

### fetch_key class method

    DuckDBData.fetch_key(
        key,
        table=None,
        schema=None,
        catalog=None,
        read_path=None,
        read_format=None,
        read_options=None,
        query=None,
        connection=None,
        connection_config=None,
        start=None,
        end=None,
        align_dates=None,
        parse_dates=None,
        to_utc=None,
        tz=None,
        index_col=None,
        squeeze=None,
        df_kwargs=None,
        **sql_kwargs
    )

Fetch a feature or symbol from a DuckDB database.

Can use a table name (which defaults to the key) or a custom query.

Args

`key` :`str`

  Feature or symbol.

  If `table` and `query` are both None, becomes the table name.

  Key can be in the `SCHEMA:TABLE` format, in this case `schema` argument will be ignored.

`table` :`str`

  Table name.

  Cannot be used together with `file` or `query`.

`schema` :`str`

  Schema name.

  Cannot be used together with `file` or `query`.

`catalog` :`str`

  Catalog name.

  Cannot be used together with ``file` or query`.

`read_path` :`path_like`

  Path to a file to read.

  Cannot be used together with `table`, `schema`, `catalog`, or `query`.

`read_format` :`str`

  Format of the file to read.

  Allowed values are "csv", "parquet", and "json".

  Requires `read_path` to be set.

`read_options` :`str` or `dict`

  Options used to read the file.

  Requires `read_path` and `read_format` to be set.

  Uses DuckDBData.format_read_options() to transform a dictionary to a string.

`query` :`str` or `DuckDBPyRelation`

  Custom query.

  Cannot be used together with `catalog`, `schema`, and `table`.

`connection` :`str` or `object`
  See DuckDBData.resolve_connection().
`connection_config` :`dict`
  See DuckDBData.resolve_connection().
`start` :`any`

  Start datetime (if datetime index) or any other start value.

  Will parse with to_timestamp() if `align_dates` is True and the index is a datetime index. Otherwise, you must ensure the correct type is provided.

  Cannot be used together with `query`. Include the condition into the query.

`end` :`any`

  End datetime (if datetime index) or any other end value.

  Will parse with to_timestamp() if `align_dates` is True and the index is a datetime index. Otherwise, you must ensure the correct type is provided.

  Cannot be used together with `query`. Include the condition into the query.

`align_dates` :`bool`

  Whether to align `start` and `end` to the timezone of the index.

  Will pull one row (using `LIMIT 1`) and use `SQLData.prepare_dt` to get the index.

`parse_dates` :`bool` or `sequence` of `str`
  See Data.prepare_dt().
`to_utc` :`bool`, `str`, `or sequence` of `str`
  See Data.prepare_dt().
`tz` :`any`

  Timezone.

  See to_timezone().

`index_col` :`int`, `str`, `or list`
  One or more columns that should become the index.
`squeeze` :`int`
  Whether to squeeze a DataFrame with one column into a Series.
`df_kwargs` :`dict`
  Keyword arguments passed to `relation.df` to convert a relation to a DataFrame.
`**sql_kwargs`
  Other keyword arguments passed to `connection.execute` to run a SQL query.

For defaults, see `custom.duckdb` in data.

### fetch_symbol class method

    DuckDBData.fetch_symbol(
        symbol,
        **kwargs
    )

Fetch the table for a symbol.

Uses DuckDBData.fetch_key().

### format_read_option class method

    DuckDBData.format_read_option(
        option
    )

Format a read option.

### format_read_options class method

    DuckDBData.format_read_options(
        options
    )

Format read options.

### format_write_option class method

    DuckDBData.format_write_option(
        option
    )

Format a write option.

### format_write_options class method

    DuckDBData.format_write_options(
        options
    )

Format write options.

### get_current_schema class method

    DuckDBData.get_current_schema(
        connection=None,
        connection_config=None
    )

Get the current schema.

### list_catalogs class method

    DuckDBData.list_catalogs(
        pattern=None,
        use_regex=False,
        incl_system=False,
        connection=None,
        connection_config=None
    )

List all catalogs.

Catalogs "system" and "temp" are skipped if `incl_system` is False.

Uses CustomData.key_match() to check each symbol against `pattern`.

### list_schemas class method

    DuckDBData.list_schemas(
        catalog_pattern=None,
        schema_pattern=None,
        use_regex=False,
        catalog=None,
        incl_system=False,
        connection=None,
        connection_config=None
    )

List all schemas.

If `catalog` is None, searches for all catalog names in the database and prefixes each schema with the respective catalog name. If `catalog` is provided, returns the schemas corresponding to this catalog without a prefix. Schemas "information_schema" and "pg_catalog" are skipped if `incl_system` is False.

Uses CustomData.key_match() to check each symbol against `pattern`.

### list_tables class method

    DuckDBData.list_tables(
        *,
        catalog_pattern=None,
        schema_pattern=None,
        table_pattern=None,
        use_regex=False,
        catalog=None,
        schema=None,
        incl_system=False,
        incl_temporary=False,
        incl_views=True,
        connection=None,
        connection_config=None
    )

List all tables and views.

If `schema` is None, searches for all schema names in the database and prefixes each table with the respective catalog and schema name (unless there's only one schema which is the current schema or `schema` is `current_schema`). If `schema` is provided, returns the tables corresponding to this schema without a prefix.

Uses CustomData.key_match() to check each schema against `schema_pattern` and each table against `table_pattern`.

### pull class method

    DuckDBData.pull(
        keys=None,
        *,
        keys_are_features=None,
        features=None,
        symbols=None,
        catalog=None,
        schema=None,
        list_tables_kwargs=None,
        read_path=None,
        read_format=None,
        connection=None,
        connection_config=None,
        share_connection=None,
        **kwargs
    )

Override Data.pull() to resolve and share the connection among the keys and use the table names available in the database in case no keys were provided.

### resolve_connection class method

    DuckDBData.resolve_connection(
        connection=None,
        **connection_config
    )

Resolve the connection.

### update_feature method

    DuckDBData.update_feature(
        feature,
        **kwargs
    )

Update data of a feature.

Uses DuckDBData.update_key().

### update_key method

    DuckDBData.update_key(
        key,
        from_last_index=None,
        **kwargs
    )

Update data of a feature or symbol.

### update_symbol method

    DuckDBData.update_symbol(
        symbol,
        **kwargs
    )

Update data for a symbol.

Uses DuckDBData.update_key().

#### Cookie consent

We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.

- Google Analytics
- GitHub

Manage settings