Skip to content
New: Star-import, signal unraveling, Python 3.12 support, and more

#
parquet module

Module with ParquetData.

## ParquetData class

    ParquetData(
        wrapper,
        data,
        single_key=True,
        classes=None,
        level_name=None,
        fetch_kwargs=None,
        returned_kwargs=None,
        last_index=None,
        delisted=None,
        tz_localize=None,
        tz_convert=None,
        missing_index=None,
        missing_columns=None,
        **kwargs
    )

Data class for fetching Parquet data using PyArrow or FastParquet.

Superclasses

- Analyzable
- AttrResolverMixin
- BaseDataMixin
- Cacheable
- Chainable
- Comparable
- Configured
- CustomData
- Data
- DataWithFeatures
- ExtPandasIndexer
- FileData
- HasSettings
- IndexingBase
- LocalData
- OHLCDataMixin
- PandasIndexer
- Pickleable
- PlotsBuilderMixin
- Prettified
- StatsBuilderMixin
- Wrapping

Inherited members

- AttrResolverMixin.deep_getattr()
- AttrResolverMixin.post_resolve_attr()
- AttrResolverMixin.pre_resolve_attr()
- AttrResolverMixin.resolve_attr()
- AttrResolverMixin.resolve_shortcut_attr()
- BaseDataMixin.assert_has_feature()
- BaseDataMixin.assert_has_symbol()
- BaseDataMixin.get_feature()
- BaseDataMixin.get_feature_idx()
- BaseDataMixin.get_symbol()
- BaseDataMixin.get_symbol_idx()
- BaseDataMixin.has_feature()
- BaseDataMixin.has_multiple_keys()
- BaseDataMixin.has_symbol()
- BaseDataMixin.prepare_key()
- BaseDataMixin.select_features()
- BaseDataMixin.select_symbols()
- Cacheable.get_ca_setup()
- Chainable.pipe()
- Configured.copy()
- Configured.equals()
- Configured.get_writeable_attrs()
- Configured.prettify()
- Configured.resolve_merge_kwargs()
- Configured.update_config()
- CustomData.get_custom_setting()
- CustomData.get_custom_settings()
- CustomData.has_custom_setting()
- CustomData.has_custom_settings()
- CustomData.key_match()
- CustomData.resolve_custom_setting()
- CustomData.set_custom_settings()
- Data.add()
- Data.add_column()
- Data.add_feature()
- Data.add_key()
- Data.add_symbol()
- Data.align_columns()
- Data.align_data()
- Data.align_index()
- Data.build_feature_config_doc()
- Data.check_dict_type()
- Data.column_stack()
- Data.concat()
- Data.dropna()
- Data.fetch()
- Data.fix_data_dict_type()
- Data.fix_dict_types_in_kwargs()
- Data.from_csv()
- Data.from_data()
- Data.from_data_str()
- Data.from_duckdb()
- Data.from_feather()
- Data.from_hdf()
- Data.from_parquet()
- Data.from_sql()
- Data.get()
- Data.get_base_setting()
- Data.get_base_settings()
- Data.get_feature_wrapper()
- Data.get_key_wrapper()
- Data.get_keys()
- Data.get_symbol_wrapper()
- Data.has_base_setting()
- Data.has_base_settings()
- Data.has_key_dict()
- Data.indexing_func()
- Data.invert()
- Data.invert_data()
- Data.merge()
- Data.modify_state()
- Data.override_feature_config_doc()
- Data.plot()
- Data.prepare_dt()
- Data.prepare_dt_column()
- Data.prepare_dt_index()
- Data.prepare_tzaware_index()
- Data.realign()
- Data.remove()
- Data.remove_columns()
- Data.remove_features()
- Data.remove_keys()
- Data.remove_symbols()
- Data.rename()
- Data.rename_columns()
- Data.rename_features()
- Data.rename_in_dict()
- Data.rename_keys()
- Data.rename_symbols()
- Data.replace()
- Data.resample()
- Data.resolve_base_setting()
- Data.resolve_columns()
- Data.resolve_features()
- Data.resolve_key_arg()
- Data.resolve_keys()
- Data.resolve_symbols()
- Data.row_stack()
- Data.run()
- Data.select()
- Data.select_classes()
- Data.select_columns()
- Data.select_delisted()
- Data.select_feature_from_dict()
- Data.select_feature_idxs()
- Data.select_feature_kwargs()
- Data.select_fetch_kwargs()
- Data.select_from_dict()
- Data.select_key_from_dict()
- Data.select_key_kwargs()
- Data.select_keys()
- Data.select_last_index()
- Data.select_returned_kwargs()
- Data.select_run_func_args()
- Data.select_run_func_kwargs()
- Data.select_symbol_from_dict()
- Data.select_symbol_idxs()
- Data.select_symbol_kwargs()
- Data.set_base_settings()
- Data.sql()
- Data.switch_class()
- Data.to_csv()
- Data.to_duckdb()
- Data.to_feather()
- Data.to_feature_oriented()
- Data.to_hdf()
- Data.to_parquet()
- Data.to_sql()
- Data.to_symbol_oriented()
- Data.transform()
- Data.try_fetch_feature()
- Data.try_fetch_symbol()
- Data.try_run()
- Data.try_update_feature()
- Data.try_update_symbol()
- Data.update()
- Data.update_classes()
- Data.update_fetch_kwargs()
- Data.update_returned_kwargs()
- Data.use_feature_config_of()
- FileData.classes
- FileData.close
- FileData.cls_dir
- FileData.column_only_select
- FileData.column_type
- FileData.columns
- FileData.config
- FileData.daily_log_returns
- FileData.daily_returns
- FileData.data
- FileData.delisted
- FileData.dict_type
- FileData.drawdowns
- FileData.feature_classes
- FileData.feature_config
- FileData.feature_oriented
- FileData.feature_wrapper
- FileData.features
- FileData.fetch_kwargs
- FileData.freq
- FileData.group_select
- FileData.has_ohlc
- FileData.has_ohlcv
- FileData.high
- FileData.hlc3
- FileData.iloc
- FileData.index
- FileData.indexing_kwargs
- FileData.is_dir_match()
- FileData.is_file_match()
- FileData.key_wrapper
- FileData.keys
- FileData.last_index
- FileData.level_name
- FileData.list_paths()
- FileData.loc
- FileData.log_returns
- FileData.low
- FileData.match_path()
- FileData.missing_columns
- FileData.missing_index
- FileData.ndim
- FileData.ohlc
- FileData.ohlc4
- FileData.ohlcv
- FileData.open
- FileData.path_to_key()
- FileData.plots_defaults
- FileData.pull()
- FileData.range_only_select
- FileData.rec_state
- FileData.resolve_keys_meta()
- FileData.returned_kwargs
- FileData.returns
- FileData.returns_acc
- FileData.self_aliases
- FileData.shape
- FileData.shape_2d
- FileData.single_feature
- FileData.single_key
- FileData.single_symbol
- FileData.stats_defaults
- FileData.symbol_classes
- FileData.symbol_oriented
- FileData.symbol_wrapper
- FileData.symbols
- FileData.trade_count
- FileData.tz_convert
- FileData.tz_localize
- FileData.volume
- FileData.vwap
- FileData.wrapper
- FileData.xloc
- HasSettings.get_path_setting()
- HasSettings.get_path_settings()
- HasSettings.get_setting()
- HasSettings.get_settings()
- HasSettings.has_path_setting()
- HasSettings.has_path_settings()
- HasSettings.has_setting()
- HasSettings.has_settings()
- HasSettings.reset_settings()
- HasSettings.resolve_setting()
- HasSettings.resolve_settings_paths()
- HasSettings.set_settings()
- IndexingBase.indexing_setter_func()
- OHLCDataMixin.get_daily_log_returns()
- OHLCDataMixin.get_daily_returns()
- OHLCDataMixin.get_drawdowns()
- OHLCDataMixin.get_log_returns()
- OHLCDataMixin.get_returns()
- OHLCDataMixin.get_returns_acc()
- PandasIndexer.xs()
- Pickleable.decode_config()
- Pickleable.decode_config_node()
- Pickleable.dumps()
- Pickleable.encode_config()
- Pickleable.encode_config_node()
- Pickleable.file_exists()
- Pickleable.getsize()
- Pickleable.load()
- Pickleable.loads()
- Pickleable.resolve_file_path()
- Pickleable.save()
- PlotsBuilderMixin.build_subplots_doc()
- PlotsBuilderMixin.override_subplots_doc()
- PlotsBuilderMixin.plots()
- StatsBuilderMixin.build_metrics_doc()
- StatsBuilderMixin.override_metrics_doc()
- StatsBuilderMixin.stats()
- Wrapping.regroup()
- Wrapping.resolve_column_stack_kwargs()
- Wrapping.resolve_row_stack_kwargs()
- Wrapping.resolve_self()
- Wrapping.resolve_stack_kwargs()
- Wrapping.select_col()
- Wrapping.select_col_from_obj()
- Wrapping.split()

### fetch_feature class method

    ParquetData.fetch_feature(
        feature,
        **kwargs
    )

Fetch the Parquet file of a feature.

Uses ParquetData.fetch_key().

### fetch_key class method

    ParquetData.fetch_key(
        key,
        path=None,
        tz=None,
        squeeze=None,
        keep_partition_cols=None,
        engine=None,
        **read_kwargs
    )

Fetch the Parquet file of a feature or symbol.

Args

`key` :`hashable`
  Feature or symbol.
`path` :`str`

  Path.

  If `path` is None, uses `key` as the path to the Parquet file.

`tz` :`any`

  Target timezone.

  See to_timezone().

`squeeze` :`int`
  Whether to squeeze a DataFrame with one column into a Series.
`keep_partition_cols` :`bool`

  Whether to return partitioning columns (if any).

  If None, will remove any partitioning column that is "group" or "group_{index}".

  Retrieves the list of partitioning columns with ParquetData.list_partition_cols().

`engine` :`str`
  See `pd.read_parquet`.
`**read_kwargs`
  Other keyword arguments passed to `pd.read_parquet`.

See https://pandas.pydata.org/docs/reference/api/pandas.read_parquet.html for other arguments.

For defaults, see `custom.parquet` in data.

### fetch_symbol class method

    ParquetData.fetch_symbol(
        symbol,
        **kwargs
    )

Fetch the Parquet file of a symbol.

Uses ParquetData.fetch_key().

### is_default_partition_col class method

    ParquetData.is_default_partition_col(
        level
    )

Return whether a partitioning column is a default partitioning column.

### is_parquet_dir class method

    ParquetData.is_parquet_dir(
        path
    )

Return whether the path is a directory that is a group itself or contains groups of Parquet partitions.

### is_parquet_file class method

    ParquetData.is_parquet_file(
        path
    )

Return whether the path is a Parquet file.

### is_parquet_group_dir class method

    ParquetData.is_parquet_group_dir(
        path
    )

Return whether the path is a directory that is a group of Parquet partitions.

Note

Assumes the Hive partitioning scheme.

### list_partition_cols class method

    ParquetData.list_partition_cols(
        path
    )

List partitioning columns under a path.

Note

Assumes the Hive partitioning scheme.

### update_feature method

    ParquetData.update_feature(
        feature,
        **kwargs
    )

Update data of a feature.

Uses ParquetData.update_key() with `key_is_feature=True`.

### update_key method

    ParquetData.update_key(
        key,
        key_is_feature=False,
        **kwargs
    )

Update data of a feature or symbol.

### update_symbol method

    ParquetData.update_symbol(
        symbol,
        **kwargs
    )

Update data for a symbol.

Uses ParquetData.update_key() with `key_is_feature=False`.

#### Cookie consent

We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.

- Google Analytics
- GitHub

Manage settings