Files

David Brazda 08f06456b7 loader/aggregator added

2024-10-30 08:35:15 +01:00

1.6 MiB

Raw Blame History

Load data¶

Make sure you have .env file in ttools or any parent dir with your Alpaca keys.

ACCOUNT1_LIVE_API_KEY=api_key
ACCOUNT1_LIVE_SECRET_KEY=secret_key

Cache directories¶

Daily trade files - DATADIR/tradecache Agg data cache - DATADIR/aggcache

DATADIR - user_data_dir from appdirs library - see config.py

In [1]:

import pandas as pd
import numpy as np
from ttools.utils import AggType
from datetime import datetime
from ttools.aggregator_vectorized import generate_time_bars_nb, aggregate_trades
from ttools.loaders import load_data, prepare_trade_cache
from ttools.utils import zoneNY
import vectorbtpro as vbt
from lightweight_charts import PlotDFAccessor, PlotSRAccessor


vbt.settings.set_theme("dark")
vbt.settings['plotting']['layout']['width'] = 1280
vbt.settings.plotting.auto_rangebreaks = True
# Set the option to display with pagination
pd.set_option('display.notebook_repr_html', True)
pd.set_option('display.max_rows', 10)  # Number of rows per page

Loaded env variables from file /Users/davidbrazda/Documents/Development/python/.env

Fetching aggregated data¶

Available aggregation types:

time based bars - AggType.OHLCV
volume based bars - AggType.OHLCV_VOL, resolution = volume threshold
dollar based bars - AggType.OHLCV_DOL, resolution = dollar threshold
renko bars - AggType.OHLCV_RENKO resolution = bricksize

In [8]:

#This is how to call LOAD function
symbol = ["BAC"]
#datetime in zoneNY 
day_start = datetime(2024, 10, 14, 9, 45, 0)
day_stop = datetime(2024, 10, 16, 15, 1, 0)
day_start = zoneNY.localize(day_start)
day_stop = zoneNY.localize(day_stop)

#requested AGG
resolution = 12 #12s bars
agg_type = AggType.OHLCV #other types AggType.OHLCV_VOL, AggType.OHLCV_DOL, AggType.OHLCV_RENKO
exclude_conditions = ['C','O','4','B','7','V','P','W','U','Z','F','9','M','6'] #None to defaults
minsize = 100 #min trade size to include
main_session_only = True
force_remote = True

ohlcv_df = load_data(symbol = symbol,
                     agg_type = agg_type,
                     resolution = resolution,
                     start_date = day_start,
                     end_date = day_stop,
                     #exclude_conditions = None,
                     minsize = minsize,
                     main_session_only = main_session_only,
                     force_remote = False
                     )

#returns symbol keyed dict with pd.DataFrame as values
bac_df = ohlcv_df["BAC"]

basic_data = vbt.Data.from_data(vbt.symbol_dict(ohlcv_df), tz_convert=zoneNY)
vbt.settings['plotting']['auto_rangebreaks'] = True
#basic_data.ohlcv.plot()
basic_data.ohlcv.data[symbol[0]].lw.plot()

Contains 3  market days

Processing market days: 100%|██████████| 3/3 [00:00<00:00, 4557.37it/s]
Processing market days to fetch: 100%|██████████| 3/3 [00:00<00:00, 481.22it/s]
Fetching data:   0%|          | 0/3 [00:00<?, ?it/s]

Loaded from CACHE /Users/davidbrazda/Library/Application Support/v2realbot/tradecache/BAC-2024-10-14.parquet
filtrujeme 09:45:00 23:59:59.999999
excluding conditions ['4', '6', '7', '9', 'B', 'C', 'F', 'M', 'O', 'P', 'U', 'V', 'W', 'Z']
Loaded from CACHE

Fetching data:  33%|███▎      | 1/3 [00:00<00:01,  1.36it/s]

 /Users/davidbrazda/Library/Application Support/v2realbot/tradecache/BAC-2024-10-15.parquet
minsize 100
filtrujeme 00:00:00 23:59:59.999999
excluding conditions ['4', '6', '7', '9', 'B', 'C', 'F', 'M', 'O', 'P', 'U', 'V', 'W', 'Z']
Loaded from CACHE /Users/davidbrazda/Library/Application Support/v2realbot/tradecache/BAC-2024-10-16.parquet
filtrujeme 00:00:00 15:01:00
excluding conditions ['4', '6', '7', '9', 'B', 'C', 'F', 'M', 'O', 'P', 'U', 'V', 'W', 'Z']
minsize 100

Fetching data: 100%|██████████| 3/3 [00:01<00:00,  1.71it/s]

minsize 100
Saved to agg_cache /Users/davidbrazda/Library/Application Support/v2realbot/aggcache/BAC-AggType.OHLCV-12-2024-10-14T09-45-00-2024-10-16T15-01-00-4679BCFMOPUVWZ-100-True.parquet

In [6]:

basic_data.ohlcv.data[symbol[0]]

Out[6]:

	open	high	low	close	volume
time
2024-10-14 09:45:00.080054045-04:00	41.9650	42.040	41.950	42.005	50000.0
2024-10-14 09:46:12.714761019-04:00	42.0050	42.100	42.000	42.095	50000.0
2024-10-14 09:47:20.320812941-04:00	42.0950	42.115	42.065	42.105	50000.0
2024-10-14 09:48:26.167362928-04:00	42.1050	42.130	42.090	42.105	50000.0
2024-10-14 09:49:06.589205027-04:00	42.1001	42.110	42.060	42.105	50000.0
...	...	...	...	...	...
2024-10-16 14:56:44.563248873-04:00	42.8900	42.890	42.880	42.880	50000.0
2024-10-16 14:57:38.830776930-04:00	42.8900	42.915	42.880	42.910	50000.0
2024-10-16 14:58:41.628561020-04:00	42.9150	42.920	42.905	42.910	50000.0
2024-10-16 14:59:50.505049944-04:00	42.9100	42.920	42.910	42.910	50000.0
2024-10-16 15:00:47.022783041-04:00	42.9100	42.910	42.880	42.880	19063.0

1352 rows × 5 columns

Prepare daily trade cache¶

This is how to prepare trade cache for given symbol and period (if daily trades are not cached they are remotely fetched.)