{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Load data\n", "\n", "Make sure you have .env file in ttools or any parent dir with your Alpaca keys.\n", "\n", "```bash\n", "ACCOUNT1_LIVE_API_KEY=api_key\n", "ACCOUNT1_LIVE_SECRET_KEY=secret_key\n", "```\n", "\n", "### Cache directories\n", "\n", "Daily trade files - DATADIR/tradecache\n", "Agg data cache - DATADIR/aggcache\n", "\n", "`DATADIR` - user_data_dir from appdirs library - see `config.py`" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TTOOLS: Loaded env variables from file /Users/davidbrazda/Documents/Development/python/.env\n" ] } ], "source": [ "import pandas as pd\n", "import numpy as np\n", "from ttools.utils import AggType\n", "from datetime import datetime\n", "from ttools.aggregator_vectorized import generate_time_bars_nb, aggregate_trades\n", "from ttools.loaders import load_data, prepare_trade_cache\n", "from ttools.utils import zoneNY\n", "import vectorbtpro as vbt\n", "from lightweight_charts import PlotDFAccessor, PlotSRAccessor\n", "\n", "\n", "vbt.settings.set_theme(\"dark\")\n", "vbt.settings['plotting']['layout']['width'] = 1280\n", "vbt.settings.plotting.auto_rangebreaks = True\n", "# Set the option to display with pagination\n", "pd.set_option('display.notebook_repr_html', True)\n", "pd.set_option('display.max_rows', 10) # Number of rows per page\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Fetching aggregated data\n", "\n", "Available aggregation types: \n", "- time based bars - AggType.OHLCV\n", "- volume based bars - AggType.OHLCV_VOL, resolution = volume threshold\n", "- dollar based bars - AggType.OHLCV_DOL, resolution = dollar threshold\n", "- renko bars - AggType.OHLCV_RENKO resolution = bricksize" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/davidbrazda/Documents/Development/python/ttools/.venv/lib/python3.10/site-packages/vectorbtpro/data/base.py:1905: UserWarning: Symbols have mismatching index. Setting missing data points to NaN.\n", " data = cls_or_self.align_index(data, missing=missing_index, silence_warnings=silence_warnings)\n" ] }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#This is how to call LOAD function\n", "symbol = [\"BAC\",\"AAPL\"]\n", "#datetime in zoneNY \n", "day_start = datetime(2024, 10, 14, 9, 45, 0)\n", "day_stop = datetime(2024, 10, 16, 15, 1, 0)\n", "day_start = zoneNY.localize(day_start)\n", "day_stop = zoneNY.localize(day_stop)\n", "\n", "#requested AGG\n", "resolution = 12 #12s bars\n", "agg_type = AggType.OHLCV #other types AggType.OHLCV_VOL, AggType.OHLCV_DOL, AggType.OHLCV_RENKO\n", "exclude_conditions = ['C','O','4','B','7','V','P','W','U','Z','F','9','M','6'] #None to defaults\n", "minsize = 100 #min trade size to include\n", "main_session_only = True\n", "force_remote = False\n", "\n", "data = load_data(symbol = symbol,\n", " agg_type = agg_type,\n", " resolution = resolution,\n", " start_date = day_start,\n", " end_date = day_stop,\n", " #exclude_conditions = None,\n", " minsize = minsize,\n", " main_session_only = main_session_only,\n", " force_remote = force_remote,\n", " return_vbt = True, #returns vbt object\n", " verbose = False\n", " )\n", "\n", "data.ohlcv.data[symbol[0]].lw.plot()\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
openhighlowclosevolume
time
2024-10-14 09:45:00-04:0041.965041.97041.95041.950017895.0
2024-10-14 09:45:12-04:0041.958941.96541.95041.96506281.0
2024-10-14 09:45:24-04:0041.965042.00541.96541.99753522.0
2024-10-14 09:45:36-04:0041.990042.00541.99042.00005960.0
2024-10-14 09:45:48-04:0042.005042.04042.00542.03009113.0
..................
2024-10-16 15:00:00-04:0042.915042.91542.91042.910012872.0
2024-10-16 15:00:12-04:0042.915042.92042.91042.92007574.0
2024-10-16 15:00:24-04:0042.920042.92042.91042.92001769.0
2024-10-16 15:00:36-04:0042.920042.92042.90542.905026599.0
2024-10-16 15:00:48-04:0042.905042.90542.88042.88009216.0
\n", "

5480 rows × 5 columns

\n", "
" ], "text/plain": [ " open high low close volume\n", "time \n", "2024-10-14 09:45:00-04:00 41.9650 41.970 41.950 41.9500 17895.0\n", "2024-10-14 09:45:12-04:00 41.9589 41.965 41.950 41.9650 6281.0\n", "2024-10-14 09:45:24-04:00 41.9650 42.005 41.965 41.9975 3522.0\n", "2024-10-14 09:45:36-04:00 41.9900 42.005 41.990 42.0000 5960.0\n", "2024-10-14 09:45:48-04:00 42.0050 42.040 42.005 42.0300 9113.0\n", "... ... ... ... ... ...\n", "2024-10-16 15:00:00-04:00 42.9150 42.915 42.910 42.9100 12872.0\n", "2024-10-16 15:00:12-04:00 42.9150 42.920 42.910 42.9200 7574.0\n", "2024-10-16 15:00:24-04:00 42.9200 42.920 42.910 42.9200 1769.0\n", "2024-10-16 15:00:36-04:00 42.9200 42.920 42.905 42.9050 26599.0\n", "2024-10-16 15:00:48-04:00 42.9050 42.905 42.880 42.8800 9216.0\n", "\n", "[5480 rows x 5 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.ohlcv.data[symbol[0]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare daily trade cache\n", "This is how to prepare trade cache for given symbol and period (if daily trades are not cached they are remotely fetched.)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "symbols = [\"BAC\", \"AAPL\"]\n", "#datetime in zoneNY \n", "day_start = datetime(2024, 10, 1, 9, 45, 0)\n", "day_stop = datetime(2024, 10, 27, 15, 1, 0)\n", "day_start = zoneNY.localize(day_start)\n", "day_stop = zoneNY.localize(day_stop)\n", "force_remote = False\n", "\n", "prepare_trade_cache(symbols, day_start, day_stop, force_remote, verbose = True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepare daily trade cache - cli script\n", "\n", "Python script prepares trade cache for specified symbols and date range.\n", "\n", "Usually 1 day takes about 35s. It is stored in /tradescache/ directory as daily file keyed by symbol.\n", "\n", "To run this script in the background with specific arguments:\n", "\n", "```bash\n", "# Running without forcing remote fetch\n", "python3 prepare_cache.py --symbols BAC AAPL --day_start 2024-10-14 --day_stop 2024-10-18 &\n", "\n", "# Running with force_remote set to True\n", "python3 prepare_cache.py --symbols BAC AAPL --day_start 2024-10-14 --day_stop 2024-10-18 --force_remote &\n", "\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" } }, "nbformat": 4, "nbformat_minor": 2 }