strategy-lab/to_explore/pyquantnews/90_MarkovChainsRegime.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e0b9b533",
   "metadata": {},
   "source": [
    "<div style=\"background-color:#000;\"><img src=\"pqn.png\"></img></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbb152ce",
   "metadata": {},
   "source": [
    "This code uses Hidden Markov Models (HMM) to identify regimes in financial time series data. It downloads historical price data, calculates returns and ranges, and uses them as features for the HMM. The model is fitted with the features to identify different market states. The identified states are then plotted to visualize market regime changes. This is useful for understanding and predicting market behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbd8082e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import yfinance as yf\n",
    "from hmmlearn import hmm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e66793fc",
   "metadata": {},
   "source": [
    "Download historical price data for SPY from Yahoo Finance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b0e43aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = yf.download(\"SPY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52141649",
   "metadata": {},
   "source": [
    "Calculate log returns of the closing prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c31978ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "returns = np.log(data.Close / data.Close.shift(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f181add",
   "metadata": {},
   "source": [
    "Calculate the range as the difference between high and low prices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f6a03ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "range = (data.High - data.Low)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a286ad5",
   "metadata": {},
   "source": [
    "Concatenate returns and range into a single DataFrame and drop any missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b6e0fed",
   "metadata": {},
   "outputs": [],
   "source": [
    "features = pd.concat([returns, range], axis=1).dropna()\n",
    "features.columns = [\"returns\", \"range\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7e18b8d",
   "metadata": {},
   "source": [
    "Initialize a Gaussian Hidden Markov Model with 3 states and fit it to the features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51c204c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = hmm.GaussianHMM(\n",
    "    n_components=3,\n",
    "    covariance_type=\"full\",\n",
    "    n_iter=1000,\n",
    ")\n",
    "model.fit(features)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b338395",
   "metadata": {},
   "source": [
    "Predict the hidden states for the given features and store them in a Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b02db04c",
   "metadata": {},
   "outputs": [],
   "source": [
    "states = pd.Series(model.predict(features), index=data.index[1:])\n",
    "states.name = \"state\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "649d1abc",
   "metadata": {},
   "source": [
    "Plot a histogram of the hidden states"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1da4348",
   "metadata": {},
   "outputs": [],
   "source": [
    "states.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99d0edca",
   "metadata": {},
   "source": [
    "Define a color map for the different states"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7944911f",
   "metadata": {},
   "outputs": [],
   "source": [
    "color_map = {\n",
    "    0.0: \"green\",\n",
    "    1.0: \"orange\",\n",
    "    2.0: \"red\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43dda8ae",
   "metadata": {},
   "source": [
    "Concatenate the closing prices and the states, drop missing values, \n",
    "set state as a hierarchical index, unstack the state index, and plot the closing prices with different colors for each state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b77b347f",
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    pd.concat([data.Close, states], axis=1)\n",
    "    .dropna()\n",
    "    .set_index(\"state\", append=True)\n",
    "    .Close\n",
    "    .unstack(\"state\")\n",
    "    .plot(color=color_map)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a9b87dd",
   "metadata": {},
   "source": [
    "<a href=\"https://pyquantnews.com/\">PyQuant News</a> is where finance practitioners level up with Python for quant finance, algorithmic trading, and market data analysis. Looking to get started? Check out the fastest growing, top-selling course to <a href=\"https://gettingstartedwithpythonforquantfinance.com/\">get started with Python for quant finance</a>. For educational purposes. Not investment advise. Use at your own risk."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}