Strategy Analysis with Pandas TA and AI/ML

This is a Work in Progress and subject to change!
Contributions are welcome and accepted!
Examples below are for educational purposes only.
NOTE: The watchlist module is independent of Pandas TA. To easily use it, copy it from your local pandas_ta installation directory into your project directory.

Required Packages

Uncomment the packages you need to install or are missing

In [1]:

#!pip install numpy
#!pip install pandas
#!pip install mplfinance
#!pip install pandas-datareader
#!pip install requests_cache
#!pip install tqdm
#!pip install alphaVantage-api # Required for Watchlist

In [2]:

%pylab inline
import datetime as dt
import random as rnd
from sys import float_info as sflt

from tqdm import tqdm

import numpy as np
import pandas as pd
pd.set_option("max_rows", 100)
pd.set_option("max_columns", 20)

import mplfinance as mpf
import pandas_ta as ta

from tqdm.notebook import trange, tqdm

from watchlist import colors, Watchlist # Is this failing? If so, copy it locally. See above.

print(f"Numpy v{np.__version__}")
print(f"Pandas v{pd.__version__}")
print(f"mplfinance v{mpf.__version__}")
print(f"\nPandas TA v{ta.version}\nTo install the Latest Version:\n$ pip install -U git+https://github.com/twopirllc/pandas-ta\n")
%matplotlib inline

Populating the interactive namespace from numpy and matplotlib
Numpy v1.20.2
Pandas v1.2.4
mplfinance v0.12.7a12

Pandas TA v0.2.74b0
To install the Latest Version:
$ pip install -U git+https://github.com/twopirllc/pandas-ta

MISC Function(s)

In [3]:

def recent_bars(df, tf: str = "1y"):
    # All Data: 0, Last Four Years: 0.25, Last Two Years: 0.5, This Year: 1, Last Half Year: 2, Last Quarter: 4
    yearly_divisor = {"all": 0, "10y": 0.1, "5y": 0.2, "4y": 0.25, "3y": 1./3, "2y": 0.5, "1y": 1, "6mo": 2, "3mo": 4}
    yd = yearly_divisor[tf] if tf in yearly_divisor.keys() else 0
    return int(ta.RATE["TRADING_DAYS_PER_YEAR"] / yd) if yd > 0 else df.shape[0]

Data Collection

In [4]:

tf = "D"
tickers = ["SPY", "QQQ", "AAPL", "TSLA", "BTC-USD"]
watch = Watchlist(tickers, tf=tf, ds_name="yahoo", timed=True)
# watch.strategy = ta.CommonStrategy # If you have a Custom Strategy, you can use it here.
watch.load(tickers, analyze=True, verbose=False)

[!] Loading All: SPY, QQQ, AAPL, TSLA, BTC-USD
[+] Downloading[yahoo]: SPY[D]
[+] Saving: /Users/kj/av_data/SPY_D.csv
[i] Runtime: 525.9195 ms (0.5259 s)
[+] Downloading[yahoo]: QQQ[D]
[+] Saving: /Users/kj/av_data/QQQ_D.csv
[i] Runtime: 480.6160 ms (0.4806 s)
[+] Downloading[yahoo]: AAPL[D]
[+] Saving: /Users/kj/av_data/AAPL_D.csv
[i] Runtime: 508.6772 ms (0.5087 s)
[+] Downloading[yahoo]: TSLA[D]
[+] Saving: /Users/kj/av_data/TSLA_D.csv
[i] Runtime: 477.4917 ms (0.4775 s)
[+] Downloading[yahoo]: BTC-USD[D]
[+] Saving: /Users/kj/av_data/BTC-USD_D.csv
[i] Runtime: 497.8067 ms (0.4978 s)

Asset Selection

In [5]:

ticker = tickers[0] # change tickers by changing the index
print(f"{ticker} {watch.data[ticker].shape}\nColumns: {', '.join(list(watch.data[ticker].columns))}")

SPY (7128, 12)
Columns: open, high, low, close, volume, dividends, split, SMA_10, SMA_20, SMA_50, SMA_200, VOL_SMA_20

Trim it

In [6]:

duration = "1y"
asset = watch.data[ticker]
recent = recent_bars(asset, duration)
asset.columns = asset.columns.str.lower()
asset.drop(columns=["dividends", "split"], errors="ignore", inplace=True)
asset = asset.copy().tail(recent)
asset

Out [6]:

	open	high	low	close	volume	sma_10	sma_20	sma_50	sma_200	vol_sma_20
date
2020-05-20	291.150942	293.168574	290.904888	292.243408	85861700	285.033035	283.029170	266.659134	292.472755	98976520.00
2020-05-21	292.105626	292.971742	289.054549	290.225769	78293900	285.741672	283.806703	267.094627	292.529205	97655730.00
2020-05-22	289.920683	290.963951	288.591985	290.776947	63958200	286.036938	284.420363	268.054838	292.587587	96600480.00
2020-05-26	297.164459	297.420365	290.796577	294.359436	88951400	286.684552	285.012367	268.671635	292.636502	97153220.00
2020-05-27	297.351478	298.778604	292.184342	298.739227	104817400	288.343939	285.888319	269.952733	292.717008	97130590.00
...	...	...	...	...	...	...	...	...	...	...
2021-05-13	407.070007	412.350006	407.019989	410.279999	106394000	415.589999	415.910498	404.295667	367.011310	79515840.00
2021-05-14	413.209991	417.489990	413.179993	416.579987	82123100	415.517999	415.876497	405.117861	367.487226	79520130.00
2021-05-17	415.390015	416.390015	413.359985	415.519989	65129200	415.249997	415.891997	405.780709	367.963577	78851665.00
2021-05-18	415.799988	416.059998	411.769989	411.940002	59266000	414.881998	415.880496	406.410032	368.409374	77722375.00
2021-05-19	406.920013	410.250000	405.329987	408.234985	66477971	414.130496	415.488745	406.856610	368.825426	77706623.55

252 rows × 10 columns

Trend Creation

A Trend is the result of some calculation or condition of one or more indicators. For simplicity, a Trend is either True or 1 and No Trend is False or 0. Using the Hello World of Trends, the Golden/Death Cross, it's Trend is Long when long = ma(close, 50) > ma(close, 200) and Short when short = ma(close, 50) < ma(close, 200) .

In [7]:

# Example Long Trends
# long = ta.sma(asset.close, 50) > ta.sma(asset.close, 200) # SMA(50) > SMA(200) "Golden/Death Cross"
# long = ta.sma(asset.close, 10) > ta.sma(asset.close, 20) # SMA(10) > SMA(20)
long = ta.ema(asset.close, 8) > ta.ema(asset.close, 21) # EMA(8) > EMA(21)
# long = ta.increasing(ta.ema(asset.close, 50))
# long = ta.macd(asset.close).iloc[:,1] > 0 # MACD Histogram is positive
# long = ta.amat(asset.close, 50, 200).AMATe_LR_2  # Long Run of AMAT(50, 200) with lookback of 2 bars

# long &= ta.increasing(ta.ema(asset.close, 50), 2) # Uncomment for further long restrictions, in this case when EMA(50) is increasing/sloping upwards
# long = 1 - long # uncomment to create a short signal of the trend

asset.ta.ema(length=8, sma=False, append=True)
asset.ta.ema(length=21, sma=False, append=True)
asset.ta.ema(length=50, sma=False, append=True)
asset.ta.percent_return(append=True)
print("TA Columns Added:")
asset[asset.columns[5:]].tail()

Out [7]:

TA Columns Added:

	sma_10	sma_20	sma_50	sma_200	vol_sma_20	EMA_8	EMA_21	EMA_50	PCTRET_1
date
2021-05-13	415.589999	415.910498	404.295667	367.011310	79515840.00	413.680581	413.448932	405.708303	0.012013
2021-05-14	415.517999	415.876497	405.117861	367.487226	79520130.00	414.324894	413.733573	406.134643	0.015355
2021-05-17	415.249997	415.891997	405.780709	367.963577	78851665.00	414.590470	413.895974	406.502696	-0.002545
2021-05-18	414.881998	415.880496	406.410032	368.409374	77722375.00	414.001478	413.718159	406.715924	-0.008616
2021-05-19	414.130496	415.488745	406.856610	368.825426	77706623.55	412.720035	413.219688	406.775495	-0.008994

Trend Signals

Given a Trend, Trend Signals returns the Trend, Trades, Entries and Exits as boolean integers. When asbool=True, it returns Trends, Entries and Exits as boolean values which is helpful when combined with the vectorbt backtesting package.

In [8]:

trendy = asset.ta.tsignals(long, asbool=False, append=True)
trendy.tail()

Out [8]:

	TS_Trends	TS_Trades	TS_Entries	TS_Exits
date
2021-05-13	1	0	0	0
2021-05-14	1	0	0	0
2021-05-17	1	0	0	0
2021-05-18	1	0	0	0
2021-05-19	0	-1	0	1

Trend Entries & Exits & Trade Table

This is a simple way to reduce the Asset DataFrame to a Trade Table with Dates, Signals, and Entries and Exits. Gives you an idea what to expect before running through a backtester such as vectorbt.

In [9]:

entries = trendy.TS_Entries * asset.close
entries = entries[~np.isclose(entries, 0)]
entries.dropna(inplace=True)
entries.name = "Entry"

exits = trendy.TS_Exits * asset.close
exits = exits[~np.isclose(exits, 0)]
exits.dropna(inplace=True)
exits.name = "Exit"

total_trades = trendy.TS_Trades.abs().sum()
rt_trades = int(trendy.TS_Trades.abs().sum() // 2)

all_trades = trendy.TS_Trades.copy().fillna(0)
all_trades = all_trades[all_trades != 0]

trades = pd.DataFrame({
    "Signal": all_trades,
    entries.name: entries,
    exits.name: exits
})

# Show some stats if there is an active trade (when there is an odd number of round trip trades)
if total_trades % 2 != 0:
    unrealized_pnl = asset.close.iloc[-1] - entries.iloc[-1]
    unrealized_pnl_pct_change = 100 * ((asset.close.iloc[-1] / entries.iloc[-1]) - 1)
    print("Current Trade:")
    print(f"Price Entry | Last:\t{entries.iloc[-1]:.4f} | {asset.close.iloc[-1]:.4f}")
    print(f"Unrealized PnL | %:\t{unrealized_pnl:.4f} | {unrealized_pnl_pct_change:.4f}%")
print(f"\nTrades Total | Round Trip:\t{total_trades} | {rt_trades}")
print(f"Trade Coverage: {100 * asset.TS_Trends.sum() / asset.shape[0]:.2f}%")

trades

Out [9]:

Trades Total | Round Trip:	10 | 5
Trade Coverage: 80.56%

	Signal	Entry	Exit
date
2020-06-18	1	306.859009	NaN
2020-06-29	-1	NaN	300.973206
2020-06-30	1	304.828522	NaN
2020-09-11	-1	NaN	330.234192
2020-10-05	1	337.213409	NaN
2020-10-28	-1	NaN	324.211609
2020-11-05	1	347.614868	NaN
2021-03-03	-1	NaN	380.174835
2021-03-10	1	388.308197	NaN
2021-05-19	-1	NaN	408.234985

In [ ]:

Visualization

Chart Display Strings

In [10]:

extime = ta.get_time(to_string=True)
first_date, last_date = asset.index[0], asset.index[-1]
f_date = f"{first_date.day_name()} {first_date.month}-{first_date.day}-{first_date.year}"
l_date = f"{last_date.day_name()} {last_date.month}-{last_date.day}-{last_date.year}"
last_ohlcv = f"Last OHLCV: ({asset.iloc[-1].open:.4f}, {asset.iloc[-1].high:.4f}, {asset.iloc[-1].low:.4f}, {asset.iloc[-1].close:.4f}, {int(asset.iloc[-1].volume)})"
ptitle = f"\n{ticker} [{tf} for {duration}({recent} bars)] from {f_date} to {l_date}\n{last_ohlcv}\n{extime}"

Trade Chart

In [11]:

# chart = asset["close"] #asset[["close", "SMA_10", "SMA_20", "SMA_50", "SMA_200"]]
# chart = asset[["close", "SMA_10", "SMA_20"]]
chart = asset[["close", "EMA_8", "EMA_21", "EMA_50"]]
chart.plot(figsize=(16, 10), color=colors("BkGrOrRd"), title=ptitle, grid=True)

Out [11]:

<AxesSubplot:title={'center':'\nSPY [D for 1y(252 bars)] from Wednesday 5-20-2020 to Wednesday 5-19-2021\nLast OHLCV: (406.9200, 410.2500, 405.3300, 408.2350, 66477971)\nWednesday May 19, 2021, NYSE: 7:20:10, Local: 11:20:10 PDT, Day 139/365 (38.00%)'}, xlabel='date'>

Long and Short Trends

Trends are either a Trend (1) or No Trend (0) depending on the Trend passed into *Trend Signals

In [12]:

long_trend = trendy.TS_Trends
short_trend = 1 - long_trend

long_trend.plot(figsize=(16, 0.85), kind="area", stacked=True, color=colors()[0], alpha=0.25) # Green Area
short_trend.plot(figsize=(16, 0.85), kind="area", stacked=True, color=colors()[1], alpha=0.25) # Red Area

Out [12]:

<AxesSubplot:xlabel='date'>

Trades or Trade Signals

The Trades are either Enter (1) or Exit (-1) or No Position/Action (0). These are based on the Trend passed into Trend Signals whether they are Long or Short Trends.

In [13]:

trendy.TS_Trades.plot(figsize=(16, 1.5), color=colors("BkBl")[0], grid=True)

Out [13]:

<AxesSubplot:xlabel='date'>

Active Returns

Active Returns are returns made during the course of the Trend. They are simply the product of the returns and the Trend

In [14]:

asset["ACTRET_1"] = trendy.TS_Trends * asset.PCTRET_1
asset[["PCTRET_1", "ACTRET_1"]].plot(figsize=(16, 3), color=colors("GyOr"), alpha=1, grid=True).axhline(0, color="black")

Out [14]:

<matplotlib.lines.Line2D at 0x12f2f8fa0>

Buy and Hold Returns (PCTRET_1) vs. Cum. Active Returns (ACTRET_1)

In [15]:

asset[["PCTRET_1", "ACTRET_1"]].cumsum().plot(figsize=(16, 3), kind="area", stacked=False, color=colors("GyOr"), title="B&H vs. Cum. Active Returns", alpha=.4, grid=True).axhline(0, color="black")

Out [15]:

<matplotlib.lines.Line2D at 0x12f280af0>

Disclaimer

All investments involve risk, and the past performance of a security, industry, sector, market, financial product, trading strategy, or individual’s trading does not guarantee future results or returns. Investors are fully responsible for any investment decisions they make. Such decisions should be based solely on an evaluation of their financial circumstances, investment objectives, risk tolerance, and liquidity needs.
Any opinions, news, research, analyses, prices, or other information offered is provided as general market commentary, and does not constitute investment advice. I will not accept liability for any loss or damage, including without limitation any loss of profit, which may arise directly or indirectly from use of or reliance on such information.

267 KiB Raw Blame History Unescape Escape