wassname/catalyst
1
0
Fork 0
mirror of https://github.com/wassname/catalyst.git synced 2026-07-28 11:18:19 +08:00
Code Issues Packages Projects Releases Wiki Activity

DOC: use sphinx docs

Browse Source
This commit is contained in:
llllllllll
2015-11-06 15:10:34 -05:00
parent 7d35a17f76
commit 39a4cf0a9e
20 changed files with 2614 additions and 2997 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/appendix.rst
+8
View File
@@ -0,0 +1,8 @@
Zipline API
-----------
For each of these api functions, the ``self`` argument is implicitly the current
:class:`~zipline.algorithm.TradingAlgorithm`
.. automodule:: zipline.api
:members:
docs/source/beginner-tutorial.rst
+818
View File
@@ -0,0 +1,818 @@
Zipline beginner tutorial
-------------------------
Basics
~~~~~~
Zipline is an open-source algorithmic trading simulator written in
Python.
The source can be found at: https://github.com/quantopian/zipline
Some benefits include:
- Realistic: slippage, transaction costs, order delays.
- Stream-based: Process each event individually, avoids look-ahead
bias.
- Batteries included: Common transforms (moving average) as well as
common risk calculations (Sharpe).
- Developed and continuously updated by
`Quantopian <https://www.quantopian.com>`__ which provides an
easy-to-use web-interface to Zipline, 10 years of minute-resolution
historical US stock data, and live-trading capabilities. This
tutorial is directed at users wishing to use Zipline without using
Quantopian. If you instead want to get started on Quantopian, see
`here <https://www.quantopian.com/faq#get-started>`__.
This tutorial assumes that you have zipline correctly installed, see the
`installation
instructions <https://github.com/quantopian/zipline#installation>`__ if
you haven't set up zipline yet.
Every ``zipline`` algorithm consists of two functions you have to
define: \* ``initialize(context)`` \* ``handle_data(context, data)``
Before the start of the algorithm, ``zipline`` calls the
``initialize()`` function and passes in a ``context`` variable.
``context`` is a persistent namespace for you to store variables you
need to access from one algorithm iteration to the next.
After the algorithm has been initialized, ``zipline`` calls the
``handle_data()`` function once for each event. At every call, it passes
the same ``context`` variable and an event-frame called ``data``
containing the current trading bar with open, high, low, and close
(OHLC) prices as well as volume for each stock in your universe. For
more information on these functions, see the `relevant part of the
Quantopian docs <https://www.quantopian.com/help#api-toplevel>`__.
My first algorithm
~~~~~~~~~~~~~~~~~~
Lets take a look at a very simple algorithm from the ``examples``
directory, ``buyapple.py``:
.. code:: python
!tail ../../zipline/examples/buyapple.py
.. parsed-literal::
# Load price data from yahoo.
data = load_from_yahoo(stocks=['AAPL'], indexes={}, start=start,
end=end)
# Create and run the algorithm.
algo = TradingAlgorithm(initialize=initialize, handle_data=handle_data,
identifiers=['AAPL'])
results = algo.run(data)
analyze(results=results)
As you can see, we first have to import some functions we would like to
use. All functions commonly used in your algorithm can be found in
``zipline.api``. Here we are using ``order()`` which takes two arguments
-- a security object, and a number specifying how many stocks you would
like to order (if negative, ``order()`` will sell/short stocks). In this
case we want to order 10 shares of Apple at each iteration. For more
documentation on ``order()``, see the `Quantopian
docs <https://www.quantopian.com/help#api-order>`__.
You don't have to use the ``symbol()`` function and could just pass in
``AAPL`` directly but it is good practice as this way your code will be
Quantopian compatible.
Finally, the ``record()`` function allows you to save the value of a
variable at each iteration. You provide it with a name for the variable
together with the variable itself: ``varname=var``. After the algorithm
finished running you will have access to each variable value you tracked
with ``record()`` under the name you provided (we will see this further
below). You also see how we can access the current price data of the
AAPL stock in the ``data`` event frame (for more information see
`here <https://www.quantopian.com/help#api-event-properties>`__.
Running the algorithm
~~~~~~~~~~~~~~~~~~~~~
To now test this algorithm on financial data, ``zipline`` provides two
interfaces. A command-line interface and an ``IPython Notebook``
interface.
Command line interface
^^^^^^^^^^^^^^^^^^^^^^
After you installed zipline you should be able to execute the following
from your command line (e.g. ``cmd.exe`` on Windows, or the Terminal app
on OSX):
.. code:: python
!run_algo.py --help
.. parsed-literal::
usage: run_algo.py [-h] [-c FILE] [--algofile ALGOFILE] [--data-frequency {minute,daily}] [--start START] [--end END]
[--capital_base CAPITAL_BASE] [--source {yahoo}] [--source_time_column SOURCE_TIME_COLUMN] [--symbols SYMBOLS]
[--output OUTPUT] [--metadata_path METADATA_PATH] [--metadata_index METADATA_INDEX] [--print-algo] [--no-print-algo]
Zipline version 0.8.0rc1.
optional arguments:
-h, --help show this help message and exit
-c FILE, --conf_file FILE
Specify config file
--algofile ALGOFILE, -f ALGOFILE
--data-frequency {minute,daily}
--start START, -s START
--end END, -e END
--capital_base CAPITAL_BASE
--source {yahoo}, -d {yahoo}
--source_time_column SOURCE_TIME_COLUMN, -t SOURCE_TIME_COLUMN
--symbols SYMBOLS
--output OUTPUT, -o OUTPUT
--metadata_path METADATA_PATH, -m METADATA_PATH
--metadata_index METADATA_INDEX, -x METADATA_INDEX
--print-algo, -p
--no-print-algo, -q
Note that you have to omit the preceding '!' when you call
``run_algo.py``, this is only required by the IPython Notebook in which
this tutorial was written.
As you can see there are a couple of flags that specify where to find
your algorithm (``-f``) as well as parameters specifying which stock
data to load from Yahoo! finance (``--symbols``) and the time-range
(``--start`` and ``--end``). Finally, you'll want to save the
performance metrics of your algorithm so that you can analyze how it
performed. This is done via the ``--output`` flag and will cause it to
write the performance ``DataFrame`` in the pickle Python file format.
Note that you can also define a configuration file with these parameters
that you can then conveniently pass to the ``-c`` option so that you
don't have to supply the command line args all the time (see the .conf
files in the examples directory).
Thus, to execute our algorithm from above and save the results to
``buyapple_out.pickle`` we would call ``run_algo.py`` as follows:
.. code:: python
!run_algo.py -f ../../zipline/examples/buyapple.py --start 2000-1-1 --end 2014-1-1 --symbols AAPL -o buyapple_out.pickle
.. parsed-literal::
AAPL
[2015-11-04 22:45:32.820166] INFO: Performance: Simulated 3521 trading days out of 3521.
[2015-11-04 22:45:32.820314] INFO: Performance: first open: 2000-01-03 14:31:00+00:00
[2015-11-04 22:45:32.820401] INFO: Performance: last close: 2013-12-31 21:00:00+00:00
``run_algo.py`` first outputs the algorithm contents. It then fetches
historical price and volume data of Apple from Yahoo! finance in the
desired time range, calls the ``initialize()`` function, and then
streams the historical stock price day-by-day through ``handle_data()``.
After each call to ``handle_data()`` we instruct ``zipline`` to order 10
stocks of AAPL. After the call of the ``order()`` function, ``zipline``
enters the ordered stock and amount in the order book. After the
``handle_data()`` function has finished, ``zipline`` looks for any open
orders and tries to fill them. If the trading volume is high enough for
this stock, the order is executed after adding the commission and
applying the slippage model which models the influence of your order on
the stock price, so your algorithm will be charged more than just the
stock price \* 10. (Note, that you can also change the commission and
slippage model that ``zipline`` uses, see the `Quantopian
docs <https://www.quantopian.com/help#ide-slippage>`__ for more
information).
Note that there is also an ``analyze()`` function printed.
``run_algo.py`` will try and look for a file with the ending with
``_analyze.py`` and the same name of the algorithm (so
``buyapple_analyze.py``) or an ``analyze()`` function directly in the
script. If an ``analyze()`` function is found it will be called *after*
the simulation has finished and passed in the performance ``DataFrame``.
(The reason for allowing specification of an ``analyze()`` function in a
separate file is that this way ``buyapple.py`` remains a valid
Quantopian algorithm that you can copy&paste to the platform).
Lets take a quick look at the performance ``DataFrame``. For this, we
use ``pandas`` from inside the IPython Notebook and print the first ten
rows. Note that ``zipline`` makes heavy usage of ``pandas``, especially
for data input and outputting so it's worth spending some time to learn
it.
.. code:: python
import pandas as pd
perf = pd.read_pickle('buyapple_out.pickle') # read in perf DataFrame
perf.head()
.. raw:: html
<div style="max-height:1000px;max-width:1500px;overflow:auto;">
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>AAPL</th>
<th>algo_volatility</th>
<th>algorithm_period_return</th>
<th>alpha</th>
<th>benchmark_period_return</th>
<th>benchmark_volatility</th>
<th>beta</th>
<th>capital_used</th>
<th>ending_cash</th>
<th>ending_exposure</th>
<th>...</th>
<th>short_exposure</th>
<th>short_value</th>
<th>shorts_count</th>
<th>sortino</th>
<th>starting_cash</th>
<th>starting_exposure</th>
<th>starting_value</th>
<th>trading_days</th>
<th>transactions</th>
<th>treasury_period_return</th>
</tr>
</thead>
<tbody>
<tr>
<th>2000-01-03 21:00:00</th>
<td>3.738314</td>
<td>0.000000e+00</td>
<td>0.000000e+00</td>
<td>-0.065800</td>
<td>-0.009549</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.00000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>0.00000</td>
<td>1</td>
<td>[]</td>
<td>0.0658</td>
</tr>
<tr>
<th>2000-01-04 21:00:00</th>
<td>3.423135</td>
<td>3.367492e-07</td>
<td>-3.000000e-08</td>
<td>-0.064897</td>
<td>-0.047528</td>
<td>0.323229</td>
<td>0.000001</td>
<td>-34.53135</td>
<td>9999965.46865</td>
<td>34.23135</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>0.00000</td>
<td>2</td>
<td>[{u'order_id': u'513357725cb64a539e3dd02b47da7...</td>
<td>0.0649</td>
</tr>
<tr>
<th>2000-01-05 21:00:00</th>
<td>3.473229</td>
<td>4.001918e-07</td>
<td>-9.906000e-09</td>
<td>-0.066196</td>
<td>-0.045697</td>
<td>0.329321</td>
<td>0.000001</td>
<td>-35.03229</td>
<td>9999930.43636</td>
<td>69.46458</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>9999965.46865</td>
<td>34.23135</td>
<td>34.23135</td>
<td>3</td>
<td>[{u'order_id': u'd7d4ad03cfec4d578c0d817dc3829...</td>
<td>0.0662</td>
</tr>
<tr>
<th>2000-01-06 21:00:00</th>
<td>3.172661</td>
<td>4.993979e-06</td>
<td>-6.410420e-07</td>
<td>-0.065758</td>
<td>-0.044785</td>
<td>0.298325</td>
<td>-0.000006</td>
<td>-32.02661</td>
<td>9999898.40975</td>
<td>95.17983</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>-12731.780516</td>
<td>9999930.43636</td>
<td>69.46458</td>
<td>69.46458</td>
<td>4</td>
<td>[{u'order_id': u'1fbf5e9bfd7c4d9cb2e8383e1085e...</td>
<td>0.0657</td>
</tr>
<tr>
<th>2000-01-07 21:00:00</th>
<td>3.322945</td>
<td>5.977002e-06</td>
<td>-2.201900e-07</td>
<td>-0.065206</td>
<td>-0.018908</td>
<td>0.375301</td>
<td>0.000005</td>
<td>-33.52945</td>
<td>9999864.88030</td>
<td>132.91780</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>-12629.274583</td>
<td>9999898.40975</td>
<td>95.17983</td>
<td>95.17983</td>
<td>5</td>
<td>[{u'order_id': u'9ea6b142ff09466b9113331a37437...</td>
<td>0.0652</td>
</tr>
</tbody>
</table>
<p>5 rows × 39 columns</p>
</div>
As you can see, there is a row for each trading day, starting on the
first business day of 2000. In the columns you can find various
information about the state of your algorithm. The very first column
``AAPL`` was placed there by the ``record()`` function mentioned earlier
and allows us to plot the price of apple. For example, we could easily
examine now how our portfolio value changed over time compared to the
AAPL stock price.
.. code:: python
%pylab inline
figsize(12, 12)
import matplotlib.pyplot as plt
ax1 = plt.subplot(211)
perf.portfolio_value.plot(ax=ax1)
ax1.set_ylabel('portfolio value')
ax2 = plt.subplot(212, sharex=ax1)
perf.AAPL.plot(ax=ax2)
ax2.set_ylabel('AAPL stock price')
.. parsed-literal::
Populating the interactive namespace from numpy and matplotlib
.. parsed-literal::
<matplotlib.text.Text at 0x7ff5c6147f90>
.. image:: tutorial_files/tutorial_11_2.png
As you can see, our algorithm performance as assessed by the
``portfolio_value`` closely matches that of the AAPL stock price. This
is not surprising as our algorithm only bought AAPL every chance it got.
IPython Notebook
~~~~~~~~~~~~~~~~
The `IPython Notebook <http://ipython.org/notebook.html>`__ is a very
powerful browser-based interface to a Python interpreter (this tutorial
was written in it). As it is already the de-facto interface for most
quantitative researchers ``zipline`` provides an easy way to run your
algorithm inside the Notebook without requiring you to use the CLI.
To use it you have to write your algorithm in a cell and let ``zipline``
know that it is supposed to run this algorithm. This is done via the
``%%zipline`` IPython magic command that is available after you
``import zipline`` from within the IPython Notebook. This magic takes
the same arguments as the command line interface described above. Thus
to run the algorithm from above with the same parameters we just have to
execute the following cell after importing ``zipline`` to register the
magic.
.. code:: python
import zipline
.. code:: python
%%zipline --start 2000-1-1 --end 2014-1-1 --symbols AAPL -o perf_ipython
from zipline.api import symbol, order, record
def initialize(context):
pass
def handle_data(context, data):
order(symbol('AAPL'), 10)
record(AAPL=data[symbol('AAPL')].price)
.. parsed-literal::
AAPL
Note that we did not have to specify an input file as above since the
magic will use the contents of the cell and look for your algorithm
functions there. Also, instead of defining an output file we are
specifying a variable name with ``-o`` that will be created in the name
space and contain the performance ``DataFrame`` we looked at above.
.. code:: python
perf_ipython.head()
.. raw:: html
<div style="max-height:1000px;max-width:1500px;overflow:auto;">
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>AAPL</th>
<th>algo_volatility</th>
<th>algorithm_period_return</th>
<th>alpha</th>
<th>benchmark_period_return</th>
<th>benchmark_volatility</th>
<th>beta</th>
<th>capital_used</th>
<th>ending_cash</th>
<th>ending_exposure</th>
<th>...</th>
<th>short_exposure</th>
<th>short_value</th>
<th>shorts_count</th>
<th>sortino</th>
<th>starting_cash</th>
<th>starting_exposure</th>
<th>starting_value</th>
<th>trading_days</th>
<th>transactions</th>
<th>treasury_period_return</th>
</tr>
</thead>
<tbody>
<tr>
<th>2000-01-03 21:00:00</th>
<td>3.738314</td>
<td>0.000000e+00</td>
<td>0.000000e+00</td>
<td>-0.065800</td>
<td>-0.009549</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.00000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>0.00000</td>
<td>1</td>
<td>[]</td>
<td>0.0658</td>
</tr>
<tr>
<th>2000-01-04 21:00:00</th>
<td>3.423135</td>
<td>3.367492e-07</td>
<td>-3.000000e-08</td>
<td>-0.064897</td>
<td>-0.047528</td>
<td>0.323229</td>
<td>0.000001</td>
<td>-34.53135</td>
<td>9999965.46865</td>
<td>34.23135</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>10000000.00000</td>
<td>0.00000</td>
<td>0.00000</td>
<td>2</td>
<td>[{u'commission': 0.3, u'amount': 10, u'sid': 0...</td>
<td>0.0649</td>
</tr>
<tr>
<th>2000-01-05 21:00:00</th>
<td>3.473229</td>
<td>4.001918e-07</td>
<td>-9.906000e-09</td>
<td>-0.066196</td>
<td>-0.045697</td>
<td>0.329321</td>
<td>0.000001</td>
<td>-35.03229</td>
<td>9999930.43636</td>
<td>69.46458</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0.000000</td>
<td>9999965.46865</td>
<td>34.23135</td>
<td>34.23135</td>
<td>3</td>
<td>[{u'commission': 0.3, u'amount': 10, u'sid': 0...</td>
<td>0.0662</td>
</tr>
<tr>
<th>2000-01-06 21:00:00</th>
<td>3.172661</td>
<td>4.993979e-06</td>
<td>-6.410420e-07</td>
<td>-0.065758</td>
<td>-0.044785</td>
<td>0.298325</td>
<td>-0.000006</td>
<td>-32.02661</td>
<td>9999898.40975</td>
<td>95.17983</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>-12731.780516</td>
<td>9999930.43636</td>
<td>69.46458</td>
<td>69.46458</td>
<td>4</td>
<td>[{u'commission': 0.3, u'amount': 10, u'sid': 0...</td>
<td>0.0657</td>
</tr>
<tr>
<th>2000-01-07 21:00:00</th>
<td>3.322945</td>
<td>5.977002e-06</td>
<td>-2.201900e-07</td>
<td>-0.065206</td>
<td>-0.018908</td>
<td>0.375301</td>
<td>0.000005</td>
<td>-33.52945</td>
<td>9999864.88030</td>
<td>132.91780</td>
<td>...</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>-12629.274583</td>
<td>9999898.40975</td>
<td>95.17983</td>
<td>95.17983</td>
<td>5</td>
<td>[{u'commission': 0.3, u'amount': 10, u'sid': 0...</td>
<td>0.0652</td>
</tr>
</tbody>
</table>
<p>5 rows × 39 columns</p>
</div>
Manual (advanced)
~~~~~~~~~~~~~~~~~
If you are happy with either way above you can safely skip this passage.
To provide a closer look at how ``zipline`` actually works it is
instructive to see how we run an algorithm without any of the interfaces
demonstrated above which hide the actual ``zipline`` API.
.. code:: python
import pytz
from datetime import datetime
from zipline.algorithm import TradingAlgorithm
from zipline.utils.factory import load_bars_from_yahoo
# Load data manually from Yahoo! finance
start = datetime(2000, 1, 1, 0, 0, 0, 0, pytz.utc)
end = datetime(2012, 1, 1, 0, 0, 0, 0, pytz.utc)
data = load_bars_from_yahoo(stocks=['AAPL'], start=start,
end=end)
# Define algorithm
def initialize(context):
pass
def handle_data(context, data):
order(symbol('AAPL'), 10)
record(AAPL=data[symbol('AAPL')].price)
# Create algorithm object passing in initialize and
# handle_data functions
algo_obj = TradingAlgorithm(initialize=initialize,
handle_data=handle_data)
# Run algorithm
perf_manual = algo_obj.run(data)
.. parsed-literal::
AAPL
As you can see, we again define the functions as above but we manually
pass them to the ``TradingAlgorithm`` class which is the main
``zipline`` class for running algorithms. We also manually load the data
using ``load_bars_from_yahoo()`` and pass it to the
``TradingAlgorithm.run()`` method which kicks off the backtest
simulation.
Access to previous prices using ``history``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Working example: Dual Moving Average Cross-Over
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The Dual Moving Average (DMA) is a classic momentum strategy. It's
probably not used by any serious trader anymore but is still very
instructive. The basic idea is that we compute two rolling or moving
averages (mavg) -- one with a longer window that is supposed to capture
long-term trends and one shorter window that is supposed to capture
short-term trends. Once the short-mavg crosses the long-mavg from below
we assume that the stock price has upwards momentum and long the stock.
If the short-mavg crosses from above we exit the positions as we assume
the stock to go down further.
As we need to have access to previous prices to implement this strategy
we need a new concept: History
``history()`` is a convenience function that keeps a rolling window of
data for you. The first argument is the number of bars you want to
collect, the second argument is the unit (either ``'1d'`` for ``'1m'``
but note that you need to have minute-level data for using ``1m``). For
a more detailed description ``history()``'s features, see the
`Quantopian docs <https://www.quantopian.com/help#ide-history>`__. While
you can directly use the ``history()`` function on Quantopian, in
``zipline`` you have to register each history container you want to use
with ``add_history()`` and pass it the same arguments as the history
function below. Lets look at the strategy which should make this clear:
.. code:: python
%%zipline --start 2000-1-1 --end 2014-1-1 --symbols AAPL -o perf_dma
from zipline.api import order_target, record, symbol, history, add_history
import numpy as np
def initialize(context):
# Register 2 histories that track daily prices,
# one with a 100 window and one with a 300 day window
add_history(100, '1d', 'price')
add_history(300, '1d', 'price')
context.i = 0
def handle_data(context, data):
# Skip first 300 days to get full windows
context.i += 1
if context.i < 300:
return
# Compute averages
# history() has to be called with the same params
# from above and returns a pandas dataframe.
short_mavg = history(100, '1d', 'price').mean()
long_mavg = history(300, '1d', 'price').mean()
# Trading logic
if short_mavg[0] > long_mavg[0]:
# order_target orders as many shares as needed to
# achieve the desired number of shares.
order_target(symbol('AAPL'), 100)
elif short_mavg[0] < long_mavg[0]:
order_target(symbol('AAPL'), 0)
# Save values for later inspection
record(AAPL=data[symbol('AAPL')].price,
short_mavg=short_mavg[0],
long_mavg=long_mavg[0])
def analyze(context, perf):
fig = plt.figure()
ax1 = fig.add_subplot(211)
perf.portfolio_value.plot(ax=ax1)
ax1.set_ylabel('portfolio value in $')
ax2 = fig.add_subplot(212)
perf['AAPL'].plot(ax=ax2)
perf[['short_mavg', 'long_mavg']].plot(ax=ax2)
perf_trans = perf.ix[[t != [] for t in perf.transactions]]
buys = perf_trans.ix[[t[0]['amount'] > 0 for t in perf_trans.transactions]]
sells = perf_trans.ix[
[t[0]['amount'] < 0 for t in perf_trans.transactions]]
ax2.plot(buys.index, perf.short_mavg.ix[buys.index],
'^', markersize=10, color='m')
ax2.plot(sells.index, perf.short_mavg.ix[sells.index],
'v', markersize=10, color='k')
ax2.set_ylabel('price in $')
plt.legend(loc=0)
plt.show()
.. parsed-literal::
AAPL
.. image:: tutorial_files/tutorial_22_1.png
Here we are explicitly defining an ``analyze()`` function that gets
automatically called once the backtest is done (this is not possible on
Quantopian currently).
Although it might not be directly apparent, the power of ``history()``
(pun intended) can not be under-estimated as most algorithms make use of
prior market developments in one form or another. You could easily
devise a strategy that trains a classifier with
`scikit-learn <http://scikit-learn.org/stable/>`__ which tries to
predict future market movements based on past prices (note, that most of
the ``scikit-learn`` functions require ``numpy.ndarray``\ s rather than
``pandas.DataFrame``\ s, so you can simply pass the underlying
``ndarray`` of a ``DataFrame`` via ``.values``).
We also used the ``order_target()`` function above. This and other
functions like it can make order management and portfolio rebalancing
much easier. See the `Quantopian documentation on order
functions <https://www.quantopian.com/help#api-order-methods>`__ fore
more details.
Conclusions
~~~~~~~~~~~
We hope that this tutorial gave you a little insight into the
architecture, API, and features of ``zipline``. For next steps, check
out some of the
`examples <https://github.com/quantopian/zipline/tree/master/zipline/examples>`__.
Feel free to ask questions on `our mailing
list <https://groups.google.com/forum/#!forum/zipline>`__, report
problems on our `GitHub issue
tracker <https://github.com/quantopian/zipline/issues?state=open>`__,
`get
involved <https://github.com/quantopian/zipline/wiki/Contribution-Requests>`__,
and `checkout Quantopian <https://quantopian.com>`__.
docs/source/conf.py
+91
View File
@@ -0,0 +1,91 @@
import sys
import os
from zipline import __version__ as version
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('..'))
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'sphinx.ext.doctest',
'sphinx.ext.extlinks',
'sphinx.ext.autosummary',
]
extlinks = dict(issue=('https://github.com/quantopian/zipline/issues/%s', '#'))
# -- Docstrings ---------------------------------------------------------------
extensions += ['numpydoc']
numpydoc_show_class_members = False
# Add any paths that contain templates here, relative to this directory.
templates_path = ['.templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Zipline'
copyright = u'2015, Quantopian Inc.'
# The full version, including alpha/beta/rc tags.
release = version
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
if not on_rtd: # only import and set the theme if we're building docs locally
try:
import sphinx_rtd_theme
except ImportError:
html_theme = 'default'
html_theme_path = []
else:
html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# The name of the Pygments (syntax highlighting) style to use.
highlight_language = 'python'
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = os.path.join('svg', 'zipline.ico')
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['.static']
# If false, no index is generated.
html_use_index = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
html_show_sphinx = False
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
html_show_copyright = True
# Output file base name for HTML help builder.
htmlhelp_basename = 'ziplinedoc'
intersphinx_mapping = {
'http://docs.python.org/dev': None,
'numpy': ('http://docs.scipy.org/doc/numpy/', None),
'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None),
'pandas': ('http://pandas.pydata.org/pandas-docs/stable/', None),
}
doctest_global_setup = "import zipline"
docs/source/index.rst
+11
View File
@@ -0,0 +1,11 @@
.. include:: ../../README.rst
Index
-----
.. toctree::
:maxdepth: 1
beginner-tutorial
releases
appendix
docs/source/releases.rst
+9
View File
@@ -0,0 +1,9 @@
=============
Release Notes
=============
.. include:: whatsnew/0.8.0.txt
.. include:: whatsnew/0.7.0.txt
.. include:: whatsnew/0.6.1.txt
docs/source/tutorial_files/tutorial_11_2.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

docs/source/tutorial_files/tutorial_22_1.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 53 KiB

docs/source/whatsnew/0.6.1.txt
+327
View File
@@ -0,0 +1,327 @@
Release 0.6.1
-------------
:Release: 0.6.1
:Date: April 23, 2014
Highlights
~~~~~~~~~~
* Major fixes to risk calculations, see Bug Fixes section.
* Port of ``history()`` function, see Enhancements section
* Start of support for Quantopian algorithm script-syntax, see ENH
section.
* conda package manager support, see Build section.
Enhancements
~~~~~~~~~~~~
* Always process new orders
i.e. on bars where ``handle_data`` isn't called, but there is 'clock'
data e.g. a consistent benchmark, process orders.
* Empty positions are now filtered from the portfolio container.
To help prevent algorithms from operating on positions that are not in
the existing universe of stocks.
Formerly, iterating over positions would return positions for stocks
which had zero shares held. (Where an explicit check in algorithm code
for ``pos.amount != 0`` could prevent from using a non-existent
position.)
* Add trading calendar for BMF&Bovespa.
* Add beginning of algo script support.
* Starts on the path of parity with the script syntax in Quantopian's IDE
on https://quantopian.com
Example:
.. code-block:: python
from datetime import datetime import pytz
from zipline import TradingAlgorithm
from zipline.utils.factory import load_from_yahoo
from zipline.api import order
def initialize(context):
context.test = 10
def handle_date(context, data):
order('AAPL', 10)
print(context.test)
if __name__ == '__main__':
import pylab as pl
start = datetime(2008, 1, 1, 0, 0, 0, 0, pytz.utc)
end = datetime(2010, 1, 1, 0, 0, 0, 0, pytz.utc)
data = load_from_yahoo(
stocks=['AAPL'],
indexes={},
start=start,
end=end)
data = data.dropna()
algo = TradingAlgorithm(
initialize=initialize,
handle_data=handle_date)
results = algo.run(data)
results.portfolio_value.plot()
pl.show()
* Add HDF5 and CSV sources.
* Limit ``handle_data`` to times with market data.
To prevent cases where custom data types had unaligned timestamps, only
call ``handle_data`` when market data passes through.
Custom data that comes before market data will still update the data
bar. But the handling of that data will only be done when there is
actionable market data.
* Extended commission PerShare method to allow a minimum cost per trade.
* Add symbol api function
A ``symbol()`` lookup feature was added to Quantopian. By adding the
same API function to zipline we can make copy&pasting of a Zipline algo
to Quantopian easier.
* Add simulated random trade source.
Added a new data source that emits events with certain user-specified
frequency (minute or daily).
This allows users to backtest and debug an algorithm in minute mode to
provide a cleaner path towards Quantopian.
* Remove dependency on benchmark for trading day calendar.
Instead of the benchmarks' index, the trading calendar is now used to
populate the environment's trading days.
Remove ``extra_date`` field, since unlike the benchmarks list, the
trading calendar can generate future dates, so dates for current day
trading do not need to be appended.
Motivations:
- The source for the open and close/early close calendar and the
trading day calendar is now the same, which should help prevent
potential issues due to misalignment.
- Allows configurations where the benchmark is provided as a generator
based data source to need to supply a second benchmark list just to
populate dates.
* Port ``history()`` API method from Quantopian.
Opens the core of the ``history()`` function that was previously only
available on the Quantopian platform.
The history method is analoguous to the ``batch_transform``
function/decorator, but with a hopefully more precise specification of
the frequency and period of the previous bar data that is captured.
Example usage:
.. code-block:: python
from zipline.api import history, add_history
def initialize(context):
add_history(bar_count=2, frequency='1d', field='price')
def handle_data(context, data):
prices = history(bar_count=2, frequency='1d', field='price')
context.last_prices = prices
N.B. this version of history lacks the backfilling capability that
allows the return a full DataFrame on the first bar.
Bug Fixes
~~~~~~~~~
* Adjust benchmark events to match market hours (:issue:`241).
Previously benchmark events were emitted at 0:00 on the day the
benchmark related to: in 'minute' emission mode this meant that the
benchmarks were emitted before any intra-day trades were processed.
* Ensure perf stats are generated for all days
When running with minutely emissions the simulator would report to the
user that it simulated 'n - 1' days (where n is the number of days
specified in the simulation params). Now the correct number of trading
days are reported as being simulated.
* Fix repr for cumulative risk metrics.
The ``__repr__`` for RiskMetricsCumulative was referring to an older
structure of the class, causing an exception when printed.
Also, now prints the last values in the metrics DataFrame.
* Prevent minute emission from crashing at end of available data.
The next day calculation was causing an error when a minute emission
algorithm reached the end of available data.
Instead of a generic exception when available data is reached, raise and
catch a named exception so that the tradesimulation loop can skip over,
since the next market close is not needed at the end.
* Fix pandas indexing in trading calendar. This could alternatively be filed
under Performance. Index using loc instead of the inefficient index-ing of
day, then time.
* Prevent crash in vwap transform due to non-existent member.
The WrongDataForTransform was referencing a ``self.fields`` member,
which did not exist.
Add a self.fields member set to ``price`` and ``volume`` and use it to
iterate over during the check.
* Fix max drawdown calculation.
The input into max drawdown was incorrect, causing the bad results. i.e.
the ``compounded_log_returns`` were not values representative of the
algorithms total return at a given time, though
``calculate_max_drawdown`` was treating the values as if they were.
Instead, the ``algorithm_period_returns`` series is now used, which does
provide the total return.
* Fix cost basis calculation.
Cost basis calculation now takes direction of txn into account.
Closing a long position or covering a short shouldn't affect the cost
basis.
* Fix floating point error in ``order()``.
Where order amounts that were near an integer could accidentally be
floored or ceilinged (depending on being postive or negative) to the
wrong integer.
e.g. an amount stored internally as -27.99999 was converted to -27
instead of -28.
* Update perf period state when positions are changed by splits.
Otherwise, ``self._position_amounts`` will be out of sync with
position.amount, etc.
* Fix misalignment of downside series calc when using exact dates.
An oddity that was exposed while working on making the return series
passed to the risk module more exact, the series comparison between the
returns and mean returns was unbalanced, because the mean returns were
not masked down to the downside data points; however, in most, if not
all cases this was papered over by the call to ``.valid()`` which was
removed in this change set.
* Check that self.logger exists before using it.
``self.logger`` is initialized as ``None`` and there is no guarantee
that users have set it, so check that it exists before trying to pass
messages to it.
* Prevent out of sync market closes in performance tracker.
In situations where the performance tracker has been reset or patched to
handle state juggling with warming up live data, the ``market_close``
member of the performance tracker could end up out of sync with the
current algo time as determined by the performance tracker.
The symptom was dividends never triggering, because the end of day
checks would not match the current time.
Fix by having the tradesimulation loop be responsible, in minute/minute
mode, for advancing the market close and passing that value to the
performance tracker, instead of having the market close advanced by the
performance tracker as well.
* Fix numerous cumulative and period risk calculations.
The calculations that are expected to change are:
- ``cumulative.beta``
- ``cumulative.alpha``
- ``cumulative.information``
- ``cumulative.sharpe``
- ``period.sortino``
How Risk Calculations Are Changing
Risk Fixes for Both Period and Cumulative
Downside Risk
Use sample instead of population for standard deviation.
Add a rounding factor, so that if the two values are close for a given
dt, that they do not count as a downside value, which would throw off
the denominator of the standard deviation of the downside diffs.
Standard Deviation Type
Across the board the standard deviation has been standardized to using a
'sample' calculation, whereas before cumulative risk was mostly using
'population'. Using ``ddof=1`` with ``np.std`` calculates as if the
values are a sample.
Cumulative Risk Fixes
Beta
Use the daily algorithm returns and benchmarks instead of annualized
mean returns.
Volatility
Use sample instead of population with standard deviation.
The volatility is an input to other calculations so this change affects
Sharpe and Information ratio calculations.
Information Ratio
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
Alpha
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
Period Risk Fixes
Sortino
Now uses the downside risk of the daily return vs. the mean algorithm
returns for the minimum acceptable return instead of the treasury
return.
The above required adding the calculation of the mean algorithm returns
for period risk.
Also, uses ``algorithm_period_returns`` and ``tresaury_period_return``
as the cumulative Sortino does, instead of using algorithm returns for
both inputs into the Sortino calculation.
Performance
~~~~~~~~~~~
* Removed ``alias_dt`` transform in favor of property on SIDData.
Adding a copy of the Event's dt field as datetime via the ``alias_dt``
generator, so that the API was forgiving and allowed both datetime and
dt on a SIDData object, was creating noticeable overhead, even on an
noop algorithms.
Instead of incurring the cost of copying the datetime value and
assigning it to the Event object on every event that is passed through
the system, add a property to SIDData which acts as an alias
``datetime`` to ``dt``.
Eventually support for ``data['foo'].datetime`` may be removed, and
could be considered deprecated.
* Remove the drop of 'null return' from cumulative returns.
The check of existence of the null return key, and the drop of said
return on every single bar was adding unneeded CPU time when an
algorithm was run with minute emissions.
Instead, add the 0.0 return with an index of the trading day before the
start date.
The removal of the ``null return`` was mainly in place so that the
period calculation was not crashing on a non-date index value; with the
index as a date, the period return can also approximate volatility (even
though the that volatility has high noise-to-signal strength because it
uses only two values as an input.)
Maintenance and Refactorings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Allow ``sim_params`` to provide data frequency for the algorithm.
In the case that ``data_frequency`` of the algorithm is None, allow the
``sim_params`` to provide the ``data_frequency``.
Also, defer to the algorithms data frequency, if provided.
Build
~~~~~
* Added support for building and releasing via conda
For those who prefer building with http://conda.pydata.org/ to compiling
locally with pip.
The following should install Zipline on many systems.
::
conda install -c quantopian zipline
Contributors
~~~~~~~~~~~~
The following people have contributed to this release, ordered by
numbers of commit:
::
49 Eddie Hebert
28 Thomas Wiecki
11 Richard Frank
2 Jamie Kirkpatrick
2 Jeremiah Lowin
1 Colin Alexander
1 Michael Schatzow
1 Moises Trovo
1 Suminda Dharmasena
docs/source/whatsnew/0.7.0.txt
+213
View File
@@ -0,0 +1,213 @@
Release 0.7.0
-------------
:Release: 0.7.0
:Date: July 25, 2014
Highlights
~~~~~~~~~~
* Command line interface to run algorithms directly.
* IPython Magic ``%%zipline`` that runs algorithm defined in an IPython
notebook cell.
* API methods for building safeguards against runaway ordering and
undesired short positions.
* New history() function to get a moving DataFrame of past market data
(replaces BatchTransform).
* A new `beginner
tutorial <http://nbviewer.ipython.org/github/quantopian/zipline/blob/master/docs/tutorial.ipynb>`__.
Enhancements
~~~~~~~~~~~~
* CLI: Adds a CLI and IPython magic for zipline.
Example:
::
python run_algo.py -f dual_moving_avg.py --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o dma.pickle
Grabs the data from yahoo finance, runs the file
dual\_moving\_avg.py (and looks for ``dual_moving_avg_analyze.py``
which, if found, will be executed after the algorithm has been run),
and outputs the perf ``DataFrame`` to ``dma.pickle`` (:issue:`325`).
- IPython magic command (at the top of an IPython notebook cell).
Example:
::
%%zipline --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o perf
Does the same as above except instead of executing the file looks
for the algorithm in the cell and instead of outputting the perf df
to a file, creates a variable in the namespace called perf (:issue:`325`).
* Adds Trading Controls to the algorithm API.
The following functions are now available on ``TradingAlgorithm``
and for algo scripts:
``set_max_order_size(self, sid=None, max_shares=None, max_notional=None)``
Set a limit on the absolute magnitude, in shares and/or total
dollar value, of any single order placed by this algorithm for a
given sid. If ``sid`` is None, then the rule is applied to any order
placed by the algorithm.
Example:
.. code-block:: python
def initialize(context):
# Algorithm will raise an exception if we attempt to place an
# order which would cause us to hold more than 10 shares
# or 1000 dollars worth of sid(24).
set_max_order_size(sid(24), max_shares=10, max_notional=1000.0)
``set_max_position_size(self, sid=None, max_shares=None, max_notional=None)``
-Set a limit on the absolute magnitude, in either shares or
dollar value, of any position held by the algorithm for a given
sid. If ``sid`` is None, then the rule is applied to any position
held by the algorithm.
Example:
.. code-block:: python
def initialize(context):
# Algorithm will raise an exception if we attempt to order more than
# 10 shares or 1000 dollars worth of sid(24) in a single order.
set_max_order_size(sid(24), max_shares=10, max_notional=1000.0)
``set_max_order_count(self, max_count)``
Set a limit on the number of orders that can be placed by the algorithm in
a single trading day.
Example:
.. code-block:: python
def initialize(context):
# Algorithm will raise an exception if more than 50 orders are placed in a day.
set_max_order_count(50)
``set_long_only(self)``
Set a rule specifying that the
algorithm may not hold short positions.
Example:
.. code-block:: python
def initialize(context):
# Algorithm will raise an exception if it attempts to place
# an order that would cause it to hold a short position.
set_long_only()
(:issue:`329`).
* Adds an ``all_api_methods`` classmethod on ``TradingAlgorithm`` that
returns a list of all ``TradingAlgorithm`` API methods (:issue:`333`).
* Expanded record() functionality for dynamic naming.
The record() function can now take positional args before the
kwargs. All original usage and functionality is the same, but now
these extra usages will work:
.. code-block:: python
name = 'Dynamically_Generated_String'
record( name, value, ... )
record( name, value1, 'name2', value2, name3=value3, name4=value4 )
The requirements are simply that the poritional args occur only
before the kwargs (:issue:`355`).
* history() has been ported from Quantopian to Zipline and provides
moving window of market data.
history() replaces BatchTransform. It is faster, works for minute level data
and has a superior interface. To use it, call ``add_history()`` inside of
``initialize()`` and then receive a pandas ``DataFrame`` by calling history()
from inside ``handle_data()``. Check out the `tutorial
<http://nbviewer.ipython.org/github/quantopian/zipline/blob/master/docs/tutorial.ipynb>`__
and an `example
<https://github.com/quantopian/zipline/blob/master/zipline/examples/dual_moving_average.py>`__.
(:issue:`345` and :issue:`357`).
* history() now supports ``1m`` window lengths (:issue:`345`).
Bug Fixes
~~~~~~~~~
* Fix alignment of trading days and open and closes in trading
environment (:issue:`331`).
* RollingPanel fix when adding/dropping new fields (:issue:`349`).
Performance
~~~~~~~~~~~
Maintenance and Refactorings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Removed undocumented and untested HDF5 and CSV data sources (:issue:`267`).
* Refactor sim\_params (:issue:`352`).
* Refactoring of history (:issue:`340`).
Build
~~~~~
* The following dependencies have been updated (zipline might work with
other versions too):
.. code-block:: diff
-pytz==2013.9
+pytz==2014.4
+numpy==1.8.1
-numpy==1.8.0
+scipy==0.12.0
+patsy==0.2.1
+statsmodels==0.5.0
-six==1.5.2
+six==1.6.1
-Cython==0.20
+Cython==0.20.1
-TA-Lib==0.4.8
+--allow-external TA-Lib --allow-unverified TA-Lib TA-Lib==0.4.8
-requests==2.2.0
+requests==2.3.0
-nose==1.3.0
+nose==1.3.3
-xlrd==0.9.2
+xlrd==0.9.3
-pep8==1.4.6
+pep8==1.5.7
-pyflakes==0.7.3
-pip-tools==0.3.4
+pyflakes==0.8.1`
-scipy==0.13.2
-tornado==3.2
-pyparsing==2.0.1
-patsy==0.2.1
-statsmodels==0.4.3
+tornado==3.2.1
+pyparsing==2.0.2
-Markdown==2.3.1
+Markdown==2.4.1
Contributors
~~~~~~~~~~~~
The following people have contributed to this release, ordered by
numbers of commit:
::
38 Scott Sanderson
29 Thomas Wiecki
26 Eddie Hebert
6 Delaney Granizo-Mackenzie
3 David Edwards
3 Richard Frank
2 Jonathan Kamens
1 Pankaj Garg
1 Tony Lambiris
1 fawce
docs/source/whatsnew/0.8.0.txt
+122
View File
@@ -0,0 +1,122 @@
Release 0.8.0
-------------
:Release: 0.8.0
:Date: TBD
Highlights
~~~~~~~~~~
* New documentation system with a new website at
`zipline.io <http://www.zipline.io>`__
* Major performance enhancements.
* Dynamic history.
Enhancements
~~~~~~~~~~~~
* Account object: Adds an account object to conext to track information about
the trading account.
Example:
.. code-block:: python
context.account.settled_cash
Returns the settled cash value that is stored on the account object.
This value is updated accordingly as the algorithm is run (:issue:`396`).
* :class:`~zipline.history.history_container.HistoryContainer` can now grow
dynamically. Calls to :func:`~zipline.api.history` will now be able to increase
the size or change the shape of the history container to be able to service the
call. :func:`~zipline.api.add_history` now acts as a preformance hint to
pre-allocate sufficient space in the container. This change is backwards
compatible with ``history``, all existing algorithms should continue to work as
intended (:issue:`412`).
* Simple transforms ported from quantopian and use history.
:class:`~zipline.protocol.SIDData` now has methods for:
- ``stddev``
- ``mavg``
- ``vwap``
- ``returns``
These methods, except for ``returns``, accept a number of days. If
you are running with minute data, then this will calculate the
number of minutes in those days, accounting for early closes and the
current time and apply the transform over the set of minutes.
``returns`` takes no parameters and will return the daily returns of
the given asset.
Example:
.. code:: python
data[security].stddev(3)
(:issue:`429`).
* New fields in Performance Period.
Performance Period has new fields accessible in return value of
``to_dict``:
- gross leverage
- net leverage
- short exposure
- long exposure
- shorts count
- longs count
(:issue:`464`).
* Allow :func:`~zipline.api.order_percent` to work with various market values
(by Jeremiah Lowin).
Currently, :func:`~zipline.api.order_percent` and
:func:`~zipline.api.order_target_percent` both operate as a percentage of
``self.portfolio.portfolio_value``. This PR lets them operate as percentages
of other important MVs.
Also adds ``context.get_market_value()``, which enables this
functionality.
For example:
.. code-block:: python
# this is how it works today (and this still works)
# put 50% of my portfolio in AAPL
order_percent('AAPL', 0.5)
# note that if this were a fully invested portfolio, it would become 150% levered.
# take half of my available cash and buy AAPL
order_percent('AAPL', 0.5, percent_of='cash')
# rebalance my short position, as a percentage of my current short
book_target_percent('MSFT', 0.1, percent_of='shorts')
# rebalance within a custom group of stocks
tech_stocks = ('AAPL', 'MSFT', 'GOOGL')
tech_filter = lambda p: p.sid in tech_stocks
for stock in tech_stocks:
order_target_percent(stock, 1/3, percent_of_fn=tech_filter)
(:issue:`477`).
* Command line option to for printing algo to stdout (by Andrea D'Amore)
(:issue:`545`).
Bug Fixes
~~~~~~~~~
* Fix a bug where the reported returns could sharply dip for random periods of
time (:issue:`378`).
Performance
~~~~~~~~~~~
* Major performance enhancements to history (by Dale Jung) (:issue:`488`).
Maintenance and Refactorings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Build
~~~~~
Documentation
~~~~~~~~~~~~~
* Switched to sphinx for the documentation (:issue:`816`).
Contributors
~~~~~~~~~~~~