Adds the data bundle concept which makes it easy for users to register
loading functions to build out minute and daily data along with an
assets db and adjustments db. By default we have provided a `quandl`
bundle which pulls from the public domain WIKI dataset. Users may
register new bundles by decorating an ingest function with
`zipline.data.bundles.register(<name>)`. This also provides a
`yahoo_equities` function for creating an ingestion function that will
load a static set of assets from yahoo.
The cli is now structured as a couple of subcommands and has been
changed to `python -m zipline`. The old behavior of `run_algo.py` has
been moved to the `run` subcommand. This is almost entirely the same
except that it now takes the name of the data bundle to use, defaulting
to `quandl`.
The next subcommand is `ingest` which takes the name of
a data bundle to ingest. This will run the loading machinery and write
the data to a specified location that `run` can find.
There is also a `clean` subcommand which deletes the data that was
written with `ingest`.
Extensions have also been added to zipline. This is an experimental
feature where users can provide an extra set of python files to run at
the start of the process. These can be used to configure aspects of
zipline. Right now the only thing that is supported in an extension file
is the registration of a new data bundle.
Changes BcolzDailyBarWriter to not be an abc, data is passed as an
iterator of (sid, dataframe) pairs to the write method.
Changes the AssetsDBWriter to be a single class which accepts an engine
at construction time and has a `write` method for writing dataframes for
the various tables. We no longer support writing the various other data
types, callers should coerce their data into a dataframe themselves. See
zipline.assets.synthetic for some helpers to do this.
Adds many new fixtures and updates some existing fixtures to use the new
ones:
WithDefaultDateBounds
A fixture that provides the suite a START_DATE and END_DATE. This is
meant to make it easy for other fixtures to synchronize their date
ranges without depending on eachother in strange ways. For example,
WithBcolzMinuteBarReader and WithBcolzDailyBarReader by default should
both have data for the same dates, so they may use depend on
WithDefaultDates without forcing a dependency between them.
WithTmpDir, WithInstanceTmpDir
Provides the suite or individual test case a temporary directory.
WithBcolzDailyBarReader
Provides the suite a BcolzDailyBarReader which reads from bcolz data
written to a temporary directory. The data will be read from
dataframes and then converted to bcolz files with
BcolzDailyBarWriter.write
WithBcolzDailyBarReaderFromCSVs
Provides the suite a BcolzDailyBarReader which reads from bcolz data
written to a temporary directory. The data will be read from a
collection of CSV files and then converted into the bcolz data through
BcolzDailyBarWriter.write_csvs
WithBcolzMinuteBarReader
Provides the suite a BcolzMinuteBarReader which reads from bcolz data
written to a temporary directory. The data will be read from
dataframes and then converted to bcolz files with
BcolzMinuteBarWriter.write
WithAdjustmentReader
Provides the suite a SQLiteAdjustmentReader which reads from an in
memory sqlite database. The data will be read from dataframes and then
converted into sqlite with SQLiteAdjustmentWriter.write
WithDataPortal
Provides each test case a DataPortal object with data from temporary
resources.
script.
Adds an option kwarg to TradingAlgorithm named 'algo_filename' that
represents the file where the algoscript came from (if any). The
run_algo.py script will pass this argument with the value passed to the
'-f' flag. The default name is '<string>' to represent that the script
is coming from a string in python and not a file. This matches the
behavior of exec and the python convention for compiling code objects.
There were sevaral places you could supply sim_params
in TradingAlgorithm (__init__, run). This got confusing
as its not clear who updated what and which one was the
correct one to use at each time.
Then there were to ways to define data_frequency, one in
__init__() and one in the sim_params which also added code
complexity.
This refactor makes it explicit that sim_params are to be
passed to __init__() only. Moreover, data_frequency is
only stored in sim_params. For backwards compatibility,
it can still be supplied separately but will link to
the one in sim_params.
For example, you could create new sim params via:
sim_params = create_simulation_parameters(data_frequency='minute')
algo = MyAlgo(sim_params)
algo.run(data)
In addition, perf_tracker only gets initialized in one place:
_create_generator() which should also make the various ways
of running an algorithm more deterministic.
This also fixes a bug with SimulationParameters where
you could not change the period_start. Unfortunately, the
current implementation still requieres an implicit call to
update the internal variables.
The IPython magic still created an output file because
the output argument was only removed after the pipeline
was run. This fix simply removes the argument before
the call to run_pipline() when running the IPython magic.
When zipline is imported it checks whether
it runs in the IPython notebook. If it does,
it registers a %%zipline magic that takes the
same arguments as the CLI with the addition of
a -o for specifying the output variable to store
the performance frame in.
The algo code in the cell is, as of yet, executed
in its own environment rather than that of the
IPython NB which is probably what we want.
Also adds cli option to save the perf dataframe
to a pickle file.
Also adds an IPython notebook buyapple example.
Add a CLI that reads in an algorithm, loads data,
run the algorithm, and output performance metrics.
The examples are adapted to the new zipline API and
analyses are split into separate files.
Also add config files that run the example
algorithms with preset settings.
Joe Jevnik