Removes support for handling dividends as part of the algorithm
simulation stream, replacing it with an API in `TradingAlgorithm` for
supplying dividends as a DataFrame.
Previously, calling order() in initalize resulted in a weird
stack trace. It now returns a well formulated error that is
readable to the user through the API. Adding a slippage
kwarg to test_algorithm and simfactor was necessary because
slippage can only be called during init. Previously initaliazed
was never set to true and calls to init-only function were sprinkled
around the code in non-init sections. Code changes were to enforce
init-only rules.
There were sevaral places you could supply sim_params
in TradingAlgorithm (__init__, run). This got confusing
as its not clear who updated what and which one was the
correct one to use at each time.
Then there were to ways to define data_frequency, one in
__init__() and one in the sim_params which also added code
complexity.
This refactor makes it explicit that sim_params are to be
passed to __init__() only. Moreover, data_frequency is
only stored in sim_params. For backwards compatibility,
it can still be supplied separately but will link to
the one in sim_params.
For example, you could create new sim params via:
sim_params = create_simulation_parameters(data_frequency='minute')
algo = MyAlgo(sim_params)
algo.run(data)
In addition, perf_tracker only gets initialized in one place:
_create_generator() which should also make the various ways
of running an algorithm more deterministic.
This also fixes a bug with SimulationParameters where
you could not change the period_start. Unfortunately, the
current implementation still requieres an implicit call to
update the internal variables.
There quite some bugs in certain corner cases. Dropping of obsolete
axes was not working correctly, roll over could cause obsolete axes
to not drop. The tests are much more stringent now as well.
The IPython magic still created an output file because
the output argument was only removed after the pipeline
was run. This fix simply removes the argument before
the call to run_pipline() when running the IPython magic.
Overhauls `HistoryContainer` in prep for support of more than one frequency.
Major changes:
- Methods/variables referring to "day" have been renamed/generalized.
- `current_day_panel` became `buffer_panel`, which is now a `RollingPanel`
- `prior_day_panel` became a dictionary mapping `Frequency` objects to
"digest panels", which are instances of `RollingPanel`.
- Hard-coded daily rollover replaced with a notion of a "current window" for
each unique frequency managed by the panel.
- When the end of the current window is reached for a given frequency, we
compute an aggregate bar (code refers to this as a "digest"), which is
appended to a panel associated with that frequency.
- Window rollover dates are managed by a pair of dictionaries,
`cur_window_starts` and `cur_window_closes`. The `Frequency` class is
responsible for computing window bounds based on the open/close of the
previous window.
- Semantic change to the `open_price` field: `open_price` now always
contains the price of the first trade occurring in the given window.
Previously it contained the price of the first minute in the window,
returning NaN it the security happened not to trade in the first minute.
Adds four new methods to the Zipline API that can be used as circuit-breakers
to interrupt the execution of an algorithm. The API methods are:
`set_max_position_size`
`set_max_order_size`
`set_max_order_count`
`set_long_only`
Internally, these methods are implemented by each registering a TradingControl
callback object with the TradingAlgorithm. During
TradingAlgorithm.__validate_order_params (and thus before any side-effects of
the order call occur), each callback's `validate` method is called with
information about the order to be placed and the algorithm's current state,
raising an exception if the callback detects that an error condition has been breached.
When zipline is imported it checks whether
it runs in the IPython notebook. If it does,
it registers a %%zipline magic that takes the
same arguments as the CLI with the addition of
a -o for specifying the output variable to store
the performance frame in.
The algo code in the cell is, as of yet, executed
in its own environment rather than that of the
IPython NB which is probably what we want.
Also adds cli option to save the perf dataframe
to a pickle file.
Also adds an IPython notebook buyapple example.
Add a CLI that reads in an algorithm, loads data,
run the algorithm, and output performance metrics.
The examples are adapted to the new zipline API and
analyses are split into separate files.
Also add config files that run the example
algorithms with preset settings.
recent 2 for 1 stock split, where 1 class C share was distributed
for each share of class A held.
Now a dividend can specify a sid and ratio of stock that will be paid
to owners of the original security. If the ratio is 2.0, then for every
existing share, two shares will be paid.
TradingAlgorithm always uses set_algo_instance in pairs of
set_algo_instance(self) and set_algo_instance(None). Refactoring this to use a
context manager.
This creates a data source for csv and hdf5 files, a generator to create a sample csv, and a pytables generator to go from a list of dated gzipped csv's in a directory to a pytables data source.
This does not add a unittest yet which we should write for the future.
This is a step towards the goal of uniting Quantopian scripts
and zipline.
To make the syntax of zipline identical to Quantopian
we break out the API methods (like order) and turn them into
functions. To access the algo object we add a thread local reference
to the current algorithm that is accessed in the API functions.
TradingAlgorithm now takes either a string or two functions
(initialize and handle_data) that it executes.
Use api method decorator for methods available in algoscript.
Ported appropriate algorithm tests from internal code.
Use date sorted sources instead, instead of sorting with second
argument of Event, etc. since the `heapq.merge` behavior is using
the second part of the tuple, thus requiring a richer set of comparison
methods, which would only be used in the test context.
Use `date_sorted_sources` instead, so that sorting is done on algo time
and source id.
Python 3 removes the `.message` attribute, so use `str` instead.
Also, the divide by zero message has changed slightly between versions,
so just check for the exception type, instead of also checking the message.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
Note that the calendar test is decorated with @nottest (as per the other calendar test functions). I've run the test to confirm the calendar works. The differences between the env (Yahoo Finance of GSPTSE) and the calendar are illustrated in the tradingcalendar_tse file and are confirmed to be errors on Yahoo Finance's part.
The code that was consuming noop_environment now uses a
real trading environment.
As more behavior relies on an accurate trading calendar, maintaining
the noop environment was a constraint that was more overhead than it
is worth.
Passing the exchange time timestamp to is_market_hours was ending
up with odd behavior due to conversion back to UTC when checking
the is_trading_day boolean.
Use the early closes to populate a DataFrame which includes
the open and close minute for each day.
To be used by the environment instead of calculating each value
mid-backtest.
Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
Instead of using all calendar days between start and end in test
sources, use the trading calendar for test sources.
Needed for an incoming refactoring of market open and close,
where the opens and closes are indexed by market days.
Most of the functions in date_utils can be done via pandas.
The other functions are no longer used for loading, etc. so remove
the date_utils module to reduce the total surface area of Zipline core.
Scott Sanderson