Fixes an issue that caused a crash if a user assigned a new column into a
returned history DataFrame. This occurred because we re-use DataFrames between
history() calls, so the new column caused an index-size mismatch on the next
attempted calculation.
Ideally we would fix this by in-place dropping the columns, but that isn't
supported in pandas 0.12.0, so instead we just return a complete copy of the
frame. We should re-evaluate this implementation when we're on a more modern
pandas version.
There were sevaral places you could supply sim_params
in TradingAlgorithm (__init__, run). This got confusing
as its not clear who updated what and which one was the
correct one to use at each time.
Then there were to ways to define data_frequency, one in
__init__() and one in the sim_params which also added code
complexity.
This refactor makes it explicit that sim_params are to be
passed to __init__() only. Moreover, data_frequency is
only stored in sim_params. For backwards compatibility,
it can still be supplied separately but will link to
the one in sim_params.
For example, you could create new sim params via:
sim_params = create_simulation_parameters(data_frequency='minute')
algo = MyAlgo(sim_params)
algo.run(data)
In addition, perf_tracker only gets initialized in one place:
_create_generator() which should also make the various ways
of running an algorithm more deterministic.
This also fixes a bug with SimulationParameters where
you could not change the period_start. Unfortunately, the
current implementation still requieres an implicit call to
update the internal variables.
Replace usage of .ix in TradingEnvironment with .loc when we know that we're
using an index key.
DataFrame.ix can be used with either integer or key-based indices, and as such
it incurs an overhead for figuring out which you meant.
The bug occurred because there is a special case in the initial window setup
code for handling the case where only a length-1 history is used for a given
frequency. Previously, the code was incorrectly calculating the period end
using a hard-coded expression for the end of the day (the correct behavior for
a length-1 '1d' history), and then using the frequency object to calculate the
period start for the window. In the case of length 1 '1m' data, this resulted
in an initial window whose start and end was the last minute of the day rather
than the first minute of the day. For non-price fields, this error doesn't
matter, because the window is only used for rolling digests (which doesn't
happen when there's only a length-1 history), and for the forward-filling logic
(which only happens on price fields). For a length-1 '1m' price, however, the
incorrect window causes us to attempt to forward-fill an empty panel, resulting
in an IndexError when we do an iloc[0] on a length-0 axis.
There quite some bugs in certain corner cases. Dropping of obsolete
axes was not working correctly, roll over could cause obsolete axes
to not drop. The tests are much more stringent now as well.
Adds a new argument, `freq_filter`, to `HistoryContainer.update_digest_panels`,
which can be used to supply a predicate determining which digest panels are
updated by the call. This is useful for supporting initialization/backfilling
of multi-frequency panels in subclasses.
Truncate non-integer order amounts in `TradingAlgorithm.order` instead of
`Blotter.order`. This fixes an issue where non-integer orders coming out of
order_value can spuriously trigger a `LongOnly` trading guard.
Example:
sid.price == 2.0
order_value(sid, 5) -> order(sid, 2.5) -> truncated to order(sid, 2.0)
order_value(sid, -5) -> order(sid, -2.5) -> LongOnlyViolation b/c 2.0 - 2.5 < 0
The IPython magic still created an output file because
the output argument was only removed after the pipeline
was run. This fix simply removes the argument before
the call to run_pipline() when running the IPython magic.
Updates `HistoryContainer.roll` to handle cases where no data is present for
the period being rolled.
We now only forward-fill the `price` field when `ffill` is specified.
Overhauls `HistoryContainer` in prep for support of more than one frequency.
Major changes:
- Methods/variables referring to "day" have been renamed/generalized.
- `current_day_panel` became `buffer_panel`, which is now a `RollingPanel`
- `prior_day_panel` became a dictionary mapping `Frequency` objects to
"digest panels", which are instances of `RollingPanel`.
- Hard-coded daily rollover replaced with a notion of a "current window" for
each unique frequency managed by the panel.
- When the end of the current window is reached for a given frequency, we
compute an aggregate bar (code refers to this as a "digest"), which is
appended to a panel associated with that frequency.
- Window rollover dates are managed by a pair of dictionaries,
`cur_window_starts` and `cur_window_closes`. The `Frequency` class is
responsible for computing window bounds based on the open/close of the
previous window.
- Semantic change to the `open_price` field: `open_price` now always
contains the price of the first trade occurring in the given window.
Previously it contained the price of the first minute in the window,
returning NaN it the security happened not to trade in the first minute.
Adds a suite of new functions for querying data from the trading calendar.
These include:
`previous_trading_day`
`minutes_for_days_in_range` (minutely version of `days_in_range`)
`previous_open_and_close` (inverse of `next_open_and_close`)
`next_market_minute`
`previous_market_minute`
`open_close_window` (get a range of opens/closes with slicing semantics)
`market_minute_window` (get a range of minutes with slicing semantics)
Also refactors `test_finance` to move `TradingEnvironment` tests into their own
TestCase.
Adds a classmethod, `instance` on `TradingEnvironment` that returns
`zipline.finance.trading.environment`, instantiating it if necessary.
This makes it possible to initialize the default environment instance in a
less-roundabout way than creating a `SimulationParameters` object.
Refactored the target order methods to separate their logic from the
other order methods.
This makes order_target the only target method that calls order()
directly.
order_target_value is passed to order_target instead of order_value.
order_target_percent is passed to order_target_value rather than
order_value.
This simplified the code and decouples the logic of target orders from
the other order methods. This allows the target order methods to be
developed independently from order_value and order_percent.
Many algorithms that use the new order methods like order_target()
will legitimately try to order 0 shares many times. The printed
warning at every turn is quite annoying and too verbose. We do not
display it on Quantopian either so I'm removing it here as well.
Filter out empty lists from `get_open_orders` so that we have consistent
behavior between the case where a user has never placed an order and the case
where the user has placed an order but it has been executed or cancelled.
A nice side-effect, which was the impetus for this change, is that you can
check if you have any open orders by doing:
```
len(get_open_orders()) == 0
```
Also adds a test for the behavior of `get_open_orders`, which was previously
lacking.
Make the portfolio property on TradingAlgorithm call `updated_portfolio`
internally. This prevents needless recomputation of the portfolio between
calls to `handle_data`, and also prevents issues where the portfolio object
could be unexpectedly modified in place in the body of a `handle_data` call.
Noteworthy finding in the course of investigating this bug:
If you modify a Python dictionary while iterating over it, the language will
only throw an exception if the size of the dictionary changes between loop
iterations; this means that you can do:
```
x = {1:1, 2:2, 3:3}
for k in x:
old_val = x[k]
del x[k]
x[f(k)] = old_val
print k
```
and you'll only get an error if f(k) is already a key in the dictionary.
This can lead to bizarre/nondeterministic behavior in the key iterator.
Make the error messages for {DoBadThing}PostInit no longer reference "the
simulation", since the algorithm may not actually be running as a simulation.
Adds four new methods to the Zipline API that can be used as circuit-breakers
to interrupt the execution of an algorithm. The API methods are:
`set_max_position_size`
`set_max_order_size`
`set_max_order_count`
`set_long_only`
Internally, these methods are implemented by each registering a TradingControl
callback object with the TradingAlgorithm. During
TradingAlgorithm.__validate_order_params (and thus before any side-effects of
the order call occur), each callback's `validate` method is called with
information about the order to be placed and the algorithm's current state,
raising an exception if the callback detects that an error condition has been breached.
TradingEnvironment class uses env_trading_calendar for trading days,
but the default trading calendar for open_and_close data, which causes
errors later, because of misalignment of trading days.
The issue can be resolved by using env_trading_calendar for
open_and_closes as well
When zipline is imported it checks whether
it runs in the IPython notebook. If it does,
it registers a %%zipline magic that takes the
same arguments as the CLI with the addition of
a -o for specifying the output variable to store
the performance frame in.
The algo code in the cell is, as of yet, executed
in its own environment rather than that of the
IPython NB which is probably what we want.
Also adds cli option to save the perf dataframe
to a pickle file.
Also adds an IPython notebook buyapple example.
Add a CLI that reads in an algorithm, loads data,
run the algorithm, and output performance metrics.
The examples are adapted to the new zipline API and
analyses are split into separate files.
Also add config files that run the example
algorithms with preset settings.
Adds the exchange property the interface for ExecutionStyle and adds an
exchange parameter to the interface of all the existing ExceutionStyles.
Subclasses wishing to support the ability to specify an exchange should set the
_exchange attribute in __init__.
Stop and limit prices both trigger when a price crosses some threshold, but
they trigger in "opposite directions". For example, on a buy, a limit price is
triggered when a price falls below a specified value, whereas a stop price
triggers when the price exceeds a specified value.
Our current stop/limit price rounding logic is asymmetric, preferring to "round
to improve" the specified price. This change makes it so that we interpret
"improvement" in opposite directions for stop vs limit prices.
recent 2 for 1 stock split, where 1 class C share was distributed
for each share of class A held.
Now a dividend can specify a sid and ratio of stock that will be paid
to owners of the original security. If the ratio is 2.0, then for every
existing share, two shares will be paid.
Add a test case in test_algorithms to verify that appropriate exceptions are
thrown if an algorithm makes a call to the order api with a stop/limit price
and a style.
Add `style` parameter to order_value, order_percent, order_target,
order_target_percent, and order_target_value methods. The style parameter is
forwarded to the underlying call to `order`.
existing `limit_price` and `stop_price` parameters. The goal of this change is
to refactor the existing ordering API to provide a cleaner interface for
defining more complex order types.
Adds a new module, zipline.finance.execution, which defines the ExecutionStyle
abstract base class, along with concrete MarketOrder, LimitOrder, StopOrder,
and StopLimitOrder subclasses.
Adds a new `style` keyword argument to the function signature of the `order`
API method, which accepts an instance of ExecutionStyle.
The existing limit_price and stop_price parameters are still supported at this
time, but are converted into the new ExecutionStyle objects before being passed
to Blotter.order.
Fixes an issue where very low limit prices were being rounded to 0.0 and
effectively resulting in market orders. Adds an explicit check to test for
this behavior.
Adds a test algorithm that tries to buy with very high limit prices/very low
stop prices and tries to sell with very low limit prices/very high stop prices.
TradingAlgorithm always uses set_algo_instance in pairs of
set_algo_instance(self) and set_algo_instance(None). Refactoring this to use a
context manager.
Thomas Wiecki