Previously the last sale price was not correctly being set on
positions when the transaction arrived before the trade event.
The last sale price was defaulted to zero and never updated. This resulted
in one holding stocks that were bough >>0 and now had value 0 from
the perspective of returns. The returns would display correctly again
when the next trade of that security happened. For most securities trading is
frequent enough that there's no issue, but for some illiquid ones it took
hours to fix itself.
Updated test_perf_tracking:TestPerformanceTracker.test_minute_tracker
This test was based on assuming that last_sale_price was zero,
allowing the sharpe ratio to be calculated. The sharpe ratio can no longer
be calculated for this specific tested scenario and the test has been changed
accordingly.
The initialization of perf_tracker had been moved from __init__
in TradingAlgorithm to _create_generator. This caused perf_tracker
to not be ready when portfolio requested it. portfolio was consequently
not ready for access in init. portfolio can now be accessed in init
again, assuming valid sim_params are passed. Otherwise it will be
available in handle_data() after _create_generator() is called.
Removes support for handling dividends as part of the algorithm
simulation stream, replacing it with an API in `TradingAlgorithm` for
supplying dividends as a DataFrame.
The function that handles a market close for daily frequency changed from
`handle_market_close` to `handle_market_close_daily`.
The function that is called at on the closing minute each day when running
minutely changed from `handle_intraday_close` to
`handle_intraday_market_close`.
Breaks out the main snapshot processing loop into its own function, and does
some minor variable renaming-shuffling.
Adds `TradingAlgorithm.on_dt_changed`, a function to be called when the
simulation dt changes, prior to processing any events.
There should be no difference in behavior as a result of this change.
We can't be sure that security identifiers can be converted
to int. In fact, most of the times they are strings. This
adds an identity function that can be overwritten if such
a conversion is necessary (as on Quantopian).
Created a new flag in TradingAlgorithm that enables subclasses to
decide if they want to handle setting self.initialized = True.
Before it was the responsibility of an overriding subclass to set
initalized = True. This was causing problems because it's easy to
forget this. Now it is the responsibility of TradingAlgorithm
unless explicity stated otherwise.
Previously, calling order() in initalize resulted in a weird
stack trace. It now returns a well formulated error that is
readable to the user through the API. Adding a slippage
kwarg to test_algorithm and simfactor was necessary because
slippage can only be called during init. Previously initaliazed
was never set to true and calls to init-only function were sprinkled
around the code in non-init sections. Code changes were to enforce
init-only rules.
Fixes an issue that caused a crash if a user assigned a new column into a
returned history DataFrame. This occurred because we re-use DataFrames between
history() calls, so the new column caused an index-size mismatch on the next
attempted calculation.
Ideally we would fix this by in-place dropping the columns, but that isn't
supported in pandas 0.12.0, so instead we just return a complete copy of the
frame. We should re-evaluate this implementation when we're on a more modern
pandas version.
There were sevaral places you could supply sim_params
in TradingAlgorithm (__init__, run). This got confusing
as its not clear who updated what and which one was the
correct one to use at each time.
Then there were to ways to define data_frequency, one in
__init__() and one in the sim_params which also added code
complexity.
This refactor makes it explicit that sim_params are to be
passed to __init__() only. Moreover, data_frequency is
only stored in sim_params. For backwards compatibility,
it can still be supplied separately but will link to
the one in sim_params.
For example, you could create new sim params via:
sim_params = create_simulation_parameters(data_frequency='minute')
algo = MyAlgo(sim_params)
algo.run(data)
In addition, perf_tracker only gets initialized in one place:
_create_generator() which should also make the various ways
of running an algorithm more deterministic.
This also fixes a bug with SimulationParameters where
you could not change the period_start. Unfortunately, the
current implementation still requieres an implicit call to
update the internal variables.
Replace usage of .ix in TradingEnvironment with .loc when we know that we're
using an index key.
DataFrame.ix can be used with either integer or key-based indices, and as such
it incurs an overhead for figuring out which you meant.
The bug occurred because there is a special case in the initial window setup
code for handling the case where only a length-1 history is used for a given
frequency. Previously, the code was incorrectly calculating the period end
using a hard-coded expression for the end of the day (the correct behavior for
a length-1 '1d' history), and then using the frequency object to calculate the
period start for the window. In the case of length 1 '1m' data, this resulted
in an initial window whose start and end was the last minute of the day rather
than the first minute of the day. For non-price fields, this error doesn't
matter, because the window is only used for rolling digests (which doesn't
happen when there's only a length-1 history), and for the forward-filling logic
(which only happens on price fields). For a length-1 '1m' price, however, the
incorrect window causes us to attempt to forward-fill an empty panel, resulting
in an IndexError when we do an iloc[0] on a length-0 axis.
There quite some bugs in certain corner cases. Dropping of obsolete
axes was not working correctly, roll over could cause obsolete axes
to not drop. The tests are much more stringent now as well.
Adds a new argument, `freq_filter`, to `HistoryContainer.update_digest_panels`,
which can be used to supply a predicate determining which digest panels are
updated by the call. This is useful for supporting initialization/backfilling
of multi-frequency panels in subclasses.
Truncate non-integer order amounts in `TradingAlgorithm.order` instead of
`Blotter.order`. This fixes an issue where non-integer orders coming out of
order_value can spuriously trigger a `LongOnly` trading guard.
Example:
sid.price == 2.0
order_value(sid, 5) -> order(sid, 2.5) -> truncated to order(sid, 2.0)
order_value(sid, -5) -> order(sid, -2.5) -> LongOnlyViolation b/c 2.0 - 2.5 < 0
The IPython magic still created an output file because
the output argument was only removed after the pipeline
was run. This fix simply removes the argument before
the call to run_pipline() when running the IPython magic.
Updates `HistoryContainer.roll` to handle cases where no data is present for
the period being rolled.
We now only forward-fill the `price` field when `ffill` is specified.
Overhauls `HistoryContainer` in prep for support of more than one frequency.
Major changes:
- Methods/variables referring to "day" have been renamed/generalized.
- `current_day_panel` became `buffer_panel`, which is now a `RollingPanel`
- `prior_day_panel` became a dictionary mapping `Frequency` objects to
"digest panels", which are instances of `RollingPanel`.
- Hard-coded daily rollover replaced with a notion of a "current window" for
each unique frequency managed by the panel.
- When the end of the current window is reached for a given frequency, we
compute an aggregate bar (code refers to this as a "digest"), which is
appended to a panel associated with that frequency.
- Window rollover dates are managed by a pair of dictionaries,
`cur_window_starts` and `cur_window_closes`. The `Frequency` class is
responsible for computing window bounds based on the open/close of the
previous window.
- Semantic change to the `open_price` field: `open_price` now always
contains the price of the first trade occurring in the given window.
Previously it contained the price of the first minute in the window,
returning NaN it the security happened not to trade in the first minute.
Adds a suite of new functions for querying data from the trading calendar.
These include:
`previous_trading_day`
`minutes_for_days_in_range` (minutely version of `days_in_range`)
`previous_open_and_close` (inverse of `next_open_and_close`)
`next_market_minute`
`previous_market_minute`
`open_close_window` (get a range of opens/closes with slicing semantics)
`market_minute_window` (get a range of minutes with slicing semantics)
Also refactors `test_finance` to move `TradingEnvironment` tests into their own
TestCase.
Adds a classmethod, `instance` on `TradingEnvironment` that returns
`zipline.finance.trading.environment`, instantiating it if necessary.
This makes it possible to initialize the default environment instance in a
less-roundabout way than creating a `SimulationParameters` object.
Refactored the target order methods to separate their logic from the
other order methods.
This makes order_target the only target method that calls order()
directly.
order_target_value is passed to order_target instead of order_value.
order_target_percent is passed to order_target_value rather than
order_value.
This simplified the code and decouples the logic of target orders from
the other order methods. This allows the target order methods to be
developed independently from order_value and order_percent.
Many algorithms that use the new order methods like order_target()
will legitimately try to order 0 shares many times. The printed
warning at every turn is quite annoying and too verbose. We do not
display it on Quantopian either so I'm removing it here as well.
Filter out empty lists from `get_open_orders` so that we have consistent
behavior between the case where a user has never placed an order and the case
where the user has placed an order but it has been executed or cancelled.
A nice side-effect, which was the impetus for this change, is that you can
check if you have any open orders by doing:
```
len(get_open_orders()) == 0
```
Also adds a test for the behavior of `get_open_orders`, which was previously
lacking.
Make the portfolio property on TradingAlgorithm call `updated_portfolio`
internally. This prevents needless recomputation of the portfolio between
calls to `handle_data`, and also prevents issues where the portfolio object
could be unexpectedly modified in place in the body of a `handle_data` call.
Noteworthy finding in the course of investigating this bug:
If you modify a Python dictionary while iterating over it, the language will
only throw an exception if the size of the dictionary changes between loop
iterations; this means that you can do:
```
x = {1:1, 2:2, 3:3}
for k in x:
old_val = x[k]
del x[k]
x[f(k)] = old_val
print k
```
and you'll only get an error if f(k) is already a key in the dictionary.
This can lead to bizarre/nondeterministic behavior in the key iterator.
Make the error messages for {DoBadThing}PostInit no longer reference "the
simulation", since the algorithm may not actually be running as a simulation.
Adds four new methods to the Zipline API that can be used as circuit-breakers
to interrupt the execution of an algorithm. The API methods are:
`set_max_position_size`
`set_max_order_size`
`set_max_order_count`
`set_long_only`
Internally, these methods are implemented by each registering a TradingControl
callback object with the TradingAlgorithm. During
TradingAlgorithm.__validate_order_params (and thus before any side-effects of
the order call occur), each callback's `validate` method is called with
information about the order to be placed and the algorithm's current state,
raising an exception if the callback detects that an error condition has been breached.
TradingEnvironment class uses env_trading_calendar for trading days,
but the default trading calendar for open_and_close data, which causes
errors later, because of misalignment of trading days.
The issue can be resolved by using env_trading_calendar for
open_and_closes as well
Delaney Granizo-Mackenzie