Previously, we have assumed that the `amounts` and `last_sale_prices`
lists have the same order as the `value_multipliers`. This is not
correct, since to populate the `amounts` and `last_sale_prices` lists
we iterate over a `dict` (self.positions). The order of this `dict`
can change in arbitrary ways when it is updated, which occurs when
we call `update_positions`. Our `value_multipliers` however are stored
in an `OrderedDict`, meaning the order of existing key/value pairs
is not changed when they are updated.
To address this issue, we make sure that `self.positions` subclasses
`OrderedDict`.
In preparation for the incoming changes which no longer push every bar
through the tradesimulation, remove the adjustment of the period's cash on
every pricing change of a held futures asset.
Instead hold the last sale price for each held future either:
- At the end of each peformance period update the last sale prices of
all held futures, so that the pnl for the next period uses values
derived from the cash difference between the end of the two periods.
- When a transaction is processed for the Future, so that the correct
amount is applied to each cash adjustment. (i.e. the cash adjustment
is reset on every change of amount of the Future being held, so that
multiple size and prices do not need to be tracked for the same asset.)
Also, remove now unused dict of payout calculation modifier, since new
calculation reads the value directly off of the asset.
Remove update_last_sale test, since the method no longer returns a cash
value.
Instead of calling a function, where the only parameter is the tracker
object, make it a method, so that the snapshot of position tracker stats
can be more easily called as `pt.stats()`.
In preparation for removal of widespread events, change the split
methods to use params for sid and cost, instead of an event, for
compatibility with lazy branch.
co-author: @jbredeche <jean@quantopian.com>
In preparation for removal of widespread events, change the commission
methods to use params for sid and cost, instead of an event, for
compatibility with lazy branch.
co-author: @jbredeche <jean@quantopian.com>
Refer to cumulative and todays performance explicitly instead of always
looping through.
The third value (minute) for which this was useful, has been removed.
Also, there are some actions where only cumulative may need application,
e.g. application of dividends. (However, this patch does not remove
dividend processing from todays performance, but opens up later patches
to make that distinction.)
- Combine the net value and exposure functions into `calc_net` since
they use the same logic.
- Change the logic to handle on empty list to using the a start value of
0.0. More concise, and reduces the number of return points from the
function to one.
Instead of having two leverage functions, whose differences were the
parameter names, add a `calc_leverage` function, with the calling code
determining whether it is gross or net by the type of exposure passed in.
Instead of calculating the position values for each stat result, e.g.
gross_exposure, net_liquidity etc.; get the positions upfront and then
calculate the period and position stats in order, passing each value
explicitly to the ones that follow it in the dependency chain.
e.g. the gross_value depends on the long_value and the short_value,
which called the position_values property for calculating both the
long_value and the short_value.
Removing the repeated calls to position_values (and
position_exposures) removes the need for the caching the last sale
prices and position amounts in separate vectors, since it is inexpensive
enough to read those values off of the positions dictionary held in the
position tracker.
This patch gives a small gain to ~500 sized portfolios, but the main
intent is to clear the path to not storing last_sale_prices on the
position objects at all. Removing all of the caching layer in this class
makes that change easier to apply. Removing the extra calls to
position_values also made this class easier to step through/reason about
when splicing in the new last sale price access, as well.
This commit removes the ability to reference a shared TradingEnvironment through the zipline.finance.trading module. In place, the classes that require a TradingEnvironment, or its child AssetFinder, contain their own references to those objects.
This commit also adds serialization utilities that allow for the pickling/unpickling of objects without unintentionally their TradingEnvironments or AssetFinders.
The minutely calculation of risk metrics had been removed with a
previous patch, remove vestigial references.
Remove a test which tested the behavior of updating the second minute of
a day.
Remove the logic that changed the datetime index of the risk metrics
depending on emission rate, now only trading_days are needed.
Remove `returns_frequency` parameter since both minute and daily
data frequency always use daily returns.
Instead of using the pandas.Series datetime index for every single
vector, get the index at the beginning of the update loop based on the
dt and then use that index to set the values.
Also, since the dt lookup is no longer needed, store the values as numpy
arrays, which are more lightweight.
Locally, this patch cuts out about 60% of the time spent in the update
method.
Use explicit references to the performance tracker instead of the proxy
lookup.
Mainly a putative change, which surfaced when reasoning about places
where the position tracker and period need access to the last sale
price.
Referencing context.account in handle_data() can block updates to account,
portfolio and performance metrics, which can cause unintended changes in
financial performance. Additionally, since context.account metrics are not
updating frequenctly enough (daily, rather than minutely) algos which base
decisions on these metrics can have undesired behavior.
To address this we do not base our decision to update performance on the
state of account_needs_update.
When we call get_account() from the market and minute close methods, we
set the performance_needs_update parameter to False since we just updated
performance.
_get_account() is renamed to _update_account() since the method does not
return account information.
Also remove redundant asset_needs_update = True lines.
By having both the trade simulation main loop route events to "process"
methods based on event type and the process methods also checking event
type, there was some duplicated effort in doing that comparison many
times.
A particular case where this was noted in profiling was for the
`process_event` function which was checking if the type was not a trade
and returning early, when in a larger universe of stocks the value
returned False 99% of the time.
Instead provide separate process functions specific to each type,
e.g. e.g. `process_trade` and `process_transaction` and route traffic to
those functions in tradesimulation.
For a universe of 160 stocks on both no-op algo and an algo that rebuys
its universe every day, saw about a 10% increase locally.
Also:
- Add process_benchmark to blotter since internal subclass relies on
logic on benchmark, this allows the internal process_trade to be a
`pass`.
- Add warning on unrecoginzed event types.
Stewart Douglas