This commit modifies the DataFrameSource and DataPanelSource to accept only Int64Indexes on the incoming data and moves the burden of mapping user identifiers to TradingAlgorithm.run().
The identifier cache's usage was nearly identical to using lookup_generic, so this commit removes identifier-keyed caching and modifies anything that uses it.
Move the responsibility of wrapping the main simulation loop in the
ZiplineAPI context from the algorithm modules generator setup to the
main trade simulation loop, so that different methods of invoking loop
do not need to duplicate how the context is set.
To make it easier for internal implementation of handle_data to
transition off of calling the ZiplineAPI every bar, to only invoking the
context once per simulation.
Instead of using the pandas.Series datetime index for every single
vector, get the index at the beginning of the update loop based on the
dt and then use that index to set the values.
Also, since the dt lookup is no longer needed, store the values as numpy
arrays, which are more lightweight.
Locally, this patch cuts out about 60% of the time spent in the update
method.
- AssetFinder no longer accepts an unused trading_calendar.
- AssetFinder correctly accepts a DataFrame as input.
- Tests for AssetFinder no longer rely on a global trading environment.
Use explicit references to the performance tracker instead of the proxy
lookup.
Mainly a putative change, which surfaced when reasoning about places
where the position tracker and period need access to the last sale
price.
This is to fix the issue where 'GOOG_L's metadata lists its symbol as 'GOOG'. To resolve this, the 'file_name' is used as the canonical symbol, if it is given.
np.allclose creates a vector of size 1 if passed a scalar, which showed
up as a bottleneck in algorithms with a large number of
order_target_percent calls.
Test sources are now defined by the sim_params period_start and period_end, rather than by the period_start and a defined 'count' of bars. This allows us to consider the sim_params.period_end as the canonical definition of the end of a simulation.
Referencing context.account in handle_data() can block updates to account,
portfolio and performance metrics, which can cause unintended changes in
financial performance. Additionally, since context.account metrics are not
updating frequenctly enough (daily, rather than minutely) algos which base
decisions on these metrics can have undesired behavior.
To address this we do not base our decision to update performance on the
state of account_needs_update.
When we call get_account() from the market and minute close methods, we
set the performance_needs_update parameter to False since we just updated
performance.
_get_account() is renamed to _update_account() since the method does not
return account information.
Also remove redundant asset_needs_update = True lines.
Make the ordering in which processing of event types both explicit and
independent of the sort ordering of the incoming sources.
The overhead of creating the list per snapshot and the iterators appears
to be marginal in the minute data case when tested locally.
This patch is intended as part of the path towards making the trade
simulation loop not depend on consuming and tracking every trade event.
The timing of where last_sale_date was needed to be changed was proving
difficult to adapt in the previous model.
Should also allow the removal of sorting of the various source streams.
By having both the trade simulation main loop route events to "process"
methods based on event type and the process methods also checking event
type, there was some duplicated effort in doing that comparison many
times.
A particular case where this was noted in profiling was for the
`process_event` function which was checking if the type was not a trade
and returning early, when in a larger universe of stocks the value
returned False 99% of the time.
Instead provide separate process functions specific to each type,
e.g. e.g. `process_trade` and `process_transaction` and route traffic to
those functions in tradesimulation.
For a universe of 160 stocks on both no-op algo and an algo that rebuys
its universe every day, saw about a 10% increase locally.
Also:
- Add process_benchmark to blotter since internal subclass relies on
logic on benchmark, this allows the internal process_trade to be a
`pass`.
- Add warning on unrecoginzed event types.
The risk containers that are actually used for reports use the
'cumulative' style container which has an index of days, not minutes.
The minute containers and copying of data etc. were causing an expanding
memory footprint.
The intraday_risk_metrics is being removed since the values are not
used; cumulative risk metrics with the last value updated to the latest
close has been used for some time.
Before the removal of intraday_risk_metrics, the position trackers
passing of benchmark returns to the cumulative risk metrics needs to no
longer depend on the calculations done by the intraday stats. So instead
use the all_benchmark_returns stored in the tracker directly.
jfkirk