Gets almost 100x speed up over iterating over the values and
summing up the values in Python.
Farms out the work to numpy and atlas by using the vector dot
product of the amounts and last sale prices.
Adds some wiring of keeping track of an index into the numpy arrays
for each position, so that value can be overwritten as events update
those amounts and sale prices.
Instead of doing the rollover by creating a new PerformancePeriod,
introduces a `rollover` method that resets the values that need
to be fresh in a new period, and moves the ending values to starting
values, and leaves positions intact.
This isn't a major runtime improvement in of itself, but it does
allow us to more easily keep track of position values from period
to period, which other improvements will use.
Instead of creating a new ndict for each position on every event,
we change the values in the object that held the previous position.
The creation of new objects on each event was incurring too much
overhead.
Changes the position type returned by performance module.
For improved speed, changes from ndict to a simple Python object,
since the cost of setting ndict values is too expensive for the
number of times that positions are returned.
Also, changes the containing type of the positions to be dictionary
with the __missing__ overloaded, instead of the ndict that had that
behavior, to reduce the penalty of using ndicts.
The creation of a new portfolio ndict on each call of handle_data
was creating a very high performance overhead.
Instead, we use the same the portfolio object for each event,
and replace the values contained within.
Gains some performance by using a 'regular' object instead of
an ndict.
Also, directly sets up the values that we return, instead of going in
between with __core_dict and then removing values.
In it's entirety performanc.as_portfolio is the current
highest bottleneck, working on reducing time spent in that function.
Gets almost 100x speed up over iterating over the values and
summing up the values in Python.
Farms out the work to numpy and atlas by using the vector dot
product of the amounts and last sale prices.
Adds some wiring of keeping track of an index into the numpy arrays
for each position, so that value can be overwritten as events update
those amounts and sale prices.
Instead of doing the rollover by creating a new PerformancePeriod,
introduces a `rollover` method that resets the values that need
to be fresh in a new period, and moves the ending values to starting
values, and leaves positions intact.
This isn't a major runtime improvement in of itself, but it does
allow us to more easily keep track of position values from period
to period, which other improvements will use.
Instead of creating a new ndict for each position on every event,
we change the values in the object that held the previous position.
The creation of new objects on each event was incurring too much
overhead.
Changes the position type returned by performance module.
For improved speed, changes from ndict to a simple Python object,
since the cost of setting ndict values is too expensive for the
number of times that positions are returned.
Also, changes the containing type of the positions to be dictionary
with the __missing__ overloaded, instead of the ndict that had that
behavior, to reduce the penalty of using ndicts.
The creation of a new portfolio ndict on each call of handle_data
was creating a very high performance overhead.
Instead, we use the same the portfolio object for each event,
and replace the values contained within.
Gains some performance by using a 'regular' object instead of
an ndict.
Also, directly sets up the values that we return, instead of going in
between with __core_dict and then removing values.
In it's entirety performanc.as_portfolio is the current
highest bottleneck, working on reducing time spent in that function.
Uses heapq.merge to sort input from mulitple sources instead of
our own sort module.
From profiling heapq.merge is more efficient than our own efforts.
update_universe is a bottleneck on large data sets.
A large portion of that bottleneck is the call to getitem while
looping over the keys, so using update while passing along the internal
__dict__
Seeing about a 40% improvement.
So that an Event can use an initial dict to set all values,
instead of needing to set initial values one by one.
i.e. enables:
```
foo = Event({'bar': 1, 'baz': 2})
```
in favor of:
```
foo = Event()
foo.bar = 1
foo.baz = 2
```
Previously, the list was generated, but only used to calculate
the number of days in the environment.
With exposing this list, working towards a path where the simulation
uses the trading days to determine when to handle market closes.
The delta was ensuring that the backtester wouldn't exceed the
delta of a bar if it were being run against live data.
However, this extra overhead of getting the current time on each
side of the handle_data adds a penalty in pure backtest mode.
Also, it makes the backtest results potentially non-repeatable,
since it is sensitive to current conditions on a box for processing
time.
Favoring having the timeout handled by whatever is running the
zipline algorithm.
There are only 6 trading days between the open and close specified
in test_perf test.
Also, removes getting the period_end off of the last trade,
since the test can now use the end date specified for the trading
environment.
Thomas Wiecki