Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
So that algorithms that specifiy market_aware, days and delta
as args can be transitioned to just specifying a window_length kwarg.
Moving towards removing market_aware and delta completely.
The eventual goal is to remove the market_aware and delta kwargs,
but removing the kwarg completely would break the init
method of EventWindow based classes for existing algorithms.
In the meantime, this ensures that only market_aware is only ever
set to True and raises an error if it is False.
Also raises exceptions if values are set for delta, which was only
used if market_aware was False
Also, since the logic for checking window length is changed,
because in market aware mode we should always be checking the
window length, this adds some sanity checking to the window length.
Takes the value set for a variable on handle_data and records it,
e.g.:
```
def initialize(self):
self.incr = 0
self.record_variables(['incr'])
def handle_data(self, data):
self.incr += 1
```
Would record a variable of `incr`.
Emits the recorded variables as part of the daily performance.
This batch combins work from:
Thomas Wiecki <thomas.wiecki@gmail.com> (@twiecki)
fawce <fawce@quantopian.com> (@fawce)
The trading day index is all business days in range minus the
non trading days we are already calculating.
Also, uses trading calendar indexes for batch transform, since the
batch transform was the only use of non_trading_days.
Instead of constantly adding and removing holidays to do market
day delta math, uses pandas DatetimeIndex to get the index of the dates
and uses the index difference to calculate market days.
The test factory was creating non-market days.
i.e. the date range spanned the weekend.
Using pandas' BDay frequency so that only business days are created.
This specific date range doesn't have holidays, so not accounting
for holidays in the factory.
Also, widens the range of the trading calendar to cover the test dates
generated by the factory which include 1990.
Previously the trading calendar began with 2002, meaning that holiday
and weekend adjustments with the data exercised by the factory did
not trigger when run with data in 1990.
This does increase the memory footprint of the tradingcalendar module.
However, only by a couple MB, so taking the hit there to enable
correct behavior.
Reducing use of ndict on the path to fixing a memory leak.
So that pulling an instance of ndict with objgraph is more likely
to be a use of ndict that is causing the leak.
Using a class level constant here suffices for the dot access
desired on messages.
Though the addition of tracking mulitple values in the window
is powerful, the changes broke behavior of existing algorithms
by changing method signatures and names.
So temporarily reverting these changes, to be pulled back in when
a way to have the multiple fields tracked with the existing API
is written, or a cutover of the API is figured out and determined.
Since the position amount and price ndarrays are one dimensional
and use real numbers, we do not need the overhead of the extra
case handling provided by numpy.vdot, which comes at a cost of
performance.
With thanks to @jlowin, for pointing out the better fit of numpy.dot.
Gets almost 100x speed up over iterating over the values and
summing up the values in Python.
Farms out the work to numpy and atlas by using the vector dot
product of the amounts and last sale prices.
Adds some wiring of keeping track of an index into the numpy arrays
for each position, so that value can be overwritten as events update
those amounts and sale prices.
Instead of doing the rollover by creating a new PerformancePeriod,
introduces a `rollover` method that resets the values that need
to be fresh in a new period, and moves the ending values to starting
values, and leaves positions intact.
This isn't a major runtime improvement in of itself, but it does
allow us to more easily keep track of position values from period
to period, which other improvements will use.
Instead of creating a new ndict for each position on every event,
we change the values in the object that held the previous position.
The creation of new objects on each event was incurring too much
overhead.
Changes the position type returned by performance module.
For improved speed, changes from ndict to a simple Python object,
since the cost of setting ndict values is too expensive for the
number of times that positions are returned.
Also, changes the containing type of the positions to be dictionary
with the __missing__ overloaded, instead of the ndict that had that
behavior, to reduce the penalty of using ndicts.
The creation of a new portfolio ndict on each call of handle_data
was creating a very high performance overhead.
Instead, we use the same the portfolio object for each event,
and replace the values contained within.
fawce