The minutely calculation of risk metrics had been removed with a
previous patch, remove vestigial references.
Remove a test which tested the behavior of updating the second minute of
a day.
Remove the logic that changed the datetime index of the risk metrics
depending on emission rate, now only trading_days are needed.
Remove `returns_frequency` parameter since both minute and daily
data frequency always use daily returns.
Instead of using the pandas.Series datetime index for every single
vector, get the index at the beginning of the update loop based on the
dt and then use that index to set the values.
Also, since the dt lookup is no longer needed, store the values as numpy
arrays, which are more lightweight.
Locally, this patch cuts out about 60% of the time spent in the update
method.
For beta calculation:
Remove `.dropna` , since it was creating a new
Series and Index which inflated memory usage as algorithm run time
progressed.
For downside risk calculations:
Instead of using pd.Series calculations, pass the underlying
numpy array which have already been sliced to the exact dt, so that the
call to `round` does not create a new Series.
Previously the class SerializeableZiplineObject was used to
house basic __setstate__ and __getstate__ methods. It wasn't
really doing much that was helpful, so it is now gone.
Risk calculations are robust to nans, except for
beta which calls numpy with the complete list of
algorithm_returns. If nans are present the result
of covar will be nan.
This is fixed by filtering out nans in
algorithm_returns.
The check of existence of the null return key, and the drop of said
return on every single bar was adding unneeded CPU time when an
algorithm was run with minute emissions.
Instead, add the 0.0 return with an index of the trading day before
the start date.
The removal of the `null return` was mainly in place so that the
period calculation was not crashing on a non-date index value;
with the index as a date, the period return can also approximate
volatility (even though the that volatility has high noise-to-signal
strength because it uses only two values as an input.)
The factoring out of the Sharpe calculation changed behavior
so that both period and cumulative return nans when there is
no volatility; however before that change period returned 0.0.
This breaks existing consumers which expected a non-nan value
for period results.
Smooth out that change by checking the value after the sharpe
has been calculated and reset nan's to 0.0
The calculations that are expected to change are:
- cumulative.beta
- cumulative.alpha
- cumulative.information
- cumulative.sharpe
- period.sortino
* Explanation of how risk calculations are changing
** Risk Fixes for Both Period and Cumulative
*** Downside Risk
Use sample instead of population for standard deviation.
Add a rounding factor, so that if the two values are close for a given
dt, that they do not count as a downside value, which would throw off
the denominator of the standard deviation of the downside diffs.
*** Standard Deviation Type
Across the board the standard deviation has been standardized to using
a 'sample' calculation, whereas before cumulative risk was monstly using
'population'. Using `ddof=1` with `np.std` calculates as if the values
are a sample.
** Cumulative Risk Fixes
*** Beta
Use the daily algorithm returns and benchmarks instead of annualized
mean returns.
*** Volatility
Use sample instead of population with standard deviation.
The volatility is an input to other calculations so this change affects
Sharpe and Information ratio calculations.
*** Information Ratio
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
*** Alpha
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
** Period Risk Fixes
*** Sortino
Use the downside risk of the daily return vs. the mean algorithm returns
for the minimum acceptable return instead of the treasury return.
The above required adding the calculation of the mean algorithm returns
for period risk.
Also, use algorithm_period_returns and tresaury_period_return as the
cumulative Sortino does, instead of using algorithm returns for both
inputs into the Sortino calculation.
* Other Supporting Changes
** answer_key
Add new mappings for downside risk and Sortino as well as
re-address the index mappings because of changes to the answer key
spread sheet.
** test_risk_cumulative
Change the decimal precision to expect higher precision.
The calculations are now more aligned with the answer key, so we can
expect higher precision. In particular now that the standard deviation
type matches everywhere in both the Python implementation and the answer
sheet, the precision of the first value no longer has to be glossed over.
** test_events_through_risk
Change the results which are used as a canary for risk changes,
since we do expect Sharpe to change with this change..
An oddity that was exposed while working on making the return series
passed to the risk module more exact, the series comparison between
the returns and mean returns was unbalanced, because the mean returns
were not masked down to the downside data points; however, in most,
if not all cases this was papered over by the call to `.valid()`
Move the downside risk calculation into the main risk module;
so that the same calculation can eventually be used by both
the period and cumulative calculations, to prevent implementation
drift.
Use slice to date, `[:dt]` instead of `pd.Series.valid` to extract
from returns containers.
Using `valid` lead to some confusion when debugging tests, because
it papers over missing data.
The use of `.valid` was based on the assumption that all values
from the zeroth date to the current algo date are populated,
with no trailing values.
`[:dt]` extracts the same data, but in a hopefully more precise
and explicit fashion.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
The __repr__ for RiskMetricsCumulative was referring to an older
structure of the class, causing an exception when printed.
Convert to printing the last values in the metrics DataFrame.
Python 3 requires submodules to have more explicit pathing, so use
the dot syntax to declare submodules which are in the same directory
as another module.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
Instead of using all calendar days between start and end in test
sources, use the trading calendar for test sources.
Needed for an incoming refactoring of market open and close,
where the opens and closes are indexed by market days.
So that the units match the other risk calculations, also
use annualized returns for beat and alpha.
Update answer key to match values calculated on the first day.
Also, update performance tracker test so that the returns used
are fractional instead of > 1, so that the annualized numbers are
more in line with real world values.
This could perhaps be labelled BUG, as well.
Change the Sharpe (and algorithm volatiilty) value used to compare
algorithms/backtests so that it is annualized and uses daily returns.
Previously, the Sharpe metric was using the same calculation style
as the fixed size periods, i.e. 3 Month, 6 Month, etc., which can
use the geometric mean when comparing against the risk free.
Change the Sharpe calculation to use the arithmetic mean differenc
against the risk free rate, using daily (non-compounded) values.
Also, use annualized mean returns.
Hold on to the values set for the returns frequency and the cont
index, mainly for debugging purposes mid-process.
It was useful to have these values when debugging why there was
an extra non-midnight time in the index.
Eddie Hebert