The calculations that are expected to change are:
- cumulative.beta
- cumulative.alpha
- cumulative.information
- cumulative.sharpe
- period.sortino
* Explanation of how risk calculations are changing
** Risk Fixes for Both Period and Cumulative
*** Downside Risk
Use sample instead of population for standard deviation.
Add a rounding factor, so that if the two values are close for a given
dt, that they do not count as a downside value, which would throw off
the denominator of the standard deviation of the downside diffs.
*** Standard Deviation Type
Across the board the standard deviation has been standardized to using
a 'sample' calculation, whereas before cumulative risk was monstly using
'population'. Using `ddof=1` with `np.std` calculates as if the values
are a sample.
** Cumulative Risk Fixes
*** Beta
Use the daily algorithm returns and benchmarks instead of annualized
mean returns.
*** Volatility
Use sample instead of population with standard deviation.
The volatility is an input to other calculations so this change affects
Sharpe and Information ratio calculations.
*** Information Ratio
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
*** Alpha
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
** Period Risk Fixes
*** Sortino
Use the downside risk of the daily return vs. the mean algorithm returns
for the minimum acceptable return instead of the treasury return.
The above required adding the calculation of the mean algorithm returns
for period risk.
Also, use algorithm_period_returns and tresaury_period_return as the
cumulative Sortino does, instead of using algorithm returns for both
inputs into the Sortino calculation.
* Other Supporting Changes
** answer_key
Add new mappings for downside risk and Sortino as well as
re-address the index mappings because of changes to the answer key
spread sheet.
** test_risk_cumulative
Change the decimal precision to expect higher precision.
The calculations are now more aligned with the answer key, so we can
expect higher precision. In particular now that the standard deviation
type matches everywhere in both the Python implementation and the answer
sheet, the precision of the first value no longer has to be glossed over.
** test_events_through_risk
Change the results which are used as a canary for risk changes,
since we do expect Sharpe to change with this change..
The risk unit tests were using the public Yahoo! data instead
of the returns from the answer key spreadsheet, change the RiskPeriod's
created in tests to use the values in the benchmark returns
column of the answer key.
Also, change the spreadsheet's benchmark volatility calculation
to use sample.
The use of population was exposed when the input values were
corrected.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
This could perhaps be labelled BUG, as well.
Change the Sharpe (and algorithm volatiilty) value used to compare
algorithms/backtests so that it is annualized and uses daily returns.
Previously, the Sharpe metric was using the same calculation style
as the fixed size periods, i.e. 3 Month, 6 Month, etc., which can
use the geometric mean when comparing against the risk free.
Change the Sharpe calculation to use the arithmetic mean differenc
against the risk free rate, using daily (non-compounded) values.
Also, use annualized mean returns.
Correct the annualization factor from being 1/sqrt(252), since
the annualization was applied to the volatility, by including
252 in the Sharpe's numerator.
Update answer key for cumulative risk:
- Annualization of Sharpe
- Use 10 year period
- Use of daily returns vectors instead of compounded return scalar.
No tests or risk module code are currently reading off of this sheet,
but developing it ahead of work in risk module so that the sheet can
be examined and vetted.
Fix the spreadsheet to apply a factor of COUNT / COUNT - 1
to the COVAR value.
Also, go back to using the C[1][1] index instead of calculating
var independently.
Add reference to updated answer key with benchmark variance cells,
and use the new cells as the reference for the benchmark variance
test.
The values changed from the original hardcoded values, due to the
change to close over close benchmarks.
So that the answer key does not onerous on the SCM repo size, add a
utility to download the answer key automatically.
Prevent re-download on every test suite run if the local answer key
matches the latest version.
Eddie Hebert