The check of existence of the null return key, and the drop of said
return on every single bar was adding unneeded CPU time when an
algorithm was run with minute emissions.
Instead, add the 0.0 return with an index of the trading day before
the start date.
The removal of the `null return` was mainly in place so that the
period calculation was not crashing on a non-date index value;
with the index as a date, the period return can also approximate
volatility (even though the that volatility has high noise-to-signal
strength because it uses only two values as an input.)
The factoring out of the Sharpe calculation changed behavior
so that both period and cumulative return nans when there is
no volatility; however before that change period returned 0.0.
This breaks existing consumers which expected a non-nan value
for period results.
Smooth out that change by checking the value after the sharpe
has been calculated and reset nan's to 0.0
The calculations that are expected to change are:
- cumulative.beta
- cumulative.alpha
- cumulative.information
- cumulative.sharpe
- period.sortino
* Explanation of how risk calculations are changing
** Risk Fixes for Both Period and Cumulative
*** Downside Risk
Use sample instead of population for standard deviation.
Add a rounding factor, so that if the two values are close for a given
dt, that they do not count as a downside value, which would throw off
the denominator of the standard deviation of the downside diffs.
*** Standard Deviation Type
Across the board the standard deviation has been standardized to using
a 'sample' calculation, whereas before cumulative risk was monstly using
'population'. Using `ddof=1` with `np.std` calculates as if the values
are a sample.
** Cumulative Risk Fixes
*** Beta
Use the daily algorithm returns and benchmarks instead of annualized
mean returns.
*** Volatility
Use sample instead of population with standard deviation.
The volatility is an input to other calculations so this change affects
Sharpe and Information ratio calculations.
*** Information Ratio
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
*** Alpha
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
** Period Risk Fixes
*** Sortino
Use the downside risk of the daily return vs. the mean algorithm returns
for the minimum acceptable return instead of the treasury return.
The above required adding the calculation of the mean algorithm returns
for period risk.
Also, use algorithm_period_returns and tresaury_period_return as the
cumulative Sortino does, instead of using algorithm returns for both
inputs into the Sortino calculation.
* Other Supporting Changes
** answer_key
Add new mappings for downside risk and Sortino as well as
re-address the index mappings because of changes to the answer key
spread sheet.
** test_risk_cumulative
Change the decimal precision to expect higher precision.
The calculations are now more aligned with the answer key, so we can
expect higher precision. In particular now that the standard deviation
type matches everywhere in both the Python implementation and the answer
sheet, the precision of the first value no longer has to be glossed over.
** test_events_through_risk
Change the results which are used as a canary for risk changes,
since we do expect Sharpe to change with this change..
An oddity that was exposed while working on making the return series
passed to the risk module more exact, the series comparison between
the returns and mean returns was unbalanced, because the mean returns
were not masked down to the downside data points; however, in most,
if not all cases this was papered over by the call to `.valid()`
Move the downside risk calculation into the main risk module;
so that the same calculation can eventually be used by both
the period and cumulative calculations, to prevent implementation
drift.
Use slice to date, `[:dt]` instead of `pd.Series.valid` to extract
from returns containers.
Using `valid` lead to some confusion when debugging tests, because
it papers over missing data.
The use of `.valid` was based on the assumption that all values
from the zeroth date to the current algo date are populated,
with no trailing values.
`[:dt]` extracts the same data, but in a hopefully more precise
and explicit fashion.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
The __repr__ for RiskMetricsCumulative was referring to an older
structure of the class, causing an exception when printed.
Convert to printing the last values in the metrics DataFrame.
Python 3 requires submodules to have more explicit pathing, so use
the dot syntax to declare submodules which are in the same directory
as another module.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
Instead of using all calendar days between start and end in test
sources, use the trading calendar for test sources.
Needed for an incoming refactoring of market open and close,
where the opens and closes are indexed by market days.
So that the units match the other risk calculations, also
use annualized returns for beat and alpha.
Update answer key to match values calculated on the first day.
Also, update performance tracker test so that the returns used
are fractional instead of > 1, so that the annualized numbers are
more in line with real world values.
This could perhaps be labelled BUG, as well.
Change the Sharpe (and algorithm volatiilty) value used to compare
algorithms/backtests so that it is annualized and uses daily returns.
Previously, the Sharpe metric was using the same calculation style
as the fixed size periods, i.e. 3 Month, 6 Month, etc., which can
use the geometric mean when comparing against the risk free.
Change the Sharpe calculation to use the arithmetic mean differenc
against the risk free rate, using daily (non-compounded) values.
Also, use annualized mean returns.
Hold on to the values set for the returns frequency and the cont
index, mainly for debugging purposes mid-process.
It was useful to have these values when debugging why there was
an extra non-midnight time in the index.
Instead of midnight to midnight for each day, use the trading
environment's market open and close for each day, so that the index
is exactly the trading minutes of each day.
Reduces the amount of memory consumed, but more importantly should
make it easier to inspect the Series that use the index and check
whether the values are correctly being filled.
Remove more use of lists for storing internal risk values to use
pandas structures, for easier matching of time to value.
Accordingy, convert use of -1 for getting last value,
to use current dt.
Continue on path of converting values stored inside of risk metrics
to use a DataFrame instead of storing multiple lists.
Also, the need for latest_dt in getting the current volatility for
the sharpe calculation, shows that we need to set the lastest_dt at
the beginning of the update loop.
Eventually, all cumulative metrics, (alpha, beta, etc.) will be
stored in the same DataFrame
For easier tracking of dt to values during debugging, but should be
some performance gains as well.
So that it is easier to add new containers, factor out the creation
of the index.
Also, make the returns frequency a parameter, to make the use of
different frequencies more clear from within the risk metrics object,
rather than hot swapping in the new frequency type via the now
removed `initialize_daily_indices`.
The eigen_values, condition_number, algorithm_covariance, and
benchmark variance, which were easy to calculate alongside beta,
since they share the same inputs, but were not passed along to performance.
Remove to trim down the number of risk report members as well as
number of calcluations done.
Can add back in if there is an expressed need for eigen_values etc.,
perhaps in an 'opt-in' type configuration.
Instead of using a pandas Series of with dictionaries as the
values treasury curves, use a DataFrame which more naturally fits
the data type of a having a timeseries with mulitple values.
Should allow easier slicing/manipulation of the treasury curves,
e.g. getting 10 year curves would now be:
```
treasury_curves['10year']
```
Also remove test that compares risk metrics batch to iterative,
since the 'iterative' calculations, replaced by the cumulative
calculations, will intentionally drift from the results in the risk
report due to annualization and other factors.
Work towards having separate calculations for the fixed periods versus
the cumulative/headline risk metrics.
Different sumbodules for each type should help make the calculations
type distinct and easier to find.
Eddie Hebert