For example, to prevent `alpha` from internally calling
`beta` a second time, pass the previously-calculated
`beta` value in.
Requires empyrical 0.1.10 from pypi
Instead of having separate ExchangeCalendar and TradingSchedule objects, we
now just have TradingCalendar. The TradingCalendar keeps track of each
session (defined as a contiguous set of minutes between an open and a close).
It's also responsible for handling the grouping logic of any given minute
to its containing session, or the next/previous session if it's not a market
minute for the given calendar.
This commit removes the ability to reference a shared TradingEnvironment through the zipline.finance.trading module. In place, the classes that require a TradingEnvironment, or its child AssetFinder, contain their own references to those objects.
This commit also adds serialization utilities that allow for the pickling/unpickling of objects without unintentionally their TradingEnvironments or AssetFinders.
The minutely calculation of risk metrics had been removed with a
previous patch, remove vestigial references.
Remove a test which tested the behavior of updating the second minute of
a day.
Remove the logic that changed the datetime index of the risk metrics
depending on emission rate, now only trading_days are needed.
Remove `returns_frequency` parameter since both minute and daily
data frequency always use daily returns.
Instead of using the pandas.Series datetime index for every single
vector, get the index at the beginning of the update loop based on the
dt and then use that index to set the values.
Also, since the dt lookup is no longer needed, store the values as numpy
arrays, which are more lightweight.
Locally, this patch cuts out about 60% of the time spent in the update
method.
For beta calculation:
Remove `.dropna` , since it was creating a new
Series and Index which inflated memory usage as algorithm run time
progressed.
For downside risk calculations:
Instead of using pd.Series calculations, pass the underlying
numpy array which have already been sliced to the exact dt, so that the
call to `round` does not create a new Series.
Previously the class SerializeableZiplineObject was used to
house basic __setstate__ and __getstate__ methods. It wasn't
really doing much that was helpful, so it is now gone.
Risk calculations are robust to nans, except for
beta which calls numpy with the complete list of
algorithm_returns. If nans are present the result
of covar will be nan.
This is fixed by filtering out nans in
algorithm_returns.
The check of existence of the null return key, and the drop of said
return on every single bar was adding unneeded CPU time when an
algorithm was run with minute emissions.
Instead, add the 0.0 return with an index of the trading day before
the start date.
The removal of the `null return` was mainly in place so that the
period calculation was not crashing on a non-date index value;
with the index as a date, the period return can also approximate
volatility (even though the that volatility has high noise-to-signal
strength because it uses only two values as an input.)
The factoring out of the Sharpe calculation changed behavior
so that both period and cumulative return nans when there is
no volatility; however before that change period returned 0.0.
This breaks existing consumers which expected a non-nan value
for period results.
Smooth out that change by checking the value after the sharpe
has been calculated and reset nan's to 0.0
The calculations that are expected to change are:
- cumulative.beta
- cumulative.alpha
- cumulative.information
- cumulative.sharpe
- period.sortino
* Explanation of how risk calculations are changing
** Risk Fixes for Both Period and Cumulative
*** Downside Risk
Use sample instead of population for standard deviation.
Add a rounding factor, so that if the two values are close for a given
dt, that they do not count as a downside value, which would throw off
the denominator of the standard deviation of the downside diffs.
*** Standard Deviation Type
Across the board the standard deviation has been standardized to using
a 'sample' calculation, whereas before cumulative risk was monstly using
'population'. Using `ddof=1` with `np.std` calculates as if the values
are a sample.
** Cumulative Risk Fixes
*** Beta
Use the daily algorithm returns and benchmarks instead of annualized
mean returns.
*** Volatility
Use sample instead of population with standard deviation.
The volatility is an input to other calculations so this change affects
Sharpe and Information ratio calculations.
*** Information Ratio
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
*** Alpha
The benchmark returns input is changed from annualized benchmark returns
to the annualized mean returns.
** Period Risk Fixes
*** Sortino
Use the downside risk of the daily return vs. the mean algorithm returns
for the minimum acceptable return instead of the treasury return.
The above required adding the calculation of the mean algorithm returns
for period risk.
Also, use algorithm_period_returns and tresaury_period_return as the
cumulative Sortino does, instead of using algorithm returns for both
inputs into the Sortino calculation.
* Other Supporting Changes
** answer_key
Add new mappings for downside risk and Sortino as well as
re-address the index mappings because of changes to the answer key
spread sheet.
** test_risk_cumulative
Change the decimal precision to expect higher precision.
The calculations are now more aligned with the answer key, so we can
expect higher precision. In particular now that the standard deviation
type matches everywhere in both the Python implementation and the answer
sheet, the precision of the first value no longer has to be glossed over.
** test_events_through_risk
Change the results which are used as a canary for risk changes,
since we do expect Sharpe to change with this change..
An oddity that was exposed while working on making the return series
passed to the risk module more exact, the series comparison between
the returns and mean returns was unbalanced, because the mean returns
were not masked down to the downside data points; however, in most,
if not all cases this was papered over by the call to `.valid()`
Move the downside risk calculation into the main risk module;
so that the same calculation can eventually be used by both
the period and cumulative calculations, to prevent implementation
drift.
Use slice to date, `[:dt]` instead of `pd.Series.valid` to extract
from returns containers.
Using `valid` lead to some confusion when debugging tests, because
it papers over missing data.
The use of `.valid` was based on the assumption that all values
from the zeroth date to the current algo date are populated,
with no trailing values.
`[:dt]` extracts the same data, but in a hopefully more precise
and explicit fashion.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
The __repr__ for RiskMetricsCumulative was referring to an older
structure of the class, causing an exception when printed.
Convert to printing the last values in the metrics DataFrame.
Scott Sanderson