Currently, just provide a way to render to some of the data extracted.
Intended to have more thorough documentation of the spreadsheet,
explaining derivation/calculations in each sheet and column.
Add indexes for Sharpe, returns and other values needed for
reading answers for cumulative risk metrics.
Prepare for unit test and matching change of implementation.
Refactor the reading of values from the Excel spreadsheet so that
parsers are configurable by index.
Needed so that we can parse columns that have dates, in addition
to floats as previously.
Instead of using the indexes defined in the answer key class
to index back into the answer key object, populate the answers
so that they are available as members of the answer key object.
Update period risk test to use new answer key structure.
Also, remove the rounding behavior from the answer sheet, leaving
the rounding to the consumer of the answer key values, so that
the values can be retrieved from the spreadsheet during answer
key __init__ without knowledge of the decimal point that the calling
code expects.
Correspondingly, change period risk tests to use
np.testing.assert_almost_equal when doing floating point comparison.
Move to a new notebook, the AnswerKeyLink will be for a permalink
to the current version of the answer key, of which the output
won't be too noisy in git.
The annotation notebook will also be kept in source control, but
without output, since the table html output is large.
For more improved viewing experience via nbviewer.ipyhton.org,
include the output of the notebook.
When saving/updating this file, a fresh kernel and evaluation of
the entire notebook should be used so that the cell numbers stay
in order.
Read tests.risk.answer_key module into an IPython notebook, to start
a base to which answer key values and explanations can be added and
display without access to Excel.
For now, the notebook just provides the latest download link for
the spreadsheet.
Update answer key for cumulative risk:
- Annualization of Sharpe
- Use 10 year period
- Use of daily returns vectors instead of compounded return scalar.
No tests or risk module code are currently reading off of this sheet,
but developing it ahead of work in risk module so that the sheet can
be examined and vetted.
Also remove test that compares risk metrics batch to iterative,
since the 'iterative' calculations, replaced by the cumulative
calculations, will intentionally drift from the results in the risk
report due to annualization and other factors.
Work towards having separate calculations for the fixed periods versus
the cumulative/headline risk metrics.
Different sumbodules for each type should help make the calculations
type distinct and easier to find.
For maintainer use, requires AWS credentials for the account where
the `zipline-test-data` bucket is hosted.
Script does the following steps which used to be manual:
- Create a key name based on the md5 of the answer key file.
- Upload the answer key to S3 bucket.
- Make the file publically downloadable over HTTP.
Fix the spreadsheet to apply a factor of COUNT / COUNT - 1
to the COVAR value.
Also, go back to using the C[1][1] index instead of calculating
var independently.
Use recent change to benchmark variance in the beta calculation,
instead of referring to the 4th quadrant of the covariance.
Also, read answers from answer key for corroboration of beta values.
The np.cov call needs a ddof of 0 to match the answer key, which uses
Excel's VAR.
When switching np.cov to use a ddof of 0, the benchmark variance is
no longer the 4th quadrant of the cov result, so use np.var directly.
Add reference to updated answer key with benchmark variance cells,
and use the new cells as the reference for the benchmark variance
test.
The values changed from the original hardcoded values, due to the
change to close over close benchmarks.
So that the answer key does not onerous on the SCM repo size, add a
utility to download the answer key automatically.
Prevent re-download on every test suite run if the local answer key
matches the latest version.
The risk tests originally were based on a spread sheet, with the
results of returns etc copy and pasted into the `test_risk` module.
Include the spreadsheet and read the values directly using a Python
Excel spreadsheet library.
The TALib transform only supported operating on the first value
of a given batch transform panel row.
Instead of returning the one value, even if an panel with multiple
sids was provided, return a dictionary that maps stock to TALib
result.
Add a `bars` keyword arg, as is used with BatchTransform.
Also, instead of overwriting the window_length kwarg with timeperiod,
always use the lookback value from the created TALib function,
as timeperiod will be an input into that value if it exists.
Calculate `window_length` in minute mode so that there are enough
days to cover the minutes in the timeperiod.
For the creation of a TALib transform use timeperiod intsead of
window_length, to be more in the style of TALib usage, since all
TALib functions may not ending up using BatchTransform, so start
the practice of adhering to TALib conventions to make porting and
explanation easier.
The "use_environment" decorator is too side-effectful (e.g.,
connecting to Yahoo! Finance or another data source) to be used as a
decorator to a function that gets evaluated during module load. This
causes problems, e.g., if Zipline is being used in a gevent
environment, when the trading environment created by the decorator
argument tries to use greenlets when gevent hasn't been fully
initialized.
Since the decorator is nothing more than a context-manager wrapper,
this commit removes the decorator and replaces its use with contexts,
i.e., "with" statements.
Instead of searching through the open orders to find the ones
that match the current transactions, now that simulate returns
the pair of transaction and order for which that transaction
was created for, that order can be used where we previously
searched for a modified order.
This should be a runtime improvement since, but not yet verified
via thorough profiling.
The TALib related tests randomly fail, skip for now, as it causes
noise when developing against the latest versions of the codebase.
Should remove skips when TALib is closer to being fully supported.
So that blotter.process_trade doesn't need to reindex the dictionary of
open orders, yield a tuple of (order, transaction) from simulate.
Also, update corresponding unit tests now that the method returns
a generator instead of a list.
Eddie Hebert