This commit removes the ability to reference a shared TradingEnvironment through the zipline.finance.trading module. In place, the classes that require a TradingEnvironment, or its child AssetFinder, contain their own references to those objects.
This commit also adds serialization utilities that allow for the pickling/unpickling of objects without unintentionally their TradingEnvironments or AssetFinders.
Test sources are now defined by the sim_params period_start and period_end, rather than by the period_start and a defined 'count' of bars. This allows us to consider the sim_params.period_end as the canonical definition of the end of a simulation.
Fixes a crash in various transforms when providing CUSTOM events whose fields
don't match the fields required for the transform.
This is fixed by requiring all `EventWindow` subclasses to supply a `fields`
property, which returns a list of strings that are required keys for any event
that can be processed by the window. Any CUSTOM events the don't supply the
required fields for a transform window are ignored by that window.
The data zipline_transform.window is always evaluating to empty,
thus the actual checks are not being used because of the
`if not data` done before running the `asserts`.
This behavior should be fixed, and we should either remove the
`not data` check, or bubble up that the check is being hit too
many times; but in the meantime, disabling this test which takes
a non-trivial amount of time to run.
When run as an algorithm, outside of unit tests the talib wrapper
does work, so the cause of the window always being empty may be to
due to the machinery of the unit test.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
- Use `print()` function for all print calls
- Fix strip and format calls that were on the outside of the
print function for some reason.
(Which were breaking in Python 3 because of print returning None.)
- Remove commented out print calls.
The TALib transform only supported operating on the first value
of a given batch transform panel row.
Instead of returning the one value, even if an panel with multiple
sids was provided, return a dictionary that maps stock to TALib
result.
Add a `bars` keyword arg, as is used with BatchTransform.
Also, instead of overwriting the window_length kwarg with timeperiod,
always use the lookback value from the created TALib function,
as timeperiod will be an input into that value if it exists.
Calculate `window_length` in minute mode so that there are enough
days to cover the minutes in the timeperiod.
For the creation of a TALib transform use timeperiod intsead of
window_length, to be more in the style of TALib usage, since all
TALib functions may not ending up using BatchTransform, so start
the practice of adhering to TALib conventions to make porting and
explanation easier.
The TALib related tests randomly fail, skip for now, as it causes
noise when developing against the latest versions of the codebase.
Should remove skips when TALib is closer to being fully supported.
When setting timeperiod in the talib function it subtracts by 1. We then used this subtracted value to set the window_length in the batch_transform which was then not passing a big enough panel. Ultimately this caused the talib transforms to always return nans.
This also makes the unittest more stringent by explicitly comparing the output of the wrapped TALib moving average to pandas rolling_mean().
Finally, this also allows passing of window_length instead of timeperiod to allow usage of the same interface as before.
So that TALib is still available, but smooth out the ability to
run tests with some issues that bear investigating.
- Ignore MAVP during tests.
- Temporarily use a "regular" member instead of __doc__ string.
(TODO: look into using `type` to generate the class)
- During tests wait until a window exists.
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.
Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.
================
Details:
- adding delorean to the requirements.
- made index symbol a parameter for loading the benchmark data. changed
messagepack storage to be symbol specific.
- ported risk, performance, algorithm, transforms, batch transforms
and associated tests to use simulation parameters and global environment
- factory and sim factory use global state and sim params
- factory method parameter names now reflect the class expected
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
Changes these tests to use market_aware==True, so that unit tests
follow the same code path as actual execution.
All use of EventWindows against data follows market_aware behavior.
These tests are the only use of market_aware==False, so heading
down the path of removing market aware completely.
The test factory was creating non-market days.
i.e. the date range spanned the weekend.
Using pandas' BDay frequency so that only business days are created.
This specific date range doesn't have holidays, so not accounting
for holidays in the factory.
Also, widens the range of the trading calendar to cover the test dates
generated by the factory which include 1990.
Previously the trading calendar began with 2002, meaning that holiday
and weekend adjustments with the data exercised by the factory did
not trigger when run with data in 1990.
This does increase the memory footprint of the tradingcalendar module.
However, only by a couple MB, so taking the hit there to enable
correct behavior.
These tests ensure that there are three, not two, empty values
at the beginning of the transform.
Also, ensures that we are using a window length of 3 on the tests.
So that wordings of errors,etc. match the window length.
Though the addition of tracking mulitple values in the window
is powerful, the changes broke behavior of existing algorithms
by changing method signatures and names.
So temporarily reverting these changes, to be pulled back in when
a way to have the multiple fields tracked with the existing API
is written, or a cutover of the API is figured out and determined.
jfkirk