Change the algorithm volatility test to use the same iterkv style
as the rest of the suite, as it was useful to be able to zero in
on the offending date when debugging changes to the risk module.
The risk unit tests were using the public Yahoo! data instead
of the returns from the answer key spreadsheet, change the RiskPeriod's
created in tests to use the values in the benchmark returns
column of the answer key.
Also, change the spreadsheet's benchmark volatility calculation
to use sample.
The use of population was exposed when the input values were
corrected.
The data zipline_transform.window is always evaluating to empty,
thus the actual checks are not being used because of the
`if not data` done before running the `asserts`.
This behavior should be fixed, and we should either remove the
`not data` check, or bubble up that the check is being hit too
many times; but in the meantime, disabling this test which takes
a non-trivial amount of time to run.
When run as an algorithm, outside of unit tests the talib wrapper
does work, so the cause of the window always being empty may be to
due to the machinery of the unit test.
This adds a new data source that emits events
with certain user-specified frequency (minute
or daily).
This allows users to backtest and debug an
algorithm in minute mode to provide a cleaner
path towards Quantopian.
Adding a copy of the Event's dt field as datetime via the
`alias_dt` generator, so that the API was forgiving and allowed
both datetime and dt on a SIDData object, was creating noticeable
overhead, even on an noop algorithms.
Instead of incurring the cost of copying the datetime value and
assigning it to the Event object on every event that is passed
through the system, add a property to SIDData which acts as an
alias `datetime` to `dt`.
Eventually support for `data['foo'].datetime` may be removed,
and could be considered deprecated.
A symbol() lookup feature was added to Quantopian.
By adding the same API function to zipline we can
make copy&pasting of a zipline algo to Quantopian
easier.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
To prevent cases where custom data types had unaligned timestamps,
only call handle_data when market data passes through.
Custom data that comes before market data will still update
the data bar. But the handling of that data will only be done
when there is actionable market data.
When running with minutely emissions the simulator would report to the
user that it simulated 'n - 1' days (where n is the number of days
specified in the simulation params). Now the correct number of trading
days are reported as being simulated.
Previously benchmark events were emitted at 0:00 on the day the
benchmark related to: in 'minute' emission mode this meant that
the benchmarks were emitted before any intra-day trades were
processed.
See: https://github.com/quantopian/zipline/issues/241
This is a step towards the goal of uniting Quantopian scripts
and zipline.
To make the syntax of zipline identical to Quantopian
we break out the API methods (like order) and turn them into
functions. To access the algo object we add a thread local reference
to the current algorithm that is accessed in the API functions.
TradingAlgorithm now takes either a string or two functions
(initialize and handle_data) that it executes.
Use api method decorator for methods available in algoscript.
Ported appropriate algorithm tests from internal code.
To help prevent algorithms from operating on positions that are
not in the existing universe of stocks.
Formerly, iterating over positions would return positions for stocks
which had zero shares held. (Where an explicit check in algorithm
code for `pos.amount != 0` could prevent from using a non-existent
position.)
Use date sorted sources instead, instead of sorting with second
argument of Event, etc. since the `heapq.merge` behavior is using
the second part of the tuple, thus requiring a richer set of comparison
methods, which would only be used in the test context.
Use `date_sorted_sources` instead, so that sorting is done on algo time
and source id.
Python 3 removes the `.message` attribute, so use `str` instead.
Also, the divide by zero message has changed slightly between versions,
so just check for the exception type, instead of also checking the message.
The rename of walk is not provided by six, so check the import error
via an exception.
Also, callback behavior slightly changes between the two versions,
so instead iterate over the walked files and call what was formerly
a callback, directly as a function.
Python 3 uses the `__next__` method instead of `next`,
and uses the syntax of `next(foo)` accordingly.
Add `__next__` and `next` side-by-side so both Python 2 and 3 have
a method that can be used during iteration.
Python 3 requires submodules to have more explicit pathing, so use
the dot syntax to declare submodules which are in the same directory
as another module.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
- Use `print()` function for all print calls
- Fix strip and format calls that were on the outside of the
print function for some reason.
(Which were breaking in Python 3 because of print returning None.)
- Remove commented out print calls.
Note that the calendar test is decorated with @nottest (as per the other calendar test functions). I've run the test to confirm the calendar works. The differences between the env (Yahoo Finance of GSPTSE) and the calendar are illustrated in the tradingcalendar_tse file and are confirmed to be errors on Yahoo Finance's part.
A bug in the create_random_simulation_parameters allows the period
start to be a non-trading day.
That bug was causing the commission tests to randomly fail, e.g.
when the period start was on Good Friday, because the commission was
created on hour three of Good Friday, instead of the next Monday.
When it hit that case, the test commission is never processed.
Defend against that bug by using the first open of the simulation
parameters which is more guaranteed to be during market hours,
when creating the test commission.
This is in place of fixing the bug in the random parameters function
or making the parameters non-random, which are other potential fixes.
Changes to trading calendar and environments for supporting market
minutes, etc. have made the non-NYSE stock exchange support lag.
Disabling the test, with the intent of bringing support back up to
parity with NYSE.
So that we can more clearly demarcate each case of buy/sell and
price compared to stop, and their expected outputs.
Also, add comment about the current behavior versus the behavior
that will be moved to in an upcoming fix.
Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
These tests use the random simulation parameters, which is leading
to an intermittent failure.
We may want to consider removing the randomness, but in the meantime
the randomness is exposing a case where the cost basis is not the value
expected, so logging the sim parameter values to help track down what
parameters cause the failure.
Eddie Hebert