Adding a copy of the Event's dt field as datetime via the
`alias_dt` generator, so that the API was forgiving and allowed
both datetime and dt on a SIDData object, was creating noticeable
overhead, even on an noop algorithms.
Instead of incurring the cost of copying the datetime value and
assigning it to the Event object on every event that is passed
through the system, add a property to SIDData which acts as an
alias `datetime` to `dt`.
Eventually support for `data['foo'].datetime` may be removed,
and could be considered deprecated.
A symbol() lookup feature was added to Quantopian.
By adding the same API function to zipline we can
make copy&pasting of a zipline algo to Quantopian
easier.
The input into max drawdown was incorrect, causing the bad results.
i.e. the `compounded_log_returns` were not values representative of
the algorithms total return at a given time, though
`calculate_max_drawdown` was treating the values as if they were.
Instead, use the `algorithm_period_returns` series, which does provide
the total return.
Update risk answer key with an Excel calculation of max drawdown
to help corroborate the calculations.
Also, remove `compounded_log_returns`, (which actually had stopped
being the `compounded_log_returns` at some point), since the max
drawdown was the only calculation using the values in that series.
To prevent cases where custom data types had unaligned timestamps,
only call handle_data when market data passes through.
Custom data that comes before market data will still update
the data bar. But the handling of that data will only be done
when there is actionable market data.
When running with minutely emissions the simulator would report to the
user that it simulated 'n - 1' days (where n is the number of days
specified in the simulation params). Now the correct number of trading
days are reported as being simulated.
Previously benchmark events were emitted at 0:00 on the day the
benchmark related to: in 'minute' emission mode this meant that
the benchmarks were emitted before any intra-day trades were
processed.
See: https://github.com/quantopian/zipline/issues/241
This is a step towards the goal of uniting Quantopian scripts
and zipline.
To make the syntax of zipline identical to Quantopian
we break out the API methods (like order) and turn them into
functions. To access the algo object we add a thread local reference
to the current algorithm that is accessed in the API functions.
TradingAlgorithm now takes either a string or two functions
(initialize and handle_data) that it executes.
Use api method decorator for methods available in algoscript.
Ported appropriate algorithm tests from internal code.
To help prevent algorithms from operating on positions that are
not in the existing universe of stocks.
Formerly, iterating over positions would return positions for stocks
which had zero shares held. (Where an explicit check in algorithm
code for `pos.amount != 0` could prevent from using a non-existent
position.)
Use date sorted sources instead, instead of sorting with second
argument of Event, etc. since the `heapq.merge` behavior is using
the second part of the tuple, thus requiring a richer set of comparison
methods, which would only be used in the test context.
Use `date_sorted_sources` instead, so that sorting is done on algo time
and source id.
Python 3 removes the `.message` attribute, so use `str` instead.
Also, the divide by zero message has changed slightly between versions,
so just check for the exception type, instead of also checking the message.
The rename of walk is not provided by six, so check the import error
via an exception.
Also, callback behavior slightly changes between the two versions,
so instead iterate over the walked files and call what was formerly
a callback, directly as a function.
Python 3 uses the `__next__` method instead of `next`,
and uses the syntax of `next(foo)` accordingly.
Add `__next__` and `next` side-by-side so both Python 2 and 3 have
a method that can be used during iteration.
Python 3 requires submodules to have more explicit pathing, so use
the dot syntax to declare submodules which are in the same directory
as another module.
Use the six module to import functions and types that are
consistent between Python 2 and 3, so that one code base can
support both versions.
- Use integer types instead of int and long.
- Use string_types instead of basestring.
- Account for iteritems, itervalues, iterkeys.
- Use six.moves for filter and zip, reduce
- Use compatible bytes for md5 hasher.
- xrange and range
- Use `print()` function for all print calls
- Fix strip and format calls that were on the outside of the
print function for some reason.
(Which were breaking in Python 3 because of print returning None.)
- Remove commented out print calls.
Note that the calendar test is decorated with @nottest (as per the other calendar test functions). I've run the test to confirm the calendar works. The differences between the env (Yahoo Finance of GSPTSE) and the calendar are illustrated in the tradingcalendar_tse file and are confirmed to be errors on Yahoo Finance's part.
A bug in the create_random_simulation_parameters allows the period
start to be a non-trading day.
That bug was causing the commission tests to randomly fail, e.g.
when the period start was on Good Friday, because the commission was
created on hour three of Good Friday, instead of the next Monday.
When it hit that case, the test commission is never processed.
Defend against that bug by using the first open of the simulation
parameters which is more guaranteed to be during market hours,
when creating the test commission.
This is in place of fixing the bug in the random parameters function
or making the parameters non-random, which are other potential fixes.
Changes to trading calendar and environments for supporting market
minutes, etc. have made the non-NYSE stock exchange support lag.
Disabling the test, with the intent of bringing support back up to
parity with NYSE.
So that we can more clearly demarcate each case of buy/sell and
price compared to stop, and their expected outputs.
Also, add comment about the current behavior versus the behavior
that will be moved to in an upcoming fix.
Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
These tests use the random simulation parameters, which is leading
to an intermittent failure.
We may want to consider removing the randomness, but in the meantime
the randomness is exposing a case where the cost basis is not the value
expected, so logging the sim parameter values to help track down what
parameters cause the failure.
So that the units match the other risk calculations, also
use annualized returns for beat and alpha.
Update answer key to match values calculated on the first day.
Also, update performance tracker test so that the returns used
are fractional instead of > 1, so that the annualized numbers are
more in line with real world values.
This could perhaps be labelled BUG, as well.
Change the Sharpe (and algorithm volatiilty) value used to compare
algorithms/backtests so that it is annualized and uses daily returns.
Previously, the Sharpe metric was using the same calculation style
as the fixed size periods, i.e. 3 Month, 6 Month, etc., which can
use the geometric mean when comparing against the risk free.
Change the Sharpe calculation to use the arithmetic mean differenc
against the risk free rate, using daily (non-compounded) values.
Also, use annualized mean returns.
Most of the functions in date_utils can be done via pandas.
The other functions are no longer used for loading, etc. so remove
the date_utils module to reduce the total surface area of Zipline core.
Continue on path of converting values stored inside of risk metrics
to use a DataFrame instead of storing multiple lists.
Also, the need for latest_dt in getting the current volatility for
the sharpe calculation, shows that we need to set the lastest_dt at
the beginning of the update loop.
Richard Frank