The call to `Panel.dropna` after the fillna was deleting all values,
if a stock stopped trading mid run and thus provided volume 0.
i.e. if any sid had 0 non-null values the entire panel of frames
would be truncated.
It's possible to avoid the collapse via by adding the `how='all'` flag
to `dropna`, however with the current tick based creation of the panel,
the `dropna` with `how='all'` should be functionally equivalent to
not dropping at all.
The dropna has been dropped in favor of leaving the drop to algorithm
code.
The recent change to the creation of the data panel ended up with
a panel with the dtype of 'object', which was causing numpy ufuncs
like `log` to crash out on an `AttributeError`.
This forces all frames in the panel to use a dtype of 'float',
we may want to look at seeting a dtype on a frame by frame basis,
e.g. 'volume' may more accurately be 'int'.
other details:
- also fixed grammatical errors in loader's status messages.
- converting the treasury curves to an ordered dict.
- moved to using a lambda for clarity as per @ehebert
- initializing calendar end dates to be midnight of current date in
- US/Eastern. Yahoo data isn't available until midnight eastern.
- added LSE reference rrules calendar (thanks to Edward Johns)
- added tests to verify LSE environment matches rrule calendar
- added a test to verify global environment behavior can be set.
- moved DailyReturn class to trading to eliminate circularity from
risk <-> trading.
- updated TradingEnvironment to be a context manager. This allows users
to run algorithms in individually isolated environments in one python
process. This is useful for managing multiple algorithms in a single
ipython notebook.
- added comments to explain behavior and useage of the global environment
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.
Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.
================
Details:
- adding delorean to the requirements.
- made index symbol a parameter for loading the benchmark data. changed
messagepack storage to be symbol specific.
- ported risk, performance, algorithm, transforms, batch transforms
and associated tests to use simulation parameters and global environment
- factory and sim factory use global state and sim params
- factory method parameter names now reflect the class expected
With this patch, on the close of markets we "fast forward" to midnight of the
next trading day and calculate the dividend payments. This patch assumes that
the dividend dates are all at midnight UTC.
For the case where the window isn't covered by the data streaming
through the simulator.
e.g. in a case where the stocks being iterated over change every
quarter, the supplemental data will fill in the 'gap' missing from
the transform since the 'new' stocks were not streaming before
the beginning of the quarter.
Of note, test cases are covered by internal suites, but this could
use tests with completely mocked data.
- Removes New Year's on Saturday, in that case there is no New Year's observed.
- Adds Monday after a July 4th Sunday
- Adds Friday before a July 4th Saturday
- Adds Monday after a Christmas Sunday
- Adds Friday before a Christmas Saturday
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
So that algorithms that specifiy market_aware, days and delta
as args can be transitioned to just specifying a window_length kwarg.
Moving towards removing market_aware and delta completely.
The eventual goal is to remove the market_aware and delta kwargs,
but removing the kwarg completely would break the init
method of EventWindow based classes for existing algorithms.
In the meantime, this ensures that only market_aware is only ever
set to True and raises an error if it is False.
Also raises exceptions if values are set for delta, which was only
used if market_aware was False
Also, since the logic for checking window length is changed,
because in market aware mode we should always be checking the
window length, this adds some sanity checking to the window length.
Thomas Wiecki