Instead of using division of the amount by itself to extract
the direction, uses math's copysign.
Should be almost functionally equivalent,
but copysign won't have a possible floating point error leading
the direction to not be exactly 1.
So that both computational and memory overhead is reduced,
this turns off serializing positions for cumulative performance.
Positions were essentially being doubled up by being stored
in both cumalative and daily.
So that transactions are kept by default.
This prepares for the addition of the serialize flag added by
@fawce.
Setting the default to True, so that the flags will be aligned.
- repeated calls with the same data window do not update batch transform
windows.
- repeated calls with the same data and same supplemental parameters do
not update batch transform results
- repeated calls with the same data and different supplemental params
do update batch transform results
If the trading_days end date is not greater than the date being
tested, (this can happen if the algorithm's end date is set to a date
that is before the latest date available saved in the msgpacks),
then trying to get the location will fail, instead searchsorted
will get the lastest date available in the trading day map to use
as a test date.
- removed use_panel
- default for refresh_period is now 0
- refresh_period will only affect the recreation of the datapanel
- user's transform method is invoked on every call to batch transform
The loader_tool currently facilitates:
- Finding the last date in each source
- Forcing a complete reload of benchmark and treasury.
- Manipulating the msgpacks so that trailing dates are dropped,
so that the condition for an update can be recreated.
This is intended to make it easier to test the update logic added
by @rday in PR #88
- the underlying dequeue shouldn't be modified, so forwarding to the
user function is a bit misleading
- if we want to provide a dequeue we should consider another class
or an EventWindow decorator.
Uses a method called 'record' that provides a key value,
instead of providing keys to extract from context.
The variables are stored internally to the algorithm in a dictionary,
and not just stared as a property of the algorithm.
Main intent behind this change is to make the API more user friendly,
since the previous recorded_variables relies on the value to be set
in the algorithms context/self, the hope is that only having to use
the `record` method means less moving pieces and a more understandable
API.
i.e., instead of:
```
def initialize(self):
recorded_variables('foo', bar')
def handle_data(self, data):
self.foo = 1
self.bar = 2
```
The API is now:
```
def initialize(self):
pass
def handle_data(self, data):
self.record(foo=1, bar=2)
```
Adds a check to see if the s_squared value is near 0.
When the number was very near 0, a very small negative floating
point, the sqrt throws a 'math domain error', this prevents that
case.
The call to `Panel.dropna` after the fillna was deleting all values,
if a stock stopped trading mid run and thus provided volume 0.
i.e. if any sid had 0 non-null values the entire panel of frames
would be truncated.
It's possible to avoid the collapse via by adding the `how='all'` flag
to `dropna`, however with the current tick based creation of the panel,
the `dropna` with `how='all'` should be functionally equivalent to
not dropping at all.
The dropna has been dropped in favor of leaving the drop to algorithm
code.
The recent change to the creation of the data panel ended up with
a panel with the dtype of 'object', which was causing numpy ufuncs
like `log` to crash out on an `AttributeError`.
This forces all frames in the panel to use a dtype of 'float',
we may want to look at seeting a dtype on a frame by frame basis,
e.g. 'volume' may more accurately be 'int'.
other details:
- also fixed grammatical errors in loader's status messages.
- converting the treasury curves to an ordered dict.
- moved to using a lambda for clarity as per @ehebert
- initializing calendar end dates to be midnight of current date in
- US/Eastern. Yahoo data isn't available until midnight eastern.
- added LSE reference rrules calendar (thanks to Edward Johns)
- added tests to verify LSE environment matches rrule calendar
- added a test to verify global environment behavior can be set.
- moved DailyReturn class to trading to eliminate circularity from
risk <-> trading.
- updated TradingEnvironment to be a context manager. This allows users
to run algorithms in individually isolated environments in one python
process. This is useful for managing multiple algorithms in a single
ipython notebook.
- added comments to explain behavior and useage of the global environment
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.
Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.
================
Details:
- adding delorean to the requirements.
- made index symbol a parameter for loading the benchmark data. changed
messagepack storage to be symbol specific.
- ported risk, performance, algorithm, transforms, batch transforms
and associated tests to use simulation parameters and global environment
- factory and sim factory use global state and sim params
- factory method parameter names now reflect the class expected
With this patch, on the close of markets we "fast forward" to midnight of the
next trading day and calculate the dividend payments. This patch assumes that
the dividend dates are all at midnight UTC.
For the case where the window isn't covered by the data streaming
through the simulator.
e.g. in a case where the stocks being iterated over change every
quarter, the supplemental data will fill in the 'gap' missing from
the transform since the 'new' stocks were not streaming before
the beginning of the quarter.
Of note, test cases are covered by internal suites, but this could
use tests with completely mocked data.
- Removes New Year's on Saturday, in that case there is no New Year's observed.
- Adds Monday after a July 4th Sunday
- Adds Friday before a July 4th Saturday
- Adds Monday after a Christmas Sunday
- Adds Friday before a Christmas Saturday
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
Eddie Hebert