and added a new test case
was not iterating over lookup date directory names, and
therefore mising all by one list of stocks.
discovered because of differing sort orders between
my local machine, other devs, and travis ci.
The underlying RollingPanel in batch_transform was always accumulating
all values to ever appear in data.
However, at any given algo time the desired return value is what the
current active sids are.
Instead, mask down to the sids that are passed in as the data parameter.
So that with minute data, 2.5 orders of magnitude of data can
be cut, allowing for longer window_lenghts, when the daily
values are what are desired for a signal.
Both risk and performance now calculate performance since inception
(cumulative) and since the open. Both periods are updated intraday
and both are reported.
Batch risk for periods starting after the end of the treasury curve
history now use most recent curve.
Work towards running an algorithm against 'live' data, which can't
be bound to the available benchmarks and treasuries, since the
benchmarks and treasury curves for that day won't be published
until that night.
Instead of having the performance tracker as part of the
tradesimulation class, hold on to it inside of the algorithm
object, so that the perf_tracker is more easily accessed for
reset behavior, etc.
- moved Order and Blotter to zipline.finance.blotter
- moved order method from AlgoSimulator to Blotter
- eliminated the set_order method in algorithm
- moved blotter to the algorithm
Refactor PerformanceTracker, Blotter, and AlgorithmSimulator to
work with handling the end of a bar at the AlgorithmSimulator level
instead of within PerformanceTracker.
- PerforamnceTracker and Blotter are longer generators,
both provide functions to process events instead.
- AlgorithmSimulator calls each from within the loop running
over the data generator.
- Change test_perf_tracker utility to be compatible with change
away from PerformanceTracker as a generator.
Has the effect of:
- Fixing the timing of order emission.
- Allow minutely emission of benchmarks, which was prevented
by the extra grouping previously caused by Blotter.
Minutely emission also depends on work for streaming benchmarks
through performance and risk at a minute granularity.
To fix the grouping of events so that (dt, events) ordering
is preserved, the tracking of order states needs to change
in the following way.
Change how order keeps track of dates:
- Change order's dt field to reflect modified date.
- Add a created field.
Change how performance keeps track of orders by:
- Map dt to transactions
- Map dt to orders
- Map order ids to keep track of updated orders.
The emission of order updates from the blotter were incorrect,
and subsequently, performance.
Previously, only the first action of the order was emitted,
fix so that all status updates are emitted.
So that a filter function can filter out sids on a 'static' source,
when the universe of sids changes.
e.g. when using fetch_csv with a changing universe of stocks.
- perf modified to let non-performance related events flow through.
- changes to support streaming non-trading data through batch transforms
and for mixing in sids with just custom data.
- allowing CUSTOM events to flow through to transforms.
- Added logic to maintain pre-specified sid filter.
So that both computational and memory overhead is reduced,
this turns off serializing positions for cumulative performance.
Positions were essentially being doubled up by being stored
in both cumalative and daily.
- repeated calls with the same data window do not update batch transform
windows.
- repeated calls with the same data and same supplemental parameters do
not update batch transform results
- repeated calls with the same data and different supplemental params
do update batch transform results
- removed use_panel
- default for refresh_period is now 0
- refresh_period will only affect the recreation of the datapanel
- user's transform method is invoked on every call to batch transform
- the underlying dequeue shouldn't be modified, so forwarding to the
user function is a bit misleading
- if we want to provide a dequeue we should consider another class
or an EventWindow decorator.
other details:
- also fixed grammatical errors in loader's status messages.
- converting the treasury curves to an ordered dict.
- moved to using a lambda for clarity as per @ehebert
- initializing calendar end dates to be midnight of current date in
- US/Eastern. Yahoo data isn't available until midnight eastern.
- added LSE reference rrules calendar (thanks to Edward Johns)
- added tests to verify LSE environment matches rrule calendar
- added a test to verify global environment behavior can be set.
- moved DailyReturn class to trading to eliminate circularity from
risk <-> trading.
- updated TradingEnvironment to be a context manager. This allows users
to run algorithms in individually isolated environments in one python
process. This is useful for managing multiple algorithms in a single
ipython notebook.
- added comments to explain behavior and useage of the global environment
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.
Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.
================
Details:
- adding delorean to the requirements.
- made index symbol a parameter for loading the benchmark data. changed
messagepack storage to be symbol specific.
- ported risk, performance, algorithm, transforms, batch transforms
and associated tests to use simulation parameters and global environment
- factory and sim factory use global state and sim params
- factory method parameter names now reflect the class expected
With this patch, on the close of markets we "fast forward" to midnight of the
next trading day and calculate the dividend payments. This patch assumes that
the dividend dates are all at midnight UTC.
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
fawce