Instead of having separate ExchangeCalendar and TradingSchedule objects, we
now just have TradingCalendar. The TradingCalendar keeps track of each
session (defined as a contiguous set of minutes between an open and a close).
It's also responsible for handling the grouping logic of any given minute
to its containing session, or the next/previous session if it's not a market
minute for the given calendar.
Refactor AlgorithmSimulator so that DAY_END is emitted for both
minute and daily emission, and that handling of end-of-minute
and end-of-day are separated
This commit removes the ability to reference a shared TradingEnvironment through the zipline.finance.trading module. In place, the classes that require a TradingEnvironment, or its child AssetFinder, contain their own references to those objects.
This commit also adds serialization utilities that allow for the pickling/unpickling of objects without unintentionally their TradingEnvironments or AssetFinders.
Move the responsibility of wrapping the main simulation loop in the
ZiplineAPI context from the algorithm modules generator setup to the
main trade simulation loop, so that different methods of invoking loop
do not need to duplicate how the context is set.
To make it easier for internal implementation of handle_data to
transition off of calling the ZiplineAPI every bar, to only invoking the
context once per simulation.
Make the ordering in which processing of event types both explicit and
independent of the sort ordering of the incoming sources.
The overhead of creating the list per snapshot and the iterators appears
to be marginal in the minute data case when tested locally.
This patch is intended as part of the path towards making the trade
simulation loop not depend on consuming and tracking every trade event.
The timing of where last_sale_date was needed to be changed was proving
difficult to adapt in the previous model.
Should also allow the removal of sorting of the various source streams.
By having both the trade simulation main loop route events to "process"
methods based on event type and the process methods also checking event
type, there was some duplicated effort in doing that comparison many
times.
A particular case where this was noted in profiling was for the
`process_event` function which was checking if the type was not a trade
and returning early, when in a larger universe of stocks the value
returned False 99% of the time.
Instead provide separate process functions specific to each type,
e.g. e.g. `process_trade` and `process_transaction` and route traffic to
those functions in tradesimulation.
For a universe of 160 stocks on both no-op algo and an algo that rebuys
its universe every day, saw about a 10% increase locally.
Also:
- Add process_benchmark to blotter since internal subclass relies on
logic on benchmark, this allows the internal process_trade to be a
`pass`.
- Add warning on unrecoginzed event types.
Remove pieces that are no longer used now that the simple transforms are
wrappers around history via the SIDData object.
Move window length related pieces into batch_transform, since the rest
of the utils module is no longer used.
Reduce overhead of the attribute access of grabbing the process_event
and process_trade from both the algorithm object and the perf_tracekr or
blotter, by assigning those functions to a variable once per snapshot.
schedule_function takes a date rule, a time rule, and a function and
will call the function, passing context and data only when the two rules
fire. This allows for code that is conditional to the datetime of the
algo.
This is implemented internally with `Event` objects which are pairings
of `EventRule`s and callbacks.
handle_data becomes a special event with a rule that always fires. This
makes the logic for handling events more complete and compact.
We guarantee that `TradingAlgorithm.updated_portfolio` is called at least once
between each perf message emission. This change moves that guaranteed call
into `AlgorithmSimulator.get_message` and adds a comment to make it clear why
updated_portfolio is being called.
The function that handles a market close for daily frequency changed from
`handle_market_close` to `handle_market_close_daily`.
The function that is called at on the closing minute each day when running
minutely changed from `handle_intraday_close` to
`handle_intraday_market_close`.
Breaks out the main snapshot processing loop into its own function, and does
some minor variable renaming-shuffling.
Adds `TradingAlgorithm.on_dt_changed`, a function to be called when the
simulation dt changes, prior to processing any events.
There should be no difference in behavior as a result of this change.
In situations where the performance tracker has been reset or patched
to handle state juggling with warming up live data, the `market_close`
member of the performance tracker could end up out of sync with the
current algo time as determined by the
The symptom was dividends never triggering, because the end of day
checks would not match the current time.
Fix by having the tradesimulation loop be responsible, in minute/minute
mode, for advancing the market close and passing that value to the
performance tracker, instead of having the market close advanced by
the performance tracker as well.
Adding a copy of the Event's dt field as datetime via the
`alias_dt` generator, so that the API was forgiving and allowed
both datetime and dt on a SIDData object, was creating noticeable
overhead, even on an noop algorithms.
Instead of incurring the cost of copying the datetime value and
assigning it to the Event object on every event that is passed
through the system, add a property to SIDData which acts as an
alias `datetime` to `dt`.
Eventually support for `data['foo'].datetime` may be removed,
and could be considered deprecated.
To prevent cases where custom data types had unaligned timestamps,
only call handle_data when market data passes through.
Custom data that comes before market data will still update
the data bar. But the handling of that data will only be done
when there is actionable market data.
The next day calculation was causing an error when a minute
emission algorithm reached the end of available data.
Instead of a generic exception when available data is reached,
raise and catch a named exception so that the tradesimulation loop
can skip over, since the next market close is not needed at the end.
When running with minutely emissions the simulator would report to the
user that it simulated 'n - 1' days (where n is the number of days
specified in the simulation params). Now the correct number of trading
days are reported as being simulated.
Jean Bredeche