In preparation for the incoming changes which no longer push every bar
through the tradesimulation, remove the adjustment of the period's cash on
every pricing change of a held futures asset.
Instead hold the last sale price for each held future either:
- At the end of each peformance period update the last sale prices of
all held futures, so that the pnl for the next period uses values
derived from the cash difference between the end of the two periods.
- When a transaction is processed for the Future, so that the correct
amount is applied to each cash adjustment. (i.e. the cash adjustment
is reset on every change of amount of the Future being held, so that
multiple size and prices do not need to be tracked for the same asset.)
Also, remove now unused dict of payout calculation modifier, since new
calculation reads the value directly off of the asset.
Remove update_last_sale test, since the method no longer returns a cash
value.
Instead of calling a function, where the only parameter is the tracker
object, make it a method, so that the snapshot of position tracker stats
can be more easily called as `pt.stats()`.
In preparation for removal of widespread events, change the split
methods to use params for sid and cost, instead of an event, for
compatibility with lazy branch.
co-author: @jbredeche <jean@quantopian.com>
In preparation for removal of widespread events, change the commission
methods to use params for sid and cost, instead of an event, for
compatibility with lazy branch.
co-author: @jbredeche <jean@quantopian.com>
Refer to cumulative and todays performance explicitly instead of always
looping through.
The third value (minute) for which this was useful, has been removed.
Also, there are some actions where only cumulative may need application,
e.g. application of dividends. (However, this patch does not remove
dividend processing from todays performance, but opens up later patches
to make that distinction.)
EarningsCalendar loader.
- Moves most of AdjustedArray back into Python. The window iterator is
the only part that's performance-intensive.
- Adds a bootleg templating system for creating specialized versions of
AdjustedArrayWindow for each concrete type we care about.
- Adds support for differently dtyped terms in pipeline. This allows us
to use datetime64s which are needed in the EarningsCalendar.
- Adds EarningsCalendar dataset for the next and previous earnings
announcements in pipeline.
- Adds in memory loader for EarningsCalendar.
- Adds blaze loader for EarningsCalendar.
Previously we were not accounting for cases where we would invoke
next_market_minute() with a time on a trading day *before* the
market open, or previous_market_minute() with a time on a trading
day *after* the market close.
This is an optimization where we're building an environment but not
using its finder. Ideally, the consumer would use just the calendar,
but it's not fully featured quite yet.
- Combine the net value and exposure functions into `calc_net` since
they use the same logic.
- Change the logic to handle on empty list to using the a start value of
0.0. More concise, and reduces the number of return points from the
function to one.
Instead of having two leverage functions, whose differences were the
parameter names, add a `calc_leverage` function, with the calling code
determining whether it is gross or net by the type of exposure passed in.
Instead of calculating the position values for each stat result, e.g.
gross_exposure, net_liquidity etc.; get the positions upfront and then
calculate the period and position stats in order, passing each value
explicitly to the ones that follow it in the dependency chain.
e.g. the gross_value depends on the long_value and the short_value,
which called the position_values property for calculating both the
long_value and the short_value.
Removing the repeated calls to position_values (and
position_exposures) removes the need for the caching the last sale
prices and position amounts in separate vectors, since it is inexpensive
enough to read those values off of the positions dictionary held in the
position tracker.
This patch gives a small gain to ~500 sized portfolios, but the main
intent is to clear the path to not storing last_sale_prices on the
position objects at all. Removing all of the caching layer in this class
makes that change easier to apply. Removing the extra calls to
position_values also made this class easier to step through/reason about
when splicing in the new last sale price access, as well.
This commit removes the ability to reference a shared TradingEnvironment through the zipline.finance.trading module. In place, the classes that require a TradingEnvironment, or its child AssetFinder, contain their own references to those objects.
This commit also adds serialization utilities that allow for the pickling/unpickling of objects without unintentionally their TradingEnvironments or AssetFinders.
The write_data methods invokes the relevant AssetDBWriter subclass
to write data to the database. update_asset_finder is no longer
a relevant method since the AssetFinder is strictly a reader class.
This patch lays the groundwork for a compute engine designed to
facilitate construction of factor-based universe screening and portfolio
allocation. It contains:
A new module, `zipline.modelling`, containing entities that can be used
to express computations as dependency graphs. Each node in such a graph
is an instance of the base `Term` class, defined in
`zipline.modelling.term`. Dependency graphs are executed by instances
of `FFCEngine`, defined in `zipline.modelling.engine`.
A new module, `zipline.data.ffc`, containing loaders and dataset
definitions for inputs to the modelling API.
New `TradingAlgorithm` api methods: `add_factor`, and `add_filter`.
These methods can only be called from `initialize`, and are used to
inform the algorithm that each day it should compute the given terms.
Computed factor results are made available through a new attribute of
the `data` object in `before_trading_start` and `handle_data`. Computed
filter results control which assets are available in the factor matrix
on each day.
The minutely calculation of risk metrics had been removed with a
previous patch, remove vestigial references.
Remove a test which tested the behavior of updating the second minute of
a day.
Remove the logic that changed the datetime index of the risk metrics
depending on emission rate, now only trading_days are needed.
Remove `returns_frequency` parameter since both minute and daily
data frequency always use daily returns.
Attack the startup bottleneck of creating the asset finders caches for a
large universe, which was between 1-2 seconds on development and
production machines.
Instead, allow the AssetFinder to be passed a sqlite3 file that has
already been populated and then hydrate asset objects only when an
equity is referenced for the first time.
To create aforementioned sqlite3, create an AssetFinder with an db_path
and `create_table` set to True. If `create_table` is set to False, the
prepopulated data in the sqlite file found at db_path will be used.
Default behavior is to use an in memory database.
Behavior that changes:
- Fuzzy lookup now only works on one character, that character needs to be
specified at write/metadata consumption time, since the fuzzy lookup key
is created by dropping the character from each symbol.
- Overwriting partially written metadata is no longer
supported. i.e. some unit tests allowed for inserting just the identifier,
and then later updating the symbol, end_date, etc.
Instead of building an upsert behavior at this time, this patch
changes the unit tests so that the data for each asset is only
inserted once.
Other notes:
- populate_cache is now removed, since there is no longer a two step
process of inserting metadata and then realizing that metadata into
assets. _spawn_asset is rolled into insert_metadata, so that a call to
insert_metadata both converts the metadata and makes it available in
the data store.
jfkirk