test_resample now fully covers the resample module.
Fix a bug exposed by increased coverage, where daily aggregation on
`high` would return `nan` for an asset instead of 1) during the
course of day `1d` history was called on non-consecutive minutes and 2)
either, a) the value for the previously inspected dt was `nan` or b)
there were only `nan`s between the previous and current dt.
`low` had a similar bug which was only triggered if the value for the
previously inspected dt was `nan`.
If subclasses have additional capital change information that
is required to correctly calculate the target values for
cash capital changes, it can now be provided via
"portfolio_value_adjustment".
Increase coverage on `ReindexSessionBarReader` so that all methods which
are considered part of the interface are covered by `test_resample`.
Fix bug in `get_value`, exposed by increased coverage, where the
`NoDataOnDate` exception was bubbling from the bcolz reader all the way
up when a session which was a holidy on the underlying reader was passed
to the reindex reader. (The reindex reader should return nan/0 in that
case.)
Also, move location of data index exceptions so that they are agnostic
to bcolz/us_equity_pricing; since the exception is now used by the
resample module to fix aforementioned bug.
BarData now takes the trading calendar as a parameter.
can_trade now checks if the asset’s exchange is open at the current or
next market minute (defined by the given trading calendar).
Remove special handling for the last session of an asset, which was
moving the last traded back a session.
If the asset has data on a session, `get_last_traded_dt` should always
return that session if it is the parameter to the method.
No longer auto-updates its internal as-of date, instead requires an explicit
as-of date from the consumer.
Take a static list of contracts (instead of needing an assetfinder).
Instead of the as_of method, the user-facing API now lets you pass in an
offset, which is defined as an integral number of sessions.
Add direct coverage on last_available_dt.
Also move reader creation into the instance fixture.
This patch attempted to add coverage on `get_last_traded_dt`, but in doing
so, revealed a bug in `BcolzDailyBarReader.get_last_traded_dt` when
requesting the last trading session of an asset.
When that is fixed, the skip can be removed.
Add a test to directly cover the first_trading_day method via the
`test_resample` suite. (The lack of coverage was exposed when testing
against real data.)
Also, refactor resample bar tests so that session bar reader is set up
in instance fixture.
Combine the equity and future readers into asset dispatch readers, so
that simulations that use both asset types can access data for each.
This patch enables `history` for future assets in algorithms; however,
it does not add extra coverage in the `test_data_portal` or `test_history`
to cover future assets. Those tests will follow, however putting this in
separately since it shows that the wrapping of the readers in the asset
dispatch reader does not break existing equity strategies.
Add `AssetDispatchSessionBarReader` and corresponding minute and session
bar version of that reader.
This reader routes requests to the appropriate reader based on the asset
type of the requested sids.
`load_raw_array` in the dispatch reader batches the sid by asset type
and then interleaves the results in the out arrays, so that the arrays
data corresponds with sids in the order that sids are passed to the
method, to meet the expected behavior of `load_raw_arrays`.
The dispatch redaer is intended for use by the data portal when using
both future and equities. The dispatch reader will also be passed to the
to the `HistoryLoader`s contained within the data portal, where the
batched `load_raw_arrays` will be used.
Also, BUG:
- Fix the return of `MinuteResampleSessionBarReader.load_raw_arrays` to
match all other readers.
- Use the input dt for the `MinuteResampleSessionBarReader.load_raw_arrays`
as a session label, instead of a minute dt, since it is a session bar
reader.
(Both of these bugs where discovered when using the resample reader for
future data in the dispatch tests.)
Working towards history results which contain mixed asset types, add
a reader which makes `load_raw_arrays` return results indexed on the
session/minute ranges specified by the specified `trading_calendar`
instead of the calendar of the backing reader.
This reader will be used to make Equity readers align with Future
readers. It is intended for use as part of another reader (which will
dispatch queries based on asset type and then recombined results) which
will be passed to the `[Minute|Session]HistoryLoaders in the data portal.
The daily/session bar reader's `spot_price` took the same parameters and
returned the same kind of output as the minute bar reader's `get_value`.
Standardize on one method to make a common interface, which may be
formally factored out in a later patch; to help enable writing reader
implementations or mixins which can be agnostic to the bar frequency.
We were mistakenly using the minute_per_day field.
We now expose from the metadata object the version from which the
metadata was read. This allows a new test that verifies the version is
read correctly.
The new TradingCalendar method is called `minute_index_to_session_labels`.
It takes a DatetimeIndex of in-order market minutes and returns a
DatetimeIndex of the corresponding sessions.
The new method is approximately 100x faster than mapping
`minute_to_session_label` over a large DatetimeIndex.
In the data portal, remove methods that make a distinction between
future and equity asset type. Instead rely on the pricing reader
dispatching.
In support of incoming work which will upsample equity history arrays to
the larger future calendar.
Also, remove perf tracker tests which were using an equity
reader/writer, to be added back in later.
* First pass.
* Improvements and fixes
- Update usages of BcolzMinuteBarWriter
- Updates with rebuilt example data
- Expose calendar from BcolzMinuteBarMetadata instead of calendar_name
- Keep market_opens and market_closes in metadata for compatibility
* Store start_session and end_session in minute bcolz metadata
- start_session replaces first_trading_day
- Add end_session to limit to correct days
* For last_available_dt, get last close from calendar to maintain tz
* Bumps version and handles earlier versionson read
* Rebuilt example data on python 3
* Indicate metadata fields that are deprecated
Implement a `SessionBarReader` which uses a minute bar reader as a
backing source, resampling the minute bars into the box around the
corresponding session data.
Also, add future/CME test cases to resample suite.
Ana Ruelas