For scaling up pricing data before writing to bcolz, the writer now
accepts a dict mapping each sid to the ratio to use. It still accepts a
single ratio as default_ohlc_ratio, which is used as a fallback if no
mapping exists for a given sid. The default is OHLC_RATIO (1000).
This allows better handling of futures pricing data, where the required
precision across root symbols is not consistent.
The daily/session bar reader's `spot_price` took the same parameters and
returned the same kind of output as the minute bar reader's `get_value`.
Standardize on one method to make a common interface, which may be
formally factored out in a later patch; to help enable writing reader
implementations or mixins which can be agnostic to the bar frequency.
We were mistakenly using the minute_per_day field.
We now expose from the metadata object the version from which the
metadata was read. This allows a new test that verifies the version is
read correctly.
In the data portal, remove methods that make a distinction between
future and equity asset type. Instead rely on the pricing reader
dispatching.
In support of incoming work which will upsample equity history arrays to
the larger future calendar.
Also, remove perf tracker tests which were using an equity
reader/writer, to be added back in later.
* First pass.
* Improvements and fixes
- Update usages of BcolzMinuteBarWriter
- Updates with rebuilt example data
- Expose calendar from BcolzMinuteBarMetadata instead of calendar_name
- Keep market_opens and market_closes in metadata for compatibility
* Store start_session and end_session in minute bcolz metadata
- start_session replaces first_trading_day
- Add end_session to limit to correct days
* For last_available_dt, get last close from calendar to maintain tz
* Bumps version and handles earlier versionson read
* Rebuilt example data on python 3
* Indicate metadata fields that are deprecated
Implement a `SessionBarReader` which uses a minute bar reader as a
backing source, resampling the minute bars into the box around the
corresponding session data.
Also, add future/CME test cases to resample suite.
Replace `DailyBarReader` with `SessionBarReader`.
This was intended to go with the patch that added the `SessionBarReader`
abstract base class.
Also, added `trading_calendar` property decorator.
* MAINT: Use TradingCalendar objects for bundles
Instead of trading days, opens, and closes, register now takes a
TradingCalendar object, along with a start_session and end_session. The
ingest function is now passed these values instead as well.
* Accept calendar name in addition to the actual object
* Updates bundles documentation for changes
* Fix typo in docs
* Use class formatting
* Force start_session and end_session within the bounds of the calendar
* Use UTC timestamps in test_core
* Document Trading Calendar API in appendix.rst
In preparation for making a resampling session bar reader, make an
abstract base class with the methods currently used by consumers of the
reader; which should assist in making a drop-in replacement of the daily
bar reader.
While pulling out the interface, it does appear that `spot_price` and
the minute bar reader's `get_value` are the same method but by different
names, showing that there may be room for having both `MinuteBarReader`
and `SessionBarReader` sharing a common `BarReader` interface.
Also, move `DailyHistoryAggregator` to `resample` module, so that tools
for converting from minute to session bars are collocated.
This patch is in preparation of adding a daily bar reader which
resamples minute data, which will be located in the `resample` module
and share the test cases and expected results in `test_resample`.
Use the future asset equity pricing reader, instead of reading directly
from the bcolz table. Required since the format for writing the future
data now uses the minute bar reader/writer pair.
Add test cases to `test_data_portal` asserting both equity and future
`get_spot_value` results.
Also, add direct coverage of last_traded_dt in the `test_data_portal`
module.
Prepares for adding test coverage of `get_last_traded_dt` for `Future` assets.
Change the mock minute data to no longer use an increasing arange, so
that a days worth of minute data can be summed and fit inside of a
uint32.
This change was required because of working on new test data that looked
like [0, 100, 200, 0, ] which was resulting in a daily rollup of 0 data,
when the coverage needed a non-0 value.
Also, factor out the resampling function, with an eye on a making it
easier to convert from minute bars to daily bars during ingest/load
processes.
TradingAlgorithm.run didn't support Panel minute bar data, and assumed
all Panel data was daily.
To rectify this, adding PanelMinuteBarReader class.
TradingAlgorithm.run decides whether to use it or PanelDailyBarReader
by assuming data is daily if and only if the time of day of every
Timestamp is identical.
Set the version to 1, with a fallback of version of 0 if version is not
found.
If the version is 0, use 390 (US Equities) for the number of minutes in
the day, so that existing files work as before.
Adds a minutes_per_day field to BcolzMinuteBarMetadata, so that the
minutes_per_day value passed to the BcolzMinuteBarWriter is written as a
part of the minute bar metadata. This allows the reader to share this
value, instead of always assuming 390.
Also rebuilds the example data to incorporate this format change.
Richard Frank