Also, add direct coverage of last_traded_dt in the `test_data_portal`
module.
Prepares for adding test coverage of `get_last_traded_dt` for `Future` assets.
Change the mock minute data to no longer use an increasing arange, so
that a days worth of minute data can be summed and fit inside of a
uint32.
This change was required because of working on new test data that looked
like [0, 100, 200, 0, ] which was resulting in a daily rollup of 0 data,
when the coverage needed a non-0 value.
Also, factor out the resampling function, with an eye on a making it
easier to convert from minute bars to daily bars during ingest/load
processes.
TradingAlgorithm.run didn't support Panel minute bar data, and assumed
all Panel data was daily.
To rectify this, adding PanelMinuteBarReader class.
TradingAlgorithm.run decides whether to use it or PanelDailyBarReader
by assuming data is daily if and only if the time of day of every
Timestamp is identical.
Set the version to 1, with a fallback of version of 0 if version is not
found.
If the version is 0, use 390 (US Equities) for the number of minutes in
the day, so that existing files work as before.
Adds a minutes_per_day field to BcolzMinuteBarMetadata, so that the
minutes_per_day value passed to the BcolzMinuteBarWriter is written as a
part of the minute bar metadata. This allows the reader to share this
value, instead of always assuming 390.
Also rebuilds the example data to incorporate this format change.
is backwards-compatible with the previous format.
In USEquityLoader, use dailyreader's trading_calendar.
This is backwards compatible and will fall back to the NYSE calendar if
the reader doesn’t have a calendar specified.
Instead of having separate ExchangeCalendar and TradingSchedule objects, we
now just have TradingCalendar. The TradingCalendar keeps track of each
session (defined as a contiguous set of minutes between an open and a close).
It's also responsible for handling the grouping logic of any given minute
to its containing session, or the next/previous session if it's not a market
minute for the given calendar.
In preparation of adding futures, add equity to the names of both the
classes and methods for writing bcolz data. Futures data will use a
different minutes per day with a separate reader. This change will allow
both equity and futures fixtures to be side by side.
Also, break out the method which generates the dataframes and trading
days member into fixtures (`EquityMinuteBarData` and
`EquityDailyBarData`) on which the `*BarReader` fixture depends. This
fixture is separated out to enable reader/writers in different formats
to use the same data setup. (There is internal code which needs to write
minute and daily bar data in a database format.)
Querying for the price field of an equity actually looks at the close
field, so we should do the same for futures. Otherwise `data.can_trade`
and `data.current` of 'price' fail for futures.
Since the first trading day is now passed directly to the DataPortal on
init, there's no need for a method that does this. Moves all the
additional logic/assignments into the init. Also corrects an issue where
we would never create certain attributes if self._first_trading_day was
None.
Adds the ability to specify the first trading day for a data portal in a
test case when using the WithDataPortal fixture.
Fix behavior in minute mode history with frequency `1d`, where on the
day immediately following an adjustment action, the overnight adjustment
would not apply. (However the adjustment would be applied after a 1 day
lag.)
The root cause of the bug was that the history data for minute mode when
using `1d` stitches together a sliding window of the daily data for
previous and the current minute. That daily data sliding window and
corresponding adjustments was being read as if the data was being viewed
from on the last day of the window; however in this case the data is
being viewed from the day after the window has completed. The difference
in view points requires the adjustments to popped and applied by the
adjusted array one index earlier. The fix uses the `extra_slot` value as
signifier on whether the data is being viewed on the following day and
then accordingly adjusts the index of the mulitpy object.
Also, change the split and merger test data ratios to have different values,
to ensure that different adjustment values are applied; as opposed to
doubling up on just one of the values.
When writing first_trading_day, it is already in the correct frame of
reference (seconds since epoch) and does not need to be transformed
further. Adjusts the reader to expect this value.
Instead of inferring it from the minute/daily writer, we now require the
first trading day to be passed explicitly, so the creator of the
DataPortal controls what is used as the first trading day.
If minutes already exist for the last existing day, adjust the number of
minutes padded to account for them. Previously we would always pad 390,
leading to a mismatch in the number of rows.
Eddie Hebert