is backwards-compatible with the previous format.
In USEquityLoader, use dailyreader's trading_calendar.
This is backwards compatible and will fall back to the NYSE calendar if
the reader doesn’t have a calendar specified.
Instead of having separate ExchangeCalendar and TradingSchedule objects, we
now just have TradingCalendar. The TradingCalendar keeps track of each
session (defined as a contiguous set of minutes between an open and a close).
It's also responsible for handling the grouping logic of any given minute
to its containing session, or the next/previous session if it's not a market
minute for the given calendar.
In preparation of adding futures, add equity to the names of both the
classes and methods for writing bcolz data. Futures data will use a
different minutes per day with a separate reader. This change will allow
both equity and futures fixtures to be side by side.
Also, break out the method which generates the dataframes and trading
days member into fixtures (`EquityMinuteBarData` and
`EquityDailyBarData`) on which the `*BarReader` fixture depends. This
fixture is separated out to enable reader/writers in different formats
to use the same data setup. (There is internal code which needs to write
minute and daily bar data in a database format.)
Querying for the price field of an equity actually looks at the close
field, so we should do the same for futures. Otherwise `data.can_trade`
and `data.current` of 'price' fail for futures.
Since the first trading day is now passed directly to the DataPortal on
init, there's no need for a method that does this. Moves all the
additional logic/assignments into the init. Also corrects an issue where
we would never create certain attributes if self._first_trading_day was
None.
Adds the ability to specify the first trading day for a data portal in a
test case when using the WithDataPortal fixture.
Fix behavior in minute mode history with frequency `1d`, where on the
day immediately following an adjustment action, the overnight adjustment
would not apply. (However the adjustment would be applied after a 1 day
lag.)
The root cause of the bug was that the history data for minute mode when
using `1d` stitches together a sliding window of the daily data for
previous and the current minute. That daily data sliding window and
corresponding adjustments was being read as if the data was being viewed
from on the last day of the window; however in this case the data is
being viewed from the day after the window has completed. The difference
in view points requires the adjustments to popped and applied by the
adjusted array one index earlier. The fix uses the `extra_slot` value as
signifier on whether the data is being viewed on the following day and
then accordingly adjusts the index of the mulitpy object.
Also, change the split and merger test data ratios to have different values,
to ensure that different adjustment values are applied; as opposed to
doubling up on just one of the values.
When writing first_trading_day, it is already in the correct frame of
reference (seconds since epoch) and does not need to be transformed
further. Adjusts the reader to expect this value.
Instead of inferring it from the minute/daily writer, we now require the
first trading day to be passed explicitly, so the creator of the
DataPortal controls what is used as the first trading day.
If minutes already exist for the last existing day, adjust the number of
minutes padded to account for them. Previously we would always pad 390,
leading to a mismatch in the number of rows.
Minutely data can now be appended to bcolz files even when
minutes in the same day have already been written. For example,
previously attempting to write data for the minute 2016-05-11 16:30
would raise an exception if any OHLCV data for 2016-05-11 had been
written to the same file.
Trying to overwrite existing minutes still raises a
BcolzMinuteOverlappingData exception.
Note that previously all sids' bcolz files ended at the same time.
This is no longer necessarily the case. The last record in each
sid's bcolz file now corresponds to the latest minute for which
OHLCV data is provided to the writer.
- Return a value from `verify_all_indices_unique` so that `panel` isn't
unconditionally `None` in `PanelDailyBarReader`.
- Fix a bug where we always set the volume of every asset to `1e9`.
- Add minimal suite of tests for get_spot_value, which catch both of the
above.
NOTE: There are still several issues with `PanelDailyBarReader`. The
docstring for `get_spot_value` claims that it will return -1 on days
where an asset didn't trade, which isn't the case. It also claims that
it will raise `NoDataOnDate` when a request is made outside the panel
range, but it just raises a KeyError. We also still have no coverage
for `load_raw_arrays`, so it's likely that there are more bugs lurking.
We are now using isoformats with ':' replaced with ';'. We cannot use a
normal isoformat because windows does not allow files or directories
with ':' in the name.
This data bundle will use the quantopian mirror of the quandl WIKI data
instead of downloading from quandl directly. This dramatically improves
the speed because we do not pay the rate limiting for quandl and we can
send the data in the format zipline expects.
Jean Bredeche