ENH: Allow arbitrary history queries.

In preparation for using `DataPortal` in notebooks, remove restriction on
the `HistoryLoader` to dates that are monotonically increasing. Notebook
usage of the `DataPortal` is more useful when the end of the history
window can be arbitrary dates without having to restart the notebook kernel.

Due to the implementation of the prefetch and caching logic, the end
date of history calls could previously only increase. e.g. `2016-11-01`,
`2016-11-02`, `2016-11-03`. This pattern was sufficient for backtesting
and live simulations, since the current time of the algorithm only ever increases.

With this change, which resets the underlying sliding window when the
last fetched idx is greater than the

Now calls to history in the same process with end dates such
`2016-11-01`, `2016-10-31`, `2015-11-02` should work.
This commit is contained in:
Eddie Hebert
2016-11-07 16:40:51 -05:00
parent 2b925ecb78
commit 6ff1d55504
2 changed files with 73 additions and 6 deletions
+15 -6
View File
@@ -379,12 +379,26 @@ class HistoryLoader(with_metaclass(ABCMeta)):
assets = self._asset_finder.retrieve_all(assets)
try:
end_ix = self._calendar.get_loc(end)
except KeyError:
raise KeyError("{0} not in calendar [{1}...{2}]".format(
end, self._calendar[0], self._calendar[-1]))
for asset in assets:
try:
asset_windows[asset] = self._window_blocks[field].get(
window = self._window_blocks[field].get(
(asset, size, is_perspective_after), end)
except KeyError:
needed_assets.append(asset)
else:
if end_ix < window.most_recent_ix:
# Window needs reset. Requested end index occurs before the
# end index from the previous history call for this window.
# Grab new window instead of rewinding adjustments.
needed_assets.append(asset)
else:
asset_windows[asset] = window
if needed_assets:
start = dts[0]
@@ -395,11 +409,6 @@ class HistoryLoader(with_metaclass(ABCMeta)):
except KeyError:
raise KeyError("{0} not in calendar [{1}...{2}]".format(
start, self._calendar[0], self._calendar[-1]))
try:
end_ix = self._calendar.get_loc(end)
except KeyError:
raise KeyError("{0} not in calendar [{1}...{2}]".format(
end, self._calendar[0], self._calendar[-1]))
cal = self._calendar
prefetch_end_ix = min(end_ix + self._prefetch_length, len(cal) - 1)
prefetch_end = cal[prefetch_end_ix]