* ENH: Adds support for supplementary asset mappings
- Adds a supplementary_mappings table to the assets.db, to hold point-
in-time mappings of sids to arbitrary categories of values, e.g.
alternative identifiers. This bumps ASSET_DB_VERSION.
- Adds supplementary_map and supplementary_map_by_sid to AssetFinder,
caches of the underlying table that are fully populated on first
access, which map the supplementary values to sids and vice versa,
respectively.
- Adds lookup_by_supplementary_mapping method, which fronts
supplementary_map to query for the asset last known to have held a
value at a given dt.
- Add get_supplementary_field method, which fronts
supplementary_map_by_sid to query for the last known value held by an
asset at a given dt.
For futures that behave like GC, use the latest roll as the back contract when
walking backwards over the window, so that when the front contract is skipped
because it never has more volume between its auto close date and the previous
auto close date, the back contract which did have volume is still used when
making comparisons to construct the chain.
Instead of maintaining a separate index into the sessions index, use the `.freq`
member of the sessions index for decrementing to the current session and finding
the previous session.
When following the release guide, installing from testpypi using the
`-i` flag failed on my, and at least one other's, development machines.
The cause of the failure appears to be that pip would look for packages,
such as `LogBook` or `pandas` on `testpypi`. However many dependencies
do not have versions that meet our version criteria. (e.g. pandas does
not have a version between 0.16.0 and 0.18.0 on testpypi.)
Instead, use `--extra-index-url` so that other packages can use `pypi`
as a fallback server, instead of being limited to `testpypi`.
To support contracts such as `PL` which should roll from F->J->N->V, add the
ability to pass a predicate function to the ordered contract chain contstrution
which returns `True` if the contract is allowed in the chain.
Convert the end minute to the its session label before calling `_active_contract`,
otherwise the volume roll finder's attempt to use the session bar reader fails
due to a non-session label Timestamp.
Fix multiple errors when attempting to generate rolls for futures which do not
roll month to month, e.g. the Eurodollar.
These errors were caused by logic that always incremented from contract to
contract by delivery month, with errors when the next contract was not part of
the quarterly roll chain and thus had not yet begun trading even though the
previous contract had autoclosed. Instead, filter out these contracts and only
allow contracts that have begun trading before the previous contract's autoclose.
This is in lieu of a more explicit specification of quarterly rolls.
Instead of requiring the roll finder to juggle the indices into the ordered
contracts, use a doubly linked list where the nodes element is the contract
with members pointing to the previous and next contracts in the chain.
Besides improving legibility in the roll finder code, this change is on the path
to adding a predicate to exclude contracts from the chain, e.g. contracts in ED
which are not in the roll schedule.
Change test results for primary chain, since new implementaton does not stop at
contract in which has not yet started when constructing the chain.
- Use `expect_bounded` to check inputs.
- Add tests for expected failures from `MACDSignal`.
- Use `float64` instead of `float` in a few places. This prevents
diverging behavior on 32-bit systems.
- Docstring edits.
- Use a RandomState with a seed so that we have repeatible results.
- Use `randint` instead of `random_integers.` `random_integers` is
deprecated.
- Use `parameter_space` to test multiple period lengths.
Previously, if input to the BcolzMinuteBarWriter had the first bar on a
non-trading minute, the next trading session would be considered the
"first day" in the input. Now, we consider the previous trading session
the "first day".
The intention is to correctly associate minutes after official trading
hours on half days with session that closed early, not the following
session (a future improvement here would be to not accept minutes
outside trading hours).
Fix common error condition which was triggered whenever the session at the end
of the prefetched history window was a session where the back contract was
active. When the back contract was the active contract, the next contract for
consideration was the front contract at the end of the window, which
definitionally always has an autoclose after the end of the window.
Instead, just start seeking backwards from the end of the window.
Also prevent lookahead bias in volume rolls, which was caused by the using the
volume for a session to determine whether that session had rolled. Information
that would not have been available at the beginning of the session.
This change makes the volume rolls overly conservative, and may be improved by
looking at vectors of the preceding volume and making the roll off of momentum.
Any DataFrame that's had `.loc` or `.iloc `called on it participates in
a cycle, which means they're not immediately garbage collected when they
go out of scope. This matters for pipeline results because they consume
multiple megabytes per column, which means that a pipeline result with
many columns can hold take up over 100MB. By manually breaking
DataFrame cycles, we can ensure that we never hold multiple pipeline
results in memory at once.
Otherwise, we either raise an exception or filter out all unsafe values.
This addresses an issue where the BcolzMinuteBarWriter would scale up
values to convert to uint32, but the resulting values were too large,
and would be mangled.
Based on the approach we take in the BcolzDailyBarWriter.
This modificaiton to the estimates loader allows the caller to pass
in an equity pricing loader which can then be used to get split data
for sids. That split data is then used to do point-in-time adjustments
of estimates data.
TST: add test for multiple estimates columns
TST: add test for multiple datasets requesting different columns
TST: add blaze versions for all next/previous tests
Richard Frank