6ff1d55504
ENH: Allow arbitrary history queries.
...
In preparation for using `DataPortal` in notebooks, remove restriction on
the `HistoryLoader` to dates that are monotonically increasing. Notebook
usage of the `DataPortal` is more useful when the end of the history
window can be arbitrary dates without having to restart the notebook kernel.
Due to the implementation of the prefetch and caching logic, the end
date of history calls could previously only increase. e.g. `2016-11-01`,
`2016-11-02`, `2016-11-03`. This pattern was sufficient for backtesting
and live simulations, since the current time of the algorithm only ever increases.
With this change, which resets the underlying sliding window when the
last fetched idx is greater than the
Now calls to history in the same process with end dates such
`2016-11-01`, `2016-10-31`, `2015-11-02` should work.
2016-11-07 16:40:51 -05:00
2b925ecb78
Merge pull request #1581 from quantopian/import-style-on-calendars
...
STY: Add space between import and open parens.
2016-11-04 14:08:05 -04:00
676fb9cb89
Merge pull request #1580 from quantopian/research-compatible-history-loader
...
ENH: Allow configurable history prefetch length.
2016-11-04 14:07:33 -04:00
336867df15
STY: Add space between import and open parens.
...
For compliance with newer flake8 versions.
Should be no functional change.
2016-11-04 13:45:19 -04:00
a3df1e3cef
ENH: Allow configurable history prefetch length.
...
To support using a `DataPortal` and `HistoryLoader` in a notebook, allow
the prefetch length to be configurable, so that it can be set to 0.
Unlike backtesting where the prefetch is useful for repeated history
windows viewed from datetimes which are monotonically increasing by a
small amount, the notebook usage of history windows needs only to
retrieve the exact data needed for the window specified.
This patch also fixes some boundary conditions related to rolls and
adjustments which were uncovered by querying for the adjustments with an
end date near the end of the window.
2016-11-04 13:30:30 -04:00
a0e36d492d
PERF: Use ctable.resize to speed up BcolzMinuteBarWriter.truncate ( #1578 )
...
This is significantly faster than the previous approach of writing a new
ctable with a slice of the existing table.
2016-11-04 10:31:41 -04:00
1a13e6172f
Merge pull request #1574 from quantopian/make-corrcoefs-window-safe
...
ENH: Mark statistical terms as window_safe.
2016-11-01 12:49:27 -04:00
2c087ef77b
ENH: Mark statistical terms as window_safe.
2016-11-01 12:16:34 -04:00
2b6fbdc202
Merge pull request #1573 from quantopian/rename-specific-assets
...
MAINT: Rename `SpecificAssets` to `StaticAssets`.
2016-11-01 12:15:10 -04:00
24931bd4dd
MAINT: Rename SpecificAssets to StaticAssets.
...
This better reflects the intended usage of accepting a fixed set of
predetermined assets.
2016-11-01 11:01:36 -04:00
62bb389199
BUG: load extensions in IPython magic ( #1543 )
2016-10-28 16:00:51 -04:00
867b878c9a
Merge pull request #1521 from quantopian/add-optimization-pass-for-initial-workspace
...
Add optimization pass for initial workspace
2016-10-28 15:42:52 -04:00
563a8b34f3
STY: Put 0 at the end. ( #1569 )
2016-10-28 15:14:22 -04:00
ad7ddd17ea
STY: cleanup
2016-10-28 15:04:18 -04:00
66a214fae9
ENH: Allow windows of NumericalExpression.
...
If all the inputs to an expr are window-safe, then the expr is itself
window-safe.
2016-10-28 15:04:18 -04:00
0f57dac4ab
TEST: Add more populate_initial_workspace tests.
...
- Tests different pipeline lengths and window lengths.
- Tests a term that depends on a window of a term that's been
precomputed.
2016-10-28 15:04:18 -04:00
7f40f7a99d
STY: remove unused imports and method, clean up docs
2016-10-28 15:04:18 -04:00
d5b901df3f
TST: add tests for postprocess and to_workspace_value
2016-10-28 15:04:18 -04:00
9822ee5783
TST: add more dispatches
2016-10-28 15:04:18 -04:00
0eb461bd16
ENH: cleanup branch based on feedback
2016-10-28 15:04:18 -04:00
92516e6609
ENH: prune the graph based on the initial workspace
2016-10-28 15:04:18 -04:00
499680553f
ENH: update aliasedmixin repr
2016-10-28 15:04:18 -04:00
f9ce7f060e
ENH: Make aliases filters, factors, and classifiers to give them their methods
2016-10-28 15:04:18 -04:00
7fc2588571
WIP: Add Alias expression.
2016-10-28 15:04:18 -04:00
4f9c6bd254
ENH: provide a hook for prepopulating the initial workspace
2016-10-28 15:04:18 -04:00
892b7cca00
ENH: Provide methods to undo the 'postprocess' step in computing pipeline output
2016-10-28 15:04:18 -04:00
9f10bbaf94
Merge pull request #1568 from quantopian/fix-microoptimizations
...
Fix microoptimizations
2016-10-28 14:57:43 -04:00
e89410dc30
MAINT: Consolidate data_portal names.
...
Rename _get_daily_window_for_sids to _get_daily_window_data.
Rename _get_minute_window_for_assets to _get_minute_window_data.
Rename _get_daily_data to get_daily_spot_value.
2016-10-28 14:35:05 -04:00
8ccdb48c40
DOC: Comment on outdated code.
2016-10-28 14:06:35 -04:00
b714a23b44
BUG: Raise SidsNotFound in retrieve_asset.
2016-10-28 14:05:49 -04:00
7e505926b6
Merge pull request #1567 from bernoullio/master
...
MAINT: Restore @property decorator
2016-10-28 12:10:58 -04:00
914dca1685
MAINT: Restore @property decorator
...
This will keep `opens`, `closes`, `early_closes`, etc to the
same pattern.
2016-10-28 23:25:53 +08:00
ac74a9dff5
Merge pull request #1561 from quantopian/micro-optimizations-2
...
Micro optimizations 2
2016-10-28 10:36:33 -04:00
e93087e452
Merge pull request #1565 from quantopian/fix-offset-history
...
BUG: Fix continuous future history with offsets.
2016-10-28 09:44:34 -04:00
8876092d29
BUG: Protect against contract offset at end of range. ( #1564 )
...
This boundary case was exposed with internal fixture data which used a
continuous future with a contract chain of size one.
2016-10-27 16:48:34 -04:00
c25da8f442
BUG: Fix continuous future history with offsets.
...
Apply offset value when writing out the rolls in a continuous future
which is offset from the primary.
2016-10-27 16:23:03 -04:00
e07d5a202a
Merge pull request #1563 from quantopian/use-same-session-for-contract-closes
...
BUG: Use proxy for settlement on future adjustments.
2016-10-27 13:31:12 -04:00
e1bafe1ecc
BUG: Use proxy for settlement on future adjustments.
...
Instead of using the difference between the session close of the front
contract before the roll and and the open of back contract on the
beginning of the roll, use the close of both at the end of the session
before the roll.
The closes of the session prior to roll is in lieu of settlement data.
2016-10-27 12:40:59 -04:00
48c725b5ea
PERF: Call concatenate directly instead of hstack.
...
Avoids a couple function calls in a hot path.
2016-10-26 23:49:48 -04:00
0cbc2ca388
PERF: Don't round until after we hstack.
2016-10-26 23:30:12 -04:00
1e889987eb
MAINT/PERF: Remove redundant method call.
...
`_get_minute_window_data` was just forwarding its input to a method with
the same signature.
2016-10-26 23:28:34 -04:00
d18080553b
PERF: Pull out loop-invariant code.
...
This shaves off 20 out of 160 seconds for an algorithm that makes a
large number of large universe, short window_length `history()` calls.
2016-10-26 23:27:33 -04:00
16e3cb50cc
PERF: Use vectorized assignment into dataframe.
...
This is a dramatic speedup (~25% in local benchmarks) for history calls
with a large number of assets and a short window length.
2016-10-26 21:10:40 -04:00
0fbf7b428b
DEV delete old *.c and *.so files with rebuild-cython.sh for Darwin/OSX ( #1560 )
2016-10-26 19:08:36 -04:00
a252bb1e3f
PERF: Refactor AdjustedArrayWindow.
...
Make `__next__` and `seek` share code instead of seek() calling
`__next__`. This avoids having to make a large number of integer
comparisons and `asanyarray` calls when seeking more than one tick
forward.
2016-10-26 17:32:27 -04:00
57a0822b60
BUG: Return NaT instead of None in daily reader.
2016-10-26 17:32:27 -04:00
52b71af848
PERF: Vectorize assignments in get_history_window.
2016-10-26 17:32:27 -04:00
fc153999e2
PERF: Remove attribute access in inner loop.
2016-10-26 17:32:27 -04:00
a1f771c1fa
MAINT: Auto-rebuild templated cython files.
2016-10-26 16:44:07 -04:00
eb2b10554b
PERF: Try cache on scalar asset lookups.
...
This provides a 15% speedup for an algo that calls `data.current` with
1000 every minute.
2016-10-26 15:22:28 -04:00
Eddie Hebert