Commit Graph

4682 Commits

Author SHA1 Message Date
Joe Jevnik bae4088cc0 ENH: cleanup branch based on feedback 2016-10-28 15:04:18 -04:00
Joe Jevnik 0123bb8a97 ENH: prune the graph based on the initial workspace 2016-10-28 15:04:18 -04:00
Joe Jevnik e3e4ad2735 ENH: update aliasedmixin repr 2016-10-28 15:04:18 -04:00
Joe Jevnik c8e40a3736 ENH: Make aliases filters, factors, and classifiers to give them their methods 2016-10-28 15:04:18 -04:00
Scott Sanderson 13c8139d45 WIP: Add Alias expression. 2016-10-28 15:04:18 -04:00
Joe Jevnik 67b35168db ENH: provide a hook for prepopulating the initial workspace 2016-10-28 15:04:18 -04:00
Joe Jevnik a1d4ceae54 ENH: Provide methods to undo the 'postprocess' step in computing pipeline output 2016-10-28 15:04:18 -04:00
Scott Sanderson 5ecb5c3754 Merge pull request #1568 from quantopian/fix-microoptimizations
Fix microoptimizations
2016-10-28 14:57:43 -04:00
Scott Sanderson 21a3f1a7ed MAINT: Consolidate data_portal names.
Rename _get_daily_window_for_sids to _get_daily_window_data.
Rename _get_minute_window_for_assets to _get_minute_window_data.
Rename _get_daily_data to get_daily_spot_value.
2016-10-28 14:35:05 -04:00
Scott Sanderson 8ccef7b9ab DOC: Comment on outdated code. 2016-10-28 14:06:35 -04:00
Scott Sanderson 1fbc17d281 BUG: Raise SidsNotFound in retrieve_asset. 2016-10-28 14:05:49 -04:00
Richard Frank 2c819a672b Merge pull request #1567 from bernoullio/master
MAINT: Restore @property decorator
2016-10-28 12:10:58 -04:00
keang 265c02b0c1 MAINT: Restore @property decorator
This will keep `opens`, `closes`, `early_closes`, etc to the
same pattern.
2016-10-28 23:25:53 +08:00
Scott Sanderson 285b5f7d77 Merge pull request #1561 from quantopian/micro-optimizations-2
Micro optimizations 2
2016-10-28 10:36:33 -04:00
Eddie Hebert 9a51efc7d2 Merge pull request #1565 from quantopian/fix-offset-history
BUG: Fix continuous future history with offsets.
2016-10-28 09:44:34 -04:00
Eddie Hebert 575a8cf048 BUG: Protect against contract offset at end of range. (#1564)
This boundary case was exposed with internal fixture data which used a
continuous future with a contract chain of size one.
2016-10-27 16:48:34 -04:00
Eddie Hebert 4235dbd758 BUG: Fix continuous future history with offsets.
Apply offset value when writing out the rolls in a continuous future
which is offset from the primary.
2016-10-27 16:23:03 -04:00
Eddie Hebert 43ca435e1a Merge pull request #1563 from quantopian/use-same-session-for-contract-closes
BUG: Use proxy for settlement on future adjustments.
2016-10-27 13:31:12 -04:00
Eddie Hebert aa021531d9 BUG: Use proxy for settlement on future adjustments.
Instead of using the difference between the session close of the front
contract before the roll and and the open of back contract on the
beginning of the roll, use the close of both at the end of the session
before the roll.

The closes of the session prior to roll is in lieu of settlement data.
2016-10-27 12:40:59 -04:00
Scott Sanderson 70cc6022f9 PERF: Call concatenate directly instead of hstack.
Avoids a couple function calls in a hot path.
2016-10-26 23:49:48 -04:00
Scott Sanderson 9e3eebc892 PERF: Don't round until after we hstack. 2016-10-26 23:30:12 -04:00
Scott Sanderson 25c78de356 MAINT/PERF: Remove redundant method call.
`_get_minute_window_data` was just forwarding its input to a method with
the same signature.
2016-10-26 23:28:34 -04:00
Scott Sanderson 5547cca779 PERF: Pull out loop-invariant code.
This shaves off 20 out of 160 seconds for an algorithm that makes a
large number of large universe, short window_length `history()` calls.
2016-10-26 23:27:33 -04:00
Scott Sanderson 9d10e28d09 PERF: Use vectorized assignment into dataframe.
This is a dramatic speedup (~25% in local benchmarks) for history calls
with a large number of assets and a short window length.
2016-10-26 21:10:40 -04:00
Paul Sutherland e7ca08038a DEV delete old *.c and *.so files with rebuild-cython.sh for Darwin/OSX (#1560) 2016-10-26 19:08:36 -04:00
Scott Sanderson 502fbf5357 PERF: Refactor AdjustedArrayWindow.
Make `__next__` and `seek` share code instead of seek() calling
`__next__`.  This avoids having to make a large number of integer
comparisons and `asanyarray` calls when seeking more than one tick
forward.
2016-10-26 17:32:27 -04:00
Scott Sanderson 85fcf0ba9b BUG: Return NaT instead of None in daily reader. 2016-10-26 17:32:27 -04:00
Scott Sanderson 20531fb9c1 PERF: Vectorize assignments in get_history_window. 2016-10-26 17:32:27 -04:00
Scott Sanderson 73cc580e0b PERF: Remove attribute access in inner loop. 2016-10-26 17:32:27 -04:00
Scott Sanderson 35a72f63a4 MAINT: Auto-rebuild templated cython files. 2016-10-26 16:44:07 -04:00
Scott Sanderson a56fc707ed PERF: Try cache on scalar asset lookups.
This provides a 15% speedup for an algo that calls `data.current` with
1000 every minute.
2016-10-26 15:22:28 -04:00
Eddie Hebert df2a091937 Merge pull request #1558 from quantopian/add-detail-to-history-calendar-mismatch
MAINT: Add more info to history calendar KeyError.
2016-10-26 15:12:08 -04:00
Eddie Hebert cd36e3702f MAINT: Add more info to history calendar KeyError.
There have been cases where the requested start or end date is not in
the history calendar.

Add the beginning and of the calendar to the KeyError to give more
detail to figure out root cause.
2016-10-26 14:41:37 -04:00
Eddie Hebert f1929342d9 Merge pull request #1556 from quantopian/volume-based-rolls
ENH: Volume based rolls for futures.
2016-10-25 15:21:41 -04:00
Eddie Hebert afbe3cdcd7 ENH: Volume based rolls for futures.
Add roll style which takes the volume of the contracts into account.
If the volume moves from the front to the back before the auto close
date, the roll is put at that session.

Also, factors out some of the common logic shared with calendar based rolls.
2016-10-25 14:08:21 -04:00
Eddie Hebert 321afc1626 Merge pull request #1555 from quantopian/return-nan-from-bcolz-daily-reader
MAINT: Return nan from daily bcolz get_value.
2016-10-25 11:49:51 -04:00
Eddie Hebert 098d38ac76 MAINT: Return nan from daily bcolz get_value.
Match the behavior of the minute bar reader, now that the session and
minute bar readers share a common interface.

isnull is slightly slower than checking against -1; however, n cases
where we check against illiquid trades in a tight loop, volume is
checked which is not using nan. The change here should be marginal with
regards to performance.
2016-10-25 11:25:09 -04:00
Eddie Hebert 0c81ee4af6 Merge pull request #1553 from quantopian/change-start-date-of-cf-tests
TST: Make test dates aligned with test data.
2016-10-25 11:16:07 -04:00
Eddie Hebert 3c7df3e997 TST: Make test dates aligned with test data.
Move dates queried near beginning of test data so that the range of data
covered does not extend beyond the beginning of the range.
i.e. the windows were covering 2016-01-25, which had no test data generated.
(Does not matter for the calendar based rolls, but is needed for volume
based rolls.)

Also, make room for having the first roll to be a day before the first auto
close by moving the first contracts auto close date back a day.

In preparation for testing volume rolls.
2016-10-25 10:21:23 -04:00
Eddie Hebert fe90cd3177 Merge pull request #1552 from quantopian/fix-session-last-traded
BUG: Fix session from minute reader's last traded.
2016-10-24 14:21:15 -04:00
Eddie Hebert fccbae25ed BUG: Fix session from minute reader's last traded.
The last traded dt provided from the session bar reader which resamples
from minutes should provide a dt that is a session label, not one that
is at the minute frequency.
2016-10-24 13:58:58 -04:00
Eddie Hebert 55a2f33052 Merge pull request #1551 from quantopian/prevent-occlusion-of-key-error-in-history
MAINT: Prevent hiding of KeyError in adjustments.
2016-10-24 12:05:22 -04:00
Eddie Hebert 9a08272262 MAINT: Prevent hiding of KeyError in adjustments.
If a KeyError occurred in the adjustment logic, the exception would be
swallowed by the try block, which was intended to just check whether or
not there was an adjustment reader adjusted.

Discovered when some logic in a futures adjustment reader were failing
because of a mismatch of minute and session labels, which resulted in no
adjustments during windows when there should have been.
2016-10-24 11:33:00 -04:00
Eddie Hebert 506832e63b Merge pull request #1549 from quantopian/speedup-resample
PERF: Speedup minute to session sampling.
2016-10-24 10:25:51 -04:00
Eddie Hebert a4205a0500 PERF: Speedup minute to session sampling.
The minute to session sampling reading was creating two DataFrame
objects, the first to hold the minute data, and then a second returned
by the `DataFrame.groupby` to sample down to sessions.

Instead use the arrays returned by the minute readers `load_raw_arrays`
and implement sampling logic which takes advantage that the minutes
being passed start with the first minute of the first session and end
with the last minute of the last session.

On my machine this takes the tests in `test/test_continuous_futures`
from ~4.0 to about ~0.1 seconds.
2016-10-24 09:59:22 -04:00
Maya Tydykov 54ebd9e362 Merge pull request #1547 from quantopian/fix-asset-selection-bug
BUG: create adjustments based on ordered assets instead of set
2016-10-24 08:56:19 -04:00
Maya Tydykov e1f008edcd TST: update adjustment tests - add gaps between sids
TST: add a seed for permuting
2016-10-21 16:53:56 -04:00
Maya Tydykov adf533d037 BUG: pass the entire list of assets 2016-10-21 16:23:38 -04:00
Maya Tydykov 99448bd122 BUG: create adjustments based on ordered assets instead of set 2016-10-21 16:23:38 -04:00
Eddie Hebert e68b77273c Merge pull request #1548 from quantopian/continuous-future-adjusted-history
ENH: Add adjusted history for continuous futures.
2016-10-21 11:22:54 -04:00