Commit Graph

3328 Commits

Author SHA1 Message Date
Richard Frank ede1eb7aa0 PERF: Look up expired futures from in-memory Futures
instead of queries to the db.
2016-02-04 18:55:34 -05:00
Joe Jevnik f5b74bf7c5 Merge pull request #980 from quantopian/odo-kwargs
ENH: allow users to specify odo kwargs in from_blaze
2016-02-03 17:28:22 -05:00
Joe Jevnik 7c8a44ecd7 ENH: allow users to specify odo kwargs in from_blaze 2016-02-03 16:16:06 -05:00
Eddie Hebert 7173e9a5c8 Merge pull request #979 from quantopian/allow-cols
ENH: Allow passing of numpy arrays to writer.
2016-02-02 14:58:06 -05:00
Eddie Hebert 27f94f83fa ENH: Allow passing of numpy arrays to writer.
For faster parsing and writing workflows, do not require a DataFrame.
2016-02-02 14:03:42 -05:00
Richard Frank d2365750fd Merge pull request #977 from quantopian/asset_comparison_overflow
BUG: Handle OverflowError by comparing False
2016-02-01 15:48:24 -05:00
Eddie Hebert 9c51daad47 Merge pull request #978 from quantopian/allow-empty-writes
ENH: Add padding method to minute bars writer.
2016-02-01 15:22:10 -05:00
Richard Frank 28b34f6409 BUG: Handle OverflowError by comparing False
instead of raising
2016-02-01 15:02:15 -05:00
Eddie Hebert 488721e805 ENH: Add padding method to minute bars writer.
So that consumers can write empty days worth of data, without needing
to construct a DataFrame with zero data force a write.

The internal loader uses `last_date_in_output_for_sid` to signify that
data has been attempted to be retrieved for all dates up until that, so
that when resuming a job those retrieval of data for those dates are not
re-attempted.

Also, used to make the write logic cleaneer, by making it only
necessary to create an array large enough for the given df.
2016-02-01 14:18:22 -05:00
Joe Jevnik b102742f69 BUG: fix deltas in blaze core loader 2016-01-28 14:08:11 -05:00
Joe Jevnik dd64a6cb73 Merge pull request #974 from jlowin/patch-4
Replace print with logger.info
2016-01-25 19:43:19 -05:00
Jeremiah Lowin e44c0d42e1 Replace print with logger.info 2016-01-25 19:22:28 -05:00
Eddie Hebert 6557553c39 Merge pull request #973 from quantopian/use-method-for-metadata
MAINT: Use metadata method for reader init.
2016-01-25 13:10:26 -05:00
Eddie Hebert 930aa1b29b MAINT: Use metadata method for reader init.
Use the preexisting metadata method when instantiating the minute bar
reader.

An internal sublcass uses the `_get_metadata` method to setup data for
directories that have not used the new writer/reader interface.
(i.e. allows for reader creation when the metadata.json file does not
exist.)
2016-01-25 12:58:43 -05:00
James Kirk 2803802736 Merge pull request #941 from quantopian/futures-tick-size
ENH: Adds tick_size and renames futures multiplier
2016-01-25 11:18:31 -05:00
Eddie Hebert e576231ae5 Merge pull request #972 from quantopian/fix-minute-bar-siblings
BUG: Fix OSError when creating sids that share dir
2016-01-25 10:41:42 -05:00
Eddie Hebert 984e934e83 BUG: Fix OSError when creating sids that share dir
Fix a bug where creating a sid bcolz file when the containing directory
was already occupied by a sid caused an OSError on attempt of creating
the directory because it already existed.

e.g. if there were two sids, `1` and `2`. The paths would be
`00/00/000001.bcolz` and `00/00/000002.bcolz` which share the same
directory `00/00`.

Fixed by checking for directory existence before calling `makedirs`.

Add test coverage which exercises writing of sids that are siblings in
the sid directory structure.
2016-01-25 10:37:50 -05:00
Jonathan Kamens 0c2a1a1654 MAINT: Upgrade sphinx to 1.3.4 in dev requirements. 2016-01-23 19:13:58 -05:00
Joe Jevnik f28982a7e5 BUG: do a notnull or notnan check based on the schema 2016-01-23 15:27:42 -05:00
Joe Jevnik dca5d347d8 BUG: lookup_expired_futures needs to coerce dts into pd.Timestamp 2016-01-23 15:27:21 -05:00
jfkirk f15245924d DOC: Whatsnew entry for assets db downgrade 2016-01-22 15:48:31 -05:00
jfkirk 5d0ffd0db4 DOC: SQLLite -> SQLite 2016-01-22 15:31:41 -05:00
jfkirk d82bd5c05d TST: Cleans up test_to_and_from_dict 2016-01-22 15:26:14 -05:00
jfkirk c9c2e3e60e TST: Updates TestPositionPerformance 2016-01-22 14:56:30 -05:00
jfkirk ece9e59ef9 ENH: Adds asset db downgrade management and tests 2016-01-22 14:56:30 -05:00
jfkirk db1e62971a ENH: Adds tick_size and renames futures multiplier 2016-01-22 14:56:30 -05:00
Joe Jevnik 110798b0be MAINT: update blaze commit again 2016-01-22 14:21:25 -05:00
Richard Frank dc95dea5d2 BLD: Only build pushes to master
now that we've turned on Build pushes
2016-01-22 08:40:46 -05:00
Joe Jevnik 88ead7202e Merge pull request #965 from quantopian/cython-stuff
PERF: make Asset comparison faster
2016-01-21 23:51:10 -05:00
Joe Jevnik 663fbd7d74 PERF: make Asset comparison faster 2016-01-21 19:11:41 -05:00
Joe Jevnik dea6af3eee Merge pull request #810 from quantopian/ffill
Blaze loader forward fill fix
2016-01-21 13:57:21 -05:00
Joe Jevnik 0f43b6b371 MAINT: fix blaze commit 2016-01-21 13:30:16 -05:00
Richard Frank 888038f796 Merge pull request #964 from quantopian/secure_snake
Secure snake
2016-01-21 13:29:06 -05:00
Joe Jevnik 9adef37373 BUG: Fix the case where multiple values happen on the same day 2016-01-21 12:43:02 -05:00
Joe Jevnik 33db0e330e BLD: update blaze 2016-01-21 12:43:02 -05:00
llllllllll 5e4b8b8f8f TST: tests for new forward fill 2016-01-21 12:43:02 -05:00
llllllllll 517ad7a5f1 TST: Adds tests for running multiple columns in one query 2016-01-21 12:43:02 -05:00
llllllllll 97298d1ad4 ENH: upgrade ffill logic to look back as far as needed 2016-01-21 12:43:02 -05:00
Richard Frank 4ee919aeb2 Merge pull request #959 from quantopian/asset_keyerror
Asset KeyError
2016-01-21 12:38:53 -05:00
Richard Frank 5b9eb04e9e BLD: Remove conditional by specifying prefix for miniconda
Don't need quotes around flake8 command
2016-01-21 12:36:25 -05:00
Richard Frank 8968d2850f BLD: Use https to download miniconda 2016-01-21 12:28:49 -05:00
Richard Frank 46a47d407c DOC: Added whatsnew 2016-01-21 11:47:32 -05:00
Richard Frank 2eeb6e78f6 BUG: Enable comparison of an Asset to an int64 on 32-bit python
We use a number of mappings keyed by int64, which otherwise raised
KeyErrors for Assets.
2016-01-21 11:47:29 -05:00
James Kirk 2cb3a7235e Merge pull request #890 from grundgruen/future_chain_order
BUG: orders future chain by min of notice and expiry
2016-01-21 11:18:57 -05:00
Eddie Hebert 603a345e2d BUG: Allow writing on first day. 2016-01-21 11:10:57 -05:00
Eddie Hebert 3a8be8c624 BUG: Need to be able to append to minute ctable. 2016-01-21 11:10:07 -05:00
Eddie Hebert 85be1b0a40 Merge pull request #963 from quantopian/minute-writer-for-one-per-sid-2
ENH: Add writer for minute bcolz format.
2016-01-21 10:59:38 -05:00
Eddie Hebert d5c3b5a15c ENH: Add writer for minute bcolz format.
Implement a writer for minute data into a format comprised of multiple
ctables, one for each individual asset, with a common 'index' shared by
all ctables where a given a dt maps to the same array index for all
equities and fields.

This format is pulled from the lazy-mainline/Q2.0 branch, with some
changes to the interface.

Add basic retrieval of values at a given dt to reader. Not yet used by
Zipline simulations, but added to support unit tests.

Also, rename stubbed out us_equity_minutes to minute_bars, since the
writer can be agnostic to asset type.
2016-01-21 10:54:27 -05:00
Richard Frank daf05c6b59 BUG: Ensure that current_sids() returns Assets instead of identifiers
Also batch lookup sids in algo.run
2016-01-21 10:32:07 -05:00
Joe Jevnik ab573731d9 BLD: make the travis build closer to a normal install 2016-01-20 18:14:07 -05:00