Commit Graph
153 Commits
Author SHA1 Message Date
Jean Bredeche f48f88861d BUG: chunk sqlite queries into groups of 999. 2015-11-20 13:39:46 -05:00
Scott Sanderson 1524944edd DOC: Many docs improvements.
- Generate links to sourcecode via the Sphinx `viewcode` extension.
- Generate reference docs for Asset/Equity/Future, AssetFinder, and
  AssetDBWriter.
- Generate reference docs for Pipeline API classes.
- Fix broken links and formatting issues in the 0.8.4 whatsnew.
- Use embedsignature in _assets.pyx so that the signatures of Asset
  subclasses are inspectable.
2015-11-19 00:15:17 -05:00
Scott Sanderson 06ab766544 MAINT: De-dupe shared filtering logic.
We filter for assets that were alive at some reference date in multiple
places.
2015-11-13 18:26:54 -05:00
Scott Sanderson a3bafd8dc7 MAINT/STY: Use dict.get instead of in check. 2015-11-13 18:26:54 -05:00
Scott Sanderson 26611b5d60 MAINT: Don't double-coerce to Timestamp.
`normalize` would already fail if the input isn't a Timestamp.
2015-11-13 18:26:54 -05:00
Scott Sanderson b9e4b62374 PERF: Fewer conversions in _convert_row_to_equity. 2015-11-13 18:26:54 -05:00
Scott Sanderson 657a132f1e ENH: Make retrieve specific type functions public.
We rely on these upstream, for better or worse, so add tests and docs.

Also adds distinct `EquitiesNotFound` and `FutureContractsNotFound`
exceptions.
2015-11-13 18:26:54 -05:00
Scott Sanderson a684063f95 DOC: Don't lie about caching in _retrieve_asset. 2015-11-13 18:26:54 -05:00
Scott Sanderson 9101636bf6 MAINT: Remove unused method. 2015-11-13 18:26:54 -05:00
Scott Sanderson 60b988a777 MAINT: Remove unused method. 2015-11-13 18:26:54 -05:00
Scott Sanderson 1c52977964 DOC: group_by_type returns a regular dict. 2015-11-13 18:26:54 -05:00
Scott Sanderson 654edaa851 BUG: Clear asset caches when mapping DataFrame.
Our DataFrame index resolution logic relies on failed lookups **not**
being cached, but not caching failed lookups is a nontrivial performance
hit when repeatedly looking up sids.  The "solution" here is to clear
the caches after writing in new assets.

The real fix for this is either:

1. Don't construct an AssetFinder until we have the datasource in hand
   in run(), or
2. Don't symbol-map the user's input source if it's a DataFrame.
   Instead we should make our data loaders pre-map the data.
2015-11-13 18:26:54 -05:00
Scott Sanderson 5bddf32062 BUG: Raise if we fail to symbol map a Frame/Panel.
Previously, we just warned here, but then we'd fail immediately trying
to write the found values into an index that's too long.
2015-11-13 18:26:54 -05:00
Scott Sanderson ff51510d88 PERF: Cache the result of failed lookups.
Otherwise we'll keep trying to look them up.
2015-11-13 18:26:54 -05:00
Scott Sanderson c818463051 MAINT: Simplify conditional. 2015-11-13 18:26:54 -05:00
jfkirk e1e160a128 MAINT: Factors-out asset db table schemas for cleanliness 2015-11-12 14:43:11 -05:00
jfkirk 85dd4b70dd MAINT: Renames 'version' table to 'version_info' for clarity 2015-11-12 14:43:11 -05:00
jfkirk 6aac54544e MAINT: Only write asset db version on db creation 2015-11-12 14:43:11 -05:00
jfkirk 48e488d423 ENH: Adds versions to asset databases 2015-11-12 14:43:11 -05:00
Maya Tydykov df492ecf7f DOC: add whatsnew entry for AssetFinderCachedEquities. 2015-11-12 11:02:30 -05:00
Maya Tydykov d0cb5bd513 ENH: add extension to AssetFinder that caches all equities in memory and uses that cache in lookup_symbol. 2015-11-12 11:02:26 -05:00
jfkirk af021f0db5 BUG: FutureChain's as_of() now properly coerces arguments 2015-11-10 14:03:19 -05:00
Scott Sanderson 90e717f6a7 Merge pull request #821 from quantopian/cme-codes-reference
ENH: Add CME code lookup table.
2015-11-05 14:19:36 -05:00
Scott Sanderson 744a998b9a ENH: Add CME code lookup table. 2015-11-05 13:57:44 -05:00
Scott Sanderson a14c61e7ff MAINT: Remove unused 'asset_type' metadata entry. 2015-11-05 13:35:05 -05:00
Richard Frank 1377d25c95 BUG: Ensure that we view the datetime64s in the index as 64bit
Using int with 32bit numpy produces an array of 2n ints, each
representing half a datetime.
2015-11-04 08:31:18 -05:00
jfkirk 40399fc743 MAINT: Removes unused allow_sid_assignment arg 2015-11-03 14:14:06 -05:00
jfkirk 915c8e800f MAINT: Removes unneeded knowledge_date logic from future_chain 2015-10-19 11:37:56 -04:00
Stewart Douglas 4e2039c9b0 ENH: Coerce user input with API method decorator
Previously we have capitalized input strings at different levels in
our code: in the user-facing API methods and in the asset finder.
This commit moves input string capitalization exclusively to the API
method to which the string was supplied. Specifically, the string is
capitalized by a preprocess API method decorator. The preprocess
decorator passes the input string to the newly defined
ensure_upper_case() method, which returns a TypeError if the argument
supplied is not a string.

ensure_upper_case() is defined in a new file, zipline/utils/input_validation.py.
The existing expect_types() method is also moved there.

Various tests in tests/test_assets.py are modified to account for the
fact that the asset finder method lookup_symol() no longer capitalizes
its supplied argument.
2015-10-08 15:41:33 -04:00
Andrew Daniels 69b734129f ENH: Add index to root_symbol column of futures_contracts 2015-10-08 14:51:42 -04:00
Andrew Daniels 9ecd7e18d0 MAINT: Improves handling for contract date info in lookup_future_chain
Improves the query for futures contract to use the date that comes first
in time (between notice_date and expiration_date) to determine cotnract
validity. If one of these is missing, we'll use the other.

Also modifies the query to order the resulting contracts by their
expiration_date if available, and to use their notice_date if not.
2015-10-08 14:16:57 -04:00
Stewart Douglas 3feb78807e MAINT: Add license to various files 2015-10-07 15:47:45 -04:00
Stewart Douglas 3cfdd6cf8c ENH: Add lookup_future_symbol method 2015-10-05 10:25:08 -04:00
Richard Frank 027d4009d8 MAINT: Share constant across modules 2015-10-02 11:53:04 -04:00
Richard Frank c8c83a9f2b MAINT: Don't bother looping through all the keys 2015-10-02 11:53:04 -04:00
Richard Frank e8c5203e31 MAINT: constraints is always True 2015-10-02 11:53:04 -04:00
Richard Frank 8b98ca7d71 PERF: Reflect the db schema once instead of per table 2015-10-02 09:58:59 -04:00
Richard Frank beac4d6f16 MAINT: Delegate to sqlalchemy metadata
instead of maintaining our own
2015-10-02 09:58:59 -04:00
Richard Frank ea02dcfb14 MAINT: Format the list when making error message 2015-10-02 09:58:59 -04:00
Richard Frank 9a745450cc MAINT: De-dupe retrieving assets 2015-10-02 09:58:59 -04:00
Richard Frank 906b68f2da MAINT: Forward along default_none 2015-10-02 09:58:59 -04:00
Richard Frank 21366e8529 PERF: Convert to ns since epoch vector-wise instead of dt by dt
No longer storing NULL, so make columns not nullable. Instead we're storing NaT.
2015-10-02 09:58:58 -04:00
Richard Frank 702cce4fea MAINT: Simplified set and dict manipulations 2015-10-02 09:58:58 -04:00
Richard Frank 3c8370114f PERF: Write in chunks of 999 instead of chunks of 1 2015-10-02 09:58:58 -04:00
Richard Frank 9d61573e52 MAINT: Instance attribute isn't used 2015-10-02 09:58:58 -04:00
Richard Frank 7424c61ada MAINT: De-duplicate AssetDBWriter subclass initialization 2015-10-02 09:58:58 -04:00
Scott Sanderson 2d683961bd MAINT: More renaming.
s/FFCEngine/PipelineEngine/
s/FFCLoader/PipelineLoader/
2015-10-01 18:03:54 -04:00
Scott Sanderson f82a01841b MAINT: Rename ALL the things.
zipline.modelling.* -> zipline.pipeline.*
zipline.data.ffc.loaders -> zipline.pipeline.loaders
tests/modelling -> tests/pipeline
2015-10-01 18:03:53 -04:00
Stewart Douglas e33fa988f2 DEP: Remove _convert_asset_str_fields
_convert_asset_str_fields was added to address the fact that the
unicode returned by SQLAlchemy was causing test failures, in
particular test_repr and test_root_symbols failed because of the
leading 'u' character in the unicode of python-2.x. Later it was
pointed out that this method would behave differently in python 2
and 3, in particular the str() method would return bytes in python
2 and unicode in python 3. After investigating it was found that
the _convert_asset_str_fields was no longer  necessary. It is unclear
why the original tests were failing. It is not due to SQLAlchemy
versions, as tests pass for version >1 and <1.
2015-10-01 15:02:23 -04:00
Warren O'Neill 66b36919a3 BUG: missing comma in _asset_timestamp_fields 2015-10-01 10:57:54 +02:00