Commit Graph
167 Commits
Author SHA1 Message Date
dmichalowicz 1ec0bced6d ENH: Add builtin factors for correlation and regression 2016-05-18 15:11:12 -04:00
Joe Jevnik 784d5f4a16 Merge pull request #1199 from quantopian/boybands-factor
BollingerBands factor
2016-05-13 15:35:10 -04:00
Scott Sanderson f4d96e065a TEST/PERF: Don't slice twice. 2016-05-13 14:44:26 -04:00
Scott Sanderson c4b69d6223 TEST: Don't mask unexpected exceptions from TALIB.
We know when we expect the error to be raised.
2016-05-13 14:32:21 -04:00
Scott Sanderson 2f90665676 TEST: Add a test for bbands output ordering. 2016-05-13 14:31:58 -04:00
Scott Sanderson cbd4ea36bc STY: No need for these to be vertical. 2016-05-13 14:31:33 -04:00
Joe Jevnik c128b69a91 STY: get Scott to stop yelling at me 2016-05-12 22:15:32 -04:00
Joe Jevnik 78db90a858 STY: flake8 2016-05-12 17:01:17 -04:00
Joe Jevnik a345e6f3f5 TST: Clean up metaclass usage in fixtures 2016-05-12 17:00:51 -04:00
Maya Tydykov 6b60e447a0 MAINT: incorporate string support
STY: remove unused imports

MAINT: change dtype to object for compatibility with python3

MAINT: rename pipeline columns and constants for clarity

MAINT: rename column
2016-05-12 10:50:31 -04:00
Joe Jevnik 2297ace20c ENH: Adds BollingerBands factor. 2016-05-11 21:41:55 -04:00
Joe Jevnik f494d6f0d1 BUG: Fix check that pipeline argument is hashable.
Adds test coverage for the caes where it is not hashable.
2016-05-11 21:37:12 -04:00
Maya Tydykov 3d521561f5 TST: update tests to handle new buyback auth design
MAINT: add back cash amount constant

BUG: fix field names

BUG: pass remaining args

WIP: make buyback units parameterized so that user can choose

BUG: fix filtering based on units parameter

WIP: test for undesired units

Revert "WIP: make buyback units parameterized so that user can choose"

This reverts commit df3b838d525bff5026eba1d81865c6645d534c88.
2016-05-11 16:24:32 -04:00
Scott Sanderson 8b1136d9d5 ENH: Validate missing_values at term construction.
Finds bugs in several bad tests that were constructing invalid terms.
2016-05-10 19:43:56 -04:00
Scott Sanderson f7e9281b14 BUG: Fix groupby with string columns.
The previous algorithm assumed that the group labels were integers. It
produced nonsense with LabelArrays (though sadly didn't crash because
numpy promotes None and void to object).
2016-05-10 16:57:59 -04:00
Scott Sanderson 2431aaefb5 BUG: Fix bad error message for element_of.
It referred to the wrong method name (`is_element`).
2016-05-10 16:57:59 -04:00
Scott Sanderson e0aeda4c3e BUG: Fix bytes/unicode issues in py3. 2016-05-05 01:46:35 -04:00
Scott Sanderson b78501e54a BUG: Fix broken isnull() on string classifiers.
Adds a special case in NullFilter to handle LabelArrays correctly.
2016-05-04 17:26:27 -04:00
Scott Sanderson 5a1ed7b1d3 ENH: Make element_of work for ints too. 2016-05-04 16:31:58 -04:00
Scott Sanderson 0922714bac DOC: Clarify test docstrings. 2016-05-04 15:54:51 -04:00
Scott Sanderson 4d42cddae4 ENH: Fail fast on outputs in CustomClassifier.
We don't support multiple outputs for CustomClassifier because we use
LabelArrays for string classifiers.
2016-05-04 15:54:50 -04:00
Scott Sanderson c40bbfae03 TEST: More tests for string predicates. 2016-05-04 15:54:50 -04:00
Scott Sanderson bb6f908036 TEST: Add test for categorical postprocessing. 2016-05-04 15:54:50 -04:00
Scott Sanderson 5f190395ad ENH: Add support for strings in Pipeline.
- Adds a new class, ``LabelArray``, which is a subclass of np.ndarray.
  LabelArray is conceptually similar to pandas.Categorical, in that it
  stores data with many duplicate values as indices into an array of
  unique values.  For string data with many duplicates (e.g. time-series
  of tickers or or industry classifications), this provides multiple
  orders of magnitude of improvement when doing string operations,
  especially string comparison/matching operations.

- Adds a new generic object "specialization" for `AdjustedArrayWindow`,
  and a corresponding ObjectOverwrite adjustment.

- Adds a new ``postprocess`` method to ``zipline.pipeline.term.Term``.
  This method is called on the final result of any pipeline expression
  after screen filtering has occurred. The default implementation of
  ``postprocess`` is identity, but Classifier overrides it to coerce
  string columns into pandas.Categoricals before presenting them to the
  user.
2016-05-04 15:50:52 -04:00
Joe Jevnik 59c8e371a2 ENH: Updates the cli, data bundles and extensions.
Adds the data bundle concept which makes it easy for users to register
loading functions to build out minute and daily data along with an
assets db and adjustments db. By default we have provided a `quandl`
bundle which pulls from the public domain WIKI dataset. Users may
register new bundles by decorating an ingest function with
`zipline.data.bundles.register(<name>)`. This also provides a
`yahoo_equities` function for creating an ingestion function that will
load a static set of assets from yahoo.

The cli is now structured as a couple of subcommands and has been
changed to `python -m zipline`. The old behavior of `run_algo.py` has
been moved to the `run` subcommand. This is almost entirely the same
except that it now takes the name of the data bundle to use, defaulting
to `quandl`.

The next subcommand is `ingest` which takes the name of
a data bundle to ingest. This will run the loading machinery and write
the data to a specified location that `run` can find.

There is also a `clean` subcommand which deletes the data that was
written with `ingest`.

Extensions have also been added to zipline. This is an experimental
feature where users can provide an extra set of python files to run at
the start of the process. These can be used to configure aspects of
zipline. Right now the only thing that is supported in an extension file
is the registration of a new data bundle.
2016-05-03 18:38:24 -04:00
Maya Tydykov 11d666daaa TST: add test for 13d filings dataset
MAINT: add 13d filings to factors init

MAINT: rename constant

MAINT: add event_date_col field
2016-04-28 11:59:49 -04:00
Maya Tydykov e726cc94c9 ENH: add 13d filings dataset to pipeline 2016-04-28 11:53:45 -04:00
Maya Tydykov 0191d9d903 MAINT: move filtering for null date rows back to dataframe
TST: test both next and prev event frame loading and use EventsLoader.

BUG: remove extra arg

MAINT: call list on zip for compatibility with python 3
2016-04-25 16:11:12 -04:00
Maya Tydykov 390295481c TST: add test for blaze loader with null data in date col
MAINT: fix blaze query
2016-04-25 11:42:10 -04:00
Maya Tydykov e41c99d077 MAINT: add an event date col field to each loader
MAINT: add event date col field and filter rows where this field is null

TST: modify tests to filter nulls in event date col

MAINT: calculate value repeats by vectorized computation on separate start and end dates.

MAINT: pass DatetimeIndex instead of list of strings
2016-04-25 11:42:08 -04:00
Maya Tydykov f8aa7c2ef4 TST: add test for case when null in expected column 2016-04-25 11:42:06 -04:00
Maya Tydykov e5ccd814e8 Merge pull request #1143 from quantopian/add-final-val-col-to-estimates
ENH: add actual value column to estimates dataset.
2016-04-21 16:23:55 -04:00
Maya Tydykov bd58140b97 ENH: add actual value column to estimates dataset. 2016-04-21 11:45:00 -04:00
dmichalowicz d9bfcaabde ENH: Support multiple outputs for custom factors 2016-04-21 10:57:29 -04:00
Maya Tydykov 1531568899 ENH: add custom dataset for estimize
MAINT: alphabetize constants

MAINT: remove obsolete column

TST: refactor tests to use common code

MAINT: remove unneeded fields from dataset

MAINT: remove obsolete earnings estimates columns and refactor
2016-04-19 11:29:03 -04:00
Joe Jevnik bc0b117dc9 MAINT: make the data loading apis more consistent.
Changes BcolzDailyBarWriter to not be an abc, data is passed as an
iterator of (sid, dataframe) pairs to the write method.

Changes the AssetsDBWriter to be a single class which accepts an engine
at construction time and has a `write` method for writing dataframes for
the various tables. We no longer support writing the various other data
types, callers should coerce their data into a dataframe themselves. See
zipline.assets.synthetic for some helpers to do this.

Adds many new fixtures and updates some existing fixtures to use the new
ones:

WithDefaultDateBounds
  A fixture that provides the suite a START_DATE and END_DATE. This is
  meant to make it easy for other fixtures to synchronize their date
  ranges without depending on eachother in strange ways. For example,
  WithBcolzMinuteBarReader and WithBcolzDailyBarReader by default should
  both have data for the same dates, so they may use depend on
  WithDefaultDates without forcing a dependency between them.

WithTmpDir, WithInstanceTmpDir
  Provides the suite or individual test case a temporary directory.

WithBcolzDailyBarReader
  Provides the suite a BcolzDailyBarReader which reads from bcolz data
  written to a temporary directory. The data will be read from
  dataframes and then converted to bcolz files with
  BcolzDailyBarWriter.write

WithBcolzDailyBarReaderFromCSVs
  Provides the suite a BcolzDailyBarReader which reads from bcolz data
  written to a temporary directory. The data will be read from a
  collection of CSV files and then converted into the bcolz data through
  BcolzDailyBarWriter.write_csvs

WithBcolzMinuteBarReader
  Provides the suite a BcolzMinuteBarReader which reads from bcolz data
  written to a temporary directory. The data will be read from
  dataframes and then converted to bcolz files with
  BcolzMinuteBarWriter.write

WithAdjustmentReader
  Provides the suite a SQLiteAdjustmentReader which reads from an in
  memory sqlite database. The data will be read from dataframes and then
  converted into sqlite with SQLiteAdjustmentWriter.write

WithDataPortal
  Provides each test case a DataPortal object with data from temporary
  resources.
2016-04-15 23:46:10 -04:00
Scott Sandersonanddmichalowicz 4449f289c2 TEST: Test that the mask is what we expect. 2016-04-07 17:29:47 -04:00
dmichalowicz 8db59b387b TST: Overhaul test case 2016-04-07 17:29:47 -04:00
dmichalowicz 5bae74adda ENH: Allow passing a mask when creating a factor 2016-04-07 17:29:47 -04:00
Eddie HebertandJean Bredeche 16fd6681a6 ENH: Rewrite of Zipline to use lazy access pattern
More documentation to follow in release notes.

Based on lazy-mainline branch, see for more details.

Also-By: Jean Bredeche <jean@quantopian.com>
Also-By: Andrew Liang <aliang@quantopian.com>
Also-By: Abhijeet Kalyan <akalyan@quantopian.com>
2016-04-04 16:12:58 -04:00
Maya Tydykov e8185a1512 MAINT: reorganize - move testing mixin to fixtures
BUG: correctly create asset finder

MAINT: rename fixture

STY: fixes for flake8

STY: add space around assignment

MAINT: add var back to constructor

MAINT: remove unused import

MAINT: compare var with None directly

MAINT: fix merge errors
2016-03-29 13:15:16 -04:00
Maya Tydykov 06dd6e958d TST: recfator tests to use fixtures
MAINT: use np.array

MAINT: return cols rather than modifying attribute
2016-03-29 13:12:50 -04:00
Maya Tydykov 8a28e82d32 ENH: add dividends to pipeline
MAINT: remove record date - not needed.

MAINT: restructure dividends dataset.

MAINT: restructure dividends factors.

WIP: update dividends tests.

MAINT: correct the way to get the 'next' event frame.
2016-03-29 13:12:50 -04:00
Scott Sanderson 9a04621781 ENH: Add eq and __ne__ to Classifier. 2016-03-28 15:46:28 -04:00
Scott Sanderson 0ebb72fe0d TEST: Explicitly use int64 everywhere.
Otherwise these tests will fail on 32-bit systems.
2016-03-28 12:21:58 -04:00
Scott Sanderson 076868f5a1 MAINT: Refactor shared code into test method. 2016-03-28 11:56:15 -04:00
Scott Sanderson fe22bde998 TEST: Test uneven buckets in quantiles. 2016-03-28 11:34:58 -04:00
Scott Sanderson c6e58af51b TEST: Test quantiles with better input.
Take the log of arange so that we know we don't depend on linearity of
the input.
2016-03-28 09:24:56 -04:00
Scott Sanderson 18bd7010b5 ENH: Improve short_reprs of classifier/normalizer.
GroupedRowTransform now shows the name of its transform, and Quantiles
shows the number of quantiles.

These are used by Pipeline.show_graph().
2016-03-25 15:11:18 -04:00
Scott Sanderson 758d6c74fc ENH: Add isnull and notnull for classifiers. 2016-03-25 15:11:18 -04:00