From pep-0008:
```
Always use a def statement instead of an assignment statement that binds a
lambda expression directly to an identifier.
Yes:
def f(x): return 2*x
No:
f = lambda x: 2*x
The first form means that the name of the resulting function object is
specifically 'f' instead of the generic '<lambda>'. This is more useful for
tracebacks and string representations in general. The use of the assignment
statement eliminates the sole benefit a lambda expression can offer over an
explicit def statement (i.e. that it can be embedded inside a larger expression)
```
Make `__next__` and `seek` share code instead of seek() calling
`__next__`. This avoids having to make a large number of integer
comparisons and `asanyarray` calls when seeking more than one tick
forward.
Limit the perspective offset to 1. There is a possibility that if a
consumer of the AdjustedArrayWindow does not fetch adjustments between
the end of the data window and the vantage points beyond the end of the
window.
Until that case has a solution, e.g. having the consumer of the
AdjustedArrayWindow include the perspective offset when calculating the
query for adjustments, limit the offsets to 1.
- Refactor `test_adjusted_array` to test a range of perspective_offsets in
all tests.
- Make perspective_offset a parameter to `AdjustedArray.traverse`
instead of `AdjustedArray`.
Add a perspective offset to `AdjustedArrayWindow` and `AdjustedArray`,
so that `HistoryLoader` does not need to twiddle with offsets to support
viewing the data from the bar after end of the window, (Which is the
case when a '1d' history window is retrieved in minute mode, which is
explained in the docstring for `HistoryLoader.history`)
Presently, this simplifies the logic in
`HistoryLoader._get_adjustments_in_range`, and other incoming
AdjustmentReader's, (e.g. the roll based adjustment reader for continous
futures.) This patch should also make it easier for history and pipeline
to converge on a singular `load_adjustments` method.
TST: fix quarter normalization test
TST: change test name
BUG: remove arg
BUG: look at dict keys
TST: add test for windowing
MAINT: raise ValueError instead of asserting
TST: add assertion to check windowing
TST: parametrize test over number of quarters forward/back.
BUG: fix adjustment calculation logic for quarter crossovers.
TST: add test for previous quarter windows
BUG: fix bugs in calculating previous windows
BUG: fix missing value for datetime
TST: add test case for missing quarter
Pandas 0.18 doesn't like having null-ish values in categoricals. Fixing
this properly requires re-thinking the semantics for missing_value on
pipeline terms, so we're punting on that until after we've upgraded to
0.18.
Pandas 0.18 deprecated passing "null-ish" values to pd.categorical. The
expectation, instead, is that you use categorical's native support for
missing data, which means the user will always get NaN's for missing
entries of the categorical.
A follow-up to this change should probably drop support for custom
missing values entirely and to use LabelArray/categorical for integer
data.
- Added test coverage for grouped and masked top/bottom.
- Added test coverage for grouped rank on datetime factors.
- Fixed an issue where grouped rank would fail on datetime inputs
because unary-negative isn't defined for datetimes. We now instead
directly invoke a function from rank.pyx that does the normalizations
as neeeded.
- Fixed an issue where GroupedRowTransform assumed that it produced the
same dtype as its input. This isn't true for rank() of a
datetime-dtype factor. GroupedRowTransform now takes a required dtype
parameter.
- Similarly, fixed an issue where GroupedRowTransform assumed that its
missing_value was the same as its parent's, which isn't true for
rank() of a datetime-dtype factor. GroupedRowTransform now takes a
required dtype parameter.
- Fixed an issue where Factor.demean() and Factor.zscore() weren't
properly cached because their static_identity included a closure that
was dynamically generated on each invocation. They both now always
use a function defined at module scope.
The previous algorithm assumed that the group labels were integers. It
produced nonsense with LabelArrays (though sadly didn't crash because
numpy promotes None and void to object).
- Fixes a bug where __setitem__ was not called when setting with a slice
on Python 2 (__setslice__ was called instead), which caused strange
behavior when setting an empty string. This is fixed by overriding
__setslice__ and forwarding to __setitem__.
- Fixes a bug where __getitem__ returned an instance of np.void when
returning a scalar. We now correctly return an entry from our
categoricals.
- Adds a new class, ``LabelArray``, which is a subclass of np.ndarray.
LabelArray is conceptually similar to pandas.Categorical, in that it
stores data with many duplicate values as indices into an array of
unique values. For string data with many duplicates (e.g. time-series
of tickers or or industry classifications), this provides multiple
orders of magnitude of improvement when doing string operations,
especially string comparison/matching operations.
- Adds a new generic object "specialization" for `AdjustedArrayWindow`,
and a corresponding ObjectOverwrite adjustment.
- Adds a new ``postprocess`` method to ``zipline.pipeline.term.Term``.
This method is called on the final result of any pipeline expression
after screen filtering has occurred. The default implementation of
``postprocess`` is identity, but Classifier overrides it to coerce
string columns into pandas.Categoricals before presenting them to the
user.
Classifiers are computations that represent grouping keys. They can be
used in conjuction with normalization functions like ``zscore`` or
``demean`` to perform normalizations over subsets of a dataset.
Notable changes:
- Added ``demean()`` and ``zscore()`` methods to ``Factor``.
- Added a classifier versions of ``Latest`` and ``CustomTermMixin``.
The .latest attribute of int64 dataset columns no produces a
classifier by default.
- Added ``Everything``, a classifier that maps all data to the same
value.
- Added ``zipline.lib.normalize``, which implements a naive, pure-Python
grouped normalize function. This will likely be moved to Cython in a
subsequent PR.
Maya Tydykov