Put the logic for reading and writing the equity price and adjustment
data into a module located in data, making it distinct from the pipeline
loader usage of the formats.
This prepares for both incoming changes of how adjustments are written,
(which includes using the bcolz daily reader as an input), as well as
eventually providing the readers to a DataPortal object.
Previously we have capitalized input strings at different levels in
our code: in the user-facing API methods and in the asset finder.
This commit moves input string capitalization exclusively to the API
method to which the string was supplied. Specifically, the string is
capitalized by a preprocess API method decorator. The preprocess
decorator passes the input string to the newly defined
ensure_upper_case() method, which returns a TypeError if the argument
supplied is not a string.
ensure_upper_case() is defined in a new file, zipline/utils/input_validation.py.
The existing expect_types() method is also moved there.
Various tests in tests/test_assets.py are modified to account for the
fact that the asset finder method lookup_symol() no longer capitalizes
its supplied argument.
Improves the query for futures contract to use the date that comes first
in time (between notice_date and expiration_date) to determine cotnract
validity. If one of these is missing, we'll use the other.
Also modifies the query to order the resulting contracts by their
expiration_date if available, and to use their notice_date if not.
- Adds `zipline.pipeline.Pipeline`, a new user-facing class for managing
pipelines of Modeling API expressions.
- Adds `attach_pipeline` and `drain_pipeline` as API methods
- Removes `add_factor` and `add_filter` as API methods. These have been
replaced two new methods on `Pipeline`: `add`, and `apply_screen`.
- Adding a `Filter` as a column no longer implicitly truncates rows from
the Modelling API output. It simply causes a new column, of dtype
`bool` to show up in the output. Removal of rows is now handled by the
new `apply_screen` method of `Pipeline`.
- Refactors the existing Modeling API tests to reflect the new APIs.
Causes doctest to not care about the module of the raised exception.
This matters because Python3 includes the module in the formatted
exception, but Python3 doesn't, so the doctest will always fail on one
or the other without this flag.
Change calculate_results to take explicit parameters for sim_params, env
and benchmark_events instead of reading those values off of the TestCase
instance.
This prepares for tests setting specific sim_params in each test case,
which is needed for an incoming refactoring of how the test data is set up.
'instant fill' execution may be on the path to deprecation, (the removal
is currently proposed by the branch which removes events in favor of
operating over dts.), so remove instant_fill from the close position
tests so that the input and output is the same between the proposed
branch and here.
(This was one of the few places where instant fill was used that
was not testing instant fill behavior explicitly, so may be better to
align this test with the rest of the suite.)
Also, change test methods to make it more clear on which day the values
the expected and actual results differed.
Instead of calculating the position values for each stat result, e.g.
gross_exposure, net_liquidity etc.; get the positions upfront and then
calculate the period and position stats in order, passing each value
explicitly to the ones that follow it in the dependency chain.
e.g. the gross_value depends on the long_value and the short_value,
which called the position_values property for calculating both the
long_value and the short_value.
Removing the repeated calls to position_values (and
position_exposures) removes the need for the caching the last sale
prices and position amounts in separate vectors, since it is inexpensive
enough to read those values off of the positions dictionary held in the
position tracker.
This patch gives a small gain to ~500 sized portfolios, but the main
intent is to clear the path to not storing last_sale_prices on the
position objects at all. Removing all of the caching layer in this class
makes that change easier to apply. Removing the extra calls to
position_values also made this class easier to step through/reason about
when splicing in the new last sale price access, as well.
lookup_symbol_resolve_multiple was identical to lookup_symbol, except that lookup_symbol performed upper-casing of the input string and lookup_symbol would return Nones. Now, lookup_symbol has a kwarg 'default_None=True' and all symbols are upper-cased on insertion and request.
- Fixes an error where Modeling API data known as of the close of `day
N` would be shown to algorithms during `before_trading_start` as of
the close of the same day. Algorithms should now only receive data
during `before_trading_start/handle_data` that was known as of the
simulation time at which the function would be called.
- All Term instances now have a `mask` attribute that must be a `Filter`
or an instance of `AssetExists()`. `mask` can be used to specify that
a Factor should be computed in a manner that ignores the values that
were not `True` in the mask.
- Changed the interface for `FFCLoader.load_adjusted_array` and
`Term._compute` from `(columns, mask)`, with mask as a DataFrame, to
`(columns, dates, assets, mask)`, where mask is a numpy array. This
is primarily to avoid having to reconstruct extra DataFrames when
using masks produced by non `AssetExists` filters.
- Adds `BoundColumn.latest`, which gives the most-recently-known value
of a column.
All terms just implement `_compute` now. (We reserve `compute` for the
public API of `CustomFactor`.)
Also removed `TestingTermMixin` and its subclasses in favor of just
using `CustomFactor.`
The initialize method of TradingAlgorithm no longer accepts and
silently ignores args and kwargs, but instead forwards them
to the user-defined function referenced by self._initialize.
To avoid passing unexpected arguments to self._initialize, the
following additional adjustments are made:
- pop 'namespace' from the kwargs supplied to TradingAlgorithm
rather than simply get()ing it
- do not pass an AssetFinder to the TradingAlgorithm in
test_modelling_algo.py, as this has been deprecated and will
cause self._initialize to fail
Eddie Hebert