Commit Graph

2838 Commits

Author SHA1 Message Date
Scott Sanderson b766ce6ebd ENH: Add zipline.utils.cache.
Implements a `CachedObject` utility class for wrapping cached results
with an expiration date.
2015-10-01 18:03:53 -04:00
Scott Sanderson 00c413e9d4 ENH: Add zipline.utils.preprocess.
Implements tools for preprocessing the arguments to user-facing
functions.
2015-10-01 18:03:53 -04:00
Scott Sanderson 75138343ba BUG: Use IGNORE_EXCEPTION_DETAIL in test_doctests.
Causes doctest to not care about the module of the raised exception.
This matters because Python3 includes the module in the formatted
exception, but Python3 doesn't, so the doctest will always fail on one
or the other without this flag.
2015-10-01 18:03:53 -04:00
Scott Sanderson 670706158a ENH: Don't write files when rendering TermGraphs.
Overhauls `zipline.modelling.visualize` to use in-memory buffers when
shelling out to `dot` and friends.

Also adds `svg`, `png`, and `jpeg` attributes to `TermGraph`, and adds a
`_repr_png_` so that `TermGraph` renders as a PNG by default.
2015-10-01 18:03:53 -04:00
Stewart Douglas e33fa988f2 DEP: Remove _convert_asset_str_fields
_convert_asset_str_fields was added to address the fact that the
unicode returned by SQLAlchemy was causing test failures, in
particular test_repr and test_root_symbols failed because of the
leading 'u' character in the unicode of python-2.x. Later it was
pointed out that this method would behave differently in python 2
and 3, in particular the str() method would return bytes in python
2 and unicode in python 3. After investigating it was found that
the _convert_asset_str_fields was no longer  necessary. It is unclear
why the original tests were failing. It is not due to SQLAlchemy
versions, as tests pass for version >1 and <1.
2015-10-01 15:02:23 -04:00
Stewart Douglas 8ba15a6f72 BUG: Correctly test for duplicate symbols 2015-10-01 15:02:23 -04:00
Eddie Hebert 39fff4aa43 Merge pull request #735 from quantopian/remove-minute-performance
MAINT: Remove unused minute performance period.
2015-10-01 10:16:52 -04:00
James Kirk a4da2761a2 Merge pull request #738 from grundgruen/asset_ts_bug
BUG: missing comma in _asset_timestamp_fields
2015-10-01 09:25:49 -04:00
Warren O'Neill 66b36919a3 BUG: missing comma in _asset_timestamp_fields 2015-10-01 10:57:54 +02:00
Eddie Hebert b655a2a592 Merge pull request #737 from quantopian/remove-random-simulation-params
Pre-lazy changes to test_perf_tracking
2015-09-29 16:26:22 -04:00
Eddie Hebert 23e7433635 TST: Make params explicit to calc results.
Change calculate_results to take explicit parameters for sim_params, env
and benchmark_events instead of reading those values off of the TestCase
instance.

This prepares for tests setting specific sim_params in each test case,
which is needed for an incoming refactoring of how the test data is set up.
2015-09-29 14:36:45 -04:00
Eddie Hebert 9581afb32a TST: Remove random simulation parameters.
For better predictability/ability to reason and step through the test
cases, use the same start and end for each run of the test suite.
2015-09-29 14:36:45 -04:00
Eddie Hebert 644cfe6a42 MAINT: Remove unused minute performance period.
Minute performance period is no longer used. The minute risk containers,
which had been removed, were the last usage of the minute performance
period.
2015-09-29 12:10:37 -04:00
Eddie Hebert b1cb177d45 Merge pull request #731 from quantopian/remove-instant-fill-from-close-pos-tests
TST: Remove instant fill from close position test.
2015-09-28 15:07:40 -04:00
Eddie Hebert cbb3d807b7 TST: Remove instant fill from close position test.
'instant fill' execution may be on the path to deprecation, (the removal
is currently proposed by the branch which removes events in favor of
operating over dts.), so remove instant_fill from the close position
tests so that the input and output is the same between the proposed
branch and here.

(This was one of the few places where instant fill was used that
was not testing instant fill behavior explicitly, so may be better to
align this test with the rest of the suite.)

Also, change test methods to make it more clear on which day the values
the expected and actual results differed.
2015-09-28 14:48:19 -04:00
Eddie Hebert 5d2217b0eb Merge pull request #726 from quantopian/explicit-position-value-calcs
MAINT: Only calc position values once per packet.
2015-09-25 17:33:53 -04:00
Eddie Hebert 20c64b591f MAINT: Clean up net calculations.
- Combine the net value and exposure functions into `calc_net` since
  they use the same logic.

- Change the logic to handle on empty list to using the a start value of
  0.0. More concise, and reduces the number of return points from the
  function to one.
2015-09-25 16:43:34 -04:00
Eddie Hebert c88412e3f1 TST: Add coverage for long and short count.
Make basic checks of position stats test all fields, by adding coverage
for `shorts_count` and `longs_count`.
2015-09-25 16:42:25 -04:00
Eddie Hebert 31d214d8ff MAINT: Combine leverage calculations.
Instead of having two leverage functions, whose differences were the
parameter names, add a `calc_leverage` function, with the calling code
determining whether it is gross or net by the type of exposure passed in.
2015-09-25 13:58:44 -04:00
Richard Frank 136eb8aa0d BUG: Running an algo with a df/panel of Assets was raising SidNotFound 2015-09-24 21:49:45 -04:00
Richard Frank 0b6b11a080 MAINT: Removed some unnecessary instance attributes 2015-09-24 21:49:45 -04:00
John Ricklefs b0de423b38 MAINT: Add indexes to company_symbol/fuzzy_symbol.
Reduces AssetFinder.lookup_symbol call times
dramatically:

With fuzzy=True: 2.32ms per call -> 391 micros
With fuzzy=False: 2.35ms  -> 451 microseconds
2015-09-24 21:14:48 -04:00
Eddie Hebert ae97e75388 MAINT: Only calc position values once per packet.
Instead of calculating the position values for each stat result, e.g.
gross_exposure, net_liquidity etc.; get the positions upfront and then
calculate the period and position stats in order, passing each value
explicitly to the ones that follow it in the dependency chain.

e.g. the gross_value depends on the long_value and the short_value,
which called the position_values property for calculating both the
long_value and the short_value.

Removing the repeated calls to position_values (and
position_exposures) removes the need for the caching the last sale
prices and position amounts in separate vectors, since it is inexpensive
enough to read those values off of the positions dictionary held in the
position tracker.

This patch gives a small gain to ~500 sized portfolios, but the main
intent is to clear the path to not storing last_sale_prices on the
position objects at all. Removing all of the caching layer in this class
makes that change easier to apply. Removing the extra calls to
position_values also made this class easier to step through/reason about
when splicing in the new last sale price access, as well.
2015-09-23 22:26:13 -04:00
John Ricklefs e3d52df88c ENH: Allow passing an existing engine to TradingEnvironment
Specifically to allow the use case of creating
an in-memory SQLite database and populating
it with assets before creating the trading
environment.
2015-09-21 15:37:38 -04:00
Scott Sanderson bd147a084e BUG: Fix crash on .latest for integer-typed columns.
Int columns get coerced to float on load, and we don't currently support
non-float columns from CustomFactors.
2015-09-21 13:20:26 -04:00
Scott Sanderson 8eb09f70d7 BUG: Don't crash when raising UnsupportedDataType.
Before this was raising a KeyError on failure to format the message
properly.
2015-09-21 13:20:26 -04:00
Scott Sanderson 85cacdf356 DOC: Typo fix. 2015-09-21 13:20:26 -04:00
John Ricklefs fd1bee9bf6 ENH: Don't trigger AssetDateBounds for orders of 0 shares. 2015-09-18 15:19:22 -04:00
John Ricklefs 38ff4cc913 BUG: Normalize dates in AssetDateBounds control checks
Assumes that if a given asset's end_date is
e.g. 9/17/2015 00:00:00 UTC that it means the
asset is still tradeable on 9/17/2015 during
the market day.
2015-09-18 15:19:15 -04:00
jfkirk 88495ce9b7 MAINT: Batch transform uses get_algo_instance 2015-09-18 14:17:53 -04:00
jfkirk 4db5e6c142 ENH: AssetDBWriter now overwrites symbol with file_name, if provided 2015-09-16 16:34:34 -04:00
jfkirk 082bc4f906 MAINT: Removes default_none from lookup_symbol 2015-09-16 14:55:42 -04:00
jfkirk c446e7f62a ENH: Adds fuzzy symbol look-up and makes it distinct from delimited symbols 2015-09-16 14:34:18 -04:00
jfkirk d84bdefef8 MAINT: Removes lookup_symbol_resolve_multiple method
lookup_symbol_resolve_multiple was identical to lookup_symbol, except that lookup_symbol performed upper-casing of the input string and lookup_symbol would return Nones. Now, lookup_symbol has a kwarg 'default_None=True' and all symbols are upper-cased on insertion and request.
2015-09-16 09:54:37 -04:00
jfkirk 29dce965d5 ENH: Simplifies AssetFinder symbol lookup by making fuzzy lookup the default 2015-09-16 09:54:11 -04:00
Scott Sanderson 26fd6fda8b ENH/BUG: Modeling API enhancements.
- Fixes an error where Modeling API data known as of the close of `day
  N` would be shown to algorithms during `before_trading_start` as of
  the close of the same day.  Algorithms should now only receive data
  during `before_trading_start/handle_data` that was known as of the
  simulation time at which the function would be called.

- All Term instances now have a `mask` attribute that must be a `Filter`
  or an instance of `AssetExists()`.  `mask` can be used to specify that
  a Factor should be computed in a manner that ignores the values that
  were not `True` in the mask.

- Changed the interface for `FFCLoader.load_adjusted_array` and
  `Term._compute` from `(columns, mask)`, with mask as a DataFrame, to
  `(columns, dates, assets, mask)`, where mask is a numpy array.  This
  is primarily to avoid having to reconstruct extra DataFrames when
  using masks produced by non `AssetExists` filters.

- Adds `BoundColumn.latest`, which gives the most-recently-known value
  of a column.
2015-09-16 01:47:11 -04:00
Scott Sanderson 691f6d95d7 STY: Use relative imports in engine.py. 2015-09-16 01:28:16 -04:00
Scott Sanderson 46882bfcb9 TST: Remove unnecessary instance attr assignments. 2015-09-16 01:28:16 -04:00
Scott Sanderson e32f50ba3a DOC: Better docstring. 2015-09-16 01:28:16 -04:00
Scott Sanderson b70f517077 DOC: Fix docstring types. 2015-09-16 01:28:16 -04:00
Scott Sanderson 4561c776a0 DOC: Fix typo in docstring. 2015-09-16 01:28:15 -04:00
Scott Sanderson 5730de25a4 DEV: Kill compute_from_{arrays,windows}.
All terms just implement `_compute` now. (We reserve `compute` for the
public API of `CustomFactor`.)

Also removed `TestingTermMixin` and its subclasses in favor of just
using `CustomFactor.`
2015-09-16 01:28:15 -04:00
Scott Sanderson 58ceb7b7bb DEV: Add zipline.utils.memoize.
- Moved zipline.utils.lazyval.
- Added `remember_last` which is just `lru_cache(1)` with simpler logic.
2015-09-16 01:28:15 -04:00
Scott Sanderson dad3bbd879 ENH: Add tools for visualizing FFC graphs. 2015-09-15 23:58:09 -04:00
Scott Sanderson ad54eedeea Revert "MAINT: AssetFinder now takes fuzzy characters on look-up, not init"
This reverts commit c16bc9f8db.  This
caused upstream breakage unexpectedly.  Postponing until we can synchronize.
2015-09-15 23:57:25 -04:00
jfkirk c16bc9f8db MAINT: AssetFinder now takes fuzzy characters on look-up, not init 2015-09-15 09:38:17 -04:00
Thomas Wiecki 6f2e1672d7 BUG: Forward initialize args and kwargs to user-defined function.
The initialize method of TradingAlgorithm no longer accepts and
silently ignores args and kwargs, but instead forwards them
to the user-defined function referenced by self._initialize.

To avoid passing unexpected arguments to self._initialize, the
following additional adjustments are made:

 - pop 'namespace' from the kwargs supplied to TradingAlgorithm
   rather than simply get()ing it

 - do not pass an AssetFinder to the TradingAlgorithm in
   test_modelling_algo.py, as this has been deprecated and will
   cause self._initialize to fail
2015-09-14 10:42:20 -04:00
jfkirk a6e677a1d7 MAINT: Removes unused __init__ from AssetDBWriter 2015-09-14 10:01:36 -04:00
jfkirk b0195c0d17 MAINT: Abstracts-away Timestamp conversion of Asset fields 2015-09-11 15:58:46 -04:00
jfkirk a6ce1e5e8d ENH: Adds auto_close_date field to Future objects 2015-09-11 14:08:03 -04:00