Currently, `order_percent()` and `order_target_percent()` both operate as a percentage of `self.portfolio.portfolio_value`. This PR lets them operate as percentages of other important MVs.
(also adds `context.get_market_value()`, which enables this functionality)
For example:
```python
order_percent('AAPL', 0.5)
order_percent('AAPL', 0.5, percent_of='cash')
order_target_percent('MSFT', 0.1, percent_of='shorts')
tech_stocks = ('AAPL', 'MSFT', 'GOOGL')
tech_filter = lambda p: p.sid in tech_stocks
for stock in tech_stocks:
order_target_percent(stock, 1/3, percent_of_fn=tech_filter)
```
Limited use of `pandas` data structures in both `HistoryContainer` and
`RollingPanel`. Where possible, methods were amended to return raw
`ndarrays` with the indexing logic done separately. This allows us to
cut down the number of times pandas objects are created both as returns
and intermediate values. The separation of indexing from data access
allowed us to minimize the times we’d make use of pandas indexes.
This required that that certain methods like `NDFrame.ffill` be replaced
with versions that work with `ndarrays`. Some of this was done via
straight numpy methods and others by access pandas internal
machinery. Outside of allowing us to use faster ndarrays, many of these
function provided speedups over their pandas counterparts as we didn’t
require the extra features like handling multiple dtypes. i.e. np.isnan
is faster than pd.isnull, but only works with certain dtypes.
and added a new test case
was not iterating over lookup date directory names, and
therefore mising all by one list of stocks.
discovered because of differing sort orders between
my local machine, other devs, and travis ci.
contstruction.
BarData can be falsey. in create_buffer_panel, the intention of the
check against bar_data was to see if it was passed at all, not if it was
truthey. In order to make that check more explicit, the check now
asserts that bar_data is not None.
getting filled with the wrong datetimes and causing errors.
Updates the logic for addressing missing datetimes and adds unit tests
for the 2 main cases (no missing datetimes, and some missing datetimes).
makes it an offset from 13:30 UTC.
This is to be more consistent with the market_close, which is an offset
from 20:00 UTC.
This also makes market_open and market_close cache the dt to offset from
for each day.
Previously, all specs had to be pre-allocated by using the 'add_history'
function. This is now no longer required and instead serves as a hint to
the HistoryContainer to pre-allocate the space for the given spec.
History can grow by increasing the length for a frequency, adding a
frequency, or adding a field. It can grow with any combination of
these.
HistoryContainer now is aware of the data_frequency of the algorithm,
and no longer uses the daily_at_midnight flag; instead, this is the
default behavior.
- NotHalfDay only worked at midnight
- week_(start|end) were actually month_(start|end)
- Removes check_args from api.
- Default offset of 30mins for market_(open|close)
schedule_function takes a date rule, a time rule, and a function and
will call the function, passing context and data only when the two rules
fire. This allows for code that is conditional to the datetime of the
algo.
This is implemented internally with `Event` objects which are pairings
of `EventRule`s and callbacks.
handle_data becomes a special event with a rule that always fires. This
makes the logic for handling events more complete and compact.
Overhaul the core HistoryContainer logic to be more robust to changing
universes.
Major Changes
-------------
* Remove `return_frame` cache. The original purpose of using
return_frames was to avoid having to create new DataFrames on each
iteration of handle_data, but we ended up having to copy the return
frames anyway because user code could mutate the frames in place.
Removing the return_frames reduces unnecessary copying, and reduces
the logic of `get_history` to just forward-filling and concatenating
two DataFrames.
* Use a `MultiIndex`ed DataFrame to represent
`last_known_prior_values`. This makes lookups faster and greatly
simplifies the logic of adding and dropping sids.
* HistoryContainer no longer attempts to determine its universe based on
the contents of its internal buffers. The TradingAlgorithm
controlling the container is now responsible for explicitly calling
`add_sids` or `drop_sids` when securities enter or leave the
algorithm's universe. These methods, along with the internal
`_realign` method, provide a clean interface for changing the universe
of securities managed by the container.
* Refactor index mutation logic in `RollingPanel` into a
`MutableIndexRollingPanel` subclass. Maintenance of the old behavior
is regrettably necessary to support `BatchTransform`.
* Refactor shared logic from `roll` and `get_history` into a single
`aggregate_ohlcv_panel` method that's responsible for collapsing an
OHLCV buffer into a frame.
In support of source that emits a subclass of Event which defines some
fields as properties instead of doubling the value in the
`Event.__dict__`
Use hasattr instead of the overridden __contains__ method of the Event
class, so that when non-algorithm facing code checks for field existence,
properties count.
Intentionally not touching the `__contains__` in Event, to avoid
changing, at the moment, any algo behavior that relies on the
`__contains__` behavior's use of `__dict__`
This commit adds support for arbitrary objects in addition to NaN
and infinity values. The object well be returned in string format
as part of the error message.
Previously order was not checking for nan values sent as
limit or stop prices. It will now raise a runtime exception
in the event that an attempt to order with a nan price is made.
Fixes a crash in various transforms when providing CUSTOM events whose fields
don't match the fields required for the transform.
This is fixed by requiring all `EventWindow` subclasses to supply a `fields`
property, which returns a list of strings that are required keys for any event
that can be processed by the window. Any CUSTOM events the don't supply the
required fields for a transform window are ignored by that window.
Previously the last sale price was not correctly being set on
positions when the transaction arrived before the trade event.
The last sale price was defaulted to zero and never updated. This resulted
in one holding stocks that were bough >>0 and now had value 0 from
the perspective of returns. The returns would display correctly again
when the next trade of that security happened. For most securities trading is
frequent enough that there's no issue, but for some illiquid ones it took
hours to fix itself.
Updated test_perf_tracking:TestPerformanceTracker.test_minute_tracker
This test was based on assuming that last_sale_price was zero,
allowing the sharpe ratio to be calculated. The sharpe ratio can no longer
be calculated for this specific tested scenario and the test has been changed
accordingly.
The initialization of perf_tracker had been moved from __init__
in TradingAlgorithm to _create_generator. This caused perf_tracker
to not be ready when portfolio requested it. portfolio was consequently
not ready for access in init. portfolio can now be accessed in init
again, assuming valid sim_params are passed. Otherwise it will be
available in handle_data() after _create_generator() is called.
Jeremiah Lowin