The check to filter out orders for zero shares wasn't truncated the
number of shares to an integer before checking, so if a fractional
amount less than 1 was being passed in, it wasn't being filtered out
even though it should have been. This is now fixed.
With the benchmark returns marked at midnight, the performance packet
for a day was emitted *before* any events for that day were processed.
Fix by expecting benchmarks marked at the market close, for backtests
that use minute data but emit performance results daily, so that the
benchmark handles at the end of day.
TST: Also, add test that exercises the event loop with minutely data,
(with benchmarks that are marked end of day), since that combination
was previously uncovered.
Working towards performance and risk logic being aware of
data frequency, as different handling of order of events based
on the data frequency is needed.
Backing out slice vs. valid(), because of an incompatiblity with
starting a minutely emitted session mid-day, since the midday start
date is not yet wired through SimulationParameters.
The slicing syntax is more explicit about declaring:
'get all returns up until the current dt'.
Also, protects against NaNs that occur before the current dt
being silently ignored.
i.e. the *_returns_cont series *should* have values from start
to current dt, but the .valid() call was occluding a bug where
it wasn't.
Both risk and performance now calculate performance since inception
(cumulative) and since the open. Both periods are updated intraday
and both are reported.
Batch risk for periods starting after the end of the treasury curve
history now use most recent curve.
Work towards running an algorithm against 'live' data, which can't
be bound to the available benchmarks and treasuries, since the
benchmarks and treasury curves for that day won't be published
until that night.
For printability in the repr when debugging algo config and state,
change the repr of TradingAlgorithm and the objects it contains
so that the more closely adhere to the repr interface of being
able to recreate an object instance.
We were converting to pd.Series for historical reasons as an artifact
during development, now that we pass dt we can just pass the float values
instead of wrapping it in a series.
- moved Order and Blotter to zipline.finance.blotter
- moved order method from AlgoSimulator to Blotter
- eliminated the set_order method in algorithm
- moved blotter to the algorithm
Refactor PerformanceTracker, Blotter, and AlgorithmSimulator to
work with handling the end of a bar at the AlgorithmSimulator level
instead of within PerformanceTracker.
- PerforamnceTracker and Blotter are longer generators,
both provide functions to process events instead.
- AlgorithmSimulator calls each from within the loop running
over the data generator.
- Change test_perf_tracker utility to be compatible with change
away from PerformanceTracker as a generator.
Has the effect of:
- Fixing the timing of order emission.
- Allow minutely emission of benchmarks, which was prevented
by the extra grouping previously caused by Blotter.
Minutely emission also depends on work for streaming benchmarks
through performance and risk at a minute granularity.
- Create different benchmark containers in performance
depending on emission rate.
- Add a minute close method which updates algorithm and
benchmark returns, and calculates the risk metrics
depending on those methods.
- Provide fake 0.0 values for annualized metrics like
sharpe, sortino, and information, until we figure out
how they should be treated in the context of minutely
calculation.
*NOTE* This does not fully work without the changes to the
simulation loop by @fawce
Eventually should to either return None or remove
progress completely, but in the meantime, return a
constant of 1.0 for progress of minute emissions.
Also, factor out the daily calculation into a property
instead of calculating during process.
Instead of creating a set of perf messages for each event during minute
emission mode, only include the messages on the last event in the bar.
Should cut down on calculations/serialization as well, as work towards
doing more 'end of bar' logic for minute benchmarks.
To fix the grouping of events so that (dt, events) ordering
is preserved, the tracking of order states needs to change
in the following way.
Change how order keeps track of dates:
- Change order's dt field to reflect modified date.
- Add a created field.
Change how performance keeps track of orders by:
- Map dt to transactions
- Map dt to orders
- Map order ids to keep track of updated orders.
The emission of order updates from the blotter were incorrect,
and subsequently, performance.
Previously, only the first action of the order was emitted,
fix so that all status updates are emitted.
The use of np.allclose introduced a severe performance penalty,
caused by the creation of two `np.array`s for each check.
Instead create and use a similar check which maintains tolerance
to floating point rounding, but operates only on scalars.
eigen vales, covariance, etc. are not calculated until the first
return is passed through, so initialize this values to None, so that
`repr` and its ilk work on a freshly created `RiskMetricsIterative`
object.
The bar timestamps for day start and finish, for NYSE traded stocks,
should be 9:31 AM EST to 4:00 PM EST, for a total of 390 minutes.
Fix starting at 9:30 AM and the creation of 391 bars.
Instead of creating a list of benchmarks in the risk module,
stream benchmarks through the system as events, starting from the
algorithm generator.
Works towards more easily setting arbritrary pricing data as
a a benchmark, as well as working towards live minutely benchmarks.
- Add transaction and order types
- Move TransactionSimulator from trading.py to tradesimulation.py
(only used by other members of the tradesimulation module)
- Make Transaction an independent event, like dividend
- Add Blotter class.
- Flatten the transaction events to be independent of trade bar events
- Make orders into events that reach performance (need to add
handling)
- Issue IDs to orders and tracking each transaction's order id.
- Make volume share slippage fill orders independently, rather than
aggregating them into a single transaction.
- Perf tracker holds orders, serializes them with transactions.
- Order state defined and maintained by order class.
- Minutely emission of orders based on last_modified date.
Also, fix double emission of performance results with the last minute.
Change the perf tracker unit tests so that it doesn't rely on an
'extra' event triggering emission.
Unlike daily, minute emission now emits at the end of the bar in
the PerformanceTracker.transform instead of waiting for the next event.
During minute emissions, it is still helpful to have a final daily
performance result, analogous to what would be the final packet in
a daily emitted backtest, so that all transactions, etc. are contained
in one place.
Prevent an extra performance result with the timestamp of the midnight of
the day from being emitted.
Fix by setting the `saved_dt` value with the dt of the first event,
before entering into the main performance loop, otherwise a performance
result with a midnight timestamp and data from just the first event is
emitted.
Jonathan Kamens