Remove the lists of DailyReturn objects in favor of using pd.Series
to store the return values.
Should make it easier to inspect the values when stepping through,
make the windowing of data to a certain range more facile by using,
and have some performance increases due to removing object creation
and member access.
These tests use the random simulation parameters, which is leading
to an intermittent failure.
We may want to consider removing the randomness, but in the meantime
the randomness is exposing a case where the cost basis is not the value
expected, so logging the sim parameter values to help track down what
parameters cause the failure.
So that the units match the other risk calculations, also
use annualized returns for beat and alpha.
Update answer key to match values calculated on the first day.
Also, update performance tracker test so that the returns used
are fractional instead of > 1, so that the annualized numbers are
more in line with real world values.
The "use_environment" decorator is too side-effectful (e.g.,
connecting to Yahoo! Finance or another data source) to be used as a
decorator to a function that gets evaluated during module load. This
causes problems, e.g., if Zipline is being used in a gevent
environment, when the trading environment created by the decorator
argument tries to use greenlets when gevent hasn't been fully
initialized.
Since the decorator is nothing more than a context-manager wrapper,
this commit removes the decorator and replaces its use with contexts,
i.e., "with" statements.
- Change the expected type for order information from the string
of the order id to an `Order` object, so that it matches the same
abstraction level as passing in an event.
- Change the order (not to be confused with the parameter named `order`)
of the parameters so that they go from left to right in order of
static -> dynamic, i.e. the parameters most likely to change within
each invoration are the amount and price, with amount more likely
to change than price.
create_transaction accepted both sid and order, which in all cases
was derived from the current event, so remove `sid` and `order`,
replacing them with event
If there is a scenario where sid and order need to be set independently
of each other, then the underlying Transaction object can be called
directly.
Looking towards making writing custom slippage models slightly easier
by removing the redundancy.
- moved Order and Blotter to zipline.finance.blotter
- moved order method from AlgoSimulator to Blotter
- eliminated the set_order method in algorithm
- moved blotter to the algorithm
Add a test suite of dividend performance that ensures a holiday
is covered, needed because tests usually go over a random test range,
so holiday coverage isn't covered in basic test.
From @fawce.
Refactor PerformanceTracker, Blotter, and AlgorithmSimulator to
work with handling the end of a bar at the AlgorithmSimulator level
instead of within PerformanceTracker.
- PerforamnceTracker and Blotter are longer generators,
both provide functions to process events instead.
- AlgorithmSimulator calls each from within the loop running
over the data generator.
- Change test_perf_tracker utility to be compatible with change
away from PerformanceTracker as a generator.
Has the effect of:
- Fixing the timing of order emission.
- Allow minutely emission of benchmarks, which was prevented
by the extra grouping previously caused by Blotter.
Minutely emission also depends on work for streaming benchmarks
through performance and risk at a minute granularity.
- Create different benchmark containers in performance
depending on emission rate.
- Add a minute close method which updates algorithm and
benchmark returns, and calculates the risk metrics
depending on those methods.
- Provide fake 0.0 values for annualized metrics like
sharpe, sortino, and information, until we figure out
how they should be treated in the context of minutely
calculation.
*NOTE* This does not fully work without the changes to the
simulation loop by @fawce
The bar timestamps for day start and finish, for NYSE traded stocks,
should be 9:31 AM EST to 4:00 PM EST, for a total of 390 minutes.
Fix starting at 9:30 AM and the creation of 391 bars.
Instead of creating a list of benchmarks in the risk module,
stream benchmarks through the system as events, starting from the
algorithm generator.
Works towards more easily setting arbritrary pricing data as
a a benchmark, as well as working towards live minutely benchmarks.
- Add transaction and order types
- Move TransactionSimulator from trading.py to tradesimulation.py
(only used by other members of the tradesimulation module)
- Make Transaction an independent event, like dividend
- Add Blotter class.
- Flatten the transaction events to be independent of trade bar events
- Make orders into events that reach performance (need to add
handling)
- Issue IDs to orders and tracking each transaction's order id.
- Make volume share slippage fill orders independently, rather than
aggregating them into a single transaction.
- Perf tracker holds orders, serializes them with transactions.
- Order state defined and maintained by order class.
- Minutely emission of orders based on last_modified date.
Also, fix double emission of performance results with the last minute.
Change the perf tracker unit tests so that it doesn't rely on an
'extra' event triggering emission.
Unlike daily, minute emission now emits at the end of the bar in
the PerformanceTracker.transform instead of waiting for the next event.
Prevent an extra performance result with the timestamp of the midnight of
the day from being emitted.
Fix by setting the `saved_dt` value with the dt of the first event,
before entering into the main performance loop, otherwise a performance
result with a midnight timestamp and data from just the first event is
emitted.
The intraday performance results were emitting all transactions
for the entire day up to that point, instead of the desired transaction
list for the current timestamp.
Add a `dt` parameter to the `to_dict` method of PerformancePeriod so
that the transactions are limited to a specific datetime.
When the parameter is `None`, a todays_performance object will
function as previously with returning all transactions for the day.
# Please enter the commit message for your changes. Lines starting
So that the environments' exchange time is used without having to
specify it independently.
Also, moves uses of Delorean.shift for the exchange conversion inside
of environment to use the exchange_dt_to_utc method.
Wires up performance tracker so that when `emission_rate` is set
to `minute`, the performance packets are sent out every minute,
instead of once per day.
Please note, the performance packets that are generated are not
ready for prime time consumption, this patch is merely a step towards
hooking up the ability to inspect minute data.
Known issues:
- The packets do not currently include risk information.
Since we need to consider how this affects the denominators
of the risk calculations.
So that Transaction object behavior is exercised, uses the Transaction
object in performance module tests instead of ndict.
Also, adds fields to the __init__ of Transaction, to make the
definition of the object more well defined.
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.
Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.
================
Details:
- adding delorean to the requirements.
- made index symbol a parameter for loading the benchmark data. changed
messagepack storage to be symbol specific.
- ported risk, performance, algorithm, transforms, batch transforms
and associated tests to use simulation parameters and global environment
- factory and sim factory use global state and sim params
- factory method parameter names now reflect the class expected
With this patch, on the close of markets we "fast forward" to midnight of the
next trading day and calculate the dividend payments. This patch assumes that
the dividend dates are all at midnight UTC.
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.
PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.
PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.
Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
Instead of doing the rollover by creating a new PerformancePeriod,
introduces a `rollover` method that resets the values that need
to be fresh in a new period, and moves the ending values to starting
values, and leaves positions intact.
This isn't a major runtime improvement in of itself, but it does
allow us to more easily keep track of position values from period
to period, which other improvements will use.
There are only 6 trading days between the open and close specified
in test_perf test.
Also, removes getting the period_end off of the last trade,
since the test can now use the end date specified for the trading
environment.
Richard Frank