Commit Graph
52 Commits
Author SHA1 Message Date
fawce 2db01b3935 created a new generator-backed trading environment for testing.
other details:
    - also fixed grammatical errors in loader's status messages.
    - converting the treasury curves to an ordered dict.
    - moved to using a lambda for clarity as per @ehebert
    - initializing calendar end dates to be midnight of current date in
    - US/Eastern. Yahoo data isn't available until midnight eastern.
2013-02-18 23:35:38 -05:00
fawceandEddie Hebert d67e5d7a4b factory had a bug in the creation of trade history that traversed daylight savings time changes. added a fix and a test. 2013-02-18 10:24:32 -05:00
fawceandEddie Hebert a4a4d38a73 TradingEnvironment allows the specification of a benchmark index and a local timezone for the exchange. This commit adds tests to verify the TradingEnvironment properly handles London Stock Exchange index, FTSE.
- added LSE reference rrules calendar (thanks to Edward Johns)
    - added tests to verify LSE environment matches rrule calendar
    - added a test to verify global environment behavior can be set.
    - moved DailyReturn class to trading to eliminate circularity from
    risk <-> trading.
    - updated TradingEnvironment to be a context manager. This allows users
    to run algorithms in individually isolated environments in one python
    process. This is useful for managing multiple algorithms in a single
    ipython notebook.
    - added comments to explain behavior and useage of the global environment
2013-02-18 10:24:32 -05:00
fawceandEddie Hebert 2c7355a0dc Refactoring of TradingEnvironment to isolate the global state: index symbol and exchange timezone. Parameters that define the simulation (start, end, and capital base) were put in a new class, SimulationParameters.
Global state for the financial simulation environment is accessed through the
zipline.finance.trading module, which now contains a module variable:
environment.

Parameters are passed into an algorithm as a keyword argument, sim_params.
SimulationParameters creates a trading day index for the test period that
can be used to find trading days, calculate distance between trading days,
and other common operations. The sim params index is just selected from the
global state.

================

Details:

    - adding delorean to the requirements.
    - made index symbol a parameter for loading the benchmark data. changed
    messagepack storage to be symbol specific.
    - ported risk, performance, algorithm, transforms, batch transforms
    and associated tests to use simulation parameters and global environment
    - factory and sim factory use global state and sim params
    - factory method parameter names now reflect the class expected
2013-02-18 10:24:32 -05:00
Thomas Wiecki 8c182ad66e ENH: load_bars_from_yahoo provides OHLC. Based on Brian Cappello's code. 2013-02-12 13:26:13 -05:00
fawceandEddie Hebert 817ed88e38 Adds dividends to performance tracking.
Algorithm returns and the risk calculations that depend on them now include
cash dividends. This commit does _not_ provide an API for user algorithms to
access dividends.

PerformanceTracker expects the dividend data to arrive as events, similar to
the way that Trades arrive. Dividends are expected to have adjusted payment
amounts that are inline with adjusted trades.

PerformanceTracker maintains state of all the unpaid dividends in the position
objects held in PerformancePeriod. Dividend objects contain all the relevant
dates (declared, ex, payment) as well as net and gross amounts. Dividends are
removed from the list as they are paid. Cash flow is not incremented until the
payment day. This creates the possibility of a dividend being owed but not
paid or realized before the end of a test. For example, a dividend with an
ex_date of today may have a pay date 2 weeks in the future. Right now the
algorithm does not receive any credit for unpaid dividends.

Tests cover buying/selling around the ex_date and payment_date, and checking
that the performance calculated is as expected.
2013-02-06 16:39:39 -05:00
Eddie Hebert 44140d2de7 Fixes dates used in test factory for batch transforms.
The test factory was creating non-market days.
i.e. the date range spanned the weekend.

Using pandas' BDay frequency so that only business days are created.

This specific date range doesn't have holidays, so not accounting
for holidays in the factory.

Also, widens the range of the trading calendar to cover the test dates
generated by the factory which include 1990.
Previously the trading calendar began with 2002, meaning that holiday
and weekend adjustments with the data exercised by the factory did
not trigger when run with data in 1990.

This does increase the memory footprint of the tradingcalendar module.
However, only by a couple MB, so taking the hit there to enable
correct behavior.
2013-01-30 14:06:56 -05:00
Eddie Hebert 799a357d98 Changes test factory to use Event instead of ndict.
As more sources are moving off of ndict,
changing the factory to use Event, so that when testing we are
exercising use of Event.
2013-01-06 17:13:07 -05:00
Thomas Wiecki 775c564ea1 ENH: Added DataPanelSource. 2012-12-13 12:55:05 -05:00
Richard Frank 095f2dd65b Date bookkeeping fixes in perf and risk
Issues appeared when we were close to the end of our
historical data.

Yielding DONE event with both perf and risk messages now
2012-12-12 15:23:26 -05:00
Richard FrankandEddie Hebert 4981c67c31 Handle missing historical data more elegantly
Updated the search for treasury data when there is none for the
test end date.
It could be that the end date is not a trading day, or we could
just be missing treasury data. In either case, we try to recover
more gracefully now, by searching as far as possible and maybe
logging a warning.

Similarly, if there is no benchmark data for the test end date,
look for the next trading day. If we really have no data,
blow up with our own explicit exception, instead of overflowing
in our search for dates in the future.
2012-12-10 13:03:25 -05:00
Eddie Hebert 5a585fc3aa Merge pull request #37 from quantopian/fix_transforms
Fix transforms
2012-12-06 14:00:35 -08:00
Thomas Wiecki 5f6839beea BUG: Refactored batch_transform unittests and fixed some bugs. 2012-12-06 12:36:47 -05:00
Eddie Hebert 3ff3a6964e Allows the capital base of an algorithm to be more easily modified.
By having run() use a capital_base member of the algorithm to
create the trading environment, the capital base should now be
configurable in the instantiation of the algorithm.

e.g.:

```
algo = LowCapitalBaseAlgorithm(capital_base=1000.0):
```
2012-12-04 10:42:58 -05:00
Eddie Hebert 8a0d6e72cd Changes the location of downloaded external data files.
So that the zipline library can be used when installed to a
write-protected location, e.g. the global site-packages,
moving the download files to a directory in the user's path,
which should be writeable.

For now, choosing a ~/.zipline/data location.
2012-11-08 22:51:03 -05:00
Thomas WieckiandEddie Hebert 126e9fdf26 Fixed batch_transform window length not being market aware.
Added accompanying unittest.

Minor refactoring of unittests and factory.
2012-11-06 21:13:27 -05:00
Thomas Wiecki ae1644fe35 Added yahoo finance loader. 2012-10-22 09:28:56 -04:00
Thomas WieckiandEddie Hebert 15ac658024 Restructured file hierarchy. Transforms now live in transforms (transform.py is now transforms/utils.py). Sources are in sources.py. VWAP is now MovingVWAP. 2012-10-19 18:10:51 -04:00
Eddie Hebert 0ab136f1c8 Adds a loader for market data when it doesn't exist locally.
Hopefully, this helps ease ramp up time for developing against
market data, without us distributing the data.

We do a check for the data when attempting to read the msgpack
files, if they don't exist the loader makes a web request and
retrieves and serializes the data for the user.

Provides a loader for:

- curves from data.treasury.gov
- benchmarks from Yahoo! Finance

Adds dependency of requests library in dev requirements.
2012-10-19 11:19:14 -04:00
Eddie Hebert 5eb4b171a9 Removes unused module. 2012-10-12 15:15:32 -04:00
Eddie Hebert 35b0bb9b26 Removes unused log_path method. 2012-10-12 15:15:07 -04:00
Eddie Hebert d98190daf7 Removes unused kwarg btrid to create_txn. 2012-10-09 14:16:48 -04:00
Eddie Hebert 9500163ed8 Adds Apache 2.0 license header to source files. 2012-10-08 17:32:41 -04:00
Eddie Hebert 15604134c7 Removes unused random source method.
Method was unused and used an unimported module.
2012-10-05 12:14:39 -04:00
Eddie Hebert 77af1ca632 Applies PEP-8 and pyflakes style to tests and zipline.
Mostly whitespace, line width and other spacing changes.
Also, removes use of deprecated has_key in favor of `in`

Going forward new patches should pass running `flake8` before
submission.
2012-10-05 12:14:09 -04:00
fawce 1caefbff43 tests are passing for independent commission model 2012-09-28 23:12:41 -04:00
Thomas Wiecki 4324f95f36 Better unittest coverage. DataFrameSource is now filtering sids. Fixed outstanding issues. 2012-09-21 11:59:46 -04:00
Thomas Wiecki 3be2f313cd Shortened test window. Renamed functions in window transform. 2012-09-19 18:39:42 -04:00
Thomas Wiecki 2fe37fa34c Fixed pandas depracation. 2012-09-19 18:02:09 -04:00
Thomas Wiecki 729a4d9058 Large refactoring and documentation of new algorithm base class and batch transform. 2012-09-19 17:49:39 -04:00
Thomas Wiecki 5d00099243 example code now retrieves data from yahoo finance. 2012-09-17 12:15:36 -04:00
fawce 81718011df cut most of protocol, cut most of log_utils. 2012-09-08 23:20:38 -04:00
Thomas Wiecki 81ea337ae6 Resolve merge conflicts. Fix: use hasattr instead of checking class dict. Turn SimulatedTradingLite into a generator. 2012-09-07 10:28:36 -04:00
Thomas Wiecki 3f801ef387 Debugging. 2012-08-23 17:38:48 -04:00
fawce f819ed67ec bugfixes 2012-08-13 17:05:14 -04:00
scottsanderson b70e76d516 commit pre-merge 2012-08-08 11:21:26 -04:00
fawce aeb50da170 fixes for unit tests, back to 50/51 passing. 2012-08-07 14:42:43 -04:00
fawce 06dc6f7acb beginning refactor to use single threaded simulator. 2012-08-06 13:11:20 -04:00
scottsanderson 4768bea646 more progress on tsc 2012-08-01 18:53:09 -04:00
scottsanderson 3621934a28 better variable names and PreTransformLayer 2012-07-28 18:24:57 -04:00
scottsanderson cef36c172d commit for fawce 2012-07-27 19:40:42 -04:00
fawce 5f878300a0 switched from pub/sub to push/pull for merge -> client socket. also purged source_id from the constructors of all datasources. 2012-07-03 12:11:22 -04:00
fawce 46ef2e136d path hacking for logging.cfg 2012-05-21 16:21:29 -04:00
Stephen Diehl c68cd849c5 Update factory location finder. 2012-05-18 11:54:11 -04:00
Stephen Diehl 244f787176 Cleaned up OOP on transforms. 2012-05-16 16:55:26 -04:00
Thomas Wiecki e26638058c Reverted path independence code in factory.py. Not sure how to fix with new path structure. 2012-05-15 17:37:27 -04:00
Stephen Diehl d399edd419 Fix ndict woes. 2012-05-14 13:49:45 -04:00
Stephen Diehl e04415e63f Remove all namedicts. 2012-05-14 11:35:43 -04:00
Stephen Diehl 73557b907f Rework imports on tests. 2012-05-14 11:17:56 -04:00
Stephen Diehl 8b95aebcf2 Refactor lots of things. 2012-05-14 10:57:40 -04:00