Merge branch 'rc-0.1.dev7'

Normalize timestamps before comparison.

README.rst

@@ -1,196 +1 @@
-========
-Catalyst
-========
-
-|version status|
-
-Catalyst is an algorithmic trading library for crypto-assets written in Python.
-It allows trading strategies to be easily expressed and backtested against historical data, providing analytics and insights regarding a particular strategy's performance.
-Catalyst will be expanded to support live-trading of crypto-assets in the coming months.
-Please visit `<enigma.co>`_ to learn about Catalyst, or refer to the 
-`whitepaper <https://www.enigma.co/enigma_catalyst.pdf>`_ for further technical details.
-
-Catalyst builds on top of the well-established `Zipline <https://github.com/quantopian/zipline>`_ project.
-We did our best to minimize structural changes to the general API to maximize compatibility with existing trading algorithms, developer knowledge, and tutorials.
-For now, please refer to the `Zipline API Docs <http://zipline.io>`_ as a general reference and bring any other questions you have to our #dev channel on `Slack <https://join.slack.com/enigmacatalyst/shared_invite/MTkzMjQ0MTg1NTczLTE0OTY3MjE3MDEtZGZmMTI5YzI3ZA>`_.
-
-Our primary contributions include the:
-
-- Introduction of an open trading calendar that permits simulation to allow trades on weekends, holidays, and outside of normal business hours.
-- Curation of OHLCV data bundle from `Poloniex's API <https://poloniex.com/support/api/>`_, which contains data in five-minute intervals as early as 2/19/2015.
-- Support for backtesting of daily trading strategies, support for five-minute backtesting is in development.
-- Addition of Bitcoin price (USDT_BTC) as a benchmark asset for comparing performance.
-
-Interested in getting involved?
-`Join us on Slack! <https://join.slack.com/enigmacatalyst/shared_invite/MTkzMjQ0MTg1NTczLTE0OTY3MjE3MDEtZGZmMTI5YzI3ZA>`_
-
-
-Installation
-============
-
-At the moment, Catalyst has some fairly specific and strict depedency requirements.
-We recommend the use of Python virtual environments if you wish to simplify the installation process, or otherwise isolate Catalyst's dependencies from your other projects.
-If you don't have ``virtualenv`` installed, see our later section on Virtual Environments.
-
-.. code-block:: bash
-
-    $ virtualenv catalyst-venv
-    $ source ./catalyst-venv/bin/activate
-    $ pip install enigma-catalyst
-
-**Note:** A successful installation will require several minutes in order to compile dependencies that expose C APIs.
-
-Dependencies
-------------
-
-Catalyst's depedencies can be found in the ``etc/requirements.txt`` file.
-If you need to install them outside of a typical ``pip install``, this is done using:
-
-.. code-block:: bash
-
-    $ pip install -r etc/requirements.txt
-
-Though not required by Catalyst directly, our example algorithms use matplotlib to visually display backtest results.
-If you wish to run any examples or use matplotlib during development, it can be installed using:
-
-.. code-block:: bash
-
-    $ pip install matplotlib
-
-**Note:** If you plan to use matplotlib and virtualenv on Mac OS X, see our later section for additional setup instructions.
-
-Getting Started
-===============
-
-The following code implements a simple buy and hold algorithm. The full source can be found in ``catalyst/examples/buy_and_hodl.py``.
-
-.. code:: python
-
-    import numpy as np
-    
-    from catalyst.api import (
-        order_target_value,
-        symbol,
-        record,
-        cancel_order,
-        get_open_orders,
-    )
-    
-    ASSET = 'USDT_BTC'
-    
-    TARGET_HODL_RATIO = 0.8
-    RESERVE_RATIO = 1.0 - TARGET_HODL_RATIO
-
-    def initialize(context):
-        context.is_buying = True
-        context.asset = symbol(ASSET)
-
-    def handle_data(context, data):
-        cash = context.portfolio.cash
-        target_hodl_value = TARGET_HODL_RATIO * context.portfolio.starting_cash
-        reserve_value = RESERVE_RATIO * context.portfolio.starting_cash
-        
-        # Cancel any outstanding orders from the previous day
-        orders = get_open_orders(context.asset) or []
-        for order in orders:
-            cancel_order(order)
-        
-        # Stop buying after passing reserve threshold
-        if cash <= reserve_value:
-            context.is_buying = False
-        
-        # Retrieve current price from pricing data
-        price = data[context.asset].price
-        
-        # Check if still buying and could (approximately) afford another purchase                    
-        if context.is_buying and cash > price:
-            # Place order to make position in asset equal to target_hodl_value
-            order_target_value(
-                context.asset,
-                target_hodl_value,
-                limit_price=1.1 * price,
-                stop_price=0.9 * price,
-            )
-        
-        # Record any state for later analysis
-        record(
-            price=price,
-            cash=context.portfolio.cash,
-            leverage=context.account.leverage,
-        )
-
-
-You can then run this algorithm using the Catalyst CLI. From the command
-line, run:
-
-.. code:: bash
-
-    $ catalyst ingest
-    $ catalyst run -f buy_and_hodl.py --start 2015-3-1 --end 2017-6-28 --capital-base 100000 -o bah.pickle
-
-This will download the crypto-asset price data from a poloniex bundle
-curated by Enigma in the specified time range and stream it through
-the algorithm and plot the resulting performance using matplotlib.
-
-You can find other examples in the ``catalyst/examples`` directory.
-
-Limitations
------------
-
-This project is currently in a pre-alpha state and has some limitations we'd like to address:
-
-- *Minimum Denomination:* The smallest tradable unit in Catalyst is equal to 1/1000th of a full coin. We plan to enable more granular increments, but have capped it at 1/1000th for the time being.
-- *Supported Assets:* Currently the poloniex bundle comes prepopulated with data for all 90 registered trading pairs. However, due to limitations in how portfolios are currently modeled, we recommend sticking to ``USDT_*`` trading pairs. USDT is an independent currency listed on Poloniex whose price is pegged to the US dollar. Currently, this list includes: ``USDT_BTC``, ``USDT_DASH``, ``USDT_ETC``, ``USDT_ETH``, ``USDT_LTC``, ``USDT_NXT``, ``USDT_REP``, ``USDT_STR``, ``USDT_XMR``, ``USDT_XRP``, and ``USDT_ZEC``. We plan to add support for basing your portfolio in arbitrary currencies and provide native support for modeling ForEx trades in the near future!
-
-Virtual Environments
-====================
-
-Here we will provide a brief tutorial for installing ``virtualenv`` and its basic usage.
-For more information regarding ``virtualenv``, please refer to this `virtualenv guide <http://python-guide-pt-br.readthedocs.io/en/latest/dev/virtualenvs/>`_.
-
-The ``virtualenv`` command can be installed using:
-
-.. code-block:: bash
-
-    $ pip install virtualenv
-
-To create a new virtual environment, choose a directory, e.g. ``/path/to/venv-dir``, where project-specific packages and files will be stored.  The environment is created by running:
-
-.. code-block:: bash
-
-    $ virtualenv /path/to/venv-dir
-
-To enter an environment, run the ``bin/activate`` script located in ``/path/to/venv-dir`` using:
-
-.. code-block:: bash
-
-    $ source /path/to/venv-dir/bin/activate
-
-Exiting an environment is accomplished using ``deactivate``, and removing it entirely is done by deleting ``/path/to/venv-dir``.
-
-OS X + virtualenv + matplotlib
--------------------------------------
-
-A note about using matplotlib in virtual enviroments on OS X: it may be necessary to run
-
-.. code-block:: python
-
-    echo "backend: TkAgg" > ~/.matplotlib/matplotlibrc
-
-in order to override the default ``macosx`` backend for your system, which may not be accessible from inside the virtual environment.
-This will allow Catalyst to open matplotlib charts from within a virtual environment, which is useful for displaying the performance of your backtests.  To learn more about matplotlib backends, please refer to the
-`matplotlib backend documentation <https://matplotlib.org/faq/usage_faq.html#what-is-a-backend>`_.
-
-Disclaimer
-==========
-
-Keep in mind that this project is still under active development, and is not recommended for production use in its current state.
-We are deeply committed to improving the overall user experience, reliability, and feature-set offered by Catalyst.
-If you have any suggestions, feedback, or general improvements regarding any of these topics, please let us know!
-
-Hello World,
-
-The Enigma Team
-
-.. |version status| image:: https://img.shields.io/pypi/pyversions/enigma-catalyst.svg
-   :target: https://pypi.python.org/pypi/enigma-catalyst
+All the documentation for `Catalyst <https://github.com/enigmampc/catalyst>`_ can be found in the `catalyst-docs wiki <https://github.com/enigmampc/catalyst-docs/wiki>`_.

catalyst/algorithm.py

@@ -742,15 +742,14 @@ class TradingAlgorithm(object):
             for perf in self.get_generator():
                 perfs.append(perf)
 
-          
             # convert perf dict to pandas dataframe
-            stats = self._create_daily_stats(perfs)
+            daily_stats = self._create_daily_stats(perfs)
 
-            self.analyze(stats)
+            self.analyze(daily_stats)
         finally:
             self.data_portal = None
 
-        return stats
+        return daily_stats
 
     def _write_and_map_id_index_to_sids(self, identifiers, as_of_date):
         # Build new Assets for identifiers that can't be resolved as
@@ -1141,12 +1140,14 @@ class TradingAlgorithm(object):
 
         date_rule = date_rule or date_rules.every_day()
         if freq is 'daily':
-            # Ignore any time rules in daily mode.
-            # every_minute in daily mode does nothing.
+            # ignore time rule in daily mode
             time_rule = time_rules.every_minute()
         else:
-            # use provided time rule or default to every minute
-            time_rule = time_rule or time_rules.every_minute()
+            # use provided time rule or default to every minute or 5 minutes
+            # based on desired data frequency.
+            time_rule = time_rule or (time_rules.every_5_minutes()
+                                      if freq is '5-minute' else
+                                      time_rules.every_minute())
 
         # Check the type of the algorithm's schedule before pulling calendar
         # Note that the ExchangeTradingSchedule is currently the only
@@ -1170,13 +1171,7 @@ class TradingAlgorithm(object):
             )
 
         self.add_event(
-            make_eventrule(
-                date_rule,
-                time_rule,
-                cal,
-                half_days=half_days,
-                data_frequency=self.data_frequency,
-            ),
+            make_eventrule(date_rule, time_rule, cal, half_days),
             func,
         )
 
@@ -1708,12 +1703,12 @@ class TradingAlgorithm(object):
         return dt
 
     @api_method
-    def set_slippage(self, equities=None, us_futures=None):
+    def set_slippage(self, us_equities=None, us_futures=None):
         """Set the slippage models for the simulation.
 
         Parameters
         ----------
-        equities : EquitySlippageModel
+        us_equities : EquitySlippageModel
             The slippage model to use for trading US equities.
         us_futures : FutureSlippageModel
             The slippage model to use for trading US futures.
@@ -1725,14 +1720,14 @@ class TradingAlgorithm(object):
         if self.initialized:
             raise SetSlippagePostInit()
 
-        if equities is not None:
-            if Equity not in equities.allowed_asset_types:
+        if us_equities is not None:
+            if Equity not in us_equities.allowed_asset_types:
                 raise IncompatibleSlippageModel(
                     asset_type='equities',
-                    given_model=equities,
-                    supported_asset_types=equities.allowed_asset_types,
+                    given_model=us_equities,
+                    supported_asset_types=us_equities.allowed_asset_types,
                 )
-            self.blotter.slippage_models[Equity] = equities
+            self.blotter.slippage_models[Equity] = us_equities
 
         if us_futures is not None:
             if Future not in us_futures.allowed_asset_types:
@@ -1744,12 +1739,12 @@ class TradingAlgorithm(object):
             self.blotter.slippage_models[Future] = us_futures
 
     @api_method
-    def set_commission(self, equities=None, us_futures=None):
+    def set_commission(self, us_equities=None, us_futures=None):
         """Sets the commission models for the simulation.
 
         Parameters
         ----------
-        equities : EquityCommissionModel
+        us_equities : EquityCommissionModel
             The commission model to use for trading US equities.
         us_futures : FutureCommissionModel
             The commission model to use for trading US futures.
@@ -1763,14 +1758,14 @@ class TradingAlgorithm(object):
         if self.initialized:
             raise SetCommissionPostInit()
 
-        if equities is not None:
-            if Equity not in equities.allowed_asset_types:
+        if us_equities is not None:
+            if Equity not in us_equities.allowed_asset_types:
                 raise IncompatibleCommissionModel(
                     asset_type='equities',
-                    given_model=equities,
-                    supported_asset_types=equities.allowed_asset_types,
+                    given_model=us_equities,
+                    supported_asset_types=us_equities.allowed_asset_types,
                 )
-            self.blotter.commission_models[Equity] = equities
+            self.blotter.commission_models[Equity] = us_equities
 
         if us_futures is not None:
             if Future not in us_futures.allowed_asset_types:

catalyst/data/_minute_bar_internal.pyx

@@ -44,7 +44,7 @@ def five_minute_value(ndarray[long_t, ndim=1] market_opens,
     q = cython.cdiv(pos, five_minutes_per_day)
     r = cython.cmod(pos, five_minutes_per_day)
 
-    return market_opens[q] + 5 * r
+    return market_opens[q] + r
 
 def find_position_of_minute(ndarray[long_t, ndim=1] market_opens,
                             ndarray[long_t, ndim=1] market_closes,
@@ -112,14 +112,10 @@ def find_position_of_five_minute(ndarray[long_t, ndim=1] market_opens,
     market_open = market_opens[market_open_loc]
     market_close = market_closes[market_open_loc]
 
-    val_open_offset = (five_minute_val - market_open)/5
-    close_open_offset = (market_close - market_open)/5
-
-    if not forward_fill and val_open_offset >= five_minutes_per_day:
+    if not forward_fill and ((five_minute_val - market_open) >= five_minutes_per_day):
         raise ValueError("Given five minutes is not between an open and a close")
 
-    # clamp offset to close index
-    delta = int_min(val_open_offset, close_open_offset)
+    delta = int_min(five_minute_val - market_open, market_close - market_open)
 
     return (market_open_loc * five_minutes_per_day) + delta
 

catalyst/data/bundles/base.py

@@ -172,6 +172,7 @@ class BaseBundle(object):
 
                 # Compile 5-minute symbol data if bundle supports 5-minute mode and
                 # persist the dataset to disk.
+                '''
                 if '5-minute' in self.frequencies:
                     five_minute_bar_writer.write(
                         self._fetch_symbol_iter(
@@ -187,6 +188,7 @@ class BaseBundle(object):
                         length=len(symbol_map),
                         show_progress=show_progress,
                     )
+                '''
 
                 # Compile minute symbol data if bundle supports minute mode and
                 # persist the dataset to disk.
@@ -296,14 +298,12 @@ class BaseBundle(object):
                     except Exception as e:
                         log.exception(
                             'Failed to load metadata from {}. '
-                            'Retrying.'.format(
-                                name=self.name,
-                            )
+                            'Retrying.'.format(self.name)
                         ) 
                 else:
                     raise ValueError(
-                        'Failed to download metadata page %d after %d '
-                        'attempts.'.format(page_number, retries),
+                        'Failed to download metadata page {} after {} '
+                        'attempts.'.format(page_number, retries)
                     )
 
 
@@ -313,7 +313,8 @@ class BaseBundle(object):
 
             # Apply selective asset filtering, useful for benchmark
             # ingestion.
-            raw = raw[raw.symbol.isin(self._asset_filter)]
+            if self._asset_filter:
+                raw = raw[raw.symbol.isin(self._asset_filter)]
 
             # Update cached value for key.
             cache[key] = raw

catalyst/data/bundles/poloniex.py

@@ -36,7 +36,7 @@ class PoloniexBundle(BaseCryptoPricingBundle):
     def frequencies(self):
         return set((
             'daily',
-            '5-minute',
+            #'5-minute',
         ))
 
     @lazyval
@@ -103,7 +103,7 @@ class PoloniexBundle(BaseCryptoPricingBundle):
         )
         raw.set_index('date', inplace=True)
 
-        scale = 1000.0
+        scale = 1
         raw.loc[:, 'open'] /= scale
         raw.loc[:, 'high'] /= scale
         raw.loc[:, 'low'] /= scale
@@ -132,7 +132,7 @@ class PoloniexBundle(BaseCryptoPricingBundle):
                          data_frequency):
         period_map = {
             'daily': 86400,
-            '5-minute': 300,
+#            '5-minute': 300,
         }
 
         try:
@@ -155,4 +155,13 @@ class PoloniexBundle(BaseCryptoPricingBundle):
             query=urlencode(query_params),
         )
 
-register_bundle(PoloniexBundle, ['USDT_BTC'])
+''' 
+As a second parameter, you can pass an array of currency pairs 
+that will be processed as an asset_filter to only process that 
+subset of assets in the bundle, such as:
+register_bundle(PoloniexBundle, ['USDT_BTC',])
+
+For a production environment make sure to use (to bundle all pairs):
+register_bundle(PoloniexBundle)
+'''
+register_bundle(PoloniexBundle)

catalyst/data/data_portal.py

@@ -289,7 +289,7 @@ class DataPortal(object):
 
         self._daily_aggregator = DailyHistoryAggregator(
             self.trading_calendar.schedule.market_open,
-            _dispatch_session_reader,
+            _dispatch_minute_reader,
             self.trading_calendar
         )
         self._history_loader = DailyHistoryLoader(

catalyst/data/five_minute_bars.py

@@ -60,7 +60,7 @@ OPEN_FIVE_MINUTES_PER_DAY = 288
 
 DEFAULT_EXPECTEDLEN_CRYPTO = OPEN_FIVE_MINUTES_PER_DAY * 366 * 15
 
-OHLC_RATIO = 1000
+OHLC_RATIO = 1000000
 
 OHLC = frozenset(['open', 'high', 'low', 'close'])
 OHLCV = frozenset(['open', 'high', 'low', 'close', 'volume'])
@@ -1151,7 +1151,6 @@ class BcolzFiveMinuteBarReader(FiveMinuteBarReader):
 
         if field != 'volume':
             value *= self._ohlc_ratio_inverse_for_sid(sid)
-        #print 'minute pos: {}, {}: {}'.format(minute_pos, field, value)
         return value
 
     def get_last_traded_dt(self, asset, dt):
@@ -1162,8 +1161,8 @@ class BcolzFiveMinuteBarReader(FiveMinuteBarReader):
 
     def _find_last_traded_five_minute_position(self, asset, dt):
         volumes = self._open_minute_file('volume', asset)
-        start_date_minute = asset.start_date.value / NANOS_IN_MINUTE
-        dt_minute = dt.value / NANOS_IN_MINUTE
+        start_date_minute = asset.start_date.value / NANOS_IN_FIVE_MINUTE
+        dt_minute = dt.value / NANOS_IN_FIVE_MINUTE
 
         try:
             # if we know of a dt before which this asset has no volume,
@@ -1228,7 +1227,7 @@ class BcolzFiveMinuteBarReader(FiveMinuteBarReader):
         return find_position_of_five_minute(
             self._market_open_values,
             self._market_close_values,
-            minute_dt.value / NANOS_IN_MINUTE,
+            minute_dt.value / NANOS_IN_FIVE_MINUTE,
             self._five_minutes_per_day,
             False,
         )
@@ -1253,19 +1252,11 @@ class BcolzFiveMinuteBarReader(FiveMinuteBarReader):
             (minutes in range, sids) with a dtype of float64, containing the
             values for the respective field over start and end dt range.
         """
-        print 'start_dt:', start_dt
-        print 'end_dt:', end_dt
-
         start_idx = self._find_position_of_five_minute(start_dt)
         end_idx = self._find_position_of_five_minute(end_dt)
 
-        print 'start_idx:', start_idx
-        print 'end_idex:', end_idx
-
         num_minutes = (end_idx - start_idx + 1)
 
-        print 'num_minutes:', num_minutes
-
         results = []
 
         indices_to_exclude = self._exclusion_indices_for_range(
@@ -1302,8 +1293,6 @@ class BcolzFiveMinuteBarReader(FiveMinuteBarReader):
                     out[:len(where), i][where] = values[where]
 
             results.append(out)
-
-        print 'results:', results
         return results
 
 

catalyst/data/loader.py

@@ -93,8 +93,8 @@ def has_data_for_dates(series_or_df, first_date, last_date):
     dts = series_or_df.index
     if not isinstance(dts, pd.DatetimeIndex):
         raise TypeError("Expected a DatetimeIndex, but got %s." % type(dts))
-    first, last = dts[[0, -1]]
-    return (first <= first_date) and (last >= last_date)
+    first, last = dts[[0, -1]].tz_localize(None)
+    return (first <= first_date.tz_localize(None)) and (last >= last_date.tz_localize(None))
 
 def load_crypto_market_data(trading_day=None,
                             trading_days=None,
@@ -134,17 +134,19 @@ def load_crypto_market_data(trading_day=None,
         trading_day,
         environ,
     )
+    # Override first_date for treasury data since we have it for many more years
+    # and is independent of crypto data
+    first_date_treasury = pd.Timestamp('1990-01-01', tz='UTC') 
     tc = ensure_treasury_data(
         bm_symbol,
-        first_date,
+        first_date_treasury,
         last_date,
         now,
         environ,
     )
     benchmark_returns = br[br.index.slice_indexer(first_date, last_date)]
-    treasury_curves = tc[tc.index.slice_indexer(first_date, last_date)]
+    treasury_curves = tc[tc.index.slice_indexer(first_date_treasury, last_date)]
     return benchmark_returns, treasury_curves
-    
 
 
 def load_market_data(trading_day=None, trading_days=None, bm_symbol='SPY',
@@ -232,6 +234,7 @@ def load_market_data(trading_day=None, trading_days=None, bm_symbol='SPY',
     treasury_curves = tc[tc.index.slice_indexer(first_date, last_date)]
     return benchmark_returns, treasury_curves
 
+
 def ensure_crypto_benchmark_data(symbol,
                                  first_date,
                                  last_date,
@@ -279,7 +282,7 @@ def ensure_crypto_benchmark_data(symbol,
             None,
             symbol,
             get_calendar(bundle.calendar_name),
-            first_date,
+            first_date - trading_day,
             last_date,
             'daily',
         )
@@ -364,6 +367,7 @@ def ensure_benchmark_data(symbol, first_date, last_date, now, trading_day,
         logger.warn("Still don't have expected data after redownload!")
     return data
 
+
 def ensure_benchmark_data(symbol, first_date, last_date, now, trading_day,
                           environ=None):
     """
@@ -478,11 +482,6 @@ def ensure_treasury_data(symbol, first_date, last_date, now, environ=None):
 
 def _load_cached_data(filename, first_date, last_date, now, resource_name,
                       environ=None):
-    if resource_name == 'benchmark':
-        from_csv = pd.Series.from_csv
-    else:
-        from_csv = pd.DataFrame.from_csv
-
     # Path for the cache.
     path = get_data_filepath(filename, environ)
 
@@ -490,8 +489,10 @@ def _load_cached_data(filename, first_date, last_date, now, resource_name,
     # yet, so don't try to read from 'path'.
     if os.path.exists(path):
         try:
-            data = from_csv(path)
-            data.index = pd.to_datetime(data.index).tz_localize('UTC')
+            data = pd.DataFrame.from_csv(path)
+            if data.empty:
+                raise ValueError("File is empty.")
+            data.index = pd.to_datetime(data.index, infer_datetime_format=True, errors='coerce' ).tz_localize('UTC')
             if has_data_for_dates(data, first_date, last_date):
                 return data
 

catalyst/data/us_equity_pricing.py

@@ -763,7 +763,7 @@ class BcolzDailyBarReader(SessionBarReader):
             if price == 0:
                 return nan
             else:
-                return price * 0.000001
+                return price * 0.001
         else:
             return price
 

catalyst/examples/buy_and_hodl.5min.py

@@ -1,143 +0,0 @@
-#!/usr/bin/env python
-#
-# Copyright 2017 Enigma MPC, Inc.
-# Copyright 2015 Quantopian, Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from catalyst.finance.slippage import VolumeShareSlippage
-
-from catalyst.api import (
-    order_target_value,
-    symbol,
-    record,
-    cancel_order,
-    get_open_orders,
-    set_slippage,
-)
-
-
-def initialize(context):
-    context.ASSET_NAME = 'USDT_BTC'
-    context.TARGET_HODL_RATIO = 0.8
-    context.RESERVE_RATIO = 1.0 - context.TARGET_HODL_RATIO
-
-    # For all trading pairs in the poloniex bundle, the default denomination
-    # currently supported by Catalyst is 1/1000th of a full coin. Use this
-    # constant to scale the price of up to that of a full coin if desired.
-    context.TICK_SIZE = 1000.0
-
-    context.is_buying = True
-    context.asset = symbol(context.ASSET_NAME)
-
-    context.i = 0
-    
-    set_slippage(equities=VolumeShareSlippage(volume_limit=0.1))
-
-def handle_data(context, data):
-    context.i += 1
-
-    starting_cash = context.portfolio.starting_cash
-    target_hodl_value = context.TARGET_HODL_RATIO * starting_cash
-    reserve_value = context.RESERVE_RATIO * starting_cash
-
-    # Cancel any outstanding orders
-    orders = get_open_orders(context.asset) or []
-    for order in orders:
-        cancel_order(order)
-    
-    # Stop buying after passing the reserve threshold
-    cash = context.portfolio.cash
-    if cash <= reserve_value:
-        context.is_buying = False
-
-    # Retrieve current asset price from pricing data
-    price = data[context.asset].price
-
-    # Check if still buying and could (approximately) afford another purchase
-    if context.is_buying and cash > price:
-        # Place order to make position in asset equal to target_hodl_value
-        order_target_value(
-            context.asset,
-            target_hodl_value,
-            limit_price=price*1.1,
-            stop_price=price*0.9,
-        )
-
-    record(
-        price=price,
-        volume=data[context.asset].volume,
-        cash=cash,
-        starting_cash=context.portfolio.starting_cash,
-        leverage=context.account.leverage,
-    )
-
-def analyze(context=None, results=None):
-    import matplotlib.pyplot as plt
-
-    # Plot the portfolio and asset data.
-    ax1 = plt.subplot(611)
-    results[['portfolio_value']].plot(ax=ax1)
-    ax1.set_ylabel('Portfolio Value (USD)')
-
-    ax2 = plt.subplot(612, sharex=ax1)
-    ax2.set_ylabel('{asset} (USD)'.format(asset=context.ASSET_NAME))
-    (context.TICK_SIZE * results[['price']]).plot(ax=ax2)
-
-    trans = results.ix[[t != [] for t in results.transactions]]
-    buys = trans.ix[
-        [t[0]['amount'] > 0 for t in trans.transactions]
-    ]
-    ax2.plot(
-        buys.index,
-        context.TICK_SIZE * results.price[buys.index],
-        '^',
-        markersize=10,
-        color='g',
-    )
-
-    ax3 = plt.subplot(613, sharex=ax1)
-    results[['leverage', 'alpha', 'beta']].plot(ax=ax3)
-    ax3.set_ylabel('Leverage ')
-
-    ax4 = plt.subplot(614, sharex=ax1)
-    results[['starting_cash', 'cash']].plot(ax=ax4)
-    ax4.set_ylabel('Cash (USD)')
-
-    results[[
-        'treasury',
-        'algorithm',
-        'benchmark',
-    ]] = results[[
-        'treasury_period_return',
-        'algorithm_period_return',
-        'benchmark_period_return',
-    ]]
-
-    ax5 = plt.subplot(615, sharex=ax1)
-    results[[
-        'treasury',
-        'algorithm',
-        'benchmark',
-    ]].plot(ax=ax5)
-    ax5.set_ylabel('Percent Change')
-
-    ax6 = plt.subplot(616, sharex=ax1)
-    (results[['volume']] / context.TICK_SIZE).plot(ax=ax6)
-    ax6.set_ylabel('Volume (mCoins/5min)')
-
-    plt.legend(loc=3)
-
-    # Show the plot.
-    plt.gcf().set_size_inches(18, 8)
-    plt.show()

catalyst/examples/buy_and_hodl.py

@@ -15,8 +15,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from catalyst.finance.slippage import VolumeShareSlippage
-
 from catalyst.api import (
     order_target_value,
     symbol,
@@ -25,6 +23,7 @@ from catalyst.api import (
     get_open_orders,
 )
 
+
 def initialize(context):
     context.ASSET_NAME = 'USDT_BTC'
     context.TARGET_HODL_RATIO = 0.8
@@ -43,6 +42,8 @@ def initialize(context):
 def handle_data(context, data):
     context.i += 1
 
+    print 'i:', context.i
+
     starting_cash = context.portfolio.starting_cash
     target_hodl_value = context.TARGET_HODL_RATIO * starting_cash
     reserve_value = context.RESERVE_RATIO * starting_cash
@@ -72,7 +73,6 @@ def handle_data(context, data):
 
     record(
         price=price,
-        volume=data[context.asset].volume,
         cash=cash,
         starting_cash=context.portfolio.starting_cash,
         leverage=context.account.leverage,
@@ -80,13 +80,12 @@ def handle_data(context, data):
 
 def analyze(context=None, results=None):
     import matplotlib.pyplot as plt
-
     # Plot the portfolio and asset data.
-    ax1 = plt.subplot(611)
+    ax1 = plt.subplot(511)
     results[['portfolio_value']].plot(ax=ax1)
     ax1.set_ylabel('Portfolio Value (USD)')
 
-    ax2 = plt.subplot(612, sharex=ax1)
+    ax2 = plt.subplot(512, sharex=ax1)
     ax2.set_ylabel('{asset} (USD)'.format(asset=context.ASSET_NAME))
     (context.TICK_SIZE * results[['price']]).plot(ax=ax2)
 
@@ -102,11 +101,11 @@ def analyze(context=None, results=None):
         color='g',
     )
 
-    ax3 = plt.subplot(613, sharex=ax1)
+    ax3 = plt.subplot(513, sharex=ax1)
     results[['leverage', 'alpha', 'beta']].plot(ax=ax3)
     ax3.set_ylabel('Leverage ')
 
-    ax4 = plt.subplot(614, sharex=ax1)
+    ax4 = plt.subplot(514, sharex=ax1)
     results[['starting_cash', 'cash']].plot(ax=ax4)
     ax4.set_ylabel('Cash (USD)')
 
@@ -120,7 +119,7 @@ def analyze(context=None, results=None):
         'benchmark_period_return',
     ]]
 
-    ax5 = plt.subplot(615, sharex=ax1)
+    ax5 = plt.subplot(515, sharex=ax1)
     results[[
         'treasury',
         'algorithm',
@@ -128,10 +127,6 @@ def analyze(context=None, results=None):
     ]].plot(ax=ax5)
     ax5.set_ylabel('Percent Change')
 
-    ax6 = plt.subplot(616, sharex=ax1)
-    (results[['volume']] / context.TICK_SIZE).plot(ax=ax6)
-    ax6.set_ylabel('Volume (mCoins/5min)')
-
     plt.legend(loc=3)
 
     # Show the plot.

catalyst/examples/dual_vwap.5min.py

@@ -1,189 +0,0 @@
-#!/usr/bin/env python
-#
-# Copyright 2017 Enigma MPC, Inc.
-# Copyright 2014 Quantopian, Inc.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from catalyst.api import (
-    order_target_percent,
-    record,
-    symbol,
-    get_open_orders,
-    set_max_leverage,
-    schedule_function,
-    date_rules,
-    time_rules,
-    attach_pipeline,
-    pipeline_output,
-)
-
-from catalyst.pipeline import Pipeline
-from catalyst.pipeline.data import CryptoPricing
-from catalyst.pipeline.factors.crypto import VWAP
-
-
-def initialize(context):
-    context.ASSET_NAME = 'USDT_BTC'
-    context.TARGET_INVESTMENT_RATIO = 0.8
-    context.SHORT_WINDOW = 30 * 288
-    context.LONG_WINDOW = 100 * 288
-
-    # For all trading pairs in the poloniex bundle, the default denomination
-    # currently supported by Catalyst is 1/1000th of a full coin. Use this
-    # constant to scale the price of up to that of a full coin if desired.
-    context.TICK_SIZE = 1000.0
-
-    context.i = 0
-    context.asset = symbol(context.ASSET_NAME)
-
-    set_max_leverage(1.0)
-
-    attach_pipeline(make_pipeline(context), 'vwap_pipeline')
-
-    schedule_function(
-        rebalance,
-        time_rule=time_rules.every_minute(),
-    )
-
-
-def before_trading_start(context, data):
-    context.pipeline_data = pipeline_output('vwap_pipeline')
-
-def make_pipeline(context):
-    return Pipeline(
-        columns={
-            'price': CryptoPricing.open.latest,
-            'volume': CryptoPricing.volume.latest,
-            'short_mavg': VWAP(window_length=context.SHORT_WINDOW),
-            'long_mavg': VWAP(window_length=context.LONG_WINDOW),
-        }
-    )
-
-def rebalance(context, data):
-    context.i += 1
-
-    # skip first LONG_WINDOW bars to fill windows
-    if context.i < context.LONG_WINDOW:
-        return
-
-    # get pipeline data for asset of interest
-    pipeline_data = context.pipeline_data
-    pipeline_data = pipeline_data[pipeline_data.index == context.asset].iloc[0]
-
-    # retrieve long and short moving averages from pipeline
-    short_mavg = pipeline_data.short_mavg
-    long_mavg = pipeline_data.long_mavg
-    price = pipeline_data.price
-    volume = pipeline_data.volume
-
-    # check that order has not already been placed
-    open_orders = get_open_orders()
-    if context.asset not in open_orders:
-        # check that the asset of interest can currently be traded
-        if data.can_trade(context.asset):
-            # adjust portfolio based on comparison of long and short vwap
-            if short_mavg > long_mavg:
-                order_target_percent(
-                    context.asset,
-                    context.TARGET_INVESTMENT_RATIO,
-                )
-            elif short_mavg < long_mavg:
-                order_target_percent(
-                    context.asset,
-                    0.0,
-                )
-
-    record(
-        price=price,
-        cash=context.portfolio.cash,
-        leverage=context.account.leverage,
-        short_mavg=short_mavg,
-        long_mavg=long_mavg,
-        volume=volume,
-    )
-    
-
-
-def analyze(context=None, results=None):
-    import matplotlib.pyplot as plt
-
-    # Plot the portfolio and asset data.
-    ax1 = plt.subplot(611)
-    results[['portfolio_value']].plot(ax=ax1)
-    ax1.set_ylabel('Portfolio value (USD)')
-
-    ax2 = plt.subplot(612, sharex=ax1)
-    ax2.set_ylabel('{asset} (USD)'.format(asset=context.ASSET_NAME))
-    (context.TICK_SIZE*results[['price', 'short_mavg', 'long_mavg']]).plot(ax=ax2)
-
-    trans = results.ix[[t != [] for t in results.transactions]]
-    amounts = [t[0]['amount'] for t in trans.transactions]
-
-    buys = trans.ix[
-        [t[0]['amount'] > 0 for t in trans.transactions]
-    ]
-    sells = trans.ix[
-        [t[0]['amount'] < 0 for t in trans.transactions]
-    ]
-
-    ax2.plot(
-        buys.index,
-        context.TICK_SIZE * results.price[buys.index],
-        '^',
-        markersize=10,
-        color='g',
-    )
-    ax2.plot(
-        sells.index,
-        context.TICK_SIZE * results.price[sells.index],
-        'v',
-        markersize=10,
-        color='r',
-    )
-
-    ax3 = plt.subplot(613, sharex=ax1)
-    results[['leverage', 'alpha', 'beta']].plot(ax=ax3)
-    ax3.set_ylabel('Leverage (USD)')
-
-    ax4 = plt.subplot(614, sharex=ax1)
-    results[['cash']].plot(ax=ax4)
-    ax4.set_ylabel('Cash (USD)')
-
-    results[[
-        'treasury',
-        'algorithm',
-        'benchmark',
-    ]] = results[[
-        'treasury_period_return',
-        'algorithm_period_return',
-        'benchmark_period_return',
-    ]]
-
-    ax5 = plt.subplot(615, sharex=ax1)
-    results[[
-        'treasury',
-        'algorithm',
-        'benchmark',
-    ]].plot(ax=ax5)
-    ax5.set_ylabel('Percent Change')
-
-    ax6 = plt.subplot(616, sharex=ax1)
-    (results[['volume']] / context.TICK_SIZE).plot(ax=ax6)
-    ax6.set_ylabel('Volume (mBTC/day)')
-
-    plt.legend(loc=3)
-
-    # Show the plot.
-    plt.gcf().set_size_inches(18, 8)
-    plt.show()

catalyst/examples/dual_vwap.py

@@ -52,7 +52,7 @@ def initialize(context):
 
     schedule_function(
         rebalance,
-        date_rule=date_rules.every_day(),
+        time_rules=times_rules.every_minute(),
     )
 
 
@@ -178,7 +178,7 @@ def analyze(context=None, results=None):
     ax5.set_ylabel('Percent Change')
 
     ax6 = plt.subplot(616, sharex=ax1)
-    (results[['volume']] / context.TICK_SIZE).plot(ax=ax6)
+    results[['volume']].plot(ax=ax6)
     ax6.set_ylabel('Volume (mBTC/day)')
 
     plt.legend(loc=3)

catalyst/finance/performance/tracker.py

@@ -189,14 +189,14 @@ class PerformanceTracker(object):
 
     @property
     def progress(self):
-        if self.emission_rate in set(('minute', '5-minute')):
+        if self.emission_rate == 'minute':
             # Fake a value
             return 1.0
         elif self.emission_rate == 'daily':
             return self.session_count / self.total_session_count
 
     def set_date(self, date):
-        if self.emission_rate in set(('minute', '5-minute')):
+        if self.emission_rate == 'minute':
             self.saved_dt = date
             self.todays_performance.period_close = self.saved_dt
 
@@ -370,9 +370,7 @@ class PerformanceTracker(object):
                                             bench_since_open,
                                             account.leverage)
 
-        assert self.emission_rate in set(('minute', '5-minute'))
         minute_packet = self.to_dict(emission_type='minute')
-
         return minute_packet
 
     def handle_market_close(self, dt, data_portal):

catalyst/finance/risk/risk.py

@@ -158,7 +158,7 @@ def choose_treasury(select_treasury, treasury_curves, start_session,
                 )
                 break
 
-        if search_day:
+        if search_day and trading_calendar.name != 'OPEN':          # Supress warning for 'OPEN' calendar
             if (search_dist is None or search_dist > 1) and \
                     search_days[0] <= end_session <= search_days[-1]:
                 message = "No rate within 1 trading day of end date = \

catalyst/pipeline/loaders/crypto_pricing_loader.py

@@ -45,7 +45,7 @@ class CryptoPricingLoader(PipelineLoader):
             reader = bundle.five_minute_bar_reader
             all_sessions = cal.all_five_minutes
 
-        elif daily_bar_reader == 'minute':
+        elif data_frequency == 'minute':
             reader = bundle.minute_bar_reader
             all_sessions = cal.all_minutes
 
@@ -57,7 +57,6 @@ class CryptoPricingLoader(PipelineLoader):
         self.raw_price_loader = reader
         self._columns = dataset.columns
         self._all_sessions = all_sessions
-        self._data_frequency = data_frequency
 
     @classmethod
     def from_files(cls, pricing_path):
@@ -107,6 +106,7 @@ class CryptoPricingLoader(PipelineLoader):
 
 
 def _shift_dates(dates, start_date, end_date, shift):
+
     try:
         start = dates.get_loc(start_date)
     except KeyError:

catalyst/sources/benchmark_source.py

@@ -51,10 +51,7 @@ class BenchmarkSource(object):
         elif benchmark_returns is not None:
             daily_series = benchmark_returns[sessions[0]:sessions[-1]]
 
-            print 'BENCHMARK_RETURNS'
-
             if self.emission_rate == "minute":
-                print 'BENCHMARK_RETURNS minute'
                 # we need to take the env's benchmark returns, which are daily,
                 # and resample them to minute
                 minutes = trading_calendar.minutes_for_sessions_in_range(
@@ -69,7 +66,6 @@ class BenchmarkSource(object):
 
                 self._precalculated_series = minute_series
             elif self.emission_rate == '5-minute':
-                print 'BENCHMARK_RETURNS 5-minute'
                 five_minutes = \
                     trading_calendar.five_minutes_for_sessions_in_range(
                         sessions[0],
@@ -83,7 +79,6 @@ class BenchmarkSource(object):
                 
                 self._precalculated_series = five_minute_series
             else:
-                print 'BENCHMARK_RETURNS daily'
                 self._precalculated_series = daily_series
         else:
             raise Exception("Must provide either benchmark_asset or "
@@ -190,7 +185,6 @@ class BenchmarkSource(object):
 
             return benchmark_series.pct_change()[1:]
         else:
-            print '----------------------------------------'
             start_date = asset.start_date
             if start_date < trading_days[0]:
                 # get the window of close prices for benchmark_asset from the

catalyst/utils/calendars/exchange_calendar_open.py

@@ -1,6 +1,7 @@
 from datetime import time
 from pytz import timezone
 
+from pandas import Timestamp
 from pandas.tseries.offsets import DateOffset
 
 from catalyst.utils.memoize import lazyval
@@ -28,3 +29,6 @@ class OpenExchangeCalendar(TradingCalendar):
     @lazyval
     def day(self):
         return DateOffset(days=1)
+
+    def __init__(self, *args, **kwargs):
+        super(OpenExchangeCalendar, self).__init__(start=Timestamp('2015-03-01', tz='UTC'), **kwargs)

catalyst/utils/events.py

@@ -47,8 +47,6 @@ __all__ = [
     'NDaysBeforeLastTradingDayOfMonth',
     'StatefulRule',
     'OncePerDay',
-    'OncePerFiveMinutes',
-    'OncePerMinute',
 
     # Factory API
     'date_rules',
@@ -554,18 +552,15 @@ class StatefulRule(EventRule):
         """
         self.should_trigger = callable_
 
-class OncePerInterval(StatefulRule):
+
+class OncePerDay(StatefulRule):
     def __init__(self, rule=None):
         self.triggered = False
 
         self.date = None
         self.next_date = None
 
-        super(OncePerInterval, self).__init__(rule)
-
-    @lazyval
-    def interval(self):
-        raise NotImplementedError
+        super(OncePerDay, self).__init__(rule)
 
     def should_trigger(self, dt):
         if self.date is None or dt >= self.next_date:
@@ -575,28 +570,11 @@ class OncePerInterval(StatefulRule):
 
             # record the timestamp for the next day, so that we can use it
             # to know if we've moved to the next day
-            self.next_date = dt + self.interval
+            self.next_date = dt + pd.Timedelta(1, unit="d")
 
         if not self.triggered and self.rule.should_trigger(dt):
             self.triggered = True
             return True
-        
-
-
-class OncePerDay(OncePerInterval):
-    @lazyval
-    def interval(self):
-        return pd.Timedelta(1, unit='d')
-
-class OncePerFiveMinutes(OncePerInterval):
-    @lazyval
-    def interval(self):
-        return pd.Timedelta(5, unit='m')
-
-class OncePerMinute(OncePerInterval):
-    @lazyval
-    def interval(self):
-        return pd.Timedelta(1, unit='m')
 
 
 # Factory API
@@ -634,11 +612,7 @@ class calendars(object):
     US_FUTURES = sentinel('US_FUTURES')
 
 
-def make_eventrule(date_rule,
-                   time_rule,
-                   cal,
-                   half_days=True,
-                   data_frequency=None):
+def make_eventrule(date_rule, time_rule, cal, half_days=True):
     """
     Constructs an event rule from the factory api.
     """
@@ -654,15 +628,4 @@ def make_eventrule(date_rule,
         nhd_rule.cal = cal
         inner_rule = date_rule & time_rule & nhd_rule
 
-    if data_frequency == 'daily':
-        return OncePerDay(rule=inner_rule)
-    elif data_frequency == '5-minute':
-        return OncePerFiveMinutes(rule=inner_rule)
-    elif data_frequency == 'minute':
-        return OncePerMinute(rule=inner_rule)
-    else:
-        raise ValueError(
-            'Cannot make event rule for data frequency: {}'.format(
-                data_frequency,
-            )
-        )
+    return OncePerDay(rule=inner_rule)