diff --git a/catalyst/examples/buy_btc.py b/catalyst/examples/buy_btc.py deleted file mode 100644 index e23b79b0..00000000 --- a/catalyst/examples/buy_btc.py +++ /dev/null @@ -1,10 +0,0 @@ -from catalyst.api import order, record, symbol - - -def initialize(context): - context.asset = symbol('btc_usd') - - -def handle_data(context, data): - order(context.asset, 1) - record(btc=data.current(context.asset, 'price')) diff --git a/catalyst/examples/buy_btc_simple.py b/catalyst/examples/buy_btc_simple.py index 78888715..ef64513b 100644 --- a/catalyst/examples/buy_btc_simple.py +++ b/catalyst/examples/buy_btc_simple.py @@ -1,3 +1,24 @@ +''' + This is a very simple example referenced in the beginner's tutorial: + https://enigmampc.github.io/catalyst/beginner-tutorial.html + + Run this example, by executing the following from your terminal: + catalyst run -f buy_btc_simple.py -x bitfinex --start 2016-1-1 --end 2017-9-30 -o buy_btc_simple_out.pickle + + If you want to run this code using another exchange, make sure that + the asset is available on that exchange. For example, if you were to run + it for exchange Poloniex, you would need to edit the following line: + + context.asset = symbol('btc_usdt') # note 'usdt' instead of 'usd' + + and specify exchange poloniex as follows: + + catalyst run -f buy_btc_simple.py -x poloniex --start 2016-1-1 --end 2017-9-30 -o buy_btc_simple_out.pickle + + To see which assets are available on each exchange, visit: + https://www.enigma.co/catalyst/status +''' + from catalyst.api import order, record, symbol def initialize(context): diff --git a/docs/live-trading-wiki.md b/docs/live-trading-wiki.md deleted file mode 100644 index a7dbd9f1..00000000 --- a/docs/live-trading-wiki.md +++ /dev/null @@ -1,105 +0,0 @@ -

Live Trading

-This document explains how to get started with live trading. - -

Supported Exchanges

-Catalyst can trade against these exchanges: - -* Bitfinex, id=`bitfinex` -* Bittrex, id=`bittrex` - -

Authentication

-Most exchanges require key/token combination for authentication. By -convention, Catalyst uses an "auth.json" file to hold this data. - -This example illustrates the convention using the Bitfinex exchange. -Here is how to generate key and secret values for bitfinex: -https://docs.bitfinex.com/v1/docs/api-access. Most exchanges follow -a similar process. - -The auth.json file: -```json -{ - "name": "bitfinex", - "key": "my-key", - "secret": "my-secret" -} -``` - -The file goes here: -``` -~/.catalyst/data/exchanges/bitfinex/auth.json -``` - -Note that the 'bitfinex' directory corresponds to the id of the Bitfinex -exchange as defined in the "Supported Exchanges" section above. -Attempting to run an algorithm where the targeted exchange is missing -its "auth.json" file will create the directory structure but result -in an error. - -

Currency Symbols

-Catalyst introduces a universal convention to reference -trading pairs and individual currencies. This -is required to ensure that the `symbol()` api predictably -returns the correct asset regardless of the targeted exchange. - -Exchanges tend to use their own convention to represent currencies -(e.g. XBT and BTC both represent Bitcoin on different exchanges). -Trading pairs are also inconsistent. For example, Bitfinex -puts the market currency before the base currency without a -separator, Bittrex puts the base currency first and uses a dash -seperator. - -Here is the Catalyst convention: - -*[Market Currency]_[Base Currency]* all lowercase. - -Currency symbols (e.g. btc, eth, ltc) follow the Bittrex convention. - -Here are some examples: -```python -# With Bitfinex -bitcoin_usd_asset = symbol('btc_usd') -ethereum_bitcoin_asset = symbol('eth_btc') - -# With Bittrex -ethereum_bitcoin_asset = symbol('eth_btc') -neo_ethereum_asset = symbol('neo_eth) -``` - -Note that the trading pairs are always referenced in the same manner. -However, not all trading pairs are available on all exchanges. An -error will occur if the specified trading pair is not trading -on the exchange. - -

Trading an Algorithm

-There is no special convention to follow when writing an -algorithm for live trading. The same algorithm should work in -backtest and live execution mode without modification. - -What differs are the arguments provided to the catalyst client or -`run_algorithm()` interface. Here is example: - -```python -run_algorithm( - initialize=initialize, - handle_data=handle_data, - analyze=analyze, - exchange_name='bitfinex', - live=True, - algo_namespace='my_algo_trading_xrp', - base_currency='btc' -) -``` - -Here is the breakdown of the new arguments: -* live: Boolean flag which enables live trading. -* exchange_name: The name of the targeted exchange - (supported values: *bitfinex*, *bittrex*). -* algo_namespace: A arbitrary label assigned to your algorithm for - data storage purposes. -* base_currency: The base currency used to calculate the - statistics of your algorithm. Currently, the base currency of all - trading pairs of your algorithm must match this value. - -Here is a complete algorithm for reference: -[Buy Low and Sell High](../catalyst/examples/buy_low_sell_high_live.py) diff --git a/docs/source/beginner-tutorial.rst b/docs/source/beginner-tutorial.rst index a19374ee..6848a806 100644 --- a/docs/source/beginner-tutorial.rst +++ b/docs/source/beginner-tutorial.rst @@ -255,7 +255,7 @@ slippage model that ``catalyst`` uses). Let's take a quick look at the performance ``DataFrame``. For this, we use ``pandas`` from inside the IPython Notebook and print the first ten -rows. and print the first ten rows. Note that ``catalyst`` makes heavy usage of +rows. Note that ``catalyst`` makes heavy usage of `pandas `_, especially for data input and outputting so it's worth spending some time to learn it. @@ -429,6 +429,10 @@ and allows us to plot the price of bitcoin. For example, we could easily examine now how our portfolio value changed over time compared to the bitcoin price. +.. code-block:: python + + %load_ext catalyst + .. code-block:: python %pylab inline @@ -483,8 +487,13 @@ but note that you need to have minute-level data for using ``1m``). This is a function we use in the ``handle_data()`` section: .. code-block:: python + + %load_ext catalyst + +.. code-block:: python + + %%catalyst --start 2016-4-1 --end 2017-9-30 -x bitfinex - %%catalyst --start 2016-1-1 --end 2017-9-30 -x bitfinex -o dma.pickle from catalyst.api import order, record, symbol, order_target def initialize(context): @@ -492,16 +501,16 @@ a function we use in the ``handle_data()`` section: context.asset = symbol('btc_usd') def handle_data(context, data): - # Skip first 300 days to get full windows + # Skip first 150 days to get full windows context.i += 1 - if context.i < 300: + if context.i < 150: return # Compute averages # data.history() has to be called with the same params # from above and returns a pandas dataframe. - short_mavg = data.history(context.asset, 'price', bar_count=100, frequency="1d").mean() - long_mavg = data.history(context.asset, 'price', bar_count=300, frequency="1d").mean() + short_mavg = data.history(context.asset, 'price', bar_count=50, frequency="1d").mean() + long_mavg = data.history(context.asset, 'price', bar_count=150, frequency="1d").mean() # Trading logic if short_mavg > long_mavg: @@ -518,7 +527,7 @@ a function we use in the ``handle_data()`` section: def analyze(context, perf): import matplotlib.pyplot as plt - fig = plt.figure() + fig = plt.figure(figsize=(12,12)) ax1 = fig.add_subplot(211) perf.portfolio_value.plot(ax=ax1) ax1.set_ylabel('portfolio value in $') diff --git a/docs/source/development-guidelines.rst b/docs/source/development-guidelines.rst index d643090d..9bc6c7b0 100644 --- a/docs/source/development-guidelines.rst +++ b/docs/source/development-guidelines.rst @@ -1,21 +1,17 @@ Development Guidelines ====================== -This page is intended for developers of Zipline, people who want to contribute to the Zipline codebase or documentation, or people who want to install from source and make local changes to their copy of Zipline. +This page is intended for developers of Catalyst, people who want to contribute to the Catalyst codebase or documentation, or people who want to install from source and make local changes to their copy of Catalyst. -All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We `track issues`__ on `GitHub`__ and also have a `mailing list`__ where you can ask questions. - -__ https://github.com/quantopian/zipline/issues -__ https://github.com/ -__ https://groups.google.com/forum/#!forum/zipline +All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We `track issues `_ on `GitHub `_ and also have a `discord group `_ where you can ask questions. Creating a Development Environment ---------------------------------- -First, you'll need to clone Zipline by running: +First, you'll need to clone Catalyst by running: .. code-block:: bash - $ git clone git@github.com:your-github-username/zipline.git + $ git clone git@github.com:enigmampc/catalyst.git Then check out to a new branch where you can make your changes: @@ -23,15 +19,13 @@ Then check out to a new branch where you can make your changes: $ git checkout -b some-short-descriptive-name -If you don't already have them, you'll need some C library dependencies. You can follow the `install guide`__ to get the appropriate dependencies. - -__ install.html +If you don't already have them, you'll need some C library dependencies. You can follow the `install guide `_ to get the appropriate dependencies. The following section assumes you already have virtualenvwrapper and pip installed on your system. Suggested installation of Python library dependencies used for development: .. code-block:: bash - $ mkvirtualenv zipline + $ mkvirtualenv catalyst $ ./etc/ordered_pip.sh ./etc/requirements.txt $ pip install -r ./etc/requirements_dev.txt $ pip install -r ./etc/requirements_blaze.txt @@ -42,104 +36,39 @@ Finally, you can build the C extensions by running: $ python setup.py build_ext --inplace -To finish, make sure `tests`__ pass. +.. To finish, make sure `tests`__ pass. -__ #style-guide-running-tests +.. __ #style-guide-running-tests -If you get an error running nosetests after setting up a fresh virtualenv, please try running +.. If you get an error running nosetests after setting up a fresh virtualenv, please try running -.. code-block:: bash +.. code-block - # where zipline is the name of your virtualenv - $ deactivate zipline - $ workon zipline +.. # where zipline is the name of your virtualenv +.. $ deactivate zipline +.. $ workon zipline -Development with Docker +.. Development with Docker +.. ----------------------- + +..If you want to work with zipline using a `Docker`__ container, you'll need to build the ``Dockerfile`` in the Zipline root directory, and then build ``Dockerfile-dev``. Instructions for building both containers can be found in ``Dockerfile`` and ``Dockerfile-dev``, respectively. + +.. __ https://docs.docker.com/get-started/ + +Git Branching Structure ----------------------- -If you want to work with zipline using a `Docker`__ container, you'll need to build the ``Dockerfile`` in the Zipline root directory, and then build ``Dockerfile-dev``. Instructions for building both containers can be found in ``Dockerfile`` and ``Dockerfile-dev``, respectively. +If you want to contribute to the codebase of Catalyst, familiarize yourself with our branching structure, a fairly standardized one for that matter, that follows what is documented in the following article: `A successful Git branching model `_. To contribute, create your local branch and submit a Pull Request (PR) to the **develop** branch. -__ https://docs.docker.com/get-started/ +.. image:: https://camo.githubusercontent.com/9bde6fb64a9542a572e0e2017cbb58d9d2c440ac/687474703a2f2f6e7669652e636f6d2f696d672f6769742d6d6f64656c4032782e706e67 - -Style Guide & Running Tests ---------------------------- - -We use `flake8`__ for checking style requirements and `nosetests`__ to run Zipline tests. Our `continuous integration`__ tools will run these commands. - -__ http://flake8.pycqa.org/en/latest/ -__ http://nose.readthedocs.io/en/latest/ -__ https://en.wikipedia.org/wiki/Continuous_integration - -Before submitting patches or pull requests, please ensure that your changes pass when running: - -.. code-block:: bash - - $ flake8 zipline tests - -In order to run tests locally, you'll need `TA-lib`__, which you can install on Linux by running: - -__ https://mrjbq7.github.io/ta-lib/install.html - -.. code-block:: bash - - $ wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz - $ tar -xvzf ta-lib-0.4.0-src.tar.gz - $ cd ta-lib/ - $ ./configure --prefix=/usr - $ make - $ sudo make install - -And for ``TA-lib`` on OS X you can just run: - -.. code-block:: bash - - $ brew install ta-lib - -Then run ``pip install`` TA-lib: - -.. code-block:: bash - - $ pip install -r ./etc/requirements_talib.txt - -You should now be free to run tests: - -.. code-block:: bash - - $ nosetests - - -Continuous Integration ----------------------- - -We use `Travis CI`__ for Linux-64 bit builds and `AppVeyor`__ for Windows-64 bit builds. - -.. note:: - - We do not currently have CI for OSX-64 bit builds. 32-bit builds may work but are not included in our integration tests. - -__ https://travis-ci.org/quantopian/zipline -__ https://ci.appveyor.com/project/quantopian/zipline - - -Packaging ---------- -To learn about how we build Zipline conda packages, you can read `this`__ section in our release process notes. - -__ release-process.html#uploading-conda-packages - Contributing to the Docs ------------------------ -If you'd like to contribute to the documentation on zipline.io, you can navigate to ``docs/source/`` where each `reStructuredText`__ (``.rst``) file is a separate section there. To add a section, create a new file called ``some-descriptive-name.rst`` and add ``some-descriptive-name`` to ``appendix.rst``. To edit a section, simply open up one of the existing files, make your changes, and save them. - -__ https://en.wikipedia.org/wiki/ReStructuredText - -We use `Sphinx`__ to generate documentation for Zipline, which you will need to install by running: - -__ http://www.sphinx-doc.org/en/stable/ +If you'd like to contribute to the documentation on enigmampc.github.io, you can navigate to ``docs/source/`` where each `reStructuredText `_ file is a separate section there. To add a section, create a new file called ``some-descriptive-name.rst`` and add ``some-descriptive-name`` to ``index.rst``. To edit a section, simply open up one of the existing files, make your changes, and save them. +We use `Sphinx `_ to generate documentation for Catalyst, which you will need to install by running: .. code-block:: bash @@ -149,7 +78,7 @@ To build and view the docs locally, run: .. code-block:: bash - # assuming you're in the Zipline root directory + # assuming you're in the Catalyst root directory $ cd docs $ make html $ {BROWSER} build/html/index.html @@ -162,7 +91,7 @@ Standard prefixes to start a commit message: .. code-block:: text - BLD: change related to building Zipline + BLD: change related to building Catalyst BUG: bug fix DEP: deprecate something, or remove a deprecated object DEV: development tool or utility @@ -172,15 +101,13 @@ Standard prefixes to start a commit message: REV: revert an earlier commit STY: style fix (whitespace, PEP8, flake8, etc) TST: addition or modification of tests - REL: related to releasing Zipline + REL: related to releasing Catalyst PERF: performance enhancements Some commit style guidelines: -Commit lines should be no longer than `72 characters`__. The first line of the commit should include one of the above prefixes. There should be an empty line between the commit subject and the body of the commit. In general, the message should be in the imperative tense. Best practice is to include not only what the change is, but why the change was made. - -__ https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project +Commit lines should be no longer than `72 characters `_. The first line of the commit should include one of the above prefixes. There should be an empty line between the commit subject and the body of the commit. In general, the message should be in the imperative tense. Best practice is to include not only what the change is, but why the change was made. **Example:** @@ -203,8 +130,6 @@ __ https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project Formatting Docstrings --------------------- -When adding or editing docstrings for classes, functions, etc, we use `numpy`__ as the canonical reference. - -__ https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt +When adding or editing docstrings for classes, functions, etc, we use `numpy `_ as the canonical reference. diff --git a/docs/source/index.rst b/docs/source/index.rst index c3e0a19a..a4554bda 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -9,8 +9,10 @@ Table of Contents install beginner-tutorial + live-trading naming-convention videos + development-guidelines .. bundles .. development-guidelines .. appendix diff --git a/docs/source/install.rst b/docs/source/install.rst index 028f9295..34855618 100644 --- a/docs/source/install.rst +++ b/docs/source/install.rst @@ -39,9 +39,10 @@ version: .. code-block:: bash + $ pip install virtualenv $ virtualenv catalyst-venv $ source ./catalyst-venv/bin/activate - $ pip install enigma- + $ pip install enigma-catalyst Though not required by Catalyst directly, our example algorithms use matplotlib to visually display the results of the trading algorithms. If you wish to run @@ -132,6 +133,36 @@ it and install it before proceeding to the next step. For windows, the easiest and best supported way to install Catalyst is to use :ref:`Conda `. +Some problems we have encountered installing the **Visual C++ Compiler** mentioned above +are as follows: + +- **The system administrator has set policies to prevent this installation**. + + In some systems, there is a default *Windows Software Restriction* policy that + prevents the installation of some software packages like this one. You'll have + to change the Registry to circumvent this: + + - Click ``Start``, and search for ``regedit`` and launch the ``Registry Editor`` + - Navigate to the following folder: + ``HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Installer`` + - If there is an entry for ``DisableMSI``, set the Value data to 0. + - If there is no such entry, click on the ``Edit`` menu -> ``New`` -> ``DWORD (32-bit) Value`` + and enter ``DisableMSI`` as the Name (and by default you get 0 as the Value Data) + +| +- **The installer has encountered an unexpected error installing this package. + This may indicate a problem with this package. The error code is 2503.** + + We have observed this when trying to install a package without enough administrator + permissions. Even when you are logged in as an Administrator, you have to explictily + install this package with administrator privileges: + + - Click ``Start`` and find ``CMD`` or ``Command Prompt`` + - Right click on it and choose ``Run as administrator`` + - ``cd`` into the folder where you downloaded ``VCForPython27.msi`` + - Run ``msiexec /i VCForPython27.msi`` + + Amazon Linux AMI ~~~~~~~~~~~~~~~~ diff --git a/docs/source/live-trading.rst b/docs/source/live-trading.rst new file mode 100644 index 00000000..7a471d68 --- /dev/null +++ b/docs/source/live-trading.rst @@ -0,0 +1,118 @@ +Live Trading +============ +This document explains how to get started with live trading. + +Supported Exchanges +^^^^^^^^^^^^^^^^^^^ +Catalyst can trade against these exchanges: + +- Bitfinex, id= ``bitfinex`` +- Bittrex, id= ``bittrex`` +- Poloniex, id= ``poloniex`` + +Authentication +^^^^^^^^^^^^^^ +Most exchanges require token key/secret combination for authentication. By +convention, Catalyst uses an ``auth.json`` file to hold this data. + +This example illustrates the convention using the *Bitfinex* exchange. +Here is how to generate key and secret values for the Bitfinex exchange: +https://docs.bitfinex.com/v1/docs/api-access. Most exchanges follow +a similar process. + +The auth.json file: + +.. code-block:: json + + { + "name": "bitfinex", + "key": "my-key", + "secret": "my-secret" + } + + +The file goes here: ``~/.catalyst/data/exchanges/bitfinex/auth.json`` + +Note that the `bitfinex` part in the directory above corresponds to the id of the Bitfinex +exchange as defined in the "Supported Exchanges" section above. +Attempting to run an algorithm where the targeted exchange is missing +its ``auth.json`` file will create the directory structure and create an empty +auth.json file, but will result in an error. + +Currency Symbols +^^^^^^^^^^^^^^^^ +Catalyst introduces a universal convention to reference +trading pairs and individual currencies. This +is required to ensure that the ``symbol()`` api predictably +returns the correct asset regardless of the targeted exchange. + +Exchanges tend to use their own convention to represent currencies +(e.g. XBT and BTC both represent Bitcoin on different exchanges). +Trading pairs are also inconsistent. For example, Bitfinex +puts the market currency before the base currency without a +separator, Bittrex puts the base currency first and uses a dash +seperator. + +Here is the Catalyst convention: + +*[Market Currency]_[Base Currency]* all lowercase. + +Currency symbols (e.g. btc, eth, ltc) follow the Bittrex convention. + +Here are some examples: + +.. code-block:: json + + # With Bitfinex + bitcoin_usd_asset = symbol('btc_usd') + ethereum_bitcoin_asset = symbol('eth_btc') + + # With Bittrex + ethereum_bitcoin_asset = symbol('eth_btc') + neo_ethereum_asset = symbol('neo_eth) + +Note that the trading pairs are always referenced in the same manner. +However, not all trading pairs are available on all exchanges. An +error will occur if the specified trading pair is not trading +on the exchange. To check which currency pairs are available on each +of the supported exchanges, see `Catalyst Market Coverage `_ diff --git a/etc/python2.7-environment.yml b/etc/python2.7-environment.yml index 34fabcbb..d59c36ae 100644 --- a/etc/python2.7-environment.yml +++ b/etc/python2.7-environment.yml @@ -1,30 +1,22 @@ name: catalyst channels: -- statiskit - defaults dependencies: - certifi=2016.2.28=py27_0 -- coverage=4.4.1=py27_0 -- nose=1.3.7=py27_1 -- openssl=1.0.2l=0 -- path.py=10.3.1=py27_0 +- mkl=2017.0.3 +- numpy=1.13.1=py27_0 +- openssl=1.0.2l - pip=9.0.1=py27_1 -- python=2.7.13=0 -- pyyaml=3.12=py27_0 -- readline=6.2=2 -- setuptools=36.4.0=py27_0 -- six=1.10.0=py27_0 -- sqlite=3.13.0=0 -- tk=8.5.18=0 +- python=2.7.13 +- scipy=0.19.1=np113py27_0 +- setuptools=36.4.0=py27_1 +- sqlite=3.13.0 +- tk=8.5.18 - wheel=0.29.0=py27_0 -- yaml=0.1.6=0 -- zlib=1.2.11=0 -- libdev=1.0.0=py27_0 -- python-dev=1.0.0=py27_0 -- python-scons=3.0.0=py27_0 +- zlib=1.2.11 - pip: - - alembic==0.9.5 - - backports.shutil-get-terminal-size==1.0.0 + - alembic==0.9.6 + - backports.functools-lru-cache==1.4 - bcolz==0.12.1 - bottleneck==1.2.1 - chardet==3.0.4 @@ -32,36 +24,22 @@ dependencies: - contextlib2==0.5.5 - cycler==0.10.0 - cyordereddict==1.0.0 - - cython==0.26.1 + - cython==0.27.1 - decorator==4.1.2 - empyrical==0.2.1 - - enigma-catalyst>=0.2.dev2 - - enum34==1.1.6 - - functools32==3.2.3.post2 - idna==2.6 - intervaltree==2.1.0 - - ipdb==0.10.3 - - ipdbplugin==1.4.5 - - ipython==5.5.0 - - ipython-genutils==0.2.0 - logbook==1.1.0 - lru-dict==1.1.6 - mako==1.0.7 - markupsafe==1.0 - - matplotlib==2.0.2 + - matplotlib==2.1.0 - multipledispatch==0.4.9 - - networkx==1.11 + - networkx==2.0 - numexpr==2.6.4 - - numpy==1.13.1 - pandas==0.19.2 - pandas-datareader==0.5.0 - - pathlib2==2.3.0 - patsy==0.4.1 - - pexpect==4.2.1 - - pickleshare==0.7.4 - - prompt-toolkit==1.0.15 - - ptyprocess==0.5.2 - - pygments==2.2.0 - pyparsing==2.2.0 - python-dateutil==2.6.1 - python-editor==1.0.3 @@ -69,16 +47,12 @@ dependencies: - requests==2.18.4 - requests-file==1.4.2 - requests-ftp==0.3.1 - - scandir==1.5 - - scipy==0.19.1 - - scons==3.0.0a20170821 - - simplegeneric==0.8.1 + - six==1.11.0 - sortedcontainers==1.5.7 - sqlalchemy==1.1.14 - statsmodels==0.8.0 - subprocess32==3.2.7 - tables==3.4.2 - toolz==0.8.2 - - traitlets==4.3.2 - urllib3==1.22 - - wcwidth==0.1.7 + - enigma-catalyst>=0.3