DOC: release notes 0.5.8

BLD: Dockerfile[-dev]: s/quantopian/enigmampc/g

Dockerfile

@@ -1,11 +1,11 @@
 #
 # Dockerfile for an image with the currently checked out version of catalyst installed. To build:
 #
-#    docker build -t quantopian/catalyst .
+#    docker build -t enigmampc/catalyst .
 #
 # To run the container:
 #
-#    docker run -v /path/to/your/notebooks:/projects -v ~/.catalyst:/root/.catalyst -p 8888:8888/tcp --name catalyst -it quantopian/catalyst
+#    docker run -v /path/to/your/notebooks:/projects -v ~/.catalyst:/root/.catalyst -p 8888:8888/tcp --name catalyst -it enigmampc/catalyst
 #
 # To access Jupyter when running docker locally (you may need to add NAT rules):
 #

Dockerfile-dev

@@ -1,15 +1,15 @@
 #
 # Dockerfile for an image with the currently checked out version of catalyst installed. To build:
 #
-#    docker build -t quantopian/catalystdev -f Dockerfile-dev .
+#    docker build -t enigmampc/catalystdev -f Dockerfile-dev .
 #
-# Note: the dev build requires a quantopian/catalyst image, which you can build as follows:
+# Note: the dev build requires a enigmampc/catalyst image, which you can build as follows:
 #
-#    docker build -t quantopian/catalyst -f Dockerfile .
+#    docker build -t enigmampc/catalyst -f Dockerfile .
 #
 # To run the container:
 #
-#    docker run -v /path/to/your/notebooks:/projects -v ~/.catalyst:/root/.catalyst -p 8888:8888/tcp --name catalystdev -it quantopian/catalystdev
+#    docker run -v /path/to/your/notebooks:/projects -v ~/.catalyst:/root/.catalyst -p 8888:8888/tcp --name catalystdev -it enigmampc/catalystdev
 #
 # To access Jupyter when running docker locally (you may need to add NAT rules):
 #
@@ -25,7 +25,7 @@
 #
 #    docker exec -it catalystdev catalyst run -f /projects/my_algo.py --start 2015-1-1 --end 2016-1-1 /projects/result.pickle
 #
-FROM quantopian/catalyst
+FROM enigmampc/catalyst
 
 WORKDIR /catalyst
 

README.rst

@@ -5,6 +5,7 @@
 
 |version tag|
 |version status|
+|forum|
 |discord|
 |twitter|
 
@@ -22,9 +23,11 @@ visit `enigma.co <https://www.enigma.co>`_ to learn more about Catalyst.
 Catalyst builds on top of the well-established 
 `Zipline <https://github.com/quantopian/zipline>`_ project. We did our best to 
 minimize structural changes to the general API to maximize compatibility with 
-existing trading algorithms, developer knowledge, and tutorials. Join us on 
+existing trading algorithms, developer knowledge, and tutorials. Join us on the 
-`Discord <https://discord.gg/SJK32GY>`_ where we have a *#catalyst_dev* channel 
+`Catalyst Forum <https://catalyst.enigma.co/>`_ for questions around Catalyst, 
-for questions around Catalyst, algorithmic trading and technical support.
+algorithmic trading and technical support. We also have a 
+`Discord <https://discord.gg/SJK32GY>`_ group with the *#catalyst_dev* and 
+*#catalyst_setup* dedicated channels.
 
 Overview
 ========
@@ -61,6 +64,9 @@ Go to our `Documentation Website <https://enigmampc.github.io/catalyst/>`_.
 .. |version status| image:: https://img.shields.io/pypi/pyversions/enigma-catalyst.svg
    :target: https://pypi.python.org/pypi/enigma-catalyst
    
+.. |forum| image:: https://img.shields.io/badge/forum-join-green.svg
+   :target: https://catalyst.enigma.co/
+
 .. |discord| image:: https://img.shields.io/badge/discord-join%20chat-green.svg
    :target: https://discordapp.com/invite/SJK32GY
 

catalyst/assets/_assets.pyx

@@ -630,23 +630,28 @@ cdef class TradingPair(Asset):
         and whose second element is a tuple of all the attributes that should
         be serialized/deserialized during pickling.
         """
-        #TODO: make sure that all fields set there
+        # added arguments for catalyst
         return (self.__class__, (self.symbol,
                                  self.exchange,
                                  self.start_date,
                                  self.asset_name,
                                  self.sid,
                                  self.leverage,
+                                 self.end_daily,
+                                 self.end_minute,
                                  self.end_date,
+                                 self.exchange_symbol,
                                  self.first_traded,
                                  self.auto_close_date,
                                  self.exchange_full,
                                  self.min_trade_size,
                                  self.max_trade_size,
+                                 self.maker,
+                                 self.taker,
                                  self.lot,
                                  self.decimals,
-                                 self.taker,
+                                 self.trading_state,
-                                 self.maker))
+                                 self.data_source))
 
 def make_asset_array(int size, Asset asset):
     cdef np.ndarray out = np.empty([size], dtype=object)

catalyst/constants.py

@@ -24,8 +24,7 @@ AUTO_INGEST = False
 
 AUTH_SERVER = 'https://data.enigma.co'
 
-# TODO: switch to mainnet
+ETH_REMOTE_NODE = 'https://mainnet.infura.io'
-ETH_REMOTE_NODE = 'https://rinkeby.infura.io/'
 
 MARKETPLACE_CONTRACT = 'https://raw.githubusercontent.com/enigmampc/' \
                        'catalyst/master/catalyst/marketplace/' \
@@ -35,7 +34,6 @@ MARKETPLACE_CONTRACT_ABI = 'https://raw.githubusercontent.com/enigmampc/' \
                            'catalyst/master/catalyst/marketplace/' \
                            'contract_marketplace_abi.json'
 
-# TODO: switch to mainnet
 ENIGMA_CONTRACT = 'https://raw.githubusercontent.com/enigmampc/' \
                   'catalyst/master/catalyst/marketplace/' \
                   'contract_enigma_address.txt'

catalyst/marketplace/contract_enigma_address.txt

@@ -1 +1 @@
-0x39a54f480d922a58c963de8091a6c9afc69db2cf
+0xf0ee6b27b759c9893ce4f094b49ad28fd15a23e4

catalyst/marketplace/contract_marketplace_address.txt

@@ -1 +1 @@
-0xa2b37c6cd52f60fd4eb46ca59fafcf22d081aebc
+0xa64927358a82254be92eb1f1cb01de68d1787004

catalyst/marketplace/marketplace.py

@@ -20,7 +20,6 @@ from requests_toolbelt.multipart.decoder import \
 from catalyst.constants import (
     LOG_LEVEL, AUTH_SERVER, ETH_REMOTE_NODE, MARKETPLACE_CONTRACT,
     MARKETPLACE_CONTRACT_ABI, ENIGMA_CONTRACT, ENIGMA_CONTRACT_ABI)
-from catalyst.utils.cli import maybe_show_progress
 from catalyst.exchange.utils.stats_utils import set_print_settings
 from catalyst.marketplace.marketplace_errors import (
     MarketplacePubAddressEmpty, MarketplaceDatasetNotFound,
@@ -70,7 +69,10 @@ class Marketplace:
                 contract_url.info().get_content_charset()).strip())
 
         abi_url = urllib.urlopen(MARKETPLACE_CONTRACT_ABI)
-        abi = json.load(abi_url)
+        abi_url = abi_url.read().decode(
+                abi_url.info().get_content_charset())
+
+        abi = json.loads(abi_url)
 
         self.mkt_contract = self.web3.eth.contract(
             self.mkt_contract_address,
@@ -84,7 +86,10 @@ class Marketplace:
                 contract_url.info().get_content_charset()).strip())
 
         abi_url = urllib.urlopen(ENIGMA_CONTRACT_ABI)
-        abi = json.load(abi_url)
+        abi_url = abi_url.read().decode(
+                abi_url.info().get_content_charset())
+
+        abi = json.loads(abi_url)
 
         self.eng_contract = self.web3.eth.contract(
             self.eng_contract_address,
@@ -518,7 +523,7 @@ class Marketplace:
                 #     iter(decoder.parts),
                 #     True,
                 #     label='Processing files') as part:
-                counter = 0
+                counter = 1
                 for part in decoder.parts:
                     log.info("Processing file {} of {}".format(
                         counter, len(decoder.parts)))
@@ -638,7 +643,7 @@ class Marketplace:
     def register(self):
         while True:
             desc = input('Enter the name of the dataset to register: ')
-            dataset = desc.lower()
+            dataset = desc.lower().strip()
             provider_info = self.mkt_contract.functions.getDataProviderInfo(
                 Web3.toHex(dataset)
             ).call()
@@ -778,26 +783,32 @@ class Marketplace:
         else:
             key, secret = get_key_secret(provider_info[0], match['wallet'])
 
-        headers = get_signed_headers(dataset, key, secret)
         filenames = glob.glob(os.path.join(datadir, '*.csv'))
 
         if not filenames:
             raise MarketplaceNoCSVFiles(datadir=datadir)
 
         files = []
-        for file in filenames:
+        for idx, file in enumerate(filenames):
+            log.info('Uploading file {} of {}: {}'.format(
+                idx+1, len(filenames), file))
+            files = []
             files.append(('file', open(file, 'rb')))
 
-        r = requests.post('{}/marketplace/publish'.format(AUTH_SERVER),
+            headers = get_signed_headers(dataset, key, secret)
-                          files=files,
+            r = requests.post('{}/marketplace/publish'.format(AUTH_SERVER),
-                          headers=headers)
+                              files=files,
+                              headers=headers)
 
-        if r.status_code != 200:
+            if r.status_code != 200:
-            raise MarketplaceHTTPRequest(request='upload file',
+                raise MarketplaceHTTPRequest(request='upload file',
-                                         error=r.status_code)
+                                             error=r.status_code)
 
-        if 'error' in r.json():
+            if 'error' in r.json():
-            raise MarketplaceHTTPRequest(request='upload file',
+                raise MarketplaceHTTPRequest(request='upload file',
-                                         error=r.json()['error'])
+                                             error=r.json()['error'])
 
-        print('Dataset {} uploaded successfully.'.format(dataset))
+            log.info('File processed successfully.')
+
+        print('\nDataset {} uploaded and processed successfully.'.format(
+            dataset))

catalyst/marketplace/utils/auth_utils.py

@@ -123,7 +123,7 @@ def get_signed_headers(ds_name, key, secret):
     -------
 
     """
-    nonce = str(int(time.time()))
+    nonce = str(int(time.time() * 1000))
 
     signature = hmac.new(
         secret.encode('utf-8'),

docs/source/install.rst

@@ -562,6 +562,10 @@ If after following the instructions above, and going through the
 *Troubleshooting* sections, you still experience problems installing Catalyst,
 you can seek additional help through the following channels:
 
+- Join our `Catalyst Forum <https://catalyst.enigma.co/>`_, and browse a variety
+  of topics and conversations around common issues that others face when using
+  Catalyst, and how to resolve them. And join the conversation!
+
 - Join our `Discord community <https://discord.gg/SJK32GY>`_, and head over 
   the #catalyst_dev channel where many other users (as well as the project 
   developers) hang out, and can assist you with your particular issue. The 

docs/source/releases.rst

@@ -2,6 +2,30 @@
 Release Notes
 =============
 
+Version 0.5.8
+^^^^^^^^^^^^^
+**Release Date**: 2018-03-29
+
+Bug Fixes
+~~~~~~~~~
+- Fix proper release of Data Marketplace on mainnet.
+
+
+Version 0.5.7
+^^^^^^^^^^^^^
+**Release Date**: 2018-03-29
+
+Build
+~~~~~
+- Data Marketplace deployed on mainnet.
+- Added progress indicators for publishing data, and made the data publishing
+  synchronous to provide feedback to the publisher.
+
+Bug Fixes
+~~~~~~~~~
+- Added arguments to the ``reduce`` function in tha Asset class :issue:`214`, 
+  :issue:`287`
+
 Version 0.5.6
 ^^^^^^^^^^^^^
 **Release Date**: 2018-03-22
@@ -9,9 +33,9 @@ Version 0.5.6
 Build
 ~~~~~
 - Data Marketplace: ensures compatibility across wallets, now fully supporting 
-`ledger`, `trezor`, `keystore`, `private key`. Partial support for `metamask` 
+  ``ledger``, ``trezor``, ``keystore``, ``private key``. Partial support for 
-(includes sign_msg, but not sign_tx). Current support for `Digital Bitbox` is 
+  ``metamask`` (includes sign_msg, but not sign_tx). Current support for 
-unknown.
+  ``Digital Bitbox`` is unknown, but believed to be supported.
 - Data Marketplace: Switched online provider from MyEtherWallet to MyCrypto.
 - Data Marketplace: Added progress indicator for data ingestion.