wassname/libcryptomarket
1
0
Fork 0
mirror of https://github.com/wassname/libcryptomarket.git synced 2026-09-07 17:00:19 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #13 from libcryptomarket/feature/#12

Browse Source 
[#12] Refactor 0.1.4
This commit is contained in:
Gavin Chan
2018-01-03 19:07:31 +08:00
committed by GitHub
parent 790980c87f 6deb3d3479
commit 710e4c0d43
21 changed files with 207 additions and 1141 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+65 -34
View File
@@ -2,13 +2,15 @@
## Objective
The library is for researchers to analysis cryptocurrency in a fast and
flexible way. Currently there are different source of API to get the
cryptocurrency market information. The sources are from websites which provides
a general comparative information among the currencies and from exchanges. The
target is to normalize the API functions from data source, and let the users
query the data without pain.
* Support API calls of top cryptocurrency exchanges
* Support trading and analysis tools for cryptocurrency market
The project aims to answer the following two questions:
1. How to get historical data, especially a long period, in a single query?
2. How to send exchange API query in an elegent fashion?
## Prerequisite
@@ -28,53 +30,82 @@ or development version
pip install git+https://github.com/libcryptomarket/libcryptomarket.git
```
## Usage
## Exchanges supported
All the query result are converted into pandas Series or DataFrame.
| Exchange | Public API | Private API |
|---|---|---|
| [Bitfinex](https://docs.bitfinex.com/v2/docs/ws-general) | v | x |
| [BitMEX](https://www.bitmex.com/api/explorer/) | v | x |
| [CoinMarketCap](https://coinmarketcap.com/api/) | v | x |
| [GDAX](https://docs.gdax.com/#api) | v | x |
| [Poloniex](https://poloniex.com/support/api/) | v | v |
### Instrument
## Basic Usage
To get a list of available currencies, run
### Generic API
You can just make a simple call to query the same function from different
exchanges. For example, if you want to get the historical data from GDAX and
Bitfinex in a 5 minute timeframe, just call
```
from libcryptomarket.instrument import get_instruments
In [1]: from libcryptomarket.api import GdaxApi, BitfinexApi
instruments = get_instruments()
In [2]: from libcryptomarket.core import historical_ticker
In [3]: from datetime import datetime
In [4]: ticker1 = historical_ticker(source=GdaxApi(), symbol="BTC-USD", period=300, start_time=datetime(2017, 12, 30, 12, 0, 0), end_time=datetime(2017, 12, 31, 12, 0, 0))
In [5]: ticker2 = historical_ticker(source=BitfinexApi(), symbol="tBTCUSD", period="5m", start_time=datetime(2017, 12, 30, 12, 0, 0), end_time=datetime(2017, 12, 31, 12, 0, 0))
```
### Historical
The historical data query automatically rolls over if the exchange response
chunk a long period data. It can guarantee that you do not need to write your
own logic to get a long period of data.
Currently, multiple data sources of historical data are supported.
For example, for the source of [Cryptocompare](https://www.cryptocompare.com/api/#),
run
### Exchange API
You can easily initialize an exchange API client and call any method stated
in the exchange API official documentation. The return value is always
the native response from library `requests`.
For example, to get GDAX order book information,
```
from datetime import datetime
from libcryptomarket.historical import get_historical_prices
In [1]: from libcryptomarket.api import GdaxApi
prices = get_historical_prices(source='cryptocompare',
symbol='LTC/BTC',
exchange='Poloniex',
period="hour",
from_time=datetime(2017, 5, 1),
to_time=datetime(2017, 8, 1))
In [2]: exchange = GdaxApi()
In [3]: exchange.products_book(product_id="BTC-USD").json()
Out[3]:
{'asks': [['14903.01', '22.60310282', 18]],
'bids': [['14903', '3.4933731', 2]],
'sequence': 4757644393}
```
Then you can get historical price in ascending order seamlessly, even though
the limit has exceeded the source limit. The application helps continue
querying until the data reaches the requirements.
For the source of [Poloniex](https://poloniex.com/support/api/), run
Also, to buy order in Poloniex,
```
prices = get_historical_prices(source='Poloniex',
symbol='LTC/BTC',
period="30m",
from_time=datetime(2016, 1, 1),
to_time=datetime(2017, 8, 1))
In [1]: from libcryptomarket.api import PoloniexApi
In [2]: exchange = PoloniexApi(public_key="<your public key>", private_key="<your private key>")
In [3]: exchange.buy(currencyPair="BTC_LTC", rate=0.001, amount=0.1, postOnly=1)
```
The exchange API methods are always delimited by underscore.
| Exchange | Method | libcryptomarket | URL |
|---|---|---|---|
| [Bitfinex](https://docs.bitfinex.com/v2/docs/ws-general) | [candles](https://docs.bitfinex.com/v2/reference#rest-public-candles) | `BitfinexApi().candles(timeframe="1m", symbol="tBTCUSD", session="hist")` | https://api.bitfinex.com/v2/candles/trade:1m:tBTCUSD/hist |
| [BitMEX](https://www.bitmex.com/api/explorer/) | [trades/bucketed](https://www.bitmex.com/api/explorer/#!/Trade/Trade_getBucketed) | `BitmexApi().trade_bucketed(binSize="1m", symbol="XBTUSD")` | https://www.bitmex.com/api/v1/trade/bucketed?binSize=1m&symbol=XBTUSD |
| [CoinMarketCap](https://coinmarketcap.com/api/) | [ticket/<id>/](https://coinmarketcap.com/api/) | `CoinMarketCapApi().ticker(id="bitcoin", convert="EUR")` | https://api.coinmarketcap.com/v1/ticker/bitcoin/?convert=EUR |
| [GDAX](https://docs.gdax.com/#api) | [/products/<product-id>/candles](https://docs.gdax.com/#get-historic-rates) | `GdaxApi().products_candles(product_id="BTC-USD")` | https://api.gdax.com/products/BTC-USD/candles |
| [Poloniex](https://poloniex.com/support/api/) | [returnOrderBook](https://poloniex.com/support/api/#returnOrderBook) | `PoloniexApi().return_order_book(currencyPair="BTC_NXT", depth=10)` | https://poloniex.com/public?command=returnOrderBook&currencyPair=BTC_NXT&depth=10 |
## Contribution
The project is targeting as a core but generic toolkit to query cryptocurrency
libcryptomarket/api/__init__.py
+7
View File
@@ -0,0 +1,7 @@
# pylint: disable-msg=W0401
# flake8: noqa
from libcryptomarket.api.bitfinex_api import BitfinexApi
from libcryptomarket.api.bitmex_api import BitmexApi
from libcryptomarket.api.coinmarketcap_api import CoinMarketCapApi
from libcryptomarket.api.gdax_api import GdaxApi
from libcryptomarket.api.poloniex_api import PoloniexApi
libcryptomarket/api/bitfinex_api.py
+2 -3
View File
@@ -44,8 +44,7 @@ class BitfinexApi(ExchangeApi):
def get_private_calls(cls):
"""Get public API calls.
"""
return {
}
return {}
@classmethod
def translate_call_name(cls, name):
@@ -112,7 +111,7 @@ class BitfinexApi(ExchangeApi):
:param name: Method name.
:param http_method: HTTP method (POST, GET, DELETE).
"""
raise NotImplementedError()
raise NotImplementedError("Not support private api at this moment.")
# @classmethod
# def _generate_auth(cls, public_key, private_key):
libcryptomarket/api/bitmex_api.py
+2 -2
View File
@@ -6,7 +6,7 @@ class BitmexApi(ExchangeApi):
"""BitMEX API connector.
"""
def __init__(self, public_key, private_key, logger=None):
def __init__(self, public_key=None, private_key=None, logger=None):
"""Constructor.
:param public_key: Public key.
@@ -82,4 +82,4 @@ class BitmexApi(ExchangeApi):
# return self._send_request(
# command=name, http_method=http_method, params=kwargs,
# public_key=self._public_key, private_key=self._private_key)
raise NotImplementedError("request private is not implemented.")
raise NotImplementedError("Not support private api at this moment.")
libcryptomarket/api/coinmarketcap_api.py
+62 -50
View File
@@ -1,64 +1,76 @@
#!/bin/python
import requests
from datetime import datetime
from libcryptomarket.api.exchange_api import ExchangeApi
TICKET_URL = "https://api.coinmarketcap.com/v1/ticker/"
class CoinMarketCapApiTicker:
"""Result class of query /ticker
class CoinMarketCapApi(ExchangeApi):
"""Coinmarketcap API.
"""
def __init__(self, **kwargs):
def __init__(self, logger=None):
"""Constructor.
Constructed from the request result like the following
{
"id": "bitcoin",
"name": "Bitcoin",
"symbol": "BTC",
"rank": "1",
"price_usd": "573.137",
"price_btc": "1.0"
"24h_volume_usd": "72855700.0",
"market_cap_usd": "9080883500.0",
"available_supply": "15844176.0",
"total_supply": "15844176.0",
"percent_change_1h": "0.04",
"percent_change_24h": "-0.3",
"percent_change_7d": "-0.57",
"last_updated": "1472762067"
}
:param public_key: Public key.
:param private_key: Private key.
:param logger: Logger.
"""
self.r_id = kwargs["id"]
self.r_name = kwargs["name"]
self.r_symbol = kwargs["symbol"]
self.r_rank = int(kwargs["rank"] or '0')
self.r_price_usd = float(kwargs["price_usd"] or '0')
self.r_price_btc = float(kwargs["price_btc"] or '0')
self.r_24h_volume_usd = float(kwargs["24h_volume_usd"] or '0')
self.r_market_cap_usd = float(kwargs["market_cap_usd"] or '0')
self.r_available_supply = float(kwargs["available_supply"] or '0')
self.r_total_supply = float(kwargs["total_supply"] or '0')
self.r_percent_change_1h = float(kwargs["percent_change_1h"] or '0')
self.r_percent_change_24h = float(kwargs["percent_change_24h"] or '0')
self.r_percent_change_7d = float(kwargs["percent_change_7d"] or '0')
self.r_last_updated = datetime.fromtimestamp(
int(kwargs["last_updated"]))
ExchangeApi.__init__(self, public_key=None, private_key=None,
logger=logger)
@classmethod
def get_url(cls):
"""Get API url.
"""
return "https://api.coinmarketcap.com/v1"
def get_ticker(coin=None):
"""Return the ticker of all coins or the particular coin.
@classmethod
def get_public_calls(cls):
"""Get public API calls.
"""
return {
"ticker": "GET",
"global": "GET"
}
It returns a list of `CoinMarketCapApiTicker` objects.
@classmethod
def get_private_calls(cls):
"""Get private API calls.
"""
return {}
:param coin: Coin id. Default None which means all coins are
queued.
"""
url = TICKET_URL
@classmethod
def translate_call_name(cls, name):
"""Translate API call name.
if coin is not None:
url += coin
The class method name is always underscored (aligned with Python
standard.) This method is to translate underscored name to exchange
API call name.
return requests.get(url).json()
:param name: Method name (underscored).
"""
return name
def _request_public(self, name, http_method, **kwargs):
"""Request public API call.
:param name: Method name.
:param http_method: HTTP method (POST, GET, DELETE).
"""
name_list = [name]
if name == "ticker" and "id" in kwargs.keys():
name_list.append(kwargs["id"])
del kwargs["id"]
name_list.append("")
return self._send_request(
command='/'.join(name_list), http_method=http_method,
public_method=True, params=kwargs)
def _request_private(self, name, http_method, **kwargs):
"""Request private API call.
:param name: Method name.
:param http_method: HTTP method (POST, GET, DELETE).
"""
raise RuntimeError("No private method provided")
libcryptomarket/api/cryptocompare_api.py
-104
View File
@@ -1,104 +0,0 @@
#!/bin/python
import requests
from datetime import datetime
import logging
API_URL = "https://min-api.cryptocompare.com/data/"
MAX_QUERY_LIMIT = 2000
logger = logging.getLogger(__name__)
class CryptocompareCoinlist:
"""Cryptocompare coinlist.
"""
def __init__(self, **kwargs):
"""Constructor.
"""
self.r_algorithm = kwargs['Algorithm'].replace('N/A', '') or ''
self.r_coinname = kwargs['CoinName'] or ''
self.r_fullname = kwargs['FullName'] or ''
self.r_fullypremined = int(kwargs['FullyPremined'].replace('N/A', '')
or 0)
self.r_id = kwargs['Id'] or ''
self.r_imageurl = kwargs.get('ImageUrl', '') or ''
self.r_name = kwargs['Name'] or ''
self.r_preminedvalue = float(kwargs['PreMinedValue'].replace('N/A', '')
or '0')
self.r_prooftype = kwargs['ProofType'].replace('N/A', '') or ''
self.r_sortorder = int(kwargs['SortOrder'].replace('N/A', '') or '0')
self.r_sponsored = kwargs['Sponsored'] or False
self.r_symbol = kwargs['Symbol'] or ''
# TotalCoinSupply and TotalCoinsFreeFloat are not supported due to
# very dirty data.
self.r_url = kwargs['Url'] or ''
class CryptocompareHisto:
"""Cryptocompare histo.
"""
def __init__(self, **kwargs):
"""Constructor.
"""
self.r_close = float(kwargs['close'])
self.r_high = float(kwargs['high'])
self.r_low = float(kwargs['low'])
self.r_open = float(kwargs['open'])
self.r_time = datetime.fromtimestamp(int(kwargs['time']))
self.r_volumefrom = float(kwargs['volumefrom'])
self.r_volumeto = float(kwargs['volumeto'])
def get_coinlist():
"""Return general info for all coins available.
"""
url = API_URL + "all/coinlist"
r = requests.get(url)
r.raise_for_status()
return r.json()
def get_histo(period, fsym, tsym, e, limit=None, toTs=None):
"""Return historical prices.
:param period: Period, one of the values of "minute", "hour" and "day".
:param fsym: From symbol.
:param tsym: To symbol.
:param e: Exchange name.
:param limit: Limit of return data. Default is None.
:param toTs: To timestamp. Default is None.
"""
valid_list = ["minute", "hour", "day"]
if period not in valid_list:
raise ValueError("Period must be in {0}".format(', '.join(valid_list)))
url = API_URL + "histo" + period
params = {
"fsym": fsym,
"tsym": tsym,
"e": e
}
if limit is not None:
params["limit"] = limit
if toTs is not None:
params["toTs"] = toTs
r = requests.get(url, params=params)
# Raise html error status
r.raise_for_status()
# The api raises a 200 for a warning, but passes a message
rjson = r.json()
if rjson.get("Message", None):
logger.warning(
'api returned message %r, for url %r',
rjson["Message"],
r.url)
return rjson
libcryptomarket/api/exchange_api.py
+2 -2
View File
@@ -45,13 +45,13 @@ class ExchangeApi:
def get_public_calls(cls):
"""Get public API calls.
"""
raise NotImplementedError("Public API calls getter not implemented")
return {}
@classmethod
def get_private_calls(cls):
"""Get private API calls.
"""
raise NotImplementedError("Private API calls getter not implemented")
return {}
@classmethod
def translate_call_name(cls, name):
libcryptomarket/api/gdax_api.py
+2 -3
View File
@@ -44,8 +44,7 @@ class GdaxApi(ExchangeApi):
def get_private_calls(cls):
"""Get public API calls.
"""
return {
}
return {}
@classmethod
def translate_call_name(cls, name):
@@ -83,7 +82,7 @@ class GdaxApi(ExchangeApi):
:param name: Method name.
:param http_method: HTTP method (POST, GET, DELETE).
"""
raise NotImplementedError()
raise NotImplementedError("request private is not implemented.")
# @classmethod
# def _generate_auth(cls, public_key, private_key):
libcryptomarket/api/poloniex_api.py
-60
View File
@@ -1,69 +1,9 @@
#!/bin/python
import requests
import hmac
import hashlib
import urllib
# from functools import partial
from libcryptomarket.api.exchange_api import ExchangeApi
API_URL = "https://poloniex.com/public?command="
VALID_PERIODS = [300, 900, 1800, 7200, 14400, 86400]
def get_return_chart_data(currency_pair, period, start, end=None):
"""Return returnChartData.
:param currency_pair: Currency pair. For example, BTC_XMR.
:param period: Period. Valid values are 300, 900, 1800, 7200, 14400 and
86400.
:param start: Start time in unix timestamp.
:param end: End time in unix timestamp. Optional.
"""
if period not in VALID_PERIODS:
raise ValueError(
"Period is not in the valid periods (%s)" % VALID_PERIODS)
url = (API_URL + "returnChartData" +
"&currencyPair={0}".format(currency_pair) +
"&period={0}".format(period) +
"&start={0}".format(start))
if end is not None:
url += "&end={0}".format(end)
r = requests.get(url)
r.raise_for_status()
rjson = r.json()
if isinstance(rjson, dict) and 'error' in rjson.keys():
raise ValueError("Query error from Poloniex API ({0})".format(rjson))
return r.json()
def get_return_order_book(currency_pair, depth=10):
"""Return returnOrderBook.
:param currency_pair: Currency pair. Specify "all" if requesting for all
symbols.
:param depth: Number of depth. Default is 10.
"""
if currency_pair is None:
raise ValueError("Currency pair cannot be None.")
params = {}
params["currencyPair"] = currency_pair
params["depth"] = depth
url = API_URL + "returnOrderBook"
r = requests.get(url, params=params)
r.raise_for_status()
rjson = r.json()
if isinstance(rjson, dict) and 'error' in rjson.keys():
raise ValueError("Query error from Poloniex API ({0})".format(rjson))
return r.json()
class PoloniexApi(ExchangeApi):
libcryptomarket/api/rest_api_connector.py
-105
View File
@@ -1,105 +0,0 @@
#!/bin/python
import requests
import urllib
import json
from time import time
class RestApiConnector(object):
"""REST API connector.
"""
def __init__(self, url, logger=None):
"""Constructor.
:param url: URL address.
:param logger: Logger. Default is None.
"""
self._logger = logger
self._url = url
@classmethod
def _generate_nonce(cls):
"""Generate an increasing unique number.
"""
return int(round(time() * 1000))
def _generate_headers(self, command, http_method, params, data,
public_key, private_key):
"""Generate headers.
:param command: Command.
:param http_method: HTTP method, for example GET.
:param params: Parameters.
:param data: Data.
:param public_key: Public key.
:param private_key: Private key.
"""
raise NotImplementedError("Not yet implemented.")
def _generate_auth(self, public_key, private_key):
"""Generate authentication.
:param public_key: Public key.
:param private_key: Private key.
"""
raise NotImplementedError("Not yet implemented.")
def _format_data(self, data):
"""Format the data to exchange desirable format.
:param data: Data.
"""
raise NotImplementedError("Not yet implemented.")
def _send_request(self, command, http_method, public_key, private_key,
params=None, data=None):
"""Send request.
:param command: API command.
:param http_method: Http method.
:param api_key: API key.
:param params: Input parameters, which will be parsed
as "?key1=value1...".
:param data: Data.
:return: JSON object.
"""
http_method = http_method.upper()
if http_method == "DELETE":
R = requests.delete
elif http_method == "GET":
R = requests.get
elif http_method == "POST":
R = requests.post
else:
raise ValueError("Http method must be either DELETE, GET or "
"POST.")
url = urllib.parse.urljoin(self._url, command)
data = self._format_data(data)
headers = self._generate_headers(command, http_method, params,
data, public_key, private_key)
auth = self._generate_auth(public_key, private_key)
if self._logger is not None:
self._logger.info(">>> OUT:\n%s" % json.dumps({
"Method": http_method,
"Url": url,
"Params": params,
"Data": data,
"Headers": headers
}))
if auth is None:
response = R(url, params=params, data=data, headers=headers)
else:
response = R(url, params=params, data=data, headers=headers,
auth=auth)
if self._logger is not None:
self._logger.info("<<< IN:\n%s" % json.dumps({
"Status code": response.status_code,
"Text": response.text
}))
return response
libcryptomarket/core/__init__.py
+5
View File
@@ -0,0 +1,5 @@
# pylint: disable-msg=W0401
# flake8: noqa
from libcryptomarket.core.historical import historical_ticker
from libcryptomarket.core.instrument import instruments
from libcryptomarket.core.order_book import order_book
libcryptomarket/core.py → libcryptomarket/core/historical.py
View File
libcryptomarket/core/instrument.py
+18
View File
@@ -0,0 +1,18 @@
import pandas as pd
def instruments(source):
"""Return instruments.
:param source: Source, an Exchange API object.
"""
# Source object name
source_name = source.__class__.__name__.lower().replace("api", "")
if source_name == "coinmarketcap":
response = source.ticker()
response.raise_for_status()
return pd.DataFrame(response.json())
else:
raise ValueError("Source (%s [%s]) does not support instruments"
% (source, source_name))
libcryptomarket/core/order_book.py
+40
View File
@@ -0,0 +1,40 @@
import pandas as pd
def order_book(source, symbol, depth=5):
"""Return the order book.
:param source: Source, an Exchange API object.
:param symbol: Symbol.
:param depth: Depth of the order book.
"""
source_name = source.__class__.__name__.lower().replace("api", "")
if source_name == "poloniex":
return _order_book_poloniex(source, symbol, depth)
else:
raise ValueError("Source (%s [%s]) does not support order book"
% (source, source_name))
def _order_book_poloniex(source, symbol, depth=5):
"""Return the order book from Poloniex
:param source: Source, an Exchange API object.
:param symbol: Symbol.
:param depth: Depth of the order book.
"""
if symbol == "all":
raise ValueError("Currently not support all symbol order book query.")
response = source.return_order_book(currencyPair=symbol, depth=depth)
response.raise_for_status()
data = response.json()
data = [pd.DataFrame(
data[side],
columns=pd.MultiIndex.from_product(
[[side], ['price', 'quantity']]),
index=range(1, depth + 1)) for side in ['bids', 'asks']]
data = pd.concat(data, axis=1).astype('float64')
return data
libcryptomarket/historical.py
-206
View File
@@ -1,206 +0,0 @@
from functools import partial
from datetime import datetime, timedelta
import pandas as pd
def get_historical_prices(source='cryptocompare', symbol=None, exchange=None,
period=None, limit=0, from_time=None, to_time=None):
"""Get historical prices.
:param source: Source of data.
:param symbol: Symbol. Default is None.
:param exchange: Exchange. Default is None.
:param period: Data frequency. Default is None, which follows the source
default value.
:param limit: Limit of records. Default is 0, which follows the source
default value.
:param from_time: From time. Default is None, which follows the source
default value.
:param to_time: To time. Default is None, which follows the source default
value.
"""
source = source.lower()
if source == 'cryptocompare':
if period is None:
raise ValueError("Input parameter period cannot be None.")
if exchange is None:
raise ValueError("Input parameter exchange cannot be None.")
if ((limit > 0) +
((from_time is not None) or (to_time is not None)) > 1):
raise ValueError("Only accept input parameter limit, or from_time"
" and to_time pair.")
if (from_time is None) ^ (to_time is None):
raise ValueError("Cannot accept either from_time or to_time is "
"None")
return _get_historical_prices_cryptocompare(symbol, exchange, period,
limit, from_time, to_time)
elif source == 'poloniex':
if exchange is not None:
raise ValueError("Poloniex does not need exchange parameter.")
# Validate and transform symbol
symbol = symbol.replace("/", "_").upper()
# Validate and transform periods
valid_periods = {
"5m": 300,
"15m": 900,
"30m": 1800,
"2h": 7200,
"4h": 14400,
"1d": 86400
}
period = period.lower()
if period not in valid_periods.keys():
raise ValueError("Periods is not valid. " +
("Valid values (%s)" % str(valid_periods)))
else:
period = valid_periods[period]
# Validate and transform to_time and from_time
from_time = from_time.timestamp()
to_time = to_time.timestamp() if to_time is not None else 9999999999
if to_time <= from_time:
raise ValueError("From time should not be greater than or "
"equal to to time.")
return _get_historical_prices_poloniex(symbol, period,
from_time, to_time)
elif source == 'bitmex':
if exchange is not None:
raise ValueError("Bitmex does not need exchange parameter.")
# Validate and transform periods
valid_periods = ["1m", "5m", "1h", "1d"]
period = period.lower()
if period not in valid_periods:
raise ValueError("Periods is not valid. " +
("Valid values (%s)" % str(valid_periods)))
# Validate and transform to_time and from_time
if (to_time is not None and from_time is not None and
to_time <= from_time):
raise ValueError("From time should not be greater than or "
"equal to to time.")
ret = []
while from_time is None or to_time is None or from_time < to_time:
# Import and query
from libcryptomarket.api.bitmex_api import BitmexApi
exchange = BitmexApi(
public_key=None,
private_key=None,
logger=None)
func = partial(exchange.trade_bucketed, symbol=symbol,
binSize=period)
if from_time is not None:
from_time_s = from_time.strftime("%Y-%m-%dT%H:%M:%S")
func = partial(func, startTime=from_time_s)
if to_time is not None:
to_time_s = to_time.strftime("%Y-%m-%dT%H:%M:%S")
func = partial(func, endTime=to_time_s)
data = func()
data.raise_for_status()
data = pd.DataFrame(data.json())
if len(data) > 0:
ret.append(data)
from_time = data.iloc[-1, :]['timestamp'][:-5]
from_time = datetime.strptime(from_time, "%Y-%m-%dT%H:%M:%S")
from_time = from_time + timedelta(seconds=1)
else:
break
return pd.concat(ret)
else:
raise ValueError("No source is called {0}".format(source))
def _get_historical_prices_cryptocompare(symbol, exchange, period,
limit, from_time, to_time):
from libcryptomarket.api.cryptocompare_api import (
get_histo, CryptocompareHisto
)
from_sym = symbol.split('/')[0]
to_sym = symbol.split('/')[1]
func = partial(get_histo, period=period, fsym=from_sym, tsym=to_sym,
e=exchange)
data = []
if limit > 0:
# Get the data by limit of records
to_time = 0
while limit > 0:
if to_time == 0:
response = func(limit=limit)['Data']
else:
response = func(limit=limit, toTs=to_time)['Data']
if len(response) == 0:
# Terminate if no further response
break
else:
data += response
limit -= len(response)
to_time = response[0]['time'] - 1
elif from_time is not None and to_time is not None:
# Get the data by time range
from libcryptomarket.api.cryptocompare_api import MAX_QUERY_LIMIT
from_time_ts = int(from_time.timestamp())
to_time_ts = int(to_time.timestamp())
while from_time_ts < to_time_ts:
response = func(limit=MAX_QUERY_LIMIT,
toTs=to_time_ts)['Data']
if len(response) == 0:
# Terminate if no further response
break
else:
data += response
to_time_ts = response[0]['time'] - 1
else:
data += func()['Data']
if len(data) == 0:
return data
data = pd.DataFrame([CryptocompareHisto(**e).__dict__ for e in data])
# Filter only valid time range
if from_time is not None and to_time is not None:
data = data[(data['r_time'] >= from_time) &
(data['r_time'] <= to_time)]
data = data.set_index(['r_time'])
data.index.name = 'datetime'
return data
def _get_historical_prices_poloniex(symbol, period, from_time, to_time):
from libcryptomarket.api.poloniex_api import get_return_chart_data
data = get_return_chart_data(symbol, period, from_time, to_time)
data = pd.DataFrame(data)
data['date'] = data['date'].map(
lambda x: pd.Timestamp(datetime.fromtimestamp(x)))
data.columns = "r_" + data.columns.str.lower()
data = data.set_index(['r_date'])
data.index.name = 'datetime'
return data
libcryptomarket/instrument.py
-26
View File
@@ -1,26 +0,0 @@
import pandas as pd
def get_instruments(source='coinmarketcap', **kwargs):
"""Return all the instruments.
"""
if source == 'coinmarketcap':
# Coinmarketcap
from libcryptomarket.api.coinmarketcap_api import (
get_ticker, CoinMarketCapApiTicker)
result = get_ticker(**kwargs)
result = [CoinMarketCapApiTicker(**e) for e in result]
elif source == 'cryptocompare':
# Cryptocompare
from libcryptomarket.api.cryptocompare_api import (
get_coinlist, CryptocompareCoinlist)
result = get_coinlist(**kwargs)
result = [CryptocompareCoinlist(**value)
for key, value in result['Data'].items()]
else:
raise ValueError("No source is called {0}".format(source))
result = pd.DataFrame([r.__dict__ for r in result])
return result
libcryptomarket/price.py
-43
View File
@@ -1,43 +0,0 @@
import pandas as pd
def get_order_book(source, symbol, depth=5):
"""Return the order book.
:param source: Data source name.
:param symbol: Symbol.
:param depth: Depth of the order book.
"""
source = source.lower()
if source == 'poloniex':
from libcryptomarket.api.poloniex_api import get_return_order_book
# Get the raw data
symbol = symbol.replace("/", "_")
data = get_return_order_book(currency_pair=symbol, depth=depth)
# Align the format as symbol="all"
if symbol == "all":
# Convert it into a multiindex dataframe with symbol at the first
# level of the columns
data = [[pd.DataFrame(
data[symbol][side],
columns=pd.MultiIndex.from_product(
[[symbol], [side], ['price', 'quantity']]),
index=range(1, depth + 1)) for side in ['bids', 'asks']]
for symbol, prices in data.items()]
data = pd.concat(sum(data, []), axis=1)
else:
# Convert it into a dataframe where bid and ask at the columns
data = [pd.DataFrame(
data[side],
columns=pd.MultiIndex.from_product(
[[side], ['price', 'quantity']]),
index=range(1, depth + 1)) for side in ['bids', 'asks']]
data = pd.concat(data, axis=1)
return data.astype('float64')
else:
raise ValueError("Source ({0}) is not yet implemented.".format(
source))
setup.py
+2 -1
View File
@@ -35,7 +35,8 @@ setup(
author="Gavin Chan",
author_email='gavincyi@gmail.com',
url='https://github.com/gavincyi/libcryptomarket',
packages=['libcryptomarket', 'libcryptomarket.api'],
packages=['libcryptomarket', 'libcryptomarket.api',
'libcryptomarket.core'],
include_package_data=True,
install_requires=requirements,
license="GNU General Public License v3",
tests/test_coinmarketcap_api.py
-134
View File
@@ -1,134 +0,0 @@
import requests
import datetime
import pandas as pd
from pandas.util.testing import assert_frame_equal
from libcryptomarket.instrument import get_instruments
def test_get_instruments_coinmarketcap(monkeypatch):
def mockreturn(url):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
url_split = url.split('/')
if url_split[-1] == "":
# Query all symbols
return [
{
"id": "bitcoin",
"name": "Bitcoin",
"symbol": "BTC",
"rank": "1",
"price_usd": "573.137",
"price_btc": "1.0",
"24h_volume_usd": "72855700.0",
"market_cap_usd": "9080883500.0",
"available_supply": "15844176.0",
"total_supply": "15844176.0",
"percent_change_1h": "0.04",
"percent_change_24h": "-0.3",
"percent_change_7d": "-0.57",
"last_updated": "1472762067"
},
{
"id": "ethereum",
"name": "Ethereum",
"symbol": "ETH",
"rank": "2",
"price_usd": "12.1844",
"price_btc": "0.021262",
"24h_volume_usd": "24085900.0",
"market_cap_usd": "1018098455.0",
"available_supply": "83557537.0",
"total_supply": "83557537.0",
"percent_change_1h": "-0.58",
"percent_change_24h": "6.34",
"percent_change_7d": "8.59",
"last_updated": "1472762062"
}]
else:
# Query the particular symbol. Now only test with bitcoin
return [
{
"id": "bitcoin",
"name": "Bitcoin",
"symbol": "BTC",
"rank": "1",
"price_usd": "573.137",
"price_btc": "1.0",
"24h_volume_usd": "72855700.0",
"market_cap_usd": "9080883500.0",
"available_supply": "15844176.0",
"total_supply": "15844176.0",
"percent_change_1h": "0.04",
"percent_change_24h": "-0.3",
"percent_change_7d": "-0.57",
"last_updated": "1472762067"
}]
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
# Test getting all coins
result = get_instruments(source='coinmarketcap')
expected_result = pd.DataFrame([
{
"r_id": "bitcoin",
"r_name": "Bitcoin",
"r_symbol": "BTC",
"r_rank": 1,
"r_price_usd": 573.137,
"r_price_btc": 1.0,
"r_24h_volume_usd": 72855700.0,
"r_market_cap_usd": 9080883500.0,
"r_available_supply": 15844176.0,
"r_total_supply": 15844176.0,
"r_percent_change_1h": 0.04,
"r_percent_change_24h": -0.3,
"r_percent_change_7d": -0.57,
"r_last_updated": datetime.datetime(2016, 9, 1, 20, 34, 27)
},
{
"r_id": "ethereum",
"r_name": "Ethereum",
"r_symbol": "ETH",
"r_rank": 2,
"r_price_usd": 12.1844,
"r_price_btc": 0.021262,
"r_24h_volume_usd": 24085900.0,
"r_market_cap_usd": 1018098455.0,
"r_available_supply": 83557537.0,
"r_total_supply": 83557537.0,
"r_percent_change_1h": -0.58,
"r_percent_change_24h": 6.34,
"r_percent_change_7d": 8.59,
"r_last_updated": datetime.datetime(2016, 9, 1, 20, 34, 22)
}])
assert_frame_equal(result.set_index(['r_id']).sort_index(),
expected_result.set_index(['r_id']).sort_index())
# Test getting only bitcoin
result = get_instruments(source='coinmarketcap', coin='bitcoin')
expected_result = pd.DataFrame([
{
"r_id": "bitcoin",
"r_name": "Bitcoin",
"r_symbol": "BTC",
"r_rank": 1,
"r_price_usd": 573.137,
"r_price_btc": 1.0,
"r_24h_volume_usd": 72855700.0,
"r_market_cap_usd": 9080883500.0,
"r_available_supply": 15844176.0,
"r_total_supply": 15844176.0,
"r_percent_change_1h": 0.04,
"r_percent_change_24h": -0.3,
"r_percent_change_7d": -0.57,
"r_last_updated": datetime.datetime(2016, 9, 1, 20, 34, 27)
}])
assert_frame_equal(result.set_index(['r_id']).sort_index(),
expected_result.set_index(['r_id']).sort_index())
tests/test_cryptocompare_api.py
-172
View File
@@ -1,172 +0,0 @@
import requests
from datetime import datetime
import pandas as pd
from pandas.util.testing import assert_frame_equal
from libcryptomarket.instrument import get_instruments
from libcryptomarket.historical import get_historical_prices
def test_get_instruments_cryptocompare(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
return {
'BaseImageUrl': 'https://www.cryptocompare.com',
'BaseLinkUrl': 'https://www.cryptocompare.com',
'Data': {
'STX': {
'Algorithm': 'N/A',
'CoinName': 'Stox',
'FullName': 'Stox (STX)',
'FullyPremined': '0',
'Id': '204716',
'ImageUrl': '/media/1383946/stx.png',
'Name': 'STX',
'PreMinedValue': 'N/A',
'ProofType': 'N/A',
'SortOrder': '1431',
'Sponsored': False,
'Symbol': 'STX',
'TotalCoinSupply': '29600000',
'TotalCoinsFreeFloat': 'N/A',
'Url': '/coins/stx/overview'},
'BCN': {
'Algorithm': 'CryptoNight',
'CoinName': 'ByteCoin',
'FullName': 'ByteCoin (BCN)',
'FullyPremined': '0',
'Id': '5280',
'ImageUrl': '/media/12318404/bcn.png',
'Name': 'BCN',
'PreMinedValue': 'N/A',
'ProofType': 'PoW',
'SortOrder': '249',
'Sponsored': False,
'Symbol': 'BCN',
'TotalCoinSupply': '184467440735',
'TotalCoinsFreeFloat': 'N/A',
'Url': '/coins/bcn/overview'}
}
}
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
# Test getting all coins
result = get_instruments(source='cryptocompare')
assert len(result) == 2
expected_result = pd.DataFrame([
{
'r_algorithm': '',
'r_coinname': 'Stox',
'r_fullname': 'Stox (STX)',
'r_fullypremined': 0,
'r_id': '204716',
'r_imageurl': '/media/1383946/stx.png',
'r_name': 'STX',
'r_preminedvalue': 0.0,
'r_prooftype': '',
'r_sortorder': 1431,
'r_sponsored': False,
'r_symbol': 'STX',
'r_url': '/coins/stx/overview'},
{
'r_algorithm': 'CryptoNight',
'r_coinname': 'ByteCoin',
'r_fullname': 'ByteCoin (BCN)',
'r_fullypremined': 0,
'r_id': '5280',
'r_imageurl': '/media/12318404/bcn.png',
'r_name': 'BCN',
'r_preminedvalue': 0.0,
'r_prooftype': 'PoW',
'r_sortorder': 249,
'r_sponsored': False,
'r_symbol': 'BCN',
'r_url': '/coins/bcn/overview'}])
assert_frame_equal(result.set_index(['r_id']).sort_index(),
expected_result.set_index(['r_id']).sort_index())
def test_get_historical_prices_cryptocompare(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
return {
'Aggregated': False,
'ConversionType': {
'conversionSymbol': '',
'type': 'force_direct'},
'Data': [
{
'close': 0.007707,
'high': 0.007716,
'low': 0.007701,
'open': 0.00771,
'time': 1510045800,
'volumefrom': 289.12,
'volumeto': 2.23
},
{
'close': 0.0077,
'high': 0.007716,
'low': 0.0077,
'open': 0.007707,
'time': 1510045860,
'volumefrom': 33.53,
'volumeto': 0.2586
}]
}
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
# Test to get historical prices
result = get_historical_prices(source='cryptocompare',
period='minute',
exchange='Poloniex',
symbol='LTC/BTC')
expected_result = pd.DataFrame([
{
'r_close': 0.007707,
'r_high': 0.007716,
'r_low': 0.007701,
'r_open': 0.00771,
'r_time': datetime(2017, 11, 7, 9, 10),
'r_volumefrom': 289.12,
'r_volumeto': 2.23
},
{
'r_close': 0.0077,
'r_high': 0.007716,
'r_low': 0.0077,
'r_open': 0.007707,
'r_time': datetime(2017, 11, 7, 9, 11),
'r_volumefrom': 33.53,
'r_volumeto': 0.2586
}]).set_index(['r_time'])
expected_result.index.name = 'datetime'
assert_frame_equal(result, expected_result)
tests/test_poloniex_api.py
-196
View File
@@ -1,196 +0,0 @@
import requests
from datetime import datetime
import pandas as pd
from pandas.util.testing import assert_frame_equal
import pytest
from libcryptomarket.historical import get_historical_prices
from libcryptomarket.price import get_order_book
def test_get_historical_prices_poloniex(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
return [
{
"date": 1405699200,
"high": 0.0045388,
"low": 0.00403001,
"open": 0.00404545,
"close": 0.00435873,
"volume": 44.34555992,
"quoteVolume": 10311.88079097,
"weightedAverage": 0.00430043
},
{
"date": 1405713600,
"high": 0.00435,
"low": 0.00412,
"open": 0.00428012,
"close": 0.00412,
"volume": 19.12271662,
"quoteVolume": 4531.85801066,
"weightedAverage": 0.00421961
}]
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
# Test to get historical prices
result = get_historical_prices(source='Poloniex',
period='4h',
symbol='BTC/XMR',
from_time=datetime(2014, 7, 18, 16, 0, 0))
expected_result = pd.DataFrame([
{
"r_date": datetime(2014, 7, 18, 16, 0, 0),
"r_high": 0.0045388,
"r_low": 0.00403001,
"r_open": 0.00404545,
"r_close": 0.00435873,
"r_volume": 44.34555992,
"r_quotevolume": 10311.88079097,
"r_weightedaverage": 0.00430043
},
{
"r_date": datetime(2014, 7, 18, 20, 0, 0),
"r_high": 0.00435,
"r_low": 0.00412,
"r_open": 0.00428012,
"r_close": 0.00412,
"r_volume": 19.12271662,
"r_quotevolume": 4531.85801066,
"r_weightedaverage": 0.00421961
}
]).set_index(['r_date'])
expected_result.index.name = 'datetime'
assert_frame_equal(result, expected_result)
def test_get_historical_prices_poloniex_invalid_period(monkeypatch):
with pytest.raises(ValueError):
get_historical_prices(
source='Poloniex',
period='29m',
symbol='BTC/XMR',
from_time=datetime(2014, 7, 18, 16, 0, 0))
def test_get_historical_prices_poloniex_invalid_instrument(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
return {"error": "Invalid currency pair."}
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
# Test to get historical prices
with pytest.raises(ValueError):
get_historical_prices(
source='Poloniex',
period='4h',
symbol='BTC/XXX',
from_time=datetime(2014, 7, 18, 16, 0, 0))
def test_get_order_book_poloniex_all_symbols(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
import json
return json.loads(
"""
{"BTC_AMP": {"asks": [["0.00002755", 16.03393139],
["0.00002756", 20.24556409]],
"bids": [["0.00002739", 20.23199043], ["0.00002723", 20.30358806]],
"isFrozen": "0",
"seq": 44422443},
"BTC_ARDR": {"asks": [["0.00003331", 62.49465571], ["0.00003332", 6567.629]],
"bids": [["0.00003285", 25], ["0.00003277", 44.83974499]],
"isFrozen": "0",
"seq": 27008837}}
"""
)
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
expected_df = pd.DataFrame([
[0.00002739, 20.231990, 0.00002755, 16.033931,
0.00003285, 25.000000, 0.00003331, 62.494656],
[0.00002723, 20.303588, 0.00002756, 20.245564,
0.00003277, 44.839745, 0.00003332, 6567.629000]],
columns=pd.MultiIndex.from_product(
[['BTC_AMP', 'BTC_ARDR'],
['bids', 'asks'],
['price', 'quantity']]),
index=[1, 2])
df = get_order_book(source="Poloniex", symbol="all", depth=2)
assert_frame_equal(expected_df.sort_index(axis=1),
df.sort_index(axis=1))
def test_get_order_book_poloniex_one_symbol(monkeypatch):
def mockreturn(url, *args, **kwargs):
# The result is from request.get(...).json()
class MockReturnClass:
@classmethod
def json(cls):
# Query all symbols
import json
return json.loads(
"""
{"asks": [["0.00002755", 16.03393139],
["0.00002756", 20.24556409]],
"bids": [["0.00002739", 20.23199043], ["0.00002723", 20.30358806]]}
"""
)
@classmethod
def raise_for_status(cls):
pass
return MockReturnClass()
monkeypatch.setattr(requests, 'get', mockreturn)
expected_df = pd.DataFrame([
[0.00002739, 20.231990, 0.00002755, 16.033931],
[0.00002723, 20.303588, 0.00002756, 20.245564]],
columns=pd.MultiIndex.from_product([['bids', 'asks'],
['price', 'quantity']]),
index=[1, 2])
df = get_order_book(source="Poloniex", symbol="BTC_AMP", depth=2)
assert_frame_equal(expected_df,
df)