mirror of
https://github.com/wassname/catalyst.git
synced 2026-08-12 11:50:11 +08:00
DOC: updated documentation
This commit is contained in:
Frederic Fortier
@@ -1,371 +0,0 @@
|
||||
API Reference
|
||||
-------------
|
||||
|
||||
Running a Backtest
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. autofunction:: catalyst.run_algorithm(...)
|
||||
|
||||
Algorithm API
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
The following methods are available for use in the ``initialize``,
|
||||
``handle_data``, and ``before_trading_start`` API functions.
|
||||
|
||||
In all listed functions, the ``self`` argument is implicitly the
|
||||
currently-executing :class:`~zipline.algorithm.TradingAlgorithm` instance.
|
||||
|
||||
Data Object
|
||||
```````````
|
||||
|
||||
.. autoclass:: catalyst.protocol.BarData
|
||||
:members:
|
||||
|
||||
Scheduling Functions
|
||||
````````````````````
|
||||
|
||||
.. autofunction:: catalyst.api.schedule_function
|
||||
|
||||
.. autoclass:: catalyst.api.date_rules
|
||||
:members:
|
||||
:undoc-members:
|
||||
|
||||
.. autoclass:: catalyst.api.time_rules
|
||||
:members:
|
||||
|
||||
Orders
|
||||
``````
|
||||
|
||||
.. autofunction:: catalyst.api.order
|
||||
|
||||
.. autofunction:: catalyst.api.order_value
|
||||
|
||||
.. autofunction:: catalyst.api.order_percent
|
||||
|
||||
.. autofunction:: catalyst.api.order_target
|
||||
|
||||
.. autofunction:: catalyst.api.order_target_value
|
||||
|
||||
.. autofunction:: catalyst.api.order_target_percent
|
||||
|
||||
.. autoclass:: catalyst.finance.execution.ExecutionStyle
|
||||
:members:
|
||||
|
||||
.. autoclass:: catalyst.finance.execution.MarketOrder
|
||||
|
||||
.. autoclass:: catalyst.finance.execution.LimitOrder
|
||||
|
||||
.. autoclass:: catalyst.finance.execution.StopOrder
|
||||
|
||||
.. autoclass:: catalyst.finance.execution.StopLimitOrder
|
||||
|
||||
.. autofunction:: catalyst.api.get_order
|
||||
|
||||
.. autofunction:: catalyst.api.get_open_orders
|
||||
|
||||
.. autofunction:: catalyst.api.cancel_order
|
||||
|
||||
Order Cancellation Policies
|
||||
'''''''''''''''''''''''''''
|
||||
|
||||
.. autofunction:: catalyst.api.set_cancel_policy
|
||||
|
||||
.. autoclass:: catalyst.finance.cancel_policy.CancelPolicy
|
||||
:members:
|
||||
|
||||
.. autofunction:: catalyst.api.EODCancel
|
||||
|
||||
.. autofunction:: catalyst.api.NeverCancel
|
||||
|
||||
|
||||
Assets
|
||||
``````
|
||||
|
||||
.. autofunction:: catalyst.api.symbol
|
||||
|
||||
.. autofunction:: catalyst.api.symbols
|
||||
|
||||
.. autofunction:: catalyst.api.set_symbol_lookup_date
|
||||
|
||||
.. autofunction:: catalyst.api.sid
|
||||
|
||||
|
||||
Trading Controls
|
||||
````````````````
|
||||
|
||||
zipline provides trading controls to help ensure that the algorithm is
|
||||
performing as expected. The functions help protect the algorithm from certian
|
||||
bugs that could cause undesirable behavior when trading with real money.
|
||||
|
||||
.. autofunction:: catalyst.api.set_do_not_order_list
|
||||
|
||||
.. autofunction:: catalyst.api.set_long_only
|
||||
|
||||
.. autofunction:: catalyst.api.set_max_leverage
|
||||
|
||||
.. autofunction:: catalyst.api.set_max_order_count
|
||||
|
||||
.. autofunction:: catalyst.api.set_max_order_size
|
||||
|
||||
.. autofunction:: catalyst.api.set_max_position_size
|
||||
|
||||
|
||||
Simulation Parameters
|
||||
`````````````````````
|
||||
|
||||
.. autofunction:: catalyst.api.set_benchmark
|
||||
|
||||
Commission Models
|
||||
'''''''''''''''''
|
||||
|
||||
.. autofunction:: catalyst.api.set_commission
|
||||
|
||||
.. autoclass:: catalyst.finance.commission.CommissionModel
|
||||
:members:
|
||||
|
||||
.. autoclass:: catalyst.finance.commission.PerShare
|
||||
|
||||
.. autoclass:: catalyst.finance.commission.PerTrade
|
||||
|
||||
.. autoclass:: catalyst.finance.commission.PerDollar
|
||||
|
||||
Slippage Models
|
||||
'''''''''''''''
|
||||
|
||||
.. autofunction:: catalyst.api.set_slippage
|
||||
|
||||
.. autoclass:: catalyst.finance.slippage.SlippageModel
|
||||
:members:
|
||||
|
||||
.. autoclass:: catalyst.finance.slippage.FixedSlippage
|
||||
|
||||
.. autoclass:: catalyst.finance.slippage.VolumeShareSlippage
|
||||
|
||||
Pipeline
|
||||
````````
|
||||
|
||||
Not supported yet.
|
||||
|
||||
.. For more information, see :ref:`pipeline-api`
|
||||
|
||||
.. .. autofunction:: catalyst.api.attach_pipeline
|
||||
|
||||
.. .. autofunction:: catalyst.api.pipeline_output
|
||||
|
||||
|
||||
Miscellaneous
|
||||
`````````````
|
||||
|
||||
.. autofunction:: catalyst.api.record
|
||||
|
||||
.. autofunction:: catalyst.api.get_environment
|
||||
|
||||
.. autofunction:: catalyst.api.fetch_csv
|
||||
|
||||
|
||||
.. _pipeline-api:
|
||||
|
||||
.. Pipeline API
|
||||
.. ~~~~~~~~~~~~
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.Pipeline
|
||||
.. :members:
|
||||
.. :member-order: groupwise
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.CustomFactor
|
||||
.. :members:
|
||||
.. :member-order: groupwise
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.filters.Filter
|
||||
.. :members: __and__, __or__
|
||||
.. :exclude-members: dtype
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.Factor
|
||||
.. :members: bottom, deciles, demean, linear_regression, pearsonr,
|
||||
.. percentile_between, quantiles, quartiles, quintiles, rank,
|
||||
.. spearmanr, top, winsorize, zscore, isnan, notnan, isfinite, eq,
|
||||
.. \__add__, \__sub__, \__mul__, \__div__, \__mod__, \__pow__,
|
||||
.. \__lt__, \__le__, \__ne__, \__ge__, \__gt__
|
||||
.. :exclude-members: dtype
|
||||
.. :member-order: bysource
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.term.Term
|
||||
.. :members:
|
||||
.. :exclude-members: compute_extra_rows, dependencies, inputs, mask, windowed
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.data.USEquityPricing
|
||||
.. :members: open, high, low, close, volume
|
||||
.. :undoc-members:
|
||||
|
||||
.. Built-in Factors
|
||||
.. ````````````````
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.AverageDollarVolume
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.BollingerBands
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.BusinessDaysSincePreviousEvent
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.BusinessDaysUntilNextEvent
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.ExponentialWeightedMovingAverage
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.ExponentialWeightedMovingStdDev
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.Latest
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.MaxDrawdown
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.Returns
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.RollingLinearRegressionOfReturns
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.RollingPearsonOfReturns
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.RollingSpearmanOfReturns
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.RSI
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.SimpleMovingAverage
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.VWAP
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.factors.WeightedAverageValue
|
||||
.. :members:
|
||||
|
||||
.. Pipeline Engine
|
||||
.. ```````````````
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.engine.PipelineEngine
|
||||
.. :members: run_pipeline, run_chunked_pipeline
|
||||
.. :member-order: bysource
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.engine.SimplePipelineEngine
|
||||
.. :members: __init__, run_pipeline, run_chunked_pipeline
|
||||
.. :member-order: bysource
|
||||
|
||||
.. .. autofunction:: zipline.pipeline.engine.default_populate_initial_workspace
|
||||
|
||||
.. Data Loaders
|
||||
.. ````````````
|
||||
|
||||
.. .. autoclass:: zipline.pipeline.loaders.equity_pricing_loader.USEquityPricingLoader
|
||||
.. :members: __init__, from_files, load_adjusted_array
|
||||
.. :member-order: bysource
|
||||
|
||||
Asset Metadata
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
.. autoclass:: catalyst.assets.Asset
|
||||
:members:
|
||||
|
||||
.. autoclass:: catalyst.assets.AssetConvertible
|
||||
:members:
|
||||
|
||||
|
||||
Trading Calendar API
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. autofunction:: catalyst.utils.calendars.get_calendar
|
||||
|
||||
.. autoclass:: catalyst.utils.calendars.TradingCalendar
|
||||
:members:
|
||||
|
||||
.. autofunction:: catalyst.utils.calendars.register_calendar
|
||||
|
||||
.. autofunction:: catalyst.utils.calendars.register_calendar_type
|
||||
|
||||
.. autofunction:: catalyst.utils.calendars.deregister_calendar
|
||||
|
||||
.. autofunction:: catalyst.utils.calendars.clear_calendars
|
||||
|
||||
|
||||
Data API
|
||||
~~~~~~~~
|
||||
|
||||
.. Writers
|
||||
.. ```````
|
||||
.. .. autoclass:: zipline.data.minute_bars.BcolzMinuteBarWriter
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.data.us_equity_pricing.BcolzDailyBarWriter
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.data.us_equity_pricing.SQLiteAdjustmentWriter
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.assets.AssetDBWriter
|
||||
.. :members:
|
||||
|
||||
.. Readers
|
||||
.. ```````
|
||||
.. .. autoclass:: zipline.data.minute_bars.BcolzMinuteBarReader
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.data.us_equity_pricing.BcolzDailyBarReader
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.data.us_equity_pricing.SQLiteAdjustmentReader
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.assets.AssetFinder
|
||||
.. :members:
|
||||
|
||||
.. .. autoclass:: zipline.data.data_portal.DataPortal
|
||||
.. :members:
|
||||
|
||||
.. Bundles
|
||||
.. ```````
|
||||
.. .. autofunction:: zipline.data.bundles.register
|
||||
|
||||
.. .. autofunction:: zipline.data.bundles.ingest(name, environ=os.environ, date=None, show_progress=True)
|
||||
|
||||
.. .. autofunction:: zipline.data.bundles.load(name, environ=os.environ, date=None)
|
||||
|
||||
.. .. autofunction:: zipline.data.bundles.unregister
|
||||
|
||||
.. .. data:: zipline.data.bundles.bundles
|
||||
|
||||
.. The bundles that have been registered as a mapping from bundle name to bundle
|
||||
.. data. This mapping is immutable and should only be updated through
|
||||
.. :func:`~zipline.data.bundles.register` or
|
||||
.. :func:`~zipline.data.bundles.unregister`.
|
||||
|
||||
.. .. autofunction:: zipline.data.bundles.yahoo_equities
|
||||
|
||||
|
||||
|
||||
Utilities
|
||||
~~~~~~~~~
|
||||
|
||||
Caching
|
||||
```````
|
||||
|
||||
.. autoclass:: catalyst.utils.cache.CachedObject
|
||||
|
||||
.. autoclass:: catalyst.utils.cache.ExpiringCache
|
||||
|
||||
.. autoclass:: catalyst.utils.cache.dataframe_cache
|
||||
|
||||
.. autoclass:: catalyst.utils.cache.working_file
|
||||
|
||||
.. autoclass:: catalyst.utils.cache.working_dir
|
||||
|
||||
Command Line
|
||||
````````````
|
||||
.. autofunction:: catalyst.utils.cli.maybe_show_progress
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,354 +0,0 @@
|
||||
.. _data-bundles:
|
||||
|
||||
Data Bundles
|
||||
------------
|
||||
|
||||
A data bundle is a collection of pricing data, adjustment data, and an asset
|
||||
database. Bundles allow us to preload all of the data we will need to run
|
||||
backtests and store the data for future runs.
|
||||
|
||||
.. _bundles-command:
|
||||
|
||||
Discovering Available Bundles
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Zipline comes with a few bundles by default as well as the ability to register
|
||||
new bundles. To see which bundles we have have available, we may run the
|
||||
``bundles`` command, for example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ zipline bundles
|
||||
my-custom-bundle 2016-05-05 20:35:19.809398
|
||||
my-custom-bundle 2016-05-05 20:34:53.654082
|
||||
my-custom-bundle 2016-05-05 20:34:48.401767
|
||||
quandl <no ingestions>
|
||||
quantopian-quandl 2016-05-05 20:06:40.894956
|
||||
|
||||
The output here shows that there are 3 bundles available:
|
||||
|
||||
- ``my-custom-bundle`` (added by the user)
|
||||
- ``quandl`` (provided by zipline)
|
||||
- ``quantopian-quandl`` (provided by zipline)
|
||||
|
||||
The dates and times next to the name show the times when the data for this
|
||||
bundle was ingested. We have run three different ingestions for
|
||||
``my-custom-bundle``. We have never ingested any data for the ``quandl`` bundle
|
||||
so it just shows ``<no ingestions>`` instead. Finally, there is only one
|
||||
ingestion for ``quantopian-quandl``.
|
||||
|
||||
.. _ingesting-data:
|
||||
|
||||
Ingesting Data
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
The first step to using a data bundle is to ingest the data. The ingestion
|
||||
process will invoke some custom bundle command and then write the data to a
|
||||
standard location that zipline can find. By default the location where ingested
|
||||
data will be written is ``$ZIPLINE_ROOT/data/<bundle>`` where by default
|
||||
``ZIPLINE_ROOT=~/.zipline``. The ingestion step may take some time as it could
|
||||
involve downloading and processing a lot of data. This can be run with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ zipline ingest [-b <bundle>]
|
||||
|
||||
|
||||
where ``<bundle>`` is the name of the bundle to ingest, defaulting to
|
||||
:ref:`quantopian-quandl <quantopian-quandl-mirror>`.
|
||||
|
||||
Old Data
|
||||
~~~~~~~~
|
||||
|
||||
When the ``ingest`` command is used it will write the new data to a subdirectory
|
||||
of ``$ZIPLINE_ROOT/data/<bundle>`` which is named with the current date. This
|
||||
makes it possible to look at older data or even run backtests with the older
|
||||
copies. Running a backtest with an old ingestion makes it easier to reproduce
|
||||
backtest results later.
|
||||
|
||||
One drawback of saving all of the data by default is that the data directory
|
||||
may grow quite large even if you do not want to use the data. As shown earlier,
|
||||
we can list all of the ingestions with the :ref:`bundles command
|
||||
<bundles-command>`. To solve the problem of leaking old data there is another
|
||||
command: ``clean``, which will clear data bundles based on some time
|
||||
constraints.
|
||||
|
||||
For example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# clean everything older than <date>
|
||||
$ zipline clean [-b <bundle>] --before <date>
|
||||
|
||||
# clean everything newer than <date>
|
||||
$ zipline clean [-b <bundle>] --after <date>
|
||||
|
||||
# keep everything in the range of [before, after] and delete the rest
|
||||
$ zipline clean [-b <bundle>] --before <date> --after <after>
|
||||
|
||||
# clean all but the last <int> runs
|
||||
$ zipline clean [-b <bundle>] --keep-last <int>
|
||||
|
||||
|
||||
Running Backtests with Data Bundles
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Now that the data has been ingested we can use it to run backtests with the
|
||||
``run`` command. The bundle to use can be specified with the ``--bundle`` option
|
||||
like:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ zipline run --bundle <bundle> --algofile algo.py ...
|
||||
|
||||
|
||||
We may also specify the date to use to look up the bundle data with the
|
||||
``--bundle-date`` option. Setting the ``--bundle-date`` will cause run to use
|
||||
the most recent bundle ingestion that is less than or equal to the
|
||||
``bundle-date``. This is how we can run backtests with older data. The reason
|
||||
that ``-bundle-date`` uses a less than or equal to relationship is that we can
|
||||
specify the date that we ran an old backtest and get the same data that would
|
||||
have been available to us on that date. The ``bundle-date`` defaults to the
|
||||
current day to use the most recent data.
|
||||
|
||||
Default Data Bundles
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. _quandl-data-bundle:
|
||||
|
||||
Quandl WIKI Bundle
|
||||
``````````````````
|
||||
|
||||
By default zipline comes with the ``quandl`` data bundle which uses quandl's
|
||||
`WIKI dataset <https://www.quandl.com/data/WIKI>`_. The quandl data bundle
|
||||
includes daily pricing data, splits, cash dividends, and asset metadata. To
|
||||
ingest the ``quandl`` data bundle we recommend creating an account on quandl.com
|
||||
to get an API key to be able to make more API requests per day. Once we have an
|
||||
API key we may run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ QUANDL_API_KEY=<api-key> zipline ingest -b quandl
|
||||
|
||||
though we may still run ``ingest`` as an anonymous quandl user (with no API
|
||||
key). We may also set the ``QUANDL_DOWNLOAD_ATTEMPTS`` environment variable to
|
||||
an integer which is the number of attempts that should be made to download data
|
||||
from quandls servers. By default ``QUANDL_DOWNLOAD_ATTEMPTS`` will be 5, meaning
|
||||
that we will retry each attempt 5 times.
|
||||
|
||||
.. note::
|
||||
|
||||
``QUANDL_DOWNLOAD_ATTEMPTS`` is not the total number of allowed failures,
|
||||
just the number of allowed failures per request. The quandl loader will make
|
||||
one request per 100 equities for the metadata followed by one request per
|
||||
equity.
|
||||
|
||||
|
||||
.. _quantopian-quandl-mirror:
|
||||
|
||||
Quantopian Quandl WIKI Mirror
|
||||
'''''''''''''''''''''''''''''
|
||||
|
||||
Quantopian provides a mirror of the quandl WIKI dataset with the data in the
|
||||
formats that zipline expects. This is available under the name:
|
||||
``quantopian-quandl`` and is the default bundle for zipline.
|
||||
|
||||
Yahoo Bundle Factories
|
||||
``````````````````````
|
||||
|
||||
Zipline also ships with a factory function for creating a data bundle out of a
|
||||
set of tickers from yahoo: :func:`~zipline.data.bundles.yahoo_equities`.
|
||||
:func:`~zipline.data.bundles.yahoo_equities` makes it easy to pre-download and
|
||||
cache the data for a set of equities from yahoo. The yahoo bundles include daily
|
||||
pricing data along with splits, cash dividends, and inferred asset metadata. To
|
||||
create a bundle from a set of equities, add the following to your
|
||||
``~/.zipline/extensions.py`` file:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from zipline.data.bundles import register, yahoo_equities
|
||||
|
||||
# these are the tickers you would like data for
|
||||
equities = {
|
||||
'AAPL',
|
||||
'MSFT',
|
||||
'GOOG',
|
||||
}
|
||||
register(
|
||||
'my-yahoo-equities-bundle', # name this whatever you like
|
||||
yahoo_equities(equities),
|
||||
)
|
||||
|
||||
|
||||
This may now be used like:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ zipline ingest -b my-yahoo-equities-bundle
|
||||
$ zipline run -f algo.py --bundle my-yahoo-equities-bundle
|
||||
|
||||
|
||||
More than one yahoo equities bundle may be registered as long as they use
|
||||
different names.
|
||||
|
||||
Writing a New Bundle
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Data bundles exist to make it easy to use different data sources with
|
||||
zipline. To add a new bundle, one must implement an ``ingest`` function.
|
||||
|
||||
The ``ingest`` function is responsible for loading the data into memory and
|
||||
passing it to a set of writer objects provided by zipline to convert the data to
|
||||
zipline's internal format. The ingest function may work by downloading data from
|
||||
a remote location like the ``quandl`` bundle or yahoo bundles or it may just
|
||||
load files that are already on the machine. The function is provided with
|
||||
writers that will write the data to the correct location transactionally. If an
|
||||
ingestion fails part way through the bundle will not be written in an incomplete
|
||||
state.
|
||||
|
||||
The signature of the ingest function should be:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ingest(environ,
|
||||
asset_db_writer,
|
||||
minute_bar_writer,
|
||||
daily_bar_writer,
|
||||
adjustment_writer,
|
||||
calendar,
|
||||
start_session,
|
||||
end_session,
|
||||
cache,
|
||||
show_progress,
|
||||
output_dir)
|
||||
|
||||
``environ``
|
||||
```````````
|
||||
|
||||
``environ`` is a mapping representing the environment variables to use. This is
|
||||
where any custom arguments needed for the ingestion should be passed, for
|
||||
example: the ``quandl`` bundle uses the enviornment to pass the API key and the
|
||||
download retry attempt count.
|
||||
|
||||
``asset_db_writer``
|
||||
```````````````````
|
||||
|
||||
``asset_db_writer`` is an instance of :class:`~zipline.assets.AssetDBWriter`.
|
||||
This is the writer for the asset metadata which provides the asset lifetimes and
|
||||
the symbol to asset id (sid) mapping. This may also contain the asset name,
|
||||
exchange and a few other columns. To write data, invoke
|
||||
:meth:`~zipline.assets.AssetDBWriter.write` with dataframes for the various
|
||||
pieces of metadata. More information about the format of the data exists in the
|
||||
docs for write.
|
||||
|
||||
``minute_bar_writer``
|
||||
`````````````````````
|
||||
|
||||
``minute_bar_writer`` is an instance of
|
||||
:class:`~zipline.data.minute_bars.BcolzMinuteBarWriter`. This writer is used to
|
||||
convert data to zipline's internal bcolz format to later be read by a
|
||||
:class:`~zipline.data.minute_bars.BcolzMinuteBarReader`. If minute data is
|
||||
provided, users should call
|
||||
:meth:`~zipline.data.minute_bars.BcolzMinuteBarWriter.write` with an iterable of
|
||||
(sid, dataframe) tuples. The ``show_progress`` argument should also be forwarded
|
||||
to this method. If the data source does not provide minute level data, then
|
||||
there is no need to call the write method. It is also acceptable to pass an
|
||||
empty iterator to :meth:`~zipline.data.minute_bars.BcolzMinuteBarWriter.write`
|
||||
to signal that there is no minutely data.
|
||||
|
||||
.. note::
|
||||
|
||||
The data passed to
|
||||
:meth:`~zipline.data.minute_bars.BcolzMinuteBarWriter.write` may be a lazy
|
||||
iterator or generator to avoid loading all of the minute data into memory at
|
||||
a single time. A given sid may also appear multiple times in the data as long
|
||||
as the dates are strictly increasing.
|
||||
|
||||
``daily_bar_writer``
|
||||
````````````````````
|
||||
|
||||
``daily_bar_writer`` is an instance of
|
||||
:class:`~zipline.data.us_equity_pricing.BcolzDailyBarWriter`. This writer is
|
||||
used to convert data into zipline's internal bcolz format to later be read by a
|
||||
:class:`~zipline.data.us_equity_pricing.BcolzDailyBarReader`. If daily data is
|
||||
provided, users should call
|
||||
:meth:`~zipline.data.minute_bars.BcolzDailyBarWriter.write` with an iterable of
|
||||
(sid dataframe) tuples. The ``show_progress`` argument should also be forwarded
|
||||
to this method. If the data shource does not provide daily data, then there is
|
||||
no need to call the write method. It is also acceptable to pass an empty
|
||||
iterable to :meth:`~zipline.data.minute_bars.BcolzMinuteBarWriter.write` to
|
||||
signal that there is no daily data. If no daily data is provided but minute data
|
||||
is provided, a daily rollup will happen to service daily history requests.
|
||||
|
||||
.. note::
|
||||
|
||||
Like the ``minute_bar_writer``, the data passed to
|
||||
:meth:`~zipline.data.minute_bars.BcolzMinuteBarWriter.write` may be a lazy
|
||||
iterable or generator to avoid loading all of the data into memory at once.
|
||||
Unlike the ``minute_bar_writer``, a sid may only appear once in the data
|
||||
iterable.
|
||||
|
||||
``adjustment_writer``
|
||||
`````````````````````
|
||||
|
||||
``adjustment_writer`` is an instance of
|
||||
:class:`~zipline.data.us_equity_pricing.SQLiteAdjustmentWriter`. This writer is
|
||||
used to store splits, mergers, dividends, and stock dividends. The data should
|
||||
be provided as dataframes and passed to
|
||||
:meth:`~zipline.data.us_equity_pricing.SQLiteAdjustmentWriter.write`. Each of
|
||||
these fields are optional, but the writer can accept as much of the data as you
|
||||
have.
|
||||
|
||||
``calendar``
|
||||
````````````
|
||||
|
||||
``calendar`` is an instance of
|
||||
:class:`zipline.utils.calendars.TradingCalendar`. The calendar is provided to
|
||||
help some bundles generate queries for the days needed.
|
||||
|
||||
``start_session``
|
||||
`````````````````
|
||||
|
||||
``start_session`` is a :class:`pandas.Timestamp` object indicating the first
|
||||
day that the bundle should load data for.
|
||||
|
||||
``end_session``
|
||||
```````````````
|
||||
|
||||
``end_session`` is a :class:`pandas.Timestamp` object indicating the last day
|
||||
that the bundle should load data for.
|
||||
|
||||
``cache``
|
||||
`````````
|
||||
|
||||
``cache`` is an instance of :class:`~zipline.utils.cache.dataframe_cache`. This
|
||||
object is a mapping from strings to dataframes. This object is provided in case
|
||||
an ingestion crashes part way through. The idea is that the ingest function
|
||||
should check the cache for raw data, if it doesn't exist in the cache, it should
|
||||
acquire it and then store it in the cache. Then it can parse and write the
|
||||
data. The cache will be cleared only after a successful load, this prevents the
|
||||
ingest function from needing to redownload all the data if there is some bug in
|
||||
the parsing. If it is very fast to get the data, for example if it is coming
|
||||
from another local file, then there is no need to use this cache.
|
||||
|
||||
``show_progress``
|
||||
`````````````````
|
||||
|
||||
``show_progress`` is a boolean indicating that the user would like to receive
|
||||
feedback about the ingest function's progress fetching and writing the
|
||||
data. Some examples for where to show how many files you have downloaded out of
|
||||
the total needed, or how far into some data conversion the ingest function
|
||||
is. One tool that may help with implementing ``show_progress`` for a loop is
|
||||
:class:`~zipline.utils.cli.maybe_show_progress`. This argument should always be
|
||||
forwarded to ``minute_bar_writer.write`` and ``daily_bar_writer.write``.
|
||||
|
||||
|
||||
``output_dir``
|
||||
``````````````
|
||||
|
||||
``output_dir`` is a string representing the file path where all the data will be
|
||||
written. ``output_dir`` will be some subdirectory of ``$ZIPLINE_ROOT`` and will
|
||||
contain the time of the start of the current ingestion. This can be used to
|
||||
directly move resources here if for some reason your ingest function can produce
|
||||
it's own outputs without the writers. For example, the ``quantopian:quandl``
|
||||
bundle uses this to directly untar the bundle into the ``output_dir``.
|
||||
@@ -1,144 +0,0 @@
|
||||
Development Guidelines
|
||||
======================
|
||||
This page is intended for developers of Catalyst, people who want to contribute to the Catalyst codebase or documentation, or people who want to install from source and make local changes to their copy of Catalyst.
|
||||
|
||||
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We `track issues <https://github.com/enigmampc/catalyst/issues>`_ on `GitHub <https://github.com/enigmampc/catalyst>`_ and also have a `discord group <https://discord.gg/SJK32GY>`_ where you can ask questions.
|
||||
|
||||
Creating a Development Environment
|
||||
----------------------------------
|
||||
|
||||
First, you'll need to clone Catalyst by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git clone git@github.com:enigmampc/catalyst.git
|
||||
|
||||
Then check out to a new branch where you can make your changes:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git checkout -b some-short-descriptive-name
|
||||
|
||||
If you don't already have them, you'll need some C library dependencies. You can follow the `install guide <install.html>`_ to get the appropriate dependencies.
|
||||
|
||||
The following section assumes you already have virtualenvwrapper and pip installed on your system. Suggested installation of Python library dependencies used for development:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ mkvirtualenv catalyst
|
||||
$ ./etc/ordered_pip.sh ./etc/requirements.txt
|
||||
$ pip install -r ./etc/requirements_dev.txt
|
||||
$ pip install -r ./etc/requirements_blaze.txt
|
||||
|
||||
Finally, you can build the C extensions by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python setup.py build_ext --inplace
|
||||
|
||||
Development with Docker
|
||||
-----------------------
|
||||
|
||||
If you want to work with zipline using a `Docker`__ container, you'll need to
|
||||
build the ``Dockerfile`` in the Zipline root directory, and then build
|
||||
``Dockerfile-dev``. Instructions for building both containers can be found in
|
||||
``Dockerfile`` and ``Dockerfile-dev``, respectively.
|
||||
|
||||
__ https://docs.docker.com/get-started/
|
||||
|
||||
Git Branching Structure
|
||||
-----------------------
|
||||
|
||||
If you want to contribute to the codebase of Catalyst, familiarize yourself with our branching structure, a fairly standardized one for that matter, that follows what is documented in the following article: `A successful Git branching model <http://nvie.com/posts/a-successful-git-branching-model/>`_. To contribute, create your local branch and submit a Pull Request (PR) to the **develop** branch.
|
||||
|
||||
.. image:: https://camo.githubusercontent.com/9bde6fb64a9542a572e0e2017cbb58d9d2c440ac/687474703a2f2f6e7669652e636f6d2f696d672f6769742d6d6f64656c4032782e706e67
|
||||
|
||||
Contributing to the Docs
|
||||
------------------------
|
||||
|
||||
If you'd like to contribute to the documentation on enigmampc.github.io, you can navigate to ``docs/source/`` where each `reStructuredText <https://en.wikipedia.org/wiki/ReStructuredText>`_ file is a separate section there. To add a section, create a new file called ``some-descriptive-name.rst`` and add ``some-descriptive-name`` to ``index.rst``. To edit a section, simply open up one of the existing files, make your changes, and save them.
|
||||
|
||||
We use `Sphinx <http://www.sphinx-doc.org/en/stable/>`_ to generate documentation for Catalyst, which you will need to install by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install -r ./etc/requirements_docs.txt
|
||||
|
||||
To build and view the docs locally, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# assuming you're in the Catalyst root directory
|
||||
$ cd docs
|
||||
$ make html
|
||||
$ {BROWSER} build/html/index.html
|
||||
|
||||
|
||||
There is a `documented issue <https://github.com/sphinx-doc/sphinx/issues/3212>`_
|
||||
with ``sphinx`` and ``docutils`` that causes the error below when trying to build
|
||||
the docs.
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
Exception occurred:
|
||||
File "(...)/env-c/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
|
||||
assert not self.context, 'len(context) = %s' % len(self.context)
|
||||
AssertionError: len(context) = 3
|
||||
|
||||
If you get this error, you need to downgrade your version of ``docutils`` as
|
||||
follows, and build the docs again:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install docutils==0.12
|
||||
|
||||
|
||||
Commit messages
|
||||
---------------
|
||||
|
||||
Standard prefixes to start a commit message:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
BLD: change related to building Catalyst
|
||||
BUG: bug fix
|
||||
DEP: deprecate something, or remove a deprecated object
|
||||
DEV: development tool or utility
|
||||
DOC: documentation
|
||||
ENH: enhancement
|
||||
MAINT: maintenance commit (refactoring, typos, etc)
|
||||
REV: revert an earlier commit
|
||||
STY: style fix (whitespace, PEP8, flake8, etc)
|
||||
TST: addition or modification of tests
|
||||
REL: related to releasing Catalyst
|
||||
PERF: performance enhancements
|
||||
|
||||
|
||||
Some commit style guidelines:
|
||||
|
||||
Commit lines should be no longer than `72 characters <https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project>`_. The first line of the commit should include one of the above prefixes. There should be an empty line between the commit subject and the body of the commit. In general, the message should be in the imperative tense. Best practice is to include not only what the change is, but why the change was made.
|
||||
|
||||
**Example:**
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
MAINT: Remove unused calculations of max_leverage, et al.
|
||||
|
||||
In the performance period the max_leverage, max_capital_used,
|
||||
cumulative_capital_used were calculated but not used.
|
||||
|
||||
At least one of those calculations, max_leverage, was causing a
|
||||
divide by zero error.
|
||||
|
||||
Instead of papering over that error, the entire calculation was
|
||||
a bit suspect so removing, with possibility of adding it back in
|
||||
later with handling the case (or raising appropriate errors) when
|
||||
the algorithm has little cash on hand.
|
||||
|
||||
|
||||
Formatting Docstrings
|
||||
---------------------
|
||||
|
||||
When adding or editing docstrings for classes, functions, etc, we use `numpy <https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt>`_ as the canonical reference.
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,122 +0,0 @@
|
||||
Features
|
||||
========
|
||||
|
||||
This page describes the features that Catalyst provides in the current version,
|
||||
and what is planned for future releases.
|
||||
|
||||
Current Functionality
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* Backtesting and live-trading modes to run your trading algorithms, with a
|
||||
seamless transition between the two.
|
||||
* Paper trading simulates order in live-trading mode.
|
||||
* Support for 3 exchanges: Bitfinex, Bittrex and Poloniex in both modes
|
||||
(backtesting and live-trading). Historical data for backtesting is provided
|
||||
with daily resolution for all three exchanges, and minute resolution for
|
||||
Bitfinex and Poloniex. No minute-resolution data is currently available for
|
||||
Bittrex. Refer to
|
||||
`Catalyst Market Coverage <https://www.enigma.co/catalyst/status>`_ for
|
||||
details.
|
||||
* Interface with over 90 exchanges available in live and paper trading modes.
|
||||
* Granular commission models which closely simulates each exchange fee
|
||||
structure in backtesting and paper trading.
|
||||
* Standardized naming convention for all asset pairs trading on any exchange in
|
||||
the form ``{market_currency}_{base_currency}``. See
|
||||
:ref:`naming`.
|
||||
* Output of performance statistics based on Pandas DataFrames to integrate
|
||||
nicely into the existing PyData ecosystem.
|
||||
* Support for accessing multiple exchanges per algorithm, which opens the door
|
||||
to cross-exchange arbitrage opportunities.
|
||||
* Support for running multiple algorithms on the same exchange independently of
|
||||
one another. Catalyst performance tracker stores just enough data to allow
|
||||
algorithms to run independently while still sharing critical data through
|
||||
exchanges.
|
||||
* Benchmark defaults to Bitcoin price (btc_usdt in Poloniex exchange) for the
|
||||
purpose of comparing performance across trading algorithms. A custom benchmark
|
||||
can be specified through ``set_benchmark()`` (but see
|
||||
`issue #86 <https://github.com/enigmampc/catalyst/issues/86>`_).
|
||||
* Support for MacOS, Linux and Windows installations.
|
||||
* Support for Python2 and Python3.
|
||||
|
||||
For additional details on the functionality added on recent releases, see the
|
||||
:doc:`Release Notes<releases>`.
|
||||
|
||||
Upcoming features
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
* Additional datasets beyond pricing data (Q1 2018)
|
||||
* API documentation (Q1 2018)
|
||||
* Support for decentralized exchanges (Q1 2018)
|
||||
* Support for data ingestion of community-contributed data sets (Q1 2018)
|
||||
* Pipeline support (Q1 2018)
|
||||
* Web UI (Q2 2018)
|
||||
|
||||
|
||||
.. _naming:
|
||||
|
||||
Naming Convention
|
||||
~~~~~~~~~~~~~~~~~
|
||||
|
||||
Catalyst introduces a standardized naming convention for all asset pairs
|
||||
trading on any exchange in the following form:
|
||||
|
||||
|
||||
**{market_currency}_{base_currency}**
|
||||
|
||||
Where {market_currency} is the asset to be traded using {base_currency} as
|
||||
the reference, both written in lowercase and separated with an underscore.
|
||||
|
||||
This standardization is needed to overcome the lack of consistency in the
|
||||
naming of assets across different exchanges, and making it easier to the user
|
||||
to refer to the asset pairs that you want to trade.
|
||||
|
||||
Catalyst maintains a `Market Coverage Overview <https://www.enigma.co/catalyst/status>`_
|
||||
where you can check the mapping between Catalyst naming pairs and that of each
|
||||
exchange. Catalyst will always expect in all its functions that you will refer to
|
||||
the asset pairs by using the Catalyst naming convention.
|
||||
|
||||
If at any point, you input the wrong name for an asset pair, you will get an error
|
||||
of that pair not found in the given exchange, and a list of pairs available on that exchange:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ catalyst ingest-exchange -x poloniex -i btc_usd
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Ingesting exchange bundle poloniex...
|
||||
Error traceback: /Volumes/Data/Users/victoris/Desktop/Enigma/user-install/catalyst-dev/catalyst/exchange/exchange.py (line 175)
|
||||
SymbolNotFoundOnExchange: Symbol btc_usd not found on exchange Poloniex.
|
||||
Choose from: ['rep_usdt', 'gno_btc', 'xvc_btc', 'pink_btc', 'sys_btc',
|
||||
'emc2_btc', 'rads_btc', 'note_btc', 'maid_btc', 'bch_btc', 'gnt_btc',
|
||||
'bcn_btc', 'rep_btc', 'bcy_btc', 'cvc_btc', 'nxt_xmr', 'zec_usdt',
|
||||
'fct_btc', 'gas_btc', 'pot_btc', 'eth_usdt', 'btc_usdt', 'lbc_btc',
|
||||
'dcr_btc', 'etc_usdt', 'omg_eth', 'amp_btc', 'xpm_btc', 'nxt_btc',
|
||||
'vtc_btc', 'steem_eth', 'blk_xmr', 'pasc_btc', 'zec_xmr', 'grc_btc',
|
||||
'nxc_btc', 'btcd_btc', 'ltc_btc', 'dash_btc', 'naut_btc', 'zec_eth',
|
||||
'zec_btc', 'burst_btc', 'zrx_eth', 'bela_btc', 'steem_btc', 'etc_btc',
|
||||
'eth_btc', 'huc_btc', 'strat_btc', 'lsk_btc', 'exp_btc', 'clam_btc',
|
||||
'rep_eth', 'dash_xmr', 'cvc_eth', 'bch_usdt', 'zrx_btc', 'dash_usdt',
|
||||
'blk_btc', 'xrp_btc', 'nxt_usdt', 'neos_btc', 'omg_btc', 'bts_btc',
|
||||
'doge_btc', 'gnt_eth', 'sbd_btc', 'gno_eth', 'xcp_btc', 'ltc_usdt',
|
||||
'btm_btc', 'xmr_usdt', 'lsk_eth', 'omni_btc', 'nav_btc', 'fldc_btc',
|
||||
'ppc_btc', 'xbc_btc', 'dgb_btc', 'sc_btc', 'btcd_xmr', 'vrc_btc',
|
||||
'ric_btc', 'str_btc', 'maid_xmr', 'xmr_btc', 'sjcx_btc', 'via_btc',
|
||||
'xem_btc', 'nmc_btc', 'etc_eth', 'ltc_xmr', 'ardr_btc', 'gas_eth',
|
||||
'flo_btc', 'xrp_usdt', 'game_btc', 'bch_eth', 'bcn_xmr', 'str_usdt']
|
||||
|
||||
In the example above, exchange Poloniex does not use USD, but uses instead the
|
||||
USDT cryptocurrency asset that is issued on the Bitcoin blockchain via the Omni
|
||||
Layer Protocol. Each USDT unit is backed by a U.S Dollar held in the reserves of
|
||||
Tether Limited. USDT can be transferred, stored, and spent, just like bitcoins
|
||||
or any other cryptocurrency. Given its 1:1 mapping to the USD, is a viable alternative.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ catalyst ingest-exchange -x poloniex -i btc_usdt
|
||||
|
||||
.. parsed-literal::
|
||||
|
||||
Ingesting exchange bundle poloniex...
|
||||
[====================================] Fetching poloniex daily candles: : 100%
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
.. include:: ../../README.rst
|
||||
|
||||
|
|
||||
|
|
||||
|
||||
Table of Contents
|
||||
-----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
install
|
||||
beginner-tutorial
|
||||
live-trading
|
||||
features
|
||||
example-algos
|
||||
utilities
|
||||
videos
|
||||
resources
|
||||
development-guidelines
|
||||
releases
|
||||
.. bundles
|
||||
.. appendix
|
||||
.. release-process
|
||||
|
||||
@@ -1,545 +0,0 @@
|
||||
Install
|
||||
=======
|
||||
|
||||
To get started with Catalyst, you will need to install it in your computer.
|
||||
Like any other piece of software, Catalyst has a number of dependencies
|
||||
(other software on which it depends to run) that you will need to install, as
|
||||
well. We recommend using a software named ``Conda`` that will manage all
|
||||
these dependencies for you, and set up the environment needed to get you up
|
||||
and running as easily as possible. This is the recommended installation method
|
||||
for Windows, MacOS and Linux. See :ref:`Installing with Conda <conda>`.
|
||||
|
||||
What conda does is create a pre-configured environment, and inside that
|
||||
environment install Catalyst using ``pip``, Python's package manager. Thus,
|
||||
as an alternative installation method for MacOS and Linux, you can install
|
||||
Catalyst directly with ``pip`` (we recommend in combination with a virtual
|
||||
environemnt). See :ref:`Installing with pip <pip>`.
|
||||
|
||||
Alternatively you can install Catalyst using ``pipenv`` which is a mix of pip
|
||||
and virtualenv. See :ref:`Installing with pipenv <pipenv>`.
|
||||
|
||||
Regardless of the method, each operating system (OS), has its own
|
||||
prerequisites, make sure to review the corresponding sections for your system:
|
||||
:ref:`Linux <linux>`, :ref:`MacOS <macos>` and :ref:`Windows <windows>`.
|
||||
|
||||
.. _conda:
|
||||
|
||||
Installing with ``conda``
|
||||
-------------------------
|
||||
|
||||
The preferred method to install Catalyst is via the ``conda`` package manager,
|
||||
which comes as part of Continuum Analytics' `Anaconda
|
||||
<http://continuum.io/downloads>`_ distribution.
|
||||
|
||||
The primary advantage of using Conda over ``pip`` is that conda natively
|
||||
understands the complex binary dependencies of packages like ``numpy`` and
|
||||
``scipy``. This means that ``conda`` can install Catalyst and its
|
||||
dependencies without requiring the use of a second tool to acquire Catalyst's
|
||||
non-Python dependencies.
|
||||
|
||||
For Windows, you will first need to install the *Microsoft Visual C++
|
||||
Compiler for Python 2.7*. Follow the instructions on the :ref:`Windows
|
||||
<windows>` section and come back here.
|
||||
|
||||
For instructions on how to install ``conda``, see the `Conda Installation
|
||||
Documentation <http://conda.pydata.org/docs/download.html>`_. Alternatively,
|
||||
you can install MiniConda, which is a smaller footprint (fewer packages and
|
||||
smaller size) than its big brother Anaconda, but it still contains all the
|
||||
main packages needed. To install MiniConda, you can follow these steps:
|
||||
|
||||
1. Download `MiniConda <https://conda.io/miniconda.html>`_. Select Python 2.7
|
||||
for your Operating System.
|
||||
2. Install MiniConda. See the `Installation Instructions
|
||||
<https://conda.io/docs/user-guide/install/index.html>`_ if you need help.
|
||||
3. Ensure the correct installation by running ``conda list`` in a Terminal
|
||||
window, which should print the list of packages installed with Conda.
|
||||
|
||||
For Windows, if you accepted the default installation options, you didn't
|
||||
check an option to add Conda to the PATH, so trying to run ``conda`` from
|
||||
a regular ``Command Prompt`` will result in the following error: ``'conda'
|
||||
is no recognized as an internal or external command, operatble program or
|
||||
batch file``. That's to be expected. You will nee to launch an ``Anaconda
|
||||
Prompt`` that was added at installation time to your list of programs
|
||||
available from the Start menu.
|
||||
|
||||
Once either Conda or MiniConda has been set up you can install Catalyst:
|
||||
|
||||
1. Download the file `python2.7-environment.yml
|
||||
<https://github.com/enigmampc/catalyst/blob/master/etc/python2.7-environment.yml>`_.
|
||||
|
||||
To download, simply click on the 'Raw' button and save the file locally
|
||||
to a folder you can remember. Make sure that the file gets saved with the
|
||||
``.yml`` extension, and nothing like a ``.txt`` file or anything else.
|
||||
|
||||
2. Open a Terminal window and enter [``cd/dir``] into the directory where you
|
||||
saved the above ``python2.7-environment.yml`` file.
|
||||
|
||||
3. Install using this file. This step can take about 5-10 minutes to install.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
conda env create -f python2.7-environment.yml
|
||||
|
||||
4. Activate the environment (which you need to do every time you start a new
|
||||
session to run Catalyst):
|
||||
|
||||
**Linux or MacOS:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
source activate catalyst
|
||||
|
||||
**Windows:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
activate catalyst
|
||||
|
||||
5. Verify that Catalyst is install correctly:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
catalyst --version
|
||||
|
||||
which should display the current version.
|
||||
|
||||
Congratulations! You now have Catalyst installed.
|
||||
|
||||
Troubleshooting ``conda`` Install
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If the command ``conda env create -f python2.7-environment.yml`` in step 3
|
||||
above failed for any reason, you can try setting up the environment manually
|
||||
with the following steps:
|
||||
|
||||
1. If the above installation failed, and you have a partially set up catalyst
|
||||
environment, remove it first. If you are starting from scratch, proceed to
|
||||
step #2:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
conda env remove --name catalyst
|
||||
|
||||
2. Create the environment:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
conda create --name catalyst python=2.7 scipy zlib
|
||||
|
||||
3. Activate the environment:
|
||||
|
||||
**Linux or MacOS:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
source activate catalyst
|
||||
|
||||
**Windows:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
activate catalyst
|
||||
|
||||
4. Install the Catalyst inside the environment:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install enigma-catalyst matplotlib
|
||||
|
||||
5. Verify that Catalyst is installed correctly:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
catalyst --version
|
||||
|
||||
which should display the current version.
|
||||
|
||||
Congratulations! You now have Catalyst properly installed.
|
||||
|
||||
.. _pip:
|
||||
|
||||
Installing with ``pip``
|
||||
-----------------------
|
||||
|
||||
Installing Catalyst via ``pip`` is slightly more involved than the average
|
||||
Python package.
|
||||
|
||||
There are two reasons for the additional complexity:
|
||||
|
||||
1. Catalyst ships several C extensions that require access to the CPython C
|
||||
API. In order to build the C extensions, ``pip`` needs access to the
|
||||
CPython header files for your Python installation.
|
||||
|
||||
2. Catalyst depends on `numpy <http://www.numpy.org/>`_, the core library for
|
||||
numerical array computing in Python. Numpy depends on having the `LAPACK
|
||||
<http://www.netlib.org/lapack>`_ linear algebra routines available.
|
||||
|
||||
Because LAPACK and the CPython headers are non-Python dependencies, the
|
||||
correctway to install them varies from platform to platform. If you'd rather
|
||||
use a single tool to install Python and non-Python dependencies, or if you're
|
||||
already using `Anaconda <http://continuum.io/downloads>`_ as your Python
|
||||
distribution, refer to the :ref:`Installing with Conda <conda>` section.
|
||||
|
||||
If you use Python for anything other than Catalyst, we **strongly** recommend
|
||||
that you install in a `virtualenv
|
||||
<https://virtualenv.readthedocs.org/en/latest>`_. The `Hitchhiker's Guide to
|
||||
Python`_ provides an `excellent tutorial on virtualenv
|
||||
<http://docs.python-guide.org/en/latest/dev/virtualenvs/>`_. Here's a
|
||||
summarized version:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install virtualenv
|
||||
$ virtualenv catalyst-venv
|
||||
$ source ./catalyst-venv/bin/activate
|
||||
|
||||
Once you've installed the necessary additional dependencies for your system
|
||||
(:ref:`Linux`, :ref:`MacOS` or :ref:`Windows`) **and have activated your virtualenv**, you should be able to simply run
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install enigma-catalyst matplotlib
|
||||
|
||||
Note that in the command above we install two different packages. The second
|
||||
one, ``matplotlib`` is a visualization library. While it's not strictly
|
||||
required to run catalyst simulations or live trading, it comes in very handy
|
||||
to visualize the performance of your algorithms, and for this reason we
|
||||
recommend you install it, as well.
|
||||
|
||||
|
||||
Troubleshooting ``pip`` Install
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
**Issue**:
|
||||
Package enigma-catalyst cannot be found
|
||||
|
||||
**Solution**:
|
||||
Make sure you have the most up-to-date version of pip installed, by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --upgrade pip
|
||||
|
||||
On Windows, the recommended command is:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python -m pip install --upgrade pip
|
||||
|
||||
----
|
||||
|
||||
**Issue**:
|
||||
Package enigma-catalyst cannot still be found, even after upgrading pip
|
||||
(see above), with an error similar to:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
Downloading/unpacking enigma-catalyst
|
||||
Could not find a version that satisfies the requirement enigma-catalyst
|
||||
(from versions: 0.1.dev9, 0.2.dev2, 0.1.dev4, 0.1.dev5, 0.1.dev3,
|
||||
0.2.dev1, 0.1.dev8, 0.1.dev6)
|
||||
Cleaning up...
|
||||
No distributions matching the version for enigma-catalyst
|
||||
|
||||
**Solution**:
|
||||
In some systems (this error has been reported in Ubuntu), pip is configured
|
||||
to only find stable versions by default. Since Catalyst is in alpha
|
||||
version, pip cannot find a matching version that satisfies the installation
|
||||
requirements. The solution is to include the `--pre` flag to include
|
||||
pre-release and development versions:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --pre enigma-catalyst
|
||||
|
||||
----
|
||||
|
||||
**Issue**:
|
||||
Package enigma-catalyst fails to install because of outdated setuptools
|
||||
|
||||
**Solution**:
|
||||
Upgrade to the most up-to-date setuptools package by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --upgrade pip setuptools
|
||||
|
||||
----
|
||||
|
||||
**Issue**:
|
||||
Missing required packages
|
||||
|
||||
**Solution**:
|
||||
Download `requirements.txt
|
||||
<https://github.com/enigmampc/catalyst/blob/master/etc/requirements.txt>`_
|
||||
(click on the *Raw* button and Right click -> Save As...) and use it to
|
||||
install all the required dependencies by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install -r requirements.txt
|
||||
|
||||
----
|
||||
|
||||
**Issue**:
|
||||
Installation fails with error:
|
||||
``fatal error: Python.h: No such file or directory``
|
||||
|
||||
**Solution**:
|
||||
Some systems (this issue has been reported in Ubuntu) require `python-dev`
|
||||
for the proper build and installation of package dependencies. The solution
|
||||
is to install python-dev, which is independent of the virtual environment.
|
||||
In Ubuntu, you would need to run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo apt-get install python-dev
|
||||
|
||||
.. _pipenv:
|
||||
|
||||
Installing with ``pipenv``
|
||||
--------------------------
|
||||
|
||||
Installing Catalyst via ``pipenv`` is perhaps easier that installing it via
|
||||
``pip`` itself but you need to install ``pipenv`` first via ``pip``.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install pipenv
|
||||
|
||||
Once ``pipenv`` is installed you can proceed by creating a project folder and
|
||||
installing Catalyst on that project automagically as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ mkdir project
|
||||
$ cd project
|
||||
$ pipenv --two
|
||||
$ pipenv install enigma-catalyst matplotlib
|
||||
|
||||
Until now the workflow compared to ``pip`` is almost identical, the difference
|
||||
is that you don't need to load manually any virtualenv however you need to use
|
||||
the `pipenv run` prefix to run the `catalyst` command as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pipenv run catalyst --version
|
||||
|
||||
If you want to know more about ``pipenv`` go to the `pipenv github repo`_
|
||||
|
||||
.. _`pipenv github repo`: https://github.com/pypa/pipenv
|
||||
|
||||
.. _linux:
|
||||
|
||||
GNU/Linux Requirements
|
||||
----------------------
|
||||
|
||||
On `Debian-derived`_ Linux distributions, you can acquire all the necessary
|
||||
binary dependencies from ``apt`` by running:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo apt-get install libatlas-base-dev python-dev gfortran pkg-config libfreetype6-dev
|
||||
|
||||
On recent `RHEL-derived`_ derived Linux distributions (e.g. Fedora), the
|
||||
following should be sufficient to acquire the necessary additional
|
||||
dependencies:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo dnf install atlas-devel gcc-c++ gcc-gfortran libgfortran python-devel redhat-rep-config
|
||||
|
||||
On `Arch Linux`_, you can acquire the additional dependencies via ``pacman``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pacman -S lapack gcc gcc-fortran pkg-config
|
||||
|
||||
.. Commenting it out until Catalyst fully supports Python 3.X
|
||||
..
|
||||
.. There are also AUR packages available for installing `Python 3.4
|
||||
.. <https://aur.archlinux.org/packages/python34/>`_ (Arch's default python is now
|
||||
.. 3.5, but Catalyst only currently supports 3.4), and `ta-lib
|
||||
.. <https://aur.archlinux.org/packages/ta-lib/>`_, an optional Catalyst dependency.
|
||||
.. Python 2 is also installable via:
|
||||
|
||||
..
|
||||
|
||||
.. $ pacman -S python2
|
||||
|
||||
Amazon Linux AMI Notes
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The packages ``pip`` and ``setuptools`` that come shipped by default are very
|
||||
outdated. Thus, you first need to run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --upgrade pip setuptools
|
||||
|
||||
The default installation is also missing the C and C++ compilers, which you
|
||||
install by:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo yum install gcc gcc-c++
|
||||
|
||||
Then you should follow the regular installation instructions outlined at the
|
||||
beginning of this page.
|
||||
|
||||
|
||||
.. _MacOS:
|
||||
|
||||
MacOS Requirements
|
||||
------------------
|
||||
|
||||
The version of Python shipped with MacOS by default is generally out of date,
|
||||
and has a number of quirks because it's used directly by the operating system.
|
||||
For these reasons, many developers choose to install and use a separate Python
|
||||
installation. The `Hitchhiker's Guide to Python`_ provides an excellent guide
|
||||
to `Installing Python on MacOS <http://docs.python-guide.org/en/latest/>`_,
|
||||
which explains how to install Python with the `Homebrew`_ manager.
|
||||
|
||||
Assuming you've installed Python with Homebrew, you'll also likely need the
|
||||
following brew packages:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ brew install freetype pkg-config gcc openssl
|
||||
|
||||
MacOS + virtualenv/conda + matplotlib
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The first time that you try to run an algorithm that loads the ``matplotlib``
|
||||
library, you may get the following error:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
RuntimeError: Python is not installed as a framework. The Mac OS X backend
|
||||
will not be able to function correctly if Python is not installed as a
|
||||
framework. See the Python documentation for more information on installing
|
||||
Python as a framework on Mac OS X. Please either reinstall Python as a
|
||||
framework, or try one of the other backends. If you are using (Ana)Conda
|
||||
please install python.app and replace the use of 'python' with 'pythonw'.
|
||||
See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more
|
||||
information.
|
||||
|
||||
This is a ``matplotlib``-specific error, that will go away once you run the
|
||||
following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ echo "backend: TkAgg" > ~/.matplotlib/matplotlibrc
|
||||
|
||||
in order to override the default ``MacOS`` backend for your system, which
|
||||
may not be accessible from inside the virtual or conda environment. This will
|
||||
allow Catalyst to open matplotlib charts from within a virtual environment,
|
||||
which is useful for displaying the performance of your backtests. To learn more
|
||||
about matplotlib backends, please refer to the
|
||||
`matplotlib backend documentation <https://matplotlib.org/faq/usage_faq.html#what-is-a-backend>`_.
|
||||
|
||||
.. _windows:
|
||||
|
||||
Windows Requirements
|
||||
--------------------
|
||||
|
||||
In Windows, you will first need to install the `Microsoft Visual C++ Compiler
|
||||
for Python 2.7
|
||||
<https://www.microsoft.com/en-us/download/details.aspx?id=44266>`_. This
|
||||
package contains the compiler and the set of system headers necessary for
|
||||
producing binary wheels for Python 2.7 packages. If it's not already in your
|
||||
system, download it and install it before proceeding to the next step.
|
||||
|
||||
Once you have the above compiler installed, the easiest and best supported way
|
||||
to install Catalyst in Windows is to use :ref:`Conda <conda>`. If you didn't
|
||||
any problems installing the compiler, jump to the :ref:`Conda <conda>` section,
|
||||
otherwise keep on reading to troubleshoot the C++ compiler installtion.
|
||||
|
||||
Some problems we have encountered installing the **Visual C++ Compiler**
|
||||
mentioned above are as follows:
|
||||
|
||||
- **The system administrator has set policies to prevent this installation**.
|
||||
|
||||
In some systems, there is a default *Windows Software Restriction* policy
|
||||
that prevents the installation of some software packages like this one.
|
||||
You'll have to change the Registry to circumvent this:
|
||||
|
||||
- Click ``Start``, and search for ``regedit`` and launch the
|
||||
``Registry Editor``
|
||||
- Navigate to the following folder:
|
||||
``HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Installer``
|
||||
- If the last folder does not exist, create it by right-clicking on the
|
||||
parent folder and choosing -> ``New`` -> ``Key`` and typing ``Installer``
|
||||
- If there is an entry for ``DisableMSI``, set the Value data to 0.
|
||||
- If there is no such entry, click on the ``Edit`` menu -> ``New`` ->
|
||||
``DWORD (32-bit) Value`` and enter ``DisableMSI`` as the Name (and by
|
||||
default you get 0 as the Value Data)
|
||||
|
||||
|
|
||||
|
||||
- **The installer has encountered an unexpected error installing this package.
|
||||
This may indicate a problem with this package. The error code is 2503.**
|
||||
|
||||
We have observed this when trying to install a package without enough
|
||||
administrator permissions. Even when you are logged in as an Administrator,
|
||||
you have to explictily install this package with administrator privileges:
|
||||
|
||||
- Click ``Start`` and find ``CMD`` or ``Command Prompt``
|
||||
- Right click on it and choose ``Run as administrator``
|
||||
- ``cd`` into the folder where you downloaded ``VCForPython27.msi``
|
||||
- Run ``msiexec /i VCForPython27.msi``
|
||||
|
||||
Updating Catalyst
|
||||
-----------------
|
||||
|
||||
Catalyst is currently in alpha and in under very active development. We release
|
||||
new minor versions every few days in response to the thorough battle testing
|
||||
that our user community puts Catalyst in. As a result, you should expect to
|
||||
update Catalyst frequently. Once installed, Catalyst can easily be updated as a
|
||||
``pip`` package regardless of the environemnt used for installation. Make sure
|
||||
you activate your environment first as you did in your first install, and then
|
||||
execute:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip uninstall enigma-catalyst
|
||||
$ pip install enigma-catalyst
|
||||
|
||||
Alternatively, you could update Catalyst issuing the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install -U enigma-catalyst
|
||||
|
||||
but this command will also upgrade all the Catalyst dependencies to the latest
|
||||
versions available, and may have unexpected side effects if a newer version of a
|
||||
dependency inadvertently breaks some functionality that Catalyst relies on.
|
||||
Thus, the first method is the recommended one.
|
||||
|
||||
Getting Help
|
||||
------------
|
||||
|
||||
If after following the instructions above, and going through the
|
||||
*Troubleshooting* sections, you still experience problems installing Catalyst,
|
||||
you can seek additional help through the following channels:
|
||||
|
||||
- Join our `Discord community <https://discord.gg/SJK32GY>`_, and head over
|
||||
the #catalyst_dev channel where many other users (as well as the project
|
||||
developers) hang out, and can assist you with your particular issue. The
|
||||
more descriptive and the more information you can provide, the easiest will
|
||||
be for others to help you out.
|
||||
|
||||
- Report the problem you are experiencing on our
|
||||
`GitHub repository <https://github.com/enigmampc/catalyst/issues>`_
|
||||
following the guidelines provided therein. Before you do so, take a moment
|
||||
to browse through all `previous reported issues
|
||||
<https://github.com/enigmampc/catalyst/issues?utf8=%E2%9C%93&q=is%3Aissue>`_
|
||||
in the likely case that someone else experienced that same issue before,
|
||||
and you get a hint on how to solve it.
|
||||
|
||||
|
||||
.. _`Debian-derived`: https://www.debian.org/misc/children-distros
|
||||
.. _`RHEL-derived`: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux_derivatives
|
||||
.. _`Arch Linux` : https://www.archlinux.org/
|
||||
.. _`Hitchhiker's Guide to Python` : http://docs.python-guide.org/en/latest/
|
||||
.. _`Homebrew` : http://brew.sh
|
||||
@@ -1,179 +0,0 @@
|
||||
Live Trading
|
||||
============
|
||||
This document explains how to get started with live trading.
|
||||
|
||||
Supported Exchanges
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Since version 0.4, Catalyst integrated with `CCXT <https://github.com/ccxt/ccxt>`_,
|
||||
a cryptocurrency trading library with support for more than 90 exchanges. The
|
||||
range of CCXT and Catalyst support for each of those exchanges varies greatly.
|
||||
The most supported exchanges are as follows:
|
||||
|
||||
The exchanges available for backtesting are fully supported in live mode:
|
||||
|
||||
- Bitfinex, id = ``bitfinex``
|
||||
- Bittrex, id = ``bittrex``
|
||||
- Poloniex, id = ``poloniex``
|
||||
|
||||
Additionally, we have successfully tested the following exchanges:
|
||||
|
||||
- Binance, id = ``binance``
|
||||
- Bitmex, id = ``bitmex``
|
||||
- GDAX, id = ``gdax``
|
||||
|
||||
As Catalyst is currently in Alpha and in under active development, you are
|
||||
encouraged to throughly test any exchange in *paper trading* mode before trading
|
||||
*live* with it.
|
||||
|
||||
Paper Trading vs Live Trading modes
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Catalyst currently supports three different modes in which you can execute your
|
||||
trading algorithm. The first is backtesting, which is covered extensively in the
|
||||
tutorial, and uses historical data to run your algorithm. There is no
|
||||
interaction with the exchange in backtesting mode, and this is the first mode
|
||||
that you should test any new algorithm.
|
||||
|
||||
Once you are confident with the simulations that you have obtained with your
|
||||
algorithm in backtesting, you may switch to live trading, where you have two
|
||||
different modes:
|
||||
* *Paper Trading*: The simulated algorithm runs in real time, and fetches
|
||||
pricing data in real time from the exchange, but the orders never reach the
|
||||
exchange, and are instead kept within Catalyst and simulated. No real currency
|
||||
is bought or sold. Think of it as a `backtesting happening in real time`.
|
||||
* *Live Trading*: This is the proper live trading mode in which an algorithm
|
||||
runs in real time, fetching pricing data from live exchanges and placing orders
|
||||
against the exchange. Real currency is transacted on the exchange driven by the
|
||||
algorithm.
|
||||
|
||||
These three modes are controlled by the following variables:
|
||||
|
||||
+---------------+-------------------------+
|
||||
| Mode | Parameters |
|
||||
+ +-------+-----------------+
|
||||
| | live | simulate_orders |
|
||||
+---------------+-------+-----------------+
|
||||
| backtesting | False | True (default) |
|
||||
+---------------+-------+-----------------+
|
||||
| paper trading | True | True |
|
||||
+---------------+-------+-----------------+
|
||||
| live trading | True | False |
|
||||
+---------------+-------+-----------------+
|
||||
|
||||
|
||||
Authentication
|
||||
^^^^^^^^^^^^^^
|
||||
Most exchanges require token key/secret combination for authentication. By
|
||||
convention, Catalyst uses an ``auth.json`` file to hold this data.
|
||||
|
||||
This example illustrates the convention using the *Bitfinex* exchange.
|
||||
Here is how to generate key and secret values for the Bitfinex exchange:
|
||||
https://docs.bitfinex.com/v1/docs/api-access. Most exchanges follow
|
||||
a similar process.
|
||||
|
||||
The auth.json file:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
{
|
||||
"name": "bitfinex",
|
||||
"key": "my-key",
|
||||
"secret": "my-secret"
|
||||
}
|
||||
|
||||
|
||||
The file goes here: ``~/.catalyst/data/exchanges/bitfinex/auth.json``
|
||||
|
||||
Note that the `bitfinex` part in the directory above corresponds to the id of the Bitfinex
|
||||
exchange as defined in the "Supported Exchanges" section above.
|
||||
Attempting to run an algorithm where the targeted exchange is missing
|
||||
its ``auth.json`` file will create the directory structure and create an empty
|
||||
auth.json file, but will result in an error.
|
||||
|
||||
Currency Symbols
|
||||
^^^^^^^^^^^^^^^^
|
||||
Catalyst introduces a universal convention to reference
|
||||
trading pairs and individual currencies. This
|
||||
is required to ensure that the ``symbol()`` api predictably
|
||||
returns the correct asset regardless of the targeted exchange.
|
||||
|
||||
Exchanges tend to use their own convention to represent currencies
|
||||
(e.g. XBT and BTC both represent Bitcoin on different exchanges).
|
||||
Trading pairs are also inconsistent. For example, Bitfinex
|
||||
puts the market currency before the base currency without a
|
||||
separator, Bittrex puts the base currency first and uses a dash
|
||||
seperator.
|
||||
|
||||
Here is the Catalyst convention:
|
||||
|
||||
*[Market Currency]_[Base Currency]* all lowercase.
|
||||
|
||||
Currency symbols (e.g. btc, eth, ltc) follow the Bittrex convention.
|
||||
|
||||
Here are some examples:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# With Bitfinex
|
||||
bitcoin_usd_asset = symbol('btc_usd')
|
||||
ethereum_bitcoin_asset = symbol('eth_btc')
|
||||
|
||||
# With Bittrex
|
||||
ethereum_bitcoin_asset = symbol('eth_btc')
|
||||
neo_ethereum_asset = symbol('neo_eth)
|
||||
|
||||
Note that the trading pairs are always referenced in the same manner.
|
||||
However, not all trading pairs are available on all exchanges. An
|
||||
error will occur if the specified trading pair is not trading
|
||||
on the exchange. To check which currency pairs are available on each
|
||||
of the supported exchanges, see
|
||||
`Catalyst Market Coverage <https://www.enigma.co/catalyst/status>`_.
|
||||
|
||||
Trading an Algorithm
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
There is no special convention to follow when writing an
|
||||
algorithm for live trading. The same algorithm should work in
|
||||
backtest and live execution mode without modification.
|
||||
|
||||
What differs are the arguments provided to the catalyst client or
|
||||
`run_algorithm()` interface. Here is the same example in both interfaces:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
catalyst live -f my_algo_code -x bitfinex -c btc -n my_algo_name
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
run_algorithm(
|
||||
initialize=initialize,
|
||||
handle_data=handle_data,
|
||||
analyze=analyze,
|
||||
exchange_name='bitfinex',
|
||||
live=True,
|
||||
algo_namespace='my_algo_name',
|
||||
base_currency='btc'
|
||||
)
|
||||
|
||||
|
||||
Here is the breakdown of the new arguments:
|
||||
|
||||
- ``live``: Boolean flag which enables live trading. It defaults to ``False``.
|
||||
- ``capital_base``: The amount of base_currency assigned to the strategy.
|
||||
It has to be lower or equal to the amount of base currency available for
|
||||
trading on the exchange. For illustration, order_target_percent(asset, 1)
|
||||
will order the capital_base amount specified here of the specified asset.
|
||||
- ``exchange_name``: The name of the targeted exchange. See the
|
||||
`CCXT Supported Exchanges <https://github.com/ccxt/ccxt/wiki/Exchange-Markets>`_
|
||||
for the full list.
|
||||
- ``algo_namespace``: A arbitrary label assigned to your algorithm for
|
||||
data storage purposes.
|
||||
- ``base_currency``: The base currency used to calculate the
|
||||
statistics of your algorithm. Currently, the base currency of all
|
||||
trading pairs of your algorithm must match this value.
|
||||
- ``simulate_orders``: Enables the paper trading mode, in which orders are
|
||||
simulated in Catalyst instead of processed on the exchange. It defaults to
|
||||
``True``.
|
||||
|
||||
Here is a complete algorithm for reference:
|
||||
`Buy Low and Sell High <https://github.com/enigmampc/catalyst/blob/master/catalyst/examples/buy_low_sell_high_live.py>`_
|
||||
@@ -1,263 +0,0 @@
|
||||
Release Process
|
||||
---------------
|
||||
|
||||
.. include:: dev-doc-message.txt
|
||||
|
||||
|
||||
Updating the Release Notes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
When we are ready to ship a new release of zipline, edit the :doc:`releases`
|
||||
page. We will have been maintaining a whatsnew file while working on the release
|
||||
with the new version. First, find that file in:
|
||||
``docs/source/whatsnew/<version>.txt``. It will be the highest version number.
|
||||
Edit the release date field to be today's date in the format:
|
||||
|
||||
::
|
||||
|
||||
<month> <day>, <year>
|
||||
|
||||
|
||||
for example, November 6, 2015.
|
||||
Remove the active development warning from the whatsnew, since it will no
|
||||
longer be pending release.
|
||||
Update the title of the release from "Development" to "Release x.x.x" and
|
||||
update the underline of the title to match the title's width.
|
||||
|
||||
If you are renaming the release at this point, you'll need to git mv the file
|
||||
and also update releases.rst to reference the renamed file.
|
||||
|
||||
To build and view the docs locally, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ cd docs
|
||||
$ make html
|
||||
$ {BROWSER} build/html/index.html
|
||||
|
||||
Updating the Python stub files
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
PyCharm and other linters and type checkers can use `Python stub files
|
||||
<https://www.python.org/dev/peps/pep-0484/#stub-files>`__ for type hinting. For
|
||||
example, we generate stub files for the :mod:`~zipline.api` namespace, since that
|
||||
namespace is populated at import time by decorators on TradingAlgorithm
|
||||
methods. Those functions are therefore hidden from static analysis tools, but
|
||||
we can generate static files to make them available. Under **Python 3**, run
|
||||
the following to generate any stub files:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python etc/gen_type_stubs.py
|
||||
|
||||
.. note::
|
||||
|
||||
In order to make stub consumers aware of the classes referred to in the
|
||||
stub, the stub file should import those classes. However, since
|
||||
``... import *`` and ``... import ... as ...`` in a stub file will export
|
||||
those imports, we import the names explicitly. For the stub for
|
||||
``zipline.api``, this is done in a header string in the
|
||||
``gen_type_stubs.py`` script mentioned above. If new classes are added as
|
||||
parameters or return types of ``zipline.api`` functions, then new imports
|
||||
should be added to that header.
|
||||
|
||||
Updating the ``__version__``
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
We use `versioneer <https://github.com/warner/python-versioneer>`__ to
|
||||
manage the ``__version__`` and ``setup.py`` version. This means that we pull
|
||||
this information from our version control's tags to ensure that they stay in
|
||||
sync and to have very fine grained version strings for development installs.
|
||||
|
||||
To upgrade the version use the git tag command like:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git tag <major>.<minor>.<micro>
|
||||
$ git push && git push --tags
|
||||
|
||||
|
||||
This will push the the code and the tag information.
|
||||
|
||||
Next, click the "Draft a new release" button on the `zipline releases page
|
||||
<https://github.com/quantopian/zipline/releases>`__. For the new release,
|
||||
choose the tag you just pushed, and publish the release.
|
||||
|
||||
Uploading PyPI packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
``sdist``
|
||||
^^^^^^^^^
|
||||
|
||||
To build the ``sdist`` (source distribution) run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python setup.py sdist
|
||||
|
||||
|
||||
from the zipline root. This will create a gzipped tarball that includes all the
|
||||
python, cython, and miscellaneous files needed to install zipline. To test that
|
||||
the source dist worked correctly, ``cd`` into an empty directory, create a new
|
||||
virtualenv and then run:
|
||||
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install <zipline-root>/dist/zipline-<major>.<minor>.<micro>.tar.gz
|
||||
$ python -c 'import zipline;print(zipline.__version__)'
|
||||
|
||||
This should print the version we are expecting to release.
|
||||
|
||||
.. note::
|
||||
|
||||
It is very important to both ``cd`` into a clean directory and make a clean
|
||||
virtualenv. Changing directories ensures that we have included all the needed
|
||||
files in the manifest. Using a clean virtualenv ensures that we have listed
|
||||
all the required packages.
|
||||
|
||||
Now that we have tested the package locally, it should be tested using the test
|
||||
PyPI server.
|
||||
|
||||
Edit your ``~/.pypirc`` file to look like:
|
||||
|
||||
::
|
||||
|
||||
[distutils]
|
||||
index-servers =
|
||||
pypi
|
||||
pypitest
|
||||
|
||||
[pypi]
|
||||
username:
|
||||
password:
|
||||
|
||||
[pypitest]
|
||||
repository: https://testpypi.python.org/pypi
|
||||
username:
|
||||
password:
|
||||
|
||||
after that, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python setup.py sdist upload -r pypitest
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
If the package version has been taken: locally update your setup.py to
|
||||
override the version with a new number. Do not use the next version, just
|
||||
append a ``.<nano>`` section to the current version. PyPI prevents the same
|
||||
package version from appearing twice, so we need to work around this when
|
||||
debugging packaging problems on the test server.
|
||||
|
||||
.. warning::
|
||||
|
||||
Do not commit the temporary version change.
|
||||
|
||||
|
||||
This will upload zipline to the pypi test server. To test installing from pypi,
|
||||
create a new virtualenv, ``cd`` into a clean directory and then run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --extra-index-url https://testpypi.python.org/pypi zipline
|
||||
$ python -c 'import zipline;print(zipline.__version__)'
|
||||
|
||||
|
||||
This should pull the package you just uploaded and then print the version
|
||||
number.
|
||||
|
||||
Now that we have tested locally and on PyPI test, it is time to upload to PyPI:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python setup.py sdist upload
|
||||
|
||||
``bdist``
|
||||
^^^^^^^^^
|
||||
|
||||
Because zipline now supports multiple versions of numpy, we're not building
|
||||
binary wheels, since they are not tagged with the version of numpy with which
|
||||
they were compiled.
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
To update `zipline.io <http://www.zipline.io/index.html>`__, checkout the
|
||||
latest master and run:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
python <zipline_root>/docs/deploy.py
|
||||
|
||||
This will build the documentation, checkout a fresh copy of the ``gh-pages``
|
||||
git branch, and copy the built docs into the zipline root.
|
||||
|
||||
.. note::
|
||||
|
||||
The docs should always be built with **Python 3**. Many of our api functions
|
||||
are wrapped by preprocessing functions which accept \*args and \**kwargs. In
|
||||
Python 3, sphinx will respect the ``__wrapped__`` attribute and display the
|
||||
correct arguments.
|
||||
|
||||
Now, using our browser of choice, view the ``index.html`` page and verify that
|
||||
the docs look correct.
|
||||
|
||||
Once we are happy, push the updated docs to the GitHub ``gh-pages`` branch.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ git add .
|
||||
$ git commit -m "DOC: update zipline.io"
|
||||
$ git push origin gh-pages
|
||||
|
||||
`zipline.io <http://www.zipline.io/index.html>`__ will update in a few moments.
|
||||
|
||||
Uploading conda packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Travis and AppVeyor build zipline conda packages for us. Once they have built
|
||||
and uploaded to anaconda.org the packages (and their dependencies) for the
|
||||
release commit to master, we should move those packages from the "ci" label to
|
||||
the "main" label. You can do this from the anaconda.org web interface. This
|
||||
is also a good time to remove all the old "ci" packages from anaconda.
|
||||
|
||||
Travis and AppVeyor only build and upload linux-64 and win-64 packages. We'll
|
||||
need to build and upload osx-64 packages manually on an OSX machine.
|
||||
|
||||
To build the conda packages for zipline locally, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python etc/conda_build_matrix.py
|
||||
|
||||
If all of the builds succeed, then this will not print anything and exit with
|
||||
``EXIT_SUCCESS``. If there are build issues, we must address them and decide
|
||||
what to do.
|
||||
|
||||
Once all of the builds in the matrix pass, we can upload them to anaconda with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python etc/conda_build_matrix.py --upload
|
||||
|
||||
If you would like to test this command by uploading to a different user, this
|
||||
may be specified with the ``--user`` flag.
|
||||
|
||||
Next Commit
|
||||
~~~~~~~~~~~
|
||||
|
||||
Push a new commit post-release that adds the whatsnew for the next release,
|
||||
which should be titled according to a micro version increment. If that next
|
||||
release turns out to be a major/minor version increment, the file can be
|
||||
renamed when that's decided. You can use ``docs/source/whatsnew/skeleton.txt``
|
||||
as a template for the new file.
|
||||
|
||||
Include the whatsnew file in ``docs/source/releases.rst``. New releases should
|
||||
appear at the top. The syntax for this is:
|
||||
|
||||
::
|
||||
|
||||
.. include:: whatsnew/<version>.txt
|
||||
@@ -1,390 +0,0 @@
|
||||
=============
|
||||
Release Notes
|
||||
=============
|
||||
|
||||
Version 0.5.2
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-02-08
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixed an issue with live candle values :issue:`216` and :issue:`199`
|
||||
|
||||
Version 0.5.1
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-02-07
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixed an issue with orders that stay open :issue:`211`
|
||||
- Fixed Jupyter issues :issue:`179`
|
||||
- Fetching multiple tickers in one call to minimize rate limit risks :issue:`174`
|
||||
- Improved live state presentation :issue:`171`
|
||||
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Introducing the Enigma Marketplace
|
||||
|
||||
Version 0.4.7
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-19
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixing issue :issue:`137` impacting the CLI
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Implemented authentication aliases (:issue:`60`)
|
||||
|
||||
Version 0.4.6
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-18
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixed some Python3 issues
|
||||
- Reading the trade log to get executed order prices on exchanges like Binance (:issue:`151`)
|
||||
- Fixed issue with market order executing price (:issue:`150` and :issue:`111`)
|
||||
- Implemented standardized symbol mapping (:issue:`157`)
|
||||
- Improved error handling for unsupported timeframes (:issue:`159`)
|
||||
- Using Bitfinex instead of Poloniex to fetch btc_usdt benchmark (:issue:`161`)
|
||||
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Added a `context.state` dict to keep arbitrary state values between runs
|
||||
- Added ability to stop live algo at specified end date
|
||||
|
||||
Version 0.4.5
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-12
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Improved order execution for exchanges supporting trade lists (:issue:`151`)
|
||||
- Fixed an issue where requesting history of multiple assets repeats values
|
||||
- Raising an error for order amounts smaller than exchange lots
|
||||
- Handling multiple req errors with tickers more gracefully (:issue:`160`)
|
||||
|
||||
Version 0.4.4
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-09
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Removed redundant capital_base validation (:issue:`142`)
|
||||
- Fixed portfolio update issue with restored state (:issue:`111`)
|
||||
- Skipping cash validation where there are open orders (:issue:`144`)
|
||||
|
||||
Version 0.4.3
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-05
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixed CLI issue (:issue:`137`)
|
||||
- Upgraded CCXT
|
||||
|
||||
Version 0.4.2
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2018-01-03
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
- Fixed cash synchronization issue (:issue:`133`)
|
||||
- Fixed positions synchronization issue (:issue:`132`)
|
||||
- Patched empyrical to resolve a np.log1p issue (:issue:`126`)
|
||||
- Fixed a paper trading issue (:issue:`124`)
|
||||
- Fixed a commission issue (:issue:`104`)
|
||||
- Fixed a poloniex specific issue in live trading (:issue:`103`)
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Caching CCXT market info to limit round-trips (:issue:`99`)
|
||||
- Tentative support for Pipeline (:issue:`96`)
|
||||
|
||||
Version 0.4.0
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-12-12
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Changed Poloniex interface (should solve :issue:`95` and :issue:`94`)
|
||||
- Solved issue with overriding commission and slippage (:issue:`87`)
|
||||
- Fixed inefficiency with Bittrex current prices (:issue:`76`)
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Integrated with CCXT
|
||||
- Added paper trading capability (`simulate_orders=True` param in live mode)
|
||||
- More granular commissions (:issue:`82`)
|
||||
- Added market orders in live mode (:issue:`81`)
|
||||
|
||||
Version 0.3.10
|
||||
~~~~~~~~~~~~~~
|
||||
**Release Date**: 2017-11-28
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed issue with fetching assets with daily frequency
|
||||
|
||||
Version 0.3.9
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-28
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed sortino warning issues (:issue:`77`)
|
||||
- Adjusted computation of last candle of data.history (:issue:`71`)
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
- Added capital_base parameter to live mode to limit cash (:issue:`79`)
|
||||
- Added support for csv ingestion (:issue:`65`)
|
||||
- Improved cash display in running stats (:issue:`80`)
|
||||
|
||||
|
||||
Version 0.3.8
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-14
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed a warning filter issue introduced with the latest release
|
||||
|
||||
Version 0.3.7
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-14
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed an SSL cert issue (:issue:`64`)
|
||||
- Fixed cumulative stats warnings (:issue:`63`)
|
||||
- Disabled auto-ingestion because of unresolved caching issues (:issue:`47`)
|
||||
- Standardized live-trading stats (:issue:`61`)
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
|
||||
- Added a mean-reversion sample algo
|
||||
- Added minutely stats in the analyze() function (:issue:`62`)
|
||||
- Added specificity to some error messages
|
||||
|
||||
Version 0.3.6
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-4
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed an issue with single bar data.history() (:issue:`55`)
|
||||
|
||||
Version 0.3.5
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-4
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Added workaround for: KeyError: Timestamp error (:issue:`53`)
|
||||
|
||||
Version 0.3.4
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-11-2
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed issue with auto-ingestion of minute data (:issue:`47`)
|
||||
- Fixed issue with sell orders in backtesting
|
||||
- Fixed data frequency issues with data.history() in backtesting
|
||||
- Fixed an issue with can_trade()
|
||||
- Reduced the commission and slippage values to account for lower volume
|
||||
transactions
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
|
||||
- Added more unit tests
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
- Improved installation notes for Windows C++ compiler and Conda
|
||||
- Addition of
|
||||
`Jupyter Notebook guide <https://enigmampc.github.io/catalyst/jupyter.html>`_
|
||||
- Addition of
|
||||
`Live Trading page <https://enigmampc.github.io/catalyst/live-trading.html>`_
|
||||
- Addition of
|
||||
`Videos page <https://enigmampc.github.io/catalyst/videos.html>`_
|
||||
- Addition of
|
||||
`Resources page <https://enigmampc.github.io/catalyst/resources.html>`_
|
||||
- Addition of `Development Guidelines
|
||||
<https://enigmampc.github.io/catalyst/development-guidelines.html>`_
|
||||
- Addition of
|
||||
`Release Notes <https://enigmampc.github.io/catalyst/releases.html>`_
|
||||
- Updated code docstrings
|
||||
|
||||
|
||||
Version 0.3.3
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-10-26
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fix missing -x in ingest-exchange
|
||||
- Fix issue with daily chunks end date (data bundles)
|
||||
- Fix issue in the prepare_chunk logic (data bundles)
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
|
||||
- Added data validation unit tests
|
||||
|
||||
|
||||
Version 0.3.2
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-10-25
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fix to work with empty data bundles
|
||||
- Fix Windows path of ``$HOME/.catalyst`` folder
|
||||
- Fix ``etc/python2.7-environment.yml`` for Windows Conda install
|
||||
- Fix hash method to create sid numbers compatible across platforms
|
||||
- Fix an issue with asset date in chunks
|
||||
|
||||
Build
|
||||
~~~~~
|
||||
|
||||
- Python3 adjustments
|
||||
- Added method to clean bundle folders, and remove symbols.json
|
||||
- Implemented and improved unit tests
|
||||
|
||||
|
||||
Version 0.3.1
|
||||
^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-10-22
|
||||
|
||||
Bug Fixes
|
||||
~~~~~~~~~
|
||||
|
||||
- Fixed OS-dependent path issue in data bundle
|
||||
- Changed handling of empty ``auth.json``, instead of throwing an error for
|
||||
missing file
|
||||
- Updated ``etc/python2.7-environment.yml`` to work with Catalyst version 0.3
|
||||
- Updated ``catalyst/examples/buy_and_hodl.py`` and
|
||||
``catalyst/examples/buy_low_sell_high.py`` to work with Catalyst version 0.3
|
||||
|
||||
|
||||
Version 0.3
|
||||
^^^^^^^^^^^
|
||||
**Release Date**: 2017-10-20
|
||||
|
||||
- Standardized live and backtesting syntax
|
||||
- Added a repository for historical data
|
||||
- Added supported for multiple exchanges per algorithm
|
||||
- Added a standardized dictionary of symbols for each exchange
|
||||
- Added auto-ingestion of bundle data while backtesting
|
||||
- Bug fixes
|
||||
|
||||
|
||||
Version 0.2.dev5
|
||||
^^^^^^^^^^^^^^^^
|
||||
**Release Date**: 2017-10-03
|
||||
|
||||
- Fixes bug in data.history function that was formatting 'volume' data as
|
||||
integers, now they are returned as floats with up to 9 decimals of precision.
|
||||
Data bundles redone.
|
||||
|
||||
Version 0.2.dev4
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-09-20
|
||||
|
||||
- Fixes bug in the pricing resolution of 1-minute data, now set to 8 decimal
|
||||
places. Pricing resolution of daily data remains set to 9 decimal places.
|
||||
- The current data bundle takes 340MB compressed for download, and 460MB
|
||||
uncompressed on disk for Catalyst to use.
|
||||
|
||||
Version 0.2.dev3
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-09-20
|
||||
|
||||
- 1-minute resolution OHLCV data bundle for backtesting from Poloniex exchange
|
||||
- Implementation of trading of fractional crypto assets (i.e. 0.01 BTC)
|
||||
- Minimum trade size of a coin can be configured on a per-coin basis, defaults
|
||||
to 0.00000001 in backtesting (most exchanges set the minimum trade to larger
|
||||
amounts, which will impact live trading)
|
||||
- Increased pricing resolution from 3 to 9 decimal places
|
||||
- The current data bundle takes 40MB compressed for download, and 99MB
|
||||
uncompressed on disk for Catalyst to use.
|
||||
|
||||
Version 0.2.dev2
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-09-07
|
||||
|
||||
- Fix path issue
|
||||
|
||||
Version 0.2.dev1
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-09-03
|
||||
|
||||
- Implementation of live trading:
|
||||
|
||||
- Comprehensive trading functionality against exchanges Bitfinex and Bittrex.
|
||||
- Support for all trading pairs available on each exchange.
|
||||
- Multiple algorithms can trade simultaneously against a single exchange
|
||||
using the same account.
|
||||
- Each algorithm has a persisted state (i.e. algorithm can be stopped and
|
||||
restarted preserving the state without data loss) that tracks all open
|
||||
orders, executed transactions and portfolio positions.
|
||||
|
||||
- Minute by minute portfolio performance metrics.
|
||||
|
||||
- Daily summary performance statistics compatible with pyfolio, a Python
|
||||
library for performance and risk analysis of financial portfolios
|
||||
|
||||
Version 0.1.dev9
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-08-28
|
||||
|
||||
- Retrieval of crypto benchmark from bundle, instead of hitting Poloniex
|
||||
exchange directly
|
||||
- Change of bundle storage provider from Dropbox to AWS
|
||||
- Fix issue with 1/1000 scaling issue of prices in bundle
|
||||
|
||||
Version 0.1.dev8
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-08-18
|
||||
|
||||
- Fixes issue in the creation of bundles (:issue:`27`)
|
||||
|
||||
|
||||
Version 0.1.dev7
|
||||
^^^^^^^^^^^^^^^^
|
||||
- Fixes issues in empty benchmark (:issue:`16`)
|
||||
- Fixes issue of normalizing timestamps before comparison (:issue:`24`)
|
||||
- Generic data bundles
|
||||
- CLI UI improvements
|
||||
|
||||
Version 0.1.dev6
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
**Release Date**: 2017-07-13
|
||||
|
||||
- Initial public release
|
||||
@@ -1,26 +0,0 @@
|
||||
Resources
|
||||
=========
|
||||
|
||||
- `Catalyst Whitepaper <https://www.enigma.co/enigma_catalyst.pdf>`_
|
||||
|
||||
|
||||
Related 3rd Party APIs
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
- `Zipline <http://www.zipline.io/appendix.html>`_ is a Pythonic Algorithmic
|
||||
Trading Library, and the project Catalyst forked off in the spring of 2017.
|
||||
- `Quantopian <https://www.quantopian.com/help>`_ provides a platform for
|
||||
freelance quantitative analysts develop, test, and use trading algorithms to
|
||||
buy and sell securities. They aim to create a crowd-sourced hedge fund by
|
||||
fostering their community of freelance traders. Quantopian's backtesting and
|
||||
live-trading engine is powered by *Zipline*.
|
||||
- `Pandas <https://pandas.pydata.org/pandas-docs/stable/api.html>`_ is a Python
|
||||
library providing high-performance, easy-to-use data structures and data
|
||||
analysis tools. Catalyst relies heavily on pandas, and many API functions
|
||||
return data as Pandas dataframes.
|
||||
- `Numpy <https://docs.scipy.org/doc/numpy/reference/>`_ is the fundamental
|
||||
package for scientific computing with Python. Some of the data computation
|
||||
that your algorithms will need, will be optimized leveraging Numpy.
|
||||
- `Matplotlib <https://matplotlib.org/1.5.3/api/index.html>`_ is a Python 2D
|
||||
plotting library that many of examples rely on to plot the performance of
|
||||
trading algorithms
|
||||
@@ -1,88 +0,0 @@
|
||||
==========
|
||||
Unit Tests
|
||||
==========
|
||||
|
||||
Exchanges
|
||||
~~~~~~~~~
|
||||
|
||||
Markets
|
||||
-------
|
||||
Sample:
|
||||
All markets in 3 random exchanges
|
||||
Test:
|
||||
Fetch all TradingPair instances
|
||||
Assert:
|
||||
No error
|
||||
|
||||
Current Ticker
|
||||
------------------
|
||||
Sample:
|
||||
3 random markets in each of the 3 random exchanges
|
||||
Test:
|
||||
Fetch current price and volume
|
||||
Assert:
|
||||
Not null and no error
|
||||
|
||||
Historical Price Data
|
||||
---------------------
|
||||
Sample:
|
||||
- 3 random markets for each of the 3 random exchanges supporting historical data
|
||||
- For each market, randomly select one supported frequency
|
||||
Test:
|
||||
Fetch historical data for each market using the selected frequency
|
||||
Assert:
|
||||
- No error and not blank
|
||||
- Date of each candle is consistent with the Catalyst desired pattern,
|
||||
- All candle start at fix intervals
|
||||
- Last candle partial and forward looking from the end date
|
||||
|
||||
Authentication and Orders
|
||||
-------------------------
|
||||
Sample:
|
||||
1 random market for each of 3 random authenticated exchanges
|
||||
Test:
|
||||
- Create one limit order randomly buying or selling at least 10% out from the current price
|
||||
- Retrieve the open order from the exchange
|
||||
- Cancel the open order
|
||||
Assert:
|
||||
No error
|
||||
|
||||
|
||||
Bundles
|
||||
~~~~~~~
|
||||
|
||||
Validate Bundle Data
|
||||
--------------------
|
||||
Sample:
|
||||
- 3 random market in bundles for exchanges supporting historical data
|
||||
- For each market, randomly selected data range available in the exchange historical data
|
||||
Test:
|
||||
- Clean the target exchange bundle
|
||||
- Ingest the selected market data for the selected data range
|
||||
- Retrieve the bundle data into a dataframe
|
||||
- Retrieve the equivalent OHLCV data from the exchange into a dataframe
|
||||
Assert:
|
||||
Matching data for the bundle and exchange
|
||||
|
||||
|
||||
Algo Stats
|
||||
----------
|
||||
Sample:
|
||||
- 2 sample algorithms with built-in stats calculator
|
||||
- 2 KPIs both calculated by each algo and by Catalyst
|
||||
Test:
|
||||
- Run each algorithm
|
||||
- Compare the results of the two methods or calculating stats
|
||||
Assert:
|
||||
- Matching stats
|
||||
|
||||
CSV Ingestion
|
||||
-------------
|
||||
Sample:
|
||||
3 random CSV files containing price data
|
||||
Test:
|
||||
- Ingest each CSV files
|
||||
- Validate with the exchange like in the 'Validate Bundle Data' test
|
||||
Assert:
|
||||
Matching data between the bundle and the exchange
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
Utilities
|
||||
=========
|
||||
|
||||
This section covers a variety of utilites that provide complimentary
|
||||
functionality to your trading algorithms. These are code snippets that you can
|
||||
add to any algorithm to add the desired functionality.
|
||||
|
||||
If you are looking for example trading algorithms, see the corresponding section.
|
||||
|
||||
Output to CSV file
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Add this script to the analyze method to create and save a CSV file with the
|
||||
results from the trading algorithm. This file will include the default
|
||||
parameters of the results DataFrame plus any recorded variables and will be
|
||||
saved in the same location where your trading algorithm is saved. The exact
|
||||
script that you need to use depends on the interface that you are using to run
|
||||
your trading algorithm, which could be the CLI or a Python Interpreter.
|
||||
|
||||
1. Script to use with CLI:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def analyze(context=None, results=None):
|
||||
import sys
|
||||
import os
|
||||
from os.path import basename
|
||||
|
||||
# Save results in CSV file
|
||||
filename = os.path.splitext(basename(sys.argv[3]))[0]
|
||||
results.to_csv(filename + '.csv')
|
||||
|
||||
2. Script to use with Python Interpreter:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def analyze(context=None, results=None):
|
||||
import os
|
||||
from os.path import basename
|
||||
|
||||
# Save results in CSV file
|
||||
filename = os.path.splitext(os.path.basename(__file__))[0]
|
||||
results.to_csv(filename + '.csv')
|
||||
|
||||
Extracting market data
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Use this script to save the price and volume data of one cryptoasset in a CSV
|
||||
file, which will be saved in the same location and with the same name as your
|
||||
Python file. To get custom data, simply modify the asset's symbol and the dates.
|
||||
Run this script directly from your development environment: python scriptname.py,
|
||||
where the contents of 'scriptname.py' are as follows. Two different version are
|
||||
provided as an example for daily- and minute-resolution data respectively:
|
||||
|
||||
Simpler case for daily data
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import os
|
||||
import pytz
|
||||
from datetime import datetime
|
||||
|
||||
from catalyst.api import record, symbol, symbols
|
||||
from catalyst.utils.run_algo import run_algorithm
|
||||
|
||||
def initialize(context):
|
||||
# Portfolio assets list
|
||||
context.asset = symbol('btc_usdt') # Bitcoin on Poloniex
|
||||
|
||||
def handle_data(context, data):
|
||||
# Variables to record for a given asset: price and volume
|
||||
price = data.current(context.asset, 'price')
|
||||
volume = data.current(context.asset, 'volume')
|
||||
record(price=price, volume=volume)
|
||||
|
||||
def analyze(context=None, results=None):
|
||||
|
||||
# Generate DataFrame with Price and Volume only
|
||||
data = results[['price','volume']]
|
||||
|
||||
# Save results in CSV file
|
||||
filename = os.path.splitext(os.path.basename(__file__))[0]
|
||||
data.to_csv(filename + '.csv')
|
||||
|
||||
''' Bitcoin data is available on Poloniex since 2015-3-1.
|
||||
Dates vary for other tokens. In the example below, we choose the
|
||||
full month of July of 2017.
|
||||
'''
|
||||
start = datetime(2017, 1, 1, 0, 0, 0, 0, pytz.utc)
|
||||
end = datetime(2017, 7, 31, 0, 0, 0, 0, pytz.utc)
|
||||
results = run_algorithm(initialize=initialize,
|
||||
handle_data=handle_data,
|
||||
analyze=analyze,
|
||||
start=start,
|
||||
end=end,
|
||||
exchange_name='poloniex',
|
||||
capital_base=10000,
|
||||
base_currency = 'usdt')
|
||||
|
||||
More versatile case for minute data
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import os
|
||||
import csv
|
||||
import pytz
|
||||
from datetime import datetime
|
||||
|
||||
from catalyst.api import record, symbol, symbols
|
||||
from catalyst.utils.run_algo import run_algorithm
|
||||
|
||||
|
||||
def initialize(context):
|
||||
# Portfolio assets list
|
||||
context.asset = symbol('btc_usdt') # Bitcoin on Poloniex
|
||||
|
||||
# Creates a .CSV file with the same name as this script to store results
|
||||
context.csvfile = open(os.path.splitext(
|
||||
os.path.basename(__file__))[0]+'.csv', 'w+')
|
||||
context.csvwriter = csv.writer(context.csvfile)
|
||||
|
||||
def handle_data(context, data):
|
||||
# Variables to record for a given asset: price and volume
|
||||
# Other options include 'open', 'high', 'open', 'close'
|
||||
# Please note that 'price' equals 'close'
|
||||
date = context.blotter.current_dt # current time in each iteration
|
||||
price = data.current(context.asset, 'price')
|
||||
volume = data.current(context.asset, 'volume')
|
||||
|
||||
# Writes one line to CSV on each iteration with the chosen variables
|
||||
context.csvwriter.writerow([date,price,volume])
|
||||
|
||||
def analyze(context=None, results=None):
|
||||
# Close open file properly at the end
|
||||
context.csvfile.close()
|
||||
|
||||
|
||||
# Bitcoin data is available from 2015-3-2. Dates vary for other tokens.
|
||||
start = datetime(2017, 7, 30, 0, 0, 0, 0, pytz.utc)
|
||||
end = datetime(2017, 7, 31, 0, 0, 0, 0, pytz.utc)
|
||||
results = run_algorithm(initialize=initialize,
|
||||
handle_data=handle_data,
|
||||
analyze=analyze,
|
||||
start=start,
|
||||
end=end,
|
||||
exchange_name='poloniex',
|
||||
data_frequency='minute',
|
||||
base_currency ='usdt',
|
||||
capital_base=10000 )
|
||||
@@ -1,63 +0,0 @@
|
||||
Videos
|
||||
======
|
||||
|
||||
|
||||
Installation: MacOS
|
||||
-------------------
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/ZnsslmHljvw" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
|
|
||||
|
|
||||
|
||||
Installation: Windows
|
||||
---------------------
|
||||
|
||||
Where things go smoothly:
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/H8HqcEbZmkk" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
|
|
||||
|
||||
Where things don't:
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/qLkQcWlUBy8" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
|
|
||||
|
|
||||
|
||||
Backtesting a Strategy
|
||||
----------------------
|
||||
|
||||
This is the first video of a two-part series on using Catalyst for algorithmic
|
||||
trading. This video implements a simple momentum strategy based on
|
||||
`mean reversion <example-algos.html#mean-reversion>`_: when the cryptoasset
|
||||
goes up quickly, we’re going to buy; when it goes down quickly, we’re going to
|
||||
sell. Hopefully, we’ll ride the waves.
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/JOBRwst9jUY" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
|
|
||||
|
|
||||
|
||||
Live Trading a Strategy
|
||||
-----------------------
|
||||
|
||||
This is the second part of the two-part series on using Catalyst for algorithmic
|
||||
trading. Having backtested `our strategy <example-algos.html#mean-reversion>`_
|
||||
in the previous video, we now take it to trade live against the Bittrex exchange.
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/NupiE-Xuglw" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
|
|
||||
|
|
||||
Reference in New Issue
Block a user