mirror of
https://github.com/wassname/ray.git
synced 2026-06-28 02:01:24 +08:00
Alok Singh
42a9233e1d
* Allow yapf to lint individual files * Add tip for using yapf * Update doc * Update script to autoformat changed py files The new default is for the script to only updated changed files to encourage using it as a pre-push hook. Travis still checks all since it's not that big an increase to runtime. * Exclude formatting thirdparty/autogen py files * Symlink .travis -> scripts Hidden directories may get glossed over otherwise. * .travis -> scripts in docs They are symlinks to the same thing, but `scripts` is more dev-friendly, while `.travis` is really only for Travis CI. * Document different yapf format functions Most devs will only need `format_changed`, and this is run by default. `format_changed` should be fast enough in most cases to work as a pre-commit hook. * Speed up yapf by only formatting changed files * Update docs 1. Mention how yapf can be used a pre-commit hook 2. rm `bash`, script is executable * Update yapf.sh * Update development.rst * Update yapf.sh * Use bash arrays for correct argument splitting Playing fast and loose with whitespace in bash is a terrible idea. * Only format non-excluded by default * Check changes against master Normally, the remote is called `origin`, but naming it explicit * Adding missing directory to `format_all` * Cleanup YAPF code Remove unused function and move around code to make clearer and adding lines give cleaner diffs. * Ensure correct files are autoformatted * Fix cmd line arg splitting Each arg has to be in its own set of quotes. * Diff against mergebase TIL there's a clean syntax for doing that, but it's too clever to belong in a shell script. We use `mapfile -t` to ensure no problems down the line with weird filenames.
86 lines
3.2 KiB
ReStructuredText
86 lines
3.2 KiB
ReStructuredText
Development Tips
|
|
================
|
|
|
|
If you are doing development on the Ray codebase, the following tips may be
|
|
helpful.
|
|
|
|
1. **Speeding up compilation:** Be sure to install Ray with
|
|
|
|
.. code-block:: shell
|
|
|
|
cd ray/python
|
|
pip install -e . --verbose
|
|
|
|
The ``-e`` means "editable", so changes you make to files in the Ray
|
|
directory will take effect without reinstalling the package. In contrast, if
|
|
you do ``python setup.py install``, files will be copied from the Ray
|
|
directory to a directory of Python packages (often something like
|
|
``/home/ubuntu/anaconda3/lib/python3.6/site-packages/ray``). This means that
|
|
changes you make to files in the Ray directory will not have any effect.
|
|
|
|
If you run into **Permission Denied** errors when running ``pip install``,
|
|
you can try adding ``--user``. You may also need to run something like ``sudo
|
|
chown -R $USER /home/ubuntu/anaconda3`` (substituting in the appropriate
|
|
path).
|
|
|
|
If you make changes to the C++ files, you will need to recompile them.
|
|
However, you do not need to rerun ``pip install -e .``. Instead, you can
|
|
recompile much more quickly by doing
|
|
|
|
.. code-block:: shell
|
|
|
|
cd ray/build
|
|
make -j8
|
|
|
|
2. **Starting processes in a debugger:** When processes are crashing, it is
|
|
often useful to start them in a debugger (``gdb`` on Linux or ``lldb`` on
|
|
MacOS). See the latest discussion about how to do this `here`_.
|
|
|
|
3. **Running tests locally:** Suppose that one of the tests (e.g.,
|
|
``runtest.py``) is failing. You can run that test locally by running
|
|
``python test/runtest.py``. However, doing so will run all of the tests which
|
|
can take a while. To run a specific test that is failing, you can do
|
|
|
|
.. code-block:: shell
|
|
|
|
cd ray
|
|
python test/runtest.py APITest.testKeywordArgs
|
|
|
|
When running tests, usually only the first test failure matters. A single
|
|
test failure often triggers the failure of subsequent tests in the same
|
|
script.
|
|
|
|
4. **Running linter locally:** To run the Python linter on a specific file, run
|
|
something like ``flake8 ray/python/ray/worker.py``. You may need to first run
|
|
``pip install flake8``.
|
|
|
|
5. **Autoformatting code**. We use ``yapf`` https://github.com/google/yapf for
|
|
linting, and the config file is located at ``.style.yapf``. We recommend
|
|
running ``.travis/yapf.sh`` prior to pushing to format changed files.
|
|
Note that some projects such as dataframes and rllib are currently excluded.
|
|
|
|
6. **Inspecting Redis shards by hand:** To inspect the primary Redis shard by
|
|
hand, you can query it with commands like the following.
|
|
|
|
.. code-block:: python
|
|
|
|
r_primary = ray.worker.global_worker.redis_client
|
|
r_primary.keys("*")
|
|
|
|
To inspect other Redis shards, you will need to create a new Redis client.
|
|
For example (assuming the relevant IP address is ``127.0.0.1`` and the
|
|
relevant port is ``1234``), you can do this as follows.
|
|
|
|
.. code-block:: python
|
|
|
|
import redis
|
|
r = redis.StrictRedis(host='127.0.0.1', port=1234)
|
|
|
|
You can find a list of the relevant IP addresses and ports by running
|
|
|
|
.. code-block:: python
|
|
|
|
r_primary.lrange('RedisShards', 0, -1)
|
|
|
|
.. _`here`: https://github.com/ray-project/ray/issues/108
|