wassname/jupyter_contrib_nbextensions
1
0
Fork 0
mirror of https://github.com/wassname/jupyter_contrib_nbextensions.git synced 2026-09-12 12:31:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #6 from jcb91/docs

Browse Source 
[docs] update readmes
This commit is contained in:
Jean-François Bercher
2017-01-05 14:27:48 +01:00
committed by GitHub
parent e2aa2495e0 aaf264d509
commit 299cf7aaa5
5 changed files with 524 additions and 386 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+154 -75
View File
@@ -1,37 +1,79 @@
KernelExecOnCells library
=========================
KernelExecOnCells library and nbextensions
==========================================
The KernelExecOnCells library provides a set of functions which enable to execute an external function, through the current Jupyter kernel, on the text of selected cells, and replace this text with the output of the external function. This scheme has been applied, for instance, to prettify code, see the [code-prettify](README_code_prettify.md) extension or to refactor it, see the [2to3](README_2to3.md) extension.
These extensions are defined as simple plugins of the library. Defining such a plugin is described in the last section below.
The KernelExecOnCells library is a shared library for creating Jupyter
nbextensions which transform code cell text using calls to the active kernel.
parameters
----------
The library uses a series of parameters, describing the configuration of the plugin. These parameters are specified as an object in the plugin source file. There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator](https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator)
or by editing the `notebook` section config file directly. Let `mod_name` be the name of the plugin module (eg code_prettify, 2to3, ...) and `LANG` the kernel language (eg julia, python, R...). These parameters are as follows:
This scheme has been applied to create several nbextensions which are also
included in the repository.
For instance, to prettify code, see the [code-prettify] nbextension, or to
refactor python 2 code for python 3, see the [2to3] extension.
These nbextensions are defined as simple plugins of the main KernelExecOnCells
library. Defining such a plugin is described in the last section below.
- `mod_name.add_toolbar_button`: Whether to add a toolbar button to
apply the kernel function to the selected cell(s), defaults to `true`.
- `mod_name.register_hotkey`: Whether to register a hotkey to apply the kernel function to
the selected cell(s). defaults to `true`.
- `mod_name.hotkey`: Hotkey to use to apply the kernel function to the selected cell(s).
defaults to `Ctrl-L`
- `mod_name.show_alerts_for_errors`: Whether to show alerts for errors in
the kernel calls. Defaults to `true`
The following give the per-kernel options as keys for the python language:
Compatible Kernels
------------------
- `mod_name.kernel_config_map_json` The value of this key is a string
which can be parsed into a json object giving the config for each kernel
language. Relevant keys, using the python language key:
The library is kernel-language agnostic, as described in the [internals]
section below. Essentially any kernel capable of interpreting and creating
json-formatted strings, and sending them to the stream output (where print
statements in most languages go) should be easy to integrate.
Hopefully, that covers pretty much all languages!
* `mod_name.kernel_config_map_json.LANG.library`: string to load the
kernel's prettifier libraries. For python, defaults to
```python
import json
import yapf.yapflib.yapf_api
```
Options
-------
The library uses a series of options, describing the configuration of the
plugin. Default values for these options are specified as an object in the
plugin source file, and can be overriden by values loaded from config.
There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator] or by editing the `notebook` section config
file directly.
If `mod_name` is the name of the plugin module (e.g. `code_prettify`, `2to3`,
...) and `LANG` the lowercased kernel language (eg julia, python, r ...), then
the options are as follows:
- `mod_name.add_toolbar_button`:
Whether to add a toolbar button to transform the selected cell(s).
Defaults to `true`.
- `mod_name.button_icon`:
A font-awesome class defining the icon used for the toolbar button and
actions. See [http://fontawesome.io/icons] for available icon classes.
Defaults to `fa-legal`.
- `mod_name.button_label`:
Toolbar button label text. Also used in the actions' help text.
Defaults to `mod_name`.
- `mod_name.register_hotkey`:
Whether to register hotkeys to transform the selected cell(s)/whole notebook.
Defaults to `true`.
- `mod_name.hotkeys.process_all`:
Hotkey to use to transform all the code cells in the notebook.
Defaults to `Ctrl-Shift-L`.
- `mod_name.hotkeys.process_selected`:
Hotkey to use to transform the selected cell(s).
Defaults to `Ctrl-L`.
- `mod_name.show_alerts_for_errors`:
Whether to show alerts for errors in the kernel calls.
Defaults to `true`.
- `mod_name.kernel_config_map_json`:
The value of this key is a string which can be parsed into a json object
giving the config for each kernel language.
The following give the per-kernel options of the parsed json, using the
language key `LANG`, to be replaced as appropriate:
* `mod_name.kernel_config_map_json.LANG.library`:
String to execute in the kernel in order to load any necessary kernel
libraries.
* `mod_name.kernel_config_map_json.LANG.replacements_json_to_kernel`:
a list of pairs of strings, used as arguments to javascript's
@@ -41,21 +83,20 @@ The following give the per-kernel options as keys for the python language:
language) be left as the default, an empty list.
* `mod_name.kernel_config_map_json.LANG.prefix` and
`mod_name.kernel_config_map_json.LANG.postfix`: strings added as
bookends to the kernel string (translated from the json string using the
replacements above) to make up the kernel prettifier call kernel's
prettifier libraries.
`mod_name.kernel_config_map_json.LANG.postfix`:
strings added as bookends to the kernel string (translated from the json
string using the replacements above) to make up the kernel prettifier call
kernel's prettifier libraries.
* `mod_name.kernel_config_map_json.LANG.trim_formatted_text`: whether
to trim whitespace from the resulting cell text. Since jupyter cells don't
usually have leading or trailing whitespace, the default behaviour is to
trim the output text, in order to prevent the output of the kernel function adding extra
newlines at the end (a common behaviour for source files, where having a
trailing newline is often considered good practice).
* `mod_name.kernel_config_map_json.LANG.trim_formatted_text`:
Whether to trim whitespace from the transformed cell text. Since jupyter
cells don't usually have leading or trailing whitespace, the default
behaviour is to trim the transformed text, in order to prevent the
transform adding extra newlines at the end (a common behaviour for source
files, where having a trailing newline is often considered good practice).
internals
Internals
---------
The model is essentially:
@@ -66,11 +107,13 @@ The model is essentially:
programming languages without much modification (e.g. a valid json string
is also a valid string in python 3, and also in python 2 when prefixed with
a `u`), and easily converted for use in others (because of its simplicity).
2. Optional regex replacements are used to translate the json-format string
into a valid kernel string. Python, R and javascript don't require this
step, but other languages may do, so it's implemented for flexibility
using the per-kernel config key `replacements_json_to_kernel`, which is a
list of pairs of arguments to javascript `String.replace`.
3. The kernel-specific prettifier call is then composed from
`kernel_config.prefix` + `kernel_text_string` + `kernel_config.postfix` and
sent to the kernel for execution. This kernel call is expected to get the
@@ -79,6 +122,7 @@ The model is essentially:
arrange. The reason for the printing text rather than simply displaying it,
is that it prevents us having to translate from a kernel string
representing a json string.
4. The callback for the kernel execution in client-side javascript parses the
printed json-format string, optionally trims trailing whitespace according
to the `trim_formatted_text` key (which defaults to `true`) in the
@@ -88,7 +132,7 @@ The process is probably best illustrated using an example for the python
implementation in `code_prettify`:
1. **At nbextension load**, the `code_prettify.kernel_config_map_json` config
parameter is parsed to give the json object
option is parsed to give the json object
```json
{
@@ -100,12 +144,12 @@ implementation in `code_prettify`:
}
```
(other kernel laguages are omitted for clarity).
(other kernel languages are omitted for clarity).
2. **On kernel becoming ready**, the nbextension looks up the config for the
kernel's language (in our example, this is the `python` key of the kernel
config json object above). It then sends the kernel config's `library`
string to the kernel for execution. Thus the python implementation above
string to the kernel for execution. Thus the python implementation above
executes
```python
@@ -138,8 +182,8 @@ implementation in `code_prettify`:
print(json.dumps(yapf.yapflib.yapf_api.FormatCode(u"msg= 'hello '+\"world\"\nprint (\n msg )")[0]))
```
4. What gets 'printed' by the kernel (i.e. returned to the javascript
stream callback) is the following json-format string:
4. What gets 'printed' by the kernel (i.e. returned to the javascript stream
callback) is the following json-format string:
```json
"msg = 'hello ' + \"world\"\nprint(msg)\n"
@@ -153,51 +197,86 @@ implementation in `code_prettify`:
print(msg)
```
adding a new plugin
-------------------
As an example, we will add a new plugin which reformats code sing the autopep8 module in python. Such a plugin, [jupyter-autopep8](https://github.com/kenkoooo/jupyter-autopep8) has be developed by @kenkoooo as a fork of an old version of `code_prettify`. Redefining it here has the advantage of using the updated and more robust architecture, in addition to the possibilty to reformat the whole notebook at once. For that extension, we just have to run `import autopep8` as the library, and then call `autopep8.fix_code` on cells' text. Hence what we have to do is:
Defining a new plugin
---------------------
As an example, we will add a new plugin which reformats code using the
[autopep8] module in python, rather than the [yapf] library used by
`code_prettify`. Such a plugin, [jupyter-autopep8] was developed by [@kenkoooo]
as a fork of an old version of `code_prettify`. Redefining it here has the
advantage of using the updated and more-robust architecture, in addition to
making it possible to reformat the whole notebook in one go.
For this new nbextension, we just have to run `import autopep8` as the kernel
library code, and then call the `autopep8.fix_code` function on cells' text.
Hence what we have to do is:
- copy `code_prettify.js` to `autopep8.js`
- update `mod_name`, `hotkeys`, `button_icon` default config values in the new
`autopep8.js`. Also update the `cfg.kernel_config_map` value to use the
correct kernel code:
```javascript
cfg.kernel_config_map = { // map of options for supported kernels
"python": {
"library": "import json\nimport autopep8",
"prefix": "print(json.dumps(autopep8.fix_code(u",
"postfix": ")))"
}
};
```
- copy `code_prettify.yaml` to `autopep8.yaml`, and update its values (name,
require, readme, plus the new defaults for hotkeys, icon, and
kernel_config_map
- copy code_prettify.js to autopep8.js
- update `mod_name`, `hotkeys`, `button_icon`
- update cfg.kernel_config_map into
```
cfg.kernel_config_map = { // map of parameters for supported kernels
"python": {
"library": "import json\nimport autopep8",
"prefix": "print(json.dumps(autopep8.fix_code(u",
"postfix": ")))"
}
};
```
- copy code_prettify.yaml into autopep8.yaml
- update values in autopep8.yaml (replace code_prettify by autopep8, update hotkeys, icon, and kernel_config_map)
- that's all :-)
Of course, one can also update the configuration of `code_prettify` using the nbextensions_configurator to use autopep8 instead of yapf to reformat python code.
Of course, for this simple case, one could equally have just updated the
configuration of `code_prettify` using the [jupyter_nbextensions_configurator]
to use [autopep8] instead of [yapf] to reformat the python code.
But, if you want two alternative prettifiers available for the same kernel
language, we need to define separate plugins.
History:
History
-------
- [@jfbercher](https://github.com/jfbercher), august 14, 2016, first version [yapf_ext]
- [@jfbercher](https://github.com/jfbercher), august 19, 2016, second version [code_prettify]
- [@jfbercher], august 14, 2016, first version, named `yapf_ext`
- [@jfbercher], august 19, 2016, second version `code_prettify`
- introduced support for R and javascript.
- changed extension name from `yapf_ext` to `code_prettify`
- [@jcb91](https://github.com/jcb91), december 2016
- [@jcb91], december 2016
- made addition of toolbar button & hotkey configurable
- reworked to avoid regex replacements for conversion to/from kernel string
formats, in favour of json-string interchange
- made kernel-specific prettifier calls configurable, allowing support for
different prettifiers & arbitrary kernels
- improved documentation
- [@jfbercher](https://github.com/jfbercher), december 2016-january 2017
- [@jfbercher], december 2016-january 2017
- added a configurable shortkey to reflow the whole notebook
- extracted most of the code to build a general library of functions, `kernel_exec_on_cell.js`, which can be used for all nbextensions which needs to exec some code (via the current kernel) on the text from cells.
- extracted most of the code to build a general library of functions,
`kernel_exec_on_cell.js`, which can be used for all nbextensions which
needs to exec some code (via the current kernel) on the text from cells.
- added 2to3 as a plugin to the shared library
- [@jcb91](https://github.com/jcb91), january 2017
- [code_prettify lib] use actions to avoid problems with auto-generated actions generated by keyboard_manager,
which were overwriting each other.This has the added benefit of allowing the action to be used in the command palette.
Also fix toolbar button removal, which was removing only the button, leaving the button group behind.
- [@jfbercher](https://github.com/jfbercher), january 2017
- updated documentations
- added autopep8 as a plugin to the shared library
- [@jcb91], january 2017
- library: Use actions to avoid problems with auto-generated actions
generated by keyboard_manager, which were overwriting each other.
Also fix toolbar button removal.
- [@jfbercher], january 2017
- updated documentation
- added autopep8 nbextension as a plugin using the shared library
[2to3]: README_2to3.md
[@jcb91]: https://github.com/jcb91
[@jfbercher]: https://github.com/jfbercher
[@kenkoooo]: https://github.com/kenkoooo
[autopep8]: https://github.com/hhatto/autopep8
[code-prettify]: README_code_prettify.md
[http://fontawesome.io/icons]: http://fontawesome.io/icons
[internals]: #Internals
[jupyter-autopep8]: https://github.com/kenkoooo/jupyter-autopep8
[jupyter_nbextensions_configurator]: https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator
[yapf]: https://github.com/google/yapf
README_2to3.md
+124 -13
View File
@@ -1,24 +1,135 @@
# A 2to3 converter
A 2to3 converter
================
This extension converts python2 code in a notebook's code cell to python3 code.
Under the hood, it uses Pythons build in [2to3](https://docs.python.org/3/library/2to3.html) function.
This nbextension converts python2 code in notebook code cells to python3 code.
The project was forked by @EWouters from [code_prettify](https://github.com/jfbercher/code_prettify) by [@jfbercher](https://github.com/jfbercher), retaining most of the code. It now shares with `code_prettify` a library of functions dedicated to kernel processing of cells text in Jupyter notebooks.
Under the hood, it uses a call to the current notebook kernel to reformat the
code.
The conversion run by the kernel uses Python's standard-library [lib2to3]
module.
The 2to3 conversion is based on [2to3_nb.py](https://gist.github.com/takluyver/c8839593c615bb2f6e80) by [@takluyver](https://github.com/takluyver) and [@fperez](https://github.com/fperez).
Possibly it will be extended to use the [futurize](http://python-future.org/automatic_conversion.html) functions so it can convert both ways.
Under the hood, it uses the KerneExecOnCells library, shared between `code_prettify`, `autopep8` and `2to3` (to date)
The nbextension provides
- a toolbar button (configurable to be added or not)
- a keyboard shortcut for reformatting the current code-cell (default shortcut
is `Alt-A`, can also be configured not to add the keyboard shortcut).
is `Ctrl-M`, can also be configured not to add the keyboard shortcut).
- a keyboard shortcut for reformatting the whole notebook (default shortcut
is `Alt-Shift-A`, can also be configured not to add the keyboard shortcut).
Syntax shall be correct. The nbextension will also point basic syntax errors.
is `Ctrl-Shift-M`, can also be configured not to add the keyboard shortcut).
Syntax needs to be correct, but the nbextension may be able to point out basic
syntax errors.
![](demo_2to3.gif)
See `code_prettify`'s [README](README.md) for the internals used by the extension and a description of the options.
Options
-------
All options are provided by the [KerneExecOnCells library] - see the
[internals] section below for details.
There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator] or by editing the `notebook` section config
file directly.
The options are as follows:
- `2to3.add_toolbar_button`:
Whether to add a toolbar button to transform the selected cell(s).
Defaults to `true`.
- `2to3.button_icon`:
A font-awesome class defining the icon used for the toolbar button and
actions. See [http://fontawesome.io/icons] for available icon classes.
Defaults to `fa-legal`.
- `2to3.button_label`:
Toolbar button label text. Also used in the actions' help text.
Defaults to `Convert Python 2 to 3`.
- `2to3.register_hotkey`:
Whether to register hotkeys to transform the selected cell(s)/whole notebook.
Defaults to `true`.
- `2to3.hotkeys.process_all`:
Hotkey to use to transform all the code cells in the notebook.
Defaults to `Ctrl-Shift-L`.
- `2to3.hotkeys.process_selected`:
Hotkey to use to transform the selected cell(s).
Defaults to `Ctrl-L`.
- `2to3.show_alerts_for_errors`:
Whether to show alerts for errors in the kernel calls.
Defaults to `true`.
- `2to3.kernel_config_map_json`:
The value of this key is a string which can be parsed into a json object
giving the config for each kernel language.
The following give the per-kernel options of the parsed json, using the
language key `python `:
* `2to3.kernel_config_map_json.python.library`:
String to execute in the kernel in order to load any necessary kernel
libraries.
* `2to3.kernel_config_map_json.python.replacements_json_to_kernel`:
a list of pairs of strings, used as arguments to javascript's
`String.replace(from, to)` to translate from a json string into a valid
representation of the same string in the kernel language. Since json
strings are particularly simple, this can often (as with the python
language) be left as the default, an empty list.
* `2to3.kernel_config_map_json.python.prefix` and
`2to3.kernel_config_map_json.python.postfix`:
Strings added as bookends to the kernel string (translated from the json
string using the replacements above) to make up the kernel prettifier call
kernel's prettifier libraries.
* `2to3.kernel_config_map_json.python.trim_formatted_text`:
Whether to trim whitespace from the transformed cell text. Since jupyter
cells don't usually have leading or trailing whitespace, the default
behaviour is to trim the transformed text, in order to prevent the
transform adding extra newlines at the end (a common behaviour for source
files, where having a trailing newline is often considered good practice).
Internals
---------
Under the hood, this nbextension uses the [KerneExecOnCells library], a shared
library for creating Jupyter nbextensions which transform code cell text using
calls to the active kernel.
See the [shared README] for the internal model used by the nbextension.
History
-------
The project was forked by [@EWouters] from [@jfbercher]'s [code_prettify],
retaining most of the code.
It has since been altered to use the [KerneExecOnCells library], a shared
library for creating Jupyter nbextensions which transform code cell text using
calls to the active kernel.
The 2to3 conversion's kernel-side python code is based on [2to3_nb.py] by
[@takluyver] and [@fperez].
It could be extended to use the [futurize] functions so it can convert both
ways.
[2to3_nb.py]: https://gist.github.com/takluyver/c8839593c615bb2f6e80
[@EWouters]: https://github.com/EWouters
[@fperez]: https://github.com/fperez
[@jfbercher]: https://github.com/jfbercher
[@takluyver]: https://github.com/takluyver
[code_prettify]: https://github.com/jfbercher/code_prettify
[futurize]: http://python-future.org/automatic_conversion.html
[http://fontawesome.io/icons]: http://fontawesome.io/icons
[internals]: #Internals
[jupyter_nbextensions_configurator]: https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator
[KerneExecOnCells library]: README.md
[lib2to3]: https://docs.python.org/3/library/2to3.html#module-lib2to3
[shared README]: README.md
README_KernelExecOncells.md
-203
View File
@@ -1,203 +0,0 @@
KernelExecOnCells library
=========================
The KernelExecOnCells library provides a set of functions which enable to execute an external function, through the current Jupyter kernel, on the text of selected cells, and replace this text with the output of the external function. This scheme has been applied, for instance, to prettify code, see the [code-prettify](README_code_prettify.md) extension or to refactor it, see the [2to3](README_2to3.md) extension.
These extensions are defined as simple plugins of the library. Defining such a plugin is described in the last section below.
parameters
----------
The library uses a series of parameters, describing the configuration of the plugin. These parameters are specified as an object in the plugin source file. There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator](https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator)
or by editing the `notebook` section config file directly. Let `mod_name` be the name of the plugin module (eg code_prettify, 2to3, ...) and `LANG` the kernel language (eg julia, python, R...). These parameters are as follows:
- `mod_name.add_toolbar_button`: Whether to add a toolbar button to
apply the kernel function to the selected cell(s), defaults to `true`.
- `mod_name.register_hotkey`: Whether to register a hotkey to apply the kernel function to
the selected cell(s). defaults to `true`.
- `mod_name.hotkey`: Hotkey to use to apply the kernel function to the selected cell(s).
defaults to `Ctrl-L`
- `mod_name.show_alerts_for_errors`: Whether to show alerts for errors in
the kernel calls. Defaults to `true`
The following give the per-kernel options as keys for the python language:
- `mod_name.kernel_config_map_json` The value of this key is a string
which can be parsed into a json object giving the config for each kernel
language. Relevant keys, using the python language key:
* `mod_name.kernel_config_map_json.LANG.library`: string to load the
kernel's prettifier libraries. For python, defaults to
```python
import json
import yapf.yapflib.yapf_api
```
* `mod_name.kernel_config_map_json.LANG.replacements_json_to_kernel`:
a list of pairs of strings, used as arguments to javascript's
`String.replace(from, to)` to translate from a json string into a valid
representation of the same string in the kernel language. Since json
strings are particularly simple, this can often (as with the python
language) be left as the default, an empty list.
* `mod_name.kernel_config_map_json.LANG.prefix` and
`mod_name.kernel_config_map_json.LANG.postfix`: strings added as
bookends to the kernel string (translated from the json string using the
replacements above) to make up the kernel prettifier call kernel's
prettifier libraries.
* `mod_name.kernel_config_map_json.LANG.trim_formatted_text`: whether
to trim whitespace from the resulting cell text. Since jupyter cells don't
usually have leading or trailing whitespace, the default behaviour is to
trim the output text, in order to prevent the output of the kernel function adding extra
newlines at the end (a common behaviour for source files, where having a
trailing newline is often considered good practice).
internals
---------
The model is essentially:
1. The cell text is grabbed by client-side javascript, then turned into a json
string using javascript `JSON.stringify`. Since json-compatible strings are
a particularly simple string format, which is compatible with many other
programming languages without much modification (e.g. a valid json string
is also a valid string in python 3, and also in python 2 when prefixed with
a `u`), and easily converted for use in others (because of its simplicity).
2. Optional regex replacements are used to translate the json-format string
into a valid kernel string. Python, R and javascript don't require this
step, but other languages may do, so it's implemented for flexibility
using the per-kernel config key `replacements_json_to_kernel`, which is a
list of pairs of arguments to javascript `String.replace`.
3. The kernel-specific prettifier call is then composed from
`kernel_config.prefix` + `kernel_text_string` + `kernel_config.postfix` and
sent to the kernel for execution. This kernel call is expected to get the
formatted cell text _printed_ as a json-compatible string. Since most
kernel languages have json packages, this should hopefully be easy to
arrange. The reason for the printing text rather than simply displaying it,
is that it prevents us having to translate from a kernel string
representing a json string.
4. The callback for the kernel execution in client-side javascript parses the
printed json-format string, optionally trims trailing whitespace according
to the `trim_formatted_text` key (which defaults to `true`) in the
per-kernel config, and then sets the cell text using the result.
The process is probably best illustrated using an example for the python
implementation in `code_prettify`:
1. **At nbextension load**, the `code_prettify.kernel_config_map_json` config
parameter is parsed to give the json object
```json
{
"python": {
"library": "import json\nimport yapf.yapflib.yapf_api",
"prefix": "print(json.dumps(yapf.yapflib.yapf_api.FormatCode(u",
"postfix": ")[0]))"
}
}
```
(other kernel laguages are omitted for clarity).
2. **On kernel becoming ready**, the nbextension looks up the config for the
kernel's language (in our example, this is the `python` key of the kernel
config json object above). It then sends the kernel config's `library`
string to the kernel for execution. Thus the python implementation above
executes
```python
import json
import yapf.yapflib.yapf_api
```
3. **On requesting a cell be prettified** which can happen by clicking the
toolbar, or with a (configurable) hotkey, the following happens:
Say the cell to be formatted contains the following ugly python code:
```python
msg= 'hello '+"world"
print (
msg )
```
Then the result of the `JSON.stringify` call will be a string containing
```json
"msg= 'hello '+\"world\"\nprint (\n msg )"
```
(note the opening and closing quotes). Concatenating this with the prefix &
postfix strings from the python kernel config above, gives us the kernel
code to execute. The call sent to the python kernel is therefore
```python
print(json.dumps(yapf.yapflib.yapf_api.FormatCode(u"msg= 'hello '+\"world\"\nprint (\n msg )")[0]))
```
4. What gets 'printed' by the kernel (i.e. returned to the javascript
stream callback) is the following json-format string:
```json
"msg = 'hello ' + \"world\"\nprint(msg)\n"
```
The default is to trim whitepace from the returned prettified text, which
results in the final prettified python code for the cell:
```python
msg = 'hello ' + "world"
print(msg)
```
adding a new plugin
-------------------
As an example, we will add a new plugin which reformats code sing the autopep8 module in python. Such a plugin, [jupyter-autopep8](https://github.com/kenkoooo/jupyter-autopep8) has be developed by @kenkoooo as a fork of an old version of `code_prettify`. Redefining it here has the advantage of using the updated and more robust architecture, in addition to the possibilty to reformat the whole notebook at once. For that extension, we just have to run `import autopep8` as the library, and then call `autopep8.fix_code` on cells' text. Hence what we have to do is:
- copy code_prettify.js to autopep8.js
- update `mod_name`, `hotkeys`, `button_icon`
- update cfg.kernel_config_map into
```
cfg.kernel_config_map = { // map of parameters for supported kernels
"python": {
"library": "import json\nimport autopep8",
"prefix": "print(json.dumps(autopep8.fix_code(u",
"postfix": ")))"
}
};
```
- copy code_prettify.yaml into autopep8.yaml
- update values in autopep8.yaml (replace code_prettify by autopep8, update hotkeys, icon, and kernel_config_map)
- that's all :-)
Of course, one can also update the configuration of `code_prettify` using the nbextensions_configurator to use autopep8 instead of yapf to reformat python code.
History:
-------
- [@jfbercher](https://github.com/jfbercher), august 14, 2016, first version [yapf_ext]
- [@jfbercher](https://github.com/jfbercher), august 19, 2016, second version [code_prettify]
- introduced support for R and javascript.
- changed extension name from `yapf_ext` to `code_prettify`
- [@jcb91](https://github.com/jcb91), december 2016
- made addition of toolbar button & hotkey configurable
- reworked to avoid regex replacements for conversion to/from kernel string
formats, in favour of json-string interchange
- made kernel-specific prettifier calls configurable, allowing support for
different prettifiers & arbitrary kernels
- improved documentation
- [@jfbercher](https://github.com/jfbercher), december 2016-january 2017
- added a configurable shortkey to reflow the whole notebook
- extracted most of the code to build a general library of functions, `kernel_exec_on_cell.js`, which can be used for all nbextensions which needs to exec some code (via the current kernel) on the text from cells.
- added 2to3 as a plugin to the shared library
- [@jcb91](https://github.com/jcb91), january 2017
- [code_prettify lib] use actions to avoid problems with auto-generated actions generated by keyboard_manager,
which were overwriting each other.This has the added benefit of allowing the action to be used in the command palette.
Also fix toolbar button removal, which was removing only the button, leaving the button group behind.
- [@jfbercher](https://github.com/jfbercher), january 2017
- updated documentations
- added autopep8 as a plugin to the shared library
README_autopep8.md
+116 -11
View File
@@ -1,27 +1,132 @@
Python code prettifying unsing autopep8
=======================================
jupyter-autopep8
================
This extension reformats/prettifies code in a notebook's code cell, uniquely for python language, using the autopep8 package.
This nbextension reformats/prettifies code in notebook python code cells.
Under the hood, it uses a call to the current notebook kernel to reformat the
code.
The conversion run by the kernel uses the python [autopep8] package, and thus is compatible only with python kernels.
Under the hood, it uses the KerneExecOnCells library, shared with `code_prettify`and `2to3`
The nbextension provides
- a toolbar button (configurable to be added or not)
- a keyboard shortcut for reformatting the current code-cell (default shortcut
is `Alt-A`, can also be configured not to add the keyboard shortcut).
- a keyboard shortcut for reformatting the whole notebook (default shortcut
is `Alt-Shift-A`, can also be configured not to add the keyboard shortcut).
Syntax shall be correct. The nbextension will also point basic syntax errors.
prerequisites
Syntax needs to be correct, but the nbextension may be able to point out basic
syntax errors.
Prerequisites
-------------
Of course, you must have the necessary kernel-specific packages installed for
Of course, you must have the necessary kernel-specific package installed for
the prettifier call to work:
- for the default python implementation, the
[autopep8](https://github.com/hhatto/autopep8) module is required:
pip install autopep8
pip install autopep8
Others you might consider using include [autopep8](https://github.com/hhatto/autopep8).
Options
-------
All options are provided by the [KerneExecOnCells library] - see the
[internals] section below for details.
There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator] or by editing the `notebook` section config
file directly.
The options are as follows:
- `autopep8.add_toolbar_button`:
Whether to add a toolbar button to transform the selected cell(s).
Defaults to `true`.
- `autopep8.button_icon`:
A font-awesome class defining the icon used for the toolbar button and
actions. See [http://fontawesome.io/icons] for available icon classes.
Defaults to `fa-cog`.
- `autopep8.button_label`:
Toolbar button label text. Also used in the actions' help text.
Defaults to `Prettify (using autopep8)`.
- `autopep8.register_hotkey`:
Whether to register hotkeys to transform the selected cell(s)/whole notebook.
Defaults to `true`.
- `autopep8.hotkeys.process_all`:
Hotkey to use to transform all the code cells in the notebook.
Defaults to `Alt-Shift-A`.
- `autopep8.hotkeys.process_selected`:
Hotkey to use to transform the selected cell(s).
Defaults to `Alt-A`.
- `autopep8.show_alerts_for_errors`:
Whether to show alerts for errors in the kernel calls.
Defaults to `true`.
- `autopep8.kernel_config_map_json`:
The value of this key is a string which can be parsed into a json object
giving the config for each kernel language.
The following give the per-kernel options of the parsed json, using the
language key `python `:
* `autopep8.kernel_config_map_json.python.library`:
String to execute in the kernel in order to load any necessary kernel
libraries.
* `autopep8.kernel_config_map_json.python.replacements_json_to_kernel`:
a list of pairs of strings, used as arguments to javascript's
`String.replace(from, to)` to translate from a json string into a valid
representation of the same string in the kernel language. Since json
strings are particularly simple, this can often (as with the python
language) be left as the default, an empty list.
* `autopep8.kernel_config_map_json.python.prefix` and
`autopep8.kernel_config_map_json.python.postfix`:
Strings added as bookends to the kernel string (translated from the json
string using the replacements above) to make up the kernel prettifier call
kernel's prettifier libraries.
* `autopep8.kernel_config_map_json.python.trim_formatted_text`:
Whether to trim whitespace from the transformed cell text. Since jupyter
cells don't usually have leading or trailing whitespace, the default
behaviour is to trim the transformed text, in order to prevent the
transform adding extra newlines at the end (a common behaviour for source
files, where having a trailing newline is often considered good practice).
Internals
---------
Under the hood, this nbextension uses the [KerneExecOnCells library], a shared
library for creating Jupyter nbextensions which transform code cell text using
calls to the active kernel.
See the [shared README] for the internal model used by the nbextension.
History
-------
The project was forked by [@kenkoooo] from [@jfbercher]'s [code_prettify],
retaining most of the code.
It has since been altered to use the [KerneExecOnCells library], a shared
library for creating Jupyter nbextensions which transform code cell text using
calls to the active kernel.
[@jfbercher]: https://github.com/jfbercher
[@kenkoooo]: https://github.com/kenkoooo
[autopep8]: https://github.com/hhatto/autopep8
[code_prettify]: https://github.com/jfbercher/code_prettify
[http://fontawesome.io/icons]: http://fontawesome.io/icons
[internals]: #Internals
[jupyter_nbextensions_configurator]: https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator
[KerneExecOnCells library]: README.md
[shared README]: README.md
README_code_prettify.md
+130 -84
View File
@@ -1,115 +1,133 @@
A Code Prettifier
=================
This extension reformats/prettifies code in a notebook's code cell.
This nbextension reformats/prettifies code in notebook code cells.
Under the hood, it uses a call to the current notebook kernel to reformat the
code. Thus the actual prettifier package has to be callable from
the current kernel language.
Example implementations for prettifiers for ipython, ir and ijavascript kernels
are provided which should work out of the box (assuming availability of the
relevant kernel-specific [prerequisites](#prerequisites) mentioned below), but
the kernel-specific prettifier calls are configurable, so the model is
applicable to essentially any kernel language.
Other languages may be added as defaults in the future, but given that there
are more than 50
[kernels](https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages)
available for Jupyter, it is not easily possible to support all of them out of
the box, unless people with experience in the relevant kernels have the time to
contribute code. For information on how the reformatting takes place, and how
to adapt it for your particular kernel/prettifier, see the
[internals](#internals) section below.
Under the hood, it uses the [KerneExecOnCells](README_kernelExecOncells.md) library, shared between `code_prettify`, `autopep8` and `2to3` (to date)
code.
Thus the actual prettifier package has to be callable from the current kernel
language.
With an appropriately-configured prettifier for the kernel in use, the
nbextension provides
- a toolbar button (configurable to be added or not)
- a keyboard shortcut for reformatting the current code-cell (default shortcut
is `Ctrl-L`, can also be configured not to add the keyboard shortcut).
- a keyboard shortcut for reformatting the whole notebook (default shortcut
is `Ctrl-Shift-L`, can also be configured not to add the keyboard shortcut).
Syntax shall be correct. The nbextension will also point basic syntax errors.
Syntax shall be correct. The nbextension will also point basic syntax errors.
Syntax shall be correct. The nbextension may also point out basic syntax errors.
![](demo-py.gif)
![](demo-R.gif)
![](demo-jv.gif)
prerequisites
Compatible Kernels
------------------
Example implementations are provided for prettifiers for ipython, ir and
ijavascript kernels which should work out of the box (assuming availability of
the relevant kernel-specific [prerequisites] mentioned below), but the
kernel-specific prettifier calls are configurable, so the model is applicable
to essentially any kernel language and prettifier library.
Other languages may be added as defaults in the future, but given that there
are more than 50 [kernels] available for Jupyter, it is not easily possible to
support all of them out of the box, unless people with experience in the
relevant kernels have the time to contribute code. For information on how the
reformatting takes place, and how to adapt it for your particular
kernel/prettifier, see the [options] and [internals] sections below.
If you implement a language that isn't yet provided by default, please submit a
PR or let us know to add it to the repo :)
Under the hood, this nbextension's functionality is provided by the
[KerneExecOnCells library], a shared library for creating Jupyter nbextensions
which transform code cell text using calls to the active kernel.
Prerequisites
-------------
Of course, you must have the necessary kernel-specific packages installed for
the prettifier call to work:
- for the default python implementation, the
[YAPF](https://github.com/google/yapf) module is required:
- for the default python implementation, the [yapf] module is required:
pip install yapf
pip install yapf
Others you might consider using include [autopep8](https://github.com/hhatto/autopep8).
Others you might consider using include [autopep8] - see [README_autopep8.md].
- for R, the default implementation uses the
[formatR](http://yihui.name/formatR/) and
[jsonlite](https://github.com/jeroenooms/jsonlite) packages:
- for R, the default implementation uses the [formatR] and [jsonlite] packages:
```r
install.packages(c("formatR", "jsonlite"), repos="http://cran.rstudio.com")
```
- for [ijavascript](http://n-riesco.github.io/ijavascript/), the
[js-beautify](https://github.com/beautify-web/js-beautify) package is used:
- for [ijavascript], the [js-beautify] package is used:
(*Under linux, in the root of your user tree = ~*)
npm install js-beautify
Under Windows, you may then need to set the `NODE_PATH` environment variable
(see [this question on stackoverflow](http://stackoverflow.com/questions/9587665/nodejs-cannot-find-installed-module-on-windows))
to it to `%AppData%\npm\node_modules` (Windows 7/8/10).
(see [this question on stackoverflow]) to it to `%AppData%\npm\node_modules`
(Windows 7/8/10).
To be done with it once and for all, add this as a System variable in the
Advanced tab of the System Properties dialog.
options
Options
-------
All options are provided by the [KerneExecOnCells library]. - see the
[internals] section below for details.
There are a few nbextension-wide options, configurable using the
[jupyter_nbextensions_configurator](https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator)
or by editing the `notebook` section config file directly:
[jupyter_nbextensions_configurator] or by editing the `notebook` section config
file directly.
The options are as follows:
- `code_prettify.add_toolbar_button`: Whether to add a toolbar button to
prettify the selected cell(s), defaults to `true`.
- `code_prettify.button_icon`: a font-awesome class defining the icon used for
the toolbar button and actions. See
[http://fontawesome.io/icons/](http://fontawesome.io/icons/)
for available icon classes. Defaults to `fa-legal`.
- `code_prettify.button_label`: Toolbar button label text. Also used in the
actions' help text. Defaults to `Code prettify`.
- `code_prettify.register_hotkey`: Whether to register a hotkey to prettify
the selected cell(s). defaults to `true`.
- `code_prettify.hotkeys.process_selected`: Hotkey to use to prettify the
selected cell(s). defaults to `Ctrl-L`
- `code_prettify.hotkeys.process_all`: Hotkey to use to prettify all
cells in the notebook. Defaults to `Ctrl-Shift-L`
- `code_prettify.show_alerts_for_errors`: Whether to show alerts for errors in
the kernel prettifying calls. Defaults to `true`
- `code_prettify.add_toolbar_button`:
Whether to add a toolbar button to transform the selected cell(s).
Defaults to `true`.
The following give the per-kernel options as keys for the python language:
- `code_prettify.button_icon`:
A font-awesome class defining the icon used for the toolbar button and
actions. See [http://fontawesome.io/icons] for available icon classes.
Defaults to `fa-legal`.
- `code_prettify.kernel_config_map_json` The value of this key is a string
which can be parsed into a json object giving the config for each kernel
language. Relevant keys, using the python language key:
- `code_prettify.button_label`:
Toolbar button label text. Also used in the actions' help text.
Defaults to `Code prettify`.
* `code_prettify.kernel_config_map_json.python.library`: string to load the
kernel's prettifier libraries. For python, defaults to
- `code_prettify.register_hotkey`:
Whether to register hotkeys to transform the selected cell(s)/whole notebook.
Defaults to `true`.
```python
import json
import yapf.yapflib.yapf_api
```
- `code_prettify.hotkeys.process_all`:
Hotkey to use to transform all the code cells in the notebook.
Defaults to `Ctrl-Shift-L`.
- `code_prettify.hotkeys.process_selected`:
Hotkey to use to transform the selected cell(s).
Defaults to `Ctrl-L`.
- `code_prettify.show_alerts_for_errors`:
Whether to show alerts for errors in the kernel calls.
Defaults to `true`.
- `code_prettify.kernel_config_map_json`:
The value of this key is a string which can be parsed into a json object
giving the config for each kernel language.
The following give the per-kernel options of the parsed json, using the
language key `python `:
* `code_prettify.kernel_config_map_json.python.library`:
String to execute in the kernel in order to load any necessary kernel
libraries.
* `code_prettify.kernel_config_map_json.python.replacements_json_to_kernel`:
a list of pairs of strings, used as arguments to javascript's
@@ -119,22 +137,26 @@ The following give the per-kernel options as keys for the python language:
language) be left as the default, an empty list.
* `code_prettify.kernel_config_map_json.python.prefix` and
`code_prettify.kernel_config_map_json.python.postfix`: strings added as
bookends to the kernel string (translated from the json string using the
replacements above) to make up the kernel prettifier call kernel's
prettifier libraries.
`code_prettify.kernel_config_map_json.python.postfix`:
Strings added as bookends to the kernel string (translated from the json
string using the replacements above) to make up the kernel prettifier call
kernel's prettifier libraries.
* `code_prettify.kernel_config_map_json.python.trim_formatted_text`: whether
to trim whitespace from the prettified cell. Since jupyter cells don't
usually have leading or trailing whitespace, the default behaviour is to
trim the prettified text, in order to prevent the prettifying adding extra
newlines at the end (a common behaviour for source files, where having a
trailing newline is often considered good practice).
* `code_prettify.kernel_config_map_json.python.trim_formatted_text`:
Whether to trim whitespace from the transformed cell text. Since jupyter
cells don't usually have leading or trailing whitespace, the default
behaviour is to trim the transformed text, in order to prevent the
transform adding extra newlines at the end (a common behaviour for source
files, where having a trailing newline is often considered good practice).
internals
Internals
---------
Under the hood, this nbextension uses the [KerneExecOnCells library], a shared
library for creating Jupyter nbextensions which transform code cell text using
calls to the active kernel.
The model is essentially:
1. The cell text is grabbed by client-side javascript, then turned into a json
@@ -143,11 +165,13 @@ The model is essentially:
programming languages without much modification (e.g. a valid json string
is also a valid string in python 3, and also in python 2 when prefixed with
a `u`), and easily converted for use in others (because of its simplicity).
2. Optional regex replacements are used to translate the json-format string
into a valid kernel string. Python, R and javascript don't require this
step, but other languages may do, so it's implemented for flexibility
using the per-kernel config key `replacements_json_to_kernel`, which is a
list of pairs of arguments to javascript `String.replace`.
3. The kernel-specific prettifier call is then composed from
`kernel_config.prefix` + `kernel_text_string` + `kernel_config.postfix` and
sent to the kernel for execution. This kernel call is expected to get the
@@ -156,6 +180,7 @@ The model is essentially:
arrange. The reason for the printing text rather than simply displaying it,
is that it prevents us having to translate from a kernel string
representing a json string.
4. The callback for the kernel execution in client-side javascript parses the
printed json-format string, optionally trims trailing whitespace according
to the `trim_formatted_text` key (which defaults to `true`) in the
@@ -165,7 +190,7 @@ The process is probably best illustrated using an example for the python
implementation:
1. **At nbextension load**, the `code_prettify.kernel_config_map_json` config
parameter is parsed to give the json object
option is parsed to give the json object
```json
{
@@ -177,12 +202,12 @@ implementation:
}
```
(other kernel laguages are omitted for clarity).
(other kernel languages are omitted for clarity).
2. **On kernel becoming ready**, the nbextension looks up the config for the
kernel's language (in our example, this is the `python` key of the kernel
config json object above). It then sends the kernel config's `library`
string to the kernel for execution. Thus the python implementation above
string to the kernel for execution. Thus the python implementation above
executes
```python
@@ -215,8 +240,8 @@ implementation:
print(json.dumps(yapf.yapflib.yapf_api.FormatCode(u"msg= 'hello '+\"world\"\nprint (\n msg )")[0]))
```
4. What gets 'printed' by the kernel (i.e. returned to the javascript
stream callback) is the following json-format string:
4. What gets 'printed' by the kernel (i.e. returned to the javascript stream
callback) is the following json-format string:
```json
"msg = 'hello ' + \"world\"\nprint(msg)\n"
@@ -231,20 +256,41 @@ implementation:
```
History:
---------
History
-------
- [@jfbercher](https://github.com/jfbercher), august 14, 2016, first version.
- [@jfbercher](https://github.com/jfbercher), august 19, 2016, second version,
- [@jfbercher], august 14, 2016, first version, named `yapf_ext`
- [@jfbercher], august 19, 2016, second version `code_prettify`
- introduced support for R and javascript.
- changed extension name from `yapf_ext` to `code_prettify`
- [@jcb91](https://github.com/jcb91), december 2016
- [@jcb91], december 2016
- made addition of toolbar button & hotkey configurable
- reworked to avoid regex replacements for conversion to/from kernel string
formats, in favour of json-string interchange
- made kernel-specific prettifier calls configurable, allowing support for
different prettifiers & arbitrary kernels
- improved documentation
- [@jfbercher](https://github.com/jfbercher), december 2016
- [@jfbercher], december 2016-january 2017
- added a configurable shortkey to reflow the whole notebook
- extracted most of the code to build a general library of functions, `kernel_exec_on_cell.js`, which can be used for all nbextensions which needs to exec some code (via the current kernel) on the text from cells.
- extracted most of the code to build a general library of functions,
`kernel_exec_on_cell.js`, which can be used for all nbextensions which
needs to exec some code (via the current kernel) on the text from cells.
[@jcb91]: https://github.com/jcb91
[@jfbercher]: https://github.com/jfbercher
[autopep8]: https://github.com/hhatto/autopep8
[formatR]: http://yihui.name/formatR
[http://fontawesome.io/icons]: http://fontawesome.io/icons
[ijavascript]: http://n-riesco.github.io/ijavascript
[internals]: #Internals
[js-beautify]: https://github.com/beautify-web/js-beautify
[jsonlite]: https://github.com/jeroenooms/jsonlite
[jupyter_nbextensions_configurator]: https://github.com/Jupyter-contrib/jupyter_nbextensions_configurator
[KerneExecOnCells library]: README.md
[kernels]: https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages
[options]: #Options
[prerequisites]: #Prerequisites
[README_autopep8.md]: README_autopep8.md
[this question on stackoverflow]: http://stackoverflow.com/questions/9587665/nodejs-cannot-find-installed-module-on-windows
[yapf]: https://github.com/google/yapf