Stephan's comments

This commit is contained in:
François Boulogne
2014-09-02 09:44:33 -04:00
parent 26e0348f3c
commit ffe7156f31
+11 -17
View File
@@ -1,13 +1,13 @@
=========================
How to parallelize loops?
=========================
========================
How to parallelize loops
========================
In image processing, we frequently apply the same algorithm
on a large batch of images. Let us define an example.
on a large batch of images. Here is an example:
.. code-block:: python
from skimage import data, color
from skimage import data, color, util
from skimage.restoration import denoise_tv_chambolle
from skimage.feature import hog
@@ -25,24 +25,18 @@ on a large batch of images. Let us define an example.
# Prepare images
hubble = data.hubble_deep_field()
width = 10
pics = [hubble[:,slice:slice+width] for slice in range(0, 1000, width)]
pics = util.view_as_windows(hubble, (width, hubble.shape[1], hubble.shape[2]), step=width)
To call the function ``task`` on each element of the list ``pics``, it is
usual to write a for loop. To measure the execution time of this loop, a function
is defined and called with ``timeit``.
usual to write a for loop. To measure the execution time of this loop, you can
use ipython and measure the execution time with ``%timeit``.
.. code-block:: python
def classic_loop():
for image in pics:
task(image)
task(image[0][0])
import timeit
print("classic_loop():", timeit.timeit("classic_loop()", setup="from __main__ import (classic_loop, task, pics)", number=1))
Alternatively, you can use ipython and measure the execution time with ``%timeit``.
.. code-block:: python
%timeit classic_loop()
@@ -51,7 +45,7 @@ Another equivalent way to code this loop is to use a comprehension list which ha
.. code-block:: python
def comprehension_loop():
[task(image) for image in pics]
[task(image[0][0]) for image in pics]
%timeit comprehension_loop()
@@ -62,6 +56,6 @@ The number of jobs can be specified.
from joblib import Parallel, delayed
def joblib_loop():
Parallel(n_jobs=4)(delayed(task)(i) for i in pics)
Parallel(n_jobs=4)(delayed(task)(i[0][0]) for i in pics)
%timeit joblib_loop()