wassname/optuna-dashboard
1
0
Fork 0
mirror of https://github.com/wassname/optuna-dashboard.git synced 2026-09-09 11:28:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #612 from keisuke-umezawa/fix/update-hitl-example-for-artifacts

Browse Source 
Update hitl tutorial for optuna.artifacts
This commit is contained in:
c-bata
2023-09-11 11:42:22 +09:00
committed by GitHub
parent 9200b9652a 633b6aaec8
commit a85bc62485
2 changed files with 25 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tutorials/hitl.rst
+15 -14
View File
@@ -99,7 +99,7 @@ To run `the script <https://github.com/optuna/optuna-dashboard/blob/main/example
.. code-block:: console
$ pip install "optuna>=3.2.0" "optuna-dashboard>=0.10.0" pillow
$ pip install "optuna>=3.3.0" "optuna-dashboard>=0.12.0" pillow
You will use SQLite for the storage backend in this tutorial. Ensure that the following library is installed:
@@ -181,7 +181,9 @@ Let’s walk through the script we used for the optimization.
.. code-block:: python
:linenos:
def suggest_and_generate_image(study: optuna.Study, artifact_backend: FileSystemBackend) -> None:
def suggest_and_generate_image(
study: optuna.Study, artifact_store: FileSystemArtifactStore
) -> None:
# 1. Ask new parameters
trial = study.ask()
r = trial.suggest_int("r", 0, 255)
@@ -194,7 +196,7 @@ Let’s walk through the script we used for the optimization.
image.save(image_path)
# 3. Upload Artifact
artifact_id = upload_artifact(artifact_backend, trial, image_path)
artifact_id = upload_artifact(trial, image_path, artifact_store)
artifact_path = get_artifact_path(trial, artifact_id)
# 4. Save Note
@@ -207,12 +209,12 @@ Let’s walk through the script we used for the optimization.
)
save_note(trial, note)
In the ``suggest_and_generate_image`` function, a new Trial is obtained and new hyperparameters are suggested for that Trial. Based on those hyperparameters, an RGB image is generated as an artifact. The generated image is then uploaded to the Artifact Storage of the Optuna Dashboard, and the image is also displayed in the Dashboard's Note. For more information on how to use the Note feature, please refer to the API Reference of :func:`~optuna_dashboard.save_note`.
In the ``suggest_and_generate_image`` function, a new Trial is obtained and new hyperparameters are suggested for that Trial. Based on those hyperparameters, an RGB image is generated as an artifact. The generated image is then uploaded to the Artifact Store of the Optuna, and the image is also displayed in the Dashboard's Note. For more information on how to use the Note feature, please refer to the API Reference of :func:`~optuna_dashboard.save_note`.
.. code-block:: python
:linenos:
def start_optimization(artifact_backend: FileSystemBackend) -> NoReturn:
def start_optimization(artifact_store: FileSystemArtifactStore) -> NoReturn:
# 1. Create Study
study = optuna.create_study(
study_name="Human-in-the-loop Optimization",
@@ -220,10 +222,10 @@ In the ``suggest_and_generate_image`` function, a new Trial is obtained and new
sampler=optuna.samplers.TPESampler(constant_liar=True, n_startup_trials=5),
load_if_exists=True,
)
# 2. Set an objective name
study.set_metric_names(["Looks like sunset color?"])
# 3. Register ChoiceWidget
register_objective_form_widgets(
study,
@@ -236,15 +238,14 @@ In the ``suggest_and_generate_image`` function, a new Trial is obtained and new
],
)
# 4. Start Optimization
# 4. Start Human-in-the-loop Optimization
n_batch = 4
while True:
running_trials = study.get_trials(deepcopy=False, states=(TrialState.RUNNING,))
if len(running_trials) >= n_batch:
time.sleep(1) # Avoid busy-loop
continue
suggest_and_generate_image(study, artifact_backend)
suggest_and_generate_image(study, artifact_store)
The function ``start_optimization`` defines our loop for HITL optimization to generate an image resembling a sunset color.
@@ -258,10 +259,10 @@ The function ``start_optimization`` defines our loop for HITL optimization to ge
def main() -> NoReturn:
tmp_path = os.path.join(os.path.dirname(__file__), "tmp")
# 1. Create Artifact Store
artifact_path = os.path.join(os.path.dirname(__file__), "artifact")
artifact_backend = FileSystemBackend(base_path=artifact_path)
artifact_store = FileSystemArtifactStore(artifact_path)
if not os.path.exists(artifact_path):
os.mkdir(artifact_path)
@@ -270,11 +271,11 @@ The function ``start_optimization`` defines our loop for HITL optimization to ge
os.mkdir(tmp_path)
# 2. Run optimize loop
start_optimization(artifact_backend)
start_optimization(artifact_store)
In the ``main`` function, at first, the locations of the Artifact Store is set.
* At #1, the :class:`~optuna_dashboard.FileSystemBackend` is created, which is one of the Artifact Storage options used in the Optuna Dashboard. Artifact Storage is used to store artifacts (data, files, etc.) generated during Optuna trials. For more information, please refer to the API Reference.
* At #1, the `FileSystemArtifactStore <https://optuna.readthedocs.io/en/stable/reference/generated/optuna.artifacts.FileSystemArtifactStore.html>`_ is created, which is one of the Artifact Store options used in the Optuna. Artifact Store is used to store artifacts (data, files, etc.) generated during Optuna trials. For more information, please refer to the API Reference.
* At #2, `start_optimization()` function, which is described above, is called.
After that, two folders are created, artifact and tmp, and then ``start_optimization`` function is called to start the HITL optimization using Optuna.
examples/hitl/main.py
+10 -8
View File
@@ -4,17 +4,19 @@ import time
from typing import NoReturn
import optuna
from optuna.artifacts import FileSystemArtifactStore
from optuna.artifacts import upload_artifact
from optuna.trial import TrialState
from optuna_dashboard import ChoiceWidget
from optuna_dashboard import register_objective_form_widgets
from optuna_dashboard import save_note
from optuna_dashboard.artifact import get_artifact_path
from optuna_dashboard.artifact import upload_artifact
from optuna_dashboard.artifact.file_system import FileSystemBackend
from PIL import Image
def suggest_and_generate_image(study: optuna.Study, artifact_backend: FileSystemBackend) -> None:
def suggest_and_generate_image(
study: optuna.Study, artifact_store: FileSystemArtifactStore
) -> None:
# 1. Ask new parameters
trial = study.ask()
r = trial.suggest_int("r", 0, 255)
@@ -27,7 +29,7 @@ def suggest_and_generate_image(study: optuna.Study, artifact_backend: FileSystem
image.save(image_path)
# 3. Upload Artifact
artifact_id = upload_artifact(artifact_backend, trial, image_path)
artifact_id = upload_artifact(trial, image_path, artifact_store)
artifact_path = get_artifact_path(trial, artifact_id)
# 4. Save Note
@@ -41,7 +43,7 @@ def suggest_and_generate_image(study: optuna.Study, artifact_backend: FileSystem
save_note(trial, note)
def start_optimization(artifact_backend: FileSystemBackend) -> NoReturn:
def start_optimization(artifact_store: FileSystemArtifactStore) -> NoReturn:
# 1. Create Study
study = optuna.create_study(
study_name="Human-in-the-loop Optimization",
@@ -72,7 +74,7 @@ def start_optimization(artifact_backend: FileSystemBackend) -> NoReturn:
if len(running_trials) >= n_batch:
time.sleep(1) # Avoid busy-loop
continue
suggest_and_generate_image(study, artifact_backend)
suggest_and_generate_image(study, artifact_store)
def main() -> NoReturn:
@@ -80,7 +82,7 @@ def main() -> NoReturn:
# 1. Create Artifact Store
artifact_path = os.path.join(os.path.dirname(__file__), "artifact")
artifact_backend = FileSystemBackend(base_path=artifact_path)
artifact_store = FileSystemArtifactStore(artifact_path)
if not os.path.exists(artifact_path):
os.mkdir(artifact_path)
@@ -89,7 +91,7 @@ def main() -> NoReturn:
os.mkdir(tmp_path)
# 2. Run optimize loop
start_optimization(artifact_backend)
start_optimization(artifact_store)
if __name__ == "__main__":