[autoscaler] Replace cluster yaml validation with json schema v… (#7261)

* replace manual cluster yaml validation with json schema
- improved error message
- support for intellisense in VSCode (or other IDEs)
- run linting
- moved schema to ray/autoscaler
- fixed typo
- remove importlib dependency

* Update python/ray/autoscaler/autoscaler.py

* read

* restrict allowed properties

* added unit test for invalid yaml
added ray[test] package (remove pytest from default dependencies)

* updated autoscaler test to use ValidationError exception

* add missing dependency

* added pytest

* replace manual cluster yaml validation with json schema
- improved error message
- support for intellisense in VSCode (or other IDEs)
- run linting
- moved schema to ray/autoscaler
- fixed typo
- remove importlib dependency

* Update python/ray/autoscaler/autoscaler.py

* read

* restrict allowed properties

* added unit test for invalid yaml
added ray[test] package (remove pytest from default dependencies)

* updated autoscaler test to use ValidationError exception

* add missing dependency

* added pytest

* removed parameterized dependency
reverted ray[test] intro

* removed parameterized

* fix_tests

* format

Co-authored-by: Ubuntu <marcozo@mc-ray-jumpbox.chcbtljllnieveqhw3e4c1ducc.xx.internal.cloudapp.net>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
This commit is contained in:
Markus Cozowicz
2020-03-10 18:58:55 -07:00
committed by GitHub
co-authored by Ubuntu Richard Liaw
parent 6163b21458
commit 49439611f1
7 changed files with 310 additions and 182 deletions
+5 -4
View File
@@ -5,6 +5,7 @@ import time
import unittest
import yaml
import copy
from jsonschema.exceptions import ValidationError
import ray
import ray.services as services
@@ -323,17 +324,17 @@ class AutoscalingTest(unittest.TestCase):
self.fail("Test config did not pass validation test!")
config["blah"] = "blah"
with pytest.raises(ValueError):
with pytest.raises(ValidationError):
validate_config(config)
del config["blah"]
config["provider"]["blah"] = "blah"
with pytest.raises(ValueError):
with pytest.raises(ValidationError):
validate_config(config)
del config["provider"]["blah"]
del config["provider"]
with pytest.raises(ValueError):
with pytest.raises(ValidationError):
validate_config(config)
def testValidateDefaultConfig(self):
@@ -346,7 +347,7 @@ class AutoscalingTest(unittest.TestCase):
config = fillout_defaults(config)
try:
validate_config(config)
except Exception:
except ValidationError:
self.fail("Default config did not pass validation test!")
def testScaleUp(self):