wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-09-11 12:43:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

General attribute-based heterogeneity support with hard and soft constraints (#248)

Browse Source 
* attribute-based heterogeneity-awareness in global scheduler and photon

* minor post-rebase fix

* photon: enforce dynamic capacity constraint on task dispatch

* globalsched: cap the number of times we try to schedule a task in round robin

* propagating ability to specify resource capacity to ray.init

* adding resources to remote function export and fetch/register

* globalsched: remove unused functions; update cached photon resource capacity (until next photon heartbeat)

* Add some integration tests.

* globalsched: cleanup + factor out constraint checking

* lots of style

* task_spec_required_resource: global refactor

* clang format

* clang format + comment update in photon

* clang format photon comment

* valgrind

* reduce verbosity for Travis

* Add test for scheduler load balancing.

* addressing comments

* refactoring global scheduler algorithm

* Minor cleanups.

* Linting.

* Fix array_test.py and linting.

* valgrind fix for photon tests

* Attempt to fix stress tests.

* fix hashmap free

* fix hashmap free comment

* memset photon resource vectors to 0 in case they get used before the first heartbeat

* More whitespace changes.

* Undo whitespace error I introduced.
This commit is contained in:
Alexey Tumanov
2017-02-09 01:34:14 -08:00
committed by Robert Nishihara
parent 1a7e1c47cb
commit dfb6107b22
22 changed files with 1037 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/common/task.c
+17
View File
@@ -58,6 +58,12 @@ struct task_spec_impl {
* has been written so far, relative to &task_spec->args_and_returns[0] +
* (task_spec->num_args + task_spec->num_returns) * sizeof(task_arg) */
int64_t args_value_offset;
/** Resource vector for this task. A resource vector maps a resource index
* (like "cpu" or "gpu") to the number of units of that resource required.
* Note that this will allow us to support arbitrary attributes:
* For example, we can have a coloring of nodes and "red" can correspond
* to 0.0, "green" to 1.0 and "yellow" to 2.0. */
double required_resources[MAX_RESOURCE_INDEX];
/** Argument and return IDs as well as offsets for pass-by-value args. */
task_arg args_and_returns[0];
};
@@ -259,6 +265,12 @@ int64_t task_args_add_val(task_spec *spec, uint8_t *data, int64_t length) {
return spec->arg_index++;
}
void task_spec_set_required_resource(task_spec *spec,
int64_t resource_index,
double value) {
spec->required_resources[resource_index] = value;
}
object_id task_return(task_spec *spec, int64_t return_index) {
/* Check that the task has been constructed. */
DCHECK(!task_ids_equal(spec->task_id, NIL_TASK_ID));
@@ -268,6 +280,11 @@ object_id task_return(task_spec *spec, int64_t return_index) {
return ret->obj_id;
}
double task_spec_get_required_resource(const task_spec *spec,
int64_t resource_index) {
return spec->required_resources[resource_index];
}
void free_task_spec(task_spec *spec) {
/* Check that the task has been constructed. */
DCHECK(!task_ids_equal(spec->task_id, NIL_TASK_ID));