Module scheduler

Classes

class ClientState
A simple object holding information about a client.
class CollectTaskMetaDataPlugin
class KilledWorker
class Scheduler
Dynamic distributed task scheduler
class TaskGroup
Collection tracking all tasks within a group
class TaskPrefix
Collection tracking all tasks within a group
class TaskState
A simple object holding information about a task.
class WorkerState
A simple object holding information about a worker.
class WorkerStatusPlugin
An plugin to share worker status with a remote observer

Functions

▶ def decide_worker(ts, all_workers, valid_workers, objective)

Decide which worker should take task *ts*.

We choose the worker that has the data on which *ts* depends.

If several workers have dependencies then we choose the less-busy worker.

Optionally provide *valid_workers* of where jobs are allowed to occur
(if all workers are allowed to take the task, pass True instead).

If the task requires data communication because no eligible worker has
all the dependencies already, then we choose to minimize the number
of bytes sent between workers.  This is determined by calling the
*objective* function.

▷ def heartbeat_interval(n)
Interval in seconds that we desire heartbeats based on number of workers
source link

▶ def validate_state(tasks, workers, clients)

Validate a current runtime state

source link

This performs a sequence of checks on the entire graph, running in about
linear time.  This raises assert errors if anything doesn't check out.

▷ def validate_task_state(ts)
Validate the given TaskState.
source link
▷ def validate_worker_state(ws)
source link

Distributed

Module scheduler

Classes

Functions