Module distributed

distributed

source link

Re-exported Classes

class Actor
Controls an object on a remote worker
[distributed.actor.Actor]
class ActorFuture
Future to an actor's method call
[distributed.actor.ActorFuture]
class Adaptive
Adaptively allocate workers based on scheduler load. A superclass.
[distributed.deploy.adaptive.Adaptive]
class Client
Connect to and submit computation to a Dask cluster
[distributed.client.Client]
class Event
Distributed Centralized Event equivalent to asyncio.Event
[distributed.event.Event]
class Executor
Deprecated: see Client
[distributed.client.Executor]
class Future
A remotely running computation
[distributed.client.Future]
class LocalCluster
Create local Scheduler and Workers
[distributed.deploy.local.LocalCluster]
class Lock
Distributed Centralized Lock
[distributed.lock.Lock]
class Nanny
A process to manage worker processes
[distributed.nanny.Nanny]
class PipInstall
A Worker Plugin to pip install a set of packages
[distributed.diagnostics.plugin.PipInstall]
class Pub
Publish data with Publish-Subscribe pattern
[distributed.pubsub.Pub]
class Queue
Distributed Queue
[distributed.queues.Queue]
class Reschedule
Reschedule this task
[distributed.worker.Reschedule]
class Scheduler
Dynamic distributed task scheduler
[distributed.scheduler.Scheduler]
class SchedulerPlugin
Interface to extend the Scheduler
[distributed.diagnostics.plugin.SchedulerPlugin]
class Security
Security configuration for a Dask cluster.
[distributed.security.Security]
class Semaphore
Semaphore
[distributed.semaphore.Semaphore]
class SpecCluster
Cluster that requires a full specification of workers
[distributed.deploy.spec.SpecCluster]
class Sub
Subscribe to a Publish/Subscribe topic
[distributed.pubsub.Sub]
class Variable
Distributed Global Variable
[distributed.variable.Variable]
class Worker
Worker node in a Dask distributed cluster
[distributed.worker.Worker]
class WorkerPlugin
Interface to extend the Worker
[distributed.diagnostics.plugin.WorkerPlugin]
class as_completed
Return futures in the order in which they complete
[distributed.client.as_completed]
class get_task_metadata
Collect task metadata within a context block
[distributed.client.get_task_metadata]
class get_task_stream
Collect task stream within a context block
[distributed.client.get_task_stream]
class performance_report
Gather performance report
[distributed.client.performance_report]
class rpc
Conveniently interact with a remote server
[distributed.core.rpc]

Re-exported Functions

def CompatibleExecutor(*args, **kwargs)
[distributed.client.CompatibleExecutor]
def SSHCluster(hosts: List[str] = None, connect_options: Union[List[dict], dict] = {}, worker_options: dict = {}, scheduler_options: dict = {}, worker_module: str = "distributed.cli.dask_worker", remote_python: Union[str, List[str]] = None, **kwargs)
Deploy a Dask cluster using SSH
[distributed.deploy.ssh.SSHCluster]
def default_client(c=None)
Return a client if one has started
[distributed.client.default_client]
def fire_and_forget(obj)
Run tasks at least once, even if we release the futures
[distributed.client.fire_and_forget]
def futures_of(o, client=None)
Future objects in a collection
[distributed.client.futures_of]
def get_client(address=None, timeout=None, resolve_address=True)
Get a client while within a task.
[distributed.worker.get_client]
def get_versions()
Get version information or return default if unable to do so.
[distributed._version.get_versions]
def get_worker()
Get the worker currently running this task
[distributed.worker.get_worker]
def local_client(*args, **kwargs)
[distributed.worker_client.local_client]
def progress(*futures, notebook=None, multi=True, complete=True, **kwargs)
Track progress of futures
[distributed.diagnostics.progressbar.progress]
def rejoin()
Have this thread rejoin the ThreadPoolExecutor
[distributed.threadpoolexecutor.rejoin]
def secede()
Have this task secede from the worker's thread pool
[distributed.worker.secede]
def sync(loop, func, *args, callback_timeout=None, **kwargs)
Run coroutine in loop running in separate thread.
[distributed.utils.sync]
def wait(fs, timeout=None, return_when=ALL_COMPLETED)
Wait until all/any futures are finished
[distributed.client.wait]

Submodules

Module distributed.actor
Module distributed.asyncio
Module distributed.batched
Module distributed.bokeh
Module distributed.cfexecutor
Module distributed.cli
Module distributed.client
Module distributed.comm
Module distributed.compatibility
Module distributed.config
Module distributed.core
Module distributed.counter
Module distributed.dashboard
Module distributed.deploy
Module distributed.diagnostics
Module distributed.diskutils
Module distributed.event
Module distributed.http
Module distributed.lock
Module distributed.locket
Module distributed.metrics
Module distributed.nanny
Module distributed.node
Module distributed.preloading
Module distributed.process
Module distributed.proctitle
Module distributed.profile
This module contains utility functions to construct and manipulate counting data structures for frames.
Module distributed.protocol
Module distributed.publish
Module distributed.pubsub
Module distributed.pytest_resourceleaks
A pytest plugin to trace resource leaks.
Module distributed.queues
Module distributed.recreate_exceptions
Module distributed.scheduler
Module distributed.security
Module distributed.semaphore
Module distributed.sizeof
Module distributed.stealing
Module distributed.system
Module distributed.system_monitor
Module distributed.tests
Module distributed.threadpoolexecutor
Modified ThreadPoolExecutor to support threads leaving the thread pool
Module distributed.utils
Module distributed.utils_comm
Module distributed.utils_perf
Module distributed.utils_test
Module distributed.variable
Module distributed.versions
utilities for package version introspection
Module distributed.worker
Module distributed.worker_client

Distributed

Module distributed

Re-exported Classes

Re-exported Functions

Submodules