Class LocalCluster

distributed.deploy.local.LocalCluster

Create local Scheduler and Workers

Declaration

class LocalCluster(SpecCluster)

Documentation

This creates a "cluster" of a scheduler and workers running on the local
machine.

Attributes

n_workers : int
```
Number of workers to start
```

processes : bool

Whether to use processes (True) or threads (False).  Defaults to True

threads_per_worker : int
```
Number of threads per each worker
```

scheduler_port : int

Port of the scheduler.  8786 by default, use 0 to choose a random port

silence_logs : logging level

Level of logs to print out to stdout.  ``logging.WARN`` by default.
Use a falsey value like False or None for no change.

host : string

Host address on which the scheduler will listen, defaults to only localhost

ip : string
```
Deprecated.  See ``host`` above.
```

dashboard_address : str

Address on which to listen for the Bokeh diagnostics server like
'localhost:8787' or '0.0.0.0:8787'.  Defaults to ':8787'.
Set to ``None`` to disable the dashboard.
Use ':0' for a random port.

worker_dashboard_address : str

Address on which to listen for the Bokeh worker diagnostics server like
'localhost:8787' or '0.0.0.0:8787'.  Defaults to None which disables the dashboard.
Use ':0' for a random port.

diagnostics_port : int
```
Deprecated.  See dashboard_address.
```

asynchronous : bool (False by default)

Set to True if using this cluster within async/await functions or within
Tornado gen.coroutines.  This should remain False for normal use.

blocked_handlers : List[str]

A list of strings specifying a blacklist of handlers to disallow on the Scheduler,
like ``['feed', 'run_function']``

service_kwargs : Dict[str, Dict]

Extra keywords to hand to the running services

security : Security or bool

Configures communication security in this cluster. Can be a security
object, or True. If True, temporary self-signed credentials will
be created automatically.

protocol : str

Protocol to use like ``tcp://``, ``tls://``, ``inproc://``
This defaults to sensible choice given other keyword arguments like
``processes`` and ``security``

interface : str

Network interface to use.  Defaults to lo/localhost

worker_class : Worker

Worker class used to instantiate workers from.

**worker_kwargs :

Extra worker arguments. Any additional keyword arguments will be passed
to the ``Worker`` class constructor.

Examples

>>> cluster = LocalCluster()  # Create a local cluster with as many workers as cores  # doctest: +SKIP
>>> cluster  # doctest: +SKIP
LocalCluster("127.0.0.1:8786", workers=8, threads=8)

>>> c = Client(cluster)  # connect to local cluster  # doctest: +SKIP

Scale the cluster to three workers

>>> cluster.scale(3)  # doctest: +SKIP

Pass extra keyword arguments to Bokeh

>>> LocalCluster(service_kwargs={'dashboard': {'prefix': '/foo'}})  # doctest: +SKIP

Methods

▶ def __init__(self, n_workers=None, threads_per_worker=None, processes=True, loop=None, ...) override
def __init__(
self,
n_workers=None,
threads_per_worker=None,
processes=True,
loop=None,
start=None,
host=None,
ip=None,
scheduler_port=0,
silence_logs=logging.WARN,
dashboard_address=":8787",
worker_dashboard_address=None,
diagnostics_port=None,
services=None,
worker_services=None,
service_kwargs=None,
asynchronous=False,
security=None,
protocol=None,
blocked_handlers=None,
interface=None,
worker_class=None,
scheduler_kwargs=None,
**worker_kwargs,
)
source link
This method overrides distributed.deploy.spec.SpecCluster.__init__.
▷ def start_worker(self, *args, **kwargs)
source link

Inherited methods

Methods inherited from distributed.deploy.spec.SpecCluster:
__await__, __exit__, adapt, new_worker_spec, plan, requested, scale
Methods inherited from distributed.deploy.cluster.Cluster:
__del__, __enter__, __repr__, asynchronous, close, dashboard_link, get_logs, logs, observed, scheduler_address, sync

Reexports

Imported in distributed.comm.tests.test_ucx.
Imported in distributed.deploy.tests.test_local.
Imported in distributed.deploy.tests.test_adaptive.
Imported in distributed.deploy.
Imported in distributed.tests.test_client_loop.
Imported in distributed.tests.test_client.
Imported in distributed.

Distributed