Class AdaptiveCore

The core logic for adaptive deployments, with none of the cluster details

Declaration

class AdaptiveCore
source link

Documentation

This class controls our adaptive scaling behavior.  It is intended to be
used as a super-class or mixin.  It expects the following state and methods:

**State**

plan: set
    A set of workers that we think should exist.
    Here and below worker is just a token, often an address or name string

requested: set
    A set of workers that the cluster class has successfully requested from
    the resource manager.  We expect that resource manager to work to make
    these exist.

observed: set
    A set of workers that have successfully checked in with the scheduler

These sets are not necessarily equivalent.  Often plan and requested will
be very similar (requesting is usually fast) but there may be a large delay
between requested and observed (often resource managers don't give us what
we want).

**Functions**

target : -> int
    Returns the target number of workers that should exist.
    This is often obtained by querying the scheduler

workers_to_close : int -> Set[worker]
    Given a target number of workers,
    returns a set of workers that we should close when we're scaling down

scale_up : int -> None
    Scales the cluster up to a target number of workers, presumably
    changing at least ``plan`` and hopefully eventually also ``requested``

scale_down : Set[worker] -> None
    Closes the provided set of workers

Attributes

Methods

Subclasses

Reexports