Module utils_comm

Classes

• class WrappedKey
  Interface for a key in a dask graph.

Functions

• ▶ def pack_data(o, d, key_types=object)

  Merge known data into tuple or dict

  source link

  Parameters

  • o :
    

    core data structures containing literals and keys

  • d : dict
    

    mapping of keys to data

  Examples

  >>> data = {'x': 1}
  >>> pack_data(('x', 'y'), data)
  (1, 'y')
  >>> pack_data({'a': 'x', 'b': 'y'}, data)  # doctest: +SKIP
  {'a': 1, 'b': 'y'}
  >>> pack_data({'a': ['x'], 'b': 'y'}, data)  # doctest: +SKIP
  {'a': [1], 'b': 'y'}

  Reexports

  • Imported in distributed.client.
  • Imported in distributed.recreate_exceptions.
  • Imported in distributed.tests.test_utils_comm.
  • Imported in distributed.worker.
• ▶ def subs_multiple(o, d)

  Perform substitutions on a tasks

  source link

  Parameters

  • o :
    

    Core data structures containing literals and keys

  • d : dict
    

    Mapping of keys to values

  Examples

  >>> dsk = {"a": (sum, ["x", 2])}
  >>> data = {"x": 1}
  >>> subs_multiple(dsk, data)  # doctest: +SKIP
  {'a': (sum, [1, 2])}

  Reexports

  • Imported in distributed.protocol.highlevelgraph.
  • Imported in distributed.tests.test_utils_comm.
• ▶ def unpack_remotedata(o, byte_keys=False, myset=None)

  Unpack WrappedKey objects from collection

  source link

  Returns original collection and set of all found WrappedKey objects

  Examples

  >>> rd = WrappedKey('mykey')
  >>> unpack_remotedata(1)
  (1, set())
  >>> unpack_remotedata(())
  ((), set())
  >>> unpack_remotedata(rd)
  ('mykey', {WrappedKey('mykey')})
  >>> unpack_remotedata([1, rd])
  ([1, 'mykey'], {WrappedKey('mykey')})
  >>> unpack_remotedata({1: rd})
  ({1: 'mykey'}, {WrappedKey('mykey')})
  >>> unpack_remotedata({1: [rd]})
  ({1: ['mykey']}, {WrappedKey('mykey')})
  
  Use the ``byte_keys=True`` keyword to force string keys
  
  >>> rd = WrappedKey(('x', 1))
  >>> unpack_remotedata(rd, byte_keys=True)
  ("('x', 1)", {WrappedKey('('x', 1)')})

  Reexports

  • Imported in distributed.client.
  • Imported in distributed.protocol.highlevelgraph.

Generated by nedoc v0.9 at 2020-12-29 14:05