Module profile
This module contains utility functions to construct and manipulate counting data structures for frames.
When performing statistical profiling we obtain many call stacks. We aggregate
these call stacks into data structures that maintain counts of how many times
each function in that call stack has been called. Because these stacks will
overlap this aggregation counting structure forms a tree, such as is commonly
visualized by profiling tools.
We represent this tree as a nested dictionary with the following form:
{
'identifier': 'root',
'description': 'A long description of the line of code being run.',
'count': 10 # the number of times we have seen this line
'children': { # callers of this line. Recursive dicts
'ident-b': {'description': ...
'identifier': 'ident-a',
'count': ...
'children': {...}},
'ident-b': {'description': ...
'identifier': 'ident-b',
'count': ...
'children': {...}}}
}source linkFunctions
▶ def call_stack(frame) Create a call text stack from a frame
▶ def get_profile(history, recent=None, start=None, stop=None, key=None) Collect profile information from a sequence of profile states
▶ def identifier(frame) A string identifier from a frame
Strings are cheaper to use as indexes into dicts than tuples or dicts
Reexports
▶ def ll_get_stack(tid) Collect low level stack information from thread id
▶ def llprocess(frames, child, state) Add counts from low level profile information onto existing state
This uses the ``stacktrace`` module to collect low level stack trace
information and place it onto the given sttate.
It is configured with the ``distributed.worker.profile.low-level`` config
entry.
See also
process
ll_get_stack
Reexports
▶ def merge(*args) Merge multiple frame states together
▶ def plot_data(state, profile_interval=0.010) Convert a profile state into data useful by Bokeh
See also
plot_figure
distributed.bokeh.components.ProfilePlot
Reexports
▶ def plot_figure(data, **kwargs) Plot profile data using Bokeh
▶ def process(frame, child, state, stop=None, omit=None) Add counts from a frame stack onto existing state
This recursively adds counts to the existing state dictionary and creates
new entries for new functions.
Examples
>>> import sys, threading
>>> ident = threading.get_ident() # replace with your thread of interest
>>> frame = sys._current_frames()[ident]
>>> state = {'children': {}, 'count': 0, 'description': 'root',
... 'identifier': 'root'}
>>> process(frame, None, state)
>>> state
{'count': 1,
'identifier': 'root',
'description': 'root',
'children': {'...'}}Reexports
▷ def repr_frame(frame) Render a frame as a line for inclusion into a text traceback
▶ def watch(thread_id=None, interval="20ms", cycle="2s", maxlen=1000, omit=None, ...) Gather profile information on a particular thread
def watch(
thread_id=None,
interval="20ms",
cycle="2s",
maxlen=1000,
omit=None,
stop=lambda: False,
)
This starts a new thread to watch a particular thread and returns a deque
that holds periodic profile information.
Parameters
- thread_id : int
- interval : str
Time per sample
- cycle : str
Time per refreshing to a new profile state
- maxlen : int
Passed onto deque, maximum number of periods
- omit : str
Don't include entries that start with this filename
- stop : callable
Function to call to see if we should stop
Returns
Reexports