SpotFlow

SpotFlow is a tool to ease the runtime analysis of Python programs. With SpotFlow, you can extract information about executed methods, run lines, argument values, return values, variable states, and thrown exceptions.

Install

git clone https://github.com/andrehora/spotflow.git
cd spotflow
pip install -e .

Quick examples

Monitoring a program

A simple code to be monitored in sample.py:

# content of sample.py
def absolute(x):
    if x < 0:
        x = -x
    return x

absolute(-10) # 10

To run and monitor function absolute in sample.py:

$ python -m spotflow -o -tt absolute sample

The result:

Running and monitoring: sample
============= Result =============
MonitoredProgram
- methods: 1
- calls: 1
MonitoredMethod
- name: absolute
- calls: 1
MethodCall
- distinct_run_lines: [2, 3, 4]
- run_lines: [2, 3, 4]
ArgState
- x: -10
ReturnState: 10

---

Another code to be monitored in sample.py, with two calls:

# content of sample.py
def sum(iter, start=0):
    count = start
    for i in iter:
        count += i
    return count

sum([1, 2]) # 3
sum([1, 2, 3], 1) # 7

To run and monitor function sum in sample.py:

$ python -m spotflow -o -tt sum sample

The result:

Running and monitoring: sample
============= Result =============
MonitoredProgram
- methods: 1
- calls: 2
MonitoredMethod
- name: sum
- calls: 2
MethodCall
- distinct_run_lines: [2, 3, 4, 5]
- run_lines: [2, 3, 4, 3, 4, 3, 5]
ArgState
- iter: [1, 2]
- start: 0
ReturnState: 3
MethodCall
- distinct_run_lines: [2, 3, 4, 5]
- run_lines: [2, 3, 4, 3, 4, 3, 4, 3, 5]
ArgState
- iter: [1, 2, 3]
- start: 1
ReturnState: 7

Monitoring a test suite

You can also monitor tests. For example, to monitor the test suite test_gzip of the Python Standard Library and print the most called methods of module gzip:

$ python -m spotflow -c -t gzip unittest test.test_gzip

The result (first lines):

Running and monitoring: unittest test.test_gzip
rank, method_name, number_of_calls
1, gzip._PaddedFile.read, 5781
2, gzip.GzipFile.readline, 1183
3, gzip.GzipFile.seek, 1108
...

Monitoring other tests of the Python Standard Library:

$ python -m spotflow -c -t ast unittest test.test_ast
$ python -m spotflow -c -t urllib unittest test.test_urlparse
$ python -m spotflow -c -t json unittest test.test_json
$ python -m spotflow -c -t calendar unittest test.test_calendar

Usage

SpotFlow can be run from the command-line. We can use SpotFlow to collect data from the execution of Python programs. For example, to run my_program.py, we could originally use the following command-line:

# Running a Python program
$ python -m my_program

The same program can be run (and monitored) under SpotFlow with following command-line:

# Running a Python program + SpotFlow
$ python -m spotflow -o -t <target> my_program

The argument -t represents the target entity to be monitored. We can pass the full name of the target method (in the format module.Class.method) or a prefix to monitor multiple methods. For example, we can pass the prefix parser (to monitor all methods of a specific module), parser.StringParser (to monitor all methods of a specific class), or the full method name (to monitor a single method). The argument -tt represents the short name of the target method (e.g., count). Use -tt to monitor functions defined in the program being run (like sample.py in the examples above), since their module is __main__. The final mandatory argument is the original command-line, which is in this case my_program.

SpotFlow only prints results when an output option is passed: -o (summary of the monitored objects), -c (summary of the calls), -p (pretty print the code), or -html (HTML report).

In a similar way, we can use SpotFlow to monitor the execution of test suites. For example, to run a test testX.py under SpotFlow, the following change would be needed with the unittest framework:

# Running unittest
$ python -m unittest testX

# Running unittest + SpotFlow
$ python -m spotflow -o -t <target> unittest testX

Monitored entities

• MonitoredProgram: This class is a repository of monitored methods, which can be used to access all collected data.

• MonitoredMethod: It represents a monitored method (or function). It has method calls and contains static information about the method/function, like name, full name, class name (for methods), file name, LOC, source code, etc.

• MethodCall: This class represents a method call that happens at runtime. It includes data about the caller, call stack, and executed lines. A method call also has a call state, which records information like argument and return states.

• CallState: This class holds the state of a method call, with information about argument states (ArgState), return states (ReturnState), thrown exceptions (ExceptionState), and yielded values (YieldState).

• State: It represents a state at runtime, such as an argument, return, yield, or exception state. All states have information about its runtime value, runtime type, and line number in code. In addition, ArgState has information about its name.