PyCG: Practical Call Graph Generation in Python

02/28/2021
by   Vitalis Salis, et al.
0

Call graphs play an important role in different contexts, such as profiling and vulnerability propagation analysis. Generating call graphs in an efficient manner can be a challenging task when it comes to high-level languages that are modular and incorporate dynamic features and higher-order functions. Despite the language's popularity, there have been very few tools aiming to generate call graphs for Python programs. Worse, these tools suffer from several effectiveness issues that limit their practicality in realistic programs. We propose a pragmatic, static approach for call graph generation in Python. We compute all assignment relations between program identifiers of functions, variables, classes, and modules through an inter-procedural analysis. Based on these assignment relations, we produce the resulting call graph by resolving all calls to potentially invoked functions. Notably, the underlying analysis is designed to be efficient and scalable, handling several Python features, such as modules, generators, function closures, and multiple inheritance. We have evaluated our prototype implementation, which we call PyCG, using two benchmarks: a micro-benchmark suite containing small Python programs and a set of macro-benchmarks with several popular real-world Python packages. Our results indicate that PyCG can efficiently handle thousands of lines of code in less than a second (0.38 seconds for 1k LoC on average). Further, it outperforms the state-of-the-art for Python in both precision and recall: PyCG achieves high rates of precision  99.2 we demonstrate how PyCG can aid dependency impact analysis by showcasing a potential enhancement to GitHub's "security advisory" notification service using a real-world example.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

04/13/2022

Timeloops: Automatic System Call Policy Learning for Containerized Microservices

In this paper we introduce Timeloops a novel technique for automatically...
08/19/2021

Svar: A Tiny C++ Header Brings Unified Interface for Multiple programming Languages

There are numerous types of programming languages developed in the last ...
02/21/2020

Graph4Code: A Machine Interpretable Knowledge Graph for Code

Knowledge graphs have proven to be extremely useful in powering diverse ...
07/27/2021

A Large-Scale Security-Oriented Static Analysis of Python Packages in PyPI

Different security issues are a common problem for open source packages ...
03/28/2021

Scalable Call Graph Constructor for Maven

As a rich source of data, Call Graphs are used for various applications ...
02/20/2019

Optimizing and Evaluating Transient Gradual Typing

Gradual typing enables programmers to combine static and dynamic typing ...
05/27/2019

Detecting Missing Dependencies and Notifiers in Puppet Programs

Puppet is a popular computer system configuration management tool. It pr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.