A Python library for alpha matting
An important step of many image editing tasks is to extract specific objects from an image in order to place them in a scene of a movie or compose them onto another background. Alpha matting describes the problem of separating the objects in the foreground from the background of an image given only a rough sketch. We introduce the PyMatting package for Python which implements various approaches to solve the alpha matting problem. Our toolbox is also able to extract the foreground of an image given the alpha matte. The implementation aims to be computationally efficient and easy to use. The source code of PyMatting is available under an open-source license at https://github.com/pymatting/pymatting.READ FULL TEXT VIEW PDF
This paper presents the philosophy, design and feature-set of Neural Net...
Pykg2vec is an open-source Python library for learning the representatio...
Analyzing the dynamical properties of mobile objects requires to extract...
The elementary transform sequence (ETS) provides a universal method of
R (Version 3.5.1 patched) has an issue with its random sampling
Replacing the background and simultaneously adjusting foreground objects...
AtomAI is an open-source software package bridging instrument-specific P...
A Python library for alpha matting
For an image with foreground pixels and background pixels , alpha matting asks to determine opacities , such that the equality
holds for every pixel
. This problem is ill-posed since, for each pixel, we have three equations (one for each color channel) with seven unknown variables. The implemented methods rely on a trimap, which is a rough classification of the input image into foreground, background and unknown pixels, to further constrain the problem. Giving the alpha matte, foreground estimation aims to extract the foregroundfrom image .
Levin et al. (2007) show that assuming local smoothness of pixel colors yields a closed-form solution to the alpha matting problem.
He et al. (2010) propose an efficient algorithm based on a large kernel matting Laplacian. They show that the computational complexity of their method is independent of the kernel size.
For given , the foreground pixels can be determined by making additional smoothness assumptions on and . Our toolbox implements the foreground estimation by Levin et al. (2007).
Furthermore, the PyMatting toolbox implements a novel multi-level approach for foreground estimation. For this method our toolbox also provides GPU implementations for OpenCL and CUDA.
The following code snippet demonstrates the usage of the library:
from pymatting import *
image = load_image( " plant_ image. png" , " RGB" )
trimap = load_image( " plant_ trimap. png" , " GRAY" )
alpha = estimate_alpha_cf( image, trimap)
foreground = estimate_foreground_cf( image, alpha)
cutout = stack_images( foreground, alpha)
save_image( " results. png" , cutout)
The estimate_alpha_cf method implements closed-form alpha estimation (Levin et al., 2007), whereas the estimate_foreground_cf method implements the closed-form foreground estimation (Levin et al., 2007). The method stack_images can be used to compose the foreground onto a new background.
More code examples at different levels of abstraction can be found in the documentation of the toolbox. PyMatting can be easily installed through pip.
Since all of the considered methods require to solve a large sparse systems of linear equations, an efficient solver is crucial for good performance. Therefore, the PyMatting package implements the conjugate gradients method (Hestenes et al., 1952) together with different preconditioners that improve convergence: Jacobi, V-cycle (Lee and Wu, 2014) and thresholded incomplete Cholesky decomposition (Jones and Plassmann, 1995).
To evaluate the performance of our implementation, we calculate the mean squared error on the unknown pixels of the benchmark images of Rhemann et al. (2009). Figure 2 shows the mean squared error to the ground truth alpha matte. Our results are consistent with the results achieved by the authors implementation (if available).
We compare the computational runtime of our solver with other solvers: PyAMG (Olson and Schroder, 2018), UMFPACK (Davis, 2004), AMGCL (Demidov, 2019), MUMPS (Amestoy et al., 2001, 2006), Eigen (Guennebaud et al., 2010) and SuperLU (Li et al., 1999). Figure 4 shows that our implemented conjugate gradients method in combination with the incomplete Cholesky decomposition preconditioner outperforms the other methods by a large margin. For the iterative solver, we use an absolute tolerance of , which we scale with the number of known pixels, i.e. pixels that are either marked as foreground or background in the trimap. The benchmarked linear system arises from the matting Laplacian by Levin et al. (2007). Figure 3 shows that our solver also outperforms the other solvers in terms of memory usage.
The PyMatting package has been tested on Windows 10, Ubuntu 16.04 and macOS 10.15.2. The package can be easily extended by adding new definitions of the graph Laplacian matrix. We plan on continuously extending our toolbox with new methods.