Stencils have long been used for decorating walls, fabrics, and even food. Application of stencils involves placing a thin sheet against a flat surface, and spray-painting through the holes in the sheet. These stencils have been generated by hand as well as using image editing applications like Adobe Photoshop. Recently, the generation of stencils has been automated [Jain et al. 2015]. By segmenting images into piecewise constant layers using the Graph Cut Algorithm and fixing the topology [Chen et al. 2011], an arbitrary image can be decomposed into a series of stencils, with one color assigned per layer. We implement this system in MATLAB (see Figure 1). We observe that such stencils are limited to hard-boundaries and only a finite number of colors; the final composite cannot capture the millions of colors present in the input image.
Here, we propose using 3-D printed stencils to produce richer composites more faithful to the original image. Our stencils can reproduce the spectrum of values, as opposed to only or . Elevation of the stencil away from the surface enables a blend between holes drilled into the stencil.
2 Procedure & Results
Our method takes as input an image and outputs a sequence of 3D printed stencils. We first decompose the input image into a user-specified number of layers. We then produce stipplings of each layer. We finally engineer a 3D model corresponding to each stippling layer. Our contribution also includes a spray-paint simulator that will predict the effects of spray-painting through an arbitrary stencil.
2.1 Image Decomposition
We outline our method for decomposing images into layers, that when alpha blended, produce a composite as close to the original as possible. We rely on the Porter-Duff over operator for our compositions [Porter and Duff 1984], defined as:
where there are layers, is the composite pixel color after layers, is the color associated with layer , and is the opacity at pixel and layer . We also compute a background color for where at every pixel. This background layer can be interpreted as ”priming” the surface.
The image decomposition problem aims to compute layers of values and colors associated with these layers. We define an energy function that takes into account the attributes we’d like to be true of a solution.
Data Term We minimize the difference between composite and input pixel value as follows:
where is the input image, is the composite, and there are pixels. The three terms correspond with three color channels, .
Smoothness Term Our smoothness term aims for smoothly varying transparency values in each layer. Since we are blending adjacent stippling dots, local sharp changes in transparency are difficult to reproduce. Let be the computed channels. We define our smoothness term as follows:
where refers to the eight neighbors of pixel .
Sparsity Term We begin with the presupposition that it is easier to replicate painted regions with or , and there will be more error associated with intermediate values. We introduce a sparsity term to drive transparency values to these extremes by penalizing intermediate values. Driving the sum of the values among the top layers to 1, independently for each pixel, achieves this goal.
We then set up a convex optimization problem, solving for layers sequentially, then solving for all the colors at once, and repeating until convergence. If we let , and tune the relative strength of the energy terms, the final objective function is:
Given a target number of layers, K-means and quantization are valid methods for generating initial color guesses for each layer. Then, we can assign each pixel to the nearest color among thechoices to initialize the layers. These layers and colors are then modified by our sequential quadratic programming with blocked coordinate descent method.
Stippling is an artistic technique whereby varying degrees of shading are approximated by small dots. A common approach to generating quality stippling involves employing Lloyd’s relaxation on a weighted centroidal Voronoi tessellation [Hertzmann 2003]. We implement such a method here. To initialize the seed points, we use importance sampling weighted by opacity. Importance sampling, unlike dithering, avoids aliasing artifacts that Lloyd’s relaxation cannot always fix.
2.3 Spray-Paint Simulator
Previous work shows that the distribution of ink expelled from a spray-paint nozzle is approximately Gaussian [Prévost et al. 2015]
. We treat each point in the CVT as a 2-D Gaussian with its center as the mean, and variance given by. We fix the radius to be , a reasonable lower bound on the smallest hole that can be drilled through a 3-D printed material [Shapeways.com 2015]. We treat , the height the nozzle lies away from the material, as a tunable parameter. For simplicity, is the same for all dots in our simulations. Tuning the Gaussian prefactor simulates the effects of time; the greater the coefficient, the more time spent painting through each hole.
Let be the set of Gaussians, where there are dots. A successful simulation must take into account that the intensity of ink at any pixel is capped at ; adding more dots near a pixel or spray-painting for more time cannot push the maximum opacity value above
. These bounds on the pixel intensity motivate using complementary probability for our simulation. We produce the simulated result as follows:
where means evaluating the height of Gaussian at pixel site . We apply equation 6 on all pixels to compute the predicted spray-painted result.
2.4 3D Model
Starting from our stippled layer, we extrude a stencil and surrounding wall with tunable height [Jacobson et al. 2015]. Functionally, the wall elevates the stencil above the surface; it also prevents unintended paint from leaving the target region. The output of this step is a 3D-printable mesh.
We have presented a pipeline to bring 3D printing to surface decoration. Future work would include printing and spray-painting the physical stencils, and comparing the composites with the input and simulated images. We also expect that additional work on the stippling algorithm would result in a more faithful simulated result.
The author is grateful to Alec Jacobson and Changxi Zheng for their generous mentorship, as well as Eitan Grinspun for helpful conversations.
This work was completed 05/15 - 12/15 and programmed in MATLAB.
- [Chen et al. 2011] Chen, C., Freedman, D., and Lampert, C. H. 2011. Enforcing topological constraints in random field image segmentation. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, IEEE, 2089–2096.
- [Hertzmann 2003] Hertzmann, A. 2003. A survey of stroke-based rendering. IEEE Computer Graphics and Applications, 4, 70–81.
- [Jacobson et al. 2015] Jacobson, A., et al., 2015. gptoolbox: Geometry processing toolbox. http://github.com/alecjacobson/gptoolbox.
- [Jain et al. 2015] Jain, A., Chen, C., Thormählen, T., Metaxas, D., and Seidel, H.-P. 2015. Multi-layer stencil creation from images. Computers & Graphics 48, 11–22.
- [Porter and Duff 1984] Porter, T., and Duff, T. 1984. Compositing digital images. In ACM Siggraph Computer Graphics, vol. 18, ACM, 253–259.
- [Prévost et al. 2015] Prévost, R., Jacobson, A., Jarosz, W., and Sorkine-Hornung, O. 2015. Large-scale painting of photographs by interactive optimization. Computer & Graphics (to appear) XX, X (Dec.).
- [Shapeways.com 2015] Shapeways.com, 2015. Design rules and detail resolution for sls 3d printing - shapeways.