Parametric Shape Modeling and Skeleton Extraction with Radial Basis Functions using Similarity Domains Network

06/01/2019 ∙ by Sedat Ozer, et al. ∙ MIT 0

We demonstrate the use of similarity domains (SDs) for shape modeling and skeleton extraction. SDs are recently proposed and they can be utilized in a neural network framework to help us analyze shapes. SDs are modeled with radial basis functions with varying shape parameters in Similarity Domains Networks (SDNs). In this paper, we demonstrate how using SDN can first help us model a pixel-based image in terms of SDs and then demonstrate how those learned SDs can be used to extract the skeleton of a shape.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Recent advances in deep learning moved attention to the neural networks based solutions for shape understanding, shape analysis and parametric shape modeling. Radial basis networks (RBNs) are a particular set of neural networks using radial basis function (RBF) kernels and in this paper, we introduce a novel shape modeling algorithm based on RBNs. RBFs have been used in the literature for many classification tasks including the original LeNET architecture

[12]. While RBFs are useful in modeling surfaces and classification tasks as in [18, 11, 22, 5, 17, 15]

, there are many challenges associated with utilizing RBFs in neural networks for parametric shape modeling. Two of those challenges include: (I) estimating the optimal number of RBFs (e.g., the number of circles in our figures) to be used in the network along with their optimal center values, and (II) estimating the optimal RBF kernel parameters by relating them to shapes geometrically. The kernel parameters are typically known as the scale or the shape parameter (representing the radius of a circle in this paper) and used interchangeably in the literature. The standard RBNs as defined in

[13] applies the same kernel parameter to each and all basis functions used in the architecture. Recent literature focused on using multiple kernels with their own kernel parameters as in [9] and [1]. While the idea of utilizing different kernels with different parameters has been heavily studied in the literature under the ”Multiple Kernel Learning” (MKL) framework as formally modeled in [1], there are not many efficient approaches and available implementations focusing on utilizing multiple kernels with their own parameters in RBNs for shape modeling. Recently, the work in [16] combined the optimization advances achieved in the kernel machines domain with the radial basis networks and introduced a novel algorithm for shape analysis. In this paper, we call that algorithm as ”Similarity Domains Network” (SDN) and discuss its benefits from both shape analysis (see Figure 1

) and skeleton extraction perspectives. As we demonstrate in this paper, the computed SDs of SDN can be used to obtain both parametric models for shapes via its SDs and their skeletons without requiring large training samples.

(a) Binary input image
(b) Altered image using SDs
(c) Visualization of all the SDs
(d) Visualization of the (+) SDs
Figure 1: The use of the shape parameters of SDN on a binary image is shown. (a) Original binary image. (b) The altered image by utilizing the SDN’s shape parameters. Each object is scaled and shifted at different scales. We run a region growing algorithm to group the kernel parameters for each object and then individually scale and shift them. (c) All the computed shape parameters of the input image are visualized. (d) Visualization of the foreground parameters.

2 Related Work

In this paper, we propose using SDs for both parametric shape modeling and for extracting the skeleton. Our proposed algorithm: SDN is related to both RBNs and kernel machines. Skeleton extraction has been widely studied in the literature as in [7, 21, 20, 8]. However, in this paper, we mainly discuss and present our novel algorithm from the RBNs perspective. In the past, the RBN related research mostly focused on computing the optimal single kernel parameter (i.e., the scale or shape parameter) to be used in all of the RBFs used in the network as in [14, 4]. While the parameter computation for multiple kernels have been heavily studied under the MKL framework in the literature (for examples, see the survey papers: [6, 10]

), the computation of multiple kernel parameters in RBNs has been mostly studied under two main approaches: using optimization or using heuristic methods. For example, in

[3]

, the authors proposed using multiple scales as opposed to using a single scale value in RBNs. Their approach utilizes first computing the standard deviation of each cluster (after applying a k-means like clustering on the data) and then using a scaled version of those standard deviations of each cluster as the shape parameter for each RBF in the network. The work in

[2] also used a similar approach by using the root-mean-square-deviation (RMSD) value between the RBF centers and the data value for each RBF in the network. The authors used a modified orthogonal least squares (OLS) algorithm to select the RBF centers. The work in [9] used k-means algorithm on the training data to choose k centers and used those centers as RBF centers. Then it used separate optimizations for computing the kernel parameters and the kernel weights (see next chapter for the formal definitions). Using additional optimization steps for different set of parameters is costly and makes it harder to interpret those parameters and to relate them to shapes geometrically and accurately. As an alternative solution, the work in [16] proposed a geometric approach by using the distance between the data samples as a geometric constraint. In [16], the author did not use the well known MKL model. Instead, he defined interpretable similarity domains concept using RBFs and developed his own optimization approach with geometric constrains similar to the original Sequential Minimal Optimization (SMO) algorithm [19]. Consequently, the SDN algorithm combines both RBN and kernel machine concepts to develop a novel algorithm with geometrically interpretable kernel parameters. In this paper, we propose using SDN for parametric shape modeling and skeleton extraction. Unlike the existing work, instead of applying an initial k-means algorithm or OLS algorithm to compute the kernel centers separately or using multiple cost functions, SDN chooses the RBF centers and their numbers automatically via its sparse modeling and uses a single cost function to be optimized with its geometric constraint. That is where SDN differs from other similar RBN works as they would have issues on computing all those parameters within a single optimization step while automatically adjusting the number of RBFs used in the network sparsely.

Figure 2: An illustration of SDN as a radial basis network. The network contains a single hidden layer. The input layer (

dimensional input vector) is connected to

radial basis functions. The output is the weighted sum of the radial basis functions’ outputs.
(a) The input image
(b) All
(c) Foreground
Figure 3: Visualization of the SDM kernel parameters at = 0.05 with zero pixel error learning. The blue area represents the background and the yellow area represents the foreground. The red dots are the RBF centers and yellow circles around them show the boundaries of SDs. The green lines are the radiuses () of SDs. The are obtained from the computed . (a) Original image: 141x178 pixels. (b) Visualization of all the from both background and foreground with total of 1393 centers. (c) Visualization of only the for the object with total of 629 foreground centers (i.e., by using only the 2.51% of all image pixels). All images are resized to fit into the figure.

3 Similarity Domains Network

RBNs typically include a single hidden layer using radial basis functions as activation functions and the hidden layer uses

different RBFs. The illustration of SDN as a radial basis network is given in Figure 2. In the figure, the hidden layer uses all of the training data as an RBF center and then through the sparse optimization, it selects a subset of the training data (e.g., subset of pixels for shape modeling). SDN represents the decision boundary as a weighted combination of Similarity Domains (SDs). A Similarity Domain is a dimensional sphere in the dimensional feature space. Each similarity domain is centered at an RBF center and modeled with a Gaussian RBF in SDN. SDN estimates the label of a given input vector x as as shown below:

(1)

where the scalar is a nonzero weight fo the RBF center , the class label of the training data and the total number of RBF centers. (.) is the Gaussian RBF kernel defined as:

(2)

where is the shape parameter for the center . The centers are automatically selected among the training data during the training via the following cost function:

(3)

where is a constant value assuring that the RBF function yields a smaller value for any given pair of samples from different classes. The shape parameter is defined as . Further details on SDs and SDN formulation can be found in [16].

4 Parametric Shape Modeling with SDN

The Gaussian RBFs and their shape parameters can be used for parametric modeling of the shapes. For that, we can save and use only the foreground (the shape’s) centers and their shape parameters to obtain a one class classifier. The computed centers of SDN can be grouped as

and , where , is the total number of centers from the (+1) class and is the total number of centers from the (-1) class. Since the Gaussian kernel functions now represent local SDs geometrically, the original decision function can now be approximated by using only (or by using only ). Therefore, we define the one-class approximation by using only the centers and their associated kernel parameters from the for any given as follows:

(4)

where the SD radius for the center is defined as and is a domain specific constant. One class approximation examples are given in Figure 0(b) where we used only the SDs from the foreground to reconstruct the altered image.

(a)
(b)
(c)
(d)
(e)
(f) Skeleton for Fig. 3(a)
(g) Skeleton for Fig. 3(b)
(h) Skeleton for Fig. 3(c)
(i) Skeleton for Fig. 3(d)
(j) Skeleton for Fig. 3(e)
Figure 4: Visualization of the shape parameters (shown in the first row) after being quantized and thresholded at various values for the image shown in Figure 2(a) and their computed skeletons (shown in the second row). Visualization of the shape parameters: (a) for ; (b) for ; (c) for ; (d) for ; (e) for . Extracted skeletons (shown as a blue line) at each of those threshold values are visualized below each image.

5 Extracting the Skeleton from SDs

Once learned and computed by the SDN, the Similarity Domains (SDs) can be used to obtain a representation of a shape’s skeleton. For that purpose, we first bin the computed shape parameters () into bins (in our experiments is set to 10). Since typically the majority of the similarity domains lay around the object (or shape) boundary, they appear in small values. Eliminating them at first, gives us a lesser number of SDs to consider for skeleton extraction. After eliminating those small SDs and their computed parameters with a simple thresholding process, we connect the centers of the remaining SDs by tracing the overlapping SDs. In the case of remaining non-overlapping SDs, we connect the closest SDs.

6 Experiments

Here, we demonstrate how to use SDN for parametric shape learning from a given single input image. Since it is hard to model shapes with the standard RBNs, and since there is no good RBN implementation was available to us, we did not use any RBN network in our experiments. The standard RBNs (as discussed earlier) have many issues and many individual steps to compute the RBN parameters including the total number of RBF centers and finding the center values along with the computation of the shape parameters at those centers. However, comparison of kernel machines (SVM) and SDN on shape modeling was already studied in the literature before (see [16]). Therefore, in this section, we focus on parametric shape modeling and skeleton extraction from SDs by using SDNs. All the images are resized to fit into the figures.

6.1 Parametric Shape Modeling with SDs

We first demonstrate visualizing the computed shape parameters of SDN on a sample image in Figure 3. Figure 2(a) shows the original input image. We used each image pixel’s 2D coordinate as the training input, and its color (being black or white) as the training labels. SDN is trained at T=0.05. SDN learned and modeled the shape and reconstructed it with zero pixel error by using 1393 SDs. Pixel error is the total number of wrongly classified pixels in the image. Figure 2(b) visualizes all the computed shape parameters of the RBF centers of SDN as circles and Figure 2(c) visualizes the ones for the foreground only. The radius of a circle in all figures is computed as where . We found the value of through a heuristic search and noticed that 2.85 suffices for all the shape experiments that we had. There are total of 629 foreground RBF centers computed by SDN (only 2.51% of all the input image pixels).

6.2 Skeleton Extraction From the SDs

Next, we demonstrate the skeleton extraction from the computed similarity domains as a proof of concept. Extracting the skeleton from the SDs as opposed to extracting it from the pixels, simplifies the computations as SDs are only a small portion of the total number of pixels (reducing the search space). To extract the skeleton from the computed SDs, we first quantize the shape parameters of the object into 10 bins and then starting from the largest bin, we select the most useful bin value to threshold the shape parameters. The remaining SD centers are connected based on their overlapping similarity domains. If multiple SDs overlap inside the same SD, we look at their centers and we ignore the SDs whose centers fall within the same SD (accepted the original SD center). That is why some points are not considered as a part of the skeleton in Figure 4. First row in Figure 4 demonstrates the remaining SD centers and their radiuses at various thresholds. The second row in the figure visualizes the extracted skeletons (shown as a blue line) from the SDs as explained in Section 5. Another example is shown in Figure 5. The learned SDs are thresholded and the corresponding skeleton as extracted from the remaining SDs are visualized as a blue line.

Bin Center: 9.93 29.12 48.32 67.51 86.71 105.90 125.09 144.29 163.48 182.68
Total Counts: 591 18 7 3 2 4 0 0 1 3
Table 1: Bin centers for the quantized foreground shape parameters () and the total number of shape parameters that fall in each bin for the image in Fig. 2(a).
(a) Input Image
(b)
(c) for
Figure 5: Visualization of the skeleton (shown as blue line) extracted from SDs on another image. (a) Input image: 64 x 83 pixels. (b) Foreground SDs. (c) Skeleton for .

7 Conclusion

In this paper, we introduced how the computed SDs of the SDN algorithm can be used to extract skeleton from shapes for the first time as a proof of concept. Instead of using and processing all the pixels to extract the skeleton of a shape, we propose to use SDs (a subset of the pixels) to extract the skeleton. The RBF shape parameters of SDN are used to define SDs and they can be used to model a shape as described in Section 4 and as visualized in our experiments. While the presented skeleton extraction algorithm is a naive solution to demonstrate the use of SDs, future work will focus on presenting more elegant solutions to extract the skeleton from SDs. SDN is a novel classification algorithm and has potential in many shape analysis applications besides the skeleton extraction. A shape can be modeled parametrically by using SDNs via shape parameters and RBF centers. A further reduction in parameters can be obtained with one class classification approximation of SDN as shown in Eq. 4. SDN can parametrically model a given single shape without requiring or using large datasets.

Acknowledgement

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research.

References

  • [1] Francis R Bach, Gert RG Lanckriet, and Michael I Jordan. Multiple kernel learning, conic duality, and the smo algorithm. In

    Proceedings of the twenty-first international conference on Machine learning

    , page 6. ACM, 2004.
  • [2] Mohammad Bataineh and Timothy Marler. Neural network for regression problems with reduced training sets. Neural networks, 95:1–9, 2017.
  • [3] Nabil Benoudjit, Cédric Archambeau, Amaury Lendasse, John Aldo Lee, Michel Verleysen, et al.

    Width optimization of the gaussian kernels in radial basis function networks.

    In ESANN, volume 2, pages 425–432, 2002.
  • [4] Jafar Biazar and Mohammad Hosami. An interval for the shape parameter in radial basis function approximation. Applied Mathematics and Computation, 315:131–149, 2017.
  • [5] Mario Botsch and Leif Kobbelt. Real-time shape editing using radial basis functions. In Computer graphics forum, volume 24, pages 611–621. Blackwell Publishing, Inc Oxford, UK and Boston, USA, 2005.
  • [6] Serhat S Bucak, Rong Jin, and Anil K Jain. Multiple kernel learning for visual object recognition: A review. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 36(7):1354–1369, 2014.
  • [7] Nicu D Cornea, Deborah Silver, and Patrick Min. Curve-skeleton properties, applications, and algorithms. IEEE Transactions on Visualization & Computer Graphics, (3):530–548, 2007.
  • [8] Ilke Demir, Camilla Hahn, Kathryn Leonard, Geraldine Morin, Dana Rahbani, Athina Panotopoulou, Amelie Fondevilla, Elena Balashova, Bastien Durix, and Adam Kortylewski. SkelNetOn 2019 Dataset and Challenge on Deep Learning for Geometric Shape Understanding. arXiv e-prints, 2019.
  • [9] Lihua Fu, Meng Zhang, and Hongwei Li. Sparse rbf networks with multi-kernels. Neural processing letters, 32(3):235–247, 2010.
  • [10] Mehmet Gönen and Ethem Alpaydın. Multiple kernel learning algorithms. The Journal of Machine Learning Research, 12:2211–2268, 2011.
  • [11] Long Jiang, Shikui Chen, and Xiangmin Jiao. Parametric shape and topology optimization: A new level set approach based on cardinal basis functions. International Journal for Numerical Methods in Engineering, 114(1):66–87, 2018.
  • [12] Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  • [13] Richard P Lippmann. Pattern classification using neural networks. IEEE communications magazine, 27(11):47–50, 1989.
  • [14] Michael Mongillo. Choosing basis functions and shape parameters for radial basis function methods. SIAM undergraduate research online, 4(190-209):2–6, 2011.
  • [15] Sedat Ozer.

    On the classification performance of support vector machines using chebyshev kernel functions.

    Master’s Thesis, University of Massachusetts, Dartmouth, 2007.
  • [16] Sedat Ozer. Similarity domains machine for scale-invariant and sparse shape modeling. IEEE Transactions on Image Processing, 28(2):534–545, 2019.
  • [17] Sedat Ozer, Masoom A Haider, Deanna L Langer, Theodorus H van der Kwast, Andrew J Evans, Miles N Wernick, John Trachtenberg, and Imam Samil Yetik. Prostate cancer localization with multispectral mri based on relevance vector machines. In Biomedical Imaging: From Nano to Macro, 2009. ISBI’09. IEEE International Symposium on, pages 73–76. IEEE, 2009.
  • [18] Sedat Ozer, Deanna L Langer, Xin Liu, Masoom A Haider, Theodorus H van der Kwast, Andrew J Evans, Yongyi Yang, Miles N Wernick, and Imam S Yetik. Supervised and unsupervised methods for prostate cancer segmentation with multispectral mri. Medical physics, 37(4):1873–1883, 2010.
  • [19] John Platt. Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods support vector learning, 3, 1999.
  • [20] Punam K Saha, Gunilla Borgefors, and Gabriella Sanniti di Baja. A survey on skeletonization algorithms and their applications. Pattern Recognition Letters, 76:3–12, 2016.
  • [21] Hari Sundar, Deborah Silver, Nikhil Gagvani, and Sven Dickinson. Skeleton based shape matching and retrieval. In 2003 Shape Modeling International., pages 130–139. IEEE, 2003.
  • [22] Sung-Hoon Yoo, Sung-Kwun Oh, and Witold Pedrycz.

    Optimized face recognition algorithm using radial basis function neural networks and its practical applications.

    Neural Networks, 69:111–125, 2015.