Obtaining Membership Functions from a Neuron Fuzzy System extended by Kohonen Network

03/29/2005 ∙ by Angelo Luis Pagliosa, et al. ∙ UDESC 0

This article presents the Neo-Fuzzy-Neuron Modified by Kohonen Network (NFN-MK), an hybrid computational model that combines fuzzy system technique and artificial neural networks. Its main task consists in the automatic generation of membership functions, in particular, triangle forms, aiming a dynamic modeling of a system. The model is tested by simulating real systems, here represented by a nonlinear mathematical function. Comparison with the results obtained by traditional neural networks, and correlated studies of neurofuzzy systems applied in system identification area, shows that the NFN-MK model has a similar performance, despite its greater simplicity.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

A traditional approach to Artificial Intelligence (AI) is known as

connectionism, and its represented by the field of Artificial Neural Network (ANN). A second approach to AI is the symbolic one, with its various branches, Fuzzy Logic (FL) among them. ANNs models offer the possibility of learning from input/output data and its functionality is inspired by biological neurons. Normally, ANNs require a relative long training time and cannot be described as a mechanism capable to explain how its results were obtained by training. Therefore, some projects involving ANNs can become complex, also lacking strong foundations. On the other hand, the FL framework deals with approximate reasoning typical from human minds. It allows the use of linguistic terminology and common sense knowledge. The main limitation of this system consists in the nonexistence of learning mechanisms, capable of generating fuzzy rules and membership functions, which depend on the specialist knowledge.

An interesting alternative consists of dealing with hybrid systems (HS), which use the advantages of both ANNs and FL. It can also employ the process model knowledge in order to decrease the duration of the project. In particular, HS are characterized by new architecture, learning methods, predefined parameters and knowledge representation, combining the fuzzy systems capacity of dealing with unprecise data and ANN ability in learning process by examples.

This work proposes a hybrid system, called Neo-Fuzzy-Neuron Modified by the Kohonen Network (NFN-MK), applied to the context of function identification. The Neo-Fuzzy-Neuron (NFN) was originally proposed by Uchino and Yamakawa uchino94 as an hybrid alternative model applied to real systems. The NFN-MK is an extension of NFN that uses a Kohonen Network to generate initial positions of triangular curves, that model the fuzzy neuron.

Ii Description of the Model

The NFK-MK model is applied here in the context of function approximations, with the objective of adjusting membership functions for a neurofuzzy system, using Kohonen self-organizing map. The neurofuzzy model used is the Neo-Fuzzy-Neuron (NFN) developed by Yamakawa

yamakawa94 ; uchino94 . This model was chosen for its short training time, compared to those typical from multilayer networks caminhas99 .

The input functions in the system are supposed to be unknown, except by samples of data. The problem consists in determining a system in which the process output and the model process become close together, according to a given criteria vas99 .

In general we suppose that the process block shown in figure 1 is nonlinear. In general this implies in difficulties for the mathematical modeling. The NFN-MK model can be advantageously used in cases like this, without the necessity of linearizing techniques, unsuitable for some inputs.

Figure 1: Block diagram for the training phase of NFN-MK

The proposed model consists in two main blocks as seen in figure 1, included in the dashed box. One of them is the original Yamakawa’s NFN yamakawa94 ; uchino94 . It works on triangular curves in its neuron fuzzy model, used for simplicity. In our model the NFN is extended by a classical application of Kohonen’s network kohonen98 . As shown in figure 1 there are two switches, used for the training process.

When switch is open and closed, the Kohonen network works, looking for the central vertices of the triangular membership functions. Initially these values are equally divided in seven fuzzy curves, which belong to the fuzzy neuron network. After this phase, the locations of the vertices are updated in the NFN block, where the base points are found by a new training.

The training on the NFN block occurs when is open and

is closed. The training proceeds like a backpropagation algorithm, finding the weights and base points of the triangular curves for each neuron. New points for these triangular curves represent rules such “

if-then” like original idea of NFN.

Here seven membership functions of triangular type are uniformly distributed in an interval

of input domain.

The values of the membership functions are based in the experiments of Shaw et al. shaw99 , which show that a change from five to seven triangular sets increases the precision in about 15%. There are no significative improvements for greater number of sets.

We note that equidistant membership functions may not be convenient in situations where there are concentration of patterns in some regions and dispersion in others shaw99 . One alternative is to deal with function non-uniformly distributed. The adjustment of the membership functions can be made using a grouping algorithm, like Kohonen networks kohonen84 ; kohonen98

. This is the main reason why we are using a very basic Kohonen network, that finds new positions of the vertices of triangular curves. In this method the network weights correspond to the values associated to the vertices of the triangular curves, and the number of neurons belonging to the processing layer corresponds to the number of fuzzy subsets for each NFN network input. The winner neuron is the one that corresponds to the shortest Euclidean distance from the input weight vector

kohonen84 ; kohonen98 .

Iii Experiment

A mathematical function that can be used as a benchmark is the Mexican hat, defined by

(1)

This function, shown in figure 2, represents the nonlinear system to be identified (cf. figure 1).

Here and are mapped to in the interval .

Figure 2: Mexican hat function used as nonlinear system

One reason for choosing this particular two-variable mathematical function for testing our system, is that the resulting 3D points can be easily visualized in a graphic. The points are used for training of Kohonen network and also for adjusting the weights in NFN (cf. figure 1).

Initially, the seven membership functions, showed in the figure 3, are equally distributed in the domain. The vertices as their limits, right and left, are complemented by three first columns in table 1.

Figure 3: Initial membership functions

The curves in figure 3 represent the initial state of neurons in NFN, which are seven for each input ( and ). The semantic values of these curves are summarized in table 2, where the notation is the usual one from fuzzy logic (FL) systems.

The next step consists in finding new vertices using a very basic Kohonen network. These new vertex values are presented in the table 1, (vertex column). This phase is computed with switch open and closed (cf. figure 1).

Once new vertices are found, the triangular curves are redrawn in according the clustering in each two curves. These curves are built to keep the convexity, in the sense that the summation is 1 for each two curves. For example, if , its neighbour curve in right side has the complement . By making these adjustments we are following the idea of clustering exhibited in the Kohonen network. Therefore, the new left and right limits for these triangular curves are found.

Since the function 2 is symmetric in the two variables, the vertex positions in the triangular curves are equally distributed. In this case, the NFN model presents fourteen fuzzy neurons, seven for each variable ( and ).

Figure 4: New membership functions adjusted by NFN-MK

By closing switch and opening (cf. figure 1), the adjustment of weights in the NFN model follows. The NFN model is trained with 225 patterns of input/output

equally distributed, during 10 epochs. The new results are presented in table

1, and the new values of the weights in NFN (cf. figure 5) are showed in last two columns of table 1. The names of the fuzzy curves are defined in table 2.

Fuzzy Curves Initial Values New Values
Left Vertex Right Left Vertex Right
NL -10.0 -10.0 -6.67 10.0 -10.0 -3.5 0.0715 -0.0643
NM -10.0 -6.67 -3.33 -10.0 -3.5 -0.2 -0.1414 0.0103
NS -6.67 -3.33 0.0 -3.5 -0.2 3.2 0.5154 0.4973
ZE -3.33 0.0 3.33 1 -0.2 3.2 6.5 -0.0824 0.0321
PS 0.0 3.33 6.67 3.2 6.5 10.0 0.0143 0.1714
PM 3.33 6.67 10.0 6.5 10.0 10.0 0.0739 -0.0463
PL 6.67 10.0 10.0 10.0 10.0 10.0 0.0 0.0358
Table 1: Parameters for the Mexican hat function
NL: Negative Large
NM: Negative Medium
NS: Negative Small
ZE: Zero
PS: Positive Small
PM: Positive Medium
PL: Positive Large
Table 2: Semantic meanings of fuzzy curves

The symbols and denote the average weights of NFN model, considering each fuzzy neuron already modified (cf. figure 5). Such values correspond the following vectors: and . Here , …, , ,…, , etc.

Figure 5: NFN model for two variables ()

The resulting sum gives

(2)

Expression (2) follows the original idea of Uchino and Yamakawa yamakawa94 ; uchino94 , where the membership functions are complementary. Thus, the indices and are associated to a desfuzzyfication into two complementary curves. The same idea is applied to and indices. A numerical evaluation of the equation (2), is given table 1. The graph in figure 4 can be easily computed for any pair of .

Iv Evaluation of the Performance

The NFN-MK is trained with 225 samples during 10 epochs, so the internal parameters become determined (neural weights and triangular curves of these neurons). After this some inputs are given. The results of these tests are shown in the figure 6. Here the points denoted by represent the simulation results. The continuous line represents the actual function.

Figure 6: Mexican hat function obtained by NFN-MK

The values for the training (225 samples and 10 epochs) as well as the model selected (Mexican hat function), were chosen in order to compare our results with those found elsewhere. The parameters chosen were provided in these references, except by the ANN model that uses a classical backpropagation algorithm with 2 neurons as input layer, 7 other in hidden layer, and 1 in output layer. The parameter considered is the number of mathematical operations necessary to evaluate a cycle (an epoch), for each neuron model in its respective architecture. As an simplification, the multiplication, sum and subtraction operations are associated to the same value. These results are shown in table 3. They show that the NFN-MK performance is equivalent to that of other models used to identify the same curve. Although our system presented a mean quadratic error (MQE) slightly greater than those obtained by NFHQ (Neuro-Fuzzy Hierarquic Quadtree) and FSOM (Fuzzy Self-Organized Map) models, it is much simpler in what concerns mathematical operations. On the other hand, the NFN-MK presented a much smaller MQE than the classical ANN.

Models Number of Operations (,,) Operations by function MQE
NFN-MK 8 2 0.0426
NFHQ 168 21 0.0150
FSOM 200 101 0.0314
NN 42 8 0.1037
Table 3: Model comparisions

V Conclusion

The objective of this work was to extend a NFN model that allowed the adjustment of membership functions of triangular type by the method of Kohonen. The result was the proposal of the Neo-Fuzzy-Neuron Modified by a Kohonen Network (NFN-MK). The model was successfully tested in function approximation, and its performance was similar to those obtained by more complex, classical neural networks.

The obtained results do not allow inferences about the generality of the NFN-MK model, but they seem to indicate that we have a viable model for system identification. What makes this model particularly interesting is the relatively reduced number of operations and function calculations involved, implying small processing times, when compared to other ANNs souza97 ; rissoli99 .

References

  • [1] W. Caminhas, H. Tavares, F. Gomide, and W. Pedrycz. Fuzzy set based neural networks: Structure, learning and aplication. 3:151–157, 1999.
  • [2] Flávio Joaquim de Souza. Modelos Neuro-Fuzzy Hierárquicos. PhD thesis, Pontifícia Universidade Católica - Rio de Janeiro, 1997.
  • [3] T. Kohonen. Self-organizing feature maps and abstractions. In I. Plander, editor, Artificial Intelligence and Information-Control Systems of Robots, pages 39–45. North-Holland, Amsterdam, 1984.
  • [4] Teuvo Kohonen. Self-organizing map. Neurocomputing, 21(1):1–6, 1998.
  • [5] Camargo H. A. Rissoli, V. R. V. and J. A. Fabri. Geração automática de regras a partir da arquitetura neuro-fuzzy para classificação de dados (nefclass). Simpósio Brasileiro de Automação Inteligente, 1999. (in Portuguese).
  • [6] I. S. Shaw and M. G. Simões. Controle e Modelagem Fuzzy. Edgard Blucher Ltda, São Paulo - SP - Brazil, 1999.
  • [7] E. Uchino and T. Yamakawa. Neo-fuzzy-neuron based new approach to system modeling, with application to actual system. In Proc. Sixth Int. Conf. on Tools with Artificial Intelligent, New Orleans, USA, pages 564–570, 1994.
  • [8] Peter Vas. Eletrical Machines And Drives. London: Oxford University Press, 1999.
  • [9] Takeshi Yamakawa. A neo fuzzy neuron and its application to system identification and prediction of chaotic behavior. In Computational Intelligence: Imitating Life, pages 383–395, 1994.