Copyright by ShunFeng Su Decomposed Fuzzy System Offered
®Copyright by Shun-Feng Su Decomposed Fuzzy System Offered by Shun-Feng Su, E-mail: sfsu@mail. ntust. edu. tw Department of Electrical Engineering, National Taiwan University of Science and Technology Nov, 2018 • 1
®Copyright by Shun-Feng Su Preface l It is usually necessary to model a system for various purposes (prediction, control, etc. ) l In traditional modeling, model structures must be defined first and modeling can be said to be parameter identification. l Neural networks or fuzzy systems have been proved to be universal approximators, which means they can model virtually any system within any accuracy. Thus, they have been pervasively employed to • 2 model different systems.
®Copyright by Shun-Feng Su Preface l. A very important advantage of using neural networks or fuzzy systems for modeling is that they can construct systems directly from the input-output relationship (data) without the use of any pre-described model or structure. l They are often referred to as model-free estimators. l Thus, they can used to model very complicated or unstructured systems, like stock trends, financial models, social • 3 behaviors, etc.
®Copyright by Shun-Feng Su Preface l Before deep learning, there almost all fuzzy systems (hardly see any neural networks for modeling) owing to learning efficiency. l For the same modeling task, usually neural networks need more epochs to converge (maybe 10~100 times) compared to that of using fuzzy systems and sometimes, may not converge at all. (for learning– as simple as possible principle) If learning is simple, the effects will also be nice in all aspects. But, if learning becomes complicated, there will • 4 generate various problems.
®Copyright by Shun-Feng Su Preface l One main characteristics of neural networks is recognition without definition. With this property, deep learning becomes meaningful. l In order to facilitate such characteristics, the learning in neural network structure is to distribute one piece of knowledge into all neurons (distributed knowledge representation). l But, such a learning behavior certainly violate the so-called minimum distribution principle and then the learning is very inefficient. • 5
®Copyright by Shun-Feng Su Preface The minimum disturbance principle [M 1, M 2] suggests that “a better way of learning should be aimed at not only reducing the output error for the current training pattern but also minimizing disturbance to the weights having already learned. ” [M 1] D. E. Rumelhart, G. E. Hinton and R. J. Williams, “Learning internal representations by error propagation”, Parallel Distribution Processing, Vol. 1, pp. 318 -362, D. E. Rumelhart and J. L. Mc. Clelland, Eds Cambrige, MA: MIT Press, 1986. [M 2] J. S. R Jang, C. -T. Sun and E. Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Prentice-Hall, 1997. • 6
®Copyright by Shun-Feng Su Preface l Another often-used system--Neural fuzzy system (NFS) is a fuzzy system equipped with the learning capability of neural networks (i. e. backpropagation learning). l In the literature, various NFSs have been proposed. • • • C. T. Lin and C. S. G. Lee, “Neural-network-based fuzzy logic control and decision systems, ” IEEE Trans. On Computers, vol. 40, no. 12, pp. 1320 -1336, 1991. J. S. R. Jang, “Adaptive-network-based fuzzy inference systems, ” IEEE Trans. on Systems, Man, Cybernetics, vol. 23, no. 3, pp. 665 -685, 1993. C. F. Juang and C. T. Lin, “An On-Line Self-Constructing Neural Fuzzy Inference Network and Its Applications, ” IEEE Trans. on Fuzz Systems, vol. 6, no. 1, pp. 12 -32, 1998. • 7
®Copyright by Shun-Feng Su Preface l. A lot of modifications about different structures, adding recurrent terms, different clustering techniques in dynamical structure formations, etc. can be found in the literature. l However, most approaches have complicated structures and then even may result in nice approximate capability, they may increase learning burden and cause ineffective learning. Such phenomena can be found in our experiments • 8
®Copyright by Shun-Feng Su Preface l In this talk, I will introduce a novel fuzzy structure -- Decomposed fuzzy system (DFS). l It can be found that the structure of the DFS not only can result in effective learning but also shows better approximate capability in modeling dynamic systems compared to those existing fuzzy systems. l DFS can also have very good control and learning performance for adaptive fuzzy control. • 9
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Ideas of Fuzzy Systems of Decomposed Fuzzy Systems l Simplified DFS l Simulation and Discussions l Conclusive remarks • 1 0
®Copyright by Shun-Feng Su Introduction of Fuzzy have been widely used in various applications. Fuzzy (or more precisely fuzzy set) is a way of uncertainty representation. Uncertainties always exist in various occasions. In addition to common-used nominal values for uncertainty entities, uncertainty representation (like fuzzy) provide an extra information for the information process. • 1 1
®Copyright by Shun-Feng Su Introduction of Fuzzy Knowledge based systems usually is constructed by rules (under what conditions, what need to be done. ) How about the current situation is between rules? A little bit close to or far way from the boundary? It should have different results. Fuzzy set can be employed to describe this membership belonging. In fact, fuzzy can also be • 1 used to model different 2 uncertainties.
®Copyright by Shun-Feng Su Introduction of Fuzzy Thus, it can be said that the fundamental idea behind fuzzy systems is to include uncertainty in the process. Such an inclusion provides extra information so that the systems can be more accurate. In other words, fuzzy is vagueness by meaning, but can provides accurate due to this extra information. Fuzzy • 1 3
®Copyright by Shun-Feng Su Introduction of Fuzzy To facilitate such processes, uncertainty reasoning is needed. It is well known that reasoning with fuzzy is not mathematical sound. In other words, the reasoning process is kind of subjective and not so accurate. (A counter case is probability, reasoning with Bayesian network) Nevertheless, fuzzy reasoning can work. Thus, it • 1 over that without can provide advantages 4 uncertainties.
®Copyright by Shun-Feng Su Fuzzy Logic Control A Fuzzy Logic Controller (FLC) is a controller described by a collection of fuzzy rules (e. g. IFTHEN rules) involving linguistic variables. A rule base system. The basic idea behind the use of Fuzzy Logic Control is to incorporate the “expert experience” of a human operator into the design of the controller in controlling a process. The utilization of linguistic variables, fuzzy control rules and approximate reasoning provides a • 1 means to incorporate human expert experience in designing the controller. 5
®Copyright by Shun-Feng Su Fuzzy Logic Control (FLC) is Fuzzy – to have fuzzy representation in dealing with the membership belonging among rules. Logic – to have logic reasoning for rule based systems. Control – to have the fuzzy system as a controller. It is nothing to do with fuzzy logic. Nowadays, people like to use fuzzy control instead of fuzzy logic control. • 1 6
®Copyright by Shun-Feng Su Fuzzy Logic Control The consequences of all matched rules must be transformed into actions. o use rules, a value ust be defined into labels. owledge usually is a rule structure d rule structures need partition. • 1 7 It is also referred to as a fuzzy system. • Feb. , 2011
®Copyright by Shun-Feng Su Fuzzy Logic Control Fuzzy logic control and fuzzy modeling both are to construct fuzzy rules. Fuzzy logic control is to provide control laws for systems to be controlled. Fuzzy modeling is to model a system by fuzzy rules. In fact, the construction of any fuzzy rule system (including FLC) can be said to be fuzzy • 1 be done by learning. modeling and usually can 8
®Copyright by Shun-Feng Su Rationale behind FLC use different control strategies for different situations compared to only one complicated controller for all situations in traditional control. The fuzzy mechanism provides the interpreting capability among rules and then with the interpreting capability, the transition between rules is gradual rather than abrupt (to avoid switch control problem). It is the so-called softening process. • 1 9
®Copyright by Shun-Feng Su Fuzzy Systems In recent development, fuzzy systems have been considered as an alternative of a nonlinear system but with a linear system in each rule so that approaches for linear systems can also be applied. Besides, various parameters are needed in fuzzy systems. Those parameters can be tuned to have excellent performance (by users or by learning mechanisms). adaptive fuzzy control • 2 neural fuzzy systems How to construct those fuzzy rules 0
®Copyright by Shun-Feng Su Fuzzy Systems Mamdani fuzzy rules : If (X is A) and (Y is B) … then (Z is C) TSK (in modeling) or TS (in control) fuzzy rules : If (X is A) and (Y is B) … then Z=f(X, Y). Note that C is a fuzzy set and f() is a crisp function. Mamdani type is hardly seen in control or modeling related applications. • Most are TS fuzzy rules. 2 1
®Copyright by Shun-Feng Su Fuzzy Systems Mamdani fuzzy rules : COA defuzzification To find the center of area, it need to use numerical integration. • 2 Difficult to take derivative 2 for the process and derivative • Feb. , 2011 is a common approach in learning.
®Copyright by Shun-Feng Su Fuzzy Systems TS fuzzy rules : Somewhat is also called COA. But without numerical integration. It is obtained as z= , Simple and easy to calculate. Most importantly, it can be used in any mathematical operations, such as derivative. where and are the firing strength and the fired result for the i-th rule and m is the rule • 2 number. 3
®Copyright by Shun-Feng Su Fuzzy System A fuzzy approximator is constructed by a set of fuzzy rules as Generally, is a fuzzy singleton (TS fuzzy model). Another type is to use a linear combination of input variables. In that case, usually, the recursive least square (RLS) approach (or recursive Kalman filter) can be used to identify those • 2 coefficients. 4
®Copyright by Shun-Feng Su Fuzzy System The fuzzy systems with the center-of area like defuzzification and product inference can be obtained as t-norm operation for all premise parts It is a universal function approximator and is written as. • 2 5
®Copyright by Shun-Feng Su Fuzzy System It should be noted that the above system is a nonlinear system. But, it can be seen that the form is virtually linear. ( ) Thus, various approaches have been proposed to handle nonlinear systems by using the linear system techniques for the linear property bearing in each rule, such as common P stability or LMI design process. • 2 6
®Copyright by Shun-Feng Su Fuzzy Modelling But how to define those fuzzy rules: Leaning from data How to define those values for ? Furthermore, are those fuzzy membership functions good enough or can they be tuned Nonlinear properties in in the learning process. those parameters As mentioned, it is generally called fuzzy • 2 modeling. 7
®Copyright by Shun-Feng Su Fuzzy Modeling l Neural fuzzy system (NFS) can be said to be a fuzzy system equipped with the learning capability of neural networks (backpropagation). l Backpropagation is to use gradient-decent like approaches for optimization. l In only for per example not for all data. the literature, various NFS structures have been proposed. However, most approaches have complicated structures and can result in nice approximate capability, • but 2 may increase learning burden and cause 8 ineffective learning.
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Ideas of Fuzzy Systems of Decomposed Fuzzy Systems l Simplified DFS l Simulation and Discussions l Conclusive remarks • 2 9
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems The idea is to use a fuzzy system (called component fuzzy systems) to represent a fuzzy rule in the traditional fuzzy structure. The original design concept is to use the parameter adaptation in a component fuzzy system to facilitate the tuning effects in a fuzzy rule. The main advantage is that all components fuzzy systems are independent. Thus, the minimum disturbance principle can be further fulfilled in DFS. • 3 0
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems Consider variation of a fuzzy set By using a set of rules to describe those variations • 3 1
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems In the structure, fuzzy variables are decomposed into several layers to have satisfactory learning performances upon the sense of minimum disturbance. To construct the DFS, the fuzzy variable is decomposed into layers and each layer corresponds to one fuzzy set in the original fuzzy partition except those two boundary fuzzy sets. • 3 2
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems • 3 3
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems Conceptually, the fuzzy partition of each layer is formed by one corresponding fuzzy set and its complement. In this study, in order to distinct those regions that are greater and less than the corresponding fuzzy set, the complement of a given fuzzy set is separated into two complement fuzzy sets. • 3 4
®Copyright by Shun-Feng Su Traditional Fuzzy Systems One fuzzy rule Learning on this rule will have • 3 great influences on its neighbor 5 rules.
®Copyright by Shun-Feng Su Traditional Fuzzy Systems One component fuzzy system (9 rules) Less correlation among component systems Another component fuzzy system (9 rules) • 3 6
• ®Copyright by Shun-Feng Su Decomposed Fuzzy Systems It can be found that the total fuzzy rule number of a DFS is large; i. e. . Nevertheless, those component fuzzy systems are independent and operate in an independent manner. Thus, it can be expected that in the learning process, there are no strong coupling effects among those component fuzzy systems and then, the learning speed of the DFS is still fast. • 3 7
• ®Copyright by Shun-Feng Su Decomposed Fuzzy Systems It can be found that for there are fuzzy component systems, each of which has 9 rules. It can be observed that each component system is a fuzzy system. Then, the average of the outputs of all component fuzzy systems is considered as the output of the DFS. Thus, the output of the DFS is calculated as • 3 Equal share for all component systems 8
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Ideas of Fuzzy Systems of Decomposed Fuzzy Systems l Simplified DFS l Simulation and Discussions l Conclusive remarks • 3 9
®Copyright by Shun-Feng Su Decomposed Fuzzy Systems A fuzzy system with a large fuzzy rule number can still result in a heavy computational burden. Thus, when the DFS is employed in real-time systems, the system may not have sufficient time to finish all calculations in one sampling time period. In order to satisfy such a real-time requirement, a combination concept similar to that of CMAC is proposed to design the • 4 so-called simplified DFS. 0
®Copyright by Shun-Feng Su Simplified DFS The concept of the simplified DFS structure is that each component fuzzy system in fact can still model the whole system domain and thus, for each layer, the simplified DFS only needs to have one component fuzzy system. Like CMAC, with overlapping, only one cell for one layer is needed. • 4 1
®Copyright by Shun-Feng Su Traditional Fuzzy Systems One component fuzzy already cover the whole universal disclosure For each set, only one component system is sufficient • 4 2
®Copyright by Shun-Feng Su Simplified DFS It is easy to see that the number of the component fuzzy systems is instead of and the total fuzzy rule number becomes. In our current implementation, only fuzzy sets from the same ordering layers of different fuzzy variables are combined to form component fuzzy systems for the simplified DFS. • 4 3
®Copyright by Shun-Feng Su Simplified DFS • 4 4
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Ideas of Fuzzy Systems of Decomposed Fuzzy Systems l Simplified DFS l Simulation and Discussions (modeling) l Conclusive remarks • 4 5
®Copyright by Shun-Feng Su DFS with BP learning • 4 • Overall scheme of the back propagation decomposed fuzzy systems 6
®Copyright by Shun-Feng Su DFS with BP learning • 4 7 for Henon chaotic sequence RMSE histories of all fuzzy systems
®Copyright by Shun-Feng Su DFS with BP learning Models BPFS FRBFNN RFRBFNN BPDFS Rules 25 25 25 81 Epoch 438 1623 1675 122 Training RMSE 3. 11 e-002 3. 05 e-002 3. 22 e-002 2. 75 e-002 Testing RMSE 3. 23 e-002 3. 07 e-002 3. 23 e-002 3. 06 e-002 • 4 8 used for chaotic sequence Performances of all fuzzy systems
®Copyright by Shun-Feng Su DFS with BP learning Models BPFS FRBFNN RFRBFNN BPDFS Rules 81 81 Epoch Training RMSE 174 337 1634 122 3. 11 e-002 3. 34 e-002 2. 77 e-002 2. 75 e-002 3. 5 e-002 3. 26 e-002 3. 95 e-002 3. 06 e-002 Testing RMSE • 4 with the same number of rules Performances of all fuzzy systems 9
®Copyright by Shun-Feng Su DFS with BP learning Models BPFS FRBFNN RFRBFNN BPDFS Rules 81 81 Epoch Training RMSE 174 337 1634 122 3. 11 e-002 3. 34 e-002 2. 77 e-002 2. 75 e-002 3. 5 e-002 3. 26 e-002 3. 95 e-002 3. 06 e-002 Testing RMSE • 5 Note that when the rule number increases, the 0 performance may be degraded.
®Copyright by Shun-Feng Su DFS with BP learning Models Rules Epoch Training RMSE Testing RMSE BPDFS (Nd = 3) 81 122 2. 75 e-002 3. 06 e-002 BPDFS(Nd =5) 225 160 2. 16 e-002 2. 34 e-002 Performances of BPDFS for Example 1 by using Nd =3 and Nd =5 In DFS, due to good learning efficient, the rule number increases, but the performance becomes better. Also the epoch number does not increase much.
®Copyright by Shun-Feng Su DFS with BP learning • 5 RMSE histories of all fuzzy 2 systems for a nonlinear model
®Copyright by Shun-Feng Su DFS with BP learning Models BPFS FRBFNN RFRBFNN BPDFS Rules Epoch 125 482 125 193 125 563 729 89 3. 83 e-002 3. 31 e-002 3. 01 e-002 2. 01 e-002 1. 34 e-002 1. 07 e-002 Training 3. 48 e-002 RMSE Testing RMSE 1. 79 e-002 • 5 3 used for a nonlinear model Performances of all fuzzy systems
®Copyright by Shun-Feng Su DFS with BP learning Models BPFS FRBFNN RFRBFNN BPDFS Rules 729 729 Epoch 162 324 245 89 Training 3. 28 e-002 RMSE 3. 53 e-002 3. 84 e-002 3. 01 e-002 Testing RMSE 1. 72 e-002 2. 06 e-002 1. 07 e-002 2. 2 e-002 • 5 with the same number of rules Performances of all fuzzy systems 4
®Copyright by Shun-Feng Su DFS with BP learning Models Rules Epoch Training RMSE Testing RMSE BPDFS (Nd =3) 729 89 3. 01 e-002 1. 07 e-002 BPDFS(Nd =5) 3375 87 3. 07 e-002 1. 11 e-002 Performances of BPDFS for Example 1 by using Nd =3 and Nd =5 In DFS, due to good learning efficient, when the rule number increases, the performance is almost the same and the epoch number becomes less.
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Simplified of Decomposed Fuzzy Systems DFS l Simulation and Discussions (Adaptive Fuzzy l Conclusive remarks Control ) • 5 6
®Copyright by Shun-Feng Su Simulation =3 is considered. The total rule numbers of the DFS, the Simplified DFS and the traditional FS are 81, 27, and 25, respectively. • 5 7
®Copyright by Shun-Feng Su Simulation The learning efficiency can easily be checked through the convergence speed of the tracking error. The function approximation capability can be evaluated by checking the magnitudes of the tracking error and of uc, where uc is the robust control law used to cope with all possible errors including the modeling error. • 5 8
®Copyright by Shun-Feng Su Simulation =3. • 5 9
®Copyright by Shun-Feng Su Simulation =3. of adaption cycle 3 to 19 for simulation 1. • 6 0 • Nov. 2013
®Copyright by Shun-Feng Su Simulation =3. The comparison of • 6 1 in simulation 1. • Nov. 2013
®Copyright by Shun-Feng Su Simulation From the figure, it can be observed that the tracking control results of the DFS and of the simplified DFS are good and similar. The simplified DFS can have fair results by using less fuzzy rules. Besides, the convergent speed of both DFSs is much better than that of the traditional fuzzy system. • 6 2
®Copyright by Shun-Feng Su Simulation =9 is considered. The total rule numbers of the DFS, the Simplified DFS and the traditional FS are 729, 81, and 121, respectively. • 6 3
®Copyright by Shun-Feng Su Simulation =9. of adaption cycle 3 to 19 for simulation 1. • 6 4
®Copyright by Shun-Feng Su Simulation =9. The comparison of • 6 5 in simulation 1. • Nov. 2013
®Copyright by Shun-Feng Su Simulation From the figure the learning speeds of both DFSs are still fast even the rule increases and remain unchanged in terms of cycles. The learning speed of the traditional fuzzy system becomes slower than that of simulation 1. It means the learning speed of DFS will not be affected significantly when the rule number increase but the traditional FS will be affected. • 6 6 disturbance The effects of minimum
®Copyright by Shun-Feng Su Simulation It can be found that the robust control magnitude of the system with either the DFS or the simplified DFS is smaller than that of simulation 1. In other words, the increase of rules in two DFSs can indeed improve the function approximation capability and most importantly, their learning efficiency (speed) is almost unchanged. • 6 7
®Copyright by Shun-Feng Su Simulation The traditional fuzzy system with a large fuzzy-rule number is supposed to have a better function approximation capability. But, in the simulation, the error is larger than that in simulation 1. We think the system may need a longer learning time to have the required performances. It is worth noting that the simplified DFS has fewer rules than the traditional fuzzy system, but its performances are better than that of the • 6 traditional one. 8
®Copyright by Shun-Feng Su Simulation =19 is considered. The total rule numbers of the DFS, the Simplified DFS and the traditional FS are 3249, 171, and 441, respectively. • 6 9
®Copyright by Shun-Feng Su Simulation =19. of adaption cycle 3 to 19 for simulation 1. • 7 0
®Copyright by Shun-Feng Su Simulation =19. The comparison of • 7 1 in simulation 1.
®Copyright by Shun-Feng Su Simulation Again, it can be observed that the increase of the rule number of two DFSs can enhance their function approximation capability to improve the tracking control performances and the learning speed is still roughly unchanged. For the traditional fuzzy system, because the required learning time is too long, the obtained tracking error can not converge to an enough small value after 19 cycles. • 7 2
®Copyright by Shun-Feng Su Simulation It can be observed that the simplified DFS has fewer rules than other two fuzzy systems do, but still have a fairly nice performance. • 7 3
®Copyright by Shun-Feng Su Outline l Preface l Introduction l Simplified l Adaptive of Decomposed Fuzzy Systems DFS Fuzzy Control l Simulation and Discussions l Conclusive remarks • 7 4
®Copyright by Shun-Feng Su Conclusions A novel fuzzy system structure called DFS is proposed to act as the fuzzy approximator for adaptive fuzzy control systems. With this structure, the coupling effects between the learning efficiency and the function approximation capability become insignificant. It can be observed that when the rule number of the DFS increases, the function approximation capability is improved to have better control performance and the • 7 learning efficiency is roughly the same. 5
®Copyright by Shun-Feng Su Conclusions For the requirement of the online applications, a simplified DFS is further proposed to have fewer rules but only slightly degrade the approximation capability. All simulation results indicate that the proposed simplified structure can still perform fairly with a reduced number of rules (lower computational burden) and even have much better control performance than that of using traditional fuzzy approximators. • 7 6
®Copyright by Shun-Feng Su Thank you for your attention! Any Questions ? ! Shun-Feng Su, Chair Professor of Department of Electrical Engineering, National Taiwan University of Science and Technology E-mail: sfsu@mail. ntust. edu. tw, • 7 7
- Slides: 77