Learning and Tuning Fuzzy Logic Controllers Through Reinforcement

Fuzzy Logic controller#

Step one: fuzzification(encoding)##

In coding the values from the sensors, one transforms the values of the sensors measurement by using the linguistic labels in the rule predictions. This process is commonly called fuzzification or encoding. The fuzzification stage requires matching the sensor measurement against the menbership functions of linguistic labels.

In modeling the hunman expert operator's knowledge, fuzzy control rules of the form
IF Error is small AND Change-in-error is small THEN
Force is small

can be used effectively when expert human operators can express the heuristics or the control knowledge that they use in controlling a process in terms of rules of the above form.

Conflict Resolution and Decision Making

As mentioned earlier, because of the partial matching attribute of fuzzy control rules and the fact that the preconditions of rules do overlap, more than one fuzzy control rule can fire at a time. The methology which is used in deciding which control action should be taken as the result of the firing of several rules can be referred to as conflict resolution. The following example, using two rules, illustrates this process. Assume that we have the following rules:

1. Rule 1: IF X is \(A_{1]\) and Y is \(B_{1}\) THEN Z is \(C_{1}\)
2. Rule 2: IF X is \(A_{2}\) and Y is \(B_{2}\) THEN Z is \(C_{2}\)
   Each rule has an antecedent, of \(if\), part containing several preconditions, and a consequent, or \(then\), part which prescribes the value of one or more output actions. Now, if we have \(x_{0}\) and \(y_{0}\) as the sensor readings for fuzzy variables \(X\) and \(Y\), then their \(truth values\) are respesented by \(\mu_{A_{1}}(x_{0})\) and \(\mu_{B_{1}}(y_{0})\) respectively for rule 1, where \(\mu_{A_{1}}\) and \(\mu_{B_{1}}\) represent the membership function for \(A_{1}\) and \(B_{2}\), respectively. Similarly for rule 2, we have \(\mu_{A_{2}}(x_{0})\) and \(\mu_{B_{2}}(y_{0})\) as the truth values of the preconditions:

\[\omega_{1} =\wedge (\mu_{A_{1}}(x_{0}),\mu_{B_{1}}(y_{0})) $$. Similarly for rule 2, $$\omega_{2} =\wedge (\mu_{A_{2}}(x_{0}),\mu_{B_{2}}(y_{0})) $$, where $\wedge$ denotes a conjunction or intersection operator. Traditionally, fuzzy logic controlles use a $minimum$ operator for $\wedge$. However, here we use a $softmin$ operator, which produces the same results in the limit but in general is not a specific as the $minimum$ operator. The reason for this is differenttialability, which we need for learning purpose. This will be dealt with in greater detail later. Using the softmin, the $strengh$ of rule 1 can be calculated by $$\omega_{1}=\frac{ \mu_{A_{1}}(x_{0}) e^{-k \mu_{A_{1}}(x_{0}) } + \mu_{B_{1}}(y_{0}) e^{-k \mu_{B_{1}}(y_{0}) } } { e^{-k \mu_{A_{1}}(x_{0}) } +e^{-k \mu_{B_{1}}(y_{0}) }}\]

Similarly for rule 2

\[\omega_{2}=\frac{ \mu_{A_{2}}(x_{0}) e^{-k \mu_{A_{2}}(x_{0}) } + \mu_{B_{2}}(y_{0}) e^{-k \mu_{B_{2}}(y_{0}) } } { e^{-k \mu_{A_{2}}(x_{0}) } +e^{-k \mu_{B_{2}}(y_{0}) }} \]

The control output of rule 1 is calculated by applying the matching strength of its precondtiontions on its conclusion. We assume that

\[z_{1} = \mu_{c_{1}}^{-1}(\omega_{1}) \]

and, for rule 2, that

\[z_{2} = \mu_{c_{2}}^{-1}(\omega_{2})$$. Here is a defuzzification procedure to compute the expression $\mu^{-1}(\omega)$, which is explained later. The above equations show that, as a result of reading sensor values $x_{0}$ and $y_{0}$, rule 1 is recommending a control action $z_{1}$ and rule 2 is recommending a control action $z_{2}$. The combination of the above rules produces a nonfuzzy control action $z*{*}$, which is calculated using a weighted averaging approach: $$z^{*} = \frac{ \sum_{i=1}^{n} \omega_{i} z_{i} }{ \sum_{i=1}^{n} \omega_{i} }$$, where $n$ is the number of rules, and $z_{i}$ is the amount of control action recommended by rule $i$, A similar procedure can be used for multiple output vairables in the consequents.\]