Early development of ANN was limited by the revelation that the
perceptron (the ANN of the time) could not implement a simple logical
transformation, the exclusive or (xor). The logical capabilities
of models can be examined by using fixed values to represent "true"
and "false" -- usually 1 and 0, respectively. The exclusive
or would then be the operation which produces a 1 when one or the
other of two inputs is 1, and produces a 0 otherwise. Regardless
of how the weights were set, a two-input perceptron could not implement
this operation. This represented a fundamental limitation of the
model for representing logical relationships. Later, it was realized
that multilayer ANN models overcame this limitation.
The capability of MPR models to implement logical relationships
was explored. It is apparent that any statement constructed with
the logical operators "and", "or", "xor"
or "not" can be easily represented using only linear interaction
terms. These operators are modeled by the following:
A and B |
A*B |
A or B |
A + B - A*B |
A xor B |
A + B - 2*A*B |
not A |
1 - A |
More complex logical statements are easily constructed by substituting
the above expressions. For example, (A .and. B) .or. C is equivalent
to A*B + C - A*B*C.
More importantly, a unique linear interaction model can be fitted
to any truth table which can be constructed. For example, Table
1 shows a truth table constructed by the rule: Z = (B .xor. C) if
.not. A, else Z = .not. (B .xor. C). Additional columns are shown
for the computed values of A*B, A*C, B*C, and A*B*C.
TABLE 1. Truth table with three independent variables.
A
|
B
|
C
|
AB
|
AC
|
BC
|
ABC
|
Z
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
0
|
1
|
0
|
0
|
0
|
0
|
1
|
0
|
1
|
0
|
0
|
0
|
0
|
0
|
1
|
0
|
1
|
1
|
0
|
0
|
1
|
0
|
0
|
1
|
0
|
0
|
0
|
0
|
0
|
0
|
1
|
1
|
0
|
1
|
0
|
1
|
0
|
0
|
0
|
1
|
1
|
0
|
1
|
0
|
0
|
0
|
0
|
1
|
1
|
1
|
1
|
1
|
1
|
1
|
1
|
|
Multilinear regression yields the first degree MPR model:
Z = A + B + C - 2AB - 2AC - 2BC + 4ABC.
A unique model can be found for any values placed in the column
under Z. Increasing the degree n of the MPR model makes it possible
to construct "truth tables" with more than two levels.
For example, if variables were allowed to have three levels, say,
-1, 0, and 1, a polynomial interaction model of degree 2 could completely
describe the truth table.
From the point of view of continuous variables, this indicates
that polynomial terms allow the model to discriminate independantly
between, say, high, medium and low values of the variables. Linear
models cannot treat these levels independantly.
The capability of MPR models to represent any logical relationship
may be one of the most important indications of the strength of
this type of model. It shows the power of the model to represent
complex behaviors in an efficient representation which is easily
determined from data.
|