The difference between autoassociative and heteroassociative networks is:
Autoassociative Networks:
- Autoassosiative networks are special kind of networks used to simulate associative processes.
- These are acheived through interaction of set of simple processing elements which are connected through weighted connections.
- They are capable to retrieve piece of data with the partial information and also capable for remembering from small portion of data.
Heteroassociative Networks:
- Heteroassociative networks stores input-output pattern pairs to recall stored output pattern by receiving noisy or incomplete version.
- In each of the pairs, an input pattern should differ from an output pattern.
- In this basic logical operations are used to determine associations among common and special features of reference patterns.
Hebb’s rule is a postulate proposed by Donald Hebb in 1949 [1]. It is a learning rule that describes how the neuronal activities influence the connection between neurons, i.e., the synaptic plasticity. It provides an algorithm to update weight of neuronal connection within neural network. Hebb’s rule provides a simplistic physiology-based model to mimic the activity dependent features of synaptic plasticity and has been widely used in the area of artificial neural network. Different versions of the rule have been proposed to make the updating rule more realistic. The weight of connection between neurons is a function of the neuronal activity. The classical Hebb’s rule indicates “neurons that fire together, wire together”. In the simplest form of Hebb’s rule, Eq. (A.1), wij stands for the weight of the connection from neuron j to neuron i
Hopfield model is an associative memory model using the Hebb’s rule for all possible pairs ij with binary units. The state variable xi of the neuron i takes on either on of the two possible values: 1 or -1, which corresponds to the firing state or not firing state, respectively
The overall performance of the recursive least-squares (RLS) algorithm is governed by the
forgetting factor. The value of this parameter leads to a compromise between low misadjustment and stability on the one hand, and fast convergence rate and tracking on the other hand.
Competitive learning is a type of unsupervised learning model used in machine learning and artificial intelligence systems. Some of the interesting new formats of machine learning projects are partially based on competitive learning include self-organizing component neural networks. That makes this an integral idea in the machine learning community.
In a
competitive learning model, there are hierarchical sets of units in the network with inhibitory and excitatory connections. The excitatory connections are between individual layers and the inhibitory connections are between units in layered clusters. Units in a cluster are either active or inactive.
Scientists explain that the configuration of active units represents an input pattern to be sent to the next level. In processes like vector quantization, professionals can see the principles of competitive learning at work. Competitive learning also exists alongside other learning models such as ensemble learning, where more than one learning unit works together in a dedicated effort toward a result.
In fact, the significant
difference between competitive learning and
Hebbian learning is
in the number
of active neurons at any one time. Whereas
neural network based on
Hebbian learning, several output neurons may be active simultaneously
in competitive learning, only a single output neuron is active at any one time.
BAM The main objective to introduce such a network model is to store hetero-associative pattern pairs.
This is used to retrieve a pattern given a noisy or incomplete pattern.
BAM Architecture:
When BAM accepts an input of
n-dimensional vector
X from set
A then the model recalls
m-dimensional vector
Y from set
B. Similarly when
Y is treated as input, the BAM recalls
X.
Algorithm:
- Storage (Learning): In this learning step of BAM, weight matrix is calculated between M pairs of patterns (fundamental memories) are stored in the synaptic weights of the network following the equation
- Testing: We have to check that the BAM recalls perfectly for corresponding and recalls for corresponding . Using,
- All pairs should be recalled accordingly.
- Retrieval: For an unknown vector X (a corrupted or incomplete version of a pattern from set A or B) to the BAM and retrieve a previously stored association:
- Initialize the BAM:
- Calculate the BAM output at iteration :
- Update the input vector :
- Repeat the iteration until convergence, when input and output remain unchanged.
Limitations of BAM:
- Storage capacity of the BAM: In the BAM, stored number of associations should not be exceeded the number of neurons in the smaller layer.
- Incorrect convergence: Always the closest association may not be produced by BAM.