Probably you already know most of the stuff i will talk about. But I want to make it clear any way.
First from the nonscientific perspective of hebian learning
I think when there are just 2 neurons and you want to "wire them together" i suspect it wouldn't be learning anything. In the and you may have a post-synaptic neuron fires whenever the pre-synaptic fires.
But when there are at least 2 pre-synaptic neurons it begins to make sense. (And you may avoid giving random numbers)
Long Term Potantiation (LTP) is usually discussed when hebian learning in process. To make a clear example lets assume 3 pre-synaptic neurons one of then (S1) has a stronger connection (weight) to post synaptic neuron while other two (W1 and W2) has a weak connection(weight).
To make it more concrete i will give (neuroscientifically non-valid) meanings to these neurons. Lets say post-synaptic neuron is a neuron that recognized motorbikes. S1 fires when you see a bike, W1 fires when there is a motor sound and W2 (lets make it arbitrary) fires when it smells cherries.
In the begining you don't have the idea how a motorbike sounds like. But when you see it S1 fires, and because you hear it W1 also fires. However W1's contribution is very weak S1 may produce enough inout for post-synaptic neuron to fire. Since W1 was firing when post-synaptic neuron is firing the connection is strengthened. Anf if you have enouh inputs after some points even without the presence of S1 post-synaptic neuron can fire. Since it didn't smell cheries when you say the motorbike W2 remained still the same.
So the message to take home is it is meaningful when there are multiple pre-synaptic inputs and the effect is to one side only.
I am quoting from "Cognitive Neuroscience" book of Gazzangia
three rules for associative LTP have bee n drawn:
- Cooperativity . More than one input must be active at the same time.
- Associativity . Weak inputs are potentiated when co-occurring with stronger inputs.
- Specificity . Only the stimulated synapse shows potentiation.
From the perspective of machine learning:
For unsupervised learning in neural networks you may want to look at:
- Boltzmann Machines
- Stochastic Maximum Likehood Learning