Introduction
So, it looks like you made it through the first two parts of my series on understanding neural networks. Thank you for coming with me on this journey. Your presence is very much appreciated.
In Part 1 – The Artificial Neuron I introduced the basic concept that underpins all artificial neural networks, and in Part 2 – Hidden layers, troublesome feedback I described two ways of connecting these neurons together to make a virtual brain.
In part 3, I’m going to introduce two ways of training these algorithms. I’ll also try to explain the differences between them and also how they do (or don’t) relate to how biological brains learn. Hopefully, I’ll explain them in a way that strips away some of the mystery. If it doesn’t make sense, I apologise – you can lay the blame for that firmly on me.
Teaching the machine
Arguably the most significant difference between the brain and traditional machines is adaptability. Machines can be built to perform specific tasks that require great strength, blistering speed, or microscopic accuracy. In highly specialised activities, they can far exceed human performance. However, there is one area where machines are woefully unable to match even simple biological organisms.
Adaptability.
Present those machines with a problem that varies even slightly from the one they’ve been designed to solve and they fail dismally. What they have never been able to do is learn new things. Machines lack even a fraction of the amazing flexibility of the biological brain.
Neural networks emerged as an attempt to solve this problem. By copying learning processes in the brain, engineers hoped to emulate intelligence. Bravely they chose to call this “artificial intelligence” – a title that remains more an aspiration than an achievement.
Early research into learning in the brain identified that the synapses that connect neurons together vary significantly in their strength. Furthermore, scientists noticed that these strengths could change quickly, and permanently. It seemed that these tiny components were the seat of learning and memory.
Two routes to wisdom – maths or magic?
The beginnings of a pattern emerged that suggested that a synapse would become stronger if its input fired shortly before the output of its target neuron fired. This was neatly summed by neuropsychologist Donald Hebb in 1949 with the phrase “neurons that fire together, wire together”. This was a very simplified view of what was happening but it did get to the heart of the matter. How do you change the weights in a network to get it to learn to solve a task, if you don’t already know the answer?
When it comes to training, there are two routes to go. One is very mathematical and provable, while the other is just a bit more mysterious and based on intuition. One relies on you knowing what outputs you want from the network, and the other assumes you can tell that things are getting better or worse. It turns out that nature chose the second path, but the maths based approach is easier to understand so I’ll start there.
I’ve already stated that the feedforward network is very capable and is a simpler beast to deal with. In part 1, I described a simple problem, known as the XOR problem in which the output of the network should be 1 (yes) if one, and only one of the inputs is also 1. This is clearly a problem where we know what the desired output is, so it is suitable for the first of the training approaches – the “backpropagation algorithm”.
Forward thinking, backwards learning
The principle is simple, especially if you start with just one artificial neuron. It assumes you know what output you wanted for a given set of inputs. For example, if I want a neuron to give a “yes” when given the input pattern “yes, yes, no” I just need to increase the weights for the first two inputs and decrease the weights for the third one until I get the answer I want. If the answer I want is a “no” then I need to do the opposite.
Remembering that “yes” is 1.0 and “no” is “0.0”, we can express the error in the neuron’s output as the difference between the desired output and the actual output. By knowing the error we can determine the direction of the change needed. This is the first factor in our learning rule.
However, as the neuron output approaches 1.0 or 0.0, the amount of change we can bring about drops to almost nothing so making a change here becomes pointless. To take this into account we include the slope (or gradient) of the neuron’s activation function as the second factor for our learning rule. (Take a look at part 1 of this series for a recap on what an activation function is, and what it looks like).
The next factor to take into account is the strength of the inputs. The higher the value of an
input, the bigger the effect it has on the output. In other words, we need to focus our learning on the inputs that contribute most to the output. To do this we include input strength in our training approach.
More haste, less speed
Finally, we want the neural network to learn many different patterns, not just one. When we try this out for real we find that what actually happens is the network either fixes on the first pattern it sees, or forgets everything and only remembers the last pattern presented. To overcome this, instead of moving the weights directly to the target value, we nudge them gently in the desired direction. To do this we multiply the weight change by a small learning rate.
What this all adds up to is a learning rule that looks like this:
Change in weight = input x error x output gradient x learning rateThis is all fine if the neuron is an output neuron because we can work out the error. However, for neurons in the hidden layers, we don’t know what their output needs to be. This is where the back-propagation bit comes in. Cutting to the chase, the answer to this problem is that we estimate the error for a hidden neuron by adding up a proportion of the error from each of the neurons it provides an input to. We call this the back-propagated error.
If you intend to build your own neural network, the feed-forward back-propagation approach is an excellent place to start. I might dedicate a post to how to do this later, but for now there are many really good posts that should help. Here’s one that’s well worth reading.
The language of music
One example of a use to which I’ve put feed-forward, back-propagation networks is in the “appreciation” of music, and other media. Well, not appreciation as such, but more the creation of a virtual “me” capable of indicating whether a person is likely to enjoy a piece of music or not.
This problem was solved successfully using very simple feed-forward networks (a total of 100 neurons across two layers) trained using the back-propagation algorithm. The first of these networks was trained on a set of 1500 musical samples ranging from modern pop to early classical to create a music based “embeddings model”.
An embeddings model is a technique used in the field of AI to turn text into numbers suitable for feeding into a neural network. It allows us to create a vector of numbers that represent the various features of a word or document. These models have the useful property that words or documents with similar meanings result in embeddings vectors that “point” in similar directions. (You can read more about text based embeddings in this Wikipedia article).
Text based embeddings models are commonplace nowadays, and I’ve used them to solve this particular problem for films based on textual synopses of their plots. Unfortunately, equivalent models relating to musical input are not available. That’s why I needed to create one myself. Once I had this model, I was able to use it to train a second neural network to emulate the musical preferences of a human subject.
With only a tiny training set (6 pieces of disliked music and 9 pieces of liked music) the AI Avatar (as I called it) was able to match the preferences of the person on samples it hadn’t been trained on over 90% of the time (11 out of 12 times to be exact). I then played entire pieces to both the human and their Avatar and plotted the preference score they each awarded to each 30 second segment.
The following graph is a comparison for just one of those pieces. (The human subject happened to be my son, Nathan). As you can see, the AI Avatar does a very decent job of predicting how the human will score the piece. Such an Avatar could then be used to scan music libraries and recommend pieces that their human might like. This could be used to create a hyper-personalised recommendation engine.
The path less travelled
In nearly all situations where someone says they trained a neural network to solve a problem, they’ll have used a feed-forward network and the back-propagation algorithm. Feed-forward because it’s the simplest to deal with, and back-propagation because it’s the easiest to get a decent result from. However, there are situations where it just won’t cut the mustard.
Firstly, to use back-propagation, you have to know what the correct output is. That’s often not the case when you’re solving novel problems. Take a robotic arm, for example. If you already know where you want the arm to move and what outputs are required to move it there, you don’t need a neural network. Existing control solutions will be far better.
Secondly, if your problem is time dependent (what came before matters) then you need feedback. This means you need a recurrent network and back-propagation can’t be used. There are “tweaks” to backpropagation that can make it work (sort of), but you’ll be trying to hammer in a nail with a screwdriver.
Thirdly, back-propagation doesn’t play nicely with scale. It has a tendency to push many neurons in the same direction which can be wasteful. Biological brains are very good at something called “sparse encoding”. What this means in practice is that only a small percentage of neurons are recruited for each specific situation, leaving the rest free to solve other problems.
Neurons that fire together, wire together
Remember Donald Hebb from earlier in this post? He discovered that biological synapses got stronger when the neuron feeding the synapse and the one fed by the synapse fired at the same time. What he and many neuroscientists since haven’t found is evidence of back-propagation from neuron to neuron. (A form of back-propagation does exist within a single neuron, but this is not related to the mechanisms used in the back-propagation algorithm).
This wiring process also seemed to be controlled by reward. Actions that led to good outcomes also led to stronger synapses. In the absence of reward, either nothing changed or synapses weakened. Learning algorithms based on this approach are often referred to as “Hebbian learning”.
(Note: I am drastically simplifying what actually goes on in the brain in order to aid understanding. Any neuroscientists reading this will be screaming at the screen right now).
So let’s jump straight to the basic Hebbian learning algorithm. In its simplest form it looks like this:
Change in weight = reward x input x output x learning rateAt face value this seems okay but there’s a problem. If we continue to reward the neural network, the weight just keeps growing, and will continue to do so. In real brains this doesn’t happen so we add a second term to the learning rule.
My preferred approach to this is to add a term that adjusts the weight to achieve a balanced neuron. This involves adding a bit to the weight if the neuron’s average rate of firing is greater than 0.5 and taking a bit off the weight if below 0.5. I prefer this approach because it is an approximation of what seems to be happening inside the brain. This additional term looks like this:
Balancing weight change = current weight x average output x learning rateTrust the process
At first glance, this approach seems very vague. Changing every neuron based solely on its local inputs and output, and an overall reward feels distinctly “hand wavy”. However, it works. In fact, it works remarkably well.
I first used this learning approach in 1989 and many times since to train networks to “solve” problems for which the desired outputs weren’t known in advance. One example is the controlling of a simulated robotic arm. The end of the arm is equipped with directional heat sensors, and the challenge is to follow a heat source that moves in three dimensions.
The neural network’s outputs are connected to the motors at each of the joints of the arm and the heat sensors are connected to its inputs. The reward value is simply a 1.0 if the arm is getting warmer and -1.0 if the arm is getting cooler. Starting from a completely virgin network with totally random weights, the network takes less than a minute of real time movement to learn to control the arm, and get it to track the heat source.
The same network can also be placed into a “virtual moth” with outputs connected to turn and move motors, and inputs connected to heat sensing eyes facing to the front-left and front-right. Starting from a similar virgin setup, the network takes a similar amount of time to control the moth and guide it to the moving heat source.
Where are we, and how did we get here?
I’ll describe these neural network training examples in more detail in later posts, but for now let’s recap on where we are. Firstly, we discussed the artificial neuron, and how it works. Then I introduced two different types of neural network, feed-forward and recurrent and touched on some of the key differences.
In this latest post, I’ve briefly explained two different training strategies. The first of these, the back-propagation algorithm is best suited to feed-forward networks and problems where the desired network outputs are well defined. It has a strong grounding in provable mathematics, but relies on mechanisms that don’t seem to exist in the biological brain. Back-propagation is not generally considered to be “biologically plausible”, and doesn’t play nicely with networks that have feedback connections.
The second approach, Hebbian learning, works for feed forward networks, but is more commonly used with recurrent networks. It has the advantage that it only requires an overall reward signal to drive its learning. It can therefore be used to solve problems where the required outputs are not known. Hebbian learning is based loosely on simple observations of how synapses in biological brains change over time, and is therefore considered to be more biologically plausible as an approach.
Overall, if the goal is adaptability, it is recurrent networks and variants of Hebbian learning that offer the most promise. However, feed-forward networks and back-propagation are useful in a wide variety of useful problems and should not be underestimated. In fact, they currently dominate most practical applications of neural networks in industry.
Where next?
In the next part of this series I’ll introduce a third type of network and another type of learning. These networks are known as “auto-associative” networks, and they provide an insight into how we animals learn to group the things we experience into different categories.