As an analogy, you can steer your car to exactly where you want it, or you can design it to steer itself and ride along as a nervous passenger. Either can get you to where you want, but not necessarily by the route you choose.<\/p>\n\n\n\n
However, there is a third way, and something that plays an important part in the learning process in you and me. It\u2019s called competitive learning, and it\u2019s how we learn to group similar things into categories. You could also call it stereotyping. It\u2019s important, because without these groupings, animals would have to deal with every encounter as if it were the first time. Previous experience would mean very little, and we\u2019d struggle to survive.<\/p>\n\n\n\n
Survival of the fittest<\/h2>\n\n\n\n
So, what is competitive learning, and what does it do for us? Despite its name, it hasn\u2019t got anything to do with competing with other neural networks. It\u2019s all about the brain competing with itself. One of the biggest differences with this approach is that it can happen without any outside intervention. No desired output is needed and no reward is required. The network just gets on with the process of putting things in boxes. <\/p>\n\n\n\n
To perform this magic, we need to build a new type of neural network, known as an auto-associative memory. If you\u2019ve ever heard of a Hopfield Network<\/a>, then you\u2019ve heard of at least one type of these specialised machines. Brains are very efficient at using just enough neurons to achieve an outcome, leaving the rest free for further experiences. Auto-associative networks are an attempt to duplicate this.<\/p>\n\n\n\n This is where competitive learning comes in. In this type of learning, we don\u2019t care which neurons fire. We just want to make sure only a few fire for each situation, and different ones fire when the situation changes.<\/p>\n\n\n\n The way this is achieved is through a winner-takes-all approach. Neurons are connected in such a way that the strongest firing neuron provides positive feedback to its nearest neighbouring neurons and negative feedback to those neurons that are further away. Each input is then connected to all of these neurons, so each one gets a chance to respond to the input pattern based on its initial weights. <\/p>\n\n\n\n The following diagram shows a simplified representation of such a network. Connectivity from all inputs to all neurons, and connectivity from one neuron to all other neurons is shown. Close neighbours are connected with a positive weight and distant neighbours with a negative weight. The graph below shows how connection weight varies with distance for a larger network. Because of the shape of this graph, it is often referred to as a \u201cMexican hat\u201d weight distribution.<\/p>\n\n\n\n When patterns of inputs are first presented to a network like this, each neuron fires at a different level depending on how closely its weights match the inputs. The neuron that best matches the pattern then \u201crecruits\u201d its nearest neighbours via positive feedback and they start to fire more strongly as well. Meanwhile, neurons further away are forced to fire less strongly due to the negative feedback that they receive from the original neuron and its new recruits.<\/p>\n\n\n\nThe winner takes it all<\/h2>\n\n\n\n
![\"\"](\"https:\/\/jonayre.uk\/blog\/wp-content\/uploads\/2024\/11\/Understanding-neural-networks-Part-4_-Self-organisation-and-unsupervised-learning.png\")
![\"\"](\"https:\/\/jonayre.uk\/blog\/wp-content\/uploads\/2024\/11\/Understanding-neural-networks-Part-4_-Self-organisation-and-unsupervised-learning-1.png\")