Self Organizing Maps Explained

Self organizing maps is a data visualization technique that reduces the dimensions of data to make it easier to understand high-dimensional data. Learn how it works. 

Published on Jun. 06, 2024
Self Organizing Maps Explained
Image: Shutterstock / Built In
Brand Studio Logo

Self organizing map (SOM) is a data visualization technique that reduces the dimensions of data to a map to make it easier to understand high-dimensional data. The technique was proposed by Teuvo Kohonen. It showcases clustering by grouping similar data together. This clustering ability was well received after a variant of SOM, growing self-organizing map, was introduced.

What Is Self Organizing Maps?

Self organizing maps is a data visualization technique that clusters similar data together to make it easier to understand high-dimensional data. The technique differs from other error-correction methods because it follows competitive learning.

We can summarize SOM as:

Illustration of how a self organizing map works
Fig.1: An illustration of what a SOM does. | Image: Vivek Vinushanth Christopher

SOM is unique as it deviates from the most generic error-correction approach and follows competitive learning.

 

How Does Self Organizing Maps Work?

Data points will be competing to get a representation in the network. First, the network is initialized with weights. Data from the training data set is randomly chosen, and the distance between nodes (weights) and sample vector is calculated. The shortest of the distances is considered the best matching unit (BMU). Then, the neighborhood of the winner node or BMU is found. The neighbor weights or nodes get updated. This process is repeated until the network is trained for every training data.

What Is a Best Matching Unit (BMU)? 

A BMU is the winner node (or weight) that has the shortest distance from itself to the sample vector. There are numerous ways to determine the distance, however, the most common one is Euclidean distance.

More on Data ScienceHistogram of Oriented Gradients: An Overview

 

Self Organizing Maps Steps

  1. Initiation: Randomization.
  2. Competition: Select winner node.
  3. Cooperation: Identify the neighborhood radius.
  4. Adaptation: Adapt weights of winners and neighbors.
  5. Smoothing: Reduce neighborhood radius as iterations increase, and introduce an optional smoothing phase.

Self Organizing Map Algorithm Parameters

  • R/C: Number of rows/columns
  • N: Number of neurons
  • MaxIter: Number of iterations
  • MaxNR: Maximum neighborhood radius of grid
  • nr(t): Neighborhood radius
  • ර(t): Learning rate
  • A: number of dimensions
  • D(wi,wj): Distance between Neuron i and j 
  • All_inputs={x1,x2….}: Input 
  • wk: All weights or nodes
self organizing maps algorithm steps
SOM algorithm steps. | Image: Vivek Vinushanth Christopher
Self organizing maps equation
Self organizing maps equations. | Image: Vivek Vinushanth Christopher

 

Self Organizing Maps Example

Let initial weights be, w1 = (0.45,0.89), w2 = (0.55,0.83), w3 = (0.95,0.32) and w4 = (0.62,0.78). And let the four neurons (N=4) N1, N2, N3 and N4 be located at the grid in Cartesian plane positions (0,0), (1,0) , (1,0) and (1,1) respectively. And suppose the network takes a two-dimensional (A=2) input vector (x1,x2). Let w_i= (w1_i,w2_i) be the weight for the neuron i. nr = 0.6. Input vector (3,1). Assume learning rate = 0.5, and it is constant throughout. Assume initial weights as noted in the diagram.

Grid and weights drawn by the author
Grid and weights in a self organizing map. | Image: Vivek Vinushanth Christopher

1. Calculate Neighborhood Radius

Since this is the first iteration, nr = 0.6.

2. Calculate the Learning Rate

ර(t) = 0.5 and constant.

3. Calcualte the Winner Node 

Find the nearest neighbor or the winner node for the given vector input (use Euclidean distance).

  • d (V,N1) = (3–0.45)² + (1–0.89)² = 6.514
  • d (V,N2) = (3–0.55)² + (1–0.83)² = 6.03
  • d (V,N3) = (3–0.62)² + (1–0.78)² = 5.71
  • d (V,N4) = (3–0.95)² + (1–0.32)² = 4.66

Since d(V,N4) is the smallest distance, the winner node is N4.

5. Calculate Distance Between the Winner Node and Other Nodes

  • d(N4,N1) = sqrt ( (0.45–0.95)² + (0.89–0.3)²) = 0.75 (> nr)
  • d(N4,N2) = sqrt ( (0.55–0.95)² + (0.89–0.32)²) = 0.648 (> nr)
  • d(N4,N3) = sqrt ( (0.62–0.95)² + (0.89–0.78)²) = 0.566 (< nr)
  • d(N4,N4) = sqrt ( (0.95–0.95)² + (0.89–0.89)²) = 0 (

6. Choose Nodes to Update Weights

Since d(N4,N3), d(N4,N4) < nr => N3,N4 chosen for updating their weights

7. Update the Weights

Next, update their weights using: w(t+1) = w(t) + Q(t) * ර(t) (x-w(t))

Updating the weight for N3:

Weights updated for N3
Weights updated for N3. | Image: Vivek Vinushanth Christopher

Updating the weight for N4:

Updating weights for N4.
Updating weights for N4. | Image: Vivek Vinushanth Christopher
A tutorial on how self organizing maps works. | Video: Thales Sehn Körting

More on Data ScienceUnderstanding the Derivative of the Sigmoid Function

 

Understanding Self Organizing Maps

In this blog, we have discussed a small example of how the weights are added for a single input. Manual calculation for every input and for each epoch is complex and hence, we didn’t touch on that for this article. However, I’m hopeful that anything can be explained through an apt example and this article. 

Frequently Asked Questions

A self organizing map is a data visualization algorithm that makes it easier to understand high dimensional data sets. The algorithm utilizes competitive learning to group clusters of data together.

Self organizing maps is a data visualization algorithm that follows five steps:

  1. Initiation: Randomly selects a data point from the training set.
  2. Competition: Selects the node with the shortest distance to the sample vector.
  3. Cooperation: Identify the neighborhood radius.
  4. Adaptation: Adapt weights of winners and neighbors.
  5. Smoothing: Reduce neighborhood radius as iterations increase, and introduce an optional smoothing phase. 
Hiring Now
Sierra Space
Aerospace • Hardware • Information Technology • Robotics
SHARE