Learning vector quantization

LVQ1

Initialize several code vectors per label. Iterate until convergence criteria is reached.

1. Sample a datum ${\displaystyle x_{i}}$, and find out the code vector ${\displaystyle w_{j}}$, such that ${\displaystyle x_{i}}$ falls within the Voronoi cell of ${\displaystyle w_{j}}$.
2. If its label ${\displaystyle y_{i}}$ is the same as that of ${\displaystyle w_{j}}$, then ${\displaystyle w_{j}\leftarrow w_{j}+\alpha _{t}(x_{i}-w_{j})}$, otherwise, ${\displaystyle w_{j}\leftarrow w_{j}-\alpha _{t}(x_{i}-w_{j})}$.

LVQ2

LVQ2 is the same as LVQ3, but with this sentence removed: "If ${\displaystyle w_{j}}$ and ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have the same class, then ${\displaystyle w_{j}\leftarrow w_{j}-\alpha _{t}(x_{i}-w_{j})}$ and ${\displaystyle w_{k}\leftarrow w_{k}+\alpha _{t}(x_{i}-w_{k})}$.". If ${\displaystyle w_{j}}$ and ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have the same class, then nothing happens.

LVQ3

Initialize several code vectors per label. Iterate until convergence criteria is reached.

1. Sample a datum ${\displaystyle x_{i}}$, and find out two code vectors ${\displaystyle w_{j},w_{k}}$ closest to it.
2. Let ${\displaystyle d_{j}:=\|x_{i}-w_{j}\|,d_{k}:=\|x_{i}-w_{k}\|}$.
3. If ${\displaystyle \min \left({\frac {d_{j}}{d_{k}}},{\frac {d_{k}}{d_{j}}}\right)>s}$, where ${\displaystyle s={\frac {1-w}{1+w}}}$, then
   • If ${\displaystyle w_{j}}$ and ${\displaystyle x_{i}}$ have the same class, and ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have different classes, then ${\displaystyle w_{j}\leftarrow w_{j}+\alpha _{t}(x_{i}-w_{j})}$ and ${\displaystyle w_{k}\leftarrow w_{k}-\alpha _{t}(x_{i}-w_{k})}$.
   • If ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have the same class, and ${\displaystyle w_{j}}$ and ${\displaystyle x_{i}}$ have different classes, then ${\displaystyle w_{j}\leftarrow w_{j}-\alpha _{t}(x_{i}-w_{j})}$ and ${\displaystyle w_{k}\leftarrow w_{k}+\alpha _{t}(x_{i}-w_{k})}$.
   • If ${\displaystyle w_{j}}$ and ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have the same class, then ${\displaystyle w_{j}\leftarrow w_{j}-\epsilon \alpha _{t}(x_{i}-w_{j})}$ and ${\displaystyle w_{k}\leftarrow w_{k}+\epsilon \alpha _{t}(x_{i}-w_{k})}$.
   • If ${\displaystyle w_{k}}$ and ${\displaystyle x_{i}}$ have different classes, and ${\displaystyle w_{j}}$ and ${\displaystyle x_{i}}$ have different classes, then the original paper simply does not explain what happens in this case, but presumably nothing happens in this case.
4. Otherwise, skip.

Note that condition ${\displaystyle \min \left({\frac {d_{j}}{d_{k}}},{\frac {d_{k}}{d_{j}}}\right)>s}$, where ${\displaystyle s={\frac {1-w}{1+w}}}$, precisely means that the point ${\displaystyle x_{i}}$ falls between two Apollonian spheres.