L2 Normalization


正则化(Regularization)、归一化(也有称为正规化、标准化,Normalization)是对数据进行预处理的方式,他们的目的都是为了让数据更便于我们的计算或获得更加泛化的结果,但并不改变问题的本质。

向量范数定义为:

要使得x归一化到单位范数,即建立一个从x到x’的映射,使得x’的范数为1,则:

即:

结论:L2范数归一化就是向量中每个元素除以向量的L2范数

Python实现:

numpy.linalg.norm

import numpy as np
from numpy import linalg as LA

x = np.random.random((10,5))
result = x/(np.sqrt(np.sum(np.square(x), axis=0)))
print(x)
print(result)

print(LA.norm(result, axis=0))

Out:

x:
array([[ 0.89611258,  0.83221169,  0.45048202,  0.33025977,  0.55290277],
       [ 0.47209991,  0.54331039,  0.1630093 ,  0.19537002,  0.28366602],
       [ 0.91545029,  0.31205771,  0.44910737,  0.43154374,  0.37949212],
       [ 0.76429001,  0.26315375,  0.1852589 ,  0.01010026,  0.90191442],
       [ 0.79275858,  0.03905429,  0.12722141,  0.34318577,  0.33879815],
       [ 0.19106657,  0.30728503,  0.31959267,  0.08441712,  0.98688497],
       [ 0.35778428,  0.56113139,  0.94248815,  0.08165487,  0.85321076],
       [ 0.40224945,  0.50467228,  0.91305646,  0.11669095,  0.18031098],
       [ 0.95299778,  0.50856775,  0.48488233,  0.69416359,  0.34814049],
       [ 0.27136071,  0.22874501,  0.41675759,  0.79987126,  0.2221969 ]])

result:
array([[ 0.42794899,  0.57010567,  0.27341624,  0.26105925,  0.30148016],
       [ 0.2254568 ,  0.37219417,  0.09893711,  0.15443344,  0.15467399],
       [ 0.43718393,  0.21377478,  0.27258191,  0.34112082,  0.20692489],
       [ 0.36499558,  0.18027318,  0.11244132,  0.00798391,  0.49178503],
       [ 0.37859108,  0.0267541 ,  0.07721595,  0.27127681,  0.18473577],
       [ 0.09124606,  0.21050526,  0.19397406,  0.0667289 ,  0.53811675],
       [ 0.17086405,  0.38440243,  0.57203519,  0.06454543,  0.46522849],
       [ 0.1920989 ,  0.34572518,  0.55417187,  0.09224027,  0.0983178 ],
       [ 0.45511517,  0.34839376,  0.29429522,  0.54871298,  0.18982985],
       [ 0.12959146,  0.15670151,  0.25294748,  0.63227134,  0.12115685]])

Out[24]: array([ 1.,  1.,  1.,  1.,  1.])

比较好的博客: zouxy09:机器学习中的范数规则化之(一)L0、L1与L2范数