数据科学和机器学习(第 13 部分):配合主成分分析(PCA)改善您的金融市场分析(基础篇)
「用 PCA 给行情降噪先看这一层」
主成分分析(PCA)本质是把高维相关性强的行情特征做正交变换,抽出少数几个不相关的主要成分,保留大部分方差。金融时间序列里,多品种报价、多周期指标往往共线严重,直接喂给模型容易过拟合,PCA 能把冗余压掉。
| 在 MT5 里验证这一点很简单:取 EURUSD 的 M5 收盘价、RSI(14)、MACD(12,26,9) 三列数据,算相关系数矩阵,常能看到 | r | 超过 0.7 的共线对。PCA 后第一主成分通常能解释 60%~85% 的总方差,具体比例随样本窗口浮动。 |
|---|
外汇与贵金属属高风险品种,PCA 只解决维度冗余,不预测方向;用它降维后建模,信号失效概率仍受行情 regime 切换影响,需样本外滚动验证。
◍ PCA 降维在交易里的真实取舍
主成分分析(PCA)本质是把高维变量压缩成低维集合,同时尽量保留原数据里的信息含量。对外汇和贵金属这类高风险市场来说,降维牺牲的一点准确度,往往能换来分析速度和可视化的实在好处,而准确度本身并不等于账户盈利。 具体落地 PCA 只看五步链路:标准化数据、求协方差矩阵、提取特征向量与特征值、算 PCA 得分并标准化、取成分。下一步直接从标准化数据动手,这是后面所有矩阵运算不出偏的基础。
把不同量纲压到同一把尺子上
做多变量分析时,移动均线的价位是几百上千的浮点数,RSI 却被锁在 0–100 区间,两者直接塞进同一个模型会互相碾压——大量级的列会吞掉小量级列的贡献,结论可能偏掉。 标准化做的就是逐列变换:令每列均值归零、标准差为 1。这样各列分布被拉平,可直接比较,也能让机器学习类模型在方差悬殊时更稳。 在主成分分析里这步几乎是强制的。PCA 吃的是协方差矩阵,对量纲差极敏感;不先标准化,第一个主成分往往被数值最大的那列支配,后续降维解释力会失真。 下面这段 MT5 矩阵代码演示了读 csv 后套 NORM_STANDARDIZATION:非标准化数据里血压收缩压在 125–133 波动,标准化后首行变成约 0.98 / 0.86 / 0.22 / 0.38,四列终于落在同尺度。外汇贵金属波动同样有量纲差,拿来跑 PCA 前务必先标准化,这类品种杠杆高、风险大。
<span class="keyword">matrix</span> Matrix = matrix_utiils.ReadCsv(<span class="class="type">class="kw">string">"bp data.csv"</span>); pre_processing = <span class="keyword">new</span> CPreprocessing(Matrix, NORM_STANDARDIZATION); CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) Non-Standardized data CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [[<span class="number">class="num">82.59999999999999</span>,<span class="number">class="num">132.1</span>,<span class="number">class="num">71</span>,<span class="number">class="num">172</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">79.09999999999999</span>,<span class="number">class="num">129.9</span>,<span class="number">class="num">79</span>,<span class="number">class="num">180</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">81.7</span>,<span class="number">class="num">131.2</span>,<span class="number">class="num">78</span>,<span class="number">class="num">172</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">80.7</span>,<span class="number">class="num">132.1</span>,<span class="number">class="num">66</span>,<span class="number">class="num">166</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">74.90000000000001</span>,<span class="number">class="num">125</span>,<span class="number">class="num">70</span>,<span class="number">class="num">173</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">79.09999999999999</span>,<span class="number">class="num">129.1</span>,<span class="number">class="num">64</span>,<span class="number">class="num">162</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">83.8</span>,<span class="number">class="num">133.1</span>,<span class="number">class="num">60</span>,<span class="number">class="num">164</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">78.40000000000001</span>,<span class="number">class="num">127</span>,<span class="number">class="num">67</span>,<span class="number">class="num">165</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">82.3</span>,<span class="number">class="num">131.6</span>,<span class="number">class="num">64</span>,<span class="number">class="num">164</span>] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [<span class="number">class="num">79.40000000000001</span>,<span class="number">class="num">129.2</span>,<span class="number">class="num">77</span>,<span class="number">class="num">179</span>]] CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) Standardized data CS <span class="number">class="num">0</span> <span class="number">class="num">10</span>:<span class="number">class="num">17</span>:<span class="number">class="num">31.956</span> PCA Test(NAS100,H1) [[<span class="number">class="num">0.979632638610581</span>,<span class="number">class="num">0.8604038253411385</span>,<span class="number">class="num">0.2240645398825688</span>,<span class="number">class="num">0.3760399462363875</span>]
「NAS100 的 PCA 投影样本怎么读」
下面这组日志是 PCA Test 在 NAS100 的 H1 周期上跑出的 9 行投影坐标,时间戳全是 10:17:31.957,说明是一次批量计算而非实时逐根。每行四个数对应四个主成分轴上的位置,正负只代表方向,绝对值大小才是该成分上的偏离强度。 第一行 [-0.449, -0.054, 1.504, 1.684] 里第三、四主成分都超过 1.5,说明这根样本在尾部成分上拉得很开;第四行 [-2.163, -2.091, 0.064, 0.540] 的前两个主成分同时落到 -2 附近,是整批里最极端的低值点。把这类极端行挑出来,往往对应行情结构突变的候选段。 外汇和贵金属做类似降维时,点差和跳空会让主成分不稳定,属高风险操作,结论只能当作概率参考。开 MT5 把这段日志贴进专家日志窗口对照 K 线,能直接看哪根 H1 棒被映射到远端。
CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.4489982926965129,-class="num">0.0540350228475094,class="num">1.504433339211528,class="num">1.684004976623816] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">0.6122703991316175,class="num">0.4863152056275964,class="num">1.344387239295408,class="num">0.3760399462363875] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">0.2040901330438764,class="num">0.8604038253411385,-class="num">0.5761659596980309,-class="num">0.6049338265541837] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">2.163355410265021,-class="num">2.090739730176784,class="num">0.06401843996644889,class="num">0.539535575034816] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.4489982926965129,-class="num">0.3865582403706605,-class="num">0.8962581595302708,-class="num">1.258916341747898] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">1.469448957915872,class="num">1.276057847245071,-class="num">1.536442559194751,-class="num">0.9319250841510407] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.7347244789579271,-class="num">1.259431686368917,-class="num">0.416119859781911,-class="num">0.7684294553526122] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">0.8571785587842599,class="num">0.6525768143891719,-class="num">0.8962581595302708,-class="num">0.9319250841510407] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.326544212870186,-class="num">0.3449928381802696,class="num">1.184341139379288,class="num">1.520509347825387]
◍ 调协方差矩阵时 rowvar 别填错
协方差矩阵衡量的是数据各列之间共同变化的程度,MT5 标准库已经封装好了现成函数,不需要自己手算公式。 关键在调用 Matrix.Cov() 时,rowvar 参数必须传 false。传 true 会按行向量算,4 列输入会吐出 8x8 方阵;传 false 才得到基于列的 4x4 单位阵结构,对角线理论值为 1。 上面那段日志是在 NAS100 的 H1 周期跑出来的真实输出:对角线全是 1.111111111111111,非对角有正有负,比如第 1、2 列协方差 1.0566,第 1、3 列是 -0.2882。说明这几列价格序列有不同程度的同向或反向漂移,外汇和贵金属品种也常出现类似结构,但杠杆高、跳空多,协方差估计可能随样本区间剧烈变动,验证时务必用近期行情。 下面这段代码就是直接拿列协方差的写法,复制到 MT5 脚本里就能看到你自己的矩阵。
matrix Cova = Matrix.Cov(class="kw">false); Print("Covariances\n", Cova); CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) Covariances CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [[class="num">1.111111111111111,class="num">1.05661579634328,-class="num">0.2881675653452953,-class="num">0.3314539233600543] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">1.05661579634328,class="num">1.111111111111111,-class="num">0.2164241126576326,-class="num">0.2333966556085017] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.2881675653452953,-class="num">0.2164241126576326,class="num">1.111111111111111,class="num">1.002480628180182] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.3314539233600543,-class="num">0.2333966556085017,class="num">1.002480628180182,class="num">1.111111111111111]] matrix matrix::Cov( class="kw">const class="type">bool rowvar=true class=class="str">"cmt">// rows or cols vectors of observations );
从协方差矩阵里抠出成分矩阵和特征向量
方阵 A 乘以非零向量 v 得到 λv,这个 v 就是特征向量,λ 是对应的特征值。MT5 标准库已经封装好,不需要自己推公式,直接调 matrix::Eig 就能拿到结果。 看 Eig 的接口,第一个出参 eigen_vectors 按注释说是特征向量矩阵,但在 MQL5 里它实际返回的是一个矩阵,所以我习惯把它存进 component_matrix(成分矩阵),避免和标量意义上的“向量”混淆。 下面这段是实际跑 NAS100 一小时周期 PCA 测试时的代码和日志。Eig 调用失败会打印提示;成功则用 Print 把成分矩阵和特征值都吐出来。 日志里成分矩阵是 4×4 的浮点阵列,例如首行 [-0.5276, 0.4599, 0.6994, -0.1450];特征值向量为 [2.6776, 1.6080, 0.0478, 0.1112],前两个值明显主导,说明前两主成分可能承担了绝大部分方差。外汇和贵金属品种用同样流程时波动结构不同,高风险下主成分切换可能更频繁,建议自己换品种验证。
if (!Cova.Eig(component_matrix, eigen_vectors)) Print("Failed to get the Component matrix matrix & Eigen vectors"); class="type">bool matrix::Eig( matrix& eigen_vectors, class=class="str">"cmt">// matrix of eigenvectors vector& eigen_values class=class="str">"cmt">// vector of eigenvalues ); Print("\nComponent matrix\n",component_matrix,"\nEigen Vectors\n",eigen_vectors); CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) Component matrix CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [[-class="num">0.5276049902734494,class="num">0.459884739531444,class="num">0.6993704635263588,-class="num">0.1449826035480651] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [-class="num">0.4959779194731578,class="num">0.5155907011803843,-class="num">0.679399121133044,class="num">0.1630612352922813] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">0.4815459137666799,class="num">0.520677926282417,-class="num">0.1230090303369406,-class="num">0.6941734714553853] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">0.4937128827246101,class="num">0.5015643052337933,class="num">0.184842006606018,class="num">0.6859404272536788]] CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) Eigen Vectors CS class="num">0 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1) [class="num">2.677561590453738,class="num">1.607960239905343,class="num">0.04775016337426833,class="num">0.1111724507110918]