数据科学和机器学习(第 13 部分):配合主成分分析(PCA)改善您的金融市场分析·进阶篇
📉

数据科学和机器学习(第 13 部分):配合主成分分析(PCA)改善您的金融市场分析·进阶篇

(2/3)· 从标准化到成分提取,把高维行情特征压成可解释的低维信号

含代码示例 第 2/3 篇
把 RSI 和移动均线直接丢进模型,尺度差一个数量级,PCA 会被大方差指标牵着走。先标准化再降维,才是金融特征工程的常规操作。很多策略回测飘逸,问题就出在跳过了这步。

「一行矩阵乘法拿到 PCA 得分」

主成分得分的求解在代码层极简:把归一化后的原始数据矩阵与成分矩阵做乘法即可,不需要自己写循环。下例在 NAS100 的 H1 周期上跑出 10 行样本、4 个主成分的原始得分矩阵。 从日志看,首行得分为 [-0.6500472384886967, 1.199407986803537, 0.1425145462368588, 0.1006701620494091],末行则是 [1.664404875867474, 1.051245703485609, 0.1413817973120564, 0.2119289033750197],说明不同 K 线在四个主成分轴上的投影差异明显。 拿到得分后还不能直接喂给模型,得再做一次标准化。示例里用 CPreprocessing 以 NORM_STANDARDIZATION 模式处理,标准化后首行变成 [-0.4187491401035159, 0.9970295470975233, 0.68746486754918, 0.3182591681100855],量纲被压到可比区间。外汇与贵金属行情受事件驱动剧烈,PCA 降维仅描述历史协方差结构,对后市方向只具概率性参考。

MQL5 / C++
pca_scores = Matrix.MatMul(component_matrix);
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)    PCA SCORES
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)    [[-class="num">0.6500472384886967,class="num">1.199407986803537,class="num">0.1425145462368588,class="num">0.1006701620494091]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [class="num">1.819562596624738,class="num">1.393614599196321,-class="num">0.1510888243020112,class="num">0.1670753033981925]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [class="num">0.2688014256048517,class="num">1.420914385142756,class="num">0.001937917070391801,-class="num">0.6847663538666366]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [-class="num">1.110534258768705,-class="num">0.06593596223641518,-class="num">0.4827665581567511,class="num">0.09571954869438426]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [class="num">2.475561333978323,-class="num">1.768915328424386,-class="num">0.0006861487484489809,class="num">0.2983796568520111]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [-class="num">0.6245145789301378,-class="num">1.503882637300733,-class="num">0.1738415909335406,-class="num">0.2393186981373224]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [-class="num">2.608156175249579,class="num">0.0662886285379769,class="num">0.1774740257067155,class="num">0.4223436077935874]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [class="num">0.4325302694103054,-class="num">1.589321053467977,class="num">0.2509606394263523,-class="num">0.337079680008286]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [-class="num">1.667608250048573,-class="num">0.2034163217366656,class="num">0.09411419638842802,-class="num">0.03495245015036286]
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)     [class="num">1.664404875867474,class="num">1.051245703485609,class="num">0.1413817973120564,class="num">0.2119289033750197]]
  pre_processing = new CPreprocessing(pca_scores_standardized, NORM_STANDARDIZATION);
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)    PCA SCORES | STANDARDIZED
CS    class="num">0    class="num">10:class="num">17:class="num">31.957    PCA Test(NAS100,H1)    [[-class="num">0.4187491401035159,class="num">0.9970295470975233,class="num">0.68746486754918,class="num">0.3182591681100855]

◍ NAS100小时图的主成分投影样本

在 NAS100 的 H1 周期上跑主成分分析,同一毫秒(10:17:31.957)输出了 9 组四维投影向量,说明降维计算是批量完成的,不是逐根 K 线触发。 看前两组:第一组是 [1.172, 1.158, -0.729, 0.528],第二组骤变为 [0.173, 1.181, 0.009, -2.165]。第三主成分从 -0.729 跳到 0.009,第四主成分从 0.528 砸到 -2.165,波动幅度超过 2 个标准差,说明样本窗口内尾部结构变化很陡。 外汇和贵金属做类似降维时,这种陡变往往对应流动性切换或数据窗口重组,属高风险场景,不能直接当方向信号。 开 MT5 把这段日志贴进专家日志窗口,对照自己 PCA 函数的 Print 格式,确认第四主成分绝对值大于 2 的频次,再决定是否拿来做特征过滤。

MQL5 / C++
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [class="num">1.172130620033975,class="num">1.15846730049564,-class="num">0.7288256625700642,class="num">0.528192723531639]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [class="num">0.1731572094549987,class="num">1.181160740523977,class="num">0.009348167869829477,-class="num">2.164823873278453]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [-class="num">0.715386880184365,-class="num">0.05481045923432144,-class="num">2.328780161211247,class="num">0.3026082735855334]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [class="num">1.594713612332284,-class="num">1.470442808583469,-class="num">0.003309859736641006,class="num">0.9432989819176616]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [-class="num">0.4023014443028848,-class="num">1.250129598312728,-class="num">0.8385809690405054,-class="num">0.7565833632510734]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [-class="num">1.68012890598631,class="num">0.05510361946569121,class="num">0.8561031894464458,class="num">1.335199254045385]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [class="num">0.2786284867625921,-class="num">1.321151824538665,class="num">1.210589566461227,-class="num">1.06564543418136]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [-class="num">1.074244269325531,-class="num">0.1690934905926844,class="num">0.4539901733759543,-class="num">0.1104988556867913]
CS       class="num">0    class="num">10:class="num">17:class="num">31.957   PCA Test(NAS100,H1)   [class="num">1.072180711318756,class="num">0.8738669736790375,class="num">0.6820006878558206,class="num">0.6699931252073736]

从 PCA 分数里抠出主成分

拿到 PCA 分数后,真正的目标是提取主成分。非标准化分数的系数其实就是各 PCA 分数列自身的方差,算出来就能知道每个成分扛了多少波动。 选成分不是拍脑袋,常见有五套准则:特征值最大、累计方差比例、碎石图拐点、凯撒(特征值>1)、交叉验证。本库只实装了方差比例、凯撒、碎石图三种,理由是计算更轻、在 MT5 上跑起来不卡。 方差比例准则默认截在 90% 以上——比如 NAS100 的 H1 测试里,四个原始变量的 PCA 分数系数为 [2.4098, 1.4472, 0.0430, 0.1001],前两个就覆盖了约 94% 的总方差,4 维直接压到 2 维。凯撒准则则只留系数大于均值(此处均值约 0.999)的那些列。 提取函数的核心是先算百分比向量,再按你传的枚举走分支。下面这段代码是系数计算和 ExtractComponents 的骨架,注意 variance 那行就是系数的来源。

MQL5 / C++
  pca_scores_coefficients.Resize(cols);
  vector v_row;

  for (class="type">class="kw">ulong i=class="num">0; i<cols; i++)
    {
      v_row = pca_scores.Col(i);

      pca_scores_coefficients[i] = v_row.Var(); class=class="str">"cmt">//variance of the pca scores
    }
class="num">2023.02.class="num">25 class="num">10:class="num">17:class="num">31.957 PCA Test(NAS100,H1)    SCORES COEFF [class="num">2.409805431408367,class="num">1.447164215914809,class="num">0.04297514703684173,class="num">0.1000552056399828]
enum criterion
  {
    CRITERION_VARIANCE,
    CRITERION_KAISER,
    CRITERION_SCREE_PLOT
  };
matrix Cpca::ExtractComponents(criterion CRITERION_)
{
  vector vars = pca_scores_coefficients;  
  vector vars_percents = (vars/(class="type">class="kw">double)vars.Sum())*class="num">100.0;

class=class="str">"cmt">//--- for Kaiser
  class="type">class="kw">double vars_mean = pca_scores_coefficients.Mean();
class=class="str">"cmt">//--- for scree
   class="type">class="kw">double x[], y[];

class=class="str">"cmt">//---
  matrix PCAS = {};

  class="type">class="kw">double sum=class="num">0;
  class="type">class="kw">ulong  max;
  vector v_cols = {};

   class="kw">switch(CRITERION_)
    {
      case  CRITERION_VARIANCE:        
       class="macro">#ifdef DEBUG_MODE
        Print("vars percentages ",vars_percents);      
       class="macro">#endif 

        for (class="type">int i=class="num">0, count=class="num">0; i<(class="type">int)cols; i++)
          {
            count++;

            max = vars_percents.ArgMax();
            sum += vars_percents[max];

            vars_percents[max] = class="num">0;

            v_cols.Resize(count);
            v_cols[count-class="num">1] = (class="type">int)max;

            if (sum >= class="num">90.0)
        class="kw">break;
          }

        PCAS.Resize(rows, v_cols.Size());

        for (class="type">class="kw">ulong i=class="num">0; i<v_cols.Size(); i++)
          PCAS.Col(pca_scores.Col((class="type">class="kw">ulong)v_cols[i]), i);

        class="kw">break;
      case  CRITERION_KAISER:

      class="macro">#ifdef DEBUG_MODE

「用碎石图挑主成分与均值阈值截尾」

PCA 降维后有两种常见保留主成分的做法:一是按系数均值做阈值过滤,二是画碎石图(Scree Plot)肉眼看拐点。代码里 CRITERION_SCREE_PLOT 分支先把 pca_scores_coefficients 升序排再反转成降序,用 ScatterCurvePlots 画出「方差—PCA序号」散点,便于判断第几主成分后斜率变平。 均值截尾那段逻辑很直接:遍历 cols 个系数,凡是大于 vars_mean 的就 count++,并把对应列从 pca_scores 拷进 PCAS 矩阵,最后 Resize 到 rows×count。注意循环里 Resize 每次都按当前 count 重设列数,属于边扫边扩,MT5 里对中小样本量可以这么写,大矩阵建议先算好 count 再一次性分配。 实跑日志里 NAS100 H1 的 PCA Test 输出前五行为:[[-0.65,1.199],[1.819,1.393],[0.268,1.420],[-1.110,-0.065],[2.475,-1.768]],说明前两主成分已承载主要方差结构。外汇与贵金属用同样流程时波动更剧烈,主成分切换可能更频繁,属高风险场景,截尾阈值建议用样本内回测调,而非固定拍脑袋。 把 vars_percents 那行加上去就能直接看累计贡献率:vars/(double)vars.Sum()*100.0 把系数转成百分比,配合碎石图能更快定保留几维。开 MT5 把这段接在你自己的 PCA 函数尾端,跑一遍 EURUSD M15 对比 NAS100 的 count 差异,会比只读文档直观。

MQL5 / C++
Print("var ",vars," scores mean ",vars_mean);
class="macro">#endif

 vars = pca_scores_coefficients;
 for (class="type">class="kw">ulong i=class="num">0, count=class="num">0; i<cols; i++)
  if (vars[i] > vars_mean)
   {
    count++;

    PCAS.Resize(rows, count);

    PCAS.Col(pca_scores.Col(i), count-class="num">1);
   }

  class="kw">break;
 case  CRITERION_SCREE_PLOT:
   v_cols.Resize(cols);
   for (class="type">class="kw">ulong i=class="num">0; i<v_cols.Size(); i++)
     v_cols[i] = (class="type">int)i+class="num">1;


    vars = pca_scores_coefficients;
    SortAscending(vars); class=class="str">"cmt">//Make sure they are in ascending first order
    ReverseOrder(vars);  class=class="str">"cmt">//Set them to descending order

    VectorToArray(v_cols, x);
    VectorToArray(vars, y);

    plt.ScatterCurvePlots("Scree plot",x,y,"variance","PCA","Variance");
class=class="str">"cmt">//---
 vars = pca_scores_coefficients;
 for (class="type">class="kw">ulong i=class="num">0, count=class="num">0; i<cols; i++)
  if (vars[i] > vars_mean)
   {
    count++;
    PCAS.Resize(rows, count);
    PCAS.Col(pca_scores.Col(i), count-class="num">1);
   }

    class="kw">break;
  }
 class="kw">return (PCAS);
}
 vector vars = pca_scores_coefficients;
 vector vars_percents = (vars/(class="type">class="kw">double)vars.Sum())*class="num">100.0;

◍ NAS100 的 PCA 投影日志长这样

在 MT5 用 EA 跑主成分分析(PCA)时,专家日志里会直接吐出每个 K 线在降维空间的坐标。上面这段是 NAS100 的 H1 周期、同一毫秒(12:03:49.579)连续打印的投影样本,第一主成分 PC1 跨度从 -2.608 到 +2.475,第二主成分 PC2 从 -1.768 到 +1.420。 注意日志里出现了两次完全相同的五行坐标块,说明脚本对窗口做了重复计算或回环打印,并不是新数据。做成分监控时,这种重复行要先在代码层去重,否则会把样本量虚增一倍。 PC1 绝对值最大的点 [-2.608, 0.066] 和 [2.475, -1.768] 分别代表该窗口内偏离均值最远的两个状态,后续做聚类或异常检测时,这两个向量值得单独拉出来看。外汇与贵金属品种的高杠杆属性意味着这类统计特征只描述历史分布,实盘信号仍可能失效。

MQL5 / C++
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">0.6245145789301378,-class="num">1.503882637300733]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">2.608156175249579,class="num">0.0662886285379769]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">0.4325302694103054,-class="num">1.589321053467977]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">1.667608250048573,-class="num">0.2034163217366656]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">1.664404875867474,class="num">1.051245703485609]]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  PCA&class="macro">#x27;S
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [[-class="num">0.6500472384886967,class="num">1.199407986803537]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">1.819562596624738,class="num">1.393614599196321]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">0.2688014256048517,class="num">1.420914385142756]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">1.110534258768705,-class="num">0.06593596223641518]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">2.475561333978323,-class="num">1.768915328424386]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">0.6245145789301378,-class="num">1.503882637300733]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">2.608156175249579,class="num">0.0662886285379769]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">0.4325302694103054,-class="num">1.589321053467977]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [-class="num">1.667608250048573,-class="num">0.2034163217366656]
CS      class="num">0      class="num">12:class="num">03:class="num">49.579    PCA Test(NAS100,H1)  [class="num">1.664404875867474,class="num">1.051245703485609]]

用 PCA 给十个振荡器瘦身

在 MT5 里堆了十种振荡器(ATR、熊市力量、MACD、柴金、CCI、DeMarker、推动力、动量、RSI、WPR),肉眼看线形高度重叠,但相关矩阵显示多数两两相关系数偏低——例如 ATR 与 MACD 仅 -0.0165,ATR 与 WPR 也才 0.3071。这说明它们虽同属振荡类,信息冗余没想象中高。 对 NAS100 的 H1 图表跑一遍相关矩阵后套用主成分分析,碎石图准则只挑出 3 个主成分,就把原本 10 个变量压到 3 个。变量数量砍掉七成,EA 的计算开销会明显降下来,这是实打实能省的资源。 下面这段是抓取十个指标缓冲并算相关矩阵的核心代码,直接在 MT5 脚本里改 period 和 bars 就能复现: 别把指标多当信息全。图表上叠一堆同类型振荡器,大概率只是视觉噪音;PCA 降维后留 3 个主成分,就够覆盖大部分波动结构,外汇和贵金属波动受消息面突袭,高风险下少算点反而更稳。

MQL5 / C++
  handles[class="num">0] = iATR(Symbol(),PERIOD_CURRENT, period);
  handles[class="num">1] = iBearsPower(Symbol(), PERIOD_CURRENT, period);
  handles[class="num">2] = iMACD(Symbol(),PERIOD_CURRENT,class="num">12, class="num">26,class="num">9,PRICE_CLOSE);
  handles[class="num">3] = iChaikin(Symbol(), PERIOD_CURRENT,class="num">12,class="num">26,MODE_SMMA,VOLUME_TICK);
  handles[class="num">4] = iCCI(Symbol(),PERIOD_CURRENT,period, PRICE_CLOSE);
  handles[class="num">5] = iDeMarker(Symbol(),PERIOD_CURRENT,period);
  handles[class="num">6] = iForce(Symbol(),PERIOD_CURRENT,period,MODE_EMA,VOLUME_TICK);
  handles[class="num">7] = iMomentum(Symbol(),PERIOD_CURRENT,period, PRICE_CLOSE);
  handles[class="num">8] = iRSI(Symbol(),PERIOD_CURRENT,period,PRICE_CLOSE);
  handles[class="num">9] = iWPR(Symbol(),PERIOD_CURRENT,period);


  for (class="type">int i=class="num">0; i<class="num">10; i++)
  {
    matrix_utiils.CopyBufferVector(handles[i],class="num">0,class="num">0,bars,buff_v);
    ind_Matrix.Col(buff_v, i); class=class="str">"cmt">//store each indicator in ind_matrix columns
  }
    Print("Oscillators Correlation Matrix\n",ind_Matrix.CorrCoef(class="kw">false));
把特征缩放交给小布
这些诊断小布盯盘的 AIGC 已内置,打开对应品种页即可看到多指标的标准化分布与协方差热力,你只需判断保留哪几个主成分。

常见问题

因为 PCA 基于协方差矩阵,对变量尺度和方差差敏感;不标准化时大量级指标会主导主成分,导致降维结果失真。
不需要,标准库已提供协方差函数,直接对数据矩阵的各列调用即可,输出为对称矩阵,对角线是各自方差。
可以,小布盯盘对应品种页内置了多指标尺度诊断,能提示哪些序列需要先标准化,避免人工漏步。
交易里准确度不等于盈利,低维特征更易可视化和提速机器学习,少量信息损失换来的简洁性往往更实用。
得分是样本在主成分轴上的投影坐标,成分指特征向量方向;前者用于重构或建模输入,后者定义降维空间。