数据处理的分组方法:用MQL5落地COMBI组合算法基础(基础篇)
(1/3)· 从单层网络与训练/测试切分讲清GMDH基础算法,避开多层迭代的认知坑
◍ 用组合算法给行情数据分组
在 MT5 里做价格行为分析时,常需要把一串历史数据按某种规则切分成若干组再统计。MQL5 的组合(combination)思路可以帮我们把 N 个样本里取 K 个的所有可能组合枚举出来,进而计算每组内的波动特征。 下面这段 MQL5 代码演示了最基础的组合生成逻辑,直接在脚本里跑就能看到控制台输出的分组结果。外汇与贵金属波动受杠杆与消息面影响,分组回测仅反映历史概率,实盘仍属高风险。 调用时把样本数组和要取的组合长度传进去,算法用递归向下钻取索引,避免重复选取同一位置的元素。你可以改 K 值观察不同分组粒度下波动率分布的差异。
class="type">void Combinations(class="type">int &arr[], class="type">int start, class="type">int k, class="type">int &temp[], class="type">int idx, class="type">int &result[][]){ if(idx == k){ class=class="str">"cmt">// 复制当前临时组合到结果 class="type">int r[]; ArrayResize(r, k); for(class="type">int i=class="num">0;i<k;i++) r[i]=temp[i]; ArrayAppend(result, r); class="kw">return; } for(class="type">int i=start;i<ArraySize(arr);i++){ temp[idx]=arr[i]; Combinations(arr, i+class="num">1, k, temp, idx+class="num">1, result); } }
COMBI 是怎么跑起来的
GMDH 里的组合算法(COMBI)是这套方法最基础的那一档,后面更复杂的变体都从它长出来。它和多层迭代算法(MIA)一样,吃的是一组变量的观测矩阵,先把样本劈成训练集和测试集两半。 训练子样本只干一件事:估多项式系数;测试子样本按某个准则挑出最小的那组模型结构,模型长什么样由它说了算。 这套计算过程在 MQL5 里是直接扩了前一篇的 GmdhModel 类来实现的,不是重写。后面还会补一个组合选择算法及其代码,最后拿比特币日线价跑一个预测模型收尾,外汇和贵金属用同思路前请记住:样本外失效是高概率事件,杠杆品种回撤可能很凶。
「COMBI 的单层组合建模逻辑」
MIA 靠多层网络堆结构,COMBI 反其道走单层:这一层里的节点数直接由输入变量个数决定,每个节点挂的是一个或多个候选模型。 以 2 个输入变量为例,COMBI 会把所有变量组合都拿来建候选模型,组合数按公式 C=2^m−1 算,m=2 时就是 3 个组合(x1、x2、x1+x2)。 每层节点上的候选模型形如 a = a1*x1 + a2*x2,a 是系数,x 是输入变量;系统实质是在解线性方程组估出 a1、a2。 模型好坏由性能指标说话,最终只留最能描述数据的那一个。外汇与贵金属行情高阶耦合强,单层组合容易漏掉非线性交叉,实盘前请在 MT5 用历史数据跑一遍确认过拟合程度。
◍ COMBI的线性骨架怎么搭
COMBI 算法在 MT5 里落地,核心是先继承一个线性模型中间层,再写具体的拟合逻辑。它最鲜明的特征是全程线性——没有非线性变换,预测就是输入向量和系数的矩阵乘法。 中间类 LinearModel 封装了这套通用线性逻辑,继承自 GmdhModel。它重写了 calculatePrediction,当输入维度不足时直接返回与输入列数等长的零向量,避免越界;否则在输入尾部补 1.0 构造偏置项,再按最优组合抽取特征列做 MatMul 得到预测值。 combi.mqh 里的 COMBI 类只需继承 LinearModel 并实现 fit() 即可复用上述预测通道。下面这段是 linearmodel.mqh 的关键代码,逐行看清楚它怎么把 x 扩维并乘系数: //+------------------------------------------------------------------+
| // | linearmodel.mqh |
|---|---|
| // | Copyright 2024, MetaQuotes Ltd. |
| // | [MQL5官方文档] |
//+------------------------------------------------------------------+ #property copyright "Copyright 2024, MetaQuotes Ltd." #property link "[MQL5官方文档] #include "gmdh.mqh" //+------------------------------------------------------------------+
| // | Class implementing the general logic of GMDH linear algorithms |
|---|
//+------------------------------------------------------------------+ class LinearModel:public GmdhModel { protected: vector calculatePrediction(vector& x) { if(x.Size()<ulong(inputColsNumber)) return vector::Zeros(ulong(inputColsNumber)); matrix modifiedX(1,x.Size()+ 1); modifiedX.Row(x,0); modifiedX[0][x.Size()] = 1.0; vector comb = bestCombinations[0][0].combination(); matrix xx(1,comb.Size()); for(ulong i = 0; i<xx.Cols(); ++i) xx[0][i] = modifiedX[0][ulong(comb[i])]; vector c,b; c = bestCombinations[0][0].bestCoeffs(); b = xx.MatMul(c); return b; } virtual matrix xDataForCombination(matrix& x, vector& comb) override { matrix out(x.Rows(), comb.Size()); for(ulong i = 0; i<out.Cols(); i++) out.Col(x.Col(int(comb[i])),i); return out; } virtual string getPolynomialPrefix(int levelIndex, int combIndex) override { return "y="; } virtual string getPolynomialVariable(int levelIndex, int coeffIndex, int coeffsNumber, vector& bestColsIndexes) override { return ((coeffIndex != coeffsNumber - 1) ? "x" + string(int(bestColsIndexes[coeffIndex]) + 1) : ""); } public:
class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| linearmodel.mqh | class=class="str">"cmt">//| Copyright class="num">2024, MetaQuotes Ltd. | class=class="str">"cmt">//| [MQL5官方文档] | class=class="str">"cmt">//+------------------------------------------------------------------+ class="macro">#class="kw">property copyright "Copyright class="num">2024, MetaQuotes Ltd." class="macro">#class="kw">property link "[MQL5官方文档] class="macro">#include "gmdh.mqh" class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Class implementing the general logic of GMDH linear algorithms | class=class="str">"cmt">//+------------------------------------------------------------------+ class LinearModel:class="kw">public GmdhModel { class="kw">protected: vector calculatePrediction(vector& x) { if(x.Size()<class="type">ulong(inputColsNumber)) class="kw">return vector::Zeros(class="type">ulong(inputColsNumber)); matrix modifiedX(class="num">1,x.Size()+ class="num">1); modifiedX.Row(x,class="num">0); modifiedX[class="num">0][x.Size()] = class="num">1.0; vector comb = bestCombinations[class="num">0][class="num">0].combination(); matrix xx(class="num">1,comb.Size()); for(class="type">ulong i = class="num">0; i<xx.Cols(); ++i) xx[class="num">0][i] = modifiedX[class="num">0][class="type">ulong(comb[i])]; vector c,b; c = bestCombinations[class="num">0][class="num">0].bestCoeffs(); b = xx.MatMul(c); class="kw">return b; } class="kw">virtual matrix xDataForCombination(matrix& x, vector& comb) class="kw">override { matrix out(x.Rows(), comb.Size()); for(class="type">ulong i = class="num">0; i<out.Cols(); i++) out.Col(x.Col(class="type">int(comb[i])),i); class="kw">return out; } class="kw">virtual class="type">class="kw">string getPolynomialPrefix(class="type">int levelIndex, class="type">int combIndex) class="kw">override { class="kw">return "y="; } class="kw">virtual class="type">class="kw">string getPolynomialVariable(class="type">int levelIndex, class="type">int coeffIndex, class="type">int coeffsNumber, vector& bestColsIndexes) class="kw">override { class="kw">return ((coeffIndex != coeffsNumber - class="num">1) ? "x" + class="type">class="kw">string(class="type">int(bestColsIndexes[coeffIndex]) + class="num">1) : ""); } class="kw">public:
COMBI 多步预测与组合择优的实现骨架
下面这段 predict 重写了基类方法,核心是把单步线性模型扩成多步:先校验 lags 必须为正整数,否则直接 Print 报错并返回长度为 1 的零向量;未训练完成时同理返回零向量,避免脏预测。
真正迭代发生在两个 for 循环:第一个把输入 x 拷进 expandedX 的前段,第二个按 lags 步数滚动调用 calculatePrediction,每步结果写回 expandedX 尾端,形成自回归式的多步前向推演。最后取 expandedX 末尾 lags 长度切片作为返回向量。
COMBI 类继承自 LinearModel,私有方法 getBest 负责从 CVector 组合池里挑最优:用 ArrayResize 把 best 与 proxys 拉到 combinations.size() 长度,再遍历把每个组合的索引与 evaluation() 分数分别存下,供后续择优。外汇与贵金属行情高波动,这类线性外推在多步预测中误差可能放大,实盘前务必在 MT5 用历史数据回测验证。
让小布替你跑这套:把 linearmodel.mqh 与 combi.mqh 放进 MQL5/Include,挂 EURUSD 的 H1 数据,把 lags 从 1 调到 5 看 expandedX 末端漂移幅度,能直观判断该模型在你品种上的衰减速度。
LinearModel(class="type">void) { } vector predict(vector& x, class="type">int lags) class="kw">override { if(lags <= class="num">0) { Print(__FUNCTION__," lags value must be a positive integer"); class="kw">return vector::Zeros(class="num">1); } if(!training_complete) { Print(__FUNCTION__," model was not successfully trained"); class="kw">return vector::Zeros(class="num">1); } vector expandedX = vector::Zeros(x.Size() + class="type">ulong(lags)); for(class="type">ulong i = class="num">0; i<x.Size(); i++) expandedX[i]=x[i]; for(class="type">int i = class="num">0; i < lags; ++i) { vector vect(x.Size(),slice,expandedX,class="type">ulong(i),x.Size()+class="type">ulong(i)-class="num">1); vector res = calculatePrediction(vect); expandedX[x.Size() + i] = res[class="num">0]; } vector vect(class="type">ulong(lags),slice,expandedX,x.Size()); class="kw">return vect; } }; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| combi.mqh | class=class="str">"cmt">//| Copyright class="num">2024, MetaQuotes Ltd. | class=class="str">"cmt">//| [MQL5官方文档] | class=class="str">"cmt">//+------------------------------------------------------------------+ class="macro">#class="kw">property copyright "Copyright class="num">2024, MetaQuotes Ltd." class="macro">#class="kw">property link "[MQL5官方文档] class="macro">#include "linearmodel.mqh" class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Class implementing combinatorial COMBI algorithm | class=class="str">"cmt">//+------------------------------------------------------------------+ class COMBI:class="kw">public LinearModel { class="kw">private: Combination getBest(CVector &combinations) { class="type">class="kw">double proxys[]; class="type">int best[]; ArrayResize(best,combinations.size()); ArrayResize(proxys,combinations.size()); for(class="type">int k = class="num">0; k<combinations.size(); k++) { best[k]=k; proxys[k]=combinations[k].evaluation();
「组合模型的拟合与剪枝落点」
COMBI 类在末尾把组合生成、剪枝和拟合接口收拢到一起。removeExtraCombinations 里先对 bestCombinations 逐层取 getBest,再对这批 top 取总最优,最后只保留一条 realBestCombinations,相当于把多层候选压成单层最优,避免组合爆炸拖垮 MT5 回测。 preparations 用 lastLevelEvaluation = DBL_MAX 初始化,并判断 level+1 是否小于特征列数,决定要不要继续往下层分裂;generateCombinations 直接调 nChooseK 按当前 level 出组合,逻辑和 GMDH 的层间扩展一致。 对外暴露的两个 fit 重载,一个吃单变量时间序列加 lags,一个吃多变量 matrix;前者默认 testsize=0.5,后者强制校验 vars.Cols()>=1 且行数与 targets 对齐,否则 Print 报错返回 false。外汇与贵金属行情用这套做特征组合拟合,过拟合和滑点风险偏高,参数建议先在历史数据小样本验证。 真正落地时,把 lags 从 1 往上调到 3~5,看 COMBI 在 EURUSD 的 M15 上测试集误差是收敛还是反弹,能直接判断该层剪枝阈值要不要改。
} MathQuickSortAscending(proxys,best,class="num">0,class="type">int(proxys.Size()-class="num">1)); class="kw">return combinations[best[class="num">0]]; } class="kw">protected: class="kw">virtual class="type">void removeExtraCombinations(class="type">void) class="kw">override { CVector2d realBestCombinations; CVector n; Combination top; for(class="type">int i = class="num">0 ; i<bestCombinations.size(); i++) { top = getBest(bestCombinations[i]); n.push_back(top); } top = getBest(n); CVector sorted; sorted.push_back(top); realBestCombinations.push_back(sorted); bestCombinations = realBestCombinations; } class="kw">virtual class="type">bool preparations(SplittedData &data, CVector &_bestCombinations) class="kw">override { lastLevelEvaluation = DBL_MAX; class="kw">return (bestCombinations.push_back(_bestCombinations) && class="type">ulong(level+class="num">1) < data.xTrain.Cols()); } class="type">void generateCombinations(class="type">int n_cols,vector &out[]) class="kw">override { GmdhModel::nChooseK(n_cols,level,out); class="kw">return; } class="kw">public: COMBI(class="type">void):LinearModel() { modelName = "COMBI"; } class="type">bool fit(vector &time_series,class="type">int lags,class="type">class="kw">double testsize=class="num">0.5,CriterionType criterion=stab) { if(lags < class="num">1) { Print(__FUNCTION__," lags must be >= class="num">1"); class="kw">return false; } PairMVXd transformed = timeSeriesTransformation(time_series,lags); SplittedData splited = splitData(transformed.first,transformed.second,testsize); Criterion criter(criterion); class="type">int pAverage = class="num">1; class="type">class="kw">double limit = class="num">0; class="type">int kBest = pAverage; if(validateInputData(testsize, pAverage, limit, kBest)) class="kw">return false; class="kw">return GmdhModel::gmdhFit(splited.xTrain, splited.yTrain, criter, kBest, testsize, pAverage, limit); } class="type">bool fit(matrix &vars,vector &targets,class="type">class="kw">double testsize=class="num">0.5,CriterionType criterion=stab) { if(vars.Cols() < class="num">1) { Print(__FUNCTION__," columns in vars must be >= class="num">1"); class="kw">return false; } if(vars.Rows() != targets.Size()) {