数据科学和机器学习(第 12 部分):自训练神经网络能否帮助您跑赢股市?·综合运用
📘

数据科学和机器学习(第 12 部分):自训练神经网络能否帮助您跑赢股市?·综合运用

第 3/3 篇

把特征压进 [0,1] 的最小-最大缩放

最小-最大缩放把任意数值特征线性映射到闭区间 [0,1],公式是 x_norm = (x - x_min) / (x_max - x_min)。x_min、x_max 取该特征在样本里的最小和最大值,映射后极值必定落在 0 和 1,不会破坏原始分布的单调性。 在 MT5 的回归神经网络类里,规范化方法由 norm_technique 枚举控制,传 NORM_MIN_MAX_SCALER 即启用本方法。下面这段构造调用里,学习率 0.01、训练 1000 轮、激活用 ReLU、损失用 MSE,并显式指定了最小-最大缩放。 [CODE] CRegNeuralNets::CRegNeuralNets(matrix &xmatrix, vector &yvector,double alpha, uint epochs, activation ACTIVATION_FUNCTION, loss LOSS_FUNCTION, norm_technique NORM_METHOD) enum norm_technique { NORM_MIN_MAX_SCALER, //Min max scaler NORM_MEAN_NORM, //Mean normalization NORM_STANDARDIZATION, //standardization NORM_NONE //Do not normalize. }; nn = new CRegNeuralNets(x_matrix,y_vector,0.01,1000, AF_RELU_, LOSS_MSE_,NORM_MIN_MAX_SCALER); CS 0 22:40:56.457 Self Trained NN EA (NAS100,M30) [ 1/1000 ] Loss = 0.19379434 | accuracy -0.581 CS 0 22:40:56.457 Self Trained NN EA (NAS100,M30) [ 2/1000 ] Loss = 0.07735744 | accuracy 0.369 CS 0 22:40:56.458 Self Trained NN EA (NAS100,M30) [ 3/1000 ] Loss = 0.04761891 | accuracy 0.611 CS 0 22:40:56.458 Self Trained NN EA (NAS100,M30) [ 4/1000 ] Loss = 0.03559318 | accuracy 0.710 CS 0 22:40:56.458 Self Trained NN EA (NAS100,M30) [ 5/1000 ] Loss = 0.02937830 | accuracy 0.760 CS 0 22:40:56.458 Self Trained NN EA (NAS100,M30) [ 6/1000 ] Loss = 0.02582918 | accuracy 0.789 CS 0 22:40:56.459 Self Trained NN EA (NAS100,M30) [ 7/1000 ] Loss = 0.02372224 | accuracy 0.806 CS 0 22:40:56.459 Self Trained NN EA (NAS100,M30) [ 8/1000 ] Loss = 0.02245222 | accuracy 0.817 CS 0 22:40:56.460 Self Trained NN EA (NAS100,M30) [ 9/1000 ] Loss = 0.02168207 | accuracy 0.823 CS 0 22:40:56.460 Self Trained NN EA (NAS100,M30 [/CODE] 上面日志是 NAS100 的 M30 图表上跑出来的前 9 轮:损失从 0.1938 降到 0.0217,准确率从 -0.581 爬到 0.823。最终训练集准确率 82.3%,在 M30 上倾向给出可用输出;但日线测出 -77%、H4 甚至返回 -11234 这类离谱值,说明单层增量规则网络对不同周期形态捕捉能力不一致。 别把正态当圣经 最小-最大缩放不是万能药。它在 M30 上表现稳,换日线或 H4 就崩,外汇和贵金属这类高波动品种尤其容易放大样本极值偏差,实盘前务必自己切周期复测。

MQL5 / C++
CRegNeuralNets::CRegNeuralNets(matrix &xmatrix, vector &yvector,class="type">class="kw">double alpha, class="type">uint epochs, activation ACTIVATION_FUNCTION, loss LOSS_FUNCTION, norm_technique NORM_METHOD)
enum norm_technique
{
   NORM_MIN_MAX_SCALER, class=class="str">"cmt">//Min max scaler
   NORM_MEAN_NORM,  class=class="str">"cmt">//Mean normalization
   NORM_STANDARDIZATION, class=class="str">"cmt">//standardization
   NORM_NONE    class=class="str">"cmt">//Do not normalize.
};
   nn = new CRegNeuralNets(x_matrix,y_vector,class="num">0.01,class="num">1000, AF_RELU_, LOSS_MSE_,NORM_MIN_MAX_SCALER);
CS       class="num">0      class="num">22:class="num">40:class="num">56.457    Self Trained NN EA(NAS100,M30) [ class="num">1/class="num">1000 ] Loss = class="num">0.19379434 | accuracy -class="num">0.581
CS       class="num">0      class="num">22:class="num">40:class="num">56.457    Self Trained NN EA(NAS100,M30) [ class="num">2/class="num">1000 ] Loss = class="num">0.07735744 | accuracy class="num">0.369
CS       class="num">0      class="num">22:class="num">40:class="num">56.458    Self Trained NN EA(NAS100,M30) [ class="num">3/class="num">1000 ] Loss = class="num">0.04761891 | accuracy class="num">0.611
CS       class="num">0      class="num">22:class="num">40:class="num">56.458    Self Trained NN EA(NAS100,M30) [ class="num">4/class="num">1000 ] Loss = class="num">0.03559318 | accuracy class="num">0.710
CS       class="num">0      class="num">22:class="num">40:class="num">56.458    Self Trained NN EA(NAS100,M30) [ class="num">5/class="num">1000 ] Loss = class="num">0.02937830 | accuracy class="num">0.760
CS       class="num">0      class="num">22:class="num">40:class="num">56.458    Self Trained NN EA(NAS100,M30) [ class="num">6/class="num">1000 ] Loss = class="num">0.02582918 | accuracy class="num">0.789
CS       class="num">0      class="num">22:class="num">40:class="num">56.459    Self Trained NN EA(NAS100,M30) [ class="num">7/class="num">1000 ] Loss = class="num">0.02372224 | accuracy class="num">0.806
CS       class="num">0      class="num">22:class="num">40:class="num">56.459    Self Trained NN EA(NAS100,M30) [ class="num">8/class="num">1000 ] Loss = class="num">0.02245222 | accuracy class="num">0.817
CS       class="num">0      class="num">22:class="num">40:class="num">56.460    Self Trained NN EA(NAS100,M30) [ class="num">9/class="num">1000 ] Loss = class="num">0.02168207 | accuracy class="num">0.823
CS       class="num">0      class="num">22:class="num">40:class="num">56.460    Self Trained NN EA(NAS100,M30

◍ 自训练神经网络只在首 tick 跑一次

一段真实 EA 日志能说明问题:NAS100 的 M30 图上,网络用 1000/1000 样本训完,Loss 落到 0.02046533,训练集准确率 0.833。这种自训练 NN 模型在外汇与贵金属上属于高风险尝试,样本内表现好不等于样本外能持续。 控制训练频数是关键。OnTick 里用 train_nn 布尔锁,第一次进来才调用 TrainNetwork(),随后立刻置 true,后续 tick 不再重训,避免每笔报价都烧 CPU 重算梯度。 TrainNetwork 的拼装方式值得照抄:x_matrix 取两个品种的同周期收盘价各填一列,y_vector 取当前品种收盘价作标签,学习率 0.01、迭代 1000 次、ReLU 激活、MSE 损失、最小最大归一化。开 MT5 把 symbol_x2 换成和你主品种有领先关系的标的,就能复现这套输入结构。

MQL5 / C++
class="type">void OnTick()
  {
class=class="str">"cmt">//---
    if (!train_nn)
      TrainNetwork(); class=class="str">"cmt">//Train the network only once
    train_nn = true;
     
  }
class=class="str">"cmt">//+------------------------------------------------------------------+
class=class="str">"cmt">//|                                                                  |
class=class="str">"cmt">//+------------------------------------------------------------------+
class="type">void TrainNetwork()
{
   matrix x_matrix(n_samples,class="num">2); vector y_vector;
   vector x_vector;

   x_vector.CopyRates(symbol_x,PERIOD_CURRENT,copy_rates_x,class="num">0,n_samples);
   x_matrix.Col(x_vector, class="num">0);
   x_vector.CopyRates(symbol_x2, PERIOD_CURRENT,copy_rates_x,class="num">0,n_samples);
   x_matrix.Col(x_vector, class="num">1);

   y_vector.CopyRates(Symbol(), PERIOD_CURRENT,COPY_RATES_CLOSE,class="num">0,n_samples);

   nn = new CRegNeuralNets(x_matrix,y_vector,class="num">0.01,class="num">1000, AF_RELU_, LOSS_MSE_,NORM_MIN_MAX_SCALER);
}

「随机打散才不会被K线顺序带偏」

做过 Python 机器学习的人对 sklearn 的 train_test_split 不陌生。拆数据不光是分训练集和测试集,核心是把样本顺序随机化,否则模型会把「数据按时间先后排列」这种结构噪声当成规律学进去。 神经网络抓的是变量之间的形态,不是样本的组织方式。若直接拿原始顺序的行情矩阵去拟合,模型可能只是在拟合排列惯性,换一段行情就失效。外汇与贵金属波动受宏观与流动性冲击,顺序相关伪形态尤其多,这类过拟合风险很高。 MQL5 里收集完特征后,用 TrainTestSplitMatrices 在随机状态 42 下做切分,70% 训练、30% 测试。随机状态固定成 42 是为了可复现,你改这个数,训练集样本就换一批,回测结果也可能跟着飘。 下面这段是实际调用方式,注意 Matrix 前三列分别是两个符号的某速率特征和当前品种收盘价,切分后丢进 CRegNeuralNets 做回归,最后用 r_squared 打测试集分数。

MQL5 / C++
class="type">void TrainNetwork()
 {
class=class="str">"cmt">//--- collecting the data
  matrix Matrix(n_samples,class="num">3); vector y_vector;
  vector x_vector; 
  
  x_vector.CopyRates(symbol_x,PERIOD_CURRENT,copy_rates_x,class="num">0,n_samples);
  Matrix.Col(x_vector, class="num">0); 
  x_vector.CopyRates(symbol_x2, PERIOD_CURRENT,copy_rates_x,class="num">0,n_samples);
  Matrix.Col(x_vector, class="num">1); 
  
  y_vector.CopyRates(Symbol(), PERIOD_CURRENT,COPY_RATES_CLOSE,class="num">0,n_samples);
  Matrix.Col(y_vector, class="num">2);
  
class=class="str">"cmt">//---
  matrix x_train, x_test; vector y_train, y_test;
  
  matrix_utils.TrainTestSplitMatrices(Matrix, x_train, y_train, x_test, y_test, class="num">0.7, class="num">42);
  
  nn = new CRegNeuralNets(x_train,y_train,class="num">0.01,class="num">1000, AF_RELU_, LOSS_MSE_,NORM_MIN_MAX_SCALER);
  
  vector test_pred = nn.ForwardPass(x_test);
    
  printf("Testing Accuracy =%.3f",metrics.r_squared(y_test, test_pred));
 }
class="type">void TrainTestSplitMatrices(matrix &matrix_,matrix &x_train,vector &y_train,matrix &x_test, vector &y_test,class="type">class="kw">double train_size=class="num">0.7,class="type">int random_state=-class="num">1)

在 OnTick 里喂数据给神经网络

实时预测的关键,是把每个 tick 到来的最新行情塞进已经训练好的网络输入向量,而不是等一根柱子走完才动手。上面这段逻辑直接在 OnTick 内完成采集与前向推算,省掉了任何中间缓存。 代码里先用 CopyRates 只取当前蜡烛的 x1、x2 两个标的数值(示例为 Apple 与 Tesla),拼成长度为 2 的 inputs 向量,再丢给 nn.ForwardPass 得到纳斯达克预测值。注意 train_nn 标志位保证 TrainNetwork 只跑一次,后续 tick 不再重训,避免重复消耗。 你在 MT5 里验证时,把 symbol_x / symbol_x2 换成自己盯的跨市场品种,copy_rates_x 对应特征索引,Comment 输出的 pred 就是该 tick 的网络输出。外汇与贵金属叠加此类跨资产模型,杠杆与滑点会放大误差,信号仅作概率参考,实盘前务必用历史 tick 回测。

MQL5 / C++
class="type">void OnTick()
  {
class=class="str">"cmt">//---
    if (!train_nn)
      TrainNetwork(); class=class="str">"cmt">//Train the network only once
    train_nn = true; 
    
    vector x1, x2;
    
    x1.CopyRates(symbol_x,PERIOD_CURRENT,copy_rates_x,class="num">0,class="num">1); class=class="str">"cmt">//only the current candle
    x2.CopyRates(symbol_x2,PERIOD_CURRENT,copy_rates_x,class="num">0,class="num">1); class=class="str">"cmt">//only the current candle
    
    vector inputs = {x1[class="num">0], x2[class="num">0]}; class=class="str">"cmt">//current values of x1 and x2 instruments | Apple & Tesla
    
    matrix OUT = nn.ForwardPass(inputs); class=class="str">"cmt">//Predicted Nasdaq value
    
    class="type">class="kw">double pred = OUT[class="num">0][class="num">0];
    
    Comment("pred ",OUT);
 }

◍ 神经网络预测价驱动的开平仓逻辑

这套自训练 EA 的核心判断只有一条:模型输出的预测价 pred 高于实时卖价 ticks.bid,就倾向做多;低于实时买价 ticks.ask,就倾向做空。止盈与止损不是固定点数,而是用预测价与当前价的差 target_gap 乘以外部输入系数再换算成 Point 小数位。 开仓前有一道硬门槛:MathAbs(pred - ticks.ask) 加上点差 spread 必须大于经纪商规定的止损最小距离 stops_level,否则整段逻辑直接跳过。这一步能避免订单因距离不足被拒,外汇与贵金属品种在高波动时 stops_level 可能瞬间拉大,需留意。 多单的止损挂在 ticks.bid 减去 target_gap*stop_loss 个 Point,止盈挂在 ticks.bid 加上 target_gap*take_profit 个 Point;空单则对称反向。PosExist 检查确保同方向不重复加仓。 下面这段代码就是 MetaEditor 里的实际写法,可直接复制到 EA 的 OnTick 里验证:先取 stops_level、最小手数、点差和最新 tick,再走上面的判断分支。

MQL5 / C++
    stops_level = (class="type">int)SymbolInfoInteger(Symbol(),SYMBOL_TRADE_STOPS_LEVEL);
    Lots = SymbolInfoDouble(Symbol(),SYMBOL_VOLUME_MIN);
    spread = (class="type">class="kw">double)SymbolInfoInteger(Symbol(), SYMBOL_SPREAD);
    
    class="type">MqlTick ticks;
    SymbolInfoTick(Symbol(), ticks);
    
    if (MathAbs(pred - ticks.ask) + spread > stops_level)
      {
        if (pred > ticks.ask && !PosExist(POSITION_TYPE_BUY))
          {
            target_gap  = pred - ticks.bid;
            
            m_trade.Buy(Lots, Symbol(), ticks.ask, ticks.bid - ((target_gap*stop_loss) * Point()) , ticks.bid + ((target_gap*take_profit) * Point()),"Self Train NN | Buy");
          }
        
        if (pred < ticks.bid && !PosExist(POSITION_TYPE_SELL))
          {
            target_gap = ticks.ask - pred;
            
            m_trade.Sell(Lots, Symbol(), ticks.bid, ticks.ask + ((target_gap*stop_loss) * Point()), ticks.ask - ((target_gap*take_profit) * Point()), "Self Train NN | Sell");
          } 
      }

「回测里的训练与测试准确率陷阱」

在 MT5 策略测试器里跑机器学习 EA,最容易被忽略的是训练集与测试集的质量差异。我曾用 NAS100 的 4 小时图,基于真实跳价回测 2023.01.01 至 2023.02.23,把模型训练函数挂在测试生命周期第一根跳价上,训练和推断几乎瞬时完成。 第一轮结果很反直觉:训练准确率仅 37.9%,测试准确率却冲到 71.7%。这种倒挂通常指向训练数据有洞——每一段缺失的行情都可能让模型学偏。该 EA 最终 78.27% 的平仓交易以亏损收场,在没动止损止盈前,这概率说明模型本身扛不住实盘噪声。 随后做了一轮短优化,挑出这组参数:copy_rates_x 用 COPY_RATES_LOW、n_samples=2950、滑点=1、止损=7.4、止盈=5.0。重跑后训练准确率 61.5%、测试准确率 63.5%,两者贴近才像能用的模型。外汇与贵金属杠杆高,回测顺滑不代表实盘能复制,开 MT5 用真实跳价复一遍再信。 下面这段测试器日志是初始烂参数的现场:copy_rates_x=1、n_samples=200、滑点=100、止损/止盈都=2.0,还出现了单日 1379 根分钟棒里缺 2 分钟真实跳价。这种数据洞正是准确率倒挂的温床。

MQL5 / C++
CS        class="num">0      class="num">15:class="num">50:class="num">47.676    Tester  NAS100,H4(Pepperstone-Demo): generating based on real ticks
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester  NAS100,H4: testing of Experts\Advisors\Self Trained NN EA.ex5 from class="num">2023.01.class="num">01 class="num">00:class="num">00 to class="num">2023.02.class="num">23 class="num">00:class="num">00 started with inputs:
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      symbol_x=Apple_Inc_(AAPL.O)
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      symbol_x2=Tesco_(TSCO.L)
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      copy_rates_x=class="num">1
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      n_samples=class="num">200
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      =
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      slippage=class="num">100
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      stop_loss=class="num">2.0
CS        class="num">0      class="num">15:class="num">50:class="num">47.677    Tester      take_profit=class="num">2.0
CS        class="num">3      class="num">15:class="num">50:class="num">49.209    Ticks   NAS100 : class="num">2023.02.class="num">21 class="num">23:class="num">59 - real ticks absent for class="num">2 minutes out of class="num">1379 total minute bars within a day
CS        class="num">0      class="num">15:class="num">50:class="num">51.466    History Tesco_(TSCO.L),H4: history begins from class="num">2022.01.class="num">04 class="num">08:class="num">00
CS        class="num">0      class="num">15:class="num">50:class="num">51.467    Self Trained NN EA(NAS100,H4)  class="num">2023.01.class="num">03 class="num">01:class="num">00:class="num">00  [ class="num">1/class="num">1000 ] Loss = class="num">0.14025037 | accuracy -class="num">1.524

NAS100 自训练神经网络的收敛轨迹

在 MT5 策略测试器里跑自训练神经网络 EA(标的 NAS100、周期 H4、回测起点 2023.01.03 01:00),前 10 个 epoch 的日志已经能看出训练动态。第 2 轮 Loss 为 0.05244676、准确率仅 0.056,到第 10 轮 Loss 降到 0.03498872、准确率升到 0.370。 整轮 1000 次迭代跑完,最终 Loss 停在 0.03452066,准确率 0.379。也就是说,即便迭代满额,模型在样本内也只能拿到约 38% 的判别命中,离随机基准不算遥遥领先。 外汇与贵金属这类高波动品种接同类自训练网络时,样本内准确率偏低往往预示样本外泛化会更弱,实盘前建议先用小资金或模拟账户验证。打开 MT5 的 Experts 日志面板,把上述 epoch 行和你的训练输出对照,就能判断自己的网络是否也卡在 0.38 附近的瓶颈。

◍ 自训练神经网络在NAS100四小时图上的回测表现

把自训练的神经网络EA跑在NAS100的H4周期上,训练集与测试集的命中率会出现明显落差。一次典型日志里,初始测试准确率为0.717,说明模型在未见数据上先给了个偏乐观的印象。 随后完整跑完1000/1000批次,训练过程记录 Loss = 0.05890808,训练集准确率降到0.615;紧接着测试准确率报出0.635。训练集和测试集相差约2个百分点,过拟合不算严重,但绝对水平刚过六成,信号本身偏弱。 这类结果提醒做外汇或贵金属量化的人:NAS100虽非外汇直盘,但指数CFD同样高波动、高杠杆,用六成出头的模型直接跟单风险很大。建议先在MT5策略测试器用相同品种周期复现日志,再决定是否加过滤条件。

MQL5 / C++
CS        class="num">0       class="num">15:class="num">50:class="num">51.662    Self Trained NN EA(NAS100,H4) class="num">2023.01.class="num">03 class="num">01:class="num">00:class="num">00  Testing Accuracy =class="num">0.717
CS        class="num">0       class="num">17:class="num">11:class="num">52.100    Self Trained NN EA(NAS100,H4) class="num">2023.01.class="num">03 class="num">01:class="num">00:class="num">00  [ class="num">1000/class="num">1000 ] Loss = class="num">0.05890808 | accuracy class="num">0.615
CS        class="num">0       class="num">17:class="num">11:class="num">52.101    Self Trained NN EA(NAS100,H4) class="num">2023.01.class="num">03 class="num">01:class="num">00:class="num">00  Testing Accuracy =class="num">0.635

「别急着下结论」

增量规则只服务于单层回归型神经网络,这一点要刻在脑子里。它本质就是若干线性回归模型的组合,比如你可以把它当成 5 个各管一摊的线性回归模型凑在一起解同一个问题。 这类单层结构理解不了变量里的复杂形态,跑不出预期结果时不必意外。它真正价值在于充当通用反向传播算法的积木,后者才撑得起深度学习的复杂网络。 作者故意让自己被训练误差带着走,是想说明:哪怕网络真能学出形态,也得抠住缩放、初始化这些小细节,很多环节做对了它才肯工作。 MALE5 函数库在 GitHub 持续迭代,目前含 metrics.mqh(准确性测量)、preprocessing.mqh(数据缩放)、matrix_utils.mqh(矩阵操控)、selftrain NN.mqh(自训练核心)及测试用 EA;外汇与贵金属杠杆高,实盘前务必在 MT5 策略测试器跑通再说。

常见问题

不同特征量纲差太大,神经网络容易只学大量纲那个。用最小-最大缩放 (x-min)/(max-min) 压到 [0,1] 能让各特征公平参与。
不用。只在首 tick 跑一次训练就够了,后续 tick 直接喂特征拿预测值,不然实时计算会卡死。
可以。小布能接入你的特征与预测输出,在对应品种页标出神经网络倾向方向,省去你盯盘口算信号。
大概率过拟合了。训练测试差太多说明只记住了历史顺序,实盘外汇贵金属高风险,别直接信高训练分。
不打散网络会顺着K线时间顺序学偏,随机打散能逼它学特征本身而不是近期走势惯性。