数据科学和机器学习（第 21 部分）：解锁神经网络，优化算法揭秘·进阶篇
📘

数据科学和机器学习（第 21 部分）：解锁神经网络，优化算法揭秘·进阶篇

第 2/3 篇
本章目录
  1. SGD 优化器的矩阵更新底层写法
  2. 小批量梯度下降的优化器接管逻辑
  3. SGD 模式下的逐样本反向传播
  4. 批梯度下降下的权重回写细节
  5. 反向传播里的权重回写与批次校验
  6. 训练循环里损失与精度的落地记录

◍ SGD 优化器的矩阵更新底层写法

在 MT5 里用 MQL5 搭神经网络，优化器最先要落地的是参数更新规则。下面这段 OptimizerSGD 把学习率默认设为 0.01，构造函数仅做成员初始化，没有额外计算开销。 update 方法里只有一行核心：parameters -= m_learning_rate * gradients。这就是最朴素的梯度下降，权重矩阵按梯度反方向走一小步，步长由学习率控制。 Batch Gradient Descent 继承自该类，特点是在整批训练集上算梯度再更新。数据集大时可能偏慢、占内存，但收敛方向倾向更稳定。外汇与贵金属行情噪声大，用 BGD 做策略参数寻优须清醒认识其过拟合高风险。 把这段代码直接丢进 MT5 的 .mqh 头文件编译，就能在自家 EA 里调用 update 验证梯度下降是否按预期收敛。

MQL5 / C++
class OptimizerSGD
  {
class="kw">protected:
   class="type">class="kw">double m_learning_rate;

class="kw">public:
                     OptimizerSGD(class="type">class="kw">double learning_rate=class="num">0.01);
                    ~OptimizerSGD(class="type">void);
                    
                     class="kw">virtual class="type">void update(matrix &parameters, matrix &gradients);
  };
class=class="str">"cmt">//+------------------------------------------------------------------+
class=class="str">"cmt">//|                                                                  |
class=class="str">"cmt">//+------------------------------------------------------------------+
OptimizerSGD::OptimizerSGD(class="type">class="kw">double learning_rate=class="num">0.01):
m_learning_rate(learning_rate)
{
}
class=class="str">"cmt">//+------------------------------------------------------------------+
class=class="str">"cmt">//|                                                                  |
class=class="str">"cmt">//+------------------------------------------------------------------+
OptimizerSGD::~OptimizerSGD(class="type">void)
{
}
class=class="str">"cmt">//+------------------------------------------------------------------+
class=class="str">"cmt">//|                                                                  |
class=class="str">"cmt">//+------------------------------------------------------------------+
class="type">void OptimizerSGD::update(matrix &parameters, matrix &gradients)
{
    parameters -= this.m_learning_rate * gradients; class=class="str">"cmt">//Simple gradient descent update rule
}
class=class="str">"cmt">//+------------------------------------------------------------------+
class=class="str">"cmt">//|  Batch Gradient Descent(BGD): This optimizer computes the       |
class=class="str">"cmt">//|  gradients of the loss function on the entire training dataset   |
class=class="str">"cmt">//|  and updates the parameters accordingly. It can be slow and     |
class=class="str">"cmt">//|  memory-intensive for large datasets but tends to provide a     |
class=class="str">"cmt">//|  stable convergence.                                            |
class=class="str">"cmt">//+------------------------------------------------------------------+
class OptimizerMinBGD: class="kw">public OptimizerSGD
  {
class="kw">public:
                     OptimizerMinBGD(class="type">class="kw">double learning_rate=class="num">0.01);
                    ~OptimizerMinBGD(class="type">void);
  };

「小批量梯度下降的优化器接管逻辑」

在回归网络里，优化器对象决定了权重和偏置怎么更新。当 batch_size 传 0 时，fit 直接用外部传入的 OptimizerMinBGD 实例；一旦 batch_size>0，backprop 内部会就地新建两个 OptimizerMinBGD 对象接管权重与偏置的更新，相当于强制走小批量梯度下降。 下面这段构造与析构只做初始化，学习率默认 0.01 透传给基类 OptimizerSGD，没有额外状态： [CODE]OptimizerMinBGD::OptimizerMinBGD(double learning_rate=0.010000): OptimizerSGD(learning_rate) { } OptimizerMinBGD::~OptimizerMinBGD(void) { }[/CODE] 脚本侧把网络结构写死成单隐藏层 5 神经元，训练超参暴露成 input 便于在 MT5 参数面板调：epochs=100、learning_rate=0.0001、batch_size=32。数据用 airfoil_noise_data.csv，按 0.7 切分训练集，x_train 先过 StandardizationScaler 再做 fit。 回测交叉验证部分用的是 10 折 KFoldCV，意味着全量样本被均分 10 份轮流出验证集。你开 MT5 把 batch_size 从 0 改成 32，能直接观察优化器分支切换后损失曲线的收敛节奏差异，外汇与贵金属行情序列若套同套框架，须注意过拟合与滑点风险偏高。

MQL5 / C++
OptimizerMinBGD::OptimizerMinBGD(class="type">class="kw">double learning_rate=class="num">0.010000): OptimizerSGD(learning_rate)
{
}
class=class="str">"cmt">//+------------------------------------------------------------------+
OptimizerMinBGD::~OptimizerMinBGD(class="type">void)
{
}
class="macro">#include <MALE5\MatrixExtend.mqh>
class="macro">#include <MALE5\preprocessing.mqh>
class="macro">#include <MALE5\metrics.mqh>
class="macro">#include <MALE5\Neural Networks\Regressor Nets.mqh>
CRegressorNets *nn;
StandardizationScaler scaler;
vector open_, high_, low_;
vector hidden_layers = {class="num">5};
input class="type">uint nn_epochs = class="num">100;
input class="type">class="kw">double nn_learning_rate = class="num">0.0001;
input class="type">uint nn_batch_size =class="num">32;
input class="type">bool show_batch = class="kw">false;
class="type">void OnStart()
  {
   class="type">class="kw">string headers;
   matrix dataset = MatrixExtend::ReadCsv("airfoil_noise_data.csv", headers);
   matrix x_train, x_test;
   vector y_train, y_test;
   MatrixExtend::TrainTestSplitMatrices(dataset, x_train, y_train, x_test, y_test, class="num">0.7);
   nn = new CRegressorNets(hidden_layers, AF_RELU_, LOSS_MSE_);
   x_train = scaler.fit_transform(x_train);
   nn.fit(x_train, y_train, new OptimizerMinBGD(nn_learning_rate), nn_epochs, nn_batch_size, show_batch);
   class="kw">delete nn;
  }
backprop CRegressorNets::backpropagation(const matrix& x, const vector &y, OptimizerSGD *optimizer, const class="type">uint epochs, class="type">uint batch_size=class="num">0, class="type">bool show_batch_progress=class="kw">false)
{
   OptimizerSGD optimizer_weights = optimizer;
   OptimizerSGD optimizer_bias = optimizer;
   if (batch_size>class="num">0)
    {
     OptimizerMinBGD optimizer_weights;
     OptimizerMinBGD optimizer_bias;
    }
   CCrossValidation cross_validation;      
   CTensors *cv_tensor;
   matrix validation_data = MatrixExtend::concatenate(x, y);
   matrix validation_x;
   vector validation_y;
   cv_tensor = cross_validation.KFoldCV(validation_data, class="num">10);
   matrix DELTA = {};
   class="type">class="kw">double actual=class="num">0, pred=class="num">0;
   }

SGD 模式下的逐样本反向传播

当 batch_size 设为 0 时，训练循环进入随机梯度下降（SGD）分支，此时对每个样本单独前向预测并回传梯度，而不是按小批量聚合。代码里用 if (batch_size==0) 显式切到 SGD，内层 for (ulong iter=0; iter<rows; iter++) 会遍历全部行数据，逐条调用 predict(x.Row(iter)) 拿到单点预测值。 反向传播从最后一层往前扫：for (int layer=(int)mlp.hidden_layers-1; layer>=0; ...) 逐层计算 DELTA，尺寸与偏置矩阵一致。权重导数 dW 由 DELTA 左乘本层输入的转置得到（dW = DELTA.MatMul(temp_inputs.Transpose())），偏置导数 dB 直接等于 DELTA。 在 MT5 里验证时，把 batch_size 传 0 跑几百个 epoch，观察 GetTickCount() 记录的单 epoch 耗时——样本数过万时 SGD 的单步更新频繁但每次矩阵很小，显存压力低但收敛波动可能偏大。外汇与贵金属品种用此类模型做信号筛选属高风险，回测表现不代表实盘概率。

MQL5 / C++
matrix temp_inputs ={};

matrix dB = {}; class=class="str">"cmt">//Bias Derivatives
matrix dW = {}; class=class="str">"cmt">//Weight Derivatives

for (class="type">class="kw">ulong epoch=class="num">0; epoch<epochs && !IsStopped(); epoch++)
  {
    class="type">class="kw">double epoch_start = GetTickCount();
    class="type">uint num_batches = (class="type">uint)MathFloor(x.Rows()/(batch_size+DBL_EPSILON));

    vector batch_loss(num_batches),
           batch_accuracy(num_batches);

     vector actual_v(class="num">1), pred_v(class="num">1), LossGradient = {};
    if (batch_size==class="num">0) class=class="str">"cmt">//Stochastic Gradient Descent
      {
       for (class="type">class="kw">ulong iter=class="num">0; iter<rows; iter++) class=class="str">"cmt">//iterate through all data points
         {
          pred = predict(x.Row(iter));
          actual = y[iter];

          pred_v[class="num">0] = pred;
          actual_v[class="num">0] = actual;
 class=class="str">"cmt">//---

          DELTA.Resize(mlp.outputs,class="num">1);

          for (class="type">int layer=(class="type">int)mlp.hidden_layers-class="num">1; layer>=class="num">0 && !IsStopped(); layer--) class=class="str">"cmt">//Loop through the network backward from last to first layer
            { class=class="str">"cmt">//..... backpropagation and finding derivatives code

             class=class="str">"cmt">//-- Observation | DeLTA matrix is same size as the bias matrix

             W = this.Weights_tensor.Get(layer);
             B = this.Bias_tensor.Get(layer);

             class=class="str">"cmt">//--- Derivatives wrt weights and bias

             dB = DELTA;
             dW = DELTA.MatMul(temp_inputs.Transpose());

             class=class="str">"cmt">//--- Weights updates

◍ 批梯度下降下的权重回写细节

上面这段逻辑走的是 Batch Gradient Descent 分支：先把样本按 batch_size 切成 num_batches 块，再逐块抽取 batch_x 与 batch_y 送进网络。注意 batch_end 的写法是 (batch_start+batch_size-1)，意味着闭区间取值，和 Get 的起止下标约定一致，少减一个 1 就会多读一行越界。 内层对每块数据仍用 for(iter=0; iter<rows; iter++) 逐点前向预测，pred_v[0] 与 actual_v[0] 分别接 predict 输出和标签。反向时 DELTA.Resize(mlp.outputs,1) 每次重置，且循环从最后一层 hidden_layers-1 倒序到 0，注释里点明 DELTA 尺寸与偏置矩阵相同——这是后续 update(B, dB) 能直接相加的前提。 optimizer_weights.update(W, dW) 与 optimizer_bias.update(B, dB) 执行完，立刻用 Weights_tensor.Add(W, layer) 和 Bias_tensor.Add(B, layer) 落盘到张量容器。开 MT5 把这段嵌进你自己的 MLP 训练循环，重点核对 batch_end 边界和 DELTA 尺寸，否则小样本下可能前几 batch 正常、末 batch 直接数组越界报错。

MQL5 / C++
optimizer_weights.update(W, dW);
optimizer_bias.update(B, dB);

this.Weights_tensor.Add(W, layer);
this.Bias_tensor.Add(B, layer);
}
   }
  }
  else class=class="str">"cmt">//Batch Gradient Descent
  {

  for (class="type">uint batch=class="num">0, batch_start=class="num">0, batch_end=batch_size; batch<num_batches; batch++, batch_start+=batch_size, batch_end=(batch_start+batch_size-class="num">1))
   {

   matrix batch_x = MatrixExtend::Get(x, batch_start, batch_end-class="num">1);
   vector batch_y = MatrixExtend::Get(y, batch_start, batch_end-class="num">1);

   rows = batch_x.Rows();

     for (class="type">class="kw">ulong iter=class="num">0; iter<rows ; iter++) class=class="str">"cmt">//iterate through all data points
      {
      pred_v[class="num">0] = predict(batch_x.Row(iter));
      actual_v[class="num">0] = y[iter];

      class=class="str">"cmt">//---

      DELTA.Resize(mlp.outputs,class="num">1);

      for (class="type">int layer=(class="type">int)mlp.hidden_layers-class="num">1; layer>=class="num">0 && !IsStopped(); layer--) class=class="str">"cmt">//Loop through the network backward from last to first layer
       {     class=class="str">"cmt">//..... backpropagation and finding derivatives code   
        }

       class=class="str">"cmt">//-- Observation | DeLTA matrix is same size as the bias matrix

「反向传播里的权重回写与批次校验」

在多层网络的训练循环里，每一层先取出当前的权重张量和偏置张量：W 与 B 分别来自 Weights_tensor 和 Bias_tensor 的指定 layer。误差反向时，偏置梯度 dB 直接等于该层 DELTA，权重梯度 dW 则是 DELTA 与前一层输入转置后的矩阵乘积，这一步对应全连接层的链式求导。 拿到梯度后交给优化器就地更新：optimizer_weights.update(W, dW) 与 optimizer_bias.update(B, dB)，随后用 Add 方法把新值写回张量容器，完成单层参数落地。注意这里 W、B 是引用式修改，回写动作不能漏，否则下一 epoch 读的还是旧参数。 一个批次跑完会用 predict(batch_x) 得到预测值，损失通过 pred_v.Loss 按预设函数计算。代码里对损失做了 NaN 防护：若数值非法或大于 1e6，则截断为 1e6，避免梯度爆炸把整个训练拖死。 批次准确率走的是 Metrics::r_squared，也就是决定系数，外接 show_batch_progress 开关时会用 printf 打出 batch[%d/%d] batch-loss %.5f accuracy %.3f。验证集则在 epoch 结束时另起循环，从 cv_tensor 拆出 validation_x / validation_y 准备独立评估，外汇与贵金属样本上过拟合概率偏高，验证损失和训练损失偏离过大时应停手调参。

MQL5 / C++
W = this.Weights_tensor.Get(layer);
B = this.Bias_tensor.Get(layer);

class=class="str">"cmt">//--- Derivatives wrt weights and bias

dB = DELTA;
dW = DELTA.MatMul(temp_inputs.Transpose());

class=class="str">"cmt">//--- Weights updates

optimizer_weights.update(W, dW);
optimizer_bias.update(B, dB);

this.Weights_tensor.Add(W, layer);
this.Bias_tensor.Add(B, layer);
}

pred_v = predict(batch_x);

batch_loss[batch] = pred_v.Loss(batch_y, ENUM_LOSS_FUNCTION(m_loss_function));
batch_loss[batch] = MathIsValidNumber(batch_loss[batch]) ? (batch_loss[batch]>class="num">1e6 ? class="num">1e6 : batch_loss[batch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan

batch_accuracy[batch] = Metrics::r_squared(batch_y, pred_v);

if (show_batch_progress)
   printf("----> batch[%d/%d] batch-loss %.5f accuracy %.3f",batch+class="num">1,num_batches,batch_loss[batch], batch_accuracy[batch]);
}
}

class=class="str">"cmt">//--- End of an epoch

vector validation_loss(cv_tensor.SIZE);
vector validation_acc(cv_tensor.SIZE);
for (class="type">class="kw">ulong i=class="num">0; i<cv_tensor.SIZE; i++)
{
   validation_data = cv_tensor.Get(i);
   MatrixExtend::XandYSplitMatrices(validation_data, validation_x, validation_y);

训练循环里损失与精度的落地记录

在每轮 epoch 收尾时，代码会先按 batch_size 是否为 0 决定取全量还是批量损失。batch_size==0 走全量前向，否则用 batch_loss 均值，两条分支都把 training_loss 和 validation_loss 写进 backprop_struct。 无论哪条分支，都调用 MathIsValidNumber 做 NaN 兜底：若数值无效或大于 1e6，强制截断为 1e6，避免发散的梯度把回测曲线打爆。validation_loss 取 validation_loss 数组的 Mean()，validation_acc 则是每轮 R² 的均值。 最后用 GetTickCount 算单轮耗时，printf 打出 'Epoch [x/y] training -> loss %.8f accuracy %.3f validation -> loss %.5f accuracy %.3f | Elapsed %s'。外汇与贵金属样本上，R² 若长期低于 0.3 往往预示过拟合，可优先调小学习率或砍隐藏层。 循环结束置 isBackProp=false 并 delete optimizer 释放指针，返回结构体给上层做早停判断。MT5 里把 epoch 打到 50 以上，观察 validation_loss 是否在 10 轮内不再下降，就能决定是否提前断训。

MQL5 / C++
vector val_preds = this.predict(validation_x);

validation_loss[i] = val_preds.Loss(validation_y, ENUM_LOSS_FUNCTION(m_loss_function));
validation_acc[i] = Metrics::r_squared(validation_y, val_preds);
					pred_v = this.predict(x);

		if (batch_size==class="num">0)
		{			
				backprop_struct.training_loss[epoch] = pred_v.Loss(y, ENUM_LOSS_FUNCTION(m_loss_function));
				backprop_struct.training_loss[epoch] = MathIsValidNumber(backprop_struct.training_loss[epoch]) ? (backprop_struct.training_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.training_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan
				backprop_struct.validation_loss[epoch] = validation_loss.Mean();
				backprop_struct.validation_loss[epoch] = MathIsValidNumber(backprop_struct.validation_loss[epoch]) ? (backprop_struct.validation_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.validation_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan
		}
		else
		{
				backprop_struct.training_loss[epoch] = batch_loss.Mean();
				backprop_struct.training_loss[epoch] = MathIsValidNumber(backprop_struct.training_loss[epoch]) ? (backprop_struct.training_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.training_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan
				backprop_struct.validation_loss[epoch] = validation_loss.Mean();
				backprop_struct.validation_loss[epoch] = MathIsValidNumber(backprop_struct.validation_loss[epoch]) ? (backprop_struct.validation_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.validation_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan
		}
				
class="type">class="kw">double epoch_stop = GetTickCount();
		printf("--> Epoch [%d/%d] training -> loss %.8f accuracy %.3f validation -> loss %.5f accuracy %.3f | Elapsed %s ",epoch+class="num">1,epochs,backprop_struct.training_loss[epoch],Metrics::r_squared(y, pred_v),backprop_struct.validation_loss[epoch],validation_acc.Mean(),this.ConvertTime((epoch_stop-epoch_start)/class="num">1000.0));
	 }

   isBackProp = class="kw">false;

	if (CheckPointer(optimizer)!=POINTER_INVALID)
		class="kw">delete optimizer;

	 class="kw">return backprop_struct;
}
	backprop_struct.training_loss[epoch] = batch_loss.Mean();
	backprop_struct.training_loss[epoch] = MathIsValidNumber(backprop_struct.training_loss[epoch]) ? (backprop_struct.training_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.training_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan
	backprop_struct.validation_loss[epoch] = validation_loss.Mean();

常见问题

把当前权重矩阵减去学习率乘梯度矩阵的逐元素乘积即可，核心就是 W = W - lr * dW，批量数据先累加梯度再除样本数。
在每个 mini-batch 前向反向跑完、梯度算好后由优化器统一回写，别在逐样本循环里写权重，否则变成在线学习。
可以，小布能对照你的训练循环代码指出反向传播中批次维度校验缺失或权重误回写的位置，并给出修正建议。
逐样本是每个样本后动权重，批梯度是攒完所有样本梯度求平均再回写，后者更新更稳定但占用内存更高。
每个 epoch 结束把 loss 和 acc 存进数组或日志文件，按步数 append，后续直接用折线图看收敛趋势。

相关阅读

小布盯盘 · 交易者自己的 价格行为学 + AIGC 分析工具。小布知识库 持续更新硬核交易干货。

本文由人工智能（AIGC）基于价格行为学进行深度分析，仅为技术方法论的探讨与分享，不构成任何形式的投资建议或交易指令。外汇保证金及贵金属交易具有高风险，并不适合所有投资者；过往表现不代表未来结果，请结合自身风险承受能力独立决策。

© 2026 小布盘 unyet.cn