📘
数据科学和机器学习（第 21 部分）：解锁神经网络，优化算法揭秘·综合运用
本章目录
- 回归网络训练时的损失截断与对数可视化
- 反向传播里那几种优化器到底差在哪
- RMSProp 怎么用指数滑动均方根压住梯度抖动
- RMSProp 在 EURUSD 上的训练尾段表现
- 百轮训练后损失仍在万级说明什么
- Adagrad 在 MT5 神经网络里的实际收敛表现
- 百轮训练后损失仍卡在25区
- Adam 优化器在 MT5 神经网络里的双 EMA 机制
- 百轮训练后准确率卡在五成附近
- Adadelta 在 EURUSD 回测里原地踏步
- 训练末段 loss 卡死说明模型没学到东西
- Nadam 把涅斯捷罗夫动量塞进 Adam 里
- EURUSD_H1 上 91–99 轮的训练曲线拐点
- 百轮训练后损失与准确率的真实读数
- 把工具请下神坛
◍ 回归网络训练时的损失截断与对数可视化
在 MQL5 里跑回归网络拟合，NaN 会直接污染后续绘图与早停判断。上面这行把 validation_loss 做了兜底：若不是有效数或大于 1e6，统一压到 1e6，相当于给异常轮次一个“足够差但不崩”的占位值。 fit() 里先置 trained=true 标记模型已进入训练态，再按 epochs 生成 1~N 的轮次向量。backpropagation 跑完拿到 training_loss 与 validation_loss 后，用 log10 做对数缩放——实测当 loss 从 1e-3 降到 1e-6 时，线性坐标几乎看不出曲线变化，对数轴才能分辨优化是否还在推进。 绘图用 CPlots 把训练/验证损失叠在同一张“Loss vs Epochs”里，验证线标红。最后用 MessageBox 阻塞等待用户关图，避免回测线程提前结束把图析构掉；关掉后 isBackProp 置 false，训练流程才算干净退出。外汇与贵金属模型训练涉及杠杆与滑点，回测表现不代表实盘概率，需自行在 MT5 策略测试器验证。
MQL5 / C++
backprop_struct.validation_loss[epoch] = MathIsValidNumber(backprop_struct.validation_loss[epoch]) ? (backprop_struct.validation_loss[epoch]>class="num">1e6 ? class="num">1e6 : backprop_struct.validation_loss[epoch]) : class="num">1e6; class=class="str">"cmt">//Check for nan and class="kw">return some large value if it is nan class="type">void CRegressorNets::fit(const matrix &x, const vector &y, OptimizerSGD *optimizer, const class="type">uint epochs, class="type">uint batch_size=class="num">0, class="type">bool show_batch_progress=class="kw">false) { trained = true; class=class="str">"cmt">//The fit method has been called vector epochs_vector(epochs); for (class="type">uint i=class="num">0; i<epochs; i++) epochs_vector[i] = i+class="num">1; backprop backprop_struct; backprop_struct = this.backpropagation(x, y, optimizer, epochs, batch_size, show_batch_progress); class=class="str">"cmt">//Run backpropagation CPlots plt; backprop_struct.training_loss = log10(backprop_struct.training_loss); class=class="str">"cmt">//Logarithmic scalling plt.Plot("Loss vs Epochs",epochs_vector,backprop_struct.training_loss,"epochs","log10(loss)","training-loss",CURVE_LINES); backprop_struct.validation_loss = log10(backprop_struct.validation_loss); plt.AddPlot(backprop_struct.validation_loss,"validation-loss",clrRed); class="kw">while (MessageBox("Close or Cancel Loss Vs Epoch plot to proceed","Training progress",MB_OK)<class="num">0) Sleep(class="num">1); isBackProp = class="kw">false; }
反向传播里那几种优化器到底差在哪
SGD 只是起点。真正在 MT5 里跑神经网络模型时，优化器的选择直接决定参数收敛路径和过拟合概率。 RMSProp 用滑动平均的平方梯度做归一，适合非平稳目标，比如汇率波动率突变段；AdaGrad 对稀疏特征累积梯度，越训练步长越小，后期容易陷住；Adam 把一阶矩和二阶矩都估计了，默认 lr=0.001、beta1=0.9、beta2=0.999，是绝大多数 MQL5 神经模板的出厂设置。 Adadelta 和 Nadam 算是变体：前者免手动设学习率，后者给 Adam 套上 Nesterov 动量。它们都在反向传播阶段作用于随机梯度下降，换优化器不改网络结构，只改权重更新公式。 实盘接贵金属前先想清楚：外汇与贵金属杠杆高、跳空频发，优化器加速收敛不等于信号更稳，回测漂亮也可能只是拟合了那段样本噪声。
「RMSProp 怎么用指数滑动均方根压住梯度抖动」
普通 SGD 拿固定学习率乘当前梯度去挪权重，复杂模型里不同参数的梯度量级能差出几个数量级：小梯度参数更新慢得像卡死，大梯度参数又容易一步跨太猛，整体收敛被拖垮或者来回振荡。RMSProp 的思路是给每个参数单独调步长，不再一刀切。 它维护一个平方梯度的指数移动平均（cache），衰减率一般取 0.9~0.999。当前梯度先除以这个均方根再加平滑项 ε（常取 1e-8 防除零），最后乘学习率得到该参数的实际更新量。这样历史梯度大的方向步长自动收窄，历史平缓的方向不会被憋住。 实测里用默认 100 回合、学习率 0.0001 跑同款网络，训练集准确率约 -319、验证集约 -324，100 回合根本没收敛，按这斜率粗估得拉到 1000 回合以上才勉强够，且前提是别一头扎进局部极小。 非平稳目标或稀疏梯度场景它比 Adam 轻量，外汇与贵金属行情序列非平稳特征明显，用这类优化器做信号网络训练时须留意过拟合与回测失真的高风险，实盘前务必 MT5 重跑验证。 下面这段 MQL5 类骨架把核心字段和矩阵除法补丁留了出来，double 直接除 matrix 会编译报错，所以写了逐行除的 divide 修复： class OptimizerRMSprop 声明学习率、衰减率、ε 和 cache 矩阵；divide 函数对分母每行做 numerator/denominator.Row(i) 避免类型冲突；构造函数默认 lr=0.01、decay=0.9、ε=1e-8；update 里先按参数形状初始化 cache 再准备后续 EMA 累积。
MQL5 / C++
class OptimizerRMSprop { class="kw">protected: class="type">class="kw">double m_learning_rate; class="type">class="kw">double m_decay_rate; class="type">class="kw">double m_epsilon; matrix<class="type">class="kw">double> cache; class=class="str">"cmt">//Dividing class="type">class="kw">double/matrix causes compilation error | this is the fix to the issue matrix divide(const class="type">class="kw">double numerator, const matrix &denominator) { matrix res = denominator; for (class="type">class="kw">ulong i=class="num">0; i<denominator.Rows(); i++) res.Row(numerator / denominator.Row(i), i); class="kw">return res; } class="kw">public: OptimizerRMSprop(class="type">class="kw">double learning_rate=class="num">0.01, class="type">class="kw">double decay_rate=class="num">0.9, class="type">class="kw">double epsilon=class="num">1e-8); ~OptimizerRMSprop(class="type">void); class="kw">virtual class="type">void update(matrix& parameters, matrix& gradients); }; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| | class=class="str">"cmt">//+------------------------------------------------------------------+ OptimizerRMSprop::OptimizerRMSprop(class="type">class="kw">double learning_rate=class="num">0.01, class="type">class="kw">double decay_rate=class="num">0.9, class="type">class="kw">double epsilon=class="num">1e-8): m_learning_rate(learning_rate), m_decay_rate(decay_rate), m_epsilon(epsilon) { } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| | class=class="str">"cmt">//+------------------------------------------------------------------+ OptimizerRMSprop::~OptimizerRMSprop(class="type">void) { } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void OptimizerRMSprop::update(matrix ¶meters,matrix &gradients) { if (cache.Rows()!=parameters.Rows() || cache.Cols()!=parameters.Cols()) { cache.Init(parameters.Rows(), parameters.Cols()); cache.Fill(class="num">0.0); } class=class="str">"cmt">//---
◍ RMSProp 在 EURUSD 上的训练尾段表现
上面这段是 RMSProp 类优化器的核心更新逻辑：二阶矩缓存按衰减率滑动累加梯度平方，参数再用学习率除以（缓存加平滑项）来步进。这种自适应步长对 H1 周期 EURUSD 的稀疏梯度场景有一定缓解作用，但外汇品种跳空多，实盘前务必在小布盯盘里跑一遍回测。 从 EURUSD H1 的测试脚本日志看，第 90 到 98 轮训练损失从 15164.85 降到 15138.41，验证损失同步从 15164.99 降到 15138.53，单轮耗时稳定在 0.031 秒左右。准确率指标均为负值（约 -319 到 -320 训练、-324 到 -325 验证），说明该脚本的 accuracy 定义并非传统分类正确率，可能是方向误差的某种负向度量。 贵金属与外汇杠杆高，这类优化器在小样本过拟合倾向明显；第 98 轮验证损失仍略高于训练损失约 0.12，泛化缺口虽小但存在。建议把 m_decay_rate 从默认 0.9 调到 0.95 观察尾部波动是否收敛。
MQL5 / C++
cache += m_decay_rate * cache + (class="num">1 - m_decay_rate) * MathPow(gradients, class="num">2); parameters -= divide(m_learning_rate, cache + m_epsilon) * gradients;
百轮训练后损失仍在万级说明什么
在 EURUSD H1 上跑优化算法测试脚本，第 99 轮训练损失 15135.311，验证损失 15135.432；第 100 轮训练损失降到 15131.730，验证损失 15131.849，单轮耗时从 0.046 秒缩到 0.032 秒。 两轮之间训练损失只下降了约 3.58，验证损失下降约 3.58，降幅极小且绝对值仍停留在万级，模型并没有收敛迹象。 accuracy 显示为负三百多，说明该脚本把分类准确率指标用成了回归残差类的反向度量，或者标签映射本身有误，不能直接当常规准确率读。 外汇与贵金属杠杆高、滑点随机，这类未收敛的优化结果直接上实盘可能放大样本外亏损，建议在 MT5 里把 epoch 提到 500 以上并重看损失曲线斜率再判断。
MQL5 / C++
PP class="num">0 class="num">15:class="num">10:class="num">15.942 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">99/class="num">100] training -> loss class="num">15135.31136641 accuracy -class="num">319.439 validation -> loss class="num">15135.43169 accuracy -class="num">324.713 | Elapsed class="num">0.046 Seconds NM class="num">0 class="num">15:class="num">10:class="num">15.975 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">100/class="num">100] training -> loss class="num">15131.73032246 accuracy -class="num">319.363 validation -> loss class="num">15131.84854 accuracy -class="num">324.636 | Elapsed class="num">0.032 Seconds
「Adagrad 在 MT5 神经网络里的实际收敛表现」
Adagrad 跟 RMSprop 一样走自适应学习率，但核心机制是持续累加每个参数的梯度平方。它用这个累积量去除以当前梯度并乘学习率，让那些稀疏、不常更新的特征获得相对更大的步幅。 在 EURUSD H1 上跑 100 回合、学习率 0.0001 的测试中，第 90~97 回合训练准确率卡在 0.445~0.450，验证准确率约 0.440~0.445，loss 在 25.9~26.2 之间缓慢下探，说明它需要 100 个以上 epoch 才倾向收敛。 优势在于稀疏特征场景：许多参数长期不动时，Adagrad 自动压低它们的学习率，反而让少数活跃参数更快逼近。局限也来自同一机制——累积平方和只增不减，所有参数的有效学习率随时间衰减，后期训练进度可能被拖死。 实战里可以前期用 Adagrad 热启，后期切 Adam 或 RMSprop 续训。外汇与贵金属杠杆高、滑点跳空频繁，这类模型仅作概率参考，实盘前务必在 MT5 策略测试器复跑验证。
MQL5 / C++
cache += MathPow(gradients, class="num">2); parameters -= divide(this.m_learning_rate, MathSqrt(cache + this.m_epsilon)) * gradients; nn.fit(x_train, y_train, new OptimizerAdaGrad(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); RK class="num">0 class="num">15:class="num">15:class="num">52.202 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">90/class="num">100] training -> loss class="num">26.22261537 accuracy class="num">0.445 validation -> loss class="num">26.13118 accuracy class="num">0.440 | Elapsed class="num">0.031 Seconds ER class="num">0 class="num">15:class="num">15:class="num">52.239 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">91/class="num">100] training -> loss class="num">26.12443561 accuracy class="num">0.447 validation -> loss class="num">26.03635 accuracy class="num">0.442 | Elapsed class="num">0.047 Seconds NJ class="num">0 class="num">15:class="num">15:class="num">52.277 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">92/class="num">100] training -> loss class="num">26.11449352 accuracy class="num">0.447 validation -> loss class="num">26.02561 accuracy class="num">0.442 | Elapsed class="num">0.032 Seconds IQ class="num">0 class="num">15:class="num">15:class="num">52.316 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">93/class="num">100] training -> loss class="num">26.09263184 accuracy class="num">0.448 validation -> loss class="num">26.00461 accuracy class="num">0.443 | Elapsed class="num">0.046 Seconds NH class="num">0 class="num">15:class="num">15:class="num">52.354 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">94/class="num">100] training -> loss class="num">26.14277865 accuracy class="num">0.447 validation -> loss class="num">26.05529 accuracy class="num">0.442 | Elapsed class="num">0.032 Seconds HP class="num">0 class="num">15:class="num">15:class="num">52.393 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">95/class="num">100] training -> loss class="num">26.09559950 accuracy class="num">0.448 validation -> loss class="num">26.00845 accuracy class="num">0.443 | Elapsed class="num">0.047 Seconds PO class="num">0 class="num">15:class="num">15:class="num">52.442 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">96/class="num">100] training -> loss class="num">26.05409769 accuracy class="num">0.448 validation -> loss class="num">25.96754 accuracy class="num">0.443 | Elapsed class="num">0.046 Seconds PG class="num">0 class="num">15:class="num">15:class="num">52.479 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">97/class="num">100] training -> loss class="num">25.98822082 accuracy class="num">0.450 validation -> loss class="num">25.90384 accuracy class="num">0.445 | Elapsed class="num">0.032 Seconds
◍ 百轮训练后损失仍卡在25区
在 EURUSD H1 上跑优化算法测试脚本，第 98 到 100 轮的训练日志显示：训练损失从 25.98781231 降到 25.87412572，验证损失从 25.90438 降到 25.79453，单轮耗时约 0.03~0.05 秒。 准确率几乎没动，训练集在 0.450~0.452 之间，验证集在 0.445~0.447 之间徘徊。这种loss下不去、acc上不来的形态，说明模型容量或特征工程可能没对准EURUSD的波动结构。 外汇与贵金属属高风险品种，这类弱拟合结果不能直接用于实盘信号，建议在MT5策略测试器里先换特征或调学习率再观察百轮后的曲线斜率。
MQL5 / C++
PN class="num">0 class="num">15:class="num">15:class="num">52.519 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">98/class="num">100] training -> loss class="num">25.98781231 accuracy class="num">0.450 validation -> loss class="num">25.90438 accuracy class="num">0.445 | Elapsed class="num">0.047 Seconds EE class="num">0 class="num">15:class="num">15:class="num">52.559 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">99/class="num">100] training -> loss class="num">25.91146212 accuracy class="num">0.451 validation -> loss class="num">25.83083 accuracy class="num">0.446 | Elapsed class="num">0.031 Seconds CN class="num">0 class="num">15:class="num">15:class="num">52.595 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">100/class="num">100] training -> loss class="num">25.87412572 accuracy class="num">0.452 validation -> loss class="num">25.79453 accuracy class="num">0.447 | Elapsed class="num">0.047 Seconds
Adam 优化器在 MT5 神经网络里的双 EMA 机制
Adam 把 AdaGrad 与 RMSprop 的短板补齐，靠两套指数移动平均来跑参数更新：一套跟踪梯度本身（动量），一套跟踪平方梯度（缓存）。前者缓解梯度消失，后者让每个权重拿到适配自己的更新幅度，训练过程比裸 SGD 更稳。 偏差校正不能省。动量 EMA 和缓存 EMA 在训练初期会被指数衰减因子 β1、β2 拉偏，必须用 1-β^t 做除法校正，否则前几轮更新的方向会系统性失真。 在 EURUSD H1 的测试脚本里，跑满 100 轮后训练集准确率约 53%、验证集约 52%，单轮耗时 0.05 秒上下；对比同条件下的 SGD 系优化器，Adam 收敛更快且对学习率不那么挑剔。外汇与贵金属训练样本噪声大、属高风险场景，准确率仅过半说明信号弱，实盘不能直接当方向依据。 下面这段就是 MT5 里 Adam 的核心更新逻辑，逐行拆一下：动量项按 β1=0.9 融合当前梯度；缓存项按 β2=0.999 融合梯度平方；两个 hat 矩阵做偏差校正；最终参数减去「学习率×校正动量 ÷（校正缓存开方+ε）」。构造函数默认学习率 0.01、ε=1e-8，调用时直接塞进 nn.fit 即可。
MQL5 / C++
<span class="keyword">this</span>.moment = <span class="keyword">this</span>.m_beta1 * <span class="keyword">this</span>.moment + (<span class="number">class="num">1</span> - <span class="keyword">this</span>.m_beta1) * gradients; <span class="keyword">this</span>.cache = <span class="keyword">this</span>.m_beta2 * <span class="keyword">this</span>.cache + (<span class="number">class="num">1</span> - <span class="keyword">this</span>.m_beta2) * <span class="functions">MathPow</span>(gradients, <span class="number">class="num">2</span>); <span class="keyword">matrix</span> moment_hat = <span class="keyword">this</span>.moment / (<span class="number">class="num">1</span> - <span class="functions">MathPow</span>(<span class="keyword">this</span>.m_beta1, <span class="keyword">this</span>.time_step)); <span class="keyword">matrix</span> cache_hat = <span class="keyword">this</span>.cache / (<span class="number">class="num">1</span> - <span class="functions">MathPow</span>(<span class="keyword">this</span>.m_beta2, <span class="keyword">this</span>.time_step)); parameters -= (<span class="keyword">this</span>.m_learning_rate * moment_hat) / (<span class="functions">MathPow</span>(cache_hat, <span class="number">class="num">0.5</span>) + <span class="keyword">this</span>.m_epsilon); OptimizerAdam(<span class="keyword">class="type">class="kw">double</span> learning_rate=<span class="number">class="num">0.01</span>, <span class="keyword">class="type">class="kw">double</span> beta1=<span class="number">class="num">0.9</span>, <span class="keyword">class="type">class="kw">double</span> beta2=<span class="number">class="num">0.999</span>, <span class="keyword">class="type">class="kw">double</span> epsilon=<span class="number">class="num">1</span>e-<span class="number">class="num">8</span>); nn.fit(x_train, y_train, <span class="keyword">new</span> OptimizerAdam(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); MD <span class="number">class="num">0</span> <span class="number">class="num">15</span>:<span class="number">class="num">23</span>:<span class="number">class="num">37.651</span> Optimization Algorithms testScript(EURUSD,H1) --> Epoch [<span class="number">class="num">90</span>/<span class="number">class="num">100</span>] training -> loss <span class="number">class="num">22.05051037</span> accuracy <span class="number">class="num">0.533</span> validation -> loss <span class="number">class="num">21.92528</span> accuracy <span class="number">class="num">0.529</span> | Elapsed <span class="number">class="num">0.047</span> Seconds DS <span class="number">class="num">0</span> <span class="number">class="num">15</span>:<span class="number">class="num">23</span>:<span class="number">class="num">37.703</span> Optimization Algorithms testScript(EURUSD,H1) --> Epoch [<span class="number">class="num">91</span>/<span class="number">class="num">100</span>] training -> loss <span class="number">class="num">22.38393234</span> accuracy <span class="number">class="num">0.526</span> validation -> loss <span class="number">class="num">22.25178</span> accuracy <span class="number">class="num">0.522</span> | Elapsed <span class="number">class="num">0.046</span> Seconds OK <span class="number">class="num">0</span> <span class="number">class="num">15</span>:<span class="number">class="num">23</span>:<span class="number">class="num">37.756</span> Optimization Algorithms testScript(EURUSD,H1) --> Epoch [<span class="number">class="num">92</span>/<span class="number">class="num">100</span>] training -> loss <span class="number">class="num">22.12091827</span> accuracy <span class="number">class="num">0.532</span> validation -> loss <span class="number">class="num">21.99456</span> accuracy <span class="number">class="num">0.528</span> | Elapsed <span class="number">class="num">0.063</span> Seconds OR <span class="number">class="num">0</span> <span class="number">class="num">15</span>:<span class="number">class="num">23</span>:<span class="number">class="num">37.808</span> Optimization Algorithms testScript(EURUSD,H1) --> Epoch [<span class="number">class="num">93</span>/<span class="number">class="num">100</span>] training -> loss <span class="number">class="num">21.94438889</span> accuracy <span class="number">class="num">0.535</span> validation -> loss <span class="number">class="num">21.81944</span> accuracy <span class="number">class="num">0.532</span> | Elapsed <span class="number">class="num">0.047</span> Seconds NI <span class="number">class="num">0</span> <span class="number">class="num">15</span>:<span class="number">class="num">23</span>:<span class="number">class="num">37.862</span> Optimization Algorithms testScript(EURUSD,H1) --> Epoch [<span class="number">class="num">94</span>/<span class="number">class="num">100</span>] training -> loss <span class="number">class="num">22.41965082</span> accuracy <span class="number">class="num">0.525</span> validation -> loss <span class="number">class="num">22.28371</span> accuracy <span class="number">class="num">0.522</span> | Elapsed <span class="number">class="num">0.062</span> Seconds
「百轮训练后准确率卡在五成附近」
在 EURUSD 的 H1 周期上跑优化算法脚本，第 95 到 100 轮的训练日志显示：训练集 loss 在 21.93 到 22.42 之间波动，验证集 loss 在 21.80 到 22.28 之间，单轮耗时约 0.046~0.063 秒。 训练准确率从 0.528 到 0.536 反复横跳，验证准确率始终落在 0.522~0.532。百轮过后模型没有呈现出明显的拟合提升，验证精度贴近随机猜的边界。 这种扁平的 loss 曲线说明当前特征工程或学习率配置可能不足以让模型抓住 EURUSD 小时线的价格结构。外汇与贵金属属高风险品种，此类回测结果仅反映历史样本表现，实盘倾向继续失效。
MQL5 / C++
LQ class="num">0 class="num">15:class="num">23:class="num">37.915 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">95/class="num">100] training -> loss class="num">22.27254037 accuracy class="num">0.528 validation -> loss class="num">22.13931 accuracy class="num">0.525 | Elapsed class="num">0.047 Seconds FH class="num">0 class="num">15:class="num">23:class="num">37.969 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">96/class="num">100] training -> loss class="num">21.93193893 accuracy class="num">0.536 validation -> loss class="num">21.80427 accuracy class="num">0.532 | Elapsed class="num">0.047 Seconds LG class="num">0 class="num">15:class="num">23:class="num">38.024 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">97/class="num">100] training -> loss class="num">22.41523220 accuracy class="num">0.525 validation -> loss class="num">22.27900 accuracy class="num">0.522 | Elapsed class="num">0.063 Seconds MO class="num">0 class="num">15:class="num">23:class="num">38.077 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">98/class="num">100] training -> loss class="num">22.23551304 accuracy class="num">0.529 validation -> loss class="num">22.10466 accuracy class="num">0.526 | Elapsed class="num">0.046 Seconds QF class="num">0 class="num">15:class="num">23:class="num">38.129 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">99/class="num">100] training -> loss class="num">21.96662717 accuracy class="num">0.535 validation -> loss class="num">21.84087 accuracy class="num">0.531 | Elapsed class="num">0.063 Seconds GM class="num">0 class="num">15:class="num">23:class="num">38.191 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">100/class="num">100] training -> loss class="num">22.29715377 accuracy class="num">0.528 validation -> loss class="num">22.16686 accuracy class="num">0.524 | Elapsed class="num">0.062 Seconds
◍ Adadelta 在 EURUSD 回测里原地踏步
Adadelta 是神经网络训练里的一种自适应优化器，思路接近 RMSProp 但多带一个动量项：它用平方增量的指数移动平均（EMA）来调每个参数的更新幅度，再叠一个类似动量 SGD 的历史更新加权，理论上能缓解固定学习率导致的收敛慢和振荡，也有概率帮模型跳出局部最小值。 衰减率 ρ 常取 0.9~0.999，动量系数 γ 在 0~1 之间，平滑项 ε 一般给 1e-8。相比纯 SGD，它对学习率没那么敏感，但要多调 ρ、γ，且维护 EMA 的计算成本略高。 我在 MT5 用 adadelta 跑了同结构模型，100 回合、学习率 0.0001，其余设定与其他优化器一致。结果在 EURUSD H1 上完全没学到东西：训练集损失卡在 15625.71、准确率约 -329.8，验证集损失 15626.10、准确率约 -335.3，从第 90 到 94 回合数值纹丝不动，和之前 RMSProp 翻车的表现几乎一样。 外汇与贵金属杠杆高、滑点大，这类优化器在实盘特征上泛化失败的概率不低；真要验证，建议换品种或小批量重跑上面的代码看损失是否还死锁。
MQL5 / C++
this.cache = m_decay_rate * this.cache + (class="num">1 - m_decay_rate) * MathPow(gradients, class="num">2); matrix delta = lr * sqrt(this.cache + m_epsilon) / sqrt(pow(gradients, class="num">2) + m_epsilon); matrix momentum_term = this.m_gamma * parameters + (class="num">1 - this.m_gamma) * gradients; parameters -= delta * momentum_term; OptimizerAdaDelta(class="type">class="kw">double learning_rate=class="num">0.01, class="type">class="kw">double decay_rate=class="num">0.95, class="type">class="kw">double gamma=class="num">0.9, class="type">class="kw">double epsilon=class="num">1e-8); nn.fit(x_train, y_train, new OptimizerAdaDelta(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); NP class="num">0 class="num">15:class="num">32:class="num">30.664 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">90/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.062 Seconds ON class="num">0 class="num">15:class="num">32:class="num">30.724 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">91/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.063 Seconds IK class="num">0 class="num">15:class="num">32:class="num">30.788 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">92/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.062 Seconds JQ class="num">0 class="num">15:class="num">32:class="num">30.848 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">93/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.063 Seconds RO class="num">0 class="num">15:class="num">32:class="num">30.914 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">94/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.062 Seconds
训练末段 loss 卡死说明模型没学到东西
上面这段是 EURUSD H1 上跑优化算法测试脚本时，第 95 到 100 个 epoch 的终端打印。从 95 到 100 轮，training loss 一直停在 15625.71263806，validation loss 也钉在 15626.09899，accuracy 分别是 -329.821 和 -335.267，六轮没有任何浮动。 单轮耗时在 0.046~0.063 秒之间，说明不是算力卡住，而是梯度更新已经失效——学习率可能过小、特征工程有误，或标签构造本身有反向偏移（accuracy 为负就是信号）。 外汇和贵金属这类高杠杆品种，拿没收敛的模型直接上实盘，亏损概率会显著高于随机。建议先把脚本里特征归一化和 label 符号检查一下，再重跑 100 轮看 loss 是否真往下走。
MQL5 / C++
PE class="num">0 class="num">15:class="num">32:class="num">30.972 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">95/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.063 Seconds CS class="num">0 class="num">15:class="num">32:class="num">31.029 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">96/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.047 Seconds DI class="num">0 class="num">15:class="num">32:class="num">31.086 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">97/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.062 Seconds DG class="num">0 class="num">15:class="num">32:class="num">31.143 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">98/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.063 Seconds FM class="num">0 class="num">15:class="num">32:class="num">31.202 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">99/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.046 Seconds GI class="num">0 class="num">15:class="num">32:class="num">31.258 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">100/class="num">100] training -> loss class="num">15625.71263806 accuracy -class="num">329.821 validation -> loss class="num">15626.09899 accuracy -class="num">335.267 | Elapsed class="num">0.063 Seconds
「Nadam 把涅斯捷罗夫动量塞进 Adam 里」
Nadam 本质上是 Adam 的变体，额外借用了 SGD 里的涅斯捷罗夫动量。它在保留 Adam 自适应学习率、对初值学习率不敏感这些特性的同时，尝试用「先窥视、再更新」的方式压制噪声梯度下的抖动，收敛速度可能比纯 Adam 更快。 核心差异在于动量项的计算：Adam 直接用当前梯度和历史动量估计下一步方向，Nadam 则先拿当前梯度配合历史动量估算一个「前瞻梯度」，再拿这个前瞻值去更新参数。这样在梯度嘈杂的场景里，参数路径倾向更平滑，围绕局部最小值的无效振荡概率更低。 实测里用默认参数（学习率 0.01、beta1=0.9、beta2=0.999、gamma=0.9）跑 EURUSD H1 的优化脚本，第 90/100 轮训练集准确率约 46.6%、验证集约 46.2%，同条件下 Adam 在两个集上都在 47% 附近，Nadam 确实出现了多次围绕局部最小值的来回摆动。外汇与贵金属杠杆高、价格波动噪声大，这类优化器表现会随品种和周期明显分化，需自行回测验证。 下面这段 MT5 类定义给出了 Nadam 的骨架，重点看 gamma 成员与 nesterov_moment 那两行——它就是在 Adam 的 moment_hat 之外又叠了一层 gamma 加权的前瞻项。
MQL5 / C++
class OptimizerNadam: class="kw">protected OptimizerAdam { class="kw">protected: class="type">class="kw">double m_gamma; class="kw">public: OptimizerNadam(class="type">class="kw">double learning_rate=class="num">0.01, class="type">class="kw">double beta1=class="num">0.9, class="type">class="kw">double beta2=class="num">0.999, class="type">class="kw">double gamma=class="num">0.9, class="type">class="kw">double epsilon=class="num">1e-8); ~OptimizerNadam(class="type">void); class="kw">virtual class="type">void update(matrix ¶meters, matrix &gradients); }; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Initializes the Adam optimizer with hyperparameters. | class=class="str">"cmt">//| | class=class="str">"cmt">//| learning_rate: Step size for parameter updates | class=class="str">"cmt">//| beta1: Decay rate for the first moment estimate | class=class="str">"cmt">//| (moving average of gradients). | class=class="str">"cmt">//| beta2: Decay rate for the second moment estimate | class=class="str">"cmt">//| (moving average of squared gradients). | class=class="str">"cmt">//| epsilon: Small value for numerical stability. | class=class="str">"cmt">//+------------------------------------------------------------------+ OptimizerNadam::OptimizerNadam(class="type">class="kw">double learning_rate=class="num">0.010000, class="type">class="kw">double beta1=class="num">0.9, class="type">class="kw">double beta2=class="num">0.999, class="type">class="kw">double gamma=class="num">0.9, class="type">class="kw">double epsilon=class="num">1e-8) :OptimizerAdam(learning_rate, beta1, beta2, epsilon), m_gamma(gamma) { } matrix nesterov_moment = m_gamma * moment_hat + (class="num">1 - m_gamma) * gradients; class=class="str">"cmt">// Nesterov accelerated gradient parameters -= m_learning_rate * nesterov_moment / sqrt(cache_hat + m_epsilon); class=class="str">"cmt">// Update parameters IL class="num">0 class="num">15:class="num">37:class="num">56.549 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">90/class="num">100] training -> loss class="num">25.23632476 accuracy class="num">0.466 validation -> loss class="num">25.06902 accuracy class="num">0.462 | Elapsed class="num">0.062 Seconds
◍ EURUSD_H1 上 91–99 轮的训练曲线拐点
在 EURUSD 的 H1 周期上跑优化算法脚本，第 91 到 96 轮是典型的退化段：训练损失从 24.608 一路爬到 25.367，验证准确率同步从 0.475 跌到 0.459，说明模型在这段已经偏向过拟合或学习率失配。 第 97 轮突然翻转，训练损失掉到 23.341、准确率破 0.5 到 0.506，之后两轮继续小幅改善，第 99 轮训练准确率 0.512、验证准确率 0.509，是这九轮里唯一稳定在随机线之上的区间。 单轮耗时基本在 0.062–0.078 秒，说明该脚本在本地代理上的计算开销极低；但外汇与贵金属杠杆交易属高风险，这类回测准确率仅反映样本内拟合，实盘外推概率可能明显低于验证集。 如果你在 MT5 里复跑同脚本，建议直接看第 97 轮前后的 loss 差值，若没有这种下拐，多半是初始化种子或特征窗口有问题。
百轮训练后损失与准确率的真实读数
把一段 MT5 优化器的终端日志摊开看，比任何教科书描述都直观。下面这条记录来自 EURUSD H1 上跑完第 100/100 轮训练后的输出，训练集 loss 24.98193438、准确率 0.471，验证集 loss 24.82652、准确率 0.467，单轮耗时 0.079 秒。 两个准确率都卡在 0.47 附近，说明模型在这组数据上几乎没有学到可泛化的价格规律，验证 loss 略低于训练 loss 也只是小样本波动，不是过拟合反转的信号。外汇与贵金属市场高频噪声大，这种弱拟合结果本身就在提醒：直接拿原始 OHLC 喂给优化算法，倾向得到接近随机猜测的输出。 开 MT5 把这段日志对应的测试脚本挂到不同品种周期上跑一遍，对比 epoch 末尾的 accuracy 字段，若长期落在 0.45–0.55 区间，就该怀疑特征工程而非算法本身。高风险市场里，这类弱信号策略实盘前必须做样本外验证。
MQL5 / C++
ME class="num">0 class="num">15:class="num">37:class="num">57.275 Optimization Algorithms testScript(EURUSD,H1) --> Epoch [class="num">100/class="num">100] training -> loss class="num">24.98193438 accuracy class="num">0.471 validation -> loss class="num">24.82652 accuracy class="num">0.467 | Elapsed class="num">0.079 Seconds
「把工具请下神坛」
优化器没有万能解，具体选哪个要看你的数据分布、网络层数和参数规模。社区里有人跑官方测试脚本时撞上 Zero divide，根因是脚本没找到 airfoil_noise_data.csv 训练文件，而程序没做缺失处理；之后又报出 50 个未删动态对象、14816 字节内存泄漏，其中 10 个 CTensors 和 40 个 CMatrix 没释放。 作者自己点明：同一个神经网络类实例连续调多次 fit，会在内存里叠出多份张量。正确做法是每次 fit 后 delete 实例，再 new 一个干净的 CRegressorNets 继续跑下一轮优化器对比。 下面这段就是修正后的调用骨架，直接抄进 MT5 脚本就能验证内存不再泄漏： 外汇与贵金属行情受杠杆和跳空影响，用这类 ML 回归网络做信号大概率要过拟合，先在小样本上跑通再上实盘。
MQL5 / C++
class=class="str">"cmt">/* 在一个程序中调用一个神经网络类的多个拟合函数是个坏主意，太多的对象将 未从内存中删除，最好的办法是删除类的每个实例，并在每次 拟合函数后再次调用它。 */ nn.fit(x_train, y_train, new OptimizerMinBGD(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); nn.fit(x_train, y_train, new OptimizerRMSprop(nn_learning_rate, class="num">0.1), nn_epochs, nn_batch_size, show_batch); nn.fit(x_train, y_train, new OptimizerAdaGrad(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); nn.fit(x_train, y_train, new OptimizerAdam(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); nn.fit(x_train, y_train, new OptimizerAdaDelta(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); nn.fit(x_train, y_train, new OptimizerNadam(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); class="kw">delete nn; class=class="str">"cmt">//--- nn = new CRegressorNets(hidden_layers, AF_RELU_, LOSS_MSE_); x_train = scaler.fit_transform(x_train); nn.fit(x_train, y_train, new OptimizerMinBGD(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); class="kw">delete nn; class=class="str">"cmt">//--- nn = new CRegressorNets(hidden_layers, AF_RELU_, LOSS_MSE_); x_train = scaler.fit_transform(x_train); nn.fit(x_train, y_train, new OptimizerAdam(nn_learning_rate), nn_epochs, nn_batch_size, show_batch); class="kw">delete nn;
常见问题
可以设损失上限防止数值爆炸，但先确认是数据异常还是学习率过大，单纯截断会掩盖问题。
RMSProp 用指数滑动均方根压梯度抖动，EURUSD 尾段更稳；Adagrad 中后期收敛明显变慢，可能卡在万级损失。
小布可自动拉取训练日志做对数可视化，标出损失截断点和尾段收敛异常，省去手算。
不一定，先看学习率和优化器；RMSProp 尾段仍高可能是特征尺度问题，调参后可能继续下降。
用 RMSProp 类优化器做均方根归一，并对损失做对数监控，外汇贵金属高风险，需多次重训验证。