神经网络变得简单（第 79 部分）：在状态上下文中的特征聚合查询（FAQ）·综合运用
（3/3）·静态环境状态漏掉了价格变动的动态概率，这篇用 FAQ 聚合查询补上最后一块拼图
「EA 初始化与逐根 K 线取数骨架」
MT5 智能交易系统在 OnInit 里优先尝试从本地加载 DOT.nnw 与 Act.nnw 两个神经网络权重文件；若文件缺失或加载失败，则当场用 CreateDescriptions 构造 dot / decoder / actor / critic 四套网络描述，并依次 DOT.Create、Decoder.Create、Actor.Create，任一步返回 false 就释放全部 CArrayObj 指针并 return INIT_FAILED。加载成功后把 DOT 内部的 OpenCL 上下文透传给 Decoder 与 Actor，这一步决定后续矩阵运算是否在显卡上跑。 行情侧靠 OnTick 驱动，但先用 IsNewBar 拦掉同根 K 线内的重复触发；随后 CopyRates 按 TimeFrame 拉取 HistoryBars 根历史数据进 Rates 数组，并 ArraySetAsSeries(Rates, true) 把索引 0 对齐到最新一根。RSI、CCI、ATR、MACD 四个指标对象各自 Refresh，再循环把 open、rsi、cci、atr、macd、signal 逐个读入 float 变量——这里 atr 在循环外声明，意味着最后一根本根的 ATR 值会留到循环外继续使用。 实盘接这套逻辑时，先把 HistoryBars 设小（比如 50）观一眼显存占用与 tick 延迟；外汇与贵金属杠杆高、滑点突变频繁，神经网络推理只是概率辅助，仓位仍须自担风险。
class=class="str">"cmt">//| Expert initialization function | //+------------------------------------------------------------------+ class="type">int OnInit() { class=class="str">"cmt">//--- ........ ........ class=class="str">"cmt">//--- load models class="type">float temp; class=class="str">"cmt">//--- if(!DOT.Load(FileName + "DOT.nnw", temp, temp, temp, dtStudied, true) || !Actor.Load(FileName + "Act.nnw", temp, temp, temp, dtStudied, true)) { CArrayObj *dot = new CArrayObj(); CArrayObj *decoder = new CArrayObj(); CArrayObj *actor = new CArrayObj(); CArrayObj *critic = new CArrayObj(); if(!CreateDescriptions(dot, decoder, actor, critic)) { class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; class="kw">return INIT_FAILED; } if(!DOT.Create(dot) || !Decoder.Create(decoder) || !Actor.Create(actor)) { class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; class="kw">return INIT_FAILED; } class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; } class=class="str">"cmt">//--- Decoder.SetOpenCL(DOT.GetOpenCL()); Actor.SetOpenCL(DOT.GetOpenCL()); class=class="str">"cmt">//--- ........ ........ class=class="str">"cmt">//--- class="kw">return(INIT_SUCCEEDED); } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert tick function | //+------------------------------------------------------------------+ class="type">void OnTick() { class=class="str">"cmt">//--- if(!IsNewBar()) class="kw">return; class="type">int bars = CopyRates(Symb.Name(), TimeFrame, iTime(Symb.Name(), TimeFrame, class="num">1), HistoryBars, Rates); if(!ArraySetAsSeries(Rates, true)) class="kw">return; class=class="str">"cmt">//--- RSI.Refresh(); CCI.Refresh(); ATR.Refresh(); MACD.Refresh(); Symb.Refresh(); Symb.RefreshRates(); class="type">float atr = class="num">0; for(class="type">int b = class="num">0; b < (class="type">int)HistoryBars; b++) { class="type">float open = (class="type">float)Rates[b].open; class="type">float rsi = (class="type">float)RSI.Main(b); class="type">float cci = (class="type">float)CCI.Main(b); atr = (class="type">float)ATR.Main(b); class="type">float macd = (class="type">float)MACD.Main(b); class="type">float sign = (class="type">float)MACD.Signal(b);
把指标与持仓压进状态数组
这段逻辑干的事很直接：先过滤掉任一指标取不到值的 K 线（rsi/cci/atr/macd/sign 任一为 EMPTY_VALUE 就跳过），避免脏数据混进训练样本。随后按 BarDescr 步长把每根 bar 的 close-high-low 相对开盘价的偏移、tick_volume/1000、以及 5 个指标值共 9 个浮点塞进 sState.state。
| 账户侧用 AccountInfoDouble 抓 balance 与 equity 写进 account[0]、[1]；持仓遍历只认当前品种，分别累加多空 volume 与 floating profit 到 account[2]~[5]。position_discount 那行有意思：用 (current - 开仓时间) * multiplyer * | profit | 做时间衰减惩罚，multiplyer = 1/(60*60*10)，相当于持仓秒数乘 1/360000 再乘利润绝对值，老仓对状态的负向权重更高。 |
|---|
最后 bAccount 只塞了两个归一化值：余额较 PrevBalance 的收益率、权益对 PrevBalance 的比值。外汇与贵金属杠杆高，这类状态拼接若用于 RL 或信号模型，过拟合和滑点吞噬概率都不小，上 MT5 跑前先手算几根 bar 的 shift 偏移确认没写串。
if(rsi == EMPTY_VALUE || cci == EMPTY_VALUE || atr == EMPTY_VALUE || macd == EMPTY_VALUE || sign == EMPTY_VALUE) class="kw">continue; class=class="str">"cmt">//--- class="type">int shift = b * BarDescr; sState.state[shift] = (class="type">float)(Rates[b].close - open); sState.state[shift + class="num">1] = (class="type">float)(Rates[b].high - open); sState.state[shift + class="num">2] = (class="type">float)(Rates[b].low - open); sState.state[shift + class="num">3] = (class="type">float)(Rates[b].tick_volume / class="num">1000.0f); sState.state[shift + class="num">4] = rsi; sState.state[shift + class="num">5] = cci; sState.state[shift + class="num">6] = atr; sState.state[shift + class="num">7] = macd; sState.state[shift + class="num">8] = sign; } bState.AssignArray(sState.state); sState.account[class="num">0] = (class="type">float)AccountInfoDouble(ACCOUNT_BALANCE); sState.account[class="num">1] = (class="type">float)AccountInfoDouble(ACCOUNT_EQUITY); class=class="str">"cmt">//--- class="type">class="kw">double buy_value = class="num">0, sell_value = class="num">0, buy_profit = class="num">0, sell_profit = class="num">0; class="type">class="kw">double position_discount = class="num">0; class="type">class="kw">double multiplyer = class="num">1.0 / (class="num">60.0 * class="num">60.0 * class="num">10.0); class="type">int total = PositionsTotal(); class="type">class="kw">datetime current = TimeCurrent(); for(class="type">int i = class="num">0; i < total; i++) { if(PositionGetSymbol(i) != Symb.Name()) class="kw">continue; class="type">class="kw">double profit = PositionGetDouble(POSITION_PROFIT); class="kw">switch((class="type">int)PositionGetInteger(POSITION_TYPE)) { case POSITION_TYPE_BUY: buy_value += PositionGetDouble(POSITION_VOLUME); buy_profit += profit; class="kw">break; case POSITION_TYPE_SELL: sell_value += PositionGetDouble(POSITION_VOLUME); sell_profit += profit; class="kw">break; } position_discount += profit - (current - PositionGetInteger(POSITION_TIME)) * multiplyer * MathAbs(profit); } sState.account[class="num">2] = (class="type">float)buy_value; sState.account[class="num">3] = (class="type">float)sell_value; sState.account[class="num">4] = (class="type">float)buy_profit; sState.account[class="num">5] = (class="type">float)sell_profit; sState.account[class="num">6] = (class="type">float)position_discount; sState.account[class="num">7] = (class="type">float)Rates[class="num">0].time; bAccount.Clear(); bAccount.Add((class="type">float)((sState.account[class="num">0] - PrevBalance) / PrevBalance)); bAccount.Add((class="type">float)(sState.account[class="num">1] / PrevBalance));
◍ 把账户状态塞进网络的喂食逻辑
这段逻辑做的是把实时账户与周期相位编码成一维特征向量，喂给后续的自编码器与策略网络。前 6 个特征值分别是权益环比变动率、账户三项原始字段，以及自由保证金、结余、净值相对前一周期余额的占比，类型全部强转 float 以压低显存开销。 时间项用了四个不同周期的三角函数：以 2023 全年秒数为分母算年相位正弦，再分别用月线、周线、日线周期秒数算余弦或正弦。这样网络能隐式感知‘现在处于年/月/周/日循环的哪个相位’，对外汇和贵金属这种带周期节律的市场可能有帮助，但高频噪声也会被一并吃进去。 特征拼满后先跑 bAccount.BufferWrite() 落盘，失败直接 return；随后 DOT 前馈、Decoder 解码、Actor 出动作，三层任意一层 feedForward 返回 false 就打印函数名加行号并退出，方便在 MT5 Experts 日志里定位断点。 出向量后做动作互斥裁剪：temp[0] 与 temp[3] 分别代表多空强度，谁大就减掉小的，保证同周期不双向开仓。下方还取了 LotsMin、LotsStep 与 StopsLevel*Point 作为下单硬约束，外汇贵金属杠杆高，这类边界值不校验就容易报‘无效交易量’错单。
bAccount.Add((class="type">float)((sState.account[class="num">1] - PrevEquity) / PrevEquity)); bAccount.Add(sState.account[class="num">2]); bAccount.Add(sState.account[class="num">3]); bAccount.Add((class="type">float)(sState.account[class="num">4] / PrevBalance)); bAccount.Add((class="type">float)(sState.account[class="num">5] / PrevBalance)); bAccount.Add((class="type">float)(sState.account[class="num">6] / PrevBalance)); class="type">class="kw">double x = (class="type">class="kw">double)Rates[class="num">0].time / (class="type">class="kw">double)(D&class="macro">#x27;class="num">2024.01.class="num">01&class="macro">#x27; - D&class="macro">#x27;class="num">2023.01.class="num">01&class="macro">#x27;); bAccount.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Rates[class="num">0].time / (class="type">class="kw">double)PeriodSeconds(PERIOD_MN1); bAccount.Add((class="type">float)MathCos(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Rates[class="num">0].time / (class="type">class="kw">double)PeriodSeconds(PERIOD_W1); bAccount.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Rates[class="num">0].time / (class="type">class="kw">double)PeriodSeconds(PERIOD_D1); bAccount.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); if(bAccount.GetIndex() >= class="num">0) if(!bAccount.BufferWrite()) class="kw">return; class=class="str">"cmt">//--- if(!DOT.feedForward((CBufferFloat*)GetPointer(bState), class="num">1, false, (CBufferFloat*)NULL)) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); class="kw">return; } if(!Decoder.feedForward((CNet*)GetPointer(DOT), LatentLayer,(CNet*)GetPointer(DOT))) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); class="kw">return; } class=class="str">"cmt">//--- Actor if(!Actor.feedForward((CNet *)GetPointer(Decoder), -class="num">1, (CBufferFloat*)GetPointer(bAccount))) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); class="kw">return; } class=class="str">"cmt">//--- PrevBalance = sState.account[class="num">0]; PrevEquity = sState.account[class="num">1]; vector<class="type">float> temp; Actor.getResults(temp); if(temp.Size() < NActions) temp = vector<class="type">float>::Zeros(NActions); class=class="str">"cmt">//--- class="type">class="kw">double min_lot = Symb.LotsMin(); class="type">class="kw">double step_lot = Symb.LotsStep(); class="type">class="kw">double stops = MathMax(Symb.StopsLevel(), class="num">1) * Symb.Point(); if(temp[class="num">0] >= temp[class="num">3]) { temp[class="num">0] -= temp[class="num">3]; temp[class="num">3] = class="num">0; } else { temp[class="num">3] -= temp[class="num">0]; temp[class="num">0] = class="num">0; } class=class="str">"cmt">//--- buy control
「双向持仓的加减量控与奖励回填」
这段逻辑把买、卖两条线拆开管，但判定骨架完全一致：先用 temp 数组的前三个元素（买向）和后三个元素（卖向）检查手数下限、止盈止损距离是否撞到 stops 硬下限。只要任一条件不满足，就直接把该方向已有仓位清掉，不再参与本轮调仓。 以买向为例，temp[0] 是模型给的目标手数，min_lot 是经纪商最小交易单位，step_lot 是手数步长。代码用 MathRound((temp[0]-min_lot)/step_lot)*step_lot 把目标手数吸附到合规网格上，避免下单被 MT5 拒。买向 TP 挂在 Ask 上方 temp[1]*MaxTP*Point 处，SL 挂在 Ask 下方 temp[2]*MaxSL*Point 处，并用 NormalizeDouble 按品种 Digits 截断报价精度。 若当前 buy_value 大于算出的 buy_lot，说明模型要减仓，调 TrailPosition 改挂止损止盈后由 ClosePartial 平掉多出部分；若小于，则 Trade.Buy 补齐差额。卖向对称处理，TP/Bid 用减、SL/Bid 用加。外汇与贵金属杠杆高，stops 设太小会被这段代码强制清仓，回测时建议先打印 stops 与 Point 乘积确认不会误杀。 清完仓位后，sState.rewards 回填三项：账户权益、1 减回撤比例、以及无持仓时的 atr/PrevBalance 惩罚项。最后把 temp 整组写进 action 并 Base.Add，添加失败直接 ExpertRemove 退出智能交易，防止脏数据继续跑。
if(temp[class="num">0] < min_lot || (temp[class="num">1] * MaxTP * Symb.Point()) <= stops || (temp[class="num">2] * MaxSL * Symb.Point()) <= stops) { if(buy_value > class="num">0) CloseByDirection(POSITION_TYPE_BUY); } else { class="type">class="kw">double buy_lot = min_lot + MathRound((class="type">class="kw">double)(temp[class="num">0] - min_lot) / step_lot) * step_lot; class="type">class="kw">double buy_tp = NormalizeDouble(Symb.Ask() + temp[class="num">1] * MaxTP * Symb.Point(), Symb.Digits()); class="type">class="kw">double buy_sl = NormalizeDouble(Symb.Ask() - temp[class="num">2] * MaxSL * Symb.Point(), Symb.Digits()); if(buy_value > class="num">0) TrailPosition(POSITION_TYPE_BUY, buy_sl, buy_tp); if(buy_value != buy_lot) { if(buy_value > buy_lot) ClosePartial(POSITION_TYPE_BUY, buy_value - buy_lot); else Trade.Buy(buy_lot - buy_value, Symb.Name(), Symb.Ask(), buy_sl, buy_tp); } } class=class="str">"cmt">//--- sell control if(temp[class="num">3] < min_lot || (temp[class="num">4] * MaxTP * Symb.Point()) <= stops || (temp[class="num">5] * MaxSL * Symb.Point()) <= stops) { if(sell_value > class="num">0) CloseByDirection(POSITION_TYPE_SELL); } else { class="type">class="kw">double sell_lot = min_lot + MathRound((class="type">class="kw">double)(temp[class="num">3] - min_lot) / step_lot) * step_lot;; class="type">class="kw">double sell_tp = NormalizeDouble(Symb.Bid() - temp[class="num">4] * MaxTP * Symb.Point(), Symb.Digits()); class="type">class="kw">double sell_sl = NormalizeDouble(Symb.Bid() + temp[class="num">5] * MaxSL * Symb.Point(), Symb.Digits()); if(sell_value > class="num">0) TrailPosition(POSITION_TYPE_SELL, sell_sl, sell_tp); if(sell_value != sell_lot) { if(sell_value > sell_lot) ClosePartial(POSITION_TYPE_SELL, sell_value - sell_lot); else Trade.Sell(sell_lot - sell_value, Symb.Name(), Symb.Bid(), sell_sl, sell_tp); } } sState.rewards[class="num">0] = bAccount[class="num">0]; sState.rewards[class="num">1] = class="num">1.0f - bAccount[class="num">1]; if((buy_value + sell_value) == class="num">0) sState.rewards[class="num">2] -= (class="type">float)(atr / PrevBalance); else sState.rewards[class="num">2] = class="num">0; for(class="type">class="kw">ulong i = class="num">0; i < NActions; i++) sState.action[i] = temp[i]; if(!Base.Add(sState)) ExpertRemove(); }
EA 初始化时四套神经网络的装载逻辑
在 MT5 的 EA 初始化阶段，先声明了四个 CNet 实例：DOT、Decoder、Actor、Critic，分别对应编码、解码、策略与价值网络。OnInit 里第一步调用 ResetLastError 清错误码，随后 LoadTotalBase 负责把历史学习数据读入内存，若返回 false 则直接打印错误号并 INIT_FAILED，EA 起不来。 紧接着尝试从磁盘加载四个 .nnw 模型文件（DOT.nnw、Dec.nnw、Act.nnw、Crt.nnw）。只要其中任意一个 Load 失败，代码会现建四组 CArrayObj 描述对象，调用 CreateDescriptions 按 NActions、HistoryBars、BarDescr 等宏生成网络结构，再逐网 Create；任一步失败同样回退 INIT_FAILED，并手动 delete 防内存泄漏。 模型就绪后，OpenCL 上下文从 DOT 取出并共享给其余三网，Actor.getResults(Result) 校验输出维度必须等于 NActions，否则报“scope of the actor does not match”。DOT.GetLayerOutput(0, Result) 再核对输入尺寸 == HistoryBars * BarDescr，不符则提示编码器输入与状态描述不匹配。最后 bGradient 缓冲区按 AccountDescr 与 NForecast 的最大值初始化，外汇与贵金属品种下这套装载若报错，EA 无法跑，属高风险调试环节，建议逐文件核对 .nnw 是否和当前宏定义一致。
CNet Decoder; CNet Actor; CNet Critic; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert initialization function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">int OnInit() { class=class="str">"cmt">//--- ResetLastError(); if(!LoadTotalBase()) { PrintFormat("Error of load study data: %d", GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- load models class="type">float temp; if(!DOT.Load(FileName + "DOT.nnw", temp, temp, temp, dtStudied, true) || !Decoder.Load(FileName + "Dec.nnw", temp, temp, temp, dtStudied, true) || !Actor.Load(FileName + "Act.nnw", temp, temp, temp, dtStudied, true) || !Critic.Load(FileName + "Crt.nnw", temp, temp, temp, dtStudied, true) ) { CArrayObj *dot = new CArrayObj(); CArrayObj *decoder = new CArrayObj(); CArrayObj *actor = new CArrayObj(); CArrayObj *critic = new CArrayObj(); if(!CreateDescriptions(dot, decoder, actor, critic)) { class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; class="kw">return INIT_FAILED; } if(!DOT.Create(dot) || !Decoder.Create(decoder) || !Actor.Create(actor) || !Critic.Create(critic)) { class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; class="kw">return INIT_FAILED; } class="kw">delete dot; class="kw">delete decoder; class="kw">delete actor; class="kw">delete critic; } OpenCL = DOT.GetOpenCL(); Decoder.SetOpenCL(OpenCL); Actor.SetOpenCL(OpenCL); Critic.SetOpenCL(OpenCL); Actor.getResults(Result); if(Result.Total() != NActions) { PrintFormat("The scope of the actor does not match the actions count(%d <> %d)", NActions, Result.Total()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- DOT.GetLayerOutput(class="num">0, Result); if(Result.Total() != (HistoryBars * BarDescr)) { PrintFormat("Input size of Encoder doesn&class="macro">#x27;t match state description(%d <> %d)", Result.Total(), (HistoryBars * BarDescr)); class="kw">return INIT_FAILED; } if(!bGradient.BufferInit(MathMax(AccountDescr, NForecast), class="num">0) || !bGradient.BufferCreate(OpenCL)) {
◍ 初始化失败与离线模型落盘的分支处理
EA 在 OnInit 里若缓冲区创建失败，会先 PrintFormat 打出错误码再 return INIT_FAILED；同样，EventChartCustom 发送自定义事件失败也走同一条路。这两处返回失败前不会触发 OnDeinit 的模型保存逻辑，因为 reason 会被置为 REASON_INITFAILED。 OnDeinit 中只在 reason 既不是 REASON_INITFAILED 也不是 REASON_RECOMPILE 时才把四个网络（Actor / DOT / Decoder / Critic）以 TimeCurrent() 时间戳写入对应 .nnw 文件。也就是说，重编译导致的卸载不会覆盖你盘面上的训练权重，这个细节能避免来回改代码把模型冲掉。 Train 函数里每轮迭代先用 GetProbTrajectories 取阈值 0.9 的概率轨迹，再用双重 MathRand() 平方做非线性采样挑 state，batch 固定为 GPTBars+48。若算出的 state<=0 就 iter-- 并 continue，相当于丢弃这次无效抽样。外汇与贵金属杠杆高，这类自研训练 EA 在实盘前务必用策略测试器跑足样本，过拟合概率不低。
PrintFormat("Error of create buffers: %d", GetLastError()); class="kw">return INIT_FAILED; } if(!EventChartCustom(ChartID(), class="num">1, class="num">0, class="num">0, "Init")) { PrintFormat("Error of create study event: %d", GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- class="kw">return(INIT_SUCCEEDED); } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert deinitialization function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void OnDeinit(const class="type">int reason) { class=class="str">"cmt">//--- if(!(reason == REASON_INITFAILED || reason == REASON_RECOMPILE)) { Actor.Save(FileName + "Act.nnw", class="num">0, class="num">0, class="num">0, TimeCurrent(), true); DOT.Save(FileName + "DOT.nnw", class="num">0, class="num">0, class="num">0, TimeCurrent(), true); Decoder.Save(FileName + "Dec.nnw", class="num">0, class="num">0, class="num">0, TimeCurrent(), true); Critic.Save(FileName + "Crt.nnw", class="num">0, class="num">0, class="num">0, TimeCurrent(), true); } class="kw">delete Result; class="kw">delete OpenCL; } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Train function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void Train(class="type">void) { class=class="str">"cmt">//--- vector<class="type">float> probability = GetProbTrajectories(Buffer, class="num">0.9); vector<class="type">float> result, target; class="type">bool Stop = false; class=class="str">"cmt">//--- class="type">uint ticks = GetTickCount(); for(class="type">int iter = class="num">0; (iter < Iterations && !IsStopped() && !Stop); iter ++) { class="type">int tr = SampleTrajectory(probability); class="type">int batch = GPTBars + class="num">48; class="type">int state = (class="type">int)((MathRand() * MathRand() / MathPow(class="num">32767, class="num">2)) * (Buffer[tr].Total - class="num">2 - PrecoderBars - batch)); if(state <= class="num">0) { iter--; class="kw">continue; } DOT.Clear(); class="type">int end = MathMin(state + batch, Buffer[tr].Total - PrecoderBars); for(class="type">int i = state; i < end; i++) { bState.AssignArray(Buffer[tr].States[i].state);
「把账户状态塞进策略网络的喂数据写法」
这段逻辑干了两件事：先跑轨迹网络（DOT + Decoder 的前向传播），再把账户历史归一化成一组特征向量喂给后续 Actor。任何一步 feedForward 失败就直接 Stop 并 break，训练循环立刻停，不会带着脏状态往下走。 账户特征里最值得盯的是那几个比值：余额变化率用 (account[0]-PrevBalance)/PrevBalance，权益相对余额用 account[1]/PrevBalance，回撤率用 (account[1]-PrevEquity)/PrevEquity。这些量纲统一到「相对于前一刻余额」之后，网络才不容易被绝对资金量带偏。外汇和贵金属杠杆高，这类归一化能降低爆仓样本对梯度的一次性冲击，但模型仍可能过拟合极端行情。 时间编码那段把时间戳分别除以年、月、周、日周期再套 sin/cos，等于手工注入周期先验。其中年周期基准写死为 D'2024.01.01' - D'2023.01.01'（约 31536000 秒），月用 PERIOD_MN1、周用 PERIOD_W1、日用 PERIOD_D1 的 PeriodSeconds()。开 MT5 把这段贴进 EA 训练循环，改一下年基准或去掉周线分量，能直接看出策略对周期敏感的程度上不下得来。
class=class="str">"cmt">//--- Trajectory if(!DOT.feedForward((CBufferFloat*)GetPointer(bState), class="num">1, false, (CBufferFloat*)NULL)) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true; class="kw">break; } if(!Decoder.feedForward((CNet*)GetPointer(DOT), LatentLayer, (CNet*)GetPointer(DOT))) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true; class="kw">break; } class=class="str">"cmt">//--- Policy class="type">float PrevBalance = Buffer[tr].States[MathMax(i - class="num">1, class="num">0)].account[class="num">0]; class="type">float PrevEquity = Buffer[tr].States[MathMax(i - class="num">1, class="num">0)].account[class="num">1]; bAccount.Clear(); bAccount.Add((Buffer[tr].States[i].account[class="num">0] - PrevBalance) / PrevBalance); bAccount.Add(Buffer[tr].States[i].account[class="num">1] / PrevBalance); bAccount.Add((Buffer[tr].States[i].account[class="num">1] - PrevEquity) / PrevEquity); bAccount.Add(Buffer[tr].States[i].account[class="num">2]); bAccount.Add(Buffer[tr].States[i].account[class="num">3]); bAccount.Add(Buffer[tr].States[i].account[class="num">4] / PrevBalance); bAccount.Add(Buffer[tr].States[i].account[class="num">5] / PrevBalance); bAccount.Add(Buffer[tr].States[i].account[class="num">6] / PrevBalance); class="type">class="kw">double time = (class="type">class="kw">double)Buffer[tr].States[i].account[class="num">7]; class="type">class="kw">double x = time / (class="type">class="kw">double)(D&class="macro">#x27;class="num">2024.01.class="num">01&class="macro">#x27; - D&class="macro">#x27;class="num">2023.01.class="num">01&class="macro">#x27;); bAccount.Add((class="type">float)MathSin(x != class="num">0 ? class="num">2.0 * M_PI * x : class="num">0)); x = time / (class="type">class="kw">double)PeriodSeconds(PERIOD_MN1); bAccount.Add((class="type">float)MathCos(x != class="num">0 ? class="num">2.0 * M_PI * x : class="num">0)); x = time / (class="type">class="kw">double)PeriodSeconds(PERIOD_W1); bAccount.Add((class="type">float)MathSin(x != class="num">0 ? class="num">2.0 * M_PI * x : class="num">0)); x = time / (class="type">class="kw">double)PeriodSeconds(PERIOD_D1); bAccount.Add((class="type">float)MathSin(x != class="num">0 ? class="num">2.0 * M_PI * x : class="num">0)); if(bAccount.GetIndex() >= class="num">0) bAccount.BufferWrite(); class=class="str">"cmt">//--- Actor
Actor-Critic 反向传播与折扣回报计算
这段训练循环把 Actor 与 Critic 串起来做前向和反向传播。前向阶段若 Actor 或 Critic 的 feedForward 返回 false，立即打印函数名与行号、置 Stop=true 并 break，避免脏梯度污染后续权重。 反向传播先用 Buffer[tr].States[i].action 给 Result 赋值，再走 Actor.backProp 与 Decoder/DOT 的梯度回传；任意一步失败同样触发 Stop 退出。注意 DOT.backPropGradient 连续三次调用，中间两次传入 LatentLayer 做隐层梯度累积，最后一次无参收尾。 回报侧用相邻两步 rewards 算时序差分：result = Buffer[i+1].rewards - Buffer[i+2].rewards * DiscFactor，DiscFactor 即折扣因子，控制未来奖励衰减强度。该值设 0.9 与设 0.99 会让策略更新方向出现明显分歧，建议直接在 MT5 里改参看 equity 曲线形变。 Critic 的反向以 Actor 为基准网络，随后 Actor 再以 bAccount、bGradient 做梯度回传；任一层 backProp 失败都会打印「__FUNCTION__ -> __LINE__」并终止本轮，方便你定位是哪一层网络维度不匹配。
if(!Actor.feedForward((CNet *)GetPointer(Decoder), -class="num">1, (CBufferFloat*)GetPointer(bAccount))) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true; class="kw">break; } class=class="str">"cmt">//--- Critic if(!Critic.feedForward((CNet *)GetPointer(Decoder), -class="num">1, (CNet*)GetPointer(Actor))) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true; class="kw">break; } Result.AssignArray(Buffer[tr].States[i].action); if(!Actor.backProp(Result, (CBufferFloat *)GetPointer(bAccount), (CBufferFloat *)GetPointer(bGradient)) || !Decoder.backPropGradient((CNet *)GetPointer(DOT), -class="num">1, -class="num">1, false) || !DOT.backPropGradient((CBufferFloat*)NULL, (CBufferFloat*)NULL, LatentLayer) || !DOT.backPropGradient((CBufferFloat*)NULL) ) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true; class="kw">break; } result.Assign(Buffer[tr].States[i + class="num">1].rewards); target.Assign(Buffer[tr].States[i + class="num">2].rewards); result = result - target * DiscFactor; Result.AssignArray(result); if(!Critic.backProp(Result, (CNet *)GetPointer(Actor)) || !Decoder.backPropGradient((CNet *)GetPointer(DOT), -class="num">1, -class="num">1, false) || !DOT.backPropGradient((CBufferFloat*)NULL, (CBufferFloat*)NULL, LatentLayer) || !DOT.backPropGradient((CBufferFloat*)NULL) || !Actor.backPropGradient((CBufferFloat *)GetPointer(bAccount), (CBufferFloat *)GetPointer(bGradient), -class="num">1, false) || !Decoder.backPropGradient((CNet *)GetPointer(DOT), -class="num">1, -class="num">1, false) || !DOT.backPropGradient((CBufferFloat*)NULL, (CBufferFloat*)NULL, LatentLayer) || !DOT.backPropGradient((CBufferFloat*)NULL) ) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); Stop = true;
◍ 训练循环里的进度与误差打印
这段代码片段处在强化学习训练主循环收尾处，核心是用定时刷新把 Actor / Critic 的进度百分比和近期平均误差推到图表上。 每过 500 毫秒（GetTickCount 差值 > 500）才更新一次 Comment，避免每 tick 刷屏拖慢 MT5 回测或实盘帧率；percent 由当前迭代 i、状态偏移 state、终点 end 与总轮数 Iterations 算出，范围 0~100。 循环结束后清空 Comment，并用 PrintFormat 把 Actor、Critic 的最终平均误差按 10.7f 精度打到日志，随后调用 ExpertRemove 让 EA 自卸载——适合跑完固定轮数就退出的训练型脚本。 外汇与贵金属行情具有高杠杆高风险，这类自训练 EA 仅建议在策略测试器内验证，实盘加载前务必确认误差收敛且过拟合可控。
class="kw">break; } if(GetTickCount() - ticks > class="num">500) { class="type">class="kw">double percent = (class="type">class="kw">double(i - state) / ((end - state)) + iter) * class="num">100.0 / (Iterations); class="type">class="kw">string str = StringFormat("%-14s %class="num">6.2f%% -> Error %class="num">15.8f\n", "Actor", percent, Actor.getRecentAverageError()); str += StringFormat("%-14s %class="num">6.2f%% -> Error %class="num">15.8f\n", "Critic", percent, Critic.getRecentAverageError()); Comment(str); ticks = GetTickCount(); } } } Comment(""); PrintFormat("%s -> %d -> %-15s %class="num">10.7f", __FUNCTION__, __LINE__, "Actor", Actor.getRecentAverageError()); PrintFormat("%s -> %d -> %-15s %class="num">10.7f", __FUNCTION__, __LINE__, "Critic", Critic.getRecentAverageError()); ExpertRemove(); class=class="str">"cmt">//--- }
「用 EURUSD 历史切片验证聚合模型」
模型跑通后必须拿真实历史去压一遍。我在 H1 周期下取 EURUSD 做训练与测试分离：用 2023 年前 7 个月数据训练，8 月数据留作测试，输入特征与奖励结构和前文保持一致。
若你手头没有之前的经验回放文件，先把 EA ...\Experts\FAQ\Research.mq5 丢进策略测试器，选训练区间做慢速优化，用随机验算灌满缓冲区；有旧文件则直接重命名为 “FAQ.bd” 续用。指标参数训练与测试必须完全一致，我准备本文时全部用了默认值。
训练阶段用 ...\Experts\FAQ\Study.mq5 在实时图表挂机，它只读书训练、不下单，所以账户余额零风险。我习惯迭代式训练：每轮用当前策略替掉随机策略去补数据，相当于围绕已有政策做在线探索，让后续训练拿到真奖励而非插值。
外汇与贵金属属高风险品种，以下仅为历史回测现象。2023 年 8 月测试集上 EA 共成交 87 笔，45 笔盈利，胜率 51.72%；盈利因子 1.61，恢复因子 1.65，最大及平均盈利均大于亏损对应值。这组合说明该聚合模型在样本外月份未明显坍塌，但换周期或品种需重测。
即插即用模块的实盘验证边界
FAQ 提出的查询聚合模块本质是即插即用结构，可挂进多数基于变换器的对象检测器，用来处理视频与时间序列类任务。动态版还能按源数据自适应生成初始化并调权重。 我们在 MQL5 里落地了这套思路，用真实历史数据训模型，再切到训练集之外的时间段回测。结果倾向支持方法有效，但训练和测试窗口都很短，样本不足以推出稳健结论。 外汇与贵金属行情属高风险环境，这类实验程序仅作方法演示，不能直接当实盘信号源。真要验证，自己开 MT5 拉长样本区间跑一遍最实在。
◍ 文中所附的程序清单
这套 LSTM 多元时间序列预测方案落地到 MT5，靠的是一组分工明确的程序文件，而不是单个 EA 包打天下。 研究阶段由 Research.mq5 与 ResearchRealORL.mq5 两个智能交易系统负责采集样本，前者跑常规窗口，后者用 Real-ORL 方法补样本；随后 Study.mq5 做模型训练，Test.mq5 做离线回测验证。 支撑逻辑的是三个类库：Trajectory.mqh 定义系统状态结构，NeuroNet.mqh 封装建网接口，NeuroNet.cl 则是 OpenCL 核函数，决定 GPU 上矩阵运算怎么跑。 想自己复现，直接下载文末 MQL5.zip（约 973 KB）解压进 MetaEditor 即可；外汇与贵金属行情高波动，任何模型输出只代表概率倾向，实盘前务必用 Test.mq5 在对应品种历史数据上重跑一遍。