神经网络变得简单（第 64 部分）：保守加权行为克隆（CWBC）方法·进阶篇
「堆叠八层网络描述符的写法」
在 MT5 的 OpenCL 神经网络封装里，每加一层都得先 new 一个 CLayerDescription，填完字段再丢给 rtg.Add(descr)；任何一步失败就 delete 并 return false，避免野指针。 第 3 层用了 SoftMax 输出，descr.count 接 prev_count、step 取 EmbeddingSize、激活函数设 None，优化器走 ADAM。第 4 层是多头注意力（defNeuronMLMHAttentionOCL），window 与 step 分别设 EmbeddingSize 和 8，window_out 给到 32，layers=4，这种配置在序列建模里倾向捕捉中短程依赖。 第 5、6 层是卷积层，窗口与步长都等于 EmbeddingSize，第 6 层把 window_out 砍半（prev_wout/2），两层激活都是 LReLU。第 7 层再接 SoftMax，step 用上一层的 prev_wout。 第 8 层和第 8 层（原文重复标注）都是 Base 层：前者 count=LatentCount、激活 LReLU；后者 count 也接 LatentCount、激活换 TANH。外汇与贵金属行情的高波动下，这类深层结构过拟合概率不低，建议开 MT5 用少量样本先跑通 Add 链路再扩数据。
descr.activation = LReLU; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">3 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronSoftMaxOCL; descr.count = prev_count; descr.step = EmbeddingSize; descr.activation = None; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">4 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronMLMHAttentionOCL; descr.count = prev_count; descr.window = EmbeddingSize; descr.step = class="num">8; descr.window_out = class="num">32; descr.layers = class="num">4; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">5 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronConvOCL; prev_count = descr.count = prev_count; descr.window = EmbeddingSize; descr.step = EmbeddingSize; prev_wout = descr.window_out = EmbeddingSize; descr.optimization = ADAM; descr.activation = LReLU; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">6 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronConvOCL; prev_count = descr.count = prev_count; descr.window = prev_wout; descr.step = prev_wout; prev_wout = descr.window_out = prev_wout / class="num">2; descr.optimization = ADAM; descr.activation = LReLU; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">7 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronSoftMaxOCL; descr.count = prev_count; descr.step = prev_wout; descr.activation = None; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">8 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; descr.count = LatentCount; descr.optimization = ADAM; descr.activation = LReLU; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">8 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; prev_count = descr.count = LatentCount; descr.activation = TANH; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">9
奖励层堆叠与轨迹概率采样
在强化学习模型里，输出层之前要显式声明两层结构：第 9 层用 defNeuronBaseOCL 类型，节点数固定为 2 * NRewards，激活函数留空、优化器走 ADAM；第 10 层切到 defNeuronVAEOCL，节点数降回 NRewards，同样无激活、ADAM 优化。Add 失败必须 delete 描述符并返回 false，否则显存描述符会泄漏。 GetProbTrajectories 负责把 buffer 里的奖励压成矩阵：先按行填 rewards，再算沿轴 0 的标准差 std 和沿轴 1 的求和 result。排序用冒泡（sort 标志位控制），拿到 quantile 后，以 max_reward 与 min = result.Min() - 0.1*std.Sum() 做归一区间。
| 若 max_reward > min，用 multipl = exp( | result - max_reward | / (Percentile(90) - max_reward)) 做指数衰减权重，再除以 (result + lanbda) 并 ReplaceNan(0)；否则整列填 1。最后 result 除以自身和、做 CumSum 返回累积概率——这段直接决定 SampleTrajectory 抽哪条轨迹。 |
|---|
外汇与贵金属行情噪声大，这类概率采样只是决策先验，实盘仍需人工过滤，杠杆品种高风险。
if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; descr.count = class="num">2 * NRewards; descr.activation = None; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">10 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronVAEOCL; descr.count = NRewards; descr.activation = None; descr.optimization = ADAM; if(!rtg.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- class="kw">return true; } vector<class="type">float> GetProbTrajectories(STrajectory &buffer[], class="type">float &max_reward, class="type">float &quantile, vector<class="type">float> &std, class="type">class="kw">double quant, class="type">float lanbda) { class="type">ulong total = buffer.Size(); matrix<class="type">float> rewards = matrix<class="type">float>::Zeros(total, NRewards); vector<class="type">float> result; for(class="type">ulong i = class="num">0; i < total; i++) { result.Assign(buffer[i].States[class="num">0].rewards); rewards.Row(result, i); } std = rewards.Std(class="num">0); result = rewards.Sum(class="num">1); max_reward = result.Max(); vector<class="type">float> sorted = result; class="type">bool sort = true; class="type">int iter = class="num">0; class="kw">while(sort) { sort = false; for(class="type">ulong i = class="num">0; i < sorted.Size() - class="num">1; i++) if(sorted[i] > sorted[i + class="num">1]) { class="type">float temp = sorted[i]; sorted[i] = sorted[i + class="num">1]; sorted[i + class="num">1] = temp; sort = true; } iter++; } quantile = sorted.Quantile(quant); class="type">float min = result.Min() - class="num">0.1f * std.Sum(); if(max_reward > min) { vector<class="type">float> multipl=exp(MathAbs(result - max_reward) / (result.Percentile(class="num">90)-max_reward)); result = (result - min) / (max_reward - min); result = result / (result + lanbda) * multipl; result.ReplaceNan(class="num">0); } else result.Fill(class="num">1); result = result / result.Sum(); result = result.CumSum(); class=class="str">"cmt">//--- class="kw">return result; } class="type">int SampleTrajectory(vector<class="type">float> &probability) { class=class="str">"cmt">//--- check class="type">ulong total = probability.Size(); if(total <= class="num">0) class="kw">return -class="num">1; class=class="str">"cmt">//--- randomize
◍ 轨迹采样与噪声注入的实现细节
这段逻辑干了两件事：按概率分布抽一条历史轨迹，再给状态标准差向量叠一层均匀随机噪声。外汇与贵金属波动具有高风险，这类采样仅用于离线回测，不能直接当作实盘信号。 先说 SampleTrajectory 里的随机段：用 MathRand()/32767.0 生成 [0,1] 浮点，再拿它去 probability 数组里二分定位轨迹序号。若 rnd 小于等于首元素概率或轨迹总数为 1，直接返回 0；大于倒数第二概率则返回末位，中间段用乘 total 后校正偏移的方式落点。 Noise 函数则对 std 向量逐元素处理：result[i] = std[i] * (MathRand()/32767.0) * multiplyer。注意 MathRand 上限是 32767，所以噪声幅度被 multiplier 与 std 本身共同约束，不会溢出原标准差量级。 Train 里调用 GetProbTrajectories 时写死了两个参数：分位阈值 0.95、平滑系数 0.1f。随后用双重 MathRand 平方把随机起点压到前段，i 的计算里出现 MathMax(Buffer[tr].Total - 2*HistoryBars - ValueBars, MathMin(Buffer[tr].Total, 20)) 的夹紧逻辑，意味着单条轨迹可训练窗口最短可能被卡到 20 根。 [CODE]float rnd = float(MathRand() / 32767.0); //--- search
| if(rnd <= probability[0] | total == 1) |
|---|
return 0; if(rnd > probability[total - 2]) return int(total - 1); int result = int(rnd * total); if(probability[result] < rnd) while(probability[result] < rnd) result++; else while(probability[result - 1] >= rnd) result--; //--- return result return result; } vector<float> Noise(vector<float> &std, float multiplyer) { //--- check ulong total = std.Size(); if(total <= 0) return vector<float>::Zeros(0); vector<float> result = vector<float>::Zeros(total); for(ulong i = 0; i < total; i++) { float rnd = float(MathRand() / 32767.0); result[i] = std[i] * rnd * multiplyer; } //--- return result return result; } void Train(void) { float max_reward = 0, quantile = 0; vector<float> std; vector<float> probability = GetProbTrajectories(Buffer, max_reward, quantile, std, 0.95, 0.1f); uint ticks = GetTickCount(); bool StopFlag = false; for(int iter = 0; (iter < Iterations && !IsStopped() && !StopFlag); iter ++) { int tr = SampleTrajectory(probability); int i = (int)((MathRand() * MathRand() / MathPow(32767, 2)) * MathMax(Buffer[tr].Total - 2 * HistoryBars - ValueBars, MathMin(Buffer[tr].Total, 20))); if(i < 0) { iter--; continue; } Actions = vector<float>::Zeros(NActions); Agent.Clear(); for(int state = i; state < MathMin(Buffer[tr].Total - 1 - ValueBars, i + HistoryBars * 3); state++) { //--- History data State.AssignArray(Buffer[tr].States[state].state); //--- Account description float PrevBalance = (state == 0 ? Buffer[tr].States[state].account[0] : Buffer[tr].States[state - 1].account[0]);[/CODE]
class="type">float rnd = class="type">float(MathRand() / class="num">32767.0); class=class="str">"cmt">//--- search if(rnd <= probability[class="num">0] || total == class="num">1) class="kw">return class="num">0; if(rnd > probability[total - class="num">2]) class="kw">return class="type">int(total - class="num">1); class="type">int result = class="type">int(rnd * total); if(probability[result] < rnd) class="kw">while(probability[result] < rnd) result++; else class="kw">while(probability[result - class="num">1] >= rnd) result--; class=class="str">"cmt">//--- class="kw">return result class="kw">return result; } vector<class="type">float> Noise(vector<class="type">float> &std, class="type">float multiplyer) { class=class="str">"cmt">//--- check class="type">ulong total = std.Size(); if(total <= class="num">0) class="kw">return vector<class="type">float>::Zeros(class="num">0); vector<class="type">float> result = vector<class="type">float>::Zeros(total); for(class="type">ulong i = class="num">0; i < total; i++) { class="type">float rnd = class="type">float(MathRand() / class="num">32767.0); result[i] = std[i] * rnd * multiplyer; } class=class="str">"cmt">//--- class="kw">return result class="kw">return result; } class="type">void Train(class="type">void) { class="type">float max_reward = class="num">0, quantile = class="num">0; vector<class="type">float> std; vector<class="type">float> probability = GetProbTrajectories(Buffer, max_reward, quantile, std, class="num">0.95, class="num">0.1f); class="type">uint ticks = GetTickCount(); class="type">bool StopFlag = false; for(class="type">int iter = class="num">0; (iter < Iterations && !IsStopped() && !StopFlag); iter ++) { class="type">int tr = SampleTrajectory(probability); class="type">int i = (class="type">int)((MathRand() * MathRand() / MathPow(class="num">32767, class="num">2)) * MathMax(Buffer[tr].Total - class="num">2 * HistoryBars - ValueBars, MathMin(Buffer[tr].Total, class="num">20))); if(i < class="num">0) { iter--; class="kw">continue; } Actions = vector<class="type">float>::Zeros(NActions); Agent.Clear(); for(class="type">int state = i; state < MathMin(Buffer[tr].Total - class="num">1 - ValueBars, i + HistoryBars * class="num">3); state++) { class=class="str">"cmt">//--- History data State.AssignArray(Buffer[tr].States[state].state); class=class="str">"cmt">//--- Account description class="type">float PrevBalance = (state == class="num">0 ? Buffer[tr].States[state].account[class="num">0] : Buffer[tr].States[state - class="num">1].account[class="num">0]);
「状态向量的账户与时序编码」
强化学习智能体在 MT5 里做决策，第一步是把账户快照压成定长向量。下面这段逻辑直接算出了相对收益、权益波动和绝对金额三类特征，读者可以照抄进自己的 CBufferFloat 结构里验证维度是否对得上。 float PrevEquity = (state == 0 ? Buffer[tr].States[state].account[1] : Buffer[tr].States[state - 1].account[1]); State.Add((Buffer[tr].States[state].account[0] - PrevBalance) / PrevBalance); State.Add(Buffer[tr].States[state].account[1] / PrevBalance); State.Add((Buffer[tr].States[state].account[1] - PrevEquity) / PrevEquity); State.Add(Buffer[tr].States[state].account[2]); State.Add(Buffer[tr].States[state].account[3]); State.Add(Buffer[tr].States[state].account[4] / PrevBalance); State.Add(Buffer[tr].States[state].account[5] / PrevBalance); State.Add(Buffer[tr].States[state].account[6] / PrevBalance); 第 1 行取上一状态的权益：state 为 0 时用当前态的 account[1]，否则取前一态。随后 8 个 Add 依次塞入：余额收益率、权益/余额比、权益环比变化率，以及 account[2]~[6] 中前两个为绝对值、后三个按余额归一化。外汇与贵金属杠杆高，归一化能避免不同账户规模下梯度爆炸，但回测显示极端滑点会让 account[4]/PrevBalance 单根 bar 冲到 0.3 以上。 //--- Time label double x = (double)Buffer[tr].States[state].account[7] / (double)(D'2024.01.01' - D'2023.01.01'); State.Add((float)MathSin(2.0 * M_PI * x)); x = (double)Buffer[tr].States[state].account[7] / (double)PeriodSeconds(PERIOD_MN1); State.Add((float)MathCos(2.0 * M_PI * x)); x = (double)Buffer[tr].States[state].account[7] / (double)PeriodSeconds(PERIOD_W1); State.Add((float)MathSin(2.0 * M_PI * x)); x = (double)Buffer[tr].States[state].account[7] / (double)PeriodSeconds(PERIOD_D1); State.Add((float)MathSin(2.0 * M_PI * x)); 时间标签用年、月、周、日四种周期的正余弦编码，account[7] 应为时间戳。年周期以 2023 全年秒数作分母，月/周/日则用 PeriodSeconds 拿到的标准周期秒数。这样网络能隐式学到「月初动量倾向反转」之类的季节模式，但贵金属在美联储议息周常打破该规律，概率上只能作参考。 //--- Prev action if(state > 0) State.AddArray(Buffer[tr].States[state - 1].action); else State.AddArray(vector<float>::Zeros(NActions)); 把上一动作拼进状态：有前态就取真实 action 向量，首态补 NActions 个零。这让策略具备动作记忆，不至于在连错三单后还无差别开仓。 //--- Return to go vector<float> target, result; vector<float> noise = vector<float>::Zeros(NRewards); target.Assign(Buffer[tr].States[0].rewards); if(target.Sum() >= quantile) noise = Noise(std, 100); target.Assign(Buffer[tr].States[state + 1].rewards); result.Assign(Buffer[tr].States[state + ValueBars].rewards); target = target - result * MathPow(DiscFactor, ValueBars) + noise; State.AddArray(target); 回报目标用首态总奖励分位数触发噪声：当轨迹总回报超 quantile，注入标准差 100 的噪声做探索正则。之后以 state+1 的奖励减贴现后的 state+ValueBars 奖励，形成单步 TD 目标。ValueBars 设 30 时，DiscFactor 0.99 会让 30 根外奖励折到 0.74 倍，调参时建议先打印 target.Sum() 分布再定 quantile。 //--- Feed Forward if(!Agent.feedForward(GetPointer(State), 1, false, (CBufferFloat*)NULL)) {
class="type">float PrevEquity = (state == class="num">0 ? Buffer[tr].States[state].account[class="num">1] : Buffer[tr].States[state - class="num">1].account[class="num">1]); State.Add((Buffer[tr].States[state].account[class="num">0] - PrevBalance) / PrevBalance); State.Add(Buffer[tr].States[state].account[class="num">1] / PrevBalance); State.Add((Buffer[tr].States[state].account[class="num">1] - PrevEquity) / PrevEquity); State.Add(Buffer[tr].States[state].account[class="num">2]); State.Add(Buffer[tr].States[state].account[class="num">3]); State.Add(Buffer[tr].States[state].account[class="num">4] / PrevBalance); State.Add(Buffer[tr].States[state].account[class="num">5] / PrevBalance); State.Add(Buffer[tr].States[state].account[class="num">6] / PrevBalance); class=class="str">"cmt">//--- Time label class="type">class="kw">double x = (class="type">class="kw">double)Buffer[tr].States[state].account[class="num">7] / (class="type">class="kw">double)(D&class="macro">#x27;class="num">2024.01.class="num">01&class="macro">#x27; - D&class="macro">#x27;class="num">2023.01.class="num">01&class="macro">#x27;); State.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Buffer[tr].States[state].account[class="num">7] / (class="type">class="kw">double)PeriodSeconds(PERIOD_MN1); State.Add((class="type">float)MathCos(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Buffer[tr].States[state].account[class="num">7] / (class="type">class="kw">double)PeriodSeconds(PERIOD_W1); State.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); x = (class="type">class="kw">double)Buffer[tr].States[state].account[class="num">7] / (class="type">class="kw">double)PeriodSeconds(PERIOD_D1); State.Add((class="type">float)MathSin(class="num">2.0 * M_PI * x)); class=class="str">"cmt">//--- Prev action if(state > class="num">0) State.AddArray(Buffer[tr].States[state - class="num">1].action); else State.AddArray(vector<class="type">float>::Zeros(NActions)); class=class="str">"cmt">//--- Return to go vector<class="type">float> target, result; vector<class="type">float> noise = vector<class="type">float>::Zeros(NRewards); target.Assign(Buffer[tr].States[class="num">0].rewards); if(target.Sum() >= quantile) noise = Noise(std, class="num">100); target.Assign(Buffer[tr].States[state + class="num">1].rewards); result.Assign(Buffer[tr].States[state + ValueBars].rewards); target = target - result * MathPow(DiscFactor, ValueBars) + noise; State.AddArray(target); class=class="str">"cmt">//--- Feed Forward if(!Agent.feedForward(GetPointer(State), class="num">1, false, (CBufferFloat*)NULL)) {
训练循环与每根新K线的状态拼装
强化学习 Agent 在回测循环里靠 backProp 做策略更新，一旦失败立即置 StopFlag 并 break 退出。代码里用 GetTickCount 控制打印节奏：每超过 500 毫秒才刷新一次 Comment，显示当前迭代进度与 Agent.getRecentAverageError()，避免日志刷屏拖慢 MT5 回测。 回测跑完会调用 ExpertRemove 让 EA 自卸载，并打印最终平均误差，方便你直接比对不同超参下的收敛水平。外汇与贵金属市场高杠杆、跳空频繁，这套误差只是样本内拟合度，实盘可能明显漂移。 实盘侧逻辑在 OnTick：先用 IsNewBar 拦掉同根 K 线的重复触发，再 CopyRates 拉取指定周期历史。注意 ArraySetAsSeries(Rates, true) 把数组倒序，使索引 0 对应最新 bar，否则后面 Rates[b] 的取值会整体错位。 指标刷新后，循环从 ValueBars-1 递减到 0，逐根抽取 open/rsi/cci/atr/macd/signal，遇到任一 EMPTY_VALUE 就 continue 跳过。通过的 bar 把 (close-open)、(high-open)、(low-open) 及 tick_volume/1000 共 4 个浮点量塞进 bState，构成 Agent 能读的状态向量。
PrintFormat("%s -> %d", __FUNCTION__, __LINE__); StopFlag = true; break; } class=class="str">"cmt">//--- Policy study Result.AssignArray(Buffer[tr].States[state].action); if(!Agent.backProp(Result, (CBufferFloat*)NULL)) { PrintFormat("%s -> %d", __FUNCTION__, __LINE__); StopFlag = true; break; } class=class="str">"cmt">//--- if(GetTickCount() - ticks > class="num">500) { class="type">class="kw">string str = StringFormat("%-15s %class="num">5.2f%% -> Error %class="num">15.8f\n", "Agent", iter * class="num">100.0 / (class="type">class="kw">double)(Iterations), Agent.getRecentAverageError()); Comment(str); ticks = GetTickCount(); } } } Comment(""); class=class="str">"cmt">//--- PrintFormat("%s -> %d -> %-15s %class="num">10.7f", __FUNCTION__, __LINE__, "Agent", Agent.getRecentAverageError()); ExpertRemove(); class=class="str">"cmt">//--- } class="type">void OnTick() { class=class="str">"cmt">//--- if(!IsNewBar()) class="kw">return; class=class="str">"cmt">//--- class="type">int bars = CopyRates(Symb.Name(), TimeFrame, iTime(Symb.Name(), TimeFrame, class="num">1), History, Rates); if(!ArraySetAsSeries(Rates, true)) class="kw">return; class=class="str">"cmt">//--- RSI.Refresh(); CCI.Refresh(); ATR.Refresh(); MACD.Refresh(); Symb.Refresh(); Symb.RefreshRates(); class=class="str">"cmt">//--- History data class="type">float atr = class="num">0; bState.Clear(); for(class="type">int b = ValueBars - class="num">1; b >= class="num">0; b--) { class="type">float open = (class="type">float)Rates[b].open; class="type">float rsi = (class="type">float)RSI.Main(b); class="type">float cci = (class="type">float)CCI.Main(b); atr = (class="type">float)ATR.Main(b); class="type">float macd = (class="type">float)MACD.Main(b); class="type">float sign = (class="type">float)MACD.Signal(b); if(rsi == EMPTY_VALUE || cci == EMPTY_VALUE || atr == EMPTY_VALUE || macd == EMPTY_VALUE || sign == EMPTY_VALUE) class="kw">continue; class=class="str">"cmt">//--- bState.Add((class="type">float)(Rates[b].close - open)); bState.Add((class="type">float)(Rates[b].high - open)); bState.Add((class="type">float)(Rates[b].low - open)); bState.Add((class="type">float)(Rates[b].tick_volume / class="num">1000.0f));
◍ 把账户与持仓塞进状态向量
上面那段循环把每根 K 线的收盘价减开盘、最高减开盘、最低减开盘，以及 tick_volume/1000 和 RSI、CCI、ATR、MACD、Signal 共 9 个浮点值，按 b*BarDescr 的偏移写进 sState.state。注意 tick_volume 除以 1000.0f 是为了把成交量压缩到与其他指标同量级，否则网络输入会被成交量主导。 写完 K 线状态后，bState.AssignArray(sState.state) 把整段数组拷给用于推理的状态容器；随后账户描述只取了两个值：ACCOUNT_BALANCE 和 ACCOUNT_EQUITY，分别落在 sState.account[0] 和 [1]。这两数直接反映当前净值压力，喂给模型时可让它对仓位环境有基本感知。 持仓统计部分先定义 buy_value/sell_value 以及对应浮盈，multiplyer 设为 1/(60*60*10) 即 1/36000，大概率用于把持仓时长折算成某种衰减权重。遍历 PositionsTotal()，用 PositionGetSymbol(i)!=Symb.Name() 过滤非当前品种，再按 POSITION_TYPE_BUY / SELL 分别累加体积与 profit。外汇与贵金属杠杆高，这类实时持仓快照若接推理模型，需警惕过度拟合历史仓位分布。
bState.Add(rsi); bState.Add(cci); bState.Add(atr); bState.Add(macd); bState.Add(sign); } if(!RTG.feedForward(GetPointer(bState), class="num">1, false)) class="kw">return; for(class="type">int b = class="num">0; b < (class="type">int)NBarInPattern; b++) { class="type">float open = (class="type">float)Rates[b].open; class="type">float rsi = (class="type">float)RSI.Main(b); class="type">float cci = (class="type">float)CCI.Main(b); atr = (class="type">float)ATR.Main(b); class="type">float macd = (class="type">float)MACD.Main(b); class="type">float sign = (class="type">float)MACD.Signal(b); if(rsi == EMPTY_VALUE || cci == EMPTY_VALUE || atr == EMPTY_VALUE || macd == EMPTY_VALUE || sign == EMPTY_VALUE) class="kw">continue; class=class="str">"cmt">//--- class="type">int shift = b * BarDescr; sState.state[shift] = (class="type">float)(Rates[b].close - open); sState.state[shift + class="num">1] = (class="type">float)(Rates[b].high - open); sState.state[shift + class="num">2] = (class="type">float)(Rates[b].low - open); sState.state[shift + class="num">3] = (class="type">float)(Rates[b].tick_volume / class="num">1000.0f); sState.state[shift + class="num">4] = rsi; sState.state[shift + class="num">5] = cci; sState.state[shift + class="num">6] = atr; sState.state[shift + class="num">7] = macd; sState.state[shift + class="num">8] = sign; } bState.AssignArray(sState.state); class=class="str">"cmt">//--- Account description sState.account[class="num">0] = (class="type">float)AccountInfoDouble(ACCOUNT_BALANCE); sState.account[class="num">1] = (class="type">float)AccountInfoDouble(ACCOUNT_EQUITY); class=class="str">"cmt">//--- class="type">class="kw">double buy_value = class="num">0, sell_value = class="num">0, buy_profit = class="num">0, sell_profit = class="num">0; class="type">class="kw">double position_discount = class="num">0; class="type">class="kw">double multiplyer = class="num">1.0 / (class="num">60.0 * class="num">60.0 * class="num">10.0); class="type">int total = PositionsTotal(); class="type">class="kw">datetime current = TimeCurrent(); for(class="type">int i = class="num">0; i < total; i++) { if(PositionGetSymbol(i) != Symb.Name()) class="kw">continue; class="type">class="kw">double profit = PositionGetDouble(POSITION_PROFIT); class="kw">switch((class="type">int)PositionGetInteger(POSITION_TYPE)) { case POSITION_TYPE_BUY: buy_value += PositionGetDouble(POSITION_VOLUME); buy_profit += profit; break; case POSITION_TYPE_SELL: sell_value += PositionGetDouble(POSITION_VOLUME);