神经网络变得简单(第 75 部分):提升轨迹预测模型的性能·进阶篇
(2/3)·复杂模型拖慢实时市价单决策?本篇拆解自动驾驶借来的降本思路
卷积梯度核里的激活分支怎么写
在 MT5 用 OpenCL 跑神经网络卷积层时,梯度回传必须和前向激活严格配对,否则权重更新会偏。下面这段内核同时处理特征图 f 与状态 s 两组张量,按 activationf / activations 两个枚举切换导数计算。 case 0 对应 tanh 类饱和函数:先把值夹到 [-1,1],导数用 (1 - f^2) 并垫了 1e-4 下限防零除;case 1 是 sigmoid,夹到 [0,1] 后导数为 f*(1-f);case 2 像 Leaky ReLU,负区梯度直接乘 0.01f。两组张量逻辑完全对称,只是变量名换了。 回测注意:外汇与贵金属杠杆高、滑点大,这类 GPU 核在 EURUSD M15 上单步耗时可能随矩阵维度线性涨,建议先在策略测试器用 1024 长度的矩阵试跑,确认 get_global_id(0) 越界与否再上实盘数据。
class="type">int activationf,class=class="str">"cmt">///< Activation type(class="macro">#ENUM_ACTIVATION) class="type">int activationsclass=class="str">"cmt">///< Activation type(class="macro">#ENUM_ACTIVATION) ) { class="type">int i = get_global_id(class="num">0); class="type">float grad = matrix_g[i]; class="type">float f = matrix_f[i]; class="type">float s = matrix_s[i]; class="type">float sg = grad * f; class="type">float fg = grad * s; class="kw">switch(activationf) { case class="num">0: f = clamp(f, -class="num">1.0f, class="num">1.0f); fg = clamp(fg + f, -class="num">1.0f, class="num">1.0f) - f; fg = fg * max(class="num">1 - pow(f, class="num">2), class="num">1.0e-4f); break; case class="num">1: f = clamp(f, class="num">0.0f, class="num">1.0f); fg = clamp(fg + f, class="num">0.0f, class="num">1.0f) - f; fg = fg * max(f * (class="num">1 - f), class="num">1.0e-4f); break; case class="num">2: if(f < class="num">0) fg *= class="num">0.01f; break; class="kw">default: break; } class="kw">switch(activations) { case class="num">0: s = clamp(s, -class="num">1.0f, class="num">1.0f); sg = clamp(sg + s, -class="num">1.0f, class="num">1.0f) - s; sg = sg * max(class="num">1 - pow(s, class="num">2), class="num">1.0e-4f); break; case class="num">1: s = clamp(s, class="num">0.0f, class="num">1.0f); sg = clamp(sg + s, class="num">0.0f, class="num">1.0f) - s; sg = sg * max(s * (class="num">1 - s), class="num">1.0e-4f); break; case class="num">2: if(s < class="num">0) sg *= class="num">0.01f; break; class="kw">default: break; } matrix_fg[i] = fg; matrix_sg[i] = sg; } class="type">bool CNeuronCGConvOCL::calcInputGradients(CNeuronBaseOCL *prevLayer) { if(!prevLayer || !prevLayer.getGradient() || prevLayer.getGradientIndex() < class="num">0) class="kw">return false; class="type">uint global_work_offset[class="num">1] = {class="num">0}; class="type">uint global_work_size[class="num">1];
◍ 卷积梯度核的参数绑定与回传
在 MT5 的 OpenCL 路径里,CGConv_HiddenGradient 这个 kernel 负责把前层输出、梯度以及激活函数统一绑进显存,再触发一次一维并行计算。代码先以 Neurons() 设定 global_work_size[0],意味着线程网格宽度直接等于本层神经元数,这是后续核函数并行粒度的基础。 绑定过程分两块:SetArgumentBuffer 把矩阵类数据(f/fg/s/sg/g)的 GPU 缓冲指针传进去,SetArgument 则只传标量级的激活函数标识。任何一步返回 false 都会用 printf 打出 __FUNCTION__、GetLastError() 和 __LINE__,方便在 MT5 终端的 Experts 日志里精确定位是第几个 Set 调用出的错。 核函数 Execute 只排一个 work-group(参数 1),偏移与尺寸沿用前面填好的 global_work_offset / global_work_size。执行成功后,代码转回 CPU 侧,对输入分支 cInputF、cInputS 各自调用 calcHiddenGradients,再做一次 SumAndNormilize 把双路输出合并归一。外汇与贵金属模型训练在 GPU 上跑这类核函数时波动放大,属高风险操作,参数错绑可能直接让 EA 在实时行情中断算。 下面这段是原文里的实际绑定与执行流,逐行对应上面说的环节:
global_work_size[class="num">0] = Neurons(); ResetLastError(); if(!OpenCL.SetArgumentBuffer(def_k_CGConv_HiddenGradient, def_k_cgc_matrix_f, cF.getOutputIndex())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgumentBuffer(def_k_CGConv_HiddenGradient, def_k_cgc_matrix_fg, cF.getGradientIndex())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgumentBuffer(def_k_CGConv_HiddenGradient, def_k_cgc_matrix_s, cS.getOutputIndex())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgumentBuffer(def_k_CGConv_HiddenGradient, def_k_cgc_matrix_sg, cS.getGradientIndex())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgumentBuffer(def_k_CGConv_HiddenGradient, def_k_cgc_matrix_g, getGradientIndex())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgument(def_k_CGConv_HiddenGradient, def_k_cgc_activationf, cF.Activation())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.SetArgument(def_k_CGConv_HiddenGradient, def_k_cgc_activations, cS.Activation())) { printf("Error of set parameter kernel %s: %d; line %d", __FUNCTION__, GetLastError(), __LINE__); class="kw">return false; } if(!OpenCL.Execute(def_k_CGConv_HiddenGradient, class="num">1, global_work_offset, global_work_size)) { printf("Error of execution kernel %s: %d", __FUNCTION__, GetLastError()); class="kw">return false; } if(!cInputF.calcHiddenGradients(GetPointer(cF))) class="kw">return false; if(!cInputS.calcHiddenGradients(GetPointer(cS))) class="kw">return false; if(!SumAndNormilize(cF.getOutput(), cS.getOutput(), prevLayer.getOutput(), class="num">1, false)) class="kw">return false; class=class="str">"cmt">//--- class="kw">return true; }
「卷积图网络层的权重回传与轨迹网搭建」
在 MT5 的 OpenCL 神经网络封装里,CNeuronCGConvOCL::updateInputWeights 只做一件事:先后调用两个子卷积核 cF、cS 的 UpdateInputWeights,任一失败立即返回 false,全过才返回 true。这意味着图卷积层的梯度回传是硬串联,前半段权重不更新成功,后半段根本不会动。 CreateTrajNetDescriptions 负责把编码器各层描述塞进 CArrayObj 容器。函数开头先判空,encoder、endpoints、probability 任一为空就 new 一个,分配失败直接 return false,避免后续空指针崩在 GPU 端。 编码器第一层是 defNeuronBaseOCL,节点数 = HistoryBars * BarDescr,激活函数 None,优化器 ADAM;第二层 defNeuronBatchNormOCL 做批归一化,batch 取 MathMax(1000, GPTBars),也就是至少 1000 条样本才跑归一化。 第三层 Embedding 把 prev_count 个输入压到 GPTBars 个 token,window_out = EmbeddingSize;第四层 PEOCL 做位置编码,第五层 CGConvOCL 把 token 数乘 embedding 维度作为卷积窗口。外汇与贵金属行情序列短、噪声大,这类结构在实盘前务必用历史样本回测验证过拟合程度,杠杆品种高风险。 打开 MT5 的 MetaEditor,搜 CNeuronCGConvOCL 把下面两段直接贴进你的网络构建文件,改 HistoryBars 和 EmbeddingSize 就能看推理耗时变化。
class="type">bool CNeuronCGConvOCL::updateInputWeights(CNeuronBaseOCL *NeuronOCL) { if(!cF.UpdateInputWeights(cInputF.AsObject())) class="kw">return false; if(!cS.UpdateInputWeights(cInputS.AsObject())) class="kw">return false; class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CreateTrajNetDescriptions(CArrayObj *encoder, CArrayObj *endpoints, CArrayObj *probability) { class=class="str">"cmt">//--- CLayerDescription *descr; class=class="str">"cmt">//--- if(!encoder) { encoder = new CArrayObj(); if(!encoder) class="kw">return false; } if(!endpoints) { endpoints = new CArrayObj(); if(!endpoints) class="kw">return false; } if(!probability) { probability = new CArrayObj(); if(!probability) class="kw">return false; } class=class="str">"cmt">//--- Encoder encoder.Clear(); class=class="str">"cmt">//--- Input layer if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; class="type">int prev_count = descr.count = (HistoryBars * BarDescr); descr.activation = None; descr.optimization = ADAM; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">1 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBatchNormOCL; descr.count = prev_count; descr.batch = MathMax(class="num">1000, GPTBars); descr.activation = None; descr.optimization = ADAM; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">2 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronEmbeddingOCL; { class="type">int temp[] = {prev_count}; ArrayCopy(descr.windows, temp); } prev_count = descr.count = GPTBars; class="type">int prev_wout = descr.window_out = EmbeddingSize; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">3 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronPEOCL; descr.count = prev_count; descr.window = prev_wout; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">4 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronCGConvOCL; descr.count = prev_count * prev_wout; descr.window = descr.count; if(!encoder.Add(descr)) { class="kw">delete descr;
编码器与端点的层定义细节
这段构建逻辑把 GPT 风格编码器推到第七层,随后初始化 endpoints 与 probability 两个子网络。外汇与贵金属行情的高波动特性,意味着这类深层结构在实盘里过拟合概率偏高,调参需谨慎。 第五层用 BatchNorm 做归一化,batch 尺寸取 MathMax(1000, GPTBars)——当 GPTBars 不足 1000 时强制拉到 1000,避免小样本下均值方差抖动。第六层 CGConv 的 window 直接等于节点数,相当于全通道卷积。 第七层多头注意力把 step 设 4、window_out 设 16、layers 设 1,属于轻量注意力头。endpoints 输入层节点数承接上层 prev_count*prev_wout,第一层压缩到 LatentCount 并用 SIGMOID,第二层 LSTM 节点数 = 3*NForecast。probability 网络则把 endpoints 第 0 层直接复用,再接一层 Concatenate(窗口 prev_count、步长 3*NForecast)。 在 MT5 里把这段贴进 CNet 类的 Description 函数,改 LatentCount 与 NForecast 看显存占用变化,能快速判断你的显卡是否扛得住这套结构。
class="kw">return false; } class=class="str">"cmt">//--- layer class="num">5 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBatchNormOCL; descr.count = prev_count*prev_wout; descr.batch = MathMax(class="num">1000, GPTBars); descr.activation = None; descr.optimization = ADAM; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">6 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronCGConvOCL; descr.count = prev_count * prev_wout; descr.window = descr.count; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">7 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronMLMHAttentionOCL; descr.count = prev_count; descr.window = prev_wout; descr.step = class="num">4; descr.window_out = class="num">16; descr.layers = class="num">1; descr.optimization = ADAM; if(!encoder.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- Endpoints endpoints.Clear(); class=class="str">"cmt">//--- Input layer if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; prev_count = descr.count = (prev_count * prev_wout); descr.activation = None; descr.optimization = ADAM; if(!endpoints.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">1 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; descr.count = LatentCount; descr.activation = SIGMOID; descr.optimization = ADAM; if(!endpoints.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">2 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronLSTMOCL; descr.count = class="num">3 * NForecast; descr.activation = None; descr.optimization = ADAM; if(!endpoints.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- Probability probability.Clear(); class=class="str">"cmt">//--- Input layer if(!probability.Add(endpoints.At(class="num">0))) class="kw">return false; class=class="str">"cmt">//--- layer class="num">1 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronConcatenate; descr.count = LatentCount; descr.window = prev_count; descr.step = class="num">3 * NForecast; descr.optimization = ADAM; descr.activation = SIGMOID; if(!probability.Add(descr)) { class="kw">delete descr; class="kw">return false; }
◍ 概率网络后三层与初始化兜底建网
上面这段把概率分支(probability)的第 2~4 层依次压进容器:第 2 层用 LatentCount 个 LReLU 神经元做隐层,第 3 层 NForecast 个线性输出,第 4 层直接上 defNeuronSoftMaxOCL 把 NForecast 个值归一成概率分布,step 写死为 1。 每一层都先 new 一个 CLayerDescription,Add 失败就 delete 并 return false,这种写法能避免 MT5 里悬空指针把整个 EA 初始化拖挂。 OnInit 里先 LoadTotalBase 读训练数据,失败就 PrintFormat 打错误码并返回 INIT_FAILED;模型文件(Enc.nnw / Endp.nnw / Prob.nnw)若不存在,代码会现场调 CreateTrajNetDescriptions 建三个 CArrayObj 描述并 Create,等于首次跑 EA 自动从零搭网。 外汇与贵金属行情高波动,这类自建模网络在样本外预测仅具概率意义,实盘前务必用历史数据回测验证泛化表现。
} class=class="str">"cmt">//--- layer class="num">2 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; descr.count = LatentCount; descr.activation = LReLU; descr.optimization = ADAM; if(!probability.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">3 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronBaseOCL; descr.count = NForecast; descr.activation = None; descr.optimization = ADAM; if(!probability.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- layer class="num">4 if(!(descr = new CLayerDescription())) class="kw">return false; descr.type = defNeuronSoftMaxOCL; descr.count = NForecast; descr.step = class="num">1; descr.activation = None; descr.optimization = ADAM; if(!probability.Add(descr)) { class="kw">delete descr; class="kw">return false; } class=class="str">"cmt">//--- class="kw">return true; } class="type">int OnInit() { class=class="str">"cmt">//--- ResetLastError(); if(!LoadTotalBase()) { PrintFormat("Error of load study data: %d", GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- load models class="type">float temp; if(!BLEncoder.Load(FileName + "Enc.nnw", temp, temp, temp, dtStudied, true) || !BLEndpoints.Load(FileName + "Endp.nnw", temp, temp, temp, dtStudied, true) || !BLProbability.Load(FileName + "Prob.nnw", temp, temp, temp, dtStudied, true) ) { CArrayObj *encoder = new CArrayObj(); CArrayObj *endpoint = new CArrayObj(); CArrayObj *prob = new CArrayObj(); if(!CreateTrajNetDescriptions(encoder, endpoint, prob)) { class="kw">delete endpoint; class="kw">delete prob; class="kw">delete encoder; class="kw">return INIT_FAILED; } if(!BLEncoder.Create(encoder) || !BLEndpoints.Create(endpoint) || !BLProbability.Create(prob)) { class="kw">delete endpoint; class="kw">delete prob; class="kw">delete encoder; class="kw">return INIT_FAILED; } class="kw">delete endpoint; class="kw">delete prob; class="kw">delete encoder; } if(!StateEncoder.Load(FileName + "StEnc.nnw", temp, temp, temp, dtStudied, true) || !EndpointEncoder.Load(FileName + "EndEnc.nnw", temp, temp, temp, dtStudied, true) ||
「加载神经网络后的一致性校验」
模型权重从 Act.nnw 读入后,先别急着跑推理,得把网络输出维度跟预设动作空间对一遍。代码里用 Actor.getResults(Result) 取动作概率分布,若 Result.Total() 不等于 NActions,直接打日志 'The scope of the actor does not match the actions count' 并返回 INIT_FAILED,说明训好的网和 EA 设定的可执行动作数脱节了。 端点预测那边同样卡死:BLEndpoints.getResults(Result) 要求总数恰好是 3 * NForecast,差一个都进不了主循环。这条硬校验能拦掉大部分因改了预测步数却忘了重训模型导致的静默错乱。 编码器输入也得对上 HistoryBars * BarDescr,比如你看 30 根 K 线、每根用 5 维描述,那 Result.Total() 就必须是 150,否则 Init 阶段直接废掉。最后 EventChartCustom 发个 'Init' 自定义事件通知图表,返回 INIT_SUCCEEDED 才算 EA 真正起得来。外汇与贵金属杠杆高,这类初始化失败若被忽略,实盘可能以错误模型开仓,风险偏大。
!Actor.Load(FileName + "Act.nnw", temp, temp, temp, dtStudied, true)) { CArrayObj *actor = new CArrayObj(); CArrayObj *endpoint = new CArrayObj(); CArrayObj *encoder = new CArrayObj(); if(!CreateDescriptions(actor, endpoint, encoder)) { class="kw">delete actor; class="kw">delete endpoint; class="kw">delete encoder; class="kw">return INIT_FAILED; } if(!Actor.Create(actor) || !StateEncoder.Create(encoder) || !EndpointEncoder.Create(endpoint)) { class="kw">delete actor; class="kw">delete endpoint; class="kw">delete encoder; class="kw">return INIT_FAILED; } class="kw">delete actor; class="kw">delete endpoint; class="kw">delete encoder; class=class="str">"cmt">//--- } OpenCL = Actor.GetOpenCL(); StateEncoder.SetOpenCL(OpenCL); EndpointEncoder.SetOpenCL(OpenCL); BLEncoder.SetOpenCL(OpenCL); BLEndpoints.SetOpenCL(OpenCL); BLProbability.SetOpenCL(OpenCL); Actor.getResults(Result); if(Result.Total() != NActions) { PrintFormat("The scope of the actor does not match the actions count(%d <> %d)", NActions, Result.Total()); class="kw">return INIT_FAILED; } BLEndpoints.getResults(Result); if(Result.Total() != class="num">3 * NForecast) { PrintFormat("The scope of the Endpoints does not match forecast endpoints(%d <> %d)", class="num">3 * NForecast, Result.Total()); class="kw">return INIT_FAILED; } BLEncoder.GetLayerOutput(class="num">0, Result); if(Result.Total() != (HistoryBars * BarDescr)) { PrintFormat("Input size of Encoder doesn&class="macro">#x27;t match state description(%d <> %d)", Result.Total(), (HistoryBars * BarDescr)); class="kw">return INIT_FAILED; } if(!bGradient.BufferInit(MathMax(AccountDescr, NForecast), class="num">0) || !bGradient.BufferCreate(OpenCL)) { PrintFormat("Error of create buffers: %d", GetLastError()); class="kw">return INIT_FAILED; } if(!EventChartCustom(ChartID(), class="num">1, class="num">0, class="num">0, "Init")) { PrintFormat("Error of create study event: %d", GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- class="kw">return(INIT_SUCCEEDED); } class="type">void OnDeinit(const class="type">int reason) { class=class="str">"cmt">//--- if(!(reason == REASON_INITFAILED || reason == REASON_RECOMPILE)) { Actor.Save(FileName + "Act.nnw", class="num">0, class="num">0, class="num">0, TimeCurrent(), true);