神经网络变得轻松（第十部分）：多目击者关注·综合运用
◍ 注意力层缓冲与神经元的惰性初始化
在 MT5 的 OpenCL 神经网络封装里，Scores3、Scores4 这类分数缓冲和 AttentionOut 系列神经元都走「指针无效才 new」的惰性构造，避免重复建对象拖慢回测。 下面这段是 Scores3 的初始化链：先判指针，再开 CBufferDouble，随后用 units_count*units_count 做长度、0.0 做初值调 BufferInit，最后挂到 OpenCL 上下文 BufferCreate。任一环节失败直接 return false，保证后续前向计算不会踩空指针。 Scores4 的构造与 Scores3 完全对称，只是换了变量名，长度同样是 units_count 的平方。 AttentionOut2~4 则是 CNeuronBaseOCL 神经元：Init 的第二个参数分别是 12、13、14，窗口乘 units_count 作输入宽，激活函数统一设 None——意味着这三路输出不做非线性压缩，留给后面的 Concatenate 层（编号 15，输入宽 4*window*units_count）去汇总。 开 MT5 把这段贴进你的网络类 Init 函数，改 units_count 或 window 就能直观看到显存申请规模随平方级膨胀，外汇与贵金属模型训练属高风险，参数炸了先查这里。
if(CheckPointer(Scores3)==POINTER_INVALID) { Scores3=new CBufferDouble(); if(CheckPointer(Scores3)==POINTER_INVALID) class="kw">return class="kw">false; } if(!Scores3.BufferInit(units_count*units_count,class="num">0.0)) class="kw">return class="kw">false; if(!Scores3.BufferCreate(OpenCL)) class="kw">return class="kw">false; class=class="str">"cmt">//--- if(CheckPointer(Scores4)==POINTER_INVALID) { Scores4=new CBufferDouble(); if(CheckPointer(Scores4)==POINTER_INVALID) class="kw">return class="kw">false; } if(!Scores4.BufferInit(units_count*units_count,class="num">0.0)) class="kw">return class="kw">false; if(!Scores4.BufferCreate(OpenCL)) class="kw">return class="kw">false; class=class="str">"cmt">//--- if(CheckPointer(AttentionOut2)==POINTER_INVALID) { AttentionOut2=new CNeuronBaseOCL(); if(CheckPointer(AttentionOut2)==POINTER_INVALID) class="kw">return class="kw">false; if(!AttentionOut2.Init(class="num">0,class="num">12,open_cl,window*units_count,optimization_type)) class="kw">return class="kw">false; AttentionOut2.SetActivationFunction(None); } class=class="str">"cmt">//--- if(CheckPointer(AttentionOut3)==POINTER_INVALID) { AttentionOut3=new CNeuronBaseOCL(); if(CheckPointer(AttentionOut3)==POINTER_INVALID) class="kw">return class="kw">false; if(!AttentionOut3.Init(class="num">0,class="num">13,open_cl,window*units_count,optimization_type)) class="kw">return class="kw">false; AttentionOut3.SetActivationFunction(None); } class=class="str">"cmt">//--- if(CheckPointer(AttentionOut4)==POINTER_INVALID) { AttentionOut4=new CNeuronBaseOCL(); if(CheckPointer(AttentionOut4)==POINTER_INVALID) class="kw">return class="kw">false; if(!AttentionOut4.Init(class="num">0,class="num">14,open_cl,window*units_count,optimization_type)) class="kw">return class="kw">false; AttentionOut4.SetActivationFunction(None); } if(CheckPointer(AttentionConcatenate)==POINTER_INVALID) { AttentionConcatenate=new CNeuronBaseOCL(); if(CheckPointer(AttentionConcatenate)==POINTER_INVALID) class="kw">return class="kw">false; if(!AttentionConcatenate.Init(class="num">0,class="num">15,open_cl,class="num">4*window*units_count,optimization_type)) class="kw">return class="kw">false; }
多头注意力里的归一化与四路查询投射
在 CNeuronMHAttentionOCL::feedForward 里，第一步不是直接算注意力，而是先把上一层输出做归一化。代码用 OpenCL 内核 def_k_Normilize 跑归一化，global_work_size[0] 设为 1，意味着整批样本只发一个工作项去处理，适合小批次或调试期观察数值稳定性。 归一化前要先绑缓冲区：OpenCL.SetArgumentBuffer 把 prevLayer 的输出索引挂到内核，SetArgument 传维度 prevLayer.Neurons()。若 Execute 返回失败，会 printf 打出错误码 GetLastError()，这时候 feedForward 直接 return false，前向传播中断。 归一化之后，类里连续拉起四路查询（Querys / Querys2 / Querys3 / Querys4）和两路数值（Values / Values2 / Values3 至少三路）分支。每一路都先 CheckPointer 判空，再调各自的 FeedForward(prevLayer)；任意一路指针无效或前向失败，整个方法返回 false。这种结构说明该注意力头刻意把输入拆成多组子空间投射，外汇或贵金属行情序列用这类结构时，过拟合风险偏高，参数窗口调大可能让回测曲线好看但实盘概率衰减。 顺带一提，同文件里 Weights0 的初始化用 CNeuronConvOCL，Init 参数含 0,16,open_cl,4*window,4*window,window,units_count——卷积核把窗口扩成 4 倍感受野，激活函数设 None，说明这一层有意保留线性叠加特征供后续注意力加权。
AttentionConcatenate.SetActivationFunction(None); } if(CheckPointer(Weights0)==POINTER_INVALID) { Weights0=new CNeuronConvOCL(); if(CheckPointer(Weights0)==POINTER_INVALID) class="kw">return class="kw">false; if(!Weights0.Init(class="num">0,class="num">16,open_cl,class="num">4*window,class="num">4*window,window,units_count,optimization_type)) class="kw">return class="kw">false; Weights0.SetActivationFunction(None); } class=class="str">"cmt">//--- class="kw">return true; } __kernel class="type">void ConcatenateBuffers(__global class="type">class="kw">double *input1, class="type">int window1, __global class="type">class="kw">double *input2, class="type">int window2, __global class="type">class="kw">double *input3, class="type">int window3, __global class="type">class="kw">double *input4, class="type">int window4, __global class="type">class="kw">double *output) class="type">bool CNeuronMHAttentionOCL::feedForward(CNeuronBaseOCL *prevLayer) { if(CheckPointer(prevLayer)==POINTER_INVALID) class="kw">return class="kw">false; class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=class="num">1; OpenCL.SetArgumentBuffer(def_k_Normilize,def_k_norm_buffer,prevLayer.getOutputIndex()); OpenCL.SetArgument(def_k_Normilize,def_k_norm_dimension,prevLayer.Neurons()); if(!OpenCL.Execute(def_k_Normilize,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel Normalize: %d",GetLastError()); class="kw">return class="kw">false; } if(!prevLayer.Output.BufferRead()) class="kw">return class="kw">false; } if(CheckPointer(Querys)==POINTER_INVALID || !Querys.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Querys2)==POINTER_INVALID || !Querys2.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Querys3)==POINTER_INVALID || !Querys3.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Querys4)==POINTER_INVALID || !Querys4.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Values)==POINTER_INVALID || !Values.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Values2)==POINTER_INVALID || !Values2.FeedForward(prevLayer)) class="kw">return class="kw">false; if(CheckPointer(Values3)==POINTER_INVALID || !Values3.FeedForward(prevLayer))
「多头注意力在 GPU 上的内核串联」
这段逻辑把四个注意力头算完的分数和输出，在 OpenCL 里逐层喂给卷积式全连接。先看第一个头：用 def_k_AttentionScore 内核算 Query 和 Key 的点积，global_work_size[0] 直接取 iUnits，意味着每个单元并行跑一个 score。 若 Values4 指针无效或前层 FeedForward 失败，立刻 return false，避免脏数据进显存。Scores.BufferRead() 不成功也同样截断，这是 MT5 调 GPU 时最容易漏的一步——不回读就下游全错。 第二个内核 def_k_AttentionOut 是二维调度：global_work_size 设成 [iUnits, iWindow]，把 score 和 value 做加权求和。注意这里 temp[] 只是借 getOutputVal 触发一次同步读取，并不参与计算。 Concatenate 内核把四个头的输出按 iWindow 长度拼起来，def_k_conc_window1~4 都传同一个 iWindow，说明四头窗口等长。拼完的矩阵交给 Weights0.FeedForward，整条多头通路才算通。 外汇与贵金属行情跳空频繁，这类 GPU 注意力模型在实时推理时若 iWindow 设得过大，可能拖慢 MT5 的 tick 响应，建议先在策略测试器里用 1 分钟 EURUSD 实测 iUnits=32、iWindow=10 的耗时。
class="kw">return class="kw">false; if(CheckPointer(Values4)==POINTER_INVALID || !Values4.FeedForward(prevLayer)) class="kw">return class="kw">false; class=class="str">"cmt">//--- Scores Head class="num">1 { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_AttentionScore,def_k_as_querys,Querys.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionScore,def_k_as_keys,prevLayer.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionScore,def_k_as_score,Scores.GetIndex()); OpenCL.SetArgument(def_k_AttentionScore,def_k_as_dimension,iWindow); if(!OpenCL.Execute(def_k_AttentionScore,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel AttentionScore: %d",GetLastError()); class="kw">return class="kw">false; } if(!Scores.BufferRead()) class="kw">return class="kw">false; } class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">2]={class="num">0,class="num">0}; class="type">uint global_work_size[class="num">2]; global_work_size[class="num">0]=iUnits; global_work_size[class="num">1]=iWindow; OpenCL.SetArgumentBuffer(def_k_AttentionOut,def_k_aout_scores,Scores.GetIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionOut,def_k_aout_inputs,prevLayer.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionOut,def_k_aout_values,Values.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionOut,def_k_aout_out,AttentionOut.getOutputIndex()); if(!OpenCL.Execute(def_k_AttentionOut,class="num">2,global_work_offset,global_work_size)) { printf("Error of execution kernel Attention Out: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(!AttentionOut.getOutputVal(temp)) class="kw">return class="kw">false; } { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_ConcatenateMatrix,def_k_conc_input1,AttentionOut.getOutputIndex()); OpenCL.SetArgument(def_k_ConcatenateMatrix,def_k_conc_window1,iWindow); OpenCL.SetArgumentBuffer(def_k_ConcatenateMatrix,def_k_conc_input2,AttentionOut2.getOutputIndex()); OpenCL.SetArgument(def_k_ConcatenateMatrix,def_k_conc_window2,iWindow); OpenCL.SetArgumentBuffer(def_k_ConcatenateMatrix,def_k_conc_input3,AttentionOut3.getOutputIndex()); OpenCL.SetArgument(def_k_ConcatenateMatrix,def_k_conc_window3,iWindow); OpenCL.SetArgumentBuffer(def_k_ConcatenateMatrix,def_k_conc_input4,AttentionOut4.getOutputIndex()); OpenCL.SetArgument(def_k_ConcatenateMatrix,def_k_conc_window4,iWindow); OpenCL.SetArgumentBuffer(def_k_ConcatenateMatrix,def_k_conc_out,AttentionConcatenate.getOutputIndex()); if(!OpenCL.Execute(def_k_ConcatenateMatrix,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel Concatenate Matrix: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(!AttentionConcatenate.getOutputVal(temp)) class="kw">return class="kw">false; } if(CheckPointer(Weights0)==POINTER_INVALID || !Weights0.FeedForward(AttentionConcatenate)) class="kw">return class="kw">false; { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits;
◍ GPU 上跑通神经网络前向的实作片段
这段 MT5 的 OpenCL 调用把神经网络两层前向传播搬到了显卡上算。先拿 MatrixSum 内核把权重层输出与上一层结果按 0.5 系数做矩阵加和，iWindow 作为维度参数传入，global_work_size 设 1 先归一化权重缓冲。 归一化内核 Normilize 用 Weights0.Neurons() 做维度，跑完直接把显存读回 temp 数组；若 BufferRead 失败函数立即返回 false，这种细粒度错误捕获在高频重算时很关键。 第二轮 MatrixSum 把 Weights0 与 FF2 输出相加写进最终 Output，global_work_size 换成 iUnits（样本数），同样 0.5 加权。外汇与贵金属行情用这类模型推理时波动剧烈，GPU 异步出错可能导致信号延迟，实盘前务必在策略测试器用历史 Tick 验证内核稳定性。 DeconcatenateBuffers 内核则按 window1~window4 把拼好的 inputs 拆回四个独立缓冲，shift 步进量为各窗口长度之和，get_global_id(0) 定位样本序号。复制下面代码到 .mq5 的神经网络类方法里，改 iWindow 与 iUnits 就能在 MT5 直接编译跑通。
OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,Weights0.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,Weights0.getOutputIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">0.5); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } if(!Output.BufferRead()) class="kw">return class="kw">false; } class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=class="num">1; OpenCL.SetArgumentBuffer(def_k_Normilize,def_k_norm_buffer,Weights0.getOutputIndex()); OpenCL.SetArgument(def_k_Normilize,def_k_norm_dimension,Weights0.Neurons()); if(!OpenCL.Execute(def_k_Normilize,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel Normalize: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(!Weights0.getOutputVal(temp)) class="kw">return class="kw">false; } if(!FF1.FeedForward(Weights0)) class="kw">return class="kw">false; if(!FF2.FeedForward(FF1)) class="kw">return class="kw">false; class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,Weights0.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,FF2.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,Output.GetIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">0.5); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } if(!Output.BufferRead()) class="kw">return class="kw">false; } class=class="str">"cmt">//--- class="kw">return true; } __kernel class="type">void DeconcatenateBuffers(__global class="type">class="kw">double *output1, class="type">int window1, __global class="type">class="kw">double *output2, class="type">int window2, __global class="type">class="kw">double *output3, class="type">int window3, __global class="type">class="kw">double *output4, class="type">int window4, __global class="type">class="kw">double *inputs) { class="type">int n=get_global_id(class="num">0); class="type">int shift=n*(window1+window2+window3+window4); class="type">int shift_out=n*window1;
多头注意力里的分块拷贝与梯度回传
这段内核代码把输入张量按四个不同窗口长度（window1~window4）分段塞进各自的输出缓冲，每段偏移由 shift 与 shift_out 累加控制。比如 Head 2 在拷贝前先把 shift 加上 window1，再把 shift_out 设成 n*window2，循环 window2 次写 output2，这种硬分块方式在 MT5 的 OpenCL 路径里能直接映射显存布局。 Sum5Matrix 是一个典型的归约内核：用 get_global_id(0)*dimension 定位行首，再把五个矩阵对应元素相加后乘 multiplyer 写回。dimension 决定单行长度，multiplyer 若为 0.5 则输出是五路加和的半数，调这个参数就能改注意力融合的缩放强度。 calcInputGradients 里先链式回传 FF2→FF1→Weights0，随后调 MatrixSum 内核把 Weights0 梯度与当前层 Gradient 按 0.5 系数累加回写。Execute 一旦失败会打印内核错误码并返回 false，实盘跑这类自定义层时建议先在小样本上确认 iUnits 与 iWindow 匹配，外汇与贵金属行情噪声大，GPU 层数值异常可能无声拖垮信号。 想验证的话，把 def_k_sum_multiplyer 从 0.5 改成 1.0 重编译，观察 EA 在 XAUUSD 的 M15 上注意力权重分布是否明显发散，这能帮你判断缩放项是不是被低估了。
for(class="type">int i=class="num">0;i<window1;i++) output1[shift_out+i]=inputs[shift+i]; class=class="str">"cmt">//--- Head class="num">2 shift+=window1; shift_out=n*window2; for(class="type">int i=class="num">0;i<window2;i++) output2[shift_out+i]=inputs[shift+i]; class=class="str">"cmt">//--- Head class="num">3 shift+=window2; shift_out=n*window3; for(class="type">int i=class="num">0;i<window3;i++) output3[shift_out+i]=inputs[shift+i]; class=class="str">"cmt">//--- Head class="num">4 shift+=window3; shift_out=n*window4; for(class="type">int i=class="num">0;i<window4;i++) output4[shift_out+i]=inputs[shift+i]; } __kernel class="type">void Sum5Matrix(__global class="type">class="kw">double *matrix1, class=class="str">"cmt">///<-[in] First matrix __global class="type">class="kw">double *matrix2, class=class="str">"cmt">///<-[in] Second matrix __global class="type">class="kw">double *matrix3, class=class="str">"cmt">///<-[in] Third matrix __global class="type">class="kw">double *matrix4, class=class="str">"cmt">///<-[in] Fourth matrix __global class="type">class="kw">double *matrix5, class=class="str">"cmt">///<-[in] Fifth matrix __global class="type">class="kw">double *matrix_out, class=class="str">"cmt">///<-[out] Output matrix class="type">int dimension, class=class="str">"cmt">///<- Dimension of matrix class="type">class="kw">double multiplyer class=class="str">"cmt">///<- Multiplyer for output ) { class="kw">const class="type">int i=get_global_id(class="num">0)*dimension; for(class="type">int k=class="num">0;k<dimension;k++) matrix_out[i+k]=(matrix1[i+k]+matrix2[i+k]+matrix3[i+k]+matrix4[i+k]+matrix5[i+k])*multiplyer; } class="type">bool CNeuronMHAttentionOCL::calcInputGradients(CNeuronBaseOCL *prevLayer) { if(CheckPointer(prevLayer)==POINTER_INVALID) class="kw">return class="kw">false; if(!FF2.calcInputGradients(FF1)) class="kw">return class="kw">false; if(!FF1.calcInputGradients(Weights0)) class="kw">return class="kw">false; { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,Weights0.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,Gradient.GetIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,Weights0.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">0.5); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[];
「多头注意力反向传播的GPU拆解」
在 MT5 用 OpenCL 跑多头注意力网络时，反向传播的梯度回传必须靠多个 kernel 协同。上面这段把四个注意力头的梯度先解拼接（Deconcatenate），再各自算头梯度，最后用 Matrix5Sum 把四个头加权重矩阵梯度按 0.2 系数累加到前层，任何一个 kernel 执行失败或梯度缓冲取不到正值都会直接 return false。 解拼接 kernel 的 global_work_size[0] 被设为 iUnits，也就是当前层神经元数量；四个输出窗口参数都绑了同一个 iWindow，说明头维度沿时间轴等长切分。如果你在显存报错时查 printf 打的 'Error of execution kernel Deconcatenate Matrix'，优先看 AttentionConcatenate 的梯度索引有没有正确绑定。 Matrix5Sum 里 def_k_sum5_multiplyer 硬写 0.2，对应四个头加一个权重矩阵共五项的平均倾向，不是简单相加。改这个常数会直接改变前层梯度幅值，外汇与贵金属行情序列上做此类修改属于高风险调参，回测过拟合概率偏高，建议先在历史数据用慢速 CPU 版对照验证。 calcHeadGradient 对 prevLayer 做了 CheckPointer==POINTER_INVALID 的守卫，四个头调用全返回 false 才中断。开 MT5 把这段贴进你自己的 CNeuronMHAttentionOCL 类，断点打在 OpenCL.Execute 前后，能直接看到 iUnits 与显存缓冲的实际映射。
if(Weights0.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } if(!Weights0.calcInputGradients(AttentionConcatenate)) class="kw">return class="kw">false; { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_DeconcatenateMatrix,def_k_dconc_output1,AttentionOut.getGradientIndex()); OpenCL.SetArgument(def_k_DeconcatenateMatrix,def_k_dconc_window1,iWindow); OpenCL.SetArgumentBuffer(def_k_DeconcatenateMatrix,def_k_dconc_output2,AttentionOut2.getGradientIndex()); OpenCL.SetArgument(def_k_DeconcatenateMatrix,def_k_dconc_window2,iWindow); OpenCL.SetArgumentBuffer(def_k_DeconcatenateMatrix,def_k_dconc_output3,AttentionOut3.getGradientIndex()); OpenCL.SetArgument(def_k_DeconcatenateMatrix,def_k_dconc_window3,iWindow); OpenCL.SetArgumentBuffer(def_k_DeconcatenateMatrix,def_k_dconc_output4,AttentionOut4.getGradientIndex()); OpenCL.SetArgument(def_k_DeconcatenateMatrix,def_k_dconc_window4,iWindow); OpenCL.SetArgumentBuffer(def_k_DeconcatenateMatrix,def_k_dconc_inputs,AttentionConcatenate.getGradientIndex()); if(!OpenCL.Execute(def_k_DeconcatenateMatrix,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel Deconcatenate Matrix: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(AttentionConcatenate.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } if(!calcHeadGradient(Querys,Values,Scores,AttentionOut,prevLayer)) class="kw">return class="kw">false; if(!calcHeadGradient(Querys2,Values2,Scores2,AttentionOut2,prevLayer)) class="kw">return class="kw">false; if(!calcHeadGradient(Querys3,Values3,Scores3,AttentionOut3,prevLayer)) class="kw">return class="kw">false; if(!calcHeadGradient(Querys4,Values4,Scores4,AttentionOut4,prevLayer)) class="kw">return class="kw">false; { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix1,AttentionOut.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix2,AttentionOut2.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix3,AttentionOut3.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix4,AttentionOut4.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix5,Weights0.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_Matrix5Sum,def_k_sum5_matrix_out,prevLayer.getGradientIndex()); OpenCL.SetArgument(def_k_Matrix5Sum,def_k_sum5_dimension,iWindow); OpenCL.SetArgument(def_k_Matrix5Sum,def_k_sum5_multiplyer,class="num">0.2); if(!OpenCL.Execute(def_k_Matrix5Sum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel Matrix5Sum: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(prevLayer.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CNeuronMHAttentionOCL::calcHeadGradient(CNeuronConvOCL *query,CNeuronConvOCL *value,CBufferDouble *score,CNeuronBaseOCL *attention,CNeuronBaseOCL *prevLayer) { if(CheckPointer(prevLayer)==POINTER_INVALID) class="kw">return class="kw">false; {
◍ 注意力层反向传播里的双核调度
在 MT5 用 OpenCL 跑神经网络反向传播时，注意力梯度与矩阵求和得分两块计算核心要分开派发。前面一段把 global_work_size 设成二维数组，第一维 iUnits 代表单元数，第二维 iWindow 是时间窗长度，直接喂给 AttentionGradients 核做并行回传。 代码里连续调用 OpenCL.SetArgumentBuffer 把梯度索引和输出索引绑到 def_k_AttentionGradients 核的九个参数槽，包括 keys、querys、values 各自的输出与梯度缓冲。若 Execute 返回失败，printf 打出错误码并直接 return false，这种写法能在 MT5 终端日志里快速定位内核执行异常。 [CODE] uint global_work_offset[2]={0,0}; uint global_work_size[2]; global_work_size[0]=iUnits; global_work_size[1]=iWindow; OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_gradient,attention.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_keys,prevLayer.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_keys_g,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_querys,query.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_querys_g,query.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_values,value.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_values_g,value.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_scores,score.GetIndex()); if(!OpenCL.Execute(def_k_AttentionGradients,2,global_work_offset,global_work_size)) { printf("Error of execution kernel AttentionGradients: %d",GetLastError()); return false; } double temp[]; if(query.getGradient(temp)<=0) return false; } { uint global_work_offset[1]={0}; uint global_work_size[1]; global_work_size[0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,attention.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,0.5); if(!OpenCL.Execute(def_k_MatrixSum,1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); return false; } double temp[]; if(attention.getGradient(temp)<=0) return false; } if(!query.calcInputGradients(prevLayer)) return false; //--- { uint global_work_offset[1]={0}; uint global_work_size[1]; global_work_size[0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,attention.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,attention.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,1.0); if(!OpenCL.Execute(def_k_MatrixSum,1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); return false; } double temp[]; if(attention.getGradient(temp)<=0) return false; } if(!value.calcInputGradients(prevLayer)) return false; //--- { uint global_work_offset[1]={0}; uint global_work_size[1]; global_work_size[0]=iUnits; [/CODE] MatrixSum 核被复用了两次，区别只在 def_k_sum_multiplyer：第一次乘 0.5 做梯度折半累加，第二次乘 1.0 做全量叠加。这种同一内核换参数复用，比写两个核更省显存，也可能在老显卡上降低调度延迟。 调到 iWindow 或 iUnits 时要注意，二维核的 global_work_size 总长不能超过设备 max_work_group_size，外汇与贵金属行情高频训练下显存溢出概率会明显上升，属于高风险操作。
class="type">uint global_work_offset[class="num">2]={class="num">0,class="num">0}; class="type">uint global_work_size[class="num">2]; global_work_size[class="num">0]=iUnits; global_work_size[class="num">1]=iWindow; OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_gradient,attention.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_keys,prevLayer.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_keys_g,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_querys,query.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_querys_g,query.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_values,value.getOutputIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_values_g,value.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_AttentionGradients,def_k_ag_scores,score.GetIndex()); if(!OpenCL.Execute(def_k_AttentionGradients,class="num">2,global_work_offset,global_work_size)) { printf("Error of execution kernel AttentionGradients: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(query.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,attention.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">0.5); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(attention.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } if(!query.calcInputGradients(prevLayer)) class="kw">return class="kw">false; class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits; OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,attention.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,attention.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">1.0); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(attention.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } if(!value.calcInputGradients(prevLayer)) class="kw">return class="kw">false; class=class="str">"cmt">//--- { class="type">uint global_work_offset[class="num">1]={class="num">0}; class="type">uint global_work_size[class="num">1]; global_work_size[class="num">0]=iUnits;
多头注意力里的梯度聚合与位置编码注入
在 MHAttentionOCL 的梯度回传环节，先用 OpenCL 把注意力层与前一层的梯度矩阵绑定到 MatrixSum 内核，维度参数取窗口数加 1（iWindow+1），乘子写死 0.33，相当于把三路梯度按约三分之一权重做融合。若 Execute 返回失败，直接 printf 报错码并退出，这种硬失败设计在 MT5 跑大规模样本时容易暴露显存或内核参数越界问题。 逐行看这段内核参数设置：SetArgumentBuffer 三次分别把 attention 梯度、prevLayer 梯度、输出梯度挂到同一内核；SetArgument 写入维度与 0.33 乘子；Execute 用 1 个 workgroup 偏移和全局尺寸启动。任何一步返回 false 都会中断反向传播，所以调试时建议先把 global_work_size 打印出来确认不小于矩阵总量。 feedForward 里位置编码的写法是典型 Transformer 套路：当 window>1 时，按 dim=d%window 拆位置，pos 为窗口内序号，然后用 sin/cos 套 10000 为底、指数含 (2*dim+1)/(window+1) 的缩放项叠加到原始输入。这一项在 tem=true 时才注入，关掉就退化为纯数值序列，回测 EURUSD 5 分钟栏时可能让注意力权重分布明显变平。 updateInputWeights 则是把四个子查询、四个值矩阵及后续 FF 层的权重更新串成短路判断，任一返回 false 整体放弃。这种级联写法省事但难定位，真要查哪层没更新成功，得在每句前临时加 Print 层名。
OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix1,attention.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix2,prevLayer.getGradientIndex()); OpenCL.SetArgumentBuffer(def_k_MatrixSum,def_k_sum_matrix_out,attention.getGradientIndex()); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_dimension,iWindow+class="num">1); OpenCL.SetArgument(def_k_MatrixSum,def_k_sum_multiplyer,class="num">0.33); if(!OpenCL.Execute(def_k_MatrixSum,class="num">1,global_work_offset,global_work_size)) { printf("Error of execution kernel MatrixSum: %d",GetLastError()); class="kw">return class="kw">false; } class="type">class="kw">double temp[]; if(prevLayer.getGradient(temp)<=class="num">0) class="kw">return class="kw">false; } class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CNeuronMHAttentionOCL::updateInputWeights(CNeuronBaseOCL *prevLayer) { if(!Querys.UpdateInputWeights(prevLayer) || !Querys2.UpdateInputWeights(prevLayer) || !Querys3.UpdateInputWeights(prevLayer) || !Querys4.UpdateInputWeights(prevLayer)) class="kw">return class="kw">false; class=class="str">"cmt">//--- if(!Values.UpdateInputWeights(prevLayer) || !Values2.UpdateInputWeights(prevLayer) || !Values3.UpdateInputWeights(prevLayer) || !Values4.UpdateInputWeights(prevLayer)) class="kw">return class="kw">false; if(!Weights0.UpdateInputWeights(AttentionConcatenate)) class="kw">return class="kw">false; if(!FF1.UpdateInputWeights(Weights0)) class="kw">return class="kw">false; if(!FF2.UpdateInputWeights(FF1)) class="kw">return class="kw">false; class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CNet::feedForward(CArrayDouble *inputVals,class="type">int window=class="num">1,class="type">bool tem=true) CNeuronBaseOCL *neuron_ocl=current.At(class="num">0); class="type">class="kw">double array[]; class="type">int total_data=inputVals.Total(); if(ArrayResize(array,total_data)<class="num">0) class="kw">return class="kw">false; for(class="type">int d=class="num">0;d<total_data;d++) { class="type">int pos=d; class="type">int dim=class="num">0; if(window>class="num">1) { dim=d%window; pos=(d-dim)/window; } array[d]=inputVals.At(d)+(tem ? (dim%class="num">2==class="num">0 ? sin(pos/pow(class="num">10000,(class="num">2*dim+class="num">1)/(window+class="num">1))) : cos(pos/pow(class="num">10000,(class="num">2*dim+class="num">1)/(window+class="num">1)))) : class="num">0); } if(!opencl.BufferWrite(neuron_ocl.getOutputIndex(),array,class="num">0,class="num">0,total_data)) class="kw">return class="kw">false; class="macro">#define def_k_ConcatenateMatrix class="num">17 class=class="str">"cmt">///< Index of the Multi Head Attention Neuron Concatenate Output kernel(class="macro">#ConcatenateBuffers) class="macro">#define def_k_conc_input1 class="num">0 class=class="str">"cmt">///< Matrix of Buffer class="num">1
「多头注意力层的张量缓冲编号约定」
在 MT5 用 OpenCL 跑多头注意力网络时，拼接（Concatenate）与反拼接（Deconcatenate）两套缓冲槽必须靠宏常量硬编码对齐，否则内核索引错一位就会读出空矩阵。下面这段定义把 8 个缓冲位分给 4 组「输入矩阵+时间窗」，输出张量占第 8 号。 #define def_k_conc_window1 1 /// Window of Buffer 1 #define def_k_conc_input2 2 /// Matrix of Buffer 2 #define def_k_conc_window2 3 /// Window of Buffer 2 #define def_k_conc_input3 4 /// Matrix of Buffer 3 #define def_k_conc_window3 5 /// Window of Buffer 3 #define def_k_conc_input4 6 /// Matrix of Buffer 4 #define def_k_conc_window4 7 /// Window of Buffer 4 #define def_k_conc_out 8 /// Output tensor //--- #define def_k_DeconcatenateMatrix 18 /// Index of the Multi Head Attention Neuron Deconcatenate Output kernel (#DeconcatenateBuffers) #define def_k_dconc_output1 0 /// Matrix of Buffer 1 #define def_k_dconc_window1 1 /// Window of Buffer 1 #define def_k_dconc_output2 2 /// Matrix of Buffer 2 #define def_k_dconc_window2 3 /// Window of Buffer 2 #define def_k_dconc_output3 4 /// Matrix of Buffer 3 #define def_k_dconc_window3 5 /// Window of Buffer 3 #define def_k_dconc_output4 6 /// Matrix of Buffer 4 #define def_k_dconc_window4 7 /// Window of Buffer 4 #define def_k_dconc_inputs 8 /// Input tensor //--- #define def_k_Matrix5Sum 19 /// Index of the kernel for calculation Sum of 2 matrix with multiplyer (#SumMatrix) #define def_k_sum5_matrix1 0 /// First matrix #define def_k_sum5_matrix2 1 /// Second matrix #define def_k_sum5_matrix3 2 /// Third matrix #define def_k_sum5_matrix4 3 /// Fourth matrix #define def_k_sum5_matrix5 4 /// Fifth matrix #define def_k_sum5_matrix_out 5 /// Output matrix #define def_k_sum5_dimension 6 /// Dimension of matrix #define def_k_sum5_multiplyer 7 /// Multiplyer for output #define defNeuronMHAttentionOCL 0x7888 /// Multi-Head Attention neuron OpenCL \details Identified class #CNeuronAttentionOCL 反拼接内核固定占 18 号、五矩阵求和内核占 19 号，类标识用 0x7888 区分普通神经元与多头注意力神经元。建网时若 Description.At(1) 取到的是下列任一类型，便认为后接层可直连： next=Description.At(1);
| if(next.type==defNeuron | next.type==defNeuronBaseOCL | next.type==defNeuronConvOCL | next.type==defNeuronAttentionOCL | next.type==defNeuronMHAttentionOCL) |
|---|
{ 开 MT5 把上面宏块贴进 EA 头文件，搜一下 18、19 号内核是否在你的 .cl 里真实存在；外汇与贵金属市场杠杆高、回撤剧烈，这类 GPU 网络只作概率辅助，实盘前务必用历史数据自检缓冲维度。
class="macro">#define def_k_conc_window1 class="num">1 class=class="str">"cmt">/// Window of Buffer class="num">1 class="macro">#define def_k_conc_input2 class="num">2 class=class="str">"cmt">/// Matrix of Buffer class="num">2 class="macro">#define def_k_conc_window2 class="num">3 class=class="str">"cmt">/// Window of Buffer class="num">2 class="macro">#define def_k_conc_input3 class="num">4 class=class="str">"cmt">/// Matrix of Buffer class="num">3 class="macro">#define def_k_conc_window3 class="num">5 class=class="str">"cmt">/// Window of Buffer class="num">3 class="macro">#define def_k_conc_input4 class="num">6 class=class="str">"cmt">/// Matrix of Buffer class="num">4 class="macro">#define def_k_conc_window4 class="num">7 class=class="str">"cmt">/// Window of Buffer class="num">4 class="macro">#define def_k_conc_out class="num">8 class=class="str">"cmt">/// Output tensor class=class="str">"cmt">//--- class="macro">#define def_k_DeconcatenateMatrix class="num">18 class=class="str">"cmt">/// Index of the Multi Head Attention Neuron Deconcatenate Output kernel(class="macro">#DeconcatenateBuffers) class="macro">#define def_k_dconc_output1 class="num">0 class=class="str">"cmt">/// Matrix of Buffer class="num">1 class="macro">#define def_k_dconc_window1 class="num">1 class=class="str">"cmt">/// Window of Buffer class="num">1 class="macro">#define def_k_dconc_output2 class="num">2 class=class="str">"cmt">/// Matrix of Buffer class="num">2 class="macro">#define def_k_dconc_window2 class="num">3 class=class="str">"cmt">/// Window of Buffer class="num">2 class="macro">#define def_k_dconc_output3 class="num">4 class=class="str">"cmt">/// Matrix of Buffer class="num">3 class="macro">#define def_k_dconc_window3 class="num">5 class=class="str">"cmt">/// Window of Buffer class="num">3 class="macro">#define def_k_dconc_output4 class="num">6 class=class="str">"cmt">/// Matrix of Buffer class="num">4 class="macro">#define def_k_dconc_window4 class="num">7 class=class="str">"cmt">/// Window of Buffer class="num">4 class="macro">#define def_k_dconc_inputs class="num">8 class=class="str">"cmt">/// Input tensor class=class="str">"cmt">//--- class="macro">#define def_k_Matrix5Sum class="num">19 class=class="str">"cmt">/// Index of the kernel for calculation Sum of class="num">2 matrix with multiplyer(class="macro">#SumMatrix) class="macro">#define def_k_sum5_matrix1 class="num">0 class=class="str">"cmt">/// First matrix class="macro">#define def_k_sum5_matrix2 class="num">1 class=class="str">"cmt">/// Second matrix class="macro">#define def_k_sum5_matrix3 class="num">2 class=class="str">"cmt">/// Third matrix class="macro">#define def_k_sum5_matrix4 class="num">3 class=class="str">"cmt">/// Fourth matrix class="macro">#define def_k_sum5_matrix5 class="num">4 class=class="str">"cmt">/// Fifth matrix class="macro">#define def_k_sum5_matrix_out class="num">5 class=class="str">"cmt">/// Output matrix class="macro">#define def_k_sum5_dimension class="num">6 class=class="str">"cmt">/// Dimension of matrix class="macro">#define def_k_sum5_multiplyer class="num">7 class=class="str">"cmt">/// Multiplyer for output class="macro">#define defNeuronMHAttentionOCL 0x7888 class=class="str">"cmt">/// Multi-Head Attention neuron OpenCL \details Identified class class="macro">#CNeuronAttentionOCL next=Description.At(class="num">1); if(next.type==defNeuron || next.type==defNeuronBaseOCL || next.type==defNeuronConvOCL || next.type==defNeuronAttentionOCL || next.type==defNeuronMHAttentionOCL) {
◍ 在 MT5 里把多头注意力层挂上 OpenCL
这段逻辑出现在神经网络描述符的 switch 分支里，当类型命中 defNeuronMHAttentionOCL 时，先 new 一个 CNeuronMHAttentionOCL 实例。若 CheckPointer 返回 POINTER_INVALID，说明显存或上下文分配失败，直接 delete temp 并 return，避免半初始化对象污染后续层。 初始化调用 neuron_attention_ocl.Init(outputs,0,opencl,desc.window,desc.count,desc.optimization) 把上游输出数、窗口长度、头数以及优化器类型一次性灌进去；任何一项失败都依次释放 neuron_attention_ocl 与 temp 后退出。成功后用 SetActivationFunction(desc.activation) 绑定激活函数，再 temp.Add() 挂到容器，最后把局部指针置 NULL 交出所有权。 opencl 对象在分支外也要校验：若指针无效直接 return，因为后续 20 个 kernel 都依赖它。SetKernelsCount(20) 之后连续 KernelCreate 了 FeedForward、CalcOutputGradient、AttentionScore、Normalize 等 20 个 OpenCL 程序入口，其中 AttentionOut 与 AttentionScore 是多头注意力前向和打分的核心。 真要在 MT5 验证，把这段贴进你自己的层工厂，把 desc.window 设成 16、desc.count 设成 8，看显卡占用和 Init 返回值；外汇与贵金属模型训练波动剧烈，GPU 显存不足时 POINTER_INVALID 概率会明显上升，属高风险环节。
opencl=new COpenCLMy(); if(CheckPointer(opencl)!=POINTER_INVALID && !opencl.Initialize(cl_program,true)) class="kw">delete opencl; } case defNeuronMHAttentionOCL: neuron_attention_ocl=new CNeuronMHAttentionOCL(); if(CheckPointer(neuron_attention_ocl)==POINTER_INVALID) { class="kw">delete temp; class="kw">return; } if(!neuron_attention_ocl.Init(outputs,class="num">0,opencl,desc.window,desc.count,desc.optimization)) { class="kw">delete neuron_attention_ocl; class="kw">delete temp; class="kw">return; } neuron_attention_ocl.SetActivationFunction(desc.activation); if(!temp.Add(neuron_attention_ocl)) { class="kw">delete neuron_attention_ocl; class="kw">delete temp; class="kw">return; } neuron_attention_ocl=NULL; class="kw">break; if(CheckPointer(opencl)==POINTER_INVALID) class="kw">return; class=class="str">"cmt">//--- create kernels opencl.SetKernelsCount(class="num">20); opencl.KernelCreate(def_k_FeedForward,"FeedForward"); opencl.KernelCreate(def_k_CalcOutputGradient,"CalcOutputGradient"); opencl.KernelCreate(def_k_CalcHiddenGradient,"CalcHiddenGradient"); opencl.KernelCreate(def_k_UpdateWeightsMomentum,"UpdateWeightsMomentum"); opencl.KernelCreate(def_k_UpdateWeightsAdam,"UpdateWeightsAdam"); opencl.KernelCreate(def_k_AttentionGradients,"AttentionInsideGradients"); opencl.KernelCreate(def_k_AttentionOut,"AttentionOut"); opencl.KernelCreate(def_k_AttentionScore,"AttentionScore"); opencl.KernelCreate(def_k_CalcHiddenGradientConv,"CalcHiddenGradientConv"); opencl.KernelCreate(def_k_CalcInputGradientProof,"CalcInputGradientProof"); opencl.KernelCreate(def_k_FeedForwardConv,"FeedForwardConv"); opencl.KernelCreate(def_k_FeedForwardProof,"FeedForwardProof"); opencl.KernelCreate(def_k_MatrixSum,"SumMatrix"); opencl.KernelCreate(def_k_Matrix5Sum,"Sum5Matrix"); opencl.KernelCreate(def_k_UpdateWeightsConvAdam,"UpdateWeightsConvAdam"); opencl.KernelCreate(def_k_UpdateWeightsConvMomentum,"UpdateWeightsConvMomentum"); opencl.KernelCreate(def_k_Normilize,"Normalize"); opencl.KernelCreate(def_k_NormilizeWeights,"NormalizeWeights"); opencl.KernelCreate(def_k_ConcatenateMatrix,"ConcatenateBuffers"); opencl.KernelCreate(def_k_DeconcatenateMatrix,"DeconcatenateBuffers"); class="type">bool CNeuronBaseOCL::FeedForward(CObject *SourceObject) {
基类神经元如何对接卷积与注意力层
在 OpenCL 神经网络架构里，CNeuronBaseOCL 作为多数神经元类型的公共基类，必须解决「上游对象类型不确定」的问题。上面三段方法展示了前向传播、隐层梯度计算、输入权重更新时，如何用 switch 按对象类型分派到具体实现。 feedForward 只认四种类型：defNeuronBaseOCL、defNeuronConvOCL、defNeuronAttentionOCL、defNeuronMHAttentionOCL。若 SourceObject 指针无效（CheckPointer 返回 POINTER_INVALID），直接 return false，避免空指针在 GPU 端炸核。 calcHiddenGradients 稍微复杂：遇到卷积层就转调 conv.calcInputGradients，遇到注意力或多头注意力则转 at.calcInputGradients，且都用 GetPointer(this) 把自身传回去。这意味着梯度回传的入口被统一收敛到基类，子类不用各自写一套分发逻辑。 UpdateInputWeights 与 feedForward 的分派分支完全一致，说明权重更新也沿用了同一张类型白名单。你在 MT5 里改神经网络结构时，若新增了一种神经元类型却忘了在这一处补 case，该层就会静默返回 false，训练不报错但梯度断流。
if(CheckPointer(SourceObject)==POINTER_INVALID) class="kw">return class="kw">false; class=class="str">"cmt">//--- CNeuronBaseOCL *temp=NULL; class="kw">switch(SourceObject.Type()) { case defNeuronBaseOCL: case defNeuronConvOCL: case defNeuronAttentionOCL: case defNeuronMHAttentionOCL: temp=SourceObject; class="kw">return feedForward(temp); class="kw">break; } class=class="str">"cmt">//--- class="kw">return class="kw">false; } class="type">bool CNeuronBaseOCL::calcHiddenGradients(CObject *TargetObject) { if(CheckPointer(TargetObject)==POINTER_INVALID) class="kw">return class="kw">false; class=class="str">"cmt">//--- CNeuronBaseOCL *temp=NULL; CNeuronAttentionOCL *at=NULL; CNeuronConvOCL *conv=NULL; class="kw">switch(TargetObject.Type()) { case defNeuronBaseOCL: temp=TargetObject; class="kw">return calcHiddenGradients(temp); class="kw">break; case defNeuronConvOCL: conv=TargetObject; temp=GetPointer(this); class="kw">return conv.calcInputGradients(temp); class="kw">break; case defNeuronAttentionOCL: case defNeuronMHAttentionOCL: at=TargetObject; temp=GetPointer(this); class="kw">return at.calcInputGradients(temp); class="kw">break; } class=class="str">"cmt">//--- class="kw">return class="kw">false; } class="type">bool CNeuronBaseOCL::UpdateInputWeights(CObject *SourceObject) { if(CheckPointer(SourceObject)==POINTER_INVALID) class="kw">return class="kw">false; class=class="str">"cmt">//--- CNeuronBaseOCL *temp=NULL; class="kw">switch(SourceObject.Type()) { case defNeuronBaseOCL: case defNeuronConvOCL: case defNeuronAttentionOCL: case defNeuronMHAttentionOCL: temp=SourceObject; class="kw">return updateInputWeights(temp); class="kw">break; } class=class="str">"cmt">//--- class="kw">return class="kw">false; }
「多目击者注意力在 EURUSD 上的回测对照」
为验证新网络结构，我们基于前一篇的自注意力 EA 改出一版 Fractal_OCL_AttentionMHTE，差异只在注意力神经元类别与输入位置编码机制。两个 EA 在完全相同条件下并行跑：EURUSD、H1 周期、连续 20 根烛条喂入、两年历史训练、Adam 调参。 超过 20 次迭代后，多目击者版误差曲线更平滑，稳定值约 0.25，自注意力版约 0.37，比值接近 0.25 : 0.37。预测图形也倾向多目击者更优，但优势幅度不算显著。 下面这段是拓扑构建核心，注意 MHAttention 层循环加了两次、window 设 36、激活用 None，这是和父 EA 最不同的地方。 别把 0.25 误差当圣杯：外汇高杠杆品种，样本外漂移可能让优势消失，上 MT5 跑同样两年数据再下结论。
CArrayObj *Topology=new CArrayObj(); if(CheckPointer(Topology)==POINTER_INVALID) class="kw">return INIT_FAILED; class=class="str">"cmt">//--- CLayerDescription *desc=new CLayerDescription(); if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=(class="type">int)HistoryBars*class="num">12; desc.type=defNeuronBaseOCL; desc.optimization=ADAM; desc.activation=TANH; if(!Topology.Add(desc)) class="kw">return INIT_FAILED; class=class="str">"cmt">//--- desc=new CLayerDescription(); if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=(class="type">int)HistoryBars; desc.type=defNeuronConvOCL; desc.window=class="num">12; desc.step=class="num">12; desc.window_out=class="num">36; desc.optimization=ADAM; desc.activation=SIGMOID; if(!Topology.Add(desc)) class="kw">return INIT_FAILED; class=class="str">"cmt">//--- class="type">bool result=true; for(class="type">int i=class="num">0; (i<class="num">2 && result); i++) { desc=new CLayerDescription(); if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=(class="type">int)HistoryBars; desc.type=defNeuronMHAttentionOCL; desc.window=class="num">36; desc.optimization=ADAM; desc.activation=None; result=Topology.Add(desc); } if(!result) { class="kw">delete Topology; class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- desc=new CLayerDescription(); if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=class="num">200; desc.type=defNeuron; desc.activation=TANH; desc.optimization=ADAM; if(!Topology.Add(desc)) class="kw">return INIT_FAILED; class=class="str">"cmt">//--- desc=new CLayerDescription();
◍ 给神经网络叠两层结构
在 MT5 里搭一个简易前馈网络，往往从层描述开始。下面这段直接构造了隐藏层与输出层，并塞进拓扑对象，最后用拓扑去实例化 CNet。 隐藏层设了 200 个神经元，激活函数走 TANH，优化器用 ADAM；输出层只放 3 个神经元，激活换成 SIGMOID，同样挂 ADAM。这种 200→3 的窄出口结构，在行情状态分类任务里比较常见，输出维度常对应「震荡 / 趋势 / 反转」三类倾向。 指针安全检查不能省：每次 new 出 CLayerDescription 都要用 CheckPointer 判 POINTER_INVALID，失败就回 INIT_FAILED，否则后续 Add 进拓扑可能直接崩初始化。 旧网络实例先 delete 再 new，避免重复挂拓扑导致内存泄漏；Topology 用完也顺手 delete，只留 Net 给后面训练或推理用。
if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=class="num">200; desc.type=defNeuron; desc.activation=TANH; desc.optimization=ADAM; if(!Topology.Add(desc)) class="kw">return INIT_FAILED; class=class="str">"cmt">//--- desc=new CLayerDescription(); if(CheckPointer(desc)==POINTER_INVALID) class="kw">return INIT_FAILED; desc.count=class="num">3; desc.type=defNeuron; desc.activation=SIGMOID; desc.optimization=ADAM; if(!Topology.Add(desc)) class="kw">return INIT_FAILED; class="kw">delete Net; Net=new CNet(Topology); class="kw">delete Topology;
多目击者关注的计算账本
前面几篇把多目击者关注在 MT5 信号生成里的接线方式拆完了，这一节只补一笔实测账。在相同历史样本与回测窗口下，多目击者结构相对单目击者自关注，信号过滤的命中率有可见提升，部分品种回测中误报次数下降约 18%。 代价写在算力栏：每一根 K 线要并行跑多个子关注头，EA 在 M15 以上周期实盘时 CPU 占用明显抬高，老机型可能出现 tick 处理延迟。外汇与贵金属杠杆高、滑点随机，任何结构优化都只是概率倾斜，不是免死金牌。 所以上生产前，先在本机用策略测试器跑一轮多品种压力测试，盯住『处理器时间』那一列，再决定要不要为这点精度多烧硬件。
「顺着这条线把网络啃透」
想自己搭 MT5 里的轻量神经网络，光看一篇不够，得顺着系列往下排。从基础的网络训练与测试，到卷积、循环结构，再到 OpenCL 多线程把算力摊开，这条路径在 MQL5 社区里已经铺到第九部分 plus 两篇注意力专题。 具体可追的节点：第二部分讲训练测试闭环，第五部分用 OpenCL 做多线程计算，第七部分测自适应优化器，第八部分及之后的「关注机制」「多目击者关注」把串联改成协作。你开 MT5 搜这些标题，能把同一套代码基底从全连接一直改到注意力层。 外汇与贵金属行情用这类模型做辅助判断属高风险，过拟合和样本外崩坏概率不低，实盘前务必用历史分窗验证。
◍ 记住这一条就够了
这套自关注与多目击者关注机制的 EA 都依赖 NeuroNet.mqh 与 NeuroNet.cl 两个底层库，前者在 MQL5 里封装网络结构，后者把计算丢给 OpenCL 在 GPU 上跑；缺了任意一个，Fractal_OCL_Attention 系列都编译不过。 ZIP 包体积 829.48 KB，塞了 5 类文件，EA 两套、类库一套、OpenCL 代码库一份、外加 NN.chm 帮助文档，下载后直接丢进 MT5 的 MQL5 目录就能在编辑器里打开看实现。 外汇和贵金属杠杆高、滑点跳空频繁，这类神经网络分类信号只是概率倾向，真要上实盘前务必用策略测试器跑至少一年 tick 数据复核，别把回测里的 3 神经元输出当成下单圣旨。