📘

交易中的神经网络：点云变换器（Pointformer）·进阶篇

第 2/3 篇

本章目录

PointFormer 神经元的接口与初始化骨架
PointFormer 前向传播里的局部到全局拼接
PointFormer 反向传播的梯度回流路径
反向传播里的梯度交换与正交约束

「PointFormer 神经元的接口与初始化骨架」

在 MT5 的 OpenCL 神经网络扩展里，CNeuronPointFormer 作为 PointNet 变体，先把反向传播与权重更新接口全部 override，这意味着它不走基类的默认梯度逻辑。构造函数仅置空，析构里 delete cbTemp 释放临时卷积缓冲；Type() 直接返回 defNeuronPointFormer 常量，用于序列化时区分神经元类型。 Init() 先调用父类 CNeuronPointNet2OCL::Init 做基础装配，失败即返回 false。随后循环 i=0、1 两次，分别挂两组局部注意力、局部-全局注意力、全局注意力与位置编码层。注意循环里 caLocalAttention[i].Init 的窗口参数写死为 64、头数 16、层数 4、units_count 与批内索引 2；caLocalGlobalAttention 则额外传入全局维度 64。这些硬编码维度决定了显存占用，调参时若改 units_count 必须同步核对 OpenCL 内核的 buffer 大小。外汇与贵金属行情用这类结构做序列建模时，过拟合概率偏高，建议先用小 batch 在策略测试器里跑通前向再扩规模。

MQL5 / C++

class="kw">virtual class="type">bool      calcInputGradients(CNeuronBaseOCL *NeuronOCL) class="kw">override;
 class="kw">virtual class="type">bool      updateInputWeights(CNeuronBaseOCL *NeuronOCL) class="kw">override;
class="kw">public:
                      CNeuronPointFormer(class="type">void) {};
                     ~CNeuronPointFormer(class="type">void) { class="kw">delete cbTemp; }
 class=class="str">"cmt">//---
 class="kw">virtual class="type">bool      Init(class="type">uint numOutputs, class="type">uint myIndex, COpenCLMy *open_cl,
                         class="type">uint window, class="type">uint units_count, class="type">uint output, class="type">bool use_tnets,
                         ENUM_OPTIMIZATION optimization_type, class="type">uint batch) class="kw">override;
 class=class="str">"cmt">//---
 class="kw">virtual class="type">int       Type(class="type">void) class="kw">override  class="kw">const   {  class="kw">return defNeuronPointFormer; }
 class=class="str">"cmt">//---
 class="kw">virtual class="type">bool      Save(class="type">int class="kw">const file_handle) class="kw">override;
 class="kw">virtual class="type">bool      Load(class="type">int class="kw">const file_handle) class="kw">override;
 class=class="str">"cmt">//--- 
 class="kw">virtual class="type">bool      WeightsUpdate(CNeuronBaseOCL *source, class="type">class="kw">float tau) class="kw">override;
 class="kw">virtual class="type">void      SetOpenCL(COpenCLMy *obj) class="kw">override;
 };
class="type">bool CNeuronPointFormer::Init(class="type">uint numOutputs, class="type">uint myIndex, COpenCLMy *open_cl,
class="type">uint window, class="type">uint units_count, class="type">uint output, class="type">bool use_tnets,
ENUM_OPTIMIZATION optimization_type, class="type">uint batch)
  {
  if(!CNeuronPointNet2OCL::Init(numOutputs, myIndex, open_cl, window, units_count, output,
use_tnets, optimization_type, batch))
      class="kw">return class="kw">false;
  for(class="type">int i = class="num">0; i < class="num">2; i++)
   {
     if(!caLocalAttention[i].Init(class="num">0, i*class="num">5, OpenCL, class="num">64, class="num">16, class="num">4, units_count, class="num">2, optimization, iBatch))
        class="kw">return class="kw">false;
     if(!caLocalGlobalAttention[i].Init(class="num">0, i*class="num">5+class="num">1, OpenCL, class="num">64, class="num">16, class="num">4, class="num">64, class="num">2, units_count,
units_count, class="num">2, class="num">2, optimization, iBatch))
        class="kw">return class="kw">false;
     if(!caGlobalAttention[i].Init(class="num">0, i*class="num">5+class="num">2, OpenCL, class="num">64, class="num">16, class="num">4, class="num">2, units_count, class="num">2, class="num">2, optimization, iBatch))
        class="kw">return class="kw">false;
     if(!caLocalPE[i].Init(class="num">0, i*class="num">5+class="num">3, OpenCL, class="num">64*units_count, optimization, iBatch))
        class="kw">return class="kw">false;

PointFormer 前向传播里的局部到全局拼接

这段实现把局部点网、局部注意力、位置编码再喂入全局注意力，构成 PointFormer 的核心推理链。初始化阶段对两层局部注意力设了不同稀疏度：caLocalAttention[0].Sparse(0.1f) 保留 10% 连接，caLocalAttention[1].Sparse(0.3f) 保留 30%，越往后局部感受野越密。 feedForward 里先走 caLocalPointNet[i]，再依次过 caLocalAttention、caLocalPE、caGlobalPE，最后用 caLocalGlobalAttention 把局部位置编码与全局位置编码做交叉。注意 global PE 的 Init 参数是 i*5+4，意味着第 0 层偏移 4、第 1 层偏移 9，两层全局位置编码的 OpenCL 内核索引不重叠。若走 cTNetG 分支，window 由 MathSqrt(cTNetG.Neurons()) 取整得到，再调 MatMul 做转置乘。以神经元数 64 为例，window=8，MatMul 维度即 8×8×8；这种开方分块在 MT5 算子上容易因 Neurons() 非完全平方数而出边界，开 MT5 跑前建议先打印 Neurons() 确认。外汇与贵金属行情下用此类模型信号属高风险，任何推断都只是概率倾向。

MQL5 / C++

  if(!caGlobalPE[i].Init(class="num">0, i*class="num">5+class="num">4, OpenCL, class="num">64*units_count, optimization, iBatch))
       class="kw">return class="kw">false;
   }
   caLocalAttention[class="num">0].Sparse(class="num">0.1f);
   caLocalAttention[class="num">1].Sparse(class="num">0.3f);
   if(!cConcatenate.Init(class="num">0, class="num">10, OpenCL, class="num">128 * units_count, optimization, iBatch))
      class="kw">return class="kw">false;
   if(!cScale.Init(class="num">0, class="num">11, OpenCL, class="num">128, class="num">128, class="num">64, units_count, class="num">1, optimization, iBatch))
      class="kw">return class="kw">false;
   if(!!cbTemp)
      class="kw">delete cbTemp;
   cbTemp = new CBufferFloat();
   if(!cbTemp ||
      !cbTemp.BufferInit(caGlobalAttention[class="num">0].Neurons(), class="num">0) ||
      !cbTemp.BufferCreate(OpenCL))
      class="kw">return class="kw">false;
class=class="str">"cmt">//---
   class="kw">return true;
   }
class="type">bool CNeuronPointFormer::feedForward(CNeuronBaseOCL *NeuronOCL)
   {
class=class="str">"cmt">//--- LocalNet
   CNeuronBaseOCL *inputs = NeuronOCL;
   for(class="type">int i = class="num">0; i < class="num">2; i++)
     {
       if(!cTNetG || i > class="num">0)
         {
          if(!caLocalPointNet[i].FeedForward(inputs))
            class="kw">return class="kw">false;
         }
       else
         {
          if(!cTurnedG)
            class="kw">return class="kw">false;
          if(!cTNetG.FeedForward(inputs))
            class="kw">return class="kw">false;
          class="type">int window = (class="type">int)MathSqrt(cTNetG.Neurons());
          if(IsStopped() ||
             !MatMul(NeuronOCL.getOutput(), cTNetG.getOutput(), cTurnedG.getOutput(),
NeuronOCL.Neurons() / window, window, window))
            class="kw">return class="kw">false;
          if(!caLocalPointNet[i].FeedForward(cTurnedG.AsObject()))
            class="kw">return class="kw">false;
         }
       class=class="str">"cmt">//--- Local Attention
       if(!caLocalAttention[i].FeedForward(caLocalPointNet[i].AsObject()))
         class="kw">return class="kw">false;
       class=class="str">"cmt">//--- Position Encoder
       if(!caLocalPE[i].FeedForward(caLocalAttention[i].AsObject()))
         class="kw">return class="kw">false;
       if(!caGlobalPE[i].FeedForward(caLocalPointNet[i].AsObject()))
         class="kw">return class="kw">false;
       class=class="str">"cmt">//--- Local to Global Attention
       if(!caLocalGlobalAttention[i].FeedForward(caLocalPE[i].AsObject(), caGlobalPE[i].getOutput()))
         class="kw">return class="kw">false;
       class=class="str">"cmt">//--- Global Attention
       if(!caGlobalAttention[i].FeedForward(caLocalGlobalAttention[i].AsObject()))

◍ PointFormer 反向传播的梯度回流路径

这段 CNeuronPointFormer::calcInputGradients 展示了 PointFormer 结构在训练时梯度如何逐层回传。它先对最终的 Scale 层和 Concat 层求梯度，再通过 DeConcat 把 64×64 维度的拼接梯度拆回两个 Global Attention 分支，拆分粒度由 cConcatenate.Neurons()/128 决定。循环从 i=1 到 i=0 逆序处理两个分支：先算 LocalGlobalAttention 的隐藏梯度，再经位置编码层 caLocalPE 回传，并带上 caGlobalPE 的输出与梯度作为旁路。LocalAttention 与 LocalPointNet 的梯度链式调用依次执行，中间用 cbTemp 缓冲区做梯度暂存与交换。注意 SumAndNormilize 调用里参数依次为 64、false、0、0、0、1，最后一位 1 代表在特征轴做归一化累加。外汇与贵金属行情高波动，这类自定义神经网络层若梯度维度配错，MT5 策略回测可能直接返回 false 中断，建议开 MT5 用注释逐行打点验证维度匹配。

MQL5 / C++

class="type">bool CNeuronPointFormer::calcInputGradients(CNeuronBaseOCL *NeuronOCL)
  {
   if(!NeuronOCL)
      class="kw">return class="kw">false;
   if(!CNeuronPointNetOCL::calcInputGradients(cScale.AsObject()))
      class="kw">return class="kw">false;
   if(!cConcatenate.calcHiddenGradients(cScale.AsObject()))
      class="kw">return class="kw">false;
   if(!DeConcat(caGlobalAttention[class="num">0].getGradient(), caGlobalAttention[class="num">1].getGradient(),
                cConcatenate.getGradient(), class="num">64, class="num">64, cConcatenate.Neurons() / class="num">128))
      class="kw">return class="kw">false;
   CNeuronBaseOCL *inputs = caGlobalAttention[class="num">0].AsObject();
   for(class="type">int i = class="num">1; i >= class="num">0; i--)
     {
      class=class="str">"cmt">//--- Global Attention
      if(!caLocalGlobalAttention[i].calcHiddenGradients(caGlobalAttention[i].AsObject()))
         class="kw">return class="kw">false;
      if(!caLocalPE[i].calcHiddenGradients(caLocalGlobalAttention[i].AsObject(), caGlobalPE[i].getOutput(),
                                           caGlobalPE[i].getGradient(),
                                           (ENUM_ACTIVATION)caGlobalPE[i].Activation()))
         class="kw">return class="kw">false;
      if(!caLocalAttention[i].calcHiddenGradients(caLocalPE[i].AsObject()))
         class="kw">return class="kw">false;
      if(!caLocalPointNet[i].calcHiddenGradients(caGlobalPE[i].AsObject()))
         class="kw">return class="kw">false;
      CBufferFloat *temp = caLocalPointNet[i].getGradient();
      caLocalPointNet[i].SetGradient(cbTemp, class="kw">false);
      cbTemp = temp;
      if(!caLocalPointNet[i].calcHiddenGradients(caLocalAttention[i].AsObject()))
         class="kw">return class="kw">false;
      if(!SumAndNormilize(caLocalPointNet[i].getGradient(), cbTemp, caLocalPointNet[i].getGradient(),
                          class="num">64, class="kw">false, class="num">0, class="num">0, class="num">0, class="num">1))
         class="kw">return class="kw">false;
      if(i > class="num">0)
        {
         temp = inputs.getGradient();
         inputs.SetGradient(cbTemp, class="kw">false);
         cbTemp = temp;
        }
      if(!cTNetG || i > class="num">0)
        {

「反向传播里的梯度交换与正交约束」

这段逻辑处在网络反向迭代的尾部，核心是把局部点网算出的隐藏梯度往上一层传，同时处理转置网络 cTurnedG 的梯度回流。若未启用转置分支，直接对 caLocalPointNet[i] 调 calcHiddenGradients，失败即返回 false，单层回传就此中断。启用 cTurnedG 时，先用 MathSqrt(cTNetG.Neurons()) 开根得到 window，把神经元数映射成方阵边长，再跑 MatMulGrad 做矩阵乘梯度，输入分块大小为 inputs.Neurons()/window。中途任一步 IsStopped 或乘积失败都直接退出，避免 EA 在终端关闭时还空转占资源。梯度指针在这里被故意互换：inputs 的梯度缓冲交给 cTurnedG，cTurnedG 的原有梯度回塞给 inputs，随后各自再算一遍隐藏梯度并做 SumAndNormilize（系数 1，不归一化到 0 维）。最后当 i>0 时以批大小 64 做累加归一，并把 inputs 指回上一层全局注意力输出，构成完整的逐层回传链。开 MT5 把这段塞进你自己的 Owl 类反向函数里，改 64 这个批参数看显存占用曲线，外汇与贵金属模型训练波动大、过拟合风险高，调参前先跑小样本验证。

MQL5 / C++

        if(!inputs.calcHiddenGradients(caLocalPointNet[i].AsObject()))
            class="kw">return class="kw">false;
      }
      else
      {
       if(!cTurnedG)
         class="kw">return class="kw">false;
       if(!cTurnedG.calcHiddenGradients(caLocalPointNet[i].AsObject()))
         class="kw">return class="kw">false;
       class="type">int window = (class="type">int)MathSqrt(cTNetG.Neurons());
       if(IsStopped() ||
          !MatMulGrad(inputs.getOutput(), inputs.getGradient(), cTNetG.getOutput(), cTNetG.getGradient(),
                     cTurnedG.getGradient(), inputs.Neurons() / window, window, window))
         class="kw">return class="kw">false;
       if(!OrthoganalLoss(cTNetG, true))
         class="kw">return class="kw">false;
       CBufferFloat *temp = inputs.getGradient();
       inputs.SetGradient(cTurnedG.getGradient(), class="kw">false);
       cTurnedG.SetGradient(temp, class="kw">false);
       if(!inputs.calcHiddenGradients(cTNetG.AsObject()))
         class="kw">return class="kw">false;
       if(!SumAndNormilize(inputs.getGradient(), cTurnedG.getGradient(), inputs.getGradient(),
                          class="num">1, class="kw">false, class="num">0, class="num">0, class="num">0, class="num">1))
         class="kw">return class="kw">false;
      }
      if(i > class="num">0)
      {
       if(!SumAndNormilize(inputs.getGradient(), cbTemp, inputs.getGradient(), class="num">64, class="kw">false, class="num">0, class="num">0, class="num">0, class="num">1))
         class="kw">return class="kw">false;
       inputs = caGlobalAttention[i - class="num">1].AsObject();
      }
      else
       inputs = NeuronOCL;
   }
class=class="str">"cmt">//---
   class="kw">return true;
   }