交易中的神经网络：使用语言模型进行时间序列预测·进阶篇
（2/3）· 当预训练语言模型碰上时间序列，分解趋势、季节性、残差才是突破泛化瓶颈的关键
很多交易者直接把大语言模型套到K线序列上，指望它自动悟出涨跌节奏，结果预测偏差比线性模型还大。理论分析表明，标准注意力机制并不具备自动分离趋势与季节性的能力，硬喂原始价格只会让模型学到噪声。本篇承接基础篇，继续深挖 TEMPO 如何用统计分解补齐这道短板。
TEMPOOCL 网络里的 PLR 梯度回传逻辑
这段代码片段来自一个把价格拆成趋势、季节、噪声三部分的神经网络类 CNeuronTEMPOOCL，其中趋势分支用到了 PLR（分段线性回归）单元。先看 do-while 循环：它从当前位置 i 向前回溯，按 step_plr 和 step_in 的步长在 plr 数组里跳，dist 取 fmax(plr[...]*lenth, 1) 并强转 int，保证最小跨度为 1 根 K 线，直到 prev_in 覆盖到目标索引 i 才停。 循环结束后做梯度拆解：other_i_gr 是季节+噪声分支的梯度，trend_i_gr 等于总趋势梯度减去 other_i_gr，也就是纯 PLR 趋势梯度。接着用 pos = i - prev_in 算相对偏移，sloat_gr = trend_i_gr * pos 是斜率累计梯度，intercept_gr = trend_i_gr 是截距梯度，分别写回 plr_gr 的两个偏移位，inputs_gr 则回填 other_i_gr。 类声明里能看到完整组件：iVariables/iSequence/iForecast/iFFT 四个维度常量，趋势侧 cPLR 与 cTrend，季节侧走 FFT（cInputFreqRe/Im、cFreqAtteention 等复数注意力），噪声侧 cResidual，预测侧用 BatchNorm+Patching+CrossAttention 拼 cSum 输出。想验证的话，在 MT5 的 MetaEditor 里搜 CNeuronTEMPOOCL，把 step_plr 从默认 4 改成 2，观察回测中趋势对短时波动的捕捉是否更密但过拟合概率上升——外汇与贵金属杠杆高，这类改参仅作方法验证，实盘须自担风险。
class="type">int dist = class="num">0; do { pos++; prev_in += dist; dist = (class="type">int)fmax(plr[shift_plr + pos * step_plr + class="num">2 * step_in] * lenth, class="num">1); } while(!(prev_in <= i && (prev_in + dist) > i)); class=class="str">"cmt">//--- get gradient class="type">float other_i_gr = other_gr[shift_in]; class="type">float trend_i_gr = trend_gr[shift_in] - other_i_gr; class=class="str">"cmt">//--- calc plr gradient pos = i - prev_in; class="type">float sloat_gr = trend_i_gr * pos; class="type">float intercept_gr = trend_i_gr; class=class="str">"cmt">//--- save result plr_gr[shift_plr + pos * step_plr] += sloat_gr; plr_gr[shift_plr + pos * step_plr + step_in] += intercept_gr; inputs_gr[shift_in] = other_i_gr; } class CNeuronTEMPOOCL : class="kw">public CNeuronBaseOCL { class="kw">protected: class=class="str">"cmt">//--- constants class="type">uint iVariables; class="type">uint iSequence; class="type">uint iForecast; class="type">uint iFFT; class=class="str">"cmt">//--- Trend CNeuronPLROCL cPLR; CNeuronBaseOCL cTrend; class=class="str">"cmt">//--- Seasons CNeuronBaseOCL cInputSeasons; CNeuronTransposeOCL cTranspose[class="num">2]; CBufferFloat cInputFreqRe; CBufferFloat cInputFreqIm; CNeuronBaseOCL cInputFreqComplex; CNeuronBaseOCL cNormFreqComplex; CBufferFloat cMeans; CBufferFloat cVariances; CNeuronComplexMLMHAttention cFreqAtteention; CNeuronBaseOCL cUnNormFreqComplex; CBufferFloat cOutputFreqRe; CBufferFloat cOutputFreqIm; CNeuronBaseOCL cOutputTimeSeriasRe; CBufferFloat cOutputTimeSeriasIm; CBufferFloat cZero; class=class="str">"cmt">//--- Noise CNeuronBaseOCL cResidual; class=class="str">"cmt">//--- Forecast CNeuronBaseOCL cConcatInput; CNeuronBatchNormOCL cNormalize; CNeuronPatching cPatching; CNeuronBatchNormOCL cNormalizePLR; CNeuronPatching cPatchingPLR; CNeuronPositionEncoder acPE[class="num">2]; CNeuronMLCrossAttentionMLKV cAttention; CNeuronTransposeOCL cTransposeAtt; CNeuronConvOCL acForecast[class="num">2]; CNeuronTransposeOCL cTransposeFrc; CNeuronRevINDenormOCL cRevIn; CNeuronConvOCL cSum; class=class="str">"cmt">//--- Complex functions class="kw">virtual class="type">bool FFT(CBufferFloat *inp_re, CBufferFloat *inp_im, CBufferFloat *out_re, CBufferFloat *out_im, class="type">bool reverse = class="kw">false); class="kw">virtual class="type">bool ComplexNormalize(class="type">void); class="kw">virtual class="type">bool ComplexUnNormalize(class="type">void); class="kw">virtual class="type">bool ComplexNormalizeGradient(class="type">void); class="kw">virtual class="type">bool ComplexUnNormalizeGradient(class="type">void); class=class="str">"cmt">//--- class="type">bool CutTrendAndOther(CBufferFloat *inputs); class="type">bool CutTrendAndOtherGradient(CBufferFloat *inputs_gr); class="type">bool CutOneFromAnother(class="type">void);
◍ TEMPO 神经元的初始化与 FFT 尺寸推算
CNeuronTEMPOOCL 类把时间序列建模拆成趋势、季节与梯度裁剪三个内部模块，对外暴露的接口里最关键是 Init 方法，它负责把外部传入的序列长度、变量数、预测步长映射到 GPU 缓冲结构。 Init 首先调用基类 CNeuronBaseOCL::Init，传入 forecast * variables 作为输出维度；若基类初始化失败直接返回 false，这意味着任何 OpenCL 上下文异常都会在第一步被拦下。 FFT 长度不是直接用 sequence，而是取大于等于 sequence 的最小 2 的幂。代码里先用 MathLog(size)/M_LN2 求以 2 为底的对数，若 2^power 仍小于 size 则 power 加 1，最终 iFFT = 2^power。例如 sequence=100 时，power 从 6（2^6=64）进位到 7，iFFT=128，这个 padding 对后续频域卷积的显存对齐有直接影响。 趋势与季节分支分别初始化 cPLR、cTrend、cInputSeasons，其中 cPLR 用 iVariables 和 iSequence 构建分段线性表示，cTrend 与 cInputSeasons 的输入维度都是 iSequence * iVariables。任一处 Init 返回 false 则整个神经元作废，外汇与贵金属行情高频跳变下，这种早退机制能避免半初始化模型参与前向传播。
class="type">bool CNeuronTEMPOOCL::Init(class="type">uint numOutputs, class="type">uint myIndex, COpenCLMy *open_cl, class="type">uint sequence, class="type">uint variables, class="type">uint forecast, class="type">uint heads, class="type">uint layers, ENUM_OPTIMIZATION optimization_type, class="type">uint batch) { class=class="str">"cmt">//--- base if(!CNeuronBaseOCL::Init(numOutputs, myIndex, open_cl, forecast * variables, optimization_type, batch)) class="kw">return class="kw">false; class=class="str">"cmt">//--- constants iVariables = variables; iForecast = forecast; iSequence = MathMax(sequence, class="num">1); class=class="str">"cmt">//--- Calculate FFTsize class="type">uint size = iSequence; class="type">int power = class="type">int(MathLog(size) / M_LN2); if(MathPow(class="num">2, power) < size) power++; iFFT = class="type">uint(MathPow(class="num">2, power)); class=class="str">"cmt">//--- trend if(!cPLR.Init(class="num">0, class="num">0, OpenCL, iVariables, iSequence, true, optimization, iBatch)) class="kw">return class="kw">false; if(!cTrend.Init(class="num">0, class="num">1, OpenCL, iSequence * iVariables, optimization, iBatch)) class="kw">return class="kw">false; class=class="str">"cmt">//--- seasons if(!cInputSeasons.Init(class="num">0, class="num">2, OpenCL, iSequence * iVariables, optimization, iBatch)) class="kw">return class="kw">false;
「频率域注意力层的 OpenCL 初始化链」
这段初始化代码把频域 Transformer 的前向通道在 GPU 上逐层落地。任何一步 BufferCreate 或 Init 失败都会直接 return false，意味着整张计算图在 MT5 的 OpenCL 上下文里建不起来，后续推理根本不会跑。 cTranspose[0] 和 [1] 分别用参数 (0,3) 与 (0,4) 做维度置换，承接 iSequence 与 iVariables 的轴交换；cInputFreqRe/Im 按 iFFT * iVariables 长度申请复数实虚部缓冲，cInputFreqComplex 则以 iFFT * iVariables * 2 的展平长度初始化，对应实部虚部拼接后的连续内存。 注意力核心 cFreqAtteention 的 Init 里写死了 head 维度 32，配合 iFFT 长度与 layers 层数决定多头拆分方式；cNormFreqComplex / cUnNormFreqComplex 复用同一 iFFT * iVariables * 2 长度做层归一化的前后变换。 噪声分支 cResidual 缓冲大小为 iSequence * iVariables，预测分支 cConcatInput 直接拉到 3 * iSequence * iVariables——说明输入把三段历史沿变量维拼接。最后 cPatching 用 window = MathMin(5, (int)iSequence-1) 算滑动窗，patches = iSequence - window + 1，当 iSequence=20 时窗为 5、patch 数 16，这个颗粒度直接影响局部时序特征抽取密度。 在 MT5 里把 iSequence 从默认改到 50 以上时，留意 cPatching 的 patches 会涨到 46，显存占用近似线性增加；外汇与贵金属行情高频噪声大，此类频域模型过拟合风险高，参数改动后务必用历史 Tick 回测验证而非直接上实盘。
if(!cTranspose[class="num">0].Init(class="num">0, class="num">3, OpenCL, iSequence, iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!cTranspose[class="num">1].Init(class="num">0, class="num">4, OpenCL, iVariables, iSequence, optimization, iBatch)) class="kw">return class="kw">false; if(!cInputFreqRe.BufferInit(iFFT * iVariables, class="num">0) || !cInputFreqRe.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cInputFreqIm.BufferInit(iFFT * iVariables, class="num">0) || !cInputFreqIm.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cInputFreqComplex.Init(class="num">0, class="num">5, OpenCL, iFFT * iVariables * class="num">2, optimization, batch)) class="kw">return class="kw">false; if(!cNormFreqComplex.Init(class="num">0, class="num">6, OpenCL, iFFT * iVariables * class="num">2, optimization, batch)) class="kw">return class="kw">false; if(!cMeans.BufferInit(iVariables, class="num">0) || !cMeans.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cVariances.BufferInit(iVariables, class="num">0) || !cVariances.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cFreqAtteention.Init(class="num">0, class="num">7, OpenCL, iFFT, class="num">32, heads, iVariables, layers, optimization, batch)) class="kw">return class="kw">false; if(!cUnNormFreqComplex.Init(class="num">0, class="num">8, OpenCL, iFFT * iVariables * class="num">2, optimization, batch)) class="kw">return class="kw">false; if(!cOutputFreqRe.BufferInit(iFFT * iVariables, class="num">0) || !cOutputFreqRe.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cOutputFreqIm.BufferInit(iFFT * iVariables, class="num">0) || !cOutputFreqIm.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cOutputTimeSeriasRe.Init(class="num">0, class="num">9, OpenCL, iFFT * iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!cOutputTimeSeriasIm.BufferInit(iFFT * iVariables, class="num">0) || !cOutputTimeSeriasIm.BufferCreate(OpenCL)) class="kw">return class="kw">false; if(!cZero.BufferInit(iFFT * iVariables, class="num">0) || !cZero.BufferCreate(OpenCL)) class="kw">return class="kw">false; class=class="str">"cmt">//--- Noise if(!cResidual.Init(class="num">0, class="num">10, OpenCL, iSequence * iVariables, optimization, iBatch)) class="kw">return class="kw">false; class=class="str">"cmt">//--- Forecast if(!cConcatInput.Init(class="num">0, class="num">11, OpenCL, class="num">3 * iSequence * iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!cNormalize.Init(class="num">0, class="num">12, OpenCL, class="num">3 * iSequence * iVariables, iBatch, optimization)) class="kw">return class="kw">false; class="type">int window = MathMin(class="num">5, (class="type">int)iSequence - class="num">1); class="type">int patches = (class="type">int)iSequence - window + class="num">1; if(!cPatching.Init(class="num">0, class="num">13, OpenCL, window, class="num">1, class="num">8, patches, class="num">3 * iVariables, optimization, iBatch))
TEMPO 模型各层的初始化与前向串联
这段 CNeuronTEMPOOCL 的 Init 把时序模型里十几个子层逐个挂到 OpenCL 上下文上，任何一层 Init 返回 false 就直接中断，整体返回 false。可以看到位置编码 acPE[0] 用了 14 号层、patch 数由 patches 控制、隐维度是 3*8*iVariables，而 cAttention 的 head 数被 MathMax(heads,1) 兜底，避免传 0 导致核崩溃。 前向 feedForward 里先跑 cPLR 做周期提取，再 CutTrendAndOther 把趋势和非趋势分量切开，随后 cTranspose[0] 对季节输入做转置。这种串行依赖意味着你在 MT5 上调 iVariables 或 patches 时，必须同步核对后面 acPE、cPatchingPLR、cAttention 的维度参数，否则会在 Init 阶段就报错。 外汇与贵金属行情受杠杆与跳空影响，这类深度学习层堆叠的预测模块仅提供概率倾向，实盘前务必用历史数据在策略测试器里跑通 Init 与 feedForward 的完整链路。
if(!acPE[class="num">0].Init(class="num">0, class="num">14, OpenCL, patches, class="num">3 * class="num">8 * iVariables, optimization, iBatch)) class="kw">return class="kw">false; class="type">int plr = cPLR.Neurons(); if(!cNormalizePLR.Init(class="num">0, class="num">15, OpenCL, plr, iBatch, optimization)) class="kw">return class="kw">false; plr = MathMax(plr/(class="num">3 * (class="type">int)iVariables),class="num">1); if(!cPatchingPLR.Init(class="num">0, class="num">16, OpenCL, class="num">3, class="num">3, class="num">8, plr, iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!acPE[class="num">1].Init(class="num">0, class="num">17, OpenCL, plr, class="num">8 * iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!cAttention.Init(class="num">0, class="num">18, OpenCL, class="num">3 * class="num">8 * iVariables, class="num">3 * iVariables, MathMax(heads, class="num">1), class="num">8 * iVariables, MathMax(heads / class="num">2, class="num">1), patches, plr, MathMax(layers, class="num">1), class="num">2, optimization, iBatch)) class="kw">return class="kw">false; if(!cTransposeAtt.Init(class="num">0, class="num">19, OpenCL, patches, class="num">3 * class="num">8 * iVariables, optimization, iBatch)) class="kw">return class="kw">false; if(!acForecast[class="num">0].Init(class="num">0, class="num">20, OpenCL, patches, patches, iForecast, class="num">3 * class="num">8 * iVariables, optimization, iBatch)) class="kw">return class="kw">false; acForecast[class="num">0].SetActivationFunction(LReLU); if(!acForecast[class="num">1].Init(class="num">0, class="num">21, OpenCL, class="num">8 * iForecast, class="num">8 * iForecast, iForecast, class="num">3 * iVariables, optimization, iBatch)) class="kw">return class="kw">false; acForecast[class="num">1].SetActivationFunction(TANH); if(!cTransposeFrc.Init(class="num">0, class="num">22, OpenCL, class="num">3 * iVariables, iForecast, optimization, iBatch)) class="kw">return class="kw">false; if(!cRevIn.Init(class="num">0, class="num">23, OpenCL, class="num">3 * iVariables * iForecast, class="num">11, GetPointer(cNormalize))) class="kw">return class="kw">false; if(!cSum.Init(class="num">0, class="num">24, OpenCL, class="num">3, class="num">3, class="num">1, iVariables, iForecast, optimization, iBatch)) class="kw">return class="kw">false; cSum.SetActivationFunction(None); SetActivationFunction(None); SetOutput(cSum.getOutput(), true); SetGradient(cSum.getGradient(), true); class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CNeuronTEMPOOCL::feedForward(CNeuronBaseOCL *NeuronOCL) { class=class="str">"cmt">//--- trend if(!cPLR.FeedForward(NeuronOCL)) class="kw">return class="kw">false; if(!CutTrendAndOther(NeuronOCL.getOutput())) class="kw">return class="kw">false; if(!cTranspose[class="num">0].FeedForward(cInputSeasons.AsObject()))
◍ 前向传播里的残差与预测拼接
这段前向传播把频域变换、噪声剥离和预测头串成一条链，任何一步返回 false 就整体中止，保证 MT5 端不会因为中间层脏数据继续跑出无意义结果。 FFT 先对转置后的输入做逆变换，再用 Concat 把实部虚部拼回复数矩阵，规模由 iFFT * iVariables 决定；ComplexNormalize 与 ComplexUnNormalize 夹着频域注意力，把能量重新分配。 注释 //--- Noise 下的 CutOneFromAnother 负责从趋势里抠掉周期成分，得到残差；紧接着 //--- Forecast 把趋势、转置输出和残差用 Concat 按 3 * iSequence * iVariables 拼起来送进归一化与分块。 后面两路位置编码 acPE[0]、acPE[1] 分别吃原始分块和 PLR 分块，过注意力后再经两层 acForecast 与转置，最终输出预测张量。外汇与贵金属行情高波动，这套链路在实盘只作概率参考，建议开 MT5 把 iFFT 与 iVariables 调小先做单品种回测。
class="kw">return class="kw">false; if(!FFT(cTranspose[class="num">0].getOutput(), NULL,GetPointer(cInputFreqRe),GetPointer(cInputFreqIm),class="kw">false)) class="kw">return class="kw">false; if(!Concat(GetPointer(cInputFreqRe), GetPointer(cInputFreqIm), cInputFreqComplex.getOutput(), class="num">1, class="num">1, iFFT * iVariables)) class="kw">return class="kw">false; if(!ComplexNormalize()) class="kw">return class="kw">false; if(!cFreqAtteention.FeedForward(cNormFreqComplex.AsObject())) class="kw">return class="kw">false; if(!ComplexUnNormalize()) class="kw">return class="kw">false; if(!DeConcat(GetPointer(cOutputFreqRe), GetPointer(cOutputFreqIm), cUnNormFreqComplex.getOutput(), class="num">1, class="num">1, iFFT * iVariables)) class="kw">return class="kw">false; if(!FFT(GetPointer(cOutputFreqRe), GetPointer(cOutputFreqIm), GetPointer(cInputFreqRe), GetPointer(cOutputTimeSeriasIm), true)) class="kw">return class="kw">false; if(!DeConcat(cOutputTimeSeriasRe.getOutput(), cOutputTimeSeriasRe.getGradient(), GetPointer(cInputFreqRe), iSequence, iFFT - iSequence, iVariables)) class="kw">return class="kw">false; if(!cTranspose[class="num">1].FeedForward(cOutputTimeSeriasRe.AsObject())) class="kw">return class="kw">false; class=class="str">"cmt">//--- Noise if(!CutOneFromAnother()) class="kw">return class="kw">false; class=class="str">"cmt">//--- Forecast if(!Concat(cTrend.getOutput(), cTranspose[class="num">1].getOutput(), cResidual.getOutput(), cConcatInput.getOutput(), class="num">1, class="num">1, class="num">1, class="num">3 * iSequence * iVariables)) class="kw">return class="kw">false; if(!cNormalize.FeedForward(cConcatInput.AsObject())) class="kw">return class="kw">false; if(!cPatching.FeedForward(cNormalize.AsObject())) class="kw">return class="kw">false; if(!acPE[class="num">0].FeedForward(cPatching.AsObject())) class="kw">return class="kw">false; if(!cNormalizePLR.FeedForward(cPLR.AsObject())) class="kw">return class="kw">false; if(!cPatchingPLR.FeedForward(cPatchingPLR.AsObject())) class="kw">return class="kw">false; if(!acPE[class="num">1].FeedForward(cPatchingPLR.AsObject())) class="kw">return class="kw">false; if(!cAttention.FeedForward(acPE[class="num">0].AsObject(), acPE[class="num">1].getOutput())) class="kw">return class="kw">false; if(!cTransposeAtt.FeedForward(cAttention.AsObject())) class="kw">return class="kw">false; if(!acForecast[class="num">0].FeedForward(cTransposeAtt.AsObject())) class="kw">return class="kw">false; if(!acForecast[class="num">1].FeedForward(acForecast[class="num">0].AsObject())) class="kw">return class="kw">false; if(!cTransposeFrc.FeedForward(acForecast[class="num">1].AsObject())) class="kw">return class="kw">false;