克服集成ONNX（Open Neural Network Exchange ）的挑战·进阶篇
（2/3）· 从简单前馈网络到多模型大数据项目，ONNX部署为何总在预处理环节翻车
很多交易者在MetaTrader 5里跑通一次ONNX前馈网络就以为打通了机器学习，等到真实项目要拉时间序列、做降维、挂多个模型时，才发现缩放没对齐、维度对不上、预测窗口错位。这些坑不在模型本身，而在部署链条的预处理与衔接处。
「把LSTM序列窗口搬进MT5实时流」
GRU、LSTM、RNN这类时间序列模型在股价类预测里常被优先选用，核心原因是它们能啃下跨周期的模式。但要用它们做预测，数据得先切成固定窗口的序列，特征和标签拆开，再 reshape 成模型吃得下的形状。 下面这段 Python 是典型的数据准备函数，time_step 控制回溯长度： def get_sequential_data(data, time_step): if dataset.empty is True: print("Failed to create sequences from an empty dataset") return Y = data.iloc[:, -1].to_numpy() # 取数据集最后一列作为目标，存进一维数组 y X = data.iloc[:, :-1].to_numpy() # 取除最后一列外的所有列作为特征，存进二维数组 x X_reshaped = [] Y_reshaped = [] for i in range(len(Y) - time_step + 1): X_reshaped.append(X[i:i + time_step]) # 按 time_step 滑窗切特征 Y_reshaped.append(Y[i + time_step - 1]) # 对应窗口末端样本作标签 return np.array(X_reshaped), np.array(Y_reshaped) 当 time_step 设为 7 时，该函数返回 x 形状为 (9994, 7, 3)、y 形状为 (9994,)。x 是三维张量，y 是一维向量，在 MT5 里写等价逻辑时必须照这个维度走。 实时预测不需要 y，但要持续重组 x。我们在 MQL5 建了 CTSDataProcessor 类，两个 extract_timeseries_data 重载分别用于训练测试和纯实时；实测在 EA 中跑出与 Python 一致的 (9994,7,3) 维度。外汇与贵金属属高风险品种，这类序列处理只解决数据形态问题，不预示任何方向。 作者训了个小 LSTM：输入层 10 神经元、单隐层，Adam 优化器，100 周期、batch_size 64、patience 5。第 77 周期收敛，loss 0.3000、accuracy 0.8876；转成 ONNX 后在 MT5 加载跑出相同精度，说明跨语言推理链路通了。实时 EA 里要额外加载保存的缩放器（均值/标准差二进制），每次喂新数据前先归一化，否则预测会偏。
def get_sequential_data(data, time_step): if dataset.empty is True: print("Failed to create sequences from an empty dataset") class="kw">return Y = data.iloc[:, -class="num">1].to_numpy() # get the last column from the dataset and assign it to y numpy 1D array X = data.iloc[:, :-class="num">1].to_numpy() # Get all the columns from data array except the last column, assign them to x numpy 2D array X_reshaped = [] Y_reshaped = [] for i in range(len(Y) - time_step + class="num">1): X_reshaped.append(X[i:i + time_step]) Y_reshaped.append(Y[i + time_step - class="num">1]) class="kw">return np.array(X_reshaped), np.array(Y_reshaped) X_reshaped, Y_reshaped = get_sequential_data(dataset, step_size) print(f"x_shape{X_reshaped.shape} y_shape{Y_reshaped.shape}") x_shape(class="num">9994, class="num">7, class="num">3) y_shape(class="num">9994,) class CTSDataProcessor { CTensors *tensor_memory[]; class=class="str">"cmt">//Tensor objects may be hard to track in memory once we class="kw">return them from a function, this keeps track of them class="type">bool xandysplit; class="kw">public: CTSDataProcessor(class="type">void); ~CTSDataProcessor(class="type">void); CTensors *extract_timeseries_data(const matrix<class="type">class="kw">double> &x, const class="type">int time_step); class=class="str">"cmt">//for real time predictions CTensors *extract_timeseries_data(const matrix<class="type">class="kw">double> &MATRIX, vector &y, const class="type">int time_step); class=class="str">"cmt">//for training and testing purposes }; CTSDataProcessor ::CTSDataProcessor(class="type">void) { xandysplit = true; class=class="str">"cmt">//by class="kw">default obtain the y vector also } class=class="str">"cmt">//+------------------------------------------------------------------+
把行情矩阵切成时间窗张量的两个重载
做序列类模型推理前，得先把 MT5 里拿到的样本矩阵按时间步展开成三维张量。下面这段给了同一函数的两个重载：一个只吐 X 时间窗，另一个顺带把 Y 标签拆出来。 第一个重载把 xandysplit 临时置 false，调内部实现后恢复 true，调用方只关心特征时序而不想要标签时直接用它就够了。 第二个重载按 time_step 滑动窗口拼行：外循环走 sample 从 0 到 x.Rows()，内循环 time_step_index 从 0 到 time_step-1，越界即 break。每步用 MatrixExtend::concatenate 把对应行纵向拼进 time_series_matrix；若开启拆分，则把 y_original 对应位置塞进 timeseries_y[0]。 实盘接数据时发现 x.Rows() 小于 time_step 会直接产出空窗，外汇与贵金属波动跳空多，这种边界最好先 assert 再喂网，否则小布盯盘里的 AIGC 信号会静默失效。
CTensors *CTSDataProcessor ::extract_timeseries_data(const matrix<class="type">class="kw">double> &x,const class="type">int time_step) { CTensors *timeseries_tensor; timeseries_tensor = new CTensors(class="num">0); ArrayResize(tensor_memory, class="num">1); tensor_memory[class="num">0] = timeseries_tensor; xandysplit = false; class=class="str">"cmt">//In this function we do not obtain the y vector vector y; timeseries_tensor = extract_timeseries_data(x, y, time_step); xandysplit = true; class=class="str">"cmt">//restore the original condition class="kw">return timeseries_tensor; } CTensors *CTSDataProcessor ::extract_timeseries_data(const matrix &MATRIX, vector &y,const class="type">int time_step) { CTensors *timeseries_tensor; timeseries_tensor = new CTensors(class="num">0); ArrayResize(tensor_memory, class="num">1); tensor_memory[class="num">0] = timeseries_tensor; matrix<class="type">class="kw">double> time_series_data = {}; matrix x = {}; class=class="str">"cmt">//store the x variables converted to timeseries vector y_original = {}; y.Init(class="num">0); if (xandysplit) class=class="str">"cmt">//if we are required to obtain the y vector also split the given matrix into x and y if (!MatrixExtend::XandYSplitMatrices(MATRIX, x, y_original)) { printf("%s failed to split the x and y matrices in order to make a tensor",__FUNCTION__); class="kw">return timeseries_tensor; } x = xandysplit ? x : MATRIX; for (class="type">ulong sample=class="num">0; sample<x.Rows(); sample++) class=class="str">"cmt">//Go throught all the samples { matrix<class="type">class="kw">double> time_series_matrix = {}; vector<class="type">class="kw">double> timeseries_y(class="num">1); for (class="type">ulong time_step_index=class="num">0; time_step_index<(class="type">ulong)time_step; time_step_index++) { if (sample + time_step_index >= x.Rows()) break; time_series_matrix = MatrixExtend::concatenate(time_series_matrix, x.Row(sample+time_step_index), class="num">0); if (xandysplit) timeseries_y[class="num">0] = y_original[sample+time_step_index]; class=class="str">"cmt">//The last value in the column is assumed to be a y value so it gets added to the y vector } }
◍ 把K线切片喂给LSTM的张量对齐细节
在 MT5 里做时序深度学习推理，最容易被忽略的是时间步长的一致性。下面这段截取自数据预处理收尾逻辑：当矩阵行数不足 time_step 时直接跳过，否则把该时间窗矩阵追加进三维张量；若做 X/Y 分离则同步拼接标签。 [CODE] if (time_series_matrix.Rows()<(ulong)time_step) continue; timeseries_tensor.Append(time_series_matrix); if (xandysplit) y = MatrixExtend::concatenate(y, timeseries_y); } return timeseries_tensor; [/CODE] 逐行拆解：第1行判断当前时间窗矩阵行数是否小于设定步长（强制转 ulong 避免符号警告），不够就 continue 丢弃残段；Append 把合规的 (time_step, 特征数) 矩阵压进张量；xandysplit 为真时把对应 y 标签纵向拼接到总标签向量；最终返回三维张量供 ONNX 推理。 EA 初始化时读入 EURUSD-OHLSignal.csv，用 time_step_=7 提取样本，日志打出 x_shape (9994, 7, 3)、y_shape (9994,)。这意味着 9994 个样本、每窗 7 根 K线、3 个特征（如 OHLC 中的 3 列）。注意：time_step_ 必须和 Python 训练时 step_size 完全一致，否则推理张量维度对不上 Onnx 模型输入会直接失败。 对应 Keras 侧结构可作对照：LSTM(units=10, input_shape=(step_size, 特征数-1))，Dense(10, relu) 后接 Dropout(0.3)，末层 softmax 输出类别概率。仅 LSTM 层就有 560 个参数、全连接首层 110 个，总参数量极小，适合 H1 级别 EURUSD 二分类。外汇与贵金属杠杆高，模型信号仅代表历史样本下的概率倾向，实盘前请在策略测试器用同周期数据复核。
if (time_series_matrix.Rows()<(class="type">ulong)time_step) class="kw">continue; timeseries_tensor.Append(time_series_matrix); if (xandysplit) y = MatrixExtend::concatenate(y, timeseries_y); } class="kw">return timeseries_tensor;
「从 Keras 训练到 MT5 加载的链路断点」
上面这段结构打印里，dropout 层输出形状是 (None, 10)、dense_1 是 (None, 2)，总参数 692 个（2.70 KB），说明这是一个极轻量的二分类 LSTM 尾巴。训练时 EarlyStopping 设在 val_loss 上、patience 未给具体值，但日志显示第 75–77 轮 val_accuracy 在 0.8983→0.9063 之间波动，最终全样本 LSTM model accuracy 打印为 0.91795，属过拟合可控的小模型。 y_train / y_test 走的是 to_categorical 做 one-hot，num_classes 直接绑死 len(classes_in_data)，这意味着你换品种或改标签数必须重跑 Python 端，不然 MT5 里推理维度会对不上。模型训完用 tf2onnx 导出，再靠 #resource 把 model.eurusd.D1.onnx 以 uchar 数组塞进 EA，这条路径是目前 MT5 跑时序深度模型最稳的桥。 MT5 侧有个暗坑：input int time_step_ = 7 必须和 Python 里 get_sequential_data 的 step_size 完全一致，注释里也写了。若两边步长错配，TSDataProcessor 拼出的张量形状会和 ONNX 输入节点冲突，predict 直接返回空。外汇与贵金属杠杆高，这类信号仅作概率参考，实盘请先开 MT5 用历史数据回测验证。
from keras.utils class="kw">import to_categorical y_train = to_categorical(y_train, num_classes=len(classes_in_data)) class="macro">#ONE-HOT encoding y_test = to_categorical(y_test, num_classes=len(classes_in_data)) class="macro">#ONE-HOT encoding early_stopping = EarlyStopping(monitor=&class="macro">#x27;val_loss&class="macro">#x27;, patience = patience, restore_best_weights=True) history = model.fit(x_train, y_train, epochs = class="num">100 , validation_data = (x_test,y_test), callbacks=[early_stopping], batch_size=class="num">64, verbose=class="num">2) X_reshaped, Y_reshaped = get_sequential_data(dataset, step_size) predictions = model.predict(X_reshaped) predictions = classes_in_data[np.argmax(predictions, axis=class="num">1)] # Find class with highest probability | converting predicted probabilities to classes from sklearn.metrics class="kw">import accuracy_score print("LSTM model accuracy: ", accuracy_score(Y_reshaped, predictions)) output_path = inp_model_name onnx_model = tf2onnx.convert.from_keras(model, output_path=output_path) print(f"saved model to {output_path}") class="macro">#include <Timeseries Deep Learning\onnx.mqh> class="macro">#include <Timeseries Deep Learning\tsdataprocessor.mqh> class="macro">#include <MALE5\metrics.mqh> class="macro">#resource "\\Files\\model.eurusd.D1.onnx" as class="type">uchar lstm_model[] class="kw">input class="type">int time_step_ = class="num">7; class=class="str">"cmt">//it is very important the time step value matches the one used during training in a python script CONNX onnx; CTSDataProcessor ts_dataprocessor; CTensors *ts_data_tensor; class CONNX { class="kw">protected: class="type">bool initialized; class="type">long onnx_handle;
用 LSTM 模型在 EURUSD 上跑出 91.8% 回测准确率
把训练好的 ONNX 模型塞进 MT5，最直接的方式是用 #resource 把 .onnx 文件和标准化器的均值、标准差二进制一并编入 EA。下面的代码片段里，lstm_model[] 承接 D1 周期的欧元美元模型，standardization_scaler_mean[] 与 standardization_scaler_std[] 则负责推理前的数据归一化。
初始化阶段先 onnx.Init(lstm_model)，失败直接 INIT_FAILED 退出；随后用 MatrixExtend::ReadCsv 读入 EURUSD-OHLSignal.csv，通过 ts_dataprocessor.extract_timeseries_data 切出张量。关键一步是 onnx.predict_bin(ts_data_tensor, classes_in_data)，它把整段历史一次性推完，返回类别预测向量，适合离线测精度。
日志给出实打实的数字：2024.04.14 07:44:16 在 EURUSD H1 上加载 D1 训练的 LSTM，对样本外切片算得 LSTM Model Accuracy: 0.9179507704622774，即约 91.8%。这仅是历史回测命中率，外汇高杠杆品种实盘信号可能随价差与滑点衰减，别直接当胜率用。
想自己复现，把模型文件和 CSV 路径换成你的品种，调 time_step_ 控制回望窗口，跑 OnInit 看 Print 出的准确率即可。
class="macro">#resource "\\Files\\model.eurusd.D1.onnx" as class="type">uchar lstm_model[] class="macro">#resource "\\Files\\EURUSD-SCALER\\mean.bin" as class="type">class="kw">double standardization_scaler_mean[] class="macro">#resource "\\Files\\EURUSD-SCALER\\std.bin" as class="type">class="kw">double standardization_scaler_std[] class="macro">#include <Timeseries Deep Learning\onnx.mqh> class="type">int OnInit() { if (!onnx.Init(lstm_model)) class="kw">return INIT_FAILED; class="type">class="kw">string headers; matrix data = MatrixExtend::ReadCsv("EURUSD-OHLSignal.csv",headers); matrix x; vector y; ts_data_tensor = ts_dataprocessor.extract_timeseries_data(data, y, time_step_); vector classes_in_data = MatrixExtend::Unique(y); vector preds = onnx.predict_bin(ts_data_tensor, classes_in_data); Print("LSTM Model Accuracy: ",Metrics::accuracy_score(y, preds)); class="kw">return(INIT_SUCCEEDED); }
◍ 把训练好的 LSTM 模型接进 MT5 实盘 tick
EURUSD D1 上跑通的 LSTM 二分类信号，关键不在网络结构，而在推理端的预处理必须和 Python 训练时完全一致。time_step_ 写死成 7，就意味着你喂给 ONNX 的必须是连续 7 根日线，且特征顺序为 open/high/low 三列，任何错位都会让预测倾向失真。 加载阶段把 mean.bin 与 std.bin 读进 StandardizationScaler，这一步直接决定归一化是否和训练分布对齐。外汇与贵金属杠杆高，模型信号仅代表历史统计概率，实盘仍可能连续止损。 OnTick 里先用 CopyRates 取最近 7 根 D1，拼成 7×3 矩阵，经 ts_dataprocessor 转成时序张量后再 scaler.transform，最后 onnx.predict_bin 出 0/1 信号并 Comment 到图表。开 MT5 把 model.eurusd.D1.onnx 和 scaler 文件丢进 Files，编译这段就能看到每日信号刷新。
class="macro">#include <Timeseries Deep Learning\tsdataprocessor.mqh> class="macro">#include <MALE5\preprocessing.mqh> class="macro">#resource "\\Files\\model.eurusd.D1.onnx" as class="type">uchar lstm_model[] class="macro">#resource "\\Files\\EURUSD-SCALER\\mean.bin" as class="type">class="kw">double standardization_scaler_mean[]; class="macro">#resource "\\Files\\EURUSD-SCALER\\std.bin" as class="type">class="kw">double standardization_scaler_std[]; class="kw">input class="type">int time_step_ = class="num">7; class=class="str">"cmt">//it is very important the time step value matches the one used during training in a python script CONNX onnx; StandardizationScaler *scaler; CTSDataProcessor ts_dataprocessor; CTensors *ts_data_tensor; class="type">MqlRates rates[]; vector classes_ = {class="num">0,class="num">1}; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert initialization function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">int OnInit() { class=class="str">"cmt">//--- if (!onnx.Init(lstm_model)) class="kw">return INIT_FAILED; scaler = new StandardizationScaler(standardization_scaler_mean, standardization_scaler_std); class=class="str">"cmt">//laoding the saved scaler class=class="str">"cmt">//--- class="kw">return(INIT_SUCCEEDED); } class="type">void OnTick() { if (CopyRates(Symbol(), PERIOD_D1, class="num">1, time_step_, rates)<-class="num">1) { printf("Failed to collect data Err=%d",GetLastError()); class="kw">return; } matrix data(time_step_, class="num">3); for (class="type">int i=class="num">0; i<time_step_; i++) class=class="str">"cmt">//Get the independent values and save them to a matrix { data[i][class="num">0] = rates[i].open; data[i][class="num">1] = rates[i].high; data[i][class="num">2] = rates[i].low; } ts_data_tensor = ts_dataprocessor.extract_timeseries_data(data, time_step_); class=class="str">"cmt">//process the new data into timeseries data = ts_data_tensor.Get(class="num">0); class=class="str">"cmt">//This tensor contains only one matrix for the recent latest bars thats why we find it at the index class="num">0 data = scaler.transform(data); class=class="str">"cmt">//Transform the new data class="type">int signal = onnx.predict_bin(data, classes_); Comment("LSTM trade signal: ",signal); }