数据科学和机器学习（第 32 部分）：保持您的 AI 模型更新，在线学习·进阶篇
（2/3）· 模型去年训练今年失效？用增量学习接住比特币新高与纳指新峰值
◍ 把 CatBoost 模型塞进 MT5 的实操链路
训练侧先把目标列并入数据框，再丢弃空行，避免后续切分时报错。特征矩阵 X 剔除了 spread、real_volume、future_close、future_open 和 target 本身，标签 y 单独取 target，按 0.7 比例随机切分（random_state=42 保证可复现）。 CatBoost 跑完 100 轮后，日志显示 bestIteration=3、bestTest=0.6927，脚本自动收缩到前 4 次迭代；但样本外准确率最终落在 0.5，说明该特征组合在 EURUSD H1 上仅接近随机猜测，外汇品种高杠杆下直接跟单风险极大。 模型导出为 Files/catboost.H1.onnx，MT5 端用 CCatBoost 加载时需先确认 FileIsExist 走 FILE_COMMON 路径，否则 OnInit 直接 INIT_FAILED。下面这段是 EA 初始化核，建议在 MT5 里新建脚本逐行验证文件是否存在、模型能否 Init 成功。 训练日志里 learn 从 0.6916 降到 0.5968、test 从 0.6934 升到 0.7010，过拟合迹象明显，调参时倾向把迭代上限压到 4 附近而不是跑满 100。
class="macro">#include <CatBoost.mqh> CCatBoost *catboost; input class="type">class="kw">string model_name = "catboost.H1.onnx"; input class="type">class="kw">string symbol = "EURUSD"; input ENUM_TIMEFRAMES timeframe = PERIOD_H1; class="type">class="kw">string common_path; class="type">int OnInit() { class=class="str">"cmt">//--- Check if the model file exists if (!FileIsExist(model_name, FILE_COMMON)) { printf("%s Onnx file doesn&class="macro">#x27;t exist",__FUNCTION__); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- Initialize a catboost model catboost = new CCatBoost(); if (!catboost.Init(model_name, ONNX_COMMON_FOLDER)) { printf("%s failed to initialize the catboost model, error = %d",__FUNCTION__,GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- }
「用刚收掉的 K 线喂给 CatBoost 做多空预判」
上面那张表是 EURUSD 一小时周期连续五根已收盘 K 线的原始字段：time、open、high、low、close、tick_volume。第 0 根收盘时间 1726772400（约 2024-09-19 21:00 UTC），收盘 1.11556，成交量 3315；到第 4 根收盘 1.11615，成交量萎缩到 1166，量价背离倾向暗示上行动能走弱。 OnTick 里我们只取「最近一根已收盘 Bar」做推理，不碰当前未定型 K 线，避免未来函数污染信号。CopyRates 的起始位置填 1、数量填 1，就是跳过正在形成的第 0 根实时 Bar，只拿它前面那根完整 Bar。 把 time、open、high、low、close、tick_volume 六个字段塞进 vector x，直接丢给 catboost.predict_bin(x)。返回 0 判为空头信号，返回 1 判为多头信号。外汇与贵金属属高风险品种，该信号仅基于历史形态概率，实战中需结合仓位与止损。 让小布替你跑这套 把上面代码贴进 MT5 策略测试器，symbol 和 timeframe 换成你盯的盘面，就能在图表左上角实时看到预测标签，先观察一百根 Bar 的命中节奏再决定是否接实盘。
class="type">void OnTick() { class=class="str">"cmt">//--- class="type">MqlRates rates[]; CopyRates(symbol, timeframe, class="num">1, class="num">1, rates); class=class="str">"cmt">//copy the recent closed bar information vector x = { (class="type">class="kw">double)rates[class="num">0].time, rates[class="num">0].open, rates[class="num">0].high, rates[class="num">0].low, rates[class="num">0].close, (class="type">class="kw">double)rates[class="num">0].tick_volume}; Comment(TimeCurrent(),"\nPredicted signal: ",catboost.predict_bin(x)==class="num">0?"Bearish":"Bullish");class=class="str">"cmt">// if the predicted signal is class="num">0 it means a bearish signal, otherwise it is a bullish signal }
让模型自己按时换血
在 MT5 里手动训练和挂载模型只是第一步，真正的价值是把整条链路交给调度器自动跑。Python 侧装一个轻量的 schedule 库就能解决定时问题：用 pip install schedule 装好后，把数据收集、训练、保存模型的逻辑封成一个函数，然后每分钟触发一次。 调度代码非常直白：schedule.every(1).minute.do(trainAndSaveCatBoost) 让 CatBoost 训练函数每分钟被呼叫；下方 while True 循环里 schedule.run_pending() 配合 time.sleep(60) 保活脚本，每 60 秒检查一次待执行任务。这样模型文件会持续被刷新到公用目录。 EA 端则用 OnTimer 接管模型热更新。OnInit 里 EventSetTimer(60) 设了 60 秒定时器，失败就返回 INIT_FAILED；到点的 OnTimer 先 delete 旧 catboost 对象释放内存，再 new 一个 CCatBoost 并从 ONNX_COMMON_FOLDER 加载同名模型，打印「New model loaded」表示换血完成。外汇与贵金属杠杆高，自动覆盖模型若训练数据偏移，可能引发行情错配，需人工抽查训练集。 别把 Sklearn 管道当万能 深度学习类如 GRU 这种 RNN 变体塞不进 Sklearn 管道，自动化训练脚本得单独写加载与序列化分支；若混用同一调度，可能卡在 pickle 环节。先用 CatBoost 跑通每分钟自更新，再扩到 GRU 更稳。
schedule.every(class="num">1).minute.do(trainAndSaveCatBoost) class="macro">#schedule catboost training # Keep the script running to execute the scheduled tasks while True: schedule.run_pending() time.sleep(class="num">60) # Wait for class="num">1 minute before checking again class="type">int OnInit() { class=class="str">"cmt">//--- Check if the model file exists .... class=class="str">"cmt">//--- Initialize a catboost model .... class=class="str">"cmt">//--- if (!EventSetTimer(class="num">60)) class=class="str">"cmt">//Execute the OnTimer function after every class="num">60 seconds { printf("%s failed to set the event timer, error = %d",__FUNCTION__,GetLastError()); class="kw">return INIT_FAILED; } class=class="str">"cmt">//--- class="kw">return(INIT_SUCCEEDED); } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert deinitialization function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void OnDeinit(const class="type">int reason) { class=class="str">"cmt">//--- if (CheckPointer(catboost) != POINTER_INVALID) class="kw">delete catboost; } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Expert tick function | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void OnTick() { class=class="str">"cmt">//--- .... } class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">void OnTimer(class="type">void) { if (CheckPointer(catboost) != POINTER_INVALID) class="kw">delete catboost; class=class="str">"cmt">//--- Load the new model after deleting the prior one from memory catboost = new CCatBoost(); if (!catboost.Init(model_name, ONNX_COMMON_FOLDER)) { printf("%s failed to initialize the catboost model, error = %d",__FUNCTION__,GetLastError()); class="kw">return; } printf("%s New model loaded",TimeToString(TimeCurrent(), TIME_DATE|TIME_MINUTES)); }
◍ 多账户并行加载 CatBoost 模型的日志现象
在同时挂多个信号源账户时，EA 的日志会按账户前缀分行刷出模型加载记录。上面这段输出里，HO 账户在 13:14:00 加载了 EURUSD D1 的在线学习 CatBoost 模型，而 FK 到 LH 共 9 个账户在 13:15:55 至 13:22:55 之间，每隔约 60 秒依次加载 GBPUSD H1 的同类型模型。 这种每分钟一个账户、错峰加载的节奏，说明调度层把模型热更新打散到了不同账户的执行线程，避免同一秒集中读写模型文件造成 IO 阻塞。 外汇与贵金属交易属高风险，多账户并行跑在线学习模型时，要核对每个账户日志里的品种与周期是否和预设一致，防止某账户误载了错周期模型导致信号错位。
HO class="num">0 class="num">13:class="num">14:class="num">00.648 Online Learning Catboost(EURUSD,D1) class="num">2024.11.class="num">18 class="num">12:class="num">14 New model loaded FK class="num">0 class="num">13:class="num">15:class="num">55.388 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">15 New model loaded JG class="num">0 class="num">13:class="num">16:class="num">55.380 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">16 New model loaded MP class="num">0 class="num">13:class="num">17:class="num">55.376 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">17 New model loaded JM class="num">0 class="num">13:class="num">18:class="num">55.377 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">18 New model loaded PF class="num">0 class="num">13:class="num">19:class="num">55.368 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">19 New model loaded CR class="num">0 class="num">13:class="num">20:class="num">55.387 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">20 New model loaded NO class="num">0 class="num">13:class="num">21:class="num">55.377 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">21 New model loaded LH class="num">0 class="num">13:class="num">22:class="num">55.379 Online Learning Catboost(GBPUSD,H1) class="num">2024.11.class="num">18 class="num">12:class="num">22 New model loaded
「GRU 模型怎么搬进 MT5 实时跑」
把 Python 里训好的 GRU 分类器落地到 MT5，关键不在模型结构，而在工程衔接：训练后除了导出 ONNX，还要把 StandardScaler 的均值和标准差存成二进制文件，否则 MQL5 端对新数据做标准化时会和训练分布漂移，预测倾向失真。 MT5 侧在 OnInit 里先取缩放器文件名，再从公共文件夹读 ONNX 模型与二进制 scaler，把均值、标准差分别填进 scaler_mean 和 scaler_std 数组；OnTimer 里定时重读这两个文件，实现模型与缩放参数的热更新。 RNN 类的时态依赖靠时间步长支撑。trainAndSaveGRU 里写死 time_step=10，所以 MQL5 取最近 10 根收盘柱作为一个样本窗口喂给模型——少于 10 根时预测可能无效。外汇与贵金属杠杆高，这类在线推断只作概率参考，实盘前请在策略测试器用历史数据验证窗口与 scaler 一致性。 下面这段是 Python 端的 GRUClassifier 核心骨架，注意输入维度 (time_step, 特征数) 和导出时 opset=13 的设定，MT5 的 ONNX 解析版本要能匹配。
class="kw">import numpy as np class="kw">import tensorflow as tf from tensorflow.keras.models class="kw">import Sequential from tensorflow.keras.layers class="kw">import GRU, Dense, Input, Dropout from keras.callbacks class="kw">import EarlyStopping from keras.optimizers class="kw">import Adam class="kw">import tf2onnx class GRUClassifier(): def __init__(self, time_step, X_train, X_test, y_train, y_test): self.X_train = X_train self.X_test = X_test self.y_train = y_train self.y_test = y_test self.model = None self.time_step = time_step self.classes_in_y = np.unique(self.y_train) def train(self, learning_rate=class="num">0.001, layers=class="num">2, neurons = class="num">50, activation="relu", batch_size=class="num">32, epochs=class="num">100, loss="binary_crossentropy", verbose=class="num">0): self.model = Sequential() self.model.add(Input(shape=(self.time_step, self.X_train.shape[class="num">2]))) self.model.add(GRU(units=neurons, activation=activation)) # input layer for layer in range(layers): # dynamically adjusting the number of hidden layers self.model.add(Dense(units=neurons, activation=activation)) self.model.add(Dropout(class="num">0.5)) self.model.add(Dense(units=len(self.classes_in_y), activation=&class="macro">#x27;softmax&class="macro">#x27;, name=&class="macro">#x27;output_layer&class="macro">#x27;)) # the output layer # Compile the model adam_optimizer = Adam(learning_rate=learning_rate) self.model.compile(optimizer=adam_optimizer, loss=loss, metrics=[&class="macro">#x27;accuracy&class="macro">#x27;]) early_stopping = EarlyStopping(monitor=&class="macro">#x27;val_loss&class="macro">#x27;, patience=class="num">10, restore_best_weights=True) history = self.model.fit(self.X_train, self.y_train, epochs=epochs, batch_size=batch_size, validation_data=(self.X_test, self.y_test), callbacks=[early_stopping], verbose=verbose) val_loss, val_accuracy = self.model.evaluate(self.X_test, self.y_test, verbose=verbose) print("Gru accuracy on validation sample = ",val_accuracy) def to_onnx(self, model_name, standard_scaler): # Convert the Keras model to ONNX spec = (tf.TensorSpec((None, self.time_step, self.X_train.shape[class="num">2]), tf.float16, name="input"),) self.model.output_names = [&class="macro">#x27;outputs&class="macro">#x27;] onnx_model, _ = tf2onnx.convert.from_keras(self.model, input_signature=spec, opset=class="num">13)
把 GRU 模型与归一化参数落盘供 MT5 调用
训练完的 ONNX 模型必须连同标准化器的均值与缩放系数一起存文件，否则 MT5 端加载后无法还原训练时的量纲。下面这段 Python 把模型序列化写盘，并分别用 tofile 导出 .standard_scaler_mean.bin 与 .standard_scaler_scale.bin，文件名自动剥掉 .onnx 后缀。 训练函数 trainAndSaveGRU 先取 1000 根 K 线（start=1），用下一根的开盘、收盘构造二分类标签：若 future_close 大于 future_open 则标 1，否则标 0；dropna 后若为空会打印 mt5.last_error() 并 shutdown。特征矩阵剔除了 spread、real_volume 及未来字段，按 train_size=0.7 且不打乱顺序切分。 time_step 固定为 10，StandardScaler 仅用训练集 fit 再 transform 测试集，避免前视偏差。序列由 create_sequences 生成，标签做 one-hot 后送 GRUClassifier；训练超参为 batch_size=64、learning_rate=0.001、relu 激活、epochs=1000、binary_crossentropy 损失。外汇与贵金属波动剧烈，这套流程仅提供概率倾向，实盘前请在 MT5 用历史数据复核归一化文件是否匹配。
# Save the ONNX model to a file
with open(model_name, "wb") as f:
f.write(onnx_model.SerializeToString())
# Save the mean and scale parameters to binary files
standard_scaler.mean_.tofile(f"{model_name.replace(&class="macro">#x27;.onnx&class="macro">#x27;,&class="macro">#x27;&class="macro">#x27;)}.standard_scaler_mean.bin")
standard_scaler.scale_.tofile(f"{model_name.replace(&class="macro">#x27;.onnx&class="macro">#x27;,&class="macro">#x27;&class="macro">#x27;)}.standard_scaler_scale.bin")
def trainAndSaveGRU():
data = getData(start=class="num">1, bars=class="num">1000)
# Preparing the target variable
data["future_open"] = data["open"].shift(-class="num">1)
data["future_close"] = data["close"].shift(-class="num">1)
target = []
for row in range(data.shape[class="num">0]):
if data["future_close"].iloc[row] > data["future_open"].iloc[row]:
target.append(class="num">1)
else:
target.append(class="num">0)
data["target"] = target
data = data.dropna()
# Check if we were able to receive some data
if (len(data)<=class="num">0):
print("Failed to obtain data from Metatrader5, error = ",mt5.last_error())
mt5.shutdown()
X = data.drop(columns = ["spread","real_volume","future_close","future_open","target"])
y = data["target"]
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=class="num">0.7, shuffle=False)
########### Preparing data for timeseries forecasting ###############
time_step = class="num">10
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
x_train_seq, y_train_seq = create_sequences(X_train, y_train, time_step)
x_test_seq, y_test_seq = create_sequences(X_test, y_test, time_step)
###### One HOt encoding #######
y_train_encoded = to_categorical(y_train_seq)
y_test_encoded = to_categorical(y_test_seq)
gru = gru_models.GRUClassifier(time_step=time_step,
X_train= x_train_seq,
y_train= y_train_encoded,
X_test= x_test_seq,
y_test= y_test_encoded
)
gru.train(
batch_size=class="num">64,
learning_rate=class="num">0.001,
activation = "relu",
epochs=class="num">1000,
loss="binary_crossentropy",