重构经典策略(第七部分):基于USDJPY的外汇市场与主权债务分析·进阶篇
(2/3)· 当美日10年债收益率差与汇率相关性达-0.85,为什么纯价量模型反而误差更低?
用箱线图拆开多组验证损失
这段 Python 片段干的事很直接:把不同预测因子组合的验证损失摊开看分布,而不是只看一条均值曲线。它先建了一个 2 行 4 列、共享 xy 轴的画布,尺寸 16×10,一次性把 8 个因子的损失箱线图排好。 循环里对每张子图调用 boxplot,数据来自 ohlc_validation_loss 的第 i 列,标题用 columns[i] 标清楚是哪一类因子。债券类数据单独跑了一遍 bonds_validation_loss,同样用 plt.plot 画折线再加 legend,肉眼比对和 OHLC 组差异。 最后 all_validation_loss 是把全部数据掺一起后的结果,箱线图能暴露离群点和方差膨胀——哪一组中位数低、须短,哪一组更可能在样本外稳住。开 Jupyter 把这三段粘进去,换你自己的 validation_loss 矩阵,就能立刻看到因子层面的稳定性断层。
fig,axs = plt.subplots(class="num">2,class="num">4,sharex=True,sharey=True,figsize=(class="num">16,class="num">10)) for i,ax in enumerate(axs.flat): ax.boxplot(ohlc_validation_loss.iloc[:,i]) ax.set_title(columns[i]) class="macro">#Our results using the bonds data bonds_validation_loss class="macro">#Visualizing the results of using the bonds predictors plt.plot(bonds_validation_loss) plt.legend(columns) class="macro">#Visualizing the results of using the bonds predictors fig,axs = plt.subplots(class="num">2,class="num">4,sharex=True,sharey=True,figsize=(class="num">16,class="num">10)) for i,ax in enumerate(axs.flat): ax.boxplot(bonds_validation_loss.iloc[:,i]) ax.set_title(columns[i]) class="macro">#Our results using all the data we have all_validation_loss class="macro">#Visualizing the results of using the bonds predictors plt.plot(all_validation_loss) plt.legend(columns) class="macro">#Visualizing the results of using the bonds predictors fig,axs = plt.subplots(class="num">2,class="num">4,sharex=True,sharey=True,figsize=(class="num">16,class="num">10)) for i,ax in enumerate(axs.flat): ax.boxplot(all_validation_loss.iloc[:,i]) ax.set_title(columns[i])
◍ 用 SHAP 与后向消除交叉验证特征权重
SHAP 值用来量化每个输入相对基线对模型预测的贡献。以线性 SVR 预测 USDJPY 走势为例,将特征按重要性自上而下排列后,收盘价处在最顶端,政府债券类数据紧接所有价格数据之后,而成交量被排在更次要的位置——这暗示债券数据可能比本地市场的 tick_volume 更具解释力。 后向消除则从完整模型起步,逐轮剔除特征直到 5 折交叉验证下的 neg_mean_squared_error 不再改善。mlxtend 跑出的入选集合为 ('open usa','high usa','tick_volume usa','open japan','low japan','close','tick_volume'),可见债券相关字段未被单独列出,但美日两侧的开高低收与量价全数保留,说明算法对纯价格结构的依赖强于 SHAP 图给出的债券偏好。 两种解释器结论不完全一致,模型解释本身带有误差,不能直接当真理。图19 显示误差率随特征剔除未剧烈跳动,即便在特征子集较小时模型也可能保持稳定,这对小样本外汇建模是正向信号,但贵金属与外汇杠杆交易高风险,关系仅为概率倾向。 想自己复现,把下面代码贴进 MT5 配套的 Python 环境即可,LinearSVR 与 SHAP 版本需提前装好。
class="macro">#Feature selection class="kw">import shap class="macro">#The SVR performed quite well, let&class="macro">#x27;s inspect it further model = LinearSVR() model.fit(train_X,train_y) class="macro">#Calculate SHAP Values explainer = shap.Explainer(model.predict,test_X) shap_values = explainer(test_X) shap.plots.beeswarm(shap_values) class="macro">#Let&class="macro">#x27;s also perform backward selection from mlxtend.feature_selection class="kw">import SequentialFeatureSelector as SFS from mlxtend.plotting class="kw">import plot_sequential_feature_selection as plot_sfs class="macro">#Reinitialize the model model = LinearSVR() class="macro">#Prepare the feature selector sfs = SFS(model, k_features=(class="num">1,train_X.shape[class="num">1]), forward=False, n_jobs = -class="num">1, scoring="neg_mean_squared_error", cv=class="num">5) class="macro">#Fit the feature selector sfs_results = sfs.fit(train_X,train_y) class="macro">#The best features we identified sfs_results.k_feature_names_ class="macro">#Prepare the plot fig1 = plot_sfs(sfs_results.get_metric_dict(),kind="std_dev") plt.title("Backward Selection on our Linear SVR") plt.grid()
「调参跑完反而印证了默认设置」
把线性支持向量回归从普通拟合推进到超参数搜索,核心是用随机化搜索在多维网格里找更优组合。下面这段 Python 代码可直接复现调优过程,重点看参数空间怎么铺的。
class="macro">#Parameter tuning from sklearn.model_selection class="kw">import RandomizedSearchCV class="macro">#Reinitialize the model model = LinearSVR() tuner = RandomizedSearchCV(model, { "epsilon":[class="num">0,class="num">0.001,class="num">0.01,class="num">0.1,class="num">25,class="num">50,class="num">100], "tol": [class="num">0.1,class="num">0.01,class="num">0.001,class="num">0.0001,class="num">0.00001], "C" : [class="num">1,class="num">5,class="num">10,class="num">50,class="num">100,class="num">1000,class="num">10000,class="num">100000], "loss":["epsilon_insensitive", "squared_epsilon_insensitive"], "fit_intercept": [False,True] }, n_jobs=-class="num">1, n_iter=class="num">100, scoring="neg_mean_squared_error" ) tuner_results = tuner.fit(train_X,train_y) tuner_results.best_params_
class="macro">#Parameter tuning from sklearn.model_selection class="kw">import RandomizedSearchCV class="macro">#Reinitialize the model model = LinearSVR() tuner = RandomizedSearchCV(model, { "epsilon":[class="num">0,class="num">0.001,class="num">0.01,class="num">0.1,class="num">25,class="num">50,class="num">100], "tol": [class="num">0.1,class="num">0.01,class="num">0.001,class="num">0.0001,class="num">0.00001], "C" : [class="num">1,class="num">5,class="num">10,class="num">50,class="num">100,class="num">1000,class="num">10000,class="num">100000], "loss":["epsilon_insensitive", "squared_epsilon_insensitive"], "fit_intercept": [False,True] }, n_jobs=-class="num">1, n_iter=class="num">100, scoring="neg_mean_squared_error" ) tuner_results = tuner.fit(train_X,train_y) tuner_results.best_params_
用交叉验证揪出过拟合
训练集上跑得漂亮不代表真能泛化,得拿没见过的数据戳一下模型。这里把线性回归、默认 LinearSVR、以及调过参的 LinearSVR 三个模型全 fit 一遍,再用 TimeSeriesSplit 做 5 折时序交叉验证,gap 设成 look_ahead 避免未来信息泄漏。 验证误差存进 overfitting_error 这个 5 行 3 列的 DataFrame,箱线图直接对比三者在未知片段上的 MSE 分布。图 20 显示调参后的 LinearSVR 平均误差最低,说明不仅压过了线性基准,也没在训练集上把噪声当规律背下来。 外汇与贵金属序列里这类过拟合特别容易藏在高频噪声中,MT5 导出的样本外切片建议手动换周期复测,别只信一次拆分的箱线图。
class="macro">#Testing for overfitting baseline_model = LinearRegression() default_model = LinearSVR() customized_model = LinearSVR(tol=class="num">0.0001,loss=&class="macro">#x27;epsilon_insensitive&class="macro">#x27;,fit_intercept=True,epsilon=,C=class="num">1) class="macro">#Fit the models baseline_model.fit(train_X,train_y) default_model.fit(train_X,train_y) customized_model.fit(train_X,train_y) class="macro">#Let&class="macro">#x27;s assess our new accuracy levels test_y = test_y.reset_index() test_X.reset_index(inplace=True) class="macro">#Create our time-series test object tscv = TimeSeriesSplit(n_splits=class="num">5,gap=look_ahead) overfitting_error = pd.DataFrame(columns=columns,index=np.arange(class="num">0,class="num">5)) class="macro">#Cross-validate each model. for j in np.arange(class="num">0,len(columns)): model = models[j] for i , (train,test) in enumerate(tscv.split(test_X)): model.fit(test_X.loc[train[class="num">0]:train[-class="num">1],predictors],test_y.loc[train[class="num">0]:train[-class="num">1],"target"]) overfitting_error.iloc[i,j] = mean_squared_error(test_y.loc[test[class="num">0]:test[-class="num">1],"target"],model.predict(test_X.loc[test[class="num">0]:test[-class="num">1],predictors])) class="macro">#Visualizing the results of using the bonds predictors fig,axs = plt.subplots(class="num">1,class="num">3,sharex=True,sharey=True,figsize=(class="num">8,class="num">4)) for i,ax in enumerate(axs.flat): ax.boxplot(overfitting_error.iloc[:,i]) ax.set_title(columns[i])
◍ 把缩放因子和线性SVR固化为ONNX
在把模型塞进MT5之前,先得解决一个隐性坑:Python里fit时用的是标准化后的数据,EA里如果直接喂原始价格就会全线漂移。做法是按列减均值、除标准差,把每个预测变量的缩放参数落盘,推理时EA端用同一套参数做变换。 下面这段代码先把均值和标准差写进一个DataFrame,再就地标准化merged_data的每一列,最后把缩放因子存到终端的MQL5\Files路径下,文件名带品种与周期便于核对。 模型在所有可获取样本上重新fit后,用skl2onnx转成ONNX,input定义为长度等于预测变量数的浮点张量,target_opset=12。存盘后用netron.start直接起本地可视化,确认输入输出格式和EA端打算传的shape一致再往下走。外汇与贵金属杠杆高,模型失效时回撤可能远超样本内表现,上线前务必在策略测试器用真实点差跑一遍。 别让缩放因子和模型分家:CSV和ONNX必须同批次生成,否则某次重训后均值对不上,EA不会报错但预测会悄悄偏掉。
class="macro">#Create scaling factors scaling_factors = pd.DataFrame(index=("mean","standard deviation"),columns=predictors) class="macro">#Write our the values for i in np.arange(class="num">0,scaling_factors.shape[class="num">1]): scaling_factors.iloc[class="num">0,i] = merged_data.loc[:,predictors[i]].mean() scaling_factors.iloc[class="num">1,i] = merged_data.loc[:,predictors[i]].std() merged_data.loc[:,predictors[i]] = ((merged_data.loc[:,predictors[i]] - scaling_factors.iloc[class="num">0,i]) / scaling_factors.iloc[class="num">1,i]) scaling_factors class="macro">#Save the scaling factors scaling_factors.to_csv("C:\Enter \Your\Path\Here\MetaQuotes\Terminal\D0E82094358C8CF3394F550E51FF075\MQL5\Files\usdjpy scaling factors.csv") class="macro">#Fit the model on all the data we have customized_model.fit(merged_data.loc[:,predictors],merged_data.loc[:,"target"]) class="macro">#Let&class="macro">#x27;s class="kw">import the libraries we need from skl2onnx.common.data_types class="kw">import FloatTensorType from skl2onnx class="kw">import convert_sklearn class="kw">import netron class="kw">import onnx class="macro">#Define the initial input types initial_types = [(&class="macro">#x27;float_input&class="macro">#x27;,FloatTensorType([class="num">1,len(predictors)]))] class="macro">#Create an ONNX representation of the model onnx_model = convert_sklearn(customized_model,initial_types=initial_types,target_opset=class="num">12) class="macro">#Save the ONNX model onnx_name = "USDJPY M1 FLOAT.onnx" onnx.save(onnx_model,onnx_name) class="macro">#Visualize the model netron.start(onnx_name)
「把 ONNX 模型塞进 EA 跑前向预测」
做 USDJPY 这类外汇品种的前向测试,第一步是把训练好的 ONNX 模型编译进程序本体。用 #resource 把 \Files\USDJPY M1 FLOAT.onnx 以 const uchar 数组形式打包,MT5 会在编译期把它嵌进 ex5,省去运行时读外部文件的路径烦恼。外汇与贵金属杠杆高,模型信号只作概率参考,实盘前务必在策略测试器跑足够样本。
全局层先留好接口:long onnx_model 存模型句柄,mean_values[15] 与 std_values[15] 对应 15 维特征的缩放均值和标准差,vector model_output 接模型输出,state 和 prediction 标记当前持仓倾向与最新预测(1 看涨 / 2 看跌)。导入 <Trade/Trade.mqh> 的 CTrade 后,开平仓不用自己拼订单指令。
辅助函数分三块:加载模型并设输入形状为 {1,15},任一步失败就返回 false 中断 OnInit;读缩放 CSV 时跳过首行列标题,从第二行起把均值和标准差填进数组;预测前用均值方差做标准化,输出转成 1/2 的二元状态,便于识别模型是否喊反转。
OnInit 顺序固定:加载 ONNX → 读缩放值 → 跑一次空预测验证模型活着的。OnDeinit 里释放句柄,别让资源泄漏拖慢终端。每次 OnTick 价格动,先取预测;无持仓就跟信号开仓并记 state,有持仓而预测反向就平掉。图 24、25 的前向测试显示,EA 在检测到反转时自动平仓这一机制是可落地的。
class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| USDJPY Bonds.mq5 | class=class="str">"cmt">//| Gamuchirai Ndawana | class=class="str">"cmt">//| [MQL5官方文档] | class=class="str">"cmt">//+------------------------------------------------------------------+ class="macro">#class="kw">property copyright "Gamuchirai Ndawana" class="macro">#class="kw">property link "[MQL5官方文档] class="macro">#class="kw">property version "class="num">1.00" class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Resources | class=class="str">"cmt">//+------------------------------------------------------------------+ class="macro">#resource "\\Files\\USDJPY M1 FLOAT.onnx" as const class="type">uchar onnx_model_buffer[]; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Global variables | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">long onnx_model; class="type">class="kw">float mean_values[class="num">15],std_values[class="num">15]; vector model_output = vector::Zeros(class="num">1); class="type">int state = class="num">0; class="type">int prediction = class="num">0; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Libraries | class=class="str">"cmt">//+------------------------------------------------------------------+ class="macro">#include <Trade/Trade.mqh> CTrade Trade; class=class="str">"cmt">//+------------------------------------------------------------------+ class=class="str">"cmt">//| Load our onnx file | class=class="str">"cmt">//+------------------------------------------------------------------+ class="type">bool load_onnx_file(class="type">void) { class=class="str">"cmt">//--- Create the model from the buffer onnx_model = OnnxCreateFromBuffer(onnx_model_buffer,ONNX_DEFAULT); class=class="str">"cmt">//--- Set the input shape class="type">ulong input_shape [] = {class="num">1,class="num">15}; class=class="str">"cmt">//--- Check if the input shape is valid if(!OnnxSetInputShape(onnx_model,class="num">0,input_shape)) {