📘

克服机器学习的局限性（第二部分）：缺乏可重复性（基础篇）

第 1/2 篇

本章目录

为什么你的回测和别人对不上
同一标的在不同经纪商处数值特性漂移
同一品种两家经纪商的行情并不重合
两家经纪商的 EURUSD 回测差异肉眼可见

「为什么你的回测和别人对不上」

很多读者照着文章里的机器学习示例跑，却复现不出同样的结果。问题大概率不出在代码，而在数据本身。全球外汇市场没有统一的价格清算中心，每家经纪商各自接路透社或其他私有源。同一秒的 EURUSD，经纪商 A 可能记涨 0.12%，经纪商 B 同日的记录却是 -0.65%，差出近 0.77 个百分点。做贵金属或外汇模型验证时，先把经纪商历史数据拉出来比对再谈训练。不同源的价格会让同一套逻辑在不同账户里走出完全相反的 equity 曲线，这是高风险市场的真实摩擦，不是算法失灵。

◍ 同一标的在不同经纪商处数值特性漂移

用 MT5 的 Python 接口拉了两家匿名经纪商（A、B）四年 EURUSD 日线，各 1460 行，但时间戳起点不同：一家回溯到 2019 年 9 月，另一家只到 2020 年 8 月。时区、夏令时、假期差异会让日线对齐出问题，这点老手也容易漏看。算 10 日收益率后差距直接显性化：A 的平均收益 0.000267，B 是 -0.000352，预期收益差了 2.3 倍；A 的收益率方差比 B 高 21%，也就是同标的在不同经纪商处风险画像都不一样。外汇和贵金属本就高波动，这种底层数据分裂会直接改写策略前提。皮尔逊相关系数只有 0.41，远不是直觉里的「接近 1」。说明超一半时间两家 EURUSD 走势不同向，拿「价格全球同步」做假设在数学上立不住。用 RSS/TSS 比率看预测难度：换到 B 经纪商，超越「永远预测均值」简单模型的效率提高 7%，即直接预测 10 日收益容易约 7%。再把均值和标准差拉来比尾部，B 的 10 日收益被夸大 147%。结论很硬：策略盈利不能默认跨经纪商可复制。做 AI 模型（ONNX 或 MQL5 内建）若只用自家经纪商数据训，客户换一家部署就可能翻车。靠谱做法是对目标经纪商单独定制训练，虽然费时但躲不掉。

同一品种两家经纪商的行情并不重合

开 MT5 终端登录后，用 copy_rates_from_pos 拉取 EURUSD 的 D1 数据，长度设 365*4=1460 根，也就是整整 4 年日线。分别用两家经纪商账号各存一份 csv，再读回来做对比——两家都返回了恰好 1460 行，但时间戳起点并不一致，说明底层数据源的拼接方式不同。把列名按经纪商加 A / B 后缀后横向合并，算 10 日收盘价差值当收益标签。肉眼先扫成交量：经纪商 A 的 tick_volume 有长期向上趋势还带周期峰值，经纪商 B 几乎无趋势、随机刺尖密集。同样是 EURUSD，两家报告的活跃度天差地别。风险维度更值得警惕。经纪商 A 的收益率波动比 B 高约 21%，最大回撤 36.79%，比 B 高出约 37%；两者 10 日收益分布零值两侧分立，价格相关性偏低，超一半时间可能反向。外汇和贵金属本就高杠杆高风险，同一标的在不同经纪商那版根本不是同一个「市场视角」。别把经纪商报价当统一真理换账号重跑这套拉数流程，把自己常用平台的 EURUSD 和对手盘比一比，回撤差超过三成就得重新评估止损布防。

MQL5 / C++

class="macro">#Load our libraries
class="kw">import pandas as pd
class="kw">import numpy as np
class="kw">import matplotlib.pyplot as plt
class="kw">import seaborn as sns
class="kw">import MetaTrader5 as mt5
class="macro">#Let us define certain constants
TF = mt5.TIMEFRAME_D1
DATA = (class="num">365 * class="num">4)
START = class="num">1
PAIR = "EURUSD"
class="macro">#Log in to the terminal
if mt5.initialize():
    print(&class="macro">#x27;Logged in successfully&class="macro">#x27;)
else:
    print(&class="macro">#x27;Failed To Log In&class="macro">#x27;)
EURUSD_BROKER_A = pd.DataFrame(mt5.copy_rates_from_pos(PAIR,TF,START,DATA))
class="macro">#Store the data we retrieved from broker A
EURUSD_BROKER_A.to_csv("EURUSD BROKER A.csv")
class="macro">#I have manually changed brokers using the MT5 terminal, you should also do the same on your side
EURUSD_BROKER_B = pd.DataFrame(mt5.copy_rates_from_pos(PAIR,TF,START,DATA))
class="macro">#Store the data we retrieved from broker B
EURUSD_BROKER_B.to_csv("EURUSD BROKER B.csv")
class="macro">#Our forecasting horizon
HORIZON = class="num">10
EURUSD_BROKER_A = pd.read_csv("EURUSD BROKER A.csv")
EURUSD_BROKER_B = pd.read_csv("EURUSD BROKER B.csv")
def format_data(f_data):
    class="macro">#First make a copy of the data, so we always preserve the original data
    f_data_copy = f_data.copy()
    class="macro">#Format the time correctly, form seconds to human readable formats
    f_data_copy[&class="macro">#x27;time&class="macro">#x27;] = pd.to_datetime(f_data_copy[&class="macro">#x27;time&class="macro">#x27;],unit=&class="macro">#x27;s&class="macro">#x27;)
    class="kw">return(f_data_copy)
A = format_data(EURUSD_BROKER_A)
B = format_data(EURUSD_BROKER_B)
# Rename all columns(except the join key)
B = B.rename(columns=lambda col: col + &class="macro">#x27; B&class="macro">#x27; if col != &class="macro">#x27;id&class="macro">#x27; else col)
A = A.rename(columns=lambda col: col + &class="macro">#x27; A&class="macro">#x27; if col != &class="macro">#x27;id&class="macro">#x27; else col)
combined = pd.concat([A,B],axis=class="num">1)
combined[&class="macro">#x27;Null&class="macro">#x27;] = class="num">0
inputs = [&class="macro">#x27;open A&class="macro">#x27;,&class="macro">#x27;high A&class="macro">#x27;,&class="macro">#x27;low A&class="macro">#x27;,&class="macro">#x27;close A&class="macro">#x27;,&class="macro">#x27;tick_volume A&class="macro">#x27;,&class="macro">#x27;spread A&class="macro">#x27;,&class="macro">#x27;open B&class="macro">#x27;,&class="macro">#x27;high B&class="macro">#x27;,&class="macro">#x27;low B&class="macro">#x27;,&class="macro">#x27;close B&class="macro">#x27;,&class="macro">#x27;tick_volume B&class="macro">#x27;,&class="macro">#x27;spread B&class="macro">#x27;]
class="macro">#Label the data
combined[&class="macro">#x27;A Target&class="macro">#x27;] = combined[&class="macro">#x27;close A&class="macro">#x27;].shift(-HORIZON) - combined[&class="macro">#x27;close A&class="macro">#x27;]
combined[&class="macro">#x27;B Target&class="macro">#x27;] = combined[&class="macro">#x27;close B&class="macro">#x27;].shift(-HORIZON) - combined[&class="macro">#x27;close B&class="macro">#x27;]
class="macro">#Drop the last HORIZON rows of data
combined = combined.iloc[:-HORIZON,:]
plt.title(&class="macro">#x27;Broker A Daily EURUSD Tick Volume&class="macro">#x27;)
plt.plot(combined[&class="macro">#x27;tick_volume A&class="macro">#x27;],class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.ylabel(&class="macro">#x27;Tick Volume&class="macro">#x27;)
plt.xlabel(&class="macro">#x27;Historical Day&class="macro">#x27;)
plt.grid()
plt.title(&class="macro">#x27;Broker B Daily EURUSD Tick Volume&class="macro">#x27;)
plt.plot(combined[&class="macro">#x27;tick_volume B&class="macro">#x27;],class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.ylabel(&class="macro">#x27;Tick Volume&class="macro">#x27;)
plt.xlabel(&class="macro">#x27;Historical Day&class="macro">#x27;)
plt.grid()
class="macro">#What&class="macro">#x27;s the average class="num">10-Day EURUSD class="kw">return from both brokers

「两家经纪商的 EURUSD 回测差异肉眼可见」

把两家经纪商近 4 年样本的 10 日 EURUSD 收益拉到一起比，均值差直接算成百分比：delta_return 那行用两列最后两列均值相减再除 A 均值乘 100，得出的字符串前 5 位就是图标题里的「预期收益差百分之几」。方差层面更刺眼。代码注释自己写了「broker A is riskier」，delta_var 用两列 var() 相除，A 的方差相对 B 高出一截，图里红色点线标出两列方差的最小值做基准。最大回撤也偏 A。delta 取两列 min() 相除，A 的最差 10 日收益比 B 恶化出的百分比被塞进标题；直方图里黑红两条标准差分布明显不重叠，热图再补一刀相关性。外汇与贵金属属高风险品种，样本结论只说明历史窗口内经纪商报价端差异，实盘点差与滑点可能放大这些偏离。开 MT5 导出自家经纪商 EURUSD 的 10 日收益，用同样 pandas 切块跑一遍，比画出来的偏差更贴近你账户的真实环境。

MQL5 / C++

delta_return = str(((combined.iloc[:,-class="num">2:].mean()[class="num">0]-combined.iloc[:,-class="num">2:].mean()[class="num">1]) / combined.iloc[:,-class="num">2:].mean()[class="num">0]) * class="num">100)
t = &class="macro">#x27;The Expected class="num">10-Day EURUSD Return Differes by &class="macro">#x27; + delta_return[:class="num">5] + &class="macro">#x27;% Between Our Brokers&class="macro">#x27;
sns.barplot(combined.iloc[:,-class="num">2:].mean(),class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.axhline(class="num">0,class="type">color=&class="macro">#x27;grey&class="macro">#x27;,linestyle=&class="macro">#x27;--&class="macro">#x27;)
plt.title(t)
plt.ylabel(&class="macro">#x27;Return&class="macro">#x27;)
plt.plot(((combined.iloc[:,-class="num">1]-combined.iloc[:,-class="num">1].mean())/combined.iloc[:,-class="num">1].std()),class="type">color=&class="macro">#x27;red&class="macro">#x27;)
plt.plot(((combined.iloc[:,-class="num">2]-combined.iloc[:,-class="num">2].mean())/combined.iloc[:,-class="num">2].std()),class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.grid()
plt.axhline(class="num">0,class="type">color=&class="macro">#x27;black&class="macro">#x27;,linestyle=&class="macro">#x27;--&class="macro">#x27;)
plt.ylabel(&class="macro">#x27;Std. Deviations From Expected class="num">10-Day EURUSD Return&class="macro">#x27;)
plt.xlabel(&class="macro">#x27;Historical Days&class="macro">#x27;)
plt.title(&class="macro">#x27;EURUSD Returns from Different Brokers May Not Always Allign&class="macro">#x27;)
plt.legend([&class="macro">#x27;Broker A&class="macro">#x27;,&class="macro">#x27;Broker B&class="macro">#x27;])
class="macro">#The variance of returns is not the same across both brokers, broker A is riskier
delta_var = str(((combined.iloc[:,-class="num">2:].var()[class="num">0]-combined.iloc[:,-class="num">2:].var()[class="num">1]) / combined.iloc[:,-class="num">2:].var()[class="num">0]) * class="num">100)
t = &class="macro">#x27;Broker A EURUSD Returns Appear to Carry &class="macro">#x27;+ delta_var[:class="num">5]+&class="macro">#x27;% Additional Risk.&class="macro">#x27;
sns.barplot(combined.iloc[:,-class="num">2:].var(),class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.axhline(np.min(combined.iloc[:,-class="num">2:].var()),class="type">color=&class="macro">#x27;red&class="macro">#x27;,linestyle=&class="macro">#x27;:&class="macro">#x27;)
plt.title(t)
plt.ylabel(&class="macro">#x27;Vriance of Returns&class="macro">#x27;)
class="macro">#Broker A also demonstrated the largest drawdown ever in our class="num">4 year sample window
delta = (((combined.iloc[:,-class="num">2:].min()[class="num">0]-combined.iloc[:,-class="num">2:].min()[class="num">1]) / combined.iloc[:,-class="num">2:].min()[class="num">0]) *class="num">100)
delta_s = str(delta)
t = &class="macro">#x27;The Largest Negative class="num">10-Day EURUSD Return Grew By: &class="macro">#x27; + delta_s[:class="num">5] + &class="macro">#x27; %&class="macro">#x27;
sns.barplot(combined.iloc[:,-class="num">2:].min(),class="type">color=&class="macro">#x27;black&class="macro">#x27;)
plt.axhline(np.max(combined.iloc[:,-class="num">2:].min()),class="type">color=&class="macro">#x27;red&class="macro">#x27;,linestyle=&class="macro">#x27;:&class="macro">#x27;)
plt.title(t)
plt.ylabel(&class="macro">#x27;Return&class="macro">#x27;)
sns.histplot(((combined.iloc[:,-class="num">2]-combined.iloc[:,-class="num">2].mean())/combined.iloc[:,-class="num">2].std()),class="type">color=&class="macro">#x27;black&class="macro">#x27;)
sns.histplot(((combined.iloc[:,-class="num">1]-combined.iloc[:,-class="num">1].mean())/combined.iloc[:,-class="num">1].std()),class="type">color=&class="macro">#x27;red&class="macro">#x27;)
plt.xlabel(&class="macro">#x27;Std. Deviations From The Expected Return&class="macro">#x27;)
plt.ylabel(&class="macro">#x27;Frequency&class="macro">#x27;)
plt.title(&class="macro">#x27;Comparing The Distribution of class="num">10-Day EURUSD Returns Between class="num">2 Brokers&class="macro">#x27;)
plt.grid()
plt.legend([&class="macro">#x27;Broker A&class="macro">#x27;,&class="macro">#x27;Broker B&class="macro">#x27;])
sns.heatmap(combined.loc[:,inputs].corr(),annot=True)

→

克服机器学习的局限性（第二部分）：缺乏可重复性·进阶篇

常见问题

大概率不是你写错了，而是行情源、点差、滑点不同导致数值特性漂移，先核对数据商和品种规格再下结论。

两家经纪商的 EURUSD 报价来源和清洗方式不同，行情并不重合，回测差异肉眼可见，换平台前必须重测。

小布可横向比对同品种在多源行情下的统计特征漂移，并标出回测结果分歧较大的参数区间，省去你手动核对的功夫。

未必。先确认实盘经纪商与回测数据源是否一致，缺乏可重复性是常见坑，对齐数据后再评估策略有效性。

用同一策略在两家经纪商历史数据各跑一遍，若净值曲线肉眼可见偏离，就说明可重复性存疑，需排查数据层。