寻找市场形态的计量经济学方法:自相关,热点图和散点图·进阶篇
(2/3)·自相关图说市场随机就完了?排除时段后相关性突然显形,这篇教你拆开看
多数人在 MT5 里跑出自相关图看见零附近波动,就默认市场纯随机放弃找形态。其实经典做法把全天混在一起算,季节性增量被噪声冲掉了。换一个角度只挑固定时段切片,记忆和规律才会浮出来。
「时段增量相关性热点图里的两个集簇」
把小时序列按单个时段拆开,再对相邻时段的收盘价差分做互相关,能直接看到一天里哪些小时的价格步调一致。2015–2020 年 EURUSD 的 H1 数据跑出来的热点图上,0–5 点与 10–14 点两团颜色最深,说明这两组时段内部的增量相关性最高。 10–14 点这一集簇的摘要统计值得细看:样本量都是 1265 个,均值在 -0.001016 到 -0.000992 之间,标准差约 0.0246,最大回撤式单步跌幅接近 -0.087。五位小数下均值全为负,意味着这几个时段里价格向下跳的概率略高于向上。 滞后参数会改相关性强度——lag 拉大,趋势分量叠加,时段间相关变高;lag 缩小则数值走低,但集簇的相对排布基本不动。单滞后时 12、13、14 点增量仍强相关,视觉上三条线几乎重叠。 外汇与贵金属属高风险品种,上述负偏移只是样本统计倾向,不等于后续必跌,开 MT5 用下方脚本复算才能确认你自己的品种是否同构。
def correlation_heatmap(symbol, lag, corrthresh): out = pd.DataFrame() rates = pd.DataFrame(MT5CopyRatesRange(symbol, MT5_TIMEFRAME_H1, class="type">class="kw">datetime(class="num">2015, class="num">1, class="num">1), class="type">class="kw">datetime(class="num">2020, class="num">1, class="num">1)), columns=[&class="macro">#x27;time&class="macro">#x27;, &class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;close&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;]) rates = rates.drop([&class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;], axis=class="num">1).set_index(&class="macro">#x27;time&class="macro">#x27;) for i in range(class="num">24): ratesH = None ratesH = rates.drop(rates.index[~rates.index.hour.isin([i])]).diff(lag).dropna() out[str(i)] = ratesH[&class="macro">#x27;close&class="macro">#x27;].reset_index(drop=True) plt.figure(figsize=(class="num">10, class="num">10)) corr = out.corr() mask = np.zeros_like(corr, dtype=np.class="type">bool) mask[np.triu_indices_from(mask)] = True sns.heatmap(corr[corr >= corrthresh], mask=mask) class="kw">return out out = correlation_heatmap(symbol=&class="macro">#x27;EURUSD&class="macro">#x27;, lag=class="num">25, corrthresh=class="num">0.9) out[[&class="macro">#x27;class="num">10&class="macro">#x27;,&class="macro">#x27;class="num">11&class="macro">#x27;,&class="macro">#x27;class="num">12&class="macro">#x27;,&class="macro">#x27;class="num">13&class="macro">#x27;,&class="macro">#x27;class="num">14&class="macro">#x27;]].describe() plt.figure(figsize=(class="num">10,class="num">5)) plt.plot(out[[&class="macro">#x27;class="num">12&class="macro">#x27;,&class="macro">#x27;class="num">13&class="macro">#x27;,&class="macro">#x27;class="num">14&class="macro">#x27;]]) plt.legend(out[[&class="macro">#x27;class="num">12&class="macro">#x27;,&class="macro">#x27;class="num">13&class="macro">#x27;,&class="macro">#x27;class="num">14&class="macro">#x27;]]) plt.show()
相邻时段增量的线性递推预测
价格增量在小时序列里存在两种依赖:一种是日内固定时段因事件重复而产生的相关性,另一种是波动在特定周期集簇带来的近邻关联。把无关时段剔掉,只留 hour 与 hour2 两个点的差额序列,就能单独研究前一小时增量对后一小时的牵引。 用 2015–2020 年 H1 数据看,第 13 与 14 小时增量的散点图高度相关;套用公式 ret[-1] = ret[0] - (ret[lag] - ret[lag-1]) 做一步预测,实际与预测增量散点形状和原图几乎一致,说明靠相关性刻画形态够用,但点并不落在直线——那是预测误差。 加 rfilter=0.03 过滤掉近零预测后,只保留绝对预测值较大的信号,噪声明显减少。把样本回退到 2000–2020,13/14 时段相关性热点图变薄,依赖减弱;该区间增量均值约 0.00047,为正却微弱,不足以支撑两时段双向交易。 3D 散点显示,以 10:00–23:00 为例,相邻时段相关性最强,距离拉远后散点趋圆,第 16 小时起相对前日 10 时已几乎无依赖。外汇与贵金属杠杆高,这种时段依赖随时段漂移,实盘前务必在 MT5 用历史数据重算热点图。
# calculate joinplot between real and predicted returns def hourly_signals_statistics(symbol, lag, hour, hour2, rfilter): rates = pd.DataFrame(MT5CopyRatesRange(symbol, MT5_TIMEFRAME_H1, class="type">class="kw">datetime(class="num">2015, class="num">1, class="num">1), class="type">class="kw">datetime(class="num">2020, class="num">1, class="num">1)), columns=[&class="macro">#x27;time&class="macro">#x27;, &class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;close&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;]) rates = rates.drop([&class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;], axis=class="num">1).set_index(&class="macro">#x27;time&class="macro">#x27;) # price differences for every hour series H = rates.drop(rates.index[~rates.index.hour.isin([hour])]).reset_index(drop=True).diff(lag).dropna() H2 = rates.drop(rates.index[~rates.index.hour.isin([hour2])]).reset_index(drop=True).diff(lag).dropna() # current returns for both hours HF = H[class="num">1:].reset_index(drop=True); HL = H2[class="num">1:].reset_index(drop=True) # previous returns for both hours HF2 = H[:-class="num">1].reset_index(drop=True); HL2 = H2[:-class="num">1].reset_index(drop=True) # Basic equation: ret[-class="num">1] = ret[class="num">0] - (ret[lag] - ret[lag-class="num">1]) # or Close[-class="num">1] = (Close[class="num">0]-Close[lag]) - ((Close[lag]-Close[lag*class="num">2]) - (Close[lag-class="num">1]-Close[lag*class="num">2-class="num">1])) predicted = HF-(HF2-HL2) real = HL # correlation joinplot between two series outcorr = pd.DataFrame() outcorr[&class="macro">#x27;Hour &class="macro">#x27; + str(hour)] = H[&class="macro">#x27;close&class="macro">#x27;] outcorr[&class="macro">#x27;Hour &class="macro">#x27; + str(hour2)] = H2[&class="macro">#x27;close&class="macro">#x27;] # real VS predicted prices out = pd.DataFrame() out[&class="macro">#x27;real&class="macro">#x27;] = real[&class="macro">#x27;close&class="macro">#x27;] out[&class="macro">#x27;predicted&class="macro">#x27;] = predicted[&class="macro">#x27;close&class="macro">#x27;] out = out.loc[((out[&class="macro">#x27;predicted&class="macro">#x27;] >= rfilter) | (out[&class="macro">#x27;predicted&class="macro">#x27;] <=- rfilter))] # plptting results from scipy class="kw">import stats
◍ 用滞后差分把隔小时波动摊开看
这段代码把 EURUSD 的 H1 收盘序列先差分再错位,构造「上一日某小时波动」对「当前日下一小时真实波动」的预测值,核心就是 hourly_signals_statistics3D 里的 predicted = HF-(HF2-HL2)。 样本区间钉死在 2015-01-01 到 2020-01-01,lag=24 意味着用一整天的位移差;调用时 hour=10、hour2=23,等于只扫伦敦—美盘重叠段之后的 13 个小时窗口。 rfilter=0.000 表示不过滤任何预测幅度,所有点都进 3D 散点;若改成 0.0005 则只留预测绝对值大于 5 个点的信号,噪声会明显变稀。 跑完 fig.show() 会弹出 plotly 三维图,x 是小时序号、y 是预测差、z 是真实差,点的大小映射预测绝对值。外汇与贵金属杠杆高、滑点跳空频繁,这种统计关联只反映历史样本内的概率结构,实盘前务必在 MT5 用真实点差重算。
sns.jointplot(x=&class="macro">#x27;Hour &class="macro">#x27; + str(hour), y=&class="macro">#x27;Hour &class="macro">#x27; + str(hour2), data=outcorr, kind="reg", height=class="num">7, ratio=class="num">6).annotate(stats.pearsonr) sns.jointplot(x=&class="macro">#x27;real&class="macro">#x27;, y=&class="macro">#x27;predicted&class="macro">#x27;, data=out, kind="reg", height=class="num">7, ratio=class="num">6).annotate(stats.pearsonr) hourly_signals_statistics(&class="macro">#x27;EURUSD&class="macro">#x27;, lag=class="num">25, hour=class="num">13, hour2=class="num">14, rfilter=class="num">0.00) # calculate joinplot between real an predicted returns def hourly_signals_statistics3D(symbol, lag, hour, hour2, rfilter): rates = pd.DataFrame(MT5CopyRatesRange(symbol, MT5_TIMEFRAME_H1, class="type">class="kw">datetime(class="num">2015, class="num">1, class="num">1), class="type">class="kw">datetime(class="num">2020, class="num">1, class="num">1)), columns=[&class="macro">#x27;time&class="macro">#x27;, &class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;close&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;]) rates = rates.drop([&class="macro">#x27;open&class="macro">#x27;, &class="macro">#x27;low&class="macro">#x27;, &class="macro">#x27;high&class="macro">#x27;, &class="macro">#x27;tick_volume&class="macro">#x27;, &class="macro">#x27;spread&class="macro">#x27;, &class="macro">#x27;real_volume&class="macro">#x27;], axis=class="num">1).set_index(&class="macro">#x27;time&class="macro">#x27;) rates = pd.DataFrame(rates[&class="macro">#x27;close&class="macro">#x27;].diff(lag)).dropna() out = pd.DataFrame(); for i in range(hour, hour2): H = None; H2 = None; HF = None; HL = None; HF2 = None; HL2 = None; predicted = None; real = None; H = rates.drop(rates.index[~rates.index.hour.isin([hour])]).reset_index(drop=True) H2 = rates.drop(rates.index[~rates.index.hour.isin([i+class="num">1])]).reset_index(drop=True) HF = H[class="num">1:].reset_index(drop=True); HL = H2[class="num">1:].reset_index(drop=True); # current hours HF2 = H[:-class="num">1].reset_index(drop=True); HL2 = H2[:-class="num">1].reset_index(drop=True) # last day hours predicted = HF-(HF2-HL2) real = HL out3D = pd.DataFrame() out3D[&class="macro">#x27;real&class="macro">#x27;] = real[&class="macro">#x27;close&class="macro">#x27;] out3D[&class="macro">#x27;predicted&class="macro">#x27;] = predicted[&class="macro">#x27;close&class="macro">#x27;] out3D[&class="macro">#x27;predictedABS&class="macro">#x27;] = predicted[&class="macro">#x27;close&class="macro">#x27;].abs() out3D[&class="macro">#x27;hour&class="macro">#x27;] = i out3D = out3D.loc[((out3D[&class="macro">#x27;predicted&class="macro">#x27;] >= rfilter) | (out3D[&class="macro">#x27;predicted&class="macro">#x27;] <=- rfilter))] out = out.append(out3D) class="kw">import plotly.express as px fig = px.scatter_3d(out, x=&class="macro">#x27;hour&class="macro">#x27;, y=&class="macro">#x27;predicted&class="macro">#x27;, z=&class="macro">#x27;real&class="macro">#x27;, size=&class="macro">#x27;predictedABS&class="macro">#x27;, class="type">color=&class="macro">#x27;hour&class="macro">#x27;, height=class="num">1000, width=class="num">1000) fig.show() hourly_signals_statistics3D(&class="macro">#x27;EURUSD&class="macro">#x27;, lag=class="num">24, hour=class="num">10, hour2=class="num">23, rfilter=class="num">0.000)
「把季节间隔写成可跑的EA逻辑」
这套EA只在识别出的形态间隔里动手:10点到14点(H1时段)才参与,其余时间不触发。它和前一篇的区别是,非形态时段不交易,而形态时段依据统计公式算出的预测增量来开仓。 核心常数 pr 由前文公式得出,用来预测下一根柱线的价格增量。若某时段增量跌到该时段最小阈值以下,就执行卖出。2015至2020年回测中,平均增量转入负区会让买入失效,这个现象你在MT5里加载数据后能自己复现。 遗传优化参数启动后,最优 Lag 落在17–30之间,接近“当天某时段增量依赖前一天同时段”的假设。复盘与正向测试里,该形态在2015–2020整段持续存在,说明计量经济学思路在外汇和贵金属上可能有效,但这类品种波动剧烈、高风险,参数漂移概率不低。 下面这段代码把阈值、滞后和止损都暴露成输入参数,你可以直接拷进MT5改 OpenThreshold 看信号密度变化。
<span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OpenThreshold = <span class="number">class="num">30</span>; <span class="comment">class=class="str">"cmt">//Open threshold</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OpenThreshold1 = <span class="number">class="num">30</span>; <span class="comment">class=class="str">"cmt">//Open threshold class="num">1</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OpenThreshold2 = <span class="number">class="num">30</span>; <span class="comment">class=class="str">"cmt">//Open threshold class="num">2</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OpenThreshold3 = <span class="number">class="num">30</span>; <span class="comment">class=class="str">"cmt">//Open threshold class="num">3</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OpenThreshold4 = <span class="number">class="num">30</span>; <span class="comment">class=class="str">"cmt">//Open threshold class="num">4</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> Lag = <span class="number">class="num">10</span>; <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> stoploss = <span class="number">class="num">150</span>; <span class="comment">class=class="str">"cmt">//Stop loss</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">int</span> OrderMagic = <span class="number">class="num">666</span>; <span class="comment">class=class="str">"cmt">//Orders magic</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">class="kw">double</span> MaximumRisk=<span class="number">class="num">0.01</span>; <span class="comment">class=class="str">"cmt">//Maximum risk</span> <span class="keyword">class="kw">input</span> <span class="keyword">class="type">class="kw">double</span> CustomLot=<span class="number">class="num">0</span>; <span class="comment">class=class="str">"cmt">//Custom lot</span> <span class="keyword">class="type">void</span> <span class="functions">OnTick</span>() { <span class="comment">class=class="str">"cmt">//---</span> <span class="keyword">if</span>(!isNewBar()) <span class="keyword">class="kw">return</span>; <span class="functions">CopyClose</span>(<span class="macro">NULL</span>, <span class="number">class="num">0</span>, <span class="number">class="num">0</span>, Lag*<span class="number">class="num">2</span>+<span class="number">class="num">1</span>, prArr); <span class="functions">ArraySetAsSeries</span>(prArr, <span class="macro">true</span>); <span class="keyword">class="kw">const</span> <span class="keyword">class="type">class="kw">double</span> pr = (prArr[<span class="number">class="num">1</span>] - prArr[Lag]) - ((prArr[Lag] - prArr[Lag*<span class="number">class="num">2</span>]) - (prArr[Lag-<span class="number">class="num">1</span>] - prArr[Lag*<span class="number">class="num">2</span>-<span class="number">class="num">1</span>])); <span class="functions">TimeToStruct</span>(<span class="functions">TimeCurrent</span>(), hours); <span class="keyword">if</span>(hours.hour >=<span class="number">class="num">10</span> && hours.hour <=<span class="number">class="num">14</span>) { <span class="comment">class=class="str">"cmt">//if(countOrders(class="num">0)==class="num">0)</span> <span class="comment">class=class="str">"cmt">// if(pr >= signal && CheckMoneyForTrade(_Symbol,LotsOptimized(),ORDER_TYPE_BUY))</span> <span class="comment">class=class="str">"cmt">// OrderSend(Symbol(),OP_BUY,LotsOptimized(), Ask,class="num">0,Bid-stoploss*_Point,NormalizeDouble(Ask + signal, _Digits),NULL,OrderMagic,INT_MIN);</span> <span class="keyword">if</span>(CheckMoneyForTrade(<span class="predefines">_Symbol</span>,LotsOptimized(),<span class="macro">ORDER_TYPE_SELL</span>)) { <span class="keyword">if</span>(pr <= -signal && hours.hour==<span class="number">class="num">10</span>) <span class="functions">OrderSend</span>(<span class="functions">Symbol</span>(),OP_SELL,LotsOptimized(), Bid,<span class="number">class="num">0</span>,Ask+stoploss*<span class="predefines">_Point</span>,<span class="functions">NormalizeDouble</span>(Bid - signal, <span class="predefines">_Digits</span>),<span class="macro">NULL</span>,OrderMagic); <span class="keyword">if</span>(pr <= -signal1 && hours.hour==<span class="number">class="num">11</span>) <span class="functions">OrderSend</span>(<span class="functions">Symbol</span>(),OP_SELL,LotsOptimized(), Bid,<span class="number">class="num">0</span>,Ask+stoploss*<span class="predefines">_Point</span>,<span class="functions">NormalizeDouble</span>(Bid - signal1, <span class="predefines">_Digits</span>),<span class="macro">NULL</span>,OrderMagic); <span class="keyword">if</span>(pr <= -signal2 && hours.hour==<span class="number">class="num">12</span>)