股票交易中的非线性回归模型·进阶篇
(2/3)·从线性指标的疲劳到微分方程的自适应比率,一篇讲清进阶建模的落地路径
◍ 七维空间里的比率起跑点
给模型喂价格之前,最头疼的不是算法本身,而是那几个初始比率怎么定。随机值试过,结果分布散到没法用;从 1 起步,优化器第一次迭代就飞了;从 0 起步又会卡在局部最小值里爬不出来。 线性项权重我最终落在 0.5:太小模型丢了趋势性,太大又过度贴着最新价跑。二次项 0.1 是刚好能抓非线性又不至于在价格突变时抽风的起点,动量项 0.2 是凭经验磨出来的,系统在这个值附近结果最稳。 Nelder-Mead 会在七维比率空间里搭单形逐步收缩,本质就是七路同时玩的冷热游戏。精度参数 xatol 和 fatol 都压到 1e-8 才稳——松一点结果跳,紧过头反而陷局部极小。 代码里 maxiter 给到 1000,但实际 300–400 步就收敛了,高波动期偶尔要多跑几步,多余迭代不改善性能,现代硬件上整轮通常不到一分钟。 调试时把比率变化实时画出来,比盯日志容易判断模型往哪偏。
def fit(self, prices): # Prepare data for training X_train, y_train = self.prepare_training_data(prices) # I found these initial values by trial and error initial_coeffs = np.array([class="num">0.5, class="num">0.1, class="num">0.3, class="num">0.1, class="num">0.2, class="num">0.1, class="num">0.0]) result = minimize( self.loss_function, initial_coeffs, args=(X_train, y_train), method=&class="macro">#x27;Nelder-Mead&class="macro">#x27;, options={ &class="macro">#x27;maxiter&class="macro">#x27;: class="num">1000, # More iterations does not improve the result &class="macro">#x27;xatol&class="macro">#x27;: class="num">1e-8, # Accuracy by ratios &class="macro">#x27;fatol&class="macro">#x27;: class="num">1e-8 # Accuracy by loss function } ) self.coefficients = result.x class="kw">return result
别被高R²骗去实盘
在 EURUSD 上第一次看到 R² 突破 0.9 时,我反复查了十遍代码,确认没有数据泄漏。模型确实解释了九成以上的价格波动,但后来明白这是双刃剑——R² 大于 0.95 往往意味着过拟合,外汇市场根本不存在那么强的可预测性,贵金属与汇价的高风险本质决定了历史拟合度不能当真。 MSE 是日常主力指标,但光看均值会吃大亏。有次 MSE 漂亮得离谱,实盘前却发现预测偶发极端异常值,不用跑就知道会亏光本金。此后我先盯误差分布的尾部:max_error 和 error_std 比平均更早发现致命问题。 MAPE 对交易员更直观,「平均误差 0.05%」比讲 R² 容易沟通。陷阱在于价格窄幅震荡时 MAPE 低得有欺骗性,剧烈波动时又瞬间飙升。 经验阈值:R² 高于 0.9 算出色,MSE 小于 0.00001 尚可,MAPE 在 0.05% 内堪称优秀。但比绝对值更重要的是指标随时间的稳定性——宁选稍逊但稳的模型,不要极佳却飘的系统。下面这段评估与验证函数可直接拷去改:
<span class="keyword">def</span> evaluate_model(self, y_true, y_pred): results = { <span class="class="type">class="kw">string">&class="macro">#x27;R²&class="macro">#x27;</span>: r2_score(y_true, y_pred), <span class="class="type">class="kw">string">&class="macro">#x27;MSE&class="macro">#x27;</span>: mean_squared_error(y_true, y_pred), <span class="class="type">class="kw">string">&class="macro">#x27;MAPE&class="macro">#x27;</span>: mean_absolute_percentage_error(y_true, y_pred) * <span class="number">class="num">100</span> } <span class="comment"># Additional statistics that often save the day</span> errors = y_pred - y_true results[<span class="class="type">class="kw">string">&class="macro">#x27;max_error&class="macro">#x27;</span>] = np.<span class="built_in">max</span>(np.<span class="built_in">abs</span>(errors)) results[<span class="class="type">class="kw">string">&class="macro">#x27;error_std&class="macro">#x27;</span>] = np.std(errors) <span class="comment"># Look separately at error distribution "tails"</span> results[<span class="class="type">class="kw">string">&class="macro">#x27;error_quantiles&class="macro">#x27;</span>] = np.percentile(np.<span class="built_in">abs</span>(errors), [<span class="number">class="num">50</span>, <span class="number">class="num">90</span>, <span class="number">class="num">95</span>, <span class="number">class="num">99</span>]) <span class="keyword">class="kw">return</span> results <span class="keyword">def</span> validate_model_performance(self): <span class="comment"># Check metrics on different timeframes</span> timeframes = [<span class="class="type">class="kw">string">&class="macro">#x27;H1&class="macro">#x27;</span>, <span class="class="type">class="kw">string">&class="macro">#x27;H4&class="macro">#x27;</span>, <span class="class="type">class="kw">string">&class="macro">#x27;D1&class="macro">#x27;</span>] <span class="keyword">for</span> tf <span class="keyword">in</span> timeframes: metrics = self.evaluate_on_timeframe(tf) <span class="keyword">if</span> <span class="keyword">not</span> self._check_metrics_thresholds(metrics): <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="comment"># Look at behavior at important historical events</span> stress_periods = self.get_stress_periods() stress_metrics = self.evaluate_on_periods(stress_periods) <span class="keyword">if</span> <span class="keyword">not</span> self._check_stress_performance(stress_metrics): <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="comment"># Check the stability of forecasts</span> stability = self.check_prediction_stability() <span class="keyword">if</span> stability < self.min_stability_threshold: <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="keyword">class="kw">return</span> <span class="literal">True</span>
<span class="keyword">def</span> evaluate_model(self, y_true, y_pred): results = { <span class="class="type">class="kw">string">&class="macro">#x27;R²&class="macro">#x27;</span>: r2_score(y_true, y_pred), <span class="class="type">class="kw">string">&class="macro">#x27;MSE&class="macro">#x27;</span>: mean_squared_error(y_true, y_pred), <span class="class="type">class="kw">string">&class="macro">#x27;MAPE&class="macro">#x27;</span>: mean_absolute_percentage_error(y_true, y_pred) * <span class="number">class="num">100</span> } <span class="comment"># Additional statistics that often save the day</span> errors = y_pred - y_true results[<span class="class="type">class="kw">string">&class="macro">#x27;max_error&class="macro">#x27;</span>] = np.<span class="built_in">max</span>(np.<span class="built_in">abs</span>(errors)) results[<span class="class="type">class="kw">string">&class="macro">#x27;error_std&class="macro">#x27;</span>] = np.std(errors) <span class="comment"># Look separately at error distribution "tails"</span> results[<span class="class="type">class="kw">string">&class="macro">#x27;error_quantiles&class="macro">#x27;</span>] = np.percentile(np.<span class="built_in">abs</span>(errors), [<span class="number">class="num">50</span>, <span class="number">class="num">90</span>, <span class="number">class="num">95</span>, <span class="number">class="num">99</span>]) <span class="keyword">class="kw">return</span> results <span class="keyword">def</span> validate_model_performance(self): <span class="comment"># Check metrics on different timeframes</span> timeframes = [<span class="class="type">class="kw">string">&class="macro">#x27;H1&class="macro">#x27;</span>, <span class="class="type">class="kw">string">&class="macro">#x27;H4&class="macro">#x27;</span>, <span class="class="type">class="kw">string">&class="macro">#x27;D1&class="macro">#x27;</span>] <span class="keyword">for</span> tf <span class="keyword">in</span> timeframes: metrics = self.evaluate_on_timeframe(tf) <span class="keyword">if</span> <span class="keyword">not</span> self._check_metrics_thresholds(metrics): <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="comment"># Look at behavior at important historical events</span> stress_periods = self.get_stress_periods() stress_metrics = self.evaluate_on_periods(stress_periods) <span class="keyword">if</span> <span class="keyword">not</span> self._check_stress_performance(stress_metrics): <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="comment"># Check the stability of forecasts</span> stability = self.check_prediction_stability() <span class="keyword">if</span> stability < self.min_stability_threshold: <span class="keyword">class="kw">return</span> <span class="literal">False</span> <span class="keyword">class="kw">return</span> <span class="literal">True</span>
「让 EA 在 MT5 上不半夜静默崩」
做交易系统最磨人的从来不是公式漂亮与否,而是真账户里 24/7 跑起来别出鬼。作者实盘踩过几次坑:一次重连死锁,一次凌晨系统悄无声息崩了、早上只能盲猜发生了什么,之后才把初始化与日志当成架构一等公民。 下面这段 Python 类骨架(对接 MT5 的封装层)就是那些惨痛经历的产物,不是教科书范例。初始化方法专门处理重连死锁,日志模块用来替代「第二天一脸茫然」。
class PriceEquationModel: def __init__(self): # Model status self.coefficients = None self.training_scores = [] self.optimization_progress = [] # Initializing the connection self._setup_logging() self._init_mt5() def _init_mt5(self): """Initializing connection to MT5""" try: if not mt5.initialize(): raise ConnectionError( "Unable to connect to MetaTrader class="num">5. " "Make sure the terminal is running" ) self.log.info("MT5 connection established") except Exception as e: self.log.critical(f"Critical initialization error: {str(e)}") raise def _safe_mt5_call(self, func, *args, retries=class="num">3, delay=class="num">5): """Secure MT5 function call with automatic recovery""" for attempt in range(retries): try: result = func(*args) if result is not None: class="kw">return result # MT5 sometimes returns None without error raise ValueError(f"MT5 returned None: {func.__name__}") except Exception as e: self.log.warning(f"Attempt {attempt + class="num">1}/{retries} failed: {str(e)}") if attempt < retries - class="num">1: time.sleep(delay) # Trying to reinitialize the connection self._init_mt5() else: raise RuntimeError(f"Call attempts exhausted {func.__name__}") def _update_model_state(self, new_coefficients): """Safely updating model ratio""" if not self._validate_coefficients(new_coefficients):
__init__ 里先 _setup_logging() 再 _init_mt5(),保证后面任何连接异常都有迹可循;_init_mt5 用 mt5.initialize() 判活,连不上直接抛 ConnectionError 并打 critical 日志,绝不带状态不明进主流程。_safe_mt5_call 设了 retries=3, delay=5,也就是最多重试 3 次、每次隔 5 秒,且 MT5 返回 None 也视作失败——这是实盘里常见的「没报错但没数据」坑。最后一次仍失败就抛 RuntimeError,强迫上层停摆而非瞎跑。
模块化不是嘴上功夫:每个组件能单测、能换。想接新指标就写新方法,换数据源只需实现同接口的连接器。外汇与贵金属杠杆高、跳空频繁,这种「宁可断不开混」的架构在真账户上可能显著降低幽灵亏损的概率。
class PriceEquationModel: def __init__(self): # Model status self.coefficients = None self.training_scores = [] self.optimization_progress = [] # Initializing the connection self._setup_logging() self._init_mt5() def _init_mt5(self): """Initializing connection to MT5""" try: if not mt5.initialize(): raise ConnectionError( "Unable to connect to MetaTrader class="num">5. " "Make sure the terminal is running" ) self.log.info("MT5 connection established") except Exception as e: self.log.critical(f"Critical initialization error: {str(e)}") raise def _safe_mt5_call(self, func, *args, retries=class="num">3, delay=class="num">5): """Secure MT5 function call with automatic recovery""" for attempt in range(retries): try: result = func(*args) if result is not None: class="kw">return result # MT5 sometimes returns None without error raise ValueError(f"MT5 returned None: {func.__name__}") except Exception as e: self.log.warning(f"Attempt {attempt + class="num">1}/{retries} failed: {str(e)}") if attempt < retries - class="num">1: time.sleep(delay) # Trying to reinitialize the connection self._init_mt5() else: raise RuntimeError(f"Call attempts exhausted {func.__name__}") def _update_model_state(self, new_coefficients): """Safely updating model ratio""" if not self._validate_coefficients(new_coefficients):
◍ 系数更新时的回滚保护
这段 Python 风格的伪代码演示了在线上模型热更新时如何避免脏状态污染。先校验传入的 ratios 是否合法,不合法直接抛异常中断,防止后续用错误比例计算权重。 核心在于用 old_coefficients 暂存旧参数,try 块里换上新系数后立即跑 _check_model_consistency() 做一致性检查;一旦返回假就主动 raise,进入 except 把系数还原再记错误日志。 这种「先备份、后验证、失败即回滚」的结构在 MT5 的 EA 参数热加载里同样适用。你可以把 self.coefficients 映射成 Expert Advisor 的输入参数数组,在 OnTester 或自定义校验函数里复刻这套逻辑,降低实盘切换模型时的崩坏概率。外汇与贵金属杠杆高,参数错误可能引发滑点放大,务必在策略沙盒先跑通再上真仓。
raise ValueError("Invalid ratios") # Save the previous state old_coefficients = self.coefficients try: self.coefficients = new_coefficients if not self._check_model_consistency(): raise ValueError("Model consistency broken") self.log.info("Model successfully updated") except Exception as e: # Roll back to the previous state self.coefficients = old_coefficients self.log.error(f"Model update error: {str(e)}") raise
MT5历史数据抓取的几个暗坑
用 Python 直连 MT5 拉历史数据,代码看着就几行,真跑起来全是断连和丢数据的坑。我们跟频繁掉线死磕了几个月,才把品种校验、可见性、取数容错和连接回收这套架构钉死。 先说 symbol_info 的硬校验。曾有系统因为配置文件里把品种名拼错,傻等几小时去交易一个根本不存在的品种;现在一律先查 symbol_info,返回 None 直接抛异常,省得后面全废。 然后是 MarketWatch 的「可见性」陷阱。品种存在不等于能取到数——没加进市场观察窗口,copy_rates 永远空手而归;更烦的是终端可能在运行中把品种「忘掉」,所以代码里发现 visible 为 False 就立刻 symbol_select 拉回来。 取数本身也不省心。copy_rates_from_pos 可能因为服务器断连、过载或本地历史不足等十几种原因返回 None,必须当场检查、出错即抛,不要试图「重试蒙混」。 时间戳是 Unix 秒,不转成标准 datetime,后面做小时级或日级序列分析基本寸步难行;转完再丢给预处理拿 close 数组。 最血泪的是 finally 里必须 mt5.shutdown()。不关连接,MT5 会先降速、再随机超时,最后终端直接卡死。外汇和贵金属波动剧烈、滑点凶猛,这类数据管道不稳会让策略信号整体失真,实盘前务必在 MT5 里把这套跑通。
def fetch_data(self, symbol="EURUSD", timeframe=mt5.TIMEFRAME_H1, bars=class="num">10000): """Loading historical data with error handling""" try: # First of all, we check the symbol itself symbol_info = mt5.symbol_info(symbol) if symbol_info is None: raise ValueError(f"Symbol {symbol} unavailable") # MT5 sometimes "loses" MarketWatch symbols if not symbol_info.visible: mt5.symbol_select(symbol, True) # Collect data rates = mt5.copy_rates_from_pos(symbol, timeframe, class="num">0, bars) if rates is None: raise ValueError("Unable to retrieve historical data") # Convert to pandas df = pd.DataFrame(rates) df[&class="macro">#x27;time&class="macro">#x27;] = pd.to_datetime(df[&class="macro">#x27;time&class="macro">#x27;], unit=&class="macro">#x27;s&class="macro">#x27;) class="kw">return self._preprocess_data(df[&class="macro">#x27;close&class="macro">#x27;].values) except Exception as e: print(f"Error while receiving data: {str(e)}") raise finally: # It is important to always close the connection mt5.shutdown()
「前瞻测试里模型没掉链子」
Nelder-Mead 优化跑得像精密钟表:408 次迭代、普通笔记本上不到一分钟收敛。拟合期 R² 0.9958,能解释 99.58% 的价格波动;均方误差 0.00000094,平均预测偏差不到 1 个点,MAPE 仅 0.06%,比多数商用系统 1–2% 的误差低了一个数量级。 参数结构也说得通:前一期价格系数 0.5517,短期记忆性很强;二次项 0.0105 和 0.0368 偏小,价格运动以线性为主;周期项系数 0.1484 印证了市场波浪式运行的经验说法。 最反常的是前瞻测试——通常模型遇新数据会退化,这里 R² 反而升到 0.9970,均方误差再降 19% 至 0.00000076,MAPE 降到 0.05%。重跑确认不是代码 bug 之后,才敢拿去给同行看。 可视化拆开看就有边界了:平静行情里预测如瑞士钟表,但重要新闻发布和突然反转时精度下滑。模型只吃价格数据、没接基本面,这是先天盲区。下面这段 Python 画图函数直接把预测/真实、误差、滚动 R² 三图叠出来,自己跑一遍比看数字直观。
def plot_model_performance(self, predictions, actuals, window=class="num">100): fig, (ax1, ax2, ax3) = plt.subplots(class="num">3, class="num">1, figsize=(class="num">15, class="num">12)) # Forecast vs. real price chart ax1.plot(actuals, &class="macro">#x27;b-&class="macro">#x27;, label=&class="macro">#x27;Real prices&class="macro">#x27;, alpha=class="num">0.7) ax1.plot(predictions, &class="macro">#x27;r--&class="macro">#x27;, label=&class="macro">#x27;Forecast&class="macro">#x27;, alpha=class="num">0.7) ax1.set_title(&class="macro">#x27;Comparing the forecast with the market&class="macro">#x27;) ax1.legend() # Error graph errors = predictions - actuals ax2.plot(errors, &class="macro">#x27;g-&class="macro">#x27;, alpha=class="num">0.5) ax2.axhline(y=class="num">0, class="type">class="kw">color=&class="macro">#x27;k&class="macro">#x27;, linestyle=&class="macro">#x27;:&class="macro">#x27;) ax2.set_title(&class="macro">#x27;Forecast errors&class="macro">#x27;) # Rolling R² rolling_r2 = [r2_score(actuals[i:i+window], predictions[i:i+window]) for i in range(len(actuals)-window)] ax3.plot(rolling_r2, &class="macro">#x27;b-&class="macro">#x27;, alpha=class="num">0.7) ax3.set_title(f&class="macro">#x27;Rolling R²(window {window})&class="macro">#x27;) plt.tight_layout() class="kw">return fig
def plot_model_performance(self, predictions, actuals, window=class="num">100): fig, (ax1, ax2, ax3) = plt.subplots(class="num">3, class="num">1, figsize=(class="num">15, class="num">12)) # Forecast vs. real price chart ax1.plot(actuals, &class="macro">#x27;b-&class="macro">#x27;, label=&class="macro">#x27;Real prices&class="macro">#x27;, alpha=class="num">0.7) ax1.plot(predictions, &class="macro">#x27;r--&class="macro">#x27;, label=&class="macro">#x27;Forecast&class="macro">#x27;, alpha=class="num">0.7) ax1.set_title(&class="macro">#x27;Comparing the forecast with the market&class="macro">#x27;) ax1.legend() # Error graph errors = predictions - actuals ax2.plot(errors, &class="macro">#x27;g-&class="macro">#x27;, alpha=class="num">0.5) ax2.axhline(y=class="num">0, class="type">class="kw">color=&class="macro">#x27;k&class="macro">#x27;, linestyle=&class="macro">#x27;:&class="macro">#x27;) ax2.set_title(&class="macro">#x27;Forecast errors&class="macro">#x27;) # Rolling R² rolling_r2 = [r2_score(actuals[i:i+window], predictions[i:i+window]) for i in range(len(actuals)-window)] ax3.plot(rolling_r2, &class="macro">#x27;b-&class="macro">#x27;, alpha=class="num">0.7) ax3.set_title(f&class="macro">#x27;Rolling R²(window {window})&class="macro">#x27;) plt.tight_layout() class="kw">return fig