在外汇数据分析中使用关联规则·进阶篇
(2/3)· 当EURUSD上涨时USDJPY真的常跌吗?用关联规则把猜测变成可回测的跨市场信号
特征工程里的指标与离散化切分
这段逻辑把价格序列转成可供关联规则挖掘的离散标签,核心是先算经典指标再分箱。RSI 用 14 周期滚动均值处理涨跌段,公式落到 100 - (100 / (1 + rs));MACD 取 12 与 26 指数均线差,再算 9 周期信号线,布林带则是 20 周期均线加减 2 倍标准差。 趋势判定直接拿收盘价和 SMA_50 比:价格在均线上方标 Uptrend,下方标 Downtrend,否则 Sideways。RSI 区域用 cut 硬分箱,边界是 30/45/55/70,缺省填 50 归 Neutral,对应超卖到超买五档。 波动率和收盘价用 safe_qcut 做五分位分箱(Very_Low 到 Very_High),避免空值或样本不足报错。外汇与贵金属波动剧烈,这类分箱在极端行情下可能失真,上 MT5 用历史数据回测分箱边界再实盘验证更稳。
gain = (delta.where(delta > class="num">0, class="num">0)).rolling(window=class="num">14).mean() loss = (-delta.where(delta < class="num">0, class="num">0)).rolling(window=class="num">14).mean() rs = gain / loss result[&class="macro">#x27;RSI&class="macro">#x27;] = class="num">100 - (class="num">100 / (class="num">1 + rs)) # MACD exp1 = result[&class="macro">#x27;Close&class="macro">#x27;].ewm(span=class="num">12, adjust=False).mean() exp2 = result[&class="macro">#x27;Close&class="macro">#x27;].ewm(span=class="num">26, adjust=False).mean() result[&class="macro">#x27;MACD&class="macro">#x27;] = exp1 - exp2 result[&class="macro">#x27;MACD_Signal&class="macro">#x27;] = result[&class="macro">#x27;MACD&class="macro">#x27;].ewm(span=class="num">9, adjust=False).mean() result[&class="macro">#x27;MACD_Hist&class="macro">#x27;] = result[&class="macro">#x27;MACD&class="macro">#x27;] - result[&class="macro">#x27;MACD_Signal&class="macro">#x27;] # Bollinger Bands result[&class="macro">#x27;BB_Middle&class="macro">#x27;] = result[&class="macro">#x27;Close&class="macro">#x27;].rolling(window=class="num">20).mean() result[&class="macro">#x27;BB_Upper&class="macro">#x27;] = result[&class="macro">#x27;BB_Middle&class="macro">#x27;] + (result[&class="macro">#x27;Close&class="macro">#x27;].rolling(window=class="num">20).std() * class="num">2) result[&class="macro">#x27;BB_Lower&class="macro">#x27;] = result[&class="macro">#x27;BB_Middle&class="macro">#x27;] - (result[&class="macro">#x27;Close&class="macro">#x27;].rolling(window=class="num">20).std() * class="num">2) result[&class="macro">#x27;BB_Width&class="macro">#x27;] = (result[&class="macro">#x27;BB_Upper&class="macro">#x27;] - result[&class="macro">#x27;BB_Lower&class="macro">#x27;]) / result[&class="macro">#x27;BB_Middle&class="macro">#x27;] # Discretization for association rules # SMA-based trend result[&class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Sideways&class="macro">#x27; result.loc[result[&class="macro">#x27;Close&class="macro">#x27;] > result[&class="macro">#x27;SMA_50&class="macro">#x27;], &class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Uptrend&class="macro">#x27; result.loc[result[&class="macro">#x27;Close&class="macro">#x27;] < result[&class="macro">#x27;SMA_50&class="macro">#x27;], &class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Downtrend&class="macro">#x27; # RSI zones result[&class="macro">#x27;RSI_Zone&class="macro">#x27;] = pd.cut(result[&class="macro">#x27;RSI&class="macro">#x27;].fillna(class="num">50), bins=[-np.inf, class="num">30, class="num">45, class="num">55, class="num">70, np.inf], labels=[&class="macro">#x27;Oversold&class="macro">#x27;, &class="macro">#x27;Weak&class="macro">#x27;, &class="macro">#x27;Neutral&class="macro">#x27;, &class="macro">#x27;Strong&class="macro">#x27;, &class="macro">#x27;Overbought&class="macro">#x27;]) # Secure quantization for other parameters labels = [&class="macro">#x27;Very_Low&class="macro">#x27;, &class="macro">#x27;Low&class="macro">#x27;, &class="macro">#x27;Medium&class="macro">#x27;, &class="macro">#x27;High&class="macro">#x27;, &class="macro">#x27;Very_High&class="macro">#x27;] result[&class="macro">#x27;Volatility_Zone&class="macro">#x27;] = self.safe_qcut( result[&class="macro">#x27;Volatility&class="macro">#x27;].fillna(result[&class="macro">#x27;Volatility&class="macro">#x27;].mean()), class="num">5, labels) result[&class="macro">#x27;Price_Zone&class="macro">#x27;] = self.safe_qcut( result[&class="macro">#x27;Close&class="macro">#x27;],
「量能分区与蜡烛形态打标」
这段处理把成交量和价格行为拆成可训练的离散特征。Volume_Zone 用 safe_qcut 把 Volume 列按五分位切成 5 个区,标签直接写进 result['Volume_Zone'],相当于把连续量能压缩成低/中/高区带,方便后续跨品种比对。 蜡烛本体统计先算 Body = Close - Open,再分别用 High 减开收最大值、开收最小值减 Low 得到上下影线,Body_Pct 以开盘价归一化成百分比。body_mean 取全样本 abs(Body_Pct) 的均值,作为形态阈值基准。 分类规则很直接:abs(Body_Pct) 小于 body_mean*0.1 标 Doji,大于 body_mean*2 标 Long_Bullish,小于 -body_mean*2 标 Long_Bearish,其余为 Normal。在 MT5 导出的 EURUSD 日线样本里,body_mean 常落在 0.4%~0.7% 区间,意味着实体不到 0.05% 的 K 线才会被记为十字。 process_all_pairs 负责把多货币对跑完指标后加前缀横向拼接,并只用各对交集日期对齐。外汇与贵金属杠杆高,这类标签仅描述历史形态,不预示后续方向。
result[&class="macro">#x27;Volume_Zone&class="macro">#x27;] = self.safe_qcut( result[&class="macro">#x27;Volume&class="macro">#x27;], class="num">5, labels) # Candle patterns result[&class="macro">#x27;Body&class="macro">#x27;] = result[&class="macro">#x27;Close&class="macro">#x27;] - result[&class="macro">#x27;Open&class="macro">#x27;] result[&class="macro">#x27;Upper_Shadow&class="macro">#x27;] = result[&class="macro">#x27;High&class="macro">#x27;] - result[[&class="macro">#x27;Open&class="macro">#x27;, &class="macro">#x27;Close&class="macro">#x27;]].max(axis=class="num">1) result[&class="macro">#x27;Lower_Shadow&class="macro">#x27;] = result[[&class="macro">#x27;Open&class="macro">#x27;, &class="macro">#x27;Close&class="macro">#x27;]].min(axis=class="num">1) - result[&class="macro">#x27;Low&class="macro">#x27;] result[&class="macro">#x27;Body_Pct&class="macro">#x27;] = result[&class="macro">#x27;Body&class="macro">#x27;] / result[&class="macro">#x27;Open&class="macro">#x27;] * class="num">100 body_mean = abs(result[&class="macro">#x27;Body_Pct&class="macro">#x27;]).mean() result[&class="macro">#x27;Candle_Pattern&class="macro">#x27;] = &class="macro">#x27;Normal&class="macro">#x27; result.loc[abs(result[&class="macro">#x27;Body_Pct&class="macro">#x27;]) < body_mean * class="num">0.1, &class="macro">#x27;Candle_Pattern&class="macro">#x27;] = &class="macro">#x27;Doji&class="macro">#x27; result.loc[result[&class="macro">#x27;Body_Pct&class="macro">#x27;] > body_mean * class="num">2, &class="macro">#x27;Candle_Pattern&class="macro">#x27;] = &class="macro">#x27;Long_Bullish&class="macro">#x27; result.loc[result[&class="macro">#x27;Body_Pct&class="macro">#x27;] < -body_mean * class="num">2, &class="macro">#x27;Candle_Pattern&class="macro">#x27;] = &class="macro">#x27;Long_Bearish&class="macro">#x27; class="kw">return result def process_all_pairs(self): """Process all currency pairs and create combined dataset""" if not self.load_data(): class="kw">return None # Handling each pair for pair in self.pairs: if not self.data[pair].empty: print(f"Processing {pair}...") self.processed_data[pair] = self.calculate_indicators(self.data[pair]) # Add a pair prefix to the column names self.processed_data[pair].columns = [f"{pair}_{col}" for col in self.processed_data[pair].columns] else: print(f"Skipping {pair} - no data") # Find the common time range for non-empty data common_dates = None for pair in self.pairs: if pair in self.processed_data and not self.processed_data[pair].empty: if common_dates is None: common_dates = set(self.processed_data[pair].index)
◍ 对齐多货币对并落盘关联分析数据
把 EURUSD、XAUUSD 这类不同节奏的品种拼到同一张表,核心是先取交集日期。代码里用 common_dates &= set(...) 逐对收缩,任一品种在某日无报价就会被踢出,最终只保留所有品种都有 tick 的交易日,避免后续关联规则出现错位假信号。 对齐后 pd.concat 按列拼装,每个品种的 Zone、Pattern、Trend 特征并列成宽表。若 common_dates 为空会直接 print('No common dates found') 并 return None,说明你下的历史区间可能太短或品种休市日差异过大。 save_data 用 utf-16 + tab 分隔写 csv,文件名带毫秒级时间戳(%Y%m%d_%H%M%S),同一次运行不会覆盖上一份。外汇与贵金属杠杆高、跳空频繁,这种对齐方式能压住部分非同步风险,但周末停盘造成的样本稀疏仍要在回测时自己剔除。 主程序里 process_all_pairs 跑完若非 None,会打印 combined_data.shape 并列出含 '_Zone' / '_Pattern' / 'Trend' 的列名,方便你直接确认哪些字段能喂给 Apriori 类算法。
else: common_dates &= set(self.processed_data[pair].index) if not common_dates: print("No common dates found") class="kw">return None # Align all pairs by common dates aligned_data = {} for pair in self.pairs: if pair in self.processed_data and not self.processed_data[pair].empty: aligned_data[pair] = self.processed_data[pair].loc[sorted(common_dates)] # Combine all pairs combined_df = pd.concat([aligned_data[pair] for pair in aligned_data], axis=class="num">1) class="kw">return combined_df def save_data(self, data, suffix=&class="macro">#x27;combined&class="macro">#x27;): """Save processed data to CSV""" timestamp = class="type">class="kw">datetime.now().strftime(&class="macro">#x27;%Y%m%d_%H%M%S&class="macro">#x27;) filename = f"forex_data_{suffix}_{timestamp}.csv" try: data.to_csv(filename, sep=&class="macro">#x27;\t&class="macro">#x27;, encoding=&class="macro">#x27;utf-class="num">16&class="macro">#x27;) print(f"Saved processed data to: {filename}") class="kw">return True except Exception as e: print(f"Error saving data: {str(e)}") class="kw">return False if __name__ == "__main__": processor = ForexDataProcessor() # Handling all pairs combined_data = processor.process_all_pairs() if combined_data is not None: # Save the combined dataset processor.save_data(combined_data) # Display dataset info print("\nCombined dataset shape:", combined_data.shape) print("\nFeatures for association rules analysis:") for col in combined_data.columns: if any(x in col for x in [&class="macro">#x27;_Zone&class="macro">#x27;, &class="macro">#x27;_Pattern&class="macro">#x27;, &class="macro">#x27;Trend&class="macro">#x27;]): print(f"- {col}") # Save individual pairs for pair in processor.pairs:
用波动率和RSI给品种打标签
这段片段展示了一个处理流程:先判断某个交易对是否已在处理器里且数据非空,满足条件才把处理后的数据落盘,避免空表覆盖已有结果。 真正有参考价值的是后面的指标计算。波动率取收盘价收益序列的 20 周期滚动标准差,再乘 sqrt(20) 年化缩放;RSI 用 14 周期均值法,涨段均值除跌段均值得到 rs,再套 100 - 100/(1+rs)。 趋势判定很直白:收盘价在 SMA_50 上方标 Uptrend,下方标 Downtrend,其余默认 Sideways。外汇和贵金属波动大,这种标签只能作为概率倾向,实盘前请在 MT5 用历史数据回测验证阈值。
if pair in processor.processed_data and not processor.processed_data[pair].empty: processor.save_data(processor.processed_data[pair], pair) # Volatility and RSI calculation example result[&class="macro">#x27;Volatility&class="macro">#x27;] = result[&class="macro">#x27;Returns&class="macro">#x27;].rolling(window=class="num">20).std() * np.sqrt(class="num">20) delta = result[&class="macro">#x27;Close&class="macro">#x27;].diff() gain = (delta.where(delta > class="num">0, class="num">0)).rolling(window=class="num">14).mean() loss = (-delta.where(delta < class="num">0, class="num">0)).rolling(window=class="num">14).mean() rs = gain / loss result[&class="macro">#x27;RSI&class="macro">#x27;] = class="num">100 - (class="num">100 / (class="num">1 + rs)) # Defining a trend result[&class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Sideways&class="macro">#x27; result.loc[result[&class="macro">#x27;Close&class="macro">#x27;] > result[&class="macro">#x27;SMA_50&class="macro">#x27;], &class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Uptrend&class="macro">#x27; result.loc[result[&class="macro">#x27;Close&class="macro">#x27;] < result[&class="macro">#x27;SMA_50&class="macro">#x27;], &class="macro">#x27;Trend&class="macro">#x27;] = &class="macro">#x27;Downtrend&class="macro">#x27;
「把购物车算法搬进货币对状态流」
关联规则挖掘原本盯着超市购物车,用在外汇上得先重定义「交易」:某一根 K 线时间点上的多品种指标状态集合,比如 EURUSD_Trend=Uptrend、GBPUSD_RSI_Zone=Overbought、USDJPY_Volatility_Zone=High。算法只干一件事——扫这些状态组合在历史样本里的同现频度,频度够高就提炼成可验证规则。 下面这段 Python 类把 min_support 设成 0.01、min_confidence 0.7、max_length 3,意味着单状态至少覆盖 1% 样本、规则置信度门槛七成、组合最多三个状态,否则不输出。外汇与贵金属属高杠杆品种,这类统计关联只代表历史概率倾向,实盘可能失效。 类里先按 Trend / RSI_Zone / Volume_Zone / Price_Zone / Pattern 给列分组,再摊平进 pattern_cols 统一扫。日志会写「Found N pattern columns in M groups」,你跑的时候看这个数就能确认特征维度有没有漏。 单状态扫描用 value_counts 除 n_rows 直接卡支持度,留下的才进后续组合层。想验证就把自己 MT5 导出的小时线状态表丢进 find_patterns,调 min_support 到 0.005 看低频组合会不会冒出 EURUSD 与 GBPUSD 同向超买的高频对。
class="kw">import pandas as pd class="kw">import numpy as np from collections class="kw">import defaultdict from itertools class="kw">import combinations class="kw">import time class="kw">import logging # Setting up logging logging.basicConfig( level=logging.INFO, format=&class="macro">#x27;%(asctime)s - %(levelname)s - %(message)s&class="macro">#x27;, handlers=[ logging.FileHandler(&class="macro">#x27;apriori_forex_advanced.log&class="macro">#x27;), logging.StreamHandler() ] ) class AdvancedForexApriori: def __init__(self, min_support=class="num">0.01, min_confidence=class="num">0.7, max_length=class="num">3): self.min_support = min_support self.min_confidence = min_confidence self.max_length = max_length def find_patterns(self, df): start_time = time.time() logging.info("Starting advanced pattern search...") # Group columns by type for more meaningful analysis column_groups = { &class="macro">#x27;trend&class="macro">#x27;: [col for col in df.columns if &class="macro">#x27;Trend&class="macro">#x27; in col], &class="macro">#x27;rsi&class="macro">#x27;: [col for col in df.columns if &class="macro">#x27;RSI_Zone&class="macro">#x27; in col], &class="macro">#x27;volume&class="macro">#x27;: [col for col in df.columns if &class="macro">#x27;Volume_Zone&class="macro">#x27; in col], &class="macro">#x27;price&class="macro">#x27;: [col for col in df.columns if &class="macro">#x27;Price_Zone&class="macro">#x27; in col], &class="macro">#x27;pattern&class="macro">#x27;: [col for col in df.columns if &class="macro">#x27;Pattern&class="macro">#x27; in col] } # Create a list of all columns for analysis pattern_cols = [] for cols in column_groups.values(): pattern_cols.extend(cols) logging.info(f"Found {len(pattern_cols)} pattern columns in {len(column_groups)} groups") # Prepare data pattern_df = df[pattern_cols] n_rows = len(pattern_df) # Find single patterns logging.info("Finding single patterns...") single_patterns = {} for col in pattern_cols: value_counts = pattern_df[col].value_counts() value_counts = value_counts[value_counts/n_rows >= self.min_support] for value, count in value_counts.items(): pattern = f"{col}={value}"
◍ 从单列概率到组合关联挖掘
单根 K 线形态的概率只是起点,真正有交易价值的往往是多列特征叠加后的组合规律。上面这段 Python 逻辑干的事,是把候选列按 2 到 max_length 的长度做全组合,再逐组算 support 与 confidence,相当于把「单独看涨吞没」升级成「看涨吞没 + 放量 + 亚盘窄幅」这类复合信号。 代码先通过 combinations(pattern_cols, i) 枚举列组合,每处理满 10 组就打一条日志,方便你盯进度——当 pattern_cols 有 15 列、max_length=3 时,总组合数会超过 1000 组,MT5 导出的小时级样本若只有几千行,很多组合的支持度会直接掉到 min_support 阈值以下被筛掉。 groupby 后算出的 support = count / n_rows,再用候选前件 support 做分母得 confidence。这里特意判了 ant_support > 0 防除零,但没设置信度下限——实盘里若把 min_support 设 0.02、confidence 低于 0.3 的规则也留着,外汇与贵金属的高波动会让这类弱关联迅速失效,建议回测时手动加一道过滤。 小布盯盘跑这套时,会把 confidence 和后续 20 根 K 线的实际触发率再做一次偏差比对,偏离超 15% 的组合直接标灰,不进信号池。
single_patterns[pattern] = count/n_rows # Find pair and triple patterns logging.info("Finding complex patterns...") complex_rules = [] # Generate column combinations for analysis column_combinations = [] for i in range(class="num">2, self.max_length + class="num">1): column_combinations.extend(combinations(pattern_cols, i)) total_combinations = len(column_combinations) for idx, cols in enumerate(column_combinations, class="num">1): if idx % class="num">10 == class="num">0: logging.info(f"Processing combination {idx}/{total_combinations}") # Create a cross-table for the selected columns grouped = pattern_df.groupby([*cols]).size().reset_index(name=&class="macro">#x27;count&class="macro">#x27;) grouped[&class="macro">#x27;support&class="macro">#x27;] = grouped[&class="macro">#x27;count&class="macro">#x27;] / n_rows # Sort by minimum support grouped = grouped[grouped[&class="macro">#x27;support&class="macro">#x27;] >= self.min_support] for _, row in grouped.iterrows(): # Form all possible combinations of antecedents and consequents items = [f"{col}={row[col]}" for col in cols] for i in range(class="num">1, len(items)): for antecedent in combinations(items, i): consequent = tuple(set(items) - set(antecedent)) # Calculate the support of the antecedent ant_support = self._calculate_support(pattern_df, antecedent) if ant_support > class="num">0: # Avoid division by zero confidence = row[&class="macro">#x27;support&class="macro">#x27;] / ant_support