神经网络变得轻松(第十九部分):使用 MQL5 的关联规则·进阶篇
(2/3)· 从 Apriori 的数据库瓶颈切换到 FP 树内存运算,多分支节点类与挖掘类逐层拆解
FP树构建与规则提取的底层调用链
在处理二元特征矩阵时,每条业务记录的特征支持度必须对齐,否则后续专有规则树的挖掘会错位。原文给出的流程里,CreatePath 先按支持度降序扫描特征,把激活态特征 id 及其支持度写进路径矩阵,路径尺寸变量从 0 起算,退出循环后再把矩阵裁到实际填充长度。 NewPath 接收树根与路径矩阵指针,先校验路径尺寸大于 0,再对根节点累加支持度,并逐节点检查或新建子节点、累加支持度。GrowsTree 则遍历源数据所有行,对每笔业务调 CreatePath 生成路径后交由 NewPath 落进 FP 树,任一步返回非正都该中断。 公开方法 CreateRules 把上述拼装起来:传入标量矩阵、目标向量、区间数、最小支持度与置信度;先校验维度与区间数大于 0,用 PrepareData 转二元,再除以样本业务数归一,调 GrowsTree 建树。买特征路径用 Mining 提取后,重置其支持度并从路径剥离,降序排专有支持度,调 CreatePositions 写入 m_BuyPositions;若剩余特征仍能做先行项就再长树,得到 m_cBuyRules 根节点。卖特征对称处理,最后清源树释放资源。 Probability 方法面向实盘调用,接收标量向量与两个 double 指针(存形态置信度),内部复用前述全套方法。外汇与贵金属波动剧烈、杠杆高风险,任何形态置信度都只是概率倾向,开 MT5 加载附件类即可逐步验证这条调用链。
class CMyTreeNode : class="kw">public CArrayObj { class="kw">protected: CMyTreeNode *m_cParent; class="type">class="kw">ulong m_iIndex; class="type">class="kw">double m_dSupport; class="kw">public: CMyTreeNode(); ~CMyTreeNode(); } CMyTreeNode::CMyTreeNode() : m_iIndex(ULONG_MAX), m_dSupport(class="num">0) { Clear(); } class CMyTreeNode : class="kw">public CArrayObj { ........ class="kw">public: ........ class=class="str">"cmt">//--- methods of access to class="kw">protected data CMyTreeNode* Parent(class="type">void) class="kw">const { class="kw">return(m_cParent); } class="type">void Parent(CMyTreeNode *node) { m_cParent = node; } class="type">void IncreaseSupport(class="type">class="kw">double support) { m_dSupport += support; } class="type">class="kw">double GetSupport(class="type">void) { class="kw">return m_dSupport; } class="type">void SetSupport(class="type">class="kw">double support) { m_dSupport = support; } class="type">class="kw">ulong ID(class="type">void) { class="kw">return m_iIndex; } class="type">void ID(class="type">class="kw">ulong ID) { m_iIndex = ID; } }; class="type">class="kw">double CMyTreeNode::GetConfidence(class="type">void) { CMyTreeNode *parent = Parent(); if(!parent) class="kw">return class="num">1; class=class="str">"cmt">//--- class="type">class="kw">double result = m_dSupport / parent.GetSupport(); class="kw">return result; } CMyTreeNode *CMyTreeNode::AddNode(class="kw">const class="type">class="kw">ulong ID, class="type">class="kw">double support = class="num">0) { CMyTreeNode *node = new CMyTreeNode(); if(!node) class="kw">return node; node.ID(ID); if(!Add(node)) { class="kw">delete node;
◍ 节点删除与关联规则挖掘的实现细节
在自定义树结构里,删除指定 ID 的节点靠 DeleteNode 遍历 m_data 数组完成。若发现空指针先尝试 Delete(i) 回收,再递归重跑,避免数组错位漏删;命中 ID 则直接 Delete(i) 返回 true,全遍历无果才返回 false。
Mining 方法承担频繁项集向下的支撑度回溯。它接收 supports 向量与 paths 矩阵,以 min_conf 做置信度门槛:当自身索引等于传入 ID 且置信度不足时直接返回 true 剪枝,否则累加子节点支撑度并在回溯时把剩余 support 写入父链。
paths 矩阵每行对应一条从根到叶的支撑度路径,代码里用 paths.Resize(row+1, paths.Cols()) 动态扩行,再 Row(vector::Zeros(...)) 置零填充。外汇与贵金属行情下用这套挖掘历史 K 线形态关联,属于高风险实验,回测结论只具概率意义。
别把空指针当异常
循环里遇到 !temp 不报错而是就地 Delete 并重跑,是这套树结构的容错习惯。若你抄代码时改成抛错,递归深度一大 MT5 容易卡死,不如保留原逻辑先在策略测试器跑通。
class="kw">return node; } node.Parent(GetPointer(this)); class=class="str">"cmt">//--- if(support > class="num">0) node.SetSupport(support); class="kw">return node; } class="type">bool CMyTreeNode::DeleteNode(class="kw">const class="type">class="kw">ulong ID) { for(class="type">int i = class="num">0; i < m_data_total; i++) { CMyTreeNode *temp = m_data[i]; if(!temp) { if(!Delete(i)) class="kw">continue; class="kw">return DeleteNode(ID); } if(temp.ID() != ID) class="kw">continue; class="kw">return Delete(i); } class=class="str">"cmt">//--- class="kw">return class="kw">false; } class="type">bool CMyTreeNode::Mining(vector &supports, matrix &paths, class="kw">const class="type">class="kw">ulong ID, class="type">class="kw">double min_conf) { if(ID == m_iIndex) if(GetConfidence() < min_conf) class="kw">return true; class="type">class="kw">double support = m_dSupport; for(class="type">int i = class="num">0; i < m_data_total; i++) { CMyTreeNode *temp = m_data[i]; if(!temp) { if(Delete(i)) i--; class="kw">continue; } if(!temp.Mining(supports, paths, (ID == m_iIndex ? ULONG_MAX : ID), min_conf)) class="kw">return class="kw">false; support -= temp.GetSupport(); if(temp.ID() == ID) if(Delete(i)) i--; } if(ID == m_iIndex || ID == ULONG_MAX) if(support > class="num">0 && !!m_cParent) { CMyTreeNode *parent = m_cParent; class="type">class="kw">ulong row = paths.Rows(); if(!paths.Resize(row + class="num">1, paths.Cols())) class="kw">return class="kw">false; if(!paths.Row(vector::Zeros(paths.Cols()), row)) class="kw">return class="kw">false; supports[m_iIndex] += support; class="kw">while(!!parent) { if(parent.ID() != ULONG_MAX) { supports[parent.ID()] += support; paths[row, parent.ID()] = support; } parent = parent.Parent(); } } class=class="str">"cmt">//--- class="kw">return true; } class CAssocRules : class="kw">public CObject {
「关联规则类的内部骨架与对外接口」
把行情离散化后做关联分析,核心载体是一个 CAssocRules 类。它在私有段维护了树根与买卖规则节点(m_cRoot / m_cBuyRules / m_cSellRules),外加分箱用的边界向量 m_vMin、m_vStep 和分段数 m_iSections,默认把特征轴切成 10 段。 持仓矩阵用了三组:m_mPositions 记录全部候选位置,m_BuyPositions 与 m_SellPositions 分别存触发买、卖规则的坐标。这种分离让你在回测时可以直接抽取某一侧的矩阵算命中率,而不用每次重跑整棵树。 私有方法里 PrepareData 的签名的参数给了明确默认值:sections=10、min_sup=0.03,意味着低于 3% 支持度的离散区间会被直接丢弃。CreateRules 作为公开入口,进一步暴露了 min_freq=0.03 与 min_prob=0.3——只有出现频率超 3% 且置信度过 30% 的规则才写进树里。 对外暴露的 Probability(vector &data, double &buy, double &sell) 是实盘调用重点:传入实时特征向量,类内部会沿树匹配路径,把买、卖后验概率通过引用回写。外汇与贵金属波动大、杠杆高,这套概率只是历史样本的条件频率,实盘信号失效概率不低,必须配合仓位控制。 文件交互给了三个虚函数 Save/Load,既能按文件句柄也能按文件名持久化规则树。训练一次跑几小时,存盘后下次 MT5 启动直接 Load,省掉重复建模的时间。
CMyTreeNode m_cRoot; CMyTreeNode m_cBuyRules; CMyTreeNode m_cSellRules; vector m_vMin; vector m_vStep; class="type">int m_iSections; matrix m_mPositions; matrix m_BuyPositions; matrix m_SellPositions; class=class="str">"cmt">//--- class="type">bool NewPath(CMyTreeNode *root, matrix &path); CMyTreeNode *CheckPath(CMyTreeNode *root, vector &path); class=class="str">"cmt">//--- class="type">bool PrepareData(matrix &data, matrix &bin_data, vector &buy, vector &sell, class="kw">const class="type">int sections = class="num">10, class="kw">const class="type">class="kw">double min_sup = class="num">0.03); matrix CreatePath(vector &bin_data, matrix &positions); matrix CreatePositions(vector &support, class="kw">const class="type">class="kw">double min_sup = class="num">0.03); class="type">bool GrowsTree(CMyTreeNode *root, matrix &bin_data, matrix &positions); class="type">class="kw">double Probability(CMyTreeNode *root, vector &data, matrix &positions); class="kw">public: CAssocRules(); ~CAssocRules(); class=class="str">"cmt">//--- class="type">bool CreateRules(matrix &data, vector &buy, vector &sell, class="type">int sections = class="num">10, class="type">class="kw">double min_freq = class="num">0.03, class="type">class="kw">double min_prob = class="num">0.3); class="type">bool Probability(vector &data, class="type">class="kw">double &buy, class="type">class="kw">double &sell); class=class="str">"cmt">//--- methods for working with files class="kw">virtual class="type">bool Save(class="kw">const class="type">int file_handle); class="kw">virtual class="type">bool Load(class="kw">const class="type">int file_handle); class="kw">virtual class="type">bool Save(class="kw">const class="type">class="kw">string file_name);
把连续行情切进离散区间的预处理
关联规则模型吃不了原始价格序列,得先把每根 K 线的多维特征压成 0/1 的二元矩阵。PrepareData 干的就是这活:默认按 10 段等分(sections=10),支持度门槛锁在 0.03,低于这个出现频率的离散状态直接丢弃。 核心计算是先求每列最小值 m_vMin 和最大值,差值除以段数得到步长 m_vStep,这里故意加了 1e-8 防止除零和边界溢出。每行数据减去最小值再除步长,就得到该特征落在第几段,Clip 到 [0, sections-1] 确保不越界。 bin_data 矩阵列数 = 特征数×段数 + 2,多出来的两列留给 buy 和 sell 标签。代码里把对应段位置置 1,再从完整矩阵里抽出买/卖标签列。support 向量用每列求和除以总行数算出来,交给 CreatePositions 按支持度从高到低冒泡排序,裁掉不满足 min_sup 的行。 在 MT5 里把 sections 从 10 调到 20,二元矩阵列数会翻倍,计算量上去但状态分得更细,过拟合风险也跟着抬升,贵金属和外汇这种高波动品种尤其要小心。
class="kw">virtual class="type">bool Load(class="kw">const class="type">class="kw">string file_name); }; class="type">bool CAssocRules::PrepareData(matrix &data, matrix &bin_data, vector &buy, vector &sell, class="kw">const class="type">int sections = class="num">10, class="kw">const class="type">class="kw">double min_sup = class="num">0.03) { class=class="str">"cmt">//--- m_iSections = sections; m_vMin = data.Min(class="num">0); vector max = data.Max(class="num">0); vector delt = max - m_vMin; m_vStep = delt / sections + class="num">1e-8; m_cBuyRules.ID(data.Cols() * m_iSections); m_cSellRules.ID(m_cBuyRules.ID() + class="num">1); bin_data = matrix::Zeros(data.Rows(), m_cSellRules.ID() + class="num">1); for(class="type">class="kw">ulong r = class="num">0; r < data.Rows(); r++) { vector pos = (data.Row(r) - m_vMin) / m_vStep; if(!pos.Clip(class="num">0, m_iSections - class="num">1)) class="kw">return class="kw">false; for(class="type">class="kw">ulong c = class="num">0; c < pos.Size(); c++) bin_data[r, c * sections + (class="type">int)pos[c]] = class="num">1; } if(!bin_data.Col(buy, m_cBuyRules.ID()) || !bin_data.Col(sell, m_cSellRules.ID())) class="kw">return class="kw">false; vector supp = bin_data.Sum(class="num">0) / bin_data.Rows(); m_mPositions = CreatePositions(supp, min_sup); class=class="str">"cmt">//--- class="kw">return true; } matrix CAssocRules::CreatePositions(vector &support, class="kw">const class="type">class="kw">double min_sup = class="num">0.03) { matrix result = matrix::Ones(support.Size(), class="num">2); result = result.CumSum(class="num">0) - class="num">1; if(!result.Col(support, class="num">1)) class="kw">return matrix::Zeros(class="num">0, class="num">0); class="type">bool change = class="kw">false; do { change = class="kw">false; class="type">class="kw">ulong total = result.Rows() - class="num">1; for(class="type">class="kw">ulong i = class="num">0; i < total; i++) { if(result[i, class="num">1] >= result[i + class="num">1, class="num">1]) class="kw">continue; if(result.SwapRows(i, i + class="num">1)) change = true; } } class="kw">while(change); class="type">int i = class="num">0; class="kw">while(result[i, class="num">1] >= min_sup) i++; if(!result.Resize(i, class="num">2))
◍ 从二值序列到关联规则树
把行情切成二值特征后,真正要算的是「哪些位置组合更常伴随买或卖」。CreatePath 干的事很直:拿一行 bin_data 和 positions 矩阵,只挑出被标记且下标合法的特征点,塞进 2×N 的 path 矩阵——第 0 行记位置索引,第 1 行记该位的值(0/1 以外也可能是权重)。 它先开一个 2×total 的零矩阵,total 是 positions 的行数;循环里遇到 pos>=size 或 bin_data[pos]==0 就跳过,有效点才写进 path 并让 vect_pos 自增。最后 Resize(2, vect_pos) 把尾部空列砍掉,若失败返回 2×0 零矩阵——这一步保证后面建树不会吃到垃圾列。 NewPath 接收 path 往树里写:从 root 起,先给根节点加 path[1,0] 的支持度,再逐个找或新建子节点,每步把对应支持度累加。GrowsTree 则对 bin_data 每一行调 CreatePath,再喂给 NewPath;任一行建路径失败就整体返回 false。 CreateRules 的入口里 sections 默认 10,意味着把连续变量分 10 段二值化。外汇与贵金属杠杆高、滑点随机,这套关联树只反映历史样本内的共现概率,实盘信号可能随 regime 切换而衰减,上 MT5 跑前先拿 EURUSD 的 M15 回测一段看看支持度分布再谈过滤。
class="kw">return matrix::Zeros(class="num">0, class="num">0); class=class="str">"cmt">//--- class="kw">return result; } matrix CAssocRules::CreatePath(vector &bin_data, matrix &positions) { class="type">class="kw">ulong size = bin_data.Size(); class=class="str">"cmt">//--- class="type">class="kw">ulong total = positions.Rows(); class="type">int vect_pos = class="num">0; matrix path = matrix::Zeros(class="num">2, total); for(class="type">class="kw">ulong c = class="num">0; c < total; c++) { class="type">class="kw">ulong pos = (class="type">class="kw">ulong)positions[c, class="num">0]; if(pos >= size) class="kw">continue; if(bin_data[pos] == class="num">0) class="kw">continue; path[class="num">0, vect_pos] = (class="type">class="kw">double)pos; path[class="num">1, vect_pos] = bin_data[pos]; vect_pos++; } if(!path.Resize(class="num">2, vect_pos)) class="kw">return matrix::Zeros(class="num">0, class="num">0); class=class="str">"cmt">//--- class="kw">return path; } class="type">bool CAssocRules::NewPath(CMyTreeNode *root, matrix &path) { class="type">class="kw">ulong total = path.Cols(); if(total <= class="num">0) class="kw">return class="kw">false; CMyTreeNode *parent = root; root.IncreaseSupport(path[class="num">1, class="num">0]); for(class="type">class="kw">ulong i = class="num">0; i < total; i++) { CMyTreeNode *temp = parent.GetNext((class="type">class="kw">ulong)path[class="num">0, i]); if(!temp) { temp = parent.AddNode((class="type">int)path[class="num">0, i], class="num">0); if(!temp) class="kw">return class="kw">false; } temp.IncreaseSupport(path[class="num">1, i]); parent = temp; } class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CAssocRules::GrowsTree(CMyTreeNode * root, matrix & bin_data, matrix &positions) { class="type">class="kw">ulong rows = bin_data.Rows(); for(class="type">class="kw">ulong r = class="num">0; r < rows; r++) { matrix path = CreatePath(bin_data.Row(r), positions); class="type">class="kw">ulong size = path.Cols(); if(size <= class="num">0) class="kw">continue; if(!NewPath(root, path)) class="kw">return class="kw">false; } class=class="str">"cmt">//--- class="kw">return true; } class="type">bool CAssocRules::CreateRules(matrix &data, vector &buy, vector &sell, class="type">int sections = class="num">10,