在交易中应用 OLAP(第 1 部分):在线分析多维数据·进阶篇
(2/3)·从记录与适配器到聚合控制类,把交易报告的任意切面交给代码自动切分
◍ 把数据适配器和聚合器接起来
在 MT5 里做交易记录的多维统计,核心不是算数,而是先把「谁提供数据」和「谁算统计」解耦。Analyst 类充当控制中心,它不在内部 new 适配器,而是靠构造函数把现成的 DataAdapter 注入进来——这就是依赖注入,换数据源不用动 Analyst 一行代码。 Analyst 只负责调适配器的 getNext 把每条 Record 塞进自己的动态数组,真正的聚合交给单独的聚合器。聚合器的底座是 MetaCube,用一维 double 数组 totals 模拟超立方体。比如按星期(0~6)看利润,dimensions 只有 1 个元素值 7;再加操作类型选择器(2 类),dimensions 变 [7,2],totals 长度就是 14。MQL5 不支持全动态多维数组,所以只能靠 offsets 算偏移把多维坐标压成一维下标。 mixIndex 把选择器返回的索引数组 k[] 转成 totals 里的序号,偏移量在 setSelectorBounds 里由各选择器范围顺序相乘得到。calculate 方法轮询记录,每条记录过一遍所有选择器,全映射成功才调 update 写值——基类里 update 是纯虚的,派生类填自己的聚合逻辑。 Analyst 还要挂一个 Display 接口,构造时传引用,acquireData 里顺手调聚合器 setSelectorBounds。最小可用链路是:建 HistoryDataAdapter → 建选择器数组 → 建具体聚合器(SumAggregator 等) → 建 LogDisplay → 用适配器+聚合器+显示建 Analyst → 依次 acquireData、聚合器 calculate、display 输出,最后删对象。外汇/贵金属历史数据统计仅反映过去,样本偏差可能导致结论失真,属高风险分析。
<span class="keyword">class="kw">template</span><<span class="keyword">class="kw">typename</span> E> <span class="keyword">class </span>Analyst { <span class="keyword">class="kw">private</span>: DataAdapter *adapter; Record *data[]; <span class="keyword">class="kw">public</span>: Analyst(DataAdapter &a): adapter(&a) { <span class="functions">ArrayResize</span>(data, adapter.reservedSize()); } ~Analyst() { <span class="keyword">class="type">int</span> n = <span class="functions">ArraySize</span>(data); <span class="keyword">for</span>(<span class="keyword">class="type">int</span> i = class="num">0; i < n; i++) { <span class="keyword">if</span>(<span class="functions">CheckPointer</span>(data[i]) == <span class="macro">POINTER_DYNAMIC</span>) <span class="keyword">class="kw">delete</span> data[i]; } } <span class="keyword">class="type">void</span> acquireData() { Record *record; <span class="keyword">class="type">int</span> i = class="num">0; <span class="keyword">class="kw">while</span>((record = adapter.getNext()) != <span class="keyword"><span class="macro">NULL</span></span>) { data[i++] = record; } <span class="functions">ArrayResize</span>(data, i); } }; <span class="keyword">class </span>MetaCube { <span class="keyword">class="kw">protected</span>: <span class="keyword">class="type">int</span> dimensions[]; <span class="keyword">class="type">int</span> offsets[]; <span class="keyword">class="type">class="kw">double</span> totals[]; <span class="keyword">class="type">class="kw">string</span> _typename; <span class="keyword">class="kw">public</span>: <span class="keyword">class="type">int</span> getDimension() <span class="keyword">class="kw">const</span> { <span class="keyword">class="kw">return</span> <span class="functions">ArraySize</span>(dimensions); } <span class="keyword">class="type">int</span> getDimensionRange(<span class="keyword">class="kw">const</span> <span class="keyword">class="type">int</span> n) <span class="keyword">class="kw">const</span> { <span class="keyword">class="kw">return</span> dimensions[n]; } <span class="keyword">class="type">int</span> getCubeSize() <span class="keyword">class="kw">const</span> { <span class="keyword">class="kw">return</span> <span class="functions">ArraySize</span>(totals); } <span class="keyword">class="kw">virtual</span> <span class="keyword">class="type">class="kw">double</span> getValue(<span class="keyword">class="kw">const</span> <span class="keyword">class="type">int</span> &indices[]) <span class="keyword">class="kw">const</span> = class="num">0; }; <span class="keyword">class="kw">template</span><<span class="keyword">class="kw">typename</span> E> <span class="keyword">class </span>Aggregator: <span class="keyword">class="kw">public</span> MetaCube { <span class="keyword">class="kw">protected</span>: <span class="keyword">class="kw">const</span> E field; <span class="keyword">class="kw">const</span> <span class="keyword">class="type">int</span> selectorCount; <span class="keyword">class="kw">const</span> Selector<E> *selectors[]; <span class="keyword">class="kw">public</span>: Aggregator(<span class="keyword">class="kw">const</span> E f, <span class="keyword">class="kw">const</span> Selector<E> *&s[]): field(f), selectorCount(<span class="functions">ArraySize</span>(s)) { <span class="functions">ArrayResize</span>(selectors, selectorCount); <span class="keyword">for</span>(<span class="keyword">class="type">int</span> i = class="num">0; i < selectorCount; i++) { selectors[i] = s[i]; } _typename = <span class="keyword">class="kw">typename</span>(<span class="keyword">this</span>); } <span class="keyword">class="type">int</span> mixIndex(<span class="keyword">class="kw">const</span> <span class="keyword">class="type">int</span> &k[]) <span class="keyword">class="kw">const</span> { <span class="keyword">class="type">int</span> result = class="num">0; <span class="keyword">for</span>(<span class="keyword">class="type">int</span> i = class="num">0; i < selectorCount; i++) {
「多维聚合的索引混排与输出遍历」
这段实现把多个选择器的维度压成一个一维数组下标,核心在 setSelectorBounds 里算 offsets:offsets[0]=1,后续每项等于前一项维度乘以上一偏移,注释里写的 1, X, Y*X 就是三维情形下的步长。mixIndex 用 k[i]*offsets[i] 累加得到线性位置,这样 N 个选择器的组合空间只需一个 totals 数组承载,省掉嵌套循环。 calculate 里对每条记录先跑选择器,任一不满足就 break 跳过;全部满足才调用 update(mixIndex(k), value)。这层逻辑让外汇或贵金属历史 Tick 的多条件统计(如某时段+某波动区间的成交量)能在单次遍历完成,MT5 回测时 CPU 占用倾向更低。 LogDisplay 的 display 用 cursors 数组做进位式遍历,从末维归零向前传播,等价于多维 for 嵌套的展开。它的循环终止条件同时检查 looping 与 IsStopped,避免脚本在终端手动停止时卡死——贵金属行情跳空时跑长历史容易触发人工中断,这点必须保留。 SumAggregator 只覆写 update 做累加,证明聚合行为可插拔:把 totals[index] += value 换成比较或平均,就能改出别的指标,不用动框架主干。
result += k[i] * offsets[i]; } class="kw">return result; } class="kw">virtual class="type">void setSelectorBounds() { ArrayResize(dimensions, selectorCount); class="type">int total = class="num">1; for(class="type">int i = class="num">0; i < selectorCount; i++) { dimensions[i] = selectors[i].getRange(); total *= dimensions[i]; } ArrayResize(totals, total); ArrayInitialize(totals, class="num">0); ArrayResize(offsets, selectorCount); offsets[class="num">0] = class="num">1; for(class="type">int i = class="num">1; i < selectorCount; i++) { offsets[i] = dimensions[i - class="num">1] * offsets[i - class="num">1]; class=class="str">"cmt">// class="num">1, X, Y*X } } class=class="str">"cmt">// 构建一个维数等于选择器数量的数组 class="kw">virtual class="type">void calculate(class="kw">const Record *&data[]) { class="type">int k[]; ArrayResize(k, selectorCount); class="type">int n = ArraySize(data); for(class="type">int i = class="num">0; i < n; i++) { class="type">int j = class="num">0; for(j = class="num">0; j < selectorCount; j++) { class="type">int d; if(!selectors[j].select(data[i], d)) class=class="str">"cmt">// 记录是否满足选择器? { class="kw">break; class=class="str">"cmt">// 若非,则跳过它 } k[j] = d; } if(j == selectorCount) { update(mixIndex(k), data[i].get(field)); } } } class="kw">virtual class="type">void update(class="kw">const class="type">int index, class="kw">const class="type">class="kw">float value) = class="num">0; class="type">class="kw">double getValue(class="kw">const class="type">int &indices[]) class="kw">const { class="kw">return totals[mixIndex(indices)]; } }; class="kw">template<class="kw">typename E> class Analyst { class="kw">private: DataAdapter *adapter; Record *data[]; Aggregator<E> *aggregator; class="kw">public: Analyst(DataAdapter &a, Aggregator<E> &g): adapter(&a), aggregator(&g) { ArrayResize(data, adapter.reservedSize()); } class="type">void acquireData() { Record *record; class="type">int i = class="num">0; class="kw">while((record = adapter.getNext()) != NULL) { data[i++] = record; } ArrayResize(data, i); aggregator.setSelectorBounds(i); } class="type">void build() { aggregator.calculate(data); } }; class="kw">template<class="kw">typename E> class Analyst { class="kw">private: ... Display *output; class="kw">public: Analyst(DataAdapter &a, Aggregator<E> &g, Display &d): adapter(&a), aggregator(&g), output(&d) { ... } class="type">void display() { output.display(aggregator); } }; class Display { class="kw">public: class="kw">virtual class="type">void display(MetaCube *metaData) = class="num">0; }; class LogDisplay: class="kw">public Display { class="kw">public: class="kw">virtual class="type">void display(MetaCube *metaData) class="kw">override { class="type">int n = metaData.getDimension(); class="type">int indices[], cursors[]; ArrayResize(indices, n); ArrayResize(cursors, n); ArrayInitialize(cursors, class="num">0); for(class="type">int i = class="num">0; i < n; i++) { indices[i] = metaData.getDimensionRange(i); } class="type">bool looping = class="kw">false; class="type">int count = class="num">0; do { ArrayPrint(cursors); Print(metaData.getValue(cursors)); for(class="type">int i = class="num">0; i < n; i++) { if(cursors[i] < indices[i] - class="num">1) { looping = true; cursors[i]++; class="kw">break; } else { cursors[i] = class="num">0; } looping = class="kw">false; } } class="kw">while(looping && !IsStopped()); } }; class="kw">template<class="kw">typename E> class SumAggregator: class="kw">public Aggregator<E> { class="kw">public: SumAggregator(class="kw">const E f, class="kw">const Selector<E> *&s[]): Aggregator(f, s) { _typename = class="kw">typename(this); } class="kw">virtual class="type">void update(class="kw">const class="type">int index, class="kw">const class="type">class="kw">float value) class="kw">override { totals[index] += value; } }; class="kw">template<class="kw">typename E>
用加权累加实现滚动均值聚合
在 MQL5 里做多选择器聚合时,普通求和会掩盖样本分布。上面这段 AverageAggregator 继承 Aggregator<E>,核心是用 counters[] 记录每个选择器桶的命中次数,再按 (原均值×旧计数+新值)/(旧计数+1) 做在线均值更新,避免每根 K 线重算全部历史。 setSelectorBounds 里先调父类铺好 totals 尺寸,再把 counters 同步 Resize 并清零;这一步若漏掉,update 里的 counters[index] 会是野值。 实际在 MT5 跑时,把 analyst.acquireData()、build()、display() 三连放在 OnCalculate 末尾,能在日志看到每个选择器对应的滚动均值随 tick 渐进收敛。外汇与贵金属波动大,这种在线统计对实时特征提取有帮助,但信号滞后倾向随窗口拉长而增大,属高风险环境下的概率性参考。
class AverageAggregator: class="kw">public Aggregator<E> { class="kw">protected: class="type">int counters[]; class="kw">public: AverageAggregator(class="kw">const E f, class="kw">const Selector<E> *&s[]): Aggregator(f, s) { _typename = class="kw">typename(this); } class="kw">virtual class="type">void setSelectorBounds() class="kw">override { Aggregator<E>::setSelectorBounds(); ArrayResize(counters, ArraySize(totals)); ArrayInitialize(counters, class="num">0); } class="kw">virtual class="type">void update(class="kw">const class="type">int index, class="kw">const class="type">class="kw">float value) class="kw">override { totals[index] = (totals[index] * counters[index] + value) / (counters[index] + class="num">1); counters[index]++; } }; analyst.acquireData(); analyst.build(); analyst.display();
◍ 动态范围怎么塞进超立方体
前面那些选择器都假设范围是写死的:一周 7 天、订单非买即卖。真到 MT5 实战里,你盯的品种列表和 EA 的 magic number 往往跑之前都不知道有几个,这时候就得靠动态词汇表把未知范围收拢成索引。 做法不复杂:先建一个 Vocabulary 类,内部用 index[] 数组存唯一值。get() 查重,命中就返回下标;没命中返回 -(数组大小+1),顺便告诉调用方“下次 add 该填哪儿”。这样 SymbolSelector 只存词汇表索引,不存字符串本身,超立方体维度直接对齐到整数区间。 词汇表在 TradeRecord 里声明成 static,整段历史共享一份。比如你同时跑 5 个黄金 EA、3 个欧美 EA,magic 动态进来,词汇表自动长到 8,MagicSelector 的范围就跟着变成 0~7,不用改一行核心逻辑。 SerialNumberSelector 是另一个极端:它的范围等于记录总数,专门沿 X 轴把每条记录“透明”铺开。配 IdentityAggregator 时超立方体恒为 2 维——X 是序号,Y 是每个字段选择器拉出来的列。想看原始开仓价、swap 这种没专选器的字段,用通用 TradeSelector 接 IdentityAggregator 就能直接读。 换成 ProgressiveTotalAggregator,X 轴变成累加和。利润字段喂进去就是余额曲线;Y 轴挂 SymbolSelector 得 N 条分品种曲线,再叠 MagicSelector 到 Z 轴就是 N*M 条组合曲线。外汇和贵金属杠杆高,这种切片只描述历史分布,不代表未来盈亏倾向。 getValue() 取单值,PairArray 能按维度把“数值+名称”成对吐出来,仅一维超立方体支持按值或按标签升降序排。多维排序原文没实现,留给你自己接。
class="kw">template<class="kw">typename T> class Vocabulary { class="kw">protected: T index[]; class="kw">public: class="type">int get(class="kw">const T &text) class="kw">const { class="type">int n = ArraySize(index); for(class="type">int i = class="num">0; i < n; i++) { if(index[i] == text) class="kw">return i; } class="kw">return -(n + class="num">1); } class="type">int add(class="kw">const T text) { class="type">int n = get(text); if(n < class="num">0) { n = -n; ArrayResize(index, n); index[n - class="num">1] = text; class="kw">return n - class="num">1; } class="kw">return n; } class="type">int size() class="kw">const { class="kw">return ArraySize(index); } T class="kw">operator[](class="kw">const class="type">int slot) class="kw">const { class="kw">return index[slot]; } }; class TradeRecord: class="kw">public Record { class="kw">private: ... class="kw">static Vocabulary<class="type">class="kw">string> symbols; class="kw">protected: class="type">void fillByOrder(class="kw">const class="type">class="kw">double balance) { ... set(FIELD_SYMBOL, symbols.add(OrderSymbol())); class=class="str">"cmt">// 保存品种作为词汇表的索引 } class="kw">public: class="kw">static class="type">int getSymbolCount() { class="kw">return symbols.size(); } class="kw">static class="type">class="kw">string getSymbol(class="kw">const class="type">int index) { class="kw">return symbols[index]; } class="kw">static class="type">int getSymbolIndex(class="kw">const class="type">class="kw">string s) { class="kw">return symbols.get(s); } class SymbolSelector: class="kw">public TradeSelector { class="kw">public: SymbolSelector(): TradeSelector(FIELD_SYMBOL) { _typename = class="kw">typename(this); } class="kw">virtual class="type">bool select(class="kw">const Record *r, class="type">int &index) class="kw">const class="kw">override { index = (class="type">int)r.get(selector); class="kw">return (index >= class="num">0); }
「把交易记录塞进向量维度」
在自定义分析类里,把持仓品种数量直接映射成一维向量的长度,是最省事的做法。下面这段覆写让 getRange 返回 TradeRecord::getSymbolCount(),也就是当前账户里出现过的交易品种数;getLabel 则按索引吐出具体品种名,方便后续按品种做分组统计。 [CODE] virtual int getRange() const override { return TradeRecord::getSymbolCount(); // 返回交易记录中的品种总数,作为向量维度 } virtual string getLabel(const int index) const override { return TradeRecord::getSymbol(index); // 按索引返回对应品种名,作为该维标签 } }; [/CODE] 取数之后要用 getVector 把某一维拉成数值数组,原型里带 consts[] 过滤条件和 PairArray* 引用返回。排序靠 SORT_BY 枚举控制,但注意它只适用于一维数值集合,多维或字符串标签堆里硬用会拿不到预期顺序。 [CODE] virtual bool getVector(const int dimension, const int &consts[], PairArray *&result, const SORT_BY sortby = SORT_BY_NONE) enum SORT_BY // 仅适用于一维数值集合 { SORT_BY_NONE, // 无 SORT_BY_VALUE_ASCENDING, // 数值 (升序) SORT_BY_VALUE_DESCENDING, // 数值 (降序) SORT_BY_LABEL_ASCENDING, // 标签 (升序) SORT_BY_LABEL_DESCENDING // 标签 (降序) }; [/CODE] 开 MT5 接上自己账户的成交历史跑一遍,若品种数超过 30 个,getRange 返回的维度会直接撑开向量规模,这时候排序枚举选错可能让后续相关性计算偏向少数活跃品种。外汇与贵金属杠杆高,历史分布不等于未来,验证时先把点差和滑点成本算进 consts 过滤再谈结论。
class="kw">virtual class="type">int getRange() class="kw">const class="kw">override { class="kw">return TradeRecord::getSymbolCount(); } class="kw">virtual class="type">class="kw">string getLabel(class="kw">const class="type">int index) class="kw">const class="kw">override { class="kw">return TradeRecord::getSymbol(index); } }; class="kw">virtual class="type">bool getVector(class="kw">const class="type">int dimension, class="kw">const class="type">int &consts[], PairArray *&result, class="kw">const SORT_BY sortby = SORT_BY_NONE) enum SORT_BY class=class="str">"cmt">// 仅适用于一维数值集合 { SORT_BY_NONE, class=class="str">"cmt">// 无 SORT_BY_VALUE_ASCENDING, class=class="str">"cmt">// 数值 (升序) SORT_BY_VALUE_DESCENDING, class=class="str">"cmt">// 数值 (降序) SORT_BY_LABEL_ASCENDING, class=class="str">"cmt">// 标签 (升序) SORT_BY_LABEL_DESCENDING class=class="str">"cmt">// 标签 (降序) };