深度神经网络 (第五部分)。 DNN 超参数的贝叶斯优化·进阶篇
（2/3）· 默认 DNN 分类结果常令人失望，网格搜索易漏最优值，本篇进阶拆解贝叶斯调参全流程
不少交易者直接拿默认 DNN 跑分类，发现胜率卡在随机边缘就弃用神经网络。其实问题常出在超参数没调，而网格搜索穷举成本高还容易错过峰值，贝叶斯优化才是更省力的路径。
◍ R脚本里把SRBM四套结构先铺好参数边界
这段 R 代码做的是把后续要跑的 SRBM（稀疏受限玻尔兹曼机）四种组合先声明好超参搜索空间，还没进 MT5 训练，但决定了你之后网格搜索的上下限。 Fact 向量写死四种激活单元：tanhUnit、maxoutUnit、softplusUnit、sigmoidUnit，对应索引 1~4；wUpd 里 4 个权重更新规则中第 2 位用 maxoutWeightUpdate，其余三个都是 weightDecayWeightUpdate。 bonds1 到 bonds4 分别覆盖 SRBM+RP、SRBM+BP、SRBM+上层+BP、SRBM+上层+RP。隐层 n1/n2 都是 1~25 的整数区间，dropout dr1/dr2 都是 0~0.5，Lr.rbm 学习率 0.01~1.0；带 fine-tune 的 bonds2/bonds3 多一个 Lr.fine 同区间，带顶层训练的 bonds3/bonds4 多一个 Lr.top 同区间。 Bs.rbm 设 100L、Bs.nn 设 50L，意味着预训练批量 100、微调批量 50。打开你的 R 环境把这段贴进去，先确认 bonds 列表维度没被截断，再接 pretrainSRBM_topLayer 函数跑上层反向传播。外汇与贵金属行情受宏观事件驱动，模型仅揭示概率倾向，实盘仍属高风险。
RandomWeightedGuessAccurcy = rwgAccurcy, RandomWeightedGuessPrecision = rwgPrecision, RandomWeightedGuessRecall = rwgRecall, RandomWeightedGuessWeightedF1 = rwgF1))) }, env) #------------------------- #-class="num">2---------------------- evalq({ #--InitParams--------------------- Fact <- c("tanhUnit","maxoutUnit","softplusUnit", "sigmoidUnit") wUpd <- c("weightDecayWeightUpdate", "maxoutWeightUpdate", "weightDecayWeightUpdate", "weightDecayWeightUpdate") #---SRBM + RP---------------- bonds1 <- list( class="macro">#n1, n2, fact1, fact2, dr1, dr2, Lr.rbm n1 = c(1L, 25L), n2 = c(1L, 25L), fact1 = c(1L, 4L), fact2 = c(1L, 4L), dr1 = c(class="num">0, class="num">0.5), dr2 = c(class="num">0, class="num">0.5), Lr.rbm = c(class="num">0.01, class="num">1.0)#, ) #---SRBM + BP---------------- bonds2 <- list( class="macro">#n1, n2, fact1, fact2, dr1, dr2, Lr.rbm, Lr.fine n1 = c(1L, 25L), n2 = c(1L, 25L), fact1 = c(1L, 4L), fact2 = c(1L, 4L), dr1 = c(class="num">0, class="num">0.5), dr2 = c(class="num">0, class="num">0.5), Lr.rbm = c(class="num">0.01, class="num">1.0), Lr.fine = c(class="num">0.01, class="num">1.0) ) #---SRBM + upperLayer + BP---- bonds3 <- list( class="macro">#n1, n2, fact1, fact2, dr1, dr2, Lr.rbm , Lr.top, Lr.fine n1 = c(1L, 25L), n2 = c(1L, 25L), fact1 = c(1L, 4L), fact2 = c(1L, 4L), dr1 = c(class="num">0, class="num">0.5), dr2 = c(class="num">0, class="num">0.5), Lr.rbm = c(class="num">0.01, class="num">1.0), Lr.top = c(class="num">0.01, class="num">1.0), Lr.fine = c(class="num">0.01, class="num">1.0) ) #---SRBM + upperLayer + RP----- bonds4 <- list( class="macro">#n1, n2, fact1, fact2, dr1, dr2, Lr.rbm, Lr.top n1 = c(1L, 25L), n2 = c(1L, 25L), fact1 = c(1L, 4L), fact2 = c(1L, 4L), dr1 = c(class="num">0, class="num">0.5), dr2 = c(class="num">0, class="num">0.5), Lr.rbm = c(class="num">0.01, class="num">1.0), Lr.top = c(class="num">0.01, class="num">1.0) ) Bs.rbm <- 100L Bs.nn <- 50L },envir = env) # SRBM + upper Layer(backpropagation) pretrainSRBM_topLayer <- function(Ln, fact1, fact2, dr1, dr2, Lr.rbm, Lr.top) # SRBM + upper Layer(backpropagation) { darch( x = X$pretrain$x, y = X$pretrain$y, xValid = X$train$x, yValid = X$train$y, #=====常量====================================== layers = Ln, paramsList = list(),
「深度网络训练参数的实盘可调清单」
下面这段配置直接决定了 DArch 深度信念网络在 MT5 端做贵金属或外汇方向判别时的拟合行为与过拟合概率。把它塞进你的 R 训练脚本，开 MT5 对照信号延迟就能验证差异。 基础项里 shuffleTrainData = T 配合 seed = 54321，保证每次重训样本乱序但结果可复现；logLevel 设 "WARN" 只吐关键告警，避免回测日志刷屏。 RBM 预训练固定跑 30 轮（rbm.numEpochs = 30L），动量从 0.5 起、终值 0.9，ramp 长度仅 1，意味着几乎瞬间进高动量区；rbm.weightDecay = 2e-04 给权重轻微惩罚，外汇高频噪声下可能降低过拟合。 NN 微调阶段 darch.numEpochs 同为 30L，但 trainLayers = c(FALSE, FALSE, TRUE) 表示只训顶层、锁住底层 RBM 特征；backpropagation 用 bp.learnRateScale = 1，未启用 rpropagation。 权重初始化选 generateWeightsGlorotUniform，normalizeWeights = T 且边界 15，能把首层尺度压住。dropout 配置中 oneMaskPerEpoch = T 让每轮掩码不同，maxout 池大小 2、单元用 ELU（alpha=2），returnBestModel = T 自动留验证集最优。贵金属杠杆高、滑点凶，这套正则倾向控方差，但实盘仍可能失效，请小资金验证。
darch = NULL, shuffleTrainData = T, seed = class="num">54321, logLevel = "WARN", class="macro">#FATAL, ERROR, WARN, DEBUG, and TRACE. #--优化 参数---------------------------------- darch.unitFunction = c(Fact[fact1], Fact[fact2], "softmaxUnit"), darch.weightUpdateFunction = c(wUpd[fact1], wUpd[fact2], "weightDecayWeightUpdate"), rbm.learnRate = Lr.rbm, bp.learnRate = Lr.top, darch.dropout = c(class="num">0, dr1, dr2), #=== 参数 RBM ============== rbm.numEpochs = 30L, rbm.allData = T, rbm.batchSize = Bs.rbm, rbm.consecutive = F, rbm.errorFunction = mseError, class="macro">#rmseError rbm.finalMomentum = class="num">0.9, rbm.initialMomentum = class="num">0.5, rbm.momentumRampLength = class="num">1, rbm.lastLayer = -class="num">1, rbm.learnRateScale = class="num">1, rbm.numCD = 1L, rbm.unitFunction = tanhUnitRbm, rbm.updateFunction = rbmUpdate, rbm.weightDecay = class="num">2e-04, #=== 参数 NN ======================== darch.numEpochs = 30L, darch.batchSize = Bs.nn, darch.trainLayers = c(FALSE, FALSE,TRUE ), darch.fineTuneFunction = "backpropagation", class="macro">#rpropagation bp.learnRateScale = class="num">1, #class="num">0.99 #--权重----------------- generateWeightsFunction = generateWeightsGlorotUniform, # generateWeightsUniform(class="kw">default), # generateWeightsGlorotUniform, # generateWeightsHeUniform. # generateWeightsNormal, # generateWeightsGlorotNormal, # generateWeightsHeNormal, darch.weightDecay = class="num">2e-04, normalizeWeights = T, normalizeWeightsBound = class="num">15, #--参数 regularization----------- darch.dither = F, darch.dropout.dropConnect = F, darch.dropout.oneMaskPerEpoch = T, darch.maxout.poolSize = 2L, darch.maxout.unitFunction = "exponentialLinearUnit", darch.elu.alpha = class="num">2, darch.returnBestModel = T
RPROP微调阶段的参数落点
上面这段是深度网络在 RPROP 微调阶段的实际调用。注意验证集被硬性截断为前 250 行：xValid 与 yValid 都用了 head(250)，意味着调参时只拿测试集头部一小撮样本盯误差，全量测试集并不进验证循环。 权重更新走的是 iRprop+ 路线。decFact=0.5 与 incFact=1.2 控制步长收缩与扩张，initDelta 锁在 1/80、maxDelta=50、minDelta=1e-6，这套边界能把更新量压在不会爆网的区间；numEpochs=50 说明微调轮数固定，不靠早停。 dropout 配置为 c(0, dr1, dr2)，输入层不丢，隐层按传入比例丢；weightDecay=2e-04 配合 normalizeWeights=T 且边界 15，权重初始化后会被重新缩放到 [-15,15]。外汇与贵金属行情用这类模型做状态分类属高风险，过拟合概率偏高，建议直接把这段丢进 R 环境改 dr1/dr2 跑一遍看验证误差曲线。
fineTuneRP <- function(Ln, fact1, fact2, dr1, dr2, Dnn) # rpropagation { darch( x = X$train$x, y = X$train$y, class="macro">#xValid = X$test$x, yValid = X$test$y, xValid = X$test$x %>% head(class="num">250), yValid = X$test$y %>% head(class="num">250), #=====常量====================================== layers = Ln, paramsList = list(), darch = Dnn, shuffleTrainData = T, seed = class="num">54321, logLevel = "WARN", class="macro">#FATAL, ERROR, WARN, DEBUG, and TRACE. rbm.numEpochs = 0L, #--优化 参数---------------------------------- darch.unitFunction = c(Fact[fact1], Fact[fact2], "softmaxUnit"), darch.weightUpdateFunction = c(wUpd[fact1], wUpd[fact2], "weightDecayWeightUpdate"), darch.dropout = c(class="num">0, dr1, dr2), #=== 参数 NN ======================== darch.numEpochs = 50L, darch.batchSize = Bs.nn, darch.trainLayers = c(TRUE,TRUE, TRUE), darch.fineTuneFunction = "rpropagation", #"rpropagation" "backpropagation" #=== 参数 RPROP ====== rprop.decFact = class="num">0.5, rprop.incFact = class="num">1.2, rprop.initDelta = class="num">1/class="num">80, rprop.maxDelta = class="num">50, rprop.method = "iRprop+", rprop.minDelta = class="num">1e-06, #--权重----------------- darch.weightDecay = class="num">2e-04, normalizeWeights = T, normalizeWeightsBound = class="num">15, #--参数 regularization----------- darch.dither = F, darch.dropout.dropConnect = F, darch.dropout.oneMaskPerEpoch = T, darch.maxout.poolSize = 2L, darch.maxout.unitFunction = "exponentialLinearUnit", darch.elu.alpha = class="num">2, darch.returnBestModel = T class="macro">#darch.returnBestModel.validationErrorFactor = class="num">0, ) }
◍ SRBM叠加随机投影的贝叶斯寻参结果
把受限玻尔兹曼机预训练与随机投影fine-tune拼成一套DNN拟合函数后，直接用贝叶斯优化搜超参。bounds给定上下界，init_points=10做初始探测，n_iter=10轮迭代，采集函数取ucb、kappa=2.576，这种配置在样本外250根K线上跑分类预测。 第7轮迭代给出当前最优解：n1=22、n2=2、fact1=3、fact2=2、dr1=0.4114、dr2=0.4818、Lr.rbm=0.7889，F1均值0.7531。外汇与贵金属行情高波动，该分数仅代表回测窗口内的分类能力，实盘可能明显衰减。 取出历史Top10参数排个序就能看到，次优组合Value集中在0.744~0.749之间，差异极小。比如n1=23/n2=8那组Lr.rbm仅0.084，却和0.789的那组几乎同分，说明该类模型对隐含层学习率并不敏感，调参时可优先锁dr1/dr2。
pretrainSRBM_topLayer(Ln, fact1, fact2, dr1, dr2, Lr.rbm, Lr.top) -> Dnn fineTuneRP(Ln, fact1, fact2, dr1, dr2, Dnn) -> Dnn predict(Dnn, newdata = X$test$x %>% tail(class="num">250) , type = "class") -> Ypred yTest <- X$test$y[ ,class="num">1] %>% tail(class="num">250) class="macro">#numIncorrect <- sum(Ypred != yTest) class="macro">#Score <- class="num">1 - round(numIncorrect/nrow(xTest), class="num">2) Score <- Evaluate(actual = yTest, predicted = Ypred)$Metrics$F1 %>% mean() class="kw">return(list(Score = Score, Pred = Ypred) } #---SRBM + RP---------------- evalq( OPT_Res1 <- BayesianOptimization(fitnes1.DNN, bounds = bonds1, init_grid_dt = NULL, init_points = class="num">10, n_iter = class="num">10, acq = "ucb", kappa = class="num">2.576, eps = class="num">0.0, verbose = TRUE) , envir = env) evalq({ OPT_Res1 %$% History %>% dplyr::arrange(desc(Value)) %>% head(class="num">10) %> dplyr::select(-Round) -> best.init1 best.init1 }, env)
「贝叶斯寻参跑出的前排权重」
上面这段是 DNN 拟合函数在 R 环境里做贝叶斯优化的实跑记录。第一轮 OPT_Res1.1 用 ucb 采集函数、kappa=2.576、各 10 次初始与迭代，跑出的最优组合是 n1=4、n2=1、fact1=1、fact2=4、dr1=0.1870、dr2=0.0、Lr.rbm=0.9728，目标值 0.7608。 把历史按 Value 降序取前 10 行作为下一轮 best.init1，头部那组 n1=4/n2=1/fact1=1/fact2=4/dr1=0.18701522/dr2≈2.22e-16/Lr.rbm=0.9728164，Value=0.7607811，和第 2 名 0.7549180 拉开约 0.006 的差距。 前 10 行里 Lr.rbm 从 0.2156837 到 0.9728164 分布很散，但 Value 都卡在 0.743~0.761 区间，说明这个适应度面在外汇/贵金属样本上比较平，局部最优不少、全局增益有限。开 MT5 接自己的品种回测时，可以先把 n1=4/n2=1 这组丢进去验，再观察 dr2 逼零是否普遍——杠杆品种的高波动可能让 dr2 不再趋近于 0。
evalq( OPT_Res1.class="num">1 <- BayesianOptimization(fitnes1.DNN, bounds = bonds1, init_grid_dt = best.init1, init_points = class="num">10, n_iter = class="num">10, acq = "ucb", kappa = class="num">2.576, eps = class="num">0.0, verbose = TRUE) , envir = env) evalq({ OPT_Res1.class="num">1 %$% History %>% dplyr::arrange(desc(Value)) %>% head(class="num">10) %>% dplyr::select(-Round) -> best.init1 best.init1 }, env)
SRBM 叠加 BP 的寻优结果怎么读
在 SRBM 预训练之后接一层 BP 微调，贝叶斯优化跑 10 轮初始点加 10 轮迭代、acq 用 ucb 且 kappa=2.576，得到的历史排序里最高 Value 落在 0.7612619。对应的结构是 n1=23、n2=24、fact1=2、fact2=1，dr1=0.4513、dr2=0.1459，Lr.rbm=0.8990、Lr.fine=0.2326。 前 8 组里 Value 从 0.7613 降到 0.7429，跨度约 0.0184，说明这组隐层宽度与学习率组合在样本内相对占优，但差距并不算悬殊。外汇与贵金属行情具有高杠杆与跳空风险，这类数值仅代表回测窗口内的拟合倾向，实盘可能明显衰减。 直接把 OPT_Res2 的 History 按 Value 降序取 head(10) 挑出 best.init2，就能拿到可复用的初始化参数；下一轮换 bonds2 边界或把 n_iter 提到 20，观察最高 Value 是否还能往上走。
evalq( OPT_Res2 <- BayesianOptimization(fitnes2.DNN, bounds = bonds2, init_grid_dt = NULL, init_points = class="num">10, n_iter = class="num">10, acq = "ucb", kappa = class="num">2.576, eps = class="num">0.0, verbose = TRUE) , envir = env) > evalq({ + OPT_Res2 %$% History %>% dplyr::arrange(desc(Value)) %>% head(class="num">10) %>% + dplyr::select(-Round) -> best.init2 + best.init2 + }, env)