Monte-Karlo Bootstrap: bor-yo'g'i 10 qator kod bilan backtest uchun ishonch intervallarini qanday olish mumkin

📄 Ushbu maqola tadqiqot maqolasiga aylandi. Bu yerda tasvirlangan bootstrap ishonch intervallari seriyali bog'liqlik sharoitida nazorat qilinadigan qamrov (coverage) testidan o'tkazildi (iid vs trade darajasidagi vs block bootstrap, aniq ma'lum haqiqat bilan 6000 tajriba). Maqolani onlayn (interaktiv versiya + PDF) bootstrap.marketmaker.cc sahifasidan o'qing, kod va ma'lumotlar github.com/suenot/bootstrap-coverage manzilida.

Siz strategiyani backtest orqali ishga tushirdingiz. PnL +42%, Sharpe 1.8, MaxDD -12% oldingiz. Natijalar ajoyib ko'rinadi. Botni ishlab chiqarishga chiqarasiz, va bir oydan keyin drawdown allaqachon -28% ga yetganini va PnL nolga qarab intilayotganini bilib qolasiz.

Nima noto'g'ri ketdi? Bu na bug, na "bozor o'zgardi" degani. Muammo shundaki, siz qarorni bitta son asosida qabul qildingiz — bitta nuqtali baho asosida. Siz strategiya +42% ko'rsatganini bilib oldingiz, lekin shu songa qanchalik ishonish mumkinligini bilmadingiz.

Bitta nuqtali baholarning muammosi

Yagona ma'lumot nuqtasi (chapda) chalg'ituvchi tasvir beradi, to'liq taqsimot (o'ngda) esa mumkin bo'lgan natijalarning haqiqiy diapazonini ochib beradi.

Tarixiy ma'lumotlar bo'yicha backtest — bozor voqealarining ma'lum bir ketma-ketligi orqali o'tkazilgan yagona yugurishdir. Natija savdolar tartibiga bog'liq: bir xil savdolar bilan bir xil strategiya, lekin boshqacha tartibda, butunlay boshqa maksimal drawdownni ko'rsatishi mumkin.

491 ta savdoni tasavvur qiling. Har bir savdo ma'lum bir daromadlilik taqsimotiga ega tasodifiy hodisadir. Tarixiy backtest bu jarayonning faqat bitta amalga oshuvini ko'rsatadi. Bu bir marta zar tashlab, zar doim to'rtga tushadi degan xulosaga kelishga o'xshaydi.

Bizga aslida nima kerak:

• Nuqtali baho emas, balki interval: "95% ehtimollik bilan, yakuniy PnL X va Y orasida bo'ladi"
• Yagona maksimal drawdown emas, balki taqsimot: "eng yomon 5% ssenariylarda drawdown Z% dan oshadi"
• O'rtacha emas, balki quyruqlar: agar omad sizning tomoningizda bo'lmasa nima bo'ladi?

Aynan shu maqsad uchun Monte-Karlo bootstrap mavjud.

Monte-Karlo bootstrap nima

Bootstrap dastlabki ma'lumotlar to'plamidan savdolarni almashtirish bilan qayta tanlash orqali minglab muqobil equity traektoriyalarini yaratadi.

Bootstrap — 1979-yilda Bredli Efron tomonidan taklif qilingan qayta tanlash usuli. G'oya nihoyatda oddiy va nafis: agar bizda ma'lumotlar namunasi bo'lsa, biz dastlabki to'plamdan elementlarni almashtirish bilan tasodifiy tanlash orqali minglab "yangi" namunalar yaratishimiz mumkin.

Backtest kontekstida bu quyidagicha ishlaydi:

1. Sizda har bir savdo uchun daromadlar massivi bor — masalan, 491 ta qiymat
2. Siz ushbu massivdan almashtirish bilan tasodifiy ravishda 491 ta qiymat tanlaysiz — ba'zi savdolar ikki marta uchraydi, ba'zilari umuman uchramaydi
3. Ushbu yangi namunadan equity egri chizig'ini quraisiz
4. Buni 10 000 marta takrorlaysiz
5. Bitta son emas, yakuniy metrikalarning taqsimotini olasiz

Har bir iteratsiya "muqobil ssenariy": savdolar tartibi va to'plami biroz boshqacha bo'lganida nima bo'lishi mumkin edi.

10 qatorda amalga oshirish

Mana to'liq ishlaydigan implementatsiya:

import numpy as np

def max_drawdown(equity_curve):
    """Calculate the maximum drawdown of an equity curve."""
    peak = np.maximum.accumulate(equity_curve)
    drawdown = (equity_curve - peak) / peak
    return drawdown.min()

trade_returns = [...]  # 491 values, e.g. [0.012, -0.005, 0.008, ...]

n_simulations = 10000
results = []

for _ in range(n_simulations):
    sampled = np.random.choice(trade_returns, size=len(trade_returns), replace=True)
    equity = np.cumprod(1 + sampled)
    results.append({
        "final_pnl": equity[-1] - 1,
        "max_dd": max_drawdown(equity),
        "sharpe": np.mean(sampled) / np.std(sampled) * np.sqrt(252)
    })

Bajarilish vaqti: oddiy noutbukda ~2 soniya. Strategiyangizning 10 000 muqobil tarixi.

Ishonch intervallarini ajratib olish

Strategiyaning asosiy metrikalari uchun ishonch intervallari: PnL, MaxDD va Sharpe Ratio, 5-, 50- va 95-protsentil chegaralari bilan ko'rsatilgan.

Endi bizda bitta son emas, taqsimot bor. Undan foydali ma'lumotni shunday ajratib olish mumkin:

import pandas as pd

df = pd.DataFrame(results)

pnl_5 = np.percentile(df['final_pnl'], 5)
pnl_50 = np.percentile(df['final_pnl'], 50)
pnl_95 = np.percentile(df['final_pnl'], 95)

dd_5 = np.percentile(df['max_dd'], 5)    # 5th — worst case
dd_50 = np.percentile(df['max_dd'], 50)
dd_95 = np.percentile(df['max_dd'], 95)  # 95th — best case

print(f"PnL:   {pnl_5:.1%} | {pnl_50:.1%} | {pnl_95:.1%}")
print(f"MaxDD: {dd_5:.1%} | {dd_50:.1%} | {dd_95:.1%}")
print(f"Sharpe: {np.percentile(df['sharpe'], 5):.2f} — {np.percentile(df['sharpe'], 95):.2f}")

Haqiqiy strategiya uchun namuna natija:

Endi farq yaqqol ko'rinadi:

• Backtest PnL +42% ko'rsatdi — lekin eng yomon 5% ssenariylarda PnL atigi +18.3%
• Backtest MaxDD -12% ko'rsatdi — lekin eng yomon 5% ssenariylarda drawdown -23.4%
• Sharpe 1.8 — lekin quyi chegara 1.12

5-protsentil — bu sizning "realistik eng yomon holatingiz". Agar strategiya 5-protsentilda foyda keltirmasa, uni ishlab chiqarishga chiqarish xavflidir.

Vizualizatsiya: Fan Chart

Monte-Karlo bootstrap tabiiy ravishda fan chart sifatida vizualizatsiya qilinadi — equity egri chiziqlarining muxlisi:

import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 2, figsize=(16, 6))

ax = axes[0]
for i in range(min(500, n_simulations)):
    sampled = np.random.choice(trade_returns, size=len(trade_returns), replace=True)
    equity = np.cumprod(1 + sampled)
    ax.plot(equity, alpha=0.02, color='#4FC3F7')

all_equities = []
for _ in range(n_simulations):
    sampled = np.random.choice(trade_returns, size=len(trade_returns), replace=True)
    equity = np.cumprod(1 + sampled)
    all_equities.append(equity)

all_equities = np.array(all_equities)
p5 = np.percentile(all_equities, 5, axis=0)
p50 = np.percentile(all_equities, 50, axis=0)
p95 = np.percentile(all_equities, 95, axis=0)

ax.fill_between(range(len(p5)), p5, p95, alpha=0.3, color='#7C4DFF', label='90% CI')
ax.plot(p50, color='#E040FB', linewidth=2, label='Median')
ax.set_title('Monte Carlo Bootstrap: Equity Curves')
ax.legend()

ax = axes[1]
ax.hist(df['final_pnl'] * 100, bins=80, color='#4FC3F7', alpha=0.7, edgecolor='#1A237E')
ax.axvline(pnl_5 * 100, color='#FF5252', linestyle='--', label=f'5th: {pnl_5:.1%}')
ax.axvline(pnl_50 * 100, color='#E040FB', linestyle='--', label=f'Median: {pnl_50:.1%}')
ax.axvline(pnl_95 * 100, color='#69F0AE', linestyle='--', label=f'95th: {pnl_95:.1%}')
ax.set_title('Distribution of Final PnL')
ax.set_xlabel('PnL, %')
ax.legend()

plt.tight_layout()
plt.savefig('monte_carlo_fan_chart.png', dpi=150)
plt.show()

Fan chart mumkin bo'lgan natijalarning tarqoqligi haqida intuitiv tushuncha beradi. Tor muxlis strategiya barqaror ekanligini bildiradi. Keng muxlis natija savdolar tartibidagi "omad"ga qattiq bog'liqligini bildiradi.

Fan chart (chapda) mumkin bo'lgan equity traektoriyalarining tarqoqligini ko'rsatadi, gistogramma (o'ngda) esa belgilangan ishonch intervallari (5%, 50%, 95%) bilan yakuniy daromadlarning zichlik taqsimotini ko'rsatadi.

Ilg'or tahlil: xarobalik ehtimoli

Xarobalik ehtimolining vizualizatsiyasi: omon qolgan equity yo'llari (siyanrang) yuqoriga qarab egiladi, xarob bo'lgan yo'llar (qizil) esa nol-equity chegarasidan pastga tushadi.

Bootstrap muhim savolga javob berish imkonini beradi: strategiyaning kapitalning X% ini yo'qotish ehtimoli qanday?

ruin_threshold = -0.20
prob_ruin = (df['max_dd'] < ruin_threshold).mean()
print(f"P(MaxDD < -20%) = {prob_ruin:.1%}")

prob_loss = (df['final_pnl'] < 0).mean()
print(f"P(PnL < 0) = {prob_loss:.1%}")

worst_5pct = df['final_pnl'].quantile(0.05)
cvar = df[df['final_pnl'] <= worst_5pct]['final_pnl'].mean()
print(f"CVaR(5%) = {cvar:.1%}")

Bu metrikalarni bitta backtest yugurishidan olish mumkin emas. Shunga qaramay, ular strategiyani ishga tushirish haqida qaror qabul qilish uchun juda muhim.

Chuqur drawdownlar nega matematik jihatdan xavfli ekanligi va daromadlar assimetriyasi qanday ishlashi haqida ko'proq ma'lumot olish uchun bizning Loss-Profit Asymmetry maqolamizni o'qing.

Klassik bootstrap qachon ishlamaydi

Usulning bilish muhim bo'lgan cheklovlari bor.

Daromadlarning avtokorrelyatsiyasi

Klassik bootstrap savdolar mustaqil deb faraz qiladi. Aslida ko'pincha bu unday emas — strategiyada ketma-ket yutuq va yo'qotish seriyalari bo'lishi mumkin. Agar avtokorrelyatsiya sezilarli bo'lsa, block bootstrapdan foydalaning:

def block_bootstrap(returns, block_size=10, n_simulations=10000):
    """Bootstrap preserving local dependency structure."""
    n = len(returns)
    results = []

    for _ in range(n_simulations):
        starts = np.random.randint(0, n - block_size + 1, size=n // block_size + 1)
        sampled = np.concatenate([returns[s:s+block_size] for s in starts])[:n]
        equity = np.cumprod(1 + sampled)
        results.append({
            "final_pnl": equity[-1] - 1,
            "max_dd": max_drawdown(equity),
        })

    return pd.DataFrame(results)

Block bootstrap ketma-ket savdolar orasidagi mahalliy bog'liqlikni saqlaydi va MaxDD uchun yanada realistik ishonch intervallarini beradi.

Block bootstrap savdolar ketma-ketligini bloklarga bo'lish va ularni almashtirish bilan qayta tanlash orqali bloklar ichidagi avtokorrelyatsiyani saqlaydi.

Bozorning statsionar emasligi

Bootstrap dastlabki savdolar taqsimoti bilan ishlaydi. Agar bozor tarkibiy jihatdan o'zgargan bo'lsa (masalan, o'zgaruvchanlik pasaygan yoki likvidlik o'zgargan bo'lsa), tarixiy savdolar vakillik qilmasligi mumkin. Buni hisobga olish uchun:

• **Aylanma oyna (rolling window)**dan foydalaning: faqat oxirgi N ta savdo bo'yicha bootstrap qiling
• So'nggi savdolarga ko'proq og'irlik bering: vaznlangan bootstrap
• Ma'lumotlarni bozor rejimlariga bo'ling va alohida bootstrap qiling

Savdolar sonining kamligi

Bootstrap n > 30 savdo bo'lganda ishonchli. Agar sizda 10 ta savdo bo'lsa — qancha qayta tanlash qilsangiz ham yordam bermaydi. 491 ta savdo — a'lo namuna; natijalarga ishonish mumkin.

Backtest barqarorligini baholash yondashuvlarini taqqoslash

Monte-Karlo bootstrap — minimal vaqtda risklarning to'liq manzarasini beradigan yagona usul.

Nazorat ro'yxati: natijalarni talqin qilish

Monte-Karlo bootstrap natijalarini quyidagicha talqin qilishni tavsiya qilamiz:

Ishlab chiqarishga chiqaring, agar:

• 5-protsentildagi PnL musbat bo'lsa
• 5-protsentildagi MaxDD risk ishtahangizga maqbul bo'lsa
• xarobalik ehtimoli < 1% bo'lsa
• 5-protsentildagi Sharpe > 0.5 bo'lsa

Qo'shimcha ishlov talab qilinadi, agar:

• 5-protsentildagi PnL nolga yaqin bo'lsa
• 5-protsentildagi MaxDD 50-protsentildagidan sezilarli darajada yomon bo'lsa
• fan chartning tarqoqligi keng bo'lsa — strategiya beqaror

Chiqarmang, agar:

• 5-protsentildagi PnL manfiy bo'lsa
• xarobalik ehtimoli > 5% bo'lsa
• Sharpe uchun ishonch intervali 0 ni o'z ichiga olsa

marketmaker.cc dagi bizning tajribamiz

marketmaker.ccda biz o'zimizning backtest dvigatelimizni ishlab chiqamiz, va Monte-Karlo bootstrap bizning pipelinemizning ajralmas qismidir. Har bir strategiya jonli savdoga tasdiqlanishidan oldin avtomatik ravishda bootstrapdan o'tadi.

Biz bootstrapni to'g'ridan-to'g'ri backtest dvigateliga integratsiya qildik: yugurishdan so'ng siz nafaqat yakuniy PnLni, balki ishonch intervallari, fan chart, xarobalik ehtimoli va block bilan standart bootstrapni taqqoslash bo'lgan to'liq hisobotni olasiz. Bu qo'shimcha 2-3 soniya vaqt oladi — bu haqiqiy risklarni tushunish uchun arzimas narx.

Bizning tajribamizdan: bitta nuqtali baho bo'yicha jozibali ko'rinadigan strategiyalarning taxminan 30% Monte-Karlo bootstrapdan keyin saralab tashlanadi. Ularning 5-protsentildagi PnL manfiy bo'lib qoladi yoki MaxDD maqbul bo'lmagan holga keladi. Bootstrapsiz bu strategiyalar ishlab chiqarishga chiqib, katta ehtimol bilan zararga olib kelgan bo'lardi.

Xulosa

Monte-Karlo bootstrap — bu taxminan 10 qator kod va taxminan 2 soniya hisoblash. U backtestdan olingan bitta sonni ishonch intervallari bilan to'liq taqsimotga aylantiradi. Bu, ehtimol, har qanday miqdoriy tahlil vositasi orasida eng yuqori ROI hisoblanadi:

• Minimal xarajat: 30 daqiqada amalga oshirish
• Maksimal foyda: strategiyaning haqiqiy risklarini tushunish
• Bog'liqliklar yo'q: faqat NumPy

Agar siz hali bootstrapdan foydalanmayotgan bo'lsangiz — uni bugunoq pipelineingizga qo'shing. Bu backtest natijalaringizga qanchalik ishonish mumkinligini bilishning yagona yo'lidir.

References

1. Efron, B. — Bootstrap Methods: Another Look at the Jackknife (1979)
2. Davison, A.C., Hinkley, D.V. — Bootstrap Methods and their Application (Cambridge)
3. Aronson, D.R. — Evidence-Based Technical Analysis: Monte Carlo permutation
4. QuantStart — Monte Carlo Simulation for Backtest Analysis
5. Marcos Lopez de Prado — Advances in Financial Machine Learning, Chapter 12: Backtesting
6. Kevin Davey — Building Winning Algorithmic Trading Systems: Monte Carlo Analysis
7. NumPy — numpy.random.choice

Citation

@software{soloviov2026montecarlobootstrap,
  author = {Soloviov, Eugen},
  title = {Monte Carlo Bootstrap: How to Get Confidence Intervals for a Backtest in 10 Lines of Code},
  year = {2026},
  url = {https://marketmaker.cc/ru/blog/post/monte-carlo-bootstrap-backtest},
  version = {0.1.0},
  description = {Why a single-point estimate from a backtest is a dangerous illusion. How Monte Carlo bootstrap in 2 seconds of computation gives you a 95\% confidence interval for PnL and MaxDD, and why this is a mandatory step before launching a strategy in production.}
}