Eugen Soloviov
Trading-systems engineer
Trading-systems engineer building bots since 2017: cross-exchange arbitrage (connected up to 30 venues), cointegration-based pairs arbitrage across spot and futures, scalping, news and sentiment-driven strategies, trend algorithms, and portfolio management and balancing algorithms. Also builds sub-millisecond order execution, big-data warehouses, backtesting engines, AI agents, and trading interfaces (incl. open-source profitmaker.cc). Stack: JS/TS, Python, Rust/Zig/Go, DevOps, backend, frontend, architecture.
Articles
Halol salbiy natija: oʻn minglab bektestlar, beshta yetakchi koin, mustahkam ustunlik yoʻq
Qidiruv va overfitting haqidagi yoʻnalishning yakuniy maqolasi — va u salbiy natija bilan tugaydi, bu esa aynan toʻgʻri natija. ETHUSDT ustida bitta simvolli, ikki taymfreymli qidiruv out-of-sample testda +16.35% va tegilmagan holdoutda +2.62% beradigan konfiguratsiyani topdi; ~37,000 sinovni hisobga olgan Deflyatsiyalangan Sharp koeffitsienti uni 0.00 gacha deflyatsiya qildi. Beshta yetakchi koin (ETH/BTC/SOL/BNB/XRP, har biri ~1.18M 1m bar) boʻyicha, out-of-sample medianasi asosida tanlaydigan kross-instrumental sinov uni uzil-kesil yoʻq qiladi: ikki taymfreymda DSR 0.24 / PBO 0.264, uch taymfreymda DSR 0.14 / PBO 0.327 — ikkalasi ham geytlardan oʻtmaydi. Chempion 5 simvoldan 1 tasida foydali, qolganlarida salbiy. Anti-overfit apparati aynan shu uchun kerak: shovqinning eng yaxshisini alfa sifatida ishga tushirishingizga yoʻl qoʻymaslik uchun.
Ko'p vaqt oralig'idagi bektestlarda kelajakka qarash yo'qligini isbotlash: kelajakni buzib, o'tmish uni ko'ra olmasligini isbotlash
Ko'p vaqt oralig'idagi (multi-timeframe) bektestlar hali yakuniy yopilish narxi mavjud bo'lmagan, shakllanayotgan yuqori vaqt oralig'idagi bar orqali kelajakni sizib chiqaradi. Kodni ko'rib chiqish orqali ishonchga erisha olmaysiz — buni sinab ko'rish kerak. Biz jonli botning yopiq-bar qoidasini aynan takrorlaymiz, so'ngra kelajakni siljitadigan tekshiruv bilan sizib chiqishning yo'qligini isbotlaymiz: kelajakdagi har bir barni buzamiz va o'tmishdagi har bir signal hamda savdo bit-baytiga o'zgarmasligini tasdiqlaymiz. 25/25 paritet tekshiruvlari, va tekshiruvning haqiqiy tishi bor.
GPU qachon o'zini oqlaydi: parametr qidiruvi roofline'i — sarlavhadagi 167x aslida 27x algoritm karra 6.2x apparat
GPUning CPUdan ustunligi paket hajmi bilan o'sadi — ko'p taymfreymli indikatorlarni oldindan hisoblashimizda har chaqiruvga bitta kombinatsiyada 54.5x dan 61 tasida 359.6x gacha — chunki kichik qidiruv kernelni ishga tushirish va uzatish ustamasini amortizatsiya qila olmaydi. Sarlavhadagi 167x ni CPUga ham naf keltiradigan 27x algoritmik yutuq karra 6.2x apparat yutug'iga ajratamiz, GPUning eng yaxshi CPUdan haqiqiy ustunligi bitta taymfreymda atigi 3.2x, ko'p taymfreymda 6.2x ekanini ko'rsatamiz va GPU sotib olish o'zini oqlashi uchun qidiruv qanchalik keng bo'lishi kerakligi haqida qaror qo'llanmasini beramiz.
GPU aniqlik tuzogi: Apple Metal ustidagi fp32 backtest qanday qilib jimgina axlat qaytaradi
Apple'ning Metal GPU'sida float64 yoq. Vektorlashtirilgan backtestni unga soddadillik bilan kochirsangiz, jozibali prefiks-yigindi WMA fp32'dan toshib ketadi — maksimal nisbiy xato 211× — biroq u baribir ishlaydi va ishonarli korinuvchi raqamlar qaytaradi. Yechim koproq aniqlik emas; bu boshqacha formulirovka: togridan-togri oynali konvolyutsiya, fp32 uchun xavfsiz 8×10⁻⁷ va bir oqimli numba'dan 55.9× tezroq. Tuzoq, arifmetika va unga tushmaganingizni qanday isbotlash.
Aniqlik darvozasi: agar arzon proksi konfiguratsiyalarni qimmat baholash kabi saralamasa, yirikdan-nozikkacha bektest sizni tezroq aldaydi
Yirikdan-nozikkacha / ko'p-aniqlikli qidiruv (ASHA, ketma-ket ikkiga bo'lish, Hyperband) minglab konfiguratsiyalarni arzon narxda saralaydi va faqat omon qolganlarni qimmat, to'liq baholashga o'tkazadi. Bu haqiqiy tezlashuv — lekin agar past aniqlikdagi reyting yuqori aniqlikdagisiga zid kelsa, u sezilmagan holda quladi. Biz fold-reyting korrelyatsiyasini o'lchadik: bitta fold'da Spearman ρ 0.03 bo'lishi mumkin (deyarli tasodifiy saralaydi), fold'lar to'planishi bilan 0.43, 0.67, 0.78, 0.91 gacha ko'tariladi. Yechim — bitta majburiy darvoza: avval ρ(arzon, to'liq)ni o'lchang va minimal aniqlik darajasini ρ ≥ 0.5 bo'lgan birinchi pog'onagacha avtomatik ko'taring.
Tasodifiy qidiruv vs aqlli qidiruv: kesishish nuqtasi baholash narxida, algoritmda emas
Bitta backtest arzon bo'lganda, ahmoqona aralashtirilgan Sobol xom o'tkazuvchanlikda g'alaba qozonadi — aqlli samplerlar (TPE, CMA-ES, ASHA) Python ask/tell solig'ini to'laydi, bu ularni 20x sekinlashtiradi, shu sababli ular teng wall-clock vaqtida ancha kamroq nuqtani baholaydi va yutqazadi. Har bir baholashni qimmat qiling (multi-TF + walk-forward foldlar) — kesishish nuqtasi teskarilanadi. Biz ikkala rejimni ham o'lchadik, shuningdek nima uchun fold-rank fidelity (ρ@1 0.03→0.43 gacha o'sishi) pruning foyda berishining zaruriy sharti ekanligini aniqladik.
Ikki oʻqli parametr fazosi: nega sweepingizning katta qismi deyarli bepul boʻlishi kerak
Barcha parametrlarni qidirish bir xil qimmatga tushmaydi. Strategiyaning parametrlari qimmat oʻqqa (indikatorlar — butun qator boʻyicha qayta hisoblanadi) va arzon oʻqqa (qaror threshold'lari — oldindan hisoblangan signallar boʻyicha O(n) oʻtish) boʻlinadi. Indikatorlar threshold'larga bogʻliq emasligi tufayli siz ularni bir marta hisoblaysiz va minglab threshold konfiguratsiyalarini ~5,600 cfg/s tezlikda sweep qilasiz — bu har bir konfiguratsiya uchun qayta hisoblashdan taxminan 1,600 marta arzon. Oʻlchamlilik laʼnatining qayta narxlanishi.
Freymvork solig'i: qachonki backtest kutubxonangiz oddiy pandas siklidan sekinroq bo'ladi
Biz sakkizta backtest dvigatelini bitta bir xil parametr sweepida solishtirdik — 150k bar, 80 ta HMA-cross kombinatsiyasi, savdolar soni 2707 da bir xil qulflangan. Eng mashhur ikkita hodisaga asoslangan freymvork qo'lda yozilgan pandas siklidan ham sekinroq chiqdi, vektorlashtirilgan/kompilyatsiya qilingan dvigatel esa xuddi shu ishni ~13,000× tezroq bajardi. Mashhur kutubxonalar hech qachon amortizatsiya qilishga mo'ljallanmagan har bir bar uchun yuklama tadqiqoti.
The Probability of Backtest Overfitting: Did Your Search Beat a Coin Flip?
The Deflated Sharpe Ratio prices the winning strategy; PBO prices the search that picked it. Combinatorially Symmetric Cross-Validation runs C(16,8) = 12,870 train/test splits over a 1000x200 performance matrix and asks: does the in-sample winner land in the bottom half out of sample? The catch almost everyone misses — PBO's null is 0.5, not 1. On 200 zero-edge strategies the best in-sample annualized Sharpe of 1.98 collapses to 0.06 out of sample and PBO = 0.476: a coin flip, fully overfit. Plant a real edge (annualized Sharpe 2.38) and PBO drops to 0.001, the in-sample 3.73 surviving to an out-of-sample 2.34. A moving-average grid on a pure random walk has no out-of-sample skill either — PBO 0.463 averaged over 60 matrices, statistically indistinguishable from the null — and on one representative matrix the mirage is vivid: a best in-sample Sharpe of 2.33 collapses to a median out-of-sample -0.22, PBO 0.573, a 63% chance of a loss.
The IPC Tax: Put the Backtest Engine Behind a Socket and Lose 13% — Almost None of It to the Socket
We ported a numba backtest kernel line-for-line to Rust and called it across a process boundary four ways, with an equivalence gate confirming identical PnL to the last trade. Shipping the entire 1.2 MB price series through a Unix socket costs ~2 ms — about 0.1% of the job. JSON-encoding the same payload costs 1348x more than raw bytes, chatty per-combo calls re-ship the data 80 times, and a per-bar call pattern would pay 2.1 s of pure IPC on a 2.0 s job. The boundary is cheap; the tax is in how you cross it.
The Deflated Sharpe Ratio: How Many of Your Backtest 'Winners' Survive Multiple Testing?
A parameter search is a machine for manufacturing luck. On pure noise — 1,000 strategies with zero true edge — the best annual Sharpe averages 1.63 and the naive significance test flags a discovery 100% of the time. We build controlled ground truth and show that the Deflated Sharpe Ratio, the Harvey-Liu haircut, and White's Reality Check restore honesty: false discoveries drop from 1.000 to 0.001-0.057, genuine edges above the noise ceiling are kept with power ~1 — and one real trap (correlated grids) where the raw DSR over-deflates and the verdict must be read across a whole band of effective-trial estimates, not one.
Objective-Function Design: The Metric You Optimize Secretly Picks Your Strategy
To search for the 'best' strategy you must first define 'best' — and that scalar silently chooses the winner. On synthetic data with a known edge (600 seeds, T=2000, 80 thresholds), a naive per-trade Sharpe crowns a lottery: it picks a sub-5%-exposure winner in 56% of seeds and degenerates in 57% — on the starkest seed, 8 trades posting an in-sample Sharpe of 21.09 that collapses to 0.13 out of sample. The honest repair is almost dull: measure on the full timeline, which never degenerates (out-of-sample 1.71). A trade-count (conf_k) shrinkage and an exposure floor can retrofit a per-trade metric, but even fully repaired they only match full-timeline Sharpe (1.70 vs 1.71) — never beat it. Goodhart's law, in a backtest, with controlled ground truth.