Irregular Time in Tick Models: Continuous-Time Encodings vs. Plain Positional Embeddings

Time bars are an arbitrary compression that destroys event timing, and activity should define bar boundaries instead — that argument is made in full, with 17 bar types and working generators, in Bar Types and Aggregation Methods for Algorithmic Trading. This article starts one step after it, at the part nobody solved: even once you have switched to tick, volume or imbalance bars, the model you feed them to still assumes regular spacing. An nn.TransformerEncoder with a learned positional embedding treats position 7 as "the seventh slot", whether that tick arrived 200 microseconds or 40 seconds after position 6. The inter-arrival time — the thing activity bars were built to preserve — gets thrown away again at the input layer.

So the question this article is about is narrow and testable: how should a sequence model be told when a tick actually happened, and does the sophisticated answer beat the trivial one?

Assumed background

The following are prerequisites, all already covered on this blog, and none of them are re-derived here:

• Microstructure noise and bid-ask bounce. Roll's implicit spread model derives the negative serial covariance in tick returns from first principles, in price units, with the common implementation error called out: Bid-Ask Spread Modeling with ML.
• Order book features. Book imbalance, weighted mid-price, depth ratio over levels 1-5, book pressure and spread/tick ratio are tabulated in the same article; the OBI and volume-weighted-mid derivations — including the caveat that the weighted mid is not Stoikov's microprice — are in DeepLOB: Deep Learning on Limit Order Books.
• Order flow and trade intensity. Trade imbalance, VPIN and Kyle's lambda are in the spread-modeling article; inter-order interval and Hawkes self-excitation intensity as timing features are in Digital Fingerprint: Trader Identification. The crude 1/dt intensity this draft originally proposed is a weaker version of the Hawkes formulation there.
• Intraday non-stationarity. The U-shaped volume pattern, widening spread in quiet periods, and time-of-day sin/cos encoding are covered in spread-modeling as market regime features.
• Labels. Both smoothing conventions (future-mean vs. current price, future-mean vs. previous-mean), the three-class discretization with threshold $\alpha$, and LOBFrame's warning about label sensitivity to $\alpha$ and $k$ are in the DeepLOB article.
• Class imbalance. With a dead zone the FLAT class dominates, which is why F1 rather than accuracy is the reportable metric — same article.
• Bar-fill duration. The wall-clock time a volume bar takes to fill is a usable feature for a tick model; the generators that produce it are in Bar Types and Aggregation Methods.

One thing worth stating explicitly because the original version of this draft got it wrong: a jump from 50.0% to 50.5% directional accuracy is not self-evidently profitable. LOBFrame's central point, discussed in the DeepLOB article, is that a predicted move must be large enough to clear the spread before accuracy means anything, and the honest negative result on this blog is what happens when you assume otherwise.

Baseline

Assume a DeepLOB-style CNN-Inception-LSTM as the baseline encoder; the architecture, the verified FI-2010 Setup 2 numbers (F1 83.40 / accuracy 84.47 at $k=10$), the LOBFrame caveats, and a working PyTorch reimplementation are all in DeepLOB: Deep Learning on Limit Order Books. Nothing below changes that encoder — the whole question is what gets added to its input representation.

Three Ways to Encode Irregular Time

Option A: delta_t as a plain feature

The trivial answer. Append $\Delta t_i = t_i - t_{i-1}$ (log-transformed, z-scored) as one more column in the feature vector, alongside OFI and book imbalance, and keep the standard learned positional embedding. Costs nothing, adds one input dimension, and is what most production tick models actually do.

This is the arm every fancier proposal has to beat, and it is the arm that gets omitted from papers proposing fancier alternatives.

Option B: continuous-time positional encoding

Replace the discrete positional embedding with a sinusoidal function of elapsed time, over learnable timescales:

$\text{TE}(\Delta t) = \left[\sin\left(\frac{\Delta t}{\tau_1}\right), \cos\left(\frac{\Delta t}{\tau_1}\right), \ldots, \sin\left(\frac{\Delta t}{\tau_d}\right), \cos\left(\frac{\Delta t}{\tau_d}\right)\right]$

where $\tau_1, \ldots, \tau_d$ are learnable timescale parameters initialized log-spaced across microseconds to seconds. The intended effect is that attention can weight events by temporal relevance rather than by slot index — a large trade 200 microseconds ago should be reachable differently than a small trade 50 milliseconds ago.

The learnable part is the interesting part. If you initialize timescales log-spaced from 1 microsecond to 10 seconds and train, where the timescales end up is itself a measurement: if they collapse toward the millisecond end, the model is telling you that everything beyond a few milliseconds is interchangeable, and that is a finding about the market, not about the architecture.

import numpy as np
import torch
import torch.nn as nn

class ContinuousTimeEncoding(nn.Module):
    """Sinusoidal encoding for irregular inter-arrival times."""
    def __init__(self, d_model: int, num_timescales: int = 64):
        super().__init__()
        log_timescales = torch.linspace(
            np.log(1e-6), np.log(10.0), num_timescales
        )
        self.log_timescales = nn.Parameter(log_timescales)
        self.proj = nn.Linear(num_timescales * 2, d_model)

    def forward(self, delta_t: torch.Tensor) -> torch.Tensor:
        """
        Args:
            delta_t: (batch, seq_len) inter-arrival times in seconds
        Returns:
            (batch, seq_len, d_model) time encoding
        """
        timescales = torch.exp(self.log_timescales)  # (num_timescales,)
        scaled = delta_t.unsqueeze(-1) / timescales.unsqueeze(0).unsqueeze(0)
        encoding = torch.cat([torch.sin(scaled), torch.cos(scaled)], dim=-1)
        return self.proj(encoding)

Dropped into a standard encoder, it replaces exactly one line — the positional embedding add:

class TickTransformer(nn.Module):
    def __init__(self, input_dim: int, d_model: int = 128,
                 nhead: int = 8, num_layers: int = 4, num_classes: int = 3):
        super().__init__()
        self.feature_proj = nn.Linear(input_dim, d_model)
        self.time_encoding = ContinuousTimeEncoding(d_model)
        encoder_layer = nn.TransformerEncoderLayer(
            d_model=d_model, nhead=nhead,
            dim_feedforward=d_model * 4,
            dropout=0.1, batch_first=True
        )
        self.encoder = nn.TransformerEncoder(
            encoder_layer, num_layers=num_layers
        )
        self.head = nn.Linear(d_model, num_classes)

    def forward(self, features: torch.Tensor,
                delta_t: torch.Tensor) -> torch.Tensor:
        h = self.feature_proj(features) + self.time_encoding(delta_t)
        h = self.encoder(h)
        return self.head(h[:, -1, :])

The encoder boilerplate itself is nothing new — the same nn.TransformerEncoderLayer scaffolding is already published as Architecture 2 in spread modeling. Only ContinuousTimeEncoding and the learnable-timescale argument are.

Option C: continuous latent state between ticks (ODE-RNN)

The most principled option treats the latent state as a continuous-time process modeled by a Neural ODE (Chen et al., 2018). Between ticks the hidden state evolves according to a learned differential equation:

$\frac{d\mathbf{h}}{dt} = f_\theta(\mathbf{h}(t), t)$

When a tick arrives, the state is updated with the observation:

$\mathbf{h}(t_i^+) = \text{Update}(\mathbf{h}(t_i^-), \mathbf{x}_i)$

This is the ODE-RNN framework. It handles irregular spacing structurally rather than as an input feature: the solver integrates for exactly $\Delta t_i$ between events, with no padding, no interpolation, and no resampling step to lose information at.

The cost is computational. ODE solvers are sequential and hard to parallelize, which makes this the arm least likely to survive a latency budget — see the latency discipline established in The IPC Tax and the backtest engine speed ladder for how such claims should be measured before being made. It is included here as the upper bound on what explicit time modeling can buy, not as a deployment candidate.

Event-Weighted Attention

A second, orthogonal idea: not all ticks are equally informative. A 100-share trade at the bid is routine. A trade that sweeps three price levels is a regime change. We can inject that prior into attention directly. Define an event importance score:

$w_i = \text{softplus}\left(\alpha_1 \cdot \log(q_i) + \alpha_2 \cdot |\Delta p_i| / \sigma + \alpha_3 \cdot \mathbb{1}[\text{sweep}_i]\right)$

where $q_i$ is trade size, $\Delta p_i$ the price change, $\sigma$ recent volatility, and $\text{sweep}_i$ flags a multi-level sweep. Add it to the attention logits before softmax:

$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}} + \mathbf{W}\right) V$

with $\mathbf{W}_{ij} = w_j$. The model keeps the ability to learn arbitrary attention patterns but starts biased toward trades that moved the market.

This is an invented formula. The functional form, the choice of three terms, and the softplus are all guesses, and nothing here shows the bias helps rather than just burning capacity.

Multi-Horizon Heads

Different horizons inform different decisions, a shared encoder with multiple horizon outputs beats separate models per horizon, and the joint loss regularizes — that argument, plus quantile outputs and interpretability, is made in Temporal Fusion Transformer for Trading. The only tick-specific part is the horizon set: 1, 10, 50 and 100 events rather than days, which means the horizons overlap heavily in wall-clock time during bursts and barely at all during quiet periods — a complication the daily-horizon literature never has to handle.

Purging Has to Be Measured in Events

Purged walk-forward validation is covered end to end in Walk-Forward Optimization (anchored, rolling, combinatorial purged CV, WFER, degradation rate), and a working purged_walk_forward() with an explicit purge and embargo gap ships in spread modeling. The look-ahead bias taxonomy quantifies what leaks of this kind do to reported Sharpe.

The one thing that is specific to tick data: adjacent ticks milliseconds apart are near-duplicates, so a purge gap sized in days is meaningless when a burst puts 500 near-identical samples inside one second. The gap has to be sized in events, and the right size is an empirical question about your instrument's burst structure, not a constant to copy from a paper.

That claim is cheap to test — sweep the purge gap in events and plot validation F1 against it. If validation F1 falls as the gap widens and then flattens, the flattening point is your gap; if it never falls, adjacent-tick leakage was not the problem you thought it was.

The Experiment That Decides This

Everything above is architecture. None of it is evidence. The article does not clear this blog's bar until the following runs:

Setup. Fix one dataset (BTC/USDT trades is the house dataset), one encoder, one label definition, and one purged split. Vary exactly one thing.

Three arms.

What to report.

1. F1 per class, not accuracy — under a dead-zone label the FLAT class dominates, and accuracy is uninformative for exactly the reason the DeepLOB article gives.
2. A confidence interval, from repeated runs with different seeds. A 0.4-point F1 gap between arms means nothing without one.
3. The fitted timescales. Print torch.exp(model.time_encoding.log_timescales) after training. Where they settle is the most interesting number in the experiment, and it costs one line to obtain.
4. Hardware and wall-clock cost per arm, in the style of The IPC Tax — measured, on disclosed hardware, median over N. If arm B costs 3x arm A's inference time for a fraction of an F1 point, that is the answer.

Report the negative result if there is one. "Continuous-time encoding buys nothing over feeding $\Delta t$ as a feature, on 30 days of BTC ticks" is a more useful post than a survey of three architectures nobody measured. It would also be consistent with the general finding on this blog that carefully constructed edges tend to evaporate under honest validation.

Where This Stands

The uncovered question is real: sequence models fed irregularly spaced events still encode position rather than time, and the blog's existing coverage — including the Transformer encoder in spread modeling — uses a plain learned positional embedding. Continuous-time encodings, ODE-RNN latent state, and event-weighted attention are three ways to fix that.

What is missing is any evidence that fixing it matters. All three are currently topics, not findings. The deciding measurement is a three-arm ablation on a fixed encoder and a fixed purged split, reporting per-class F1 with a confidence interval and the fitted timescales — and it has not been run.