Page 1
Stochastly / Working paper 02 / Synthetic methodology 1 STOCHASTLY / WORKING PAPER 02 Validation in event time Label maturity, overlapping information and chronological evaluation Mahé · 21 September 2026 Abstract A timestamp split can separate rows while leaving their information intervals connected. This working paper describes validation in terms of the time at which an observation becomes usable, the interval used to construct its label and the state of its fitted transformations. A deterministic audit of 120 synthetic events shows exactly which training labels intersect a test interval and which remain unresolved at a forward fitting boundary. An analytical calculation explains why overlapping forward returns can be highly correlated even when their underlying innovations are independent. We distinguish retrospective purged evaluation from a deployable chronological experiment, then specify the artifacts needed to reproduce each. The numerical results concern interval geometry; no forecast accuracy or trading performance is measured. A prediction record needs several clocks. A price can have an exchange timestamp, a vendor arrival timestamp and a local processing timestamp. A feature has a latest input availability time. A label has a completion time. The decision that consumes them has its own deadline. Chronological validation becomes ambiguous if these clocks are collapsed into one column named date. Consider a decision at index t with a label constructed from the following eight return increments. The label is unavailable until the last required increment arrives. A training row dated before a model-fit cutoff can therefore carry a target that would become available after that cutoff. Sorting by its decision index does not repair this mismatch. Our audit treats label support as the inclusive interval [t,t+8], a deliberately conservative convention for shared boundary observations. Where a production target uses half-open intervals, that convention must be changed explicitly and tested. Endpoint choices affect purge counts. Set the convention from the information dependencies before examining results. Figure 1. Synthetic event geometry. The test-support interval extends beyond its last decision. Purge and embargo remove different training events; the forward fit excludes immature labels.
Page 2
Stochastly / Working paper 02 / Synthetic methodology 2 1. A filtration for research records Let F_u represent the information available by operational time u. A feature used at decision t must be measurable with respect to F_t. A fitted model used at t must be a function of training inputs and labels available before its fit completes. These conditions constrain preprocessing, model selection, calibration, state updates and the final prediction function. For label i, a_i is its first relevant information time and b_i its completion time. For test labels j, remove a training event whenever their declared support intervals intersect. This is an explicit dependency rule. If a feature also consumes observations over a longer interval, record that support separately and extend the overlap test to the dependencies the evaluation is intended to exclude. The rule should follow the estimand. A retrospective comparison can allow training periods after a held-out block when its purpose is to measure interpolation under a stable process. A deployment simulation asks what a model could have learned before each prediction. It restricts training to the past, even if distant future rows have no direct label overlap. Purging alone does not turn the former experiment into the latter. Leave-future-out evaluation addresses prediction of subsequent observations using earlier information [1]. Empirical comparisons of time-series validation methods also show that the data-generating setting and evaluation design matter [2]. These references motivate declaring the task before choosing a splitting scheme. Neither reference supplies a universal embargo length for a trading pipeline. Availability metadata should be stored before feature construction. When a fundamental value is revised, its economic period and publication time differ. When a bar aggregates trades, the final close is unknown until the bar completes. A feature store that discards earlier vintages can make a later replay appear causal despite using information unavailable at the original decision. An operational audit therefore asks for the last source event used by each feature, the maturity of every fitted label and the model version active at every prediction. These can be tested mechanically. A plot with separated train and test colours is insufficient evidence if the dependency graph remains unexamined. Equations in plain text (1) b_i < t_fit. For each test event j, the intersection of [a_i,b_i] and [a_j,b_j] is empty.
Page 3
Stochastly / Working paper 02 / Synthetic methodology 3 2. An exact interval audit We construct decision indices 0 through 119 with inclusive label support [t,t+8]. The held-out decisions are 60 through 79; their combined label support extends through 87. There are 100 candidate training events outside the held-out decisions. Sixteen of them intersect test-label support: eight before the held-out decisions and eight after them. Table I. Membership audit Inclusive label support [t,t+8]; every count is derived. Quantity Count Total decisions 120 Held-out decisions 20 Candidate training events 100 Label-overlap exclusions 16 Additional embargo exclusions 4 Retrospective eligible training 80 Naive forward training at 60 60 Mature forward training at 60 52 An additional four-index embargo removes events starting at 88 through 91 after the test-support boundary. This leaves 80 events in the retrospective training set. The four-index buffer is a disclosed demonstration parameter with no empirical optimization. It has no effect on the preceding overlap count because the two exclusions are recorded separately. For a forward fit at index 60, the naive date filter t < 60 retains 60 events. Requiring completed labels strictly before 60 gives t + 8 < 60, leaving 52 events. The difference can be established without fitting any machine-learning model. It is an audit of the information available at the fitting boundary. The full membership lists are saved in results.json. Reproduction tests assert the counts and the boundary indices. This is important because a purge implementation can produce the expected total while removing the wrong side of a window. Matching membership is stronger evidence than matching counts alone. The example assumes regular, equally spaced events and a constant target horizon. Real labels may terminate at a barrier, a corporate event or a variable holding period. Then b_i varies by event, and vectorized arithmetic using a single horizon can silently under-purge long labels. The general overlap predicate still applies once the actual completion timestamps are available.
Page 4
Stochastly / Working paper 02 / Synthetic methodology 4 3. Overlapping labels reduce independent information Let independent, mean-zero increments have variance sigma squared, and define an H-step forward label as their sum. Adjacent labels share H-1 increments. Counting the shared terms gives their covariance and correlation directly. With H = 8, adjacent labels have correlation 7/8 = 0.875 despite independent underlying increments. The correlation declines linearly to zero at lag eight under this construction. This calculation isolates dependence created by label design; it makes no claim about empirical market autocorrelation. Figure 2. Analytical correlation of overlapping eight-increment labels under independent equal-variance innovations. No empirical market correlation was measured. The variance of the sample mean includes the lag covariances, weighted by how many pairs remain in a finite sample. For a long sample from this moving-sum process, the long-run variance relative to the marginal variance approaches H. The usual independent-observation standard error can therefore be too small by approximately sqrt(H). A precise finite-sample calculation should retain the pair-count weights. This information loss is relevant when reporting accuracy, mean returns or differences between model scores. Thousands of overlapping labels can represent far fewer independent episodes. Increasing the number of folds or constructing many complementary partitions does not generate new underlying increments. Block resampling, long-run variance estimators and event-cluster aggregation are possible responses, subject to their assumptions. Their parameters should reflect the data-generating dependence and the target statistic. Automatically setting a block length equal to H addresses the specified overlap mechanism in this example; additional persistence, clustering or nonstationarity can require a different design. Even a correctly estimated standard error does not remove the bias caused by selecting a model on the same evaluation scores. Dependence accounting and separation of tuning from evaluation remain distinct requirements. Their outputs should be documented separately so that a reader can identify what each check establishes. Equations in plain text (2) y_t = sum from i=1 to H of epsilon_(t+i). Corr(y_t,y_(t+k)) = 1-k/H for 0 <= k < H.
Page 5
Stochastly / Working paper 02 / Synthetic methodology 5 4. Nested fitting and stateful transformations A chronological experiment can be represented as a sequence of model states. At each outer boundary, collect the labels that have matured, fit preprocessing on permitted training rows, select hyperparameters using an inner design, freeze the resulting state and evaluate the next untouched interval. The outer score summarizes that sequence of fits. A model fitted once using information from all intervals has a different information set. Global normalization is an especially quiet source of contamination. A scaler fitted to the complete dataset incorporates future distributional information even if the downstream estimator is fitted to earlier rows. Imputation, feature selection, dimensionality reduction, probability calibration and threshold selection have the same requirement: their fit state belongs to a specific training information set. Some features are naturally stateful. A rolling estimate needs a warm-up history; an online regime model needs its filtering state. Retrospectively smoothed states can use later observations. Their numerical agreement with a plausible chart does not establish causality. Store whether an output is filtered, predicted or smoothed, and verify that the chosen variant matches its role in the decision process. A useful adversarial test changes observations strictly after a decision deadline and recomputes earlier outputs. Those outputs should remain unchanged within a declared numerical tolerance. A second test truncates the dataset at the deadline and compares the available predictions with the corresponding prefix of a full replay. Together these tests can detect accidental future reads, though they cannot establish the accuracy of external timestamps. Hyperparameter selection must also be contained inside each outer training window if the claimed procedure retunes over time. Carrying the globally best hyperparameter into the historical windows evaluates a different, informed procedure. If the research specification fixes hyperparameters before the entire evaluation, preserve that specification and its timestamp instead. For operational realism, declare refit frequency, computation delay, missing-data treatment and failure behaviour. A model that misses its decision deadline cannot be replaced after seeing the realised return. The failed prediction and the fallback actually available at that time belong in the record.
Page 6
Stochastly / Working paper 02 / Synthetic methodology 6 5. Reproducibility and limitations The accompanying script derives the interval memberships by Boolean predicates and verifies the boundary cases with assertions. The results are deterministic; no fitting, random train-test split or market-data access is involved in this paper. The forward-return correlation curve is an analytical consequence of independent increments and a fixed horizon. Its values are not measured correlations from a trading dataset. A production adaptation should add irregular timestamps, variable-duration labels, market closures, publication lags and revised inputs. It should test both exact-boundary touches and missing completion times. Unknown availability should cause an explicit exclusion or a named limitation. Assigning an arbitrary early timestamp turns incomplete provenance into an apparently usable observation. Our conservative inclusive-support convention can discard more rows than a minimal return-increment overlap rule. This is a design choice made visible in the protocol. A different convention is admissible when justified by the actual construction of features and labels; the resulting membership lists and counts should change together. The demonstration is not a universal recommendation to discard a fixed percentage of a dataset. The audit also leaves economic validation open. A temporally valid forecast may carry negligible trading value after execution costs, or it may be too unstable to support a decision. Conversely, an apparent improvement on contaminated labels cannot be rescued by a convincing economic narrative. Temporal integrity establishes the meaning of an evaluation, while effect size, uncertainty and deployability require their own evidence. For publication, retain the splitting code, event-support schema, fitted-state identifiers and held-out predictions alongside aggregate scores. A reader should be able to reconstruct why each row was admitted or excluded. This record exposes the information used to construct a temporal boundary. References [1] Buerkner, P.-C., Gabry, J. and Vehtari, A. (2020). Approximate Leave-Future-Out Cross-Validation for Bayesian Time Series Models. https://arxiv.org/abs/1902.06281 [2] Cerqueira, V., Torgo, L. and Mozetic, I. (2020). Evaluating time series forecasting models: an empirical study on performance estimation methods. https://arxiv.org/abs/1905.11744 [3] Newey, W. K. and West, K. D. (1987). A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix. https://www.nber.org/papers/t0055 Research disclosures Methodological working paper; not peer reviewed. AI-assisted drafting and computation prepared for Mahe's review. The analysis uses synthetic event indices and analytical examples. No forecast model, market backtest, investment recommendation or application-run performance is represented by these results.
