# Known limits of the research

Each entry names the perimeter it touches, the direction of its effect, and the control available in the build today.

## An entry carries a mechanism, not an apology

An entry earns its place by naming three things: the quantity or the decision it touches, the direction in which it moves a result, and what the build offers against it today. A limit stated without those is an impression, and it stays out.

Each entry states a condition, and that condition is what makes a number produced by this engine interpretable. A reader who skips the entry reads the number wider than it was measured.

## What the file contains, and what it leaves out

### Survivorship

Scope: equity universes built from a currently-listed index
Effect: may overstate performance, since delisted names are absent from the file
Control: load a universe file that carries delisted tickers

### Point-in-time on prices

Scope: price series; alternative series are covered by the next entry
Effect: a revised price can flatter a rule fitted on the revised value
Control: alternative series can carry an explicit vintage field

### Point-in-time on alternative series

Scope: macro and alternative series whose publication date is missing from the source
Effect: an observation can be read earlier than it was published, which flatters a rule fitted on it
Control: the app states that it cannot promise point-in-time for that series, and one alignment stage controls the date shift

### A bar column read as an external series

Scope: alternative-data nodes wired onto an OHLCV file
Effect: a node could bind to CLOSE or VOLUME as if it were an external series, and 0 checks raised
Control: the loader keeps observation dates and knowledge dates in separate fields

### A missing column in an engine kernel

Scope: a V9 kernel reading a column the bar file does not carry
Effect: the run proceeds on a fabricated constant, so the figure is not a measurement of your data
Control: the result carries a data warning that marks the figure as not measured

## What is assumed about getting filled

### Capacity and market impact

Scope: sizing and fills on any run
Effect: fills are assumed executable at the size you set
Control: cost nodes apply the spread, commission and slippage you declare

### Undeclared costs

Scope: a graph wiring no cost node
Effect: the run is gross, and gross overstates what the account would have kept
Control: the gross label travels with the run into the verdict and the exported paper

## What the gate proves, and where it stops

### Boundary margin of the causality gate

Scope: backends holding a position across the truncation cut
Effect: a dependency confined to the boundary window passes the production gate — a backend reading one bar ahead is measured at exactly zero disagreement
Control: a second validation replays the same test with the margin set to zero

### What the causality validation covers

Scope: the shipped kernels, on synthetic bars, in memory
Effect: it establishes the causality of a kernel, and covers 0 properties of a real data set
Control: the validation states its perimeter and reports not checked when the engine is absent

## What the corrections settle, and what stays open

### Multiplicity filters, and stops there

Scope: any run scoring more than one configuration
Effect: a configuration surviving the correction is not thereby deployable
Control: the multiplicity result states this scope; deployment is judged separately

### Too few independent observations

Scope: runs with fewer than 30 effective observations, fewer than 30 common bars, or fewer than 2 finished configurations
Effect: a number would be produced from a sample that cannot support it
Control: the result is returned as insufficient with the figure that failed

### A ceiling on what is corrected

Scope: runs above 400 configurations, and validation matrices above 3000 bars
Effect: beyond the ceiling the correction runs on a sample, and the matrix keeps the most recent bars
Control: the sampling is declared in the report rather than truncated in silence

### False-discovery control assumes a dependence structure

Scope: the Benjamini-Hochberg procedure over correlated configurations
Effect: two configurations long and short on the same signal can be negatively correlated, which BH does not cover
Control: BY is emitted beside BH so the gap between them stays visible

### The overfitting probability weighs in no automatic score

Scope: PBO by CSCV
Effect: it is biased high under a hundred configurations, and splitting the same grid across two runs changes the penalty without the data moving
Control: it is excluded from the score and remains available as a node to wire

### Purging is applied on the out-of-sample side

Scope: CSCV block boundaries
Effect: without the embargo the overfitting probability comes out optimistic — measured 0.306 against 0.357 at a 120-bar overlap
Control: the embargo is applied and the report states that the in-sample side is not purged

## Two different outcomes, and they were written as one

### A kernel the translator does not cover

Scope: graphs built on a kernel outside the 18 translated ones
Effect: the export is refused by name and no file is written
Control: Check the exact supported translations in the [export kernel reference](https://stochastly.com/docs/exports).

### A translation below the coverage threshold

Scope: a translated kernel whose exported rule flags fewer of the engine's entries than the threshold
Effect: the file IS written below 0.8; the result carries verified:false and the measured coverage
Control: the reservation travels in the generated file header

### What the coverage number measures

Scope: the compiled and transcribed targets
Effect: it is a recall on the engine's entries with a two-bar tolerance, and extra signals carry no penalty, so it states no equivalence
Control: the exported file states that the measurement used synthetic paths

### Distance between a translated file and the run

Scope: the transcribed targets, measured over 1000 draws of 3000 bars
Effect: the worst target disagrees on 16.9 % of triggered bars (pine_avant)
Control: the campaign, its interval and its registry are published under export fidelity

## What the placebo test can and cannot separate

### The sizing placebo tests alignment, not stationarity

Scope: a modulator applied to position size
Effect: a modulator that reads ahead aligns perfectly and passes with the best score
Control: both limits are stated; the null group is enumerated when small enough, sampled and declared otherwise

### Stationarity tests are informational

Scope: ADF, KPSS, Jarque-Bera, Engle-Granger, Johansen
Effect: no threshold is wired to them, so 0 configurations are excluded on their result
Control: p-values and critical values are returned for you to read; KPSS states that same-sample parameter estimation is conservative

## What a published count is, and is not

### A declared status is not a proof

Scope: the runnable-node count
Effect: a status declared in the catalogue sets a ceiling on a result, and does not establish one
Control: the published count includes checks that were actually executed

### A recorded control is not an executed failure

Scope: the count of validations carrying a deliberately failing case
Effect: the failing case has to be triggered before it demonstrates detection
Control: the limit is written into the measurement itself

## A result has a market regime

Splitting the recorded result by regime reveals where the aggregate can hide different behaviour. This is a description of this sample, not proof that the same regimes or returns will recur.

[Screenshot: Regime split in Stochastly, Full desk · 155 nodes, light theme.](https://stochastly.com/screenshots-current/research-gaps--regime-split--desk.png)

The recorded result separated into calm and volatile observations.

## The tails can depart from the reference

A tail quantile plot compares the recorded return distribution with its reference. Departures matter when interpreting risk estimates: the reference curve is an assumption, not a guarantee about extreme losses.

[Screenshot: QQ tail plot in Stochastly, Full desk · 155 nodes, light theme.](https://stochastly.com/screenshots-current/research-gaps--qq-tail-plot--desk.png)

Empirical return quantiles against a normal reference; deviations in the tails remain visible.

---
Source: https://stochastly.com/research/gaps