Using macro and alternative data

A macro value stamped March was published weeks later. Joining that value to a March bar assumes a publication delay of zero, which is the most favourable case available. The result is a look-ahead that leaves no trace in the equity curve.

Revisions add a second layer. A series published once and restated twice has three histories, and the current file holds the last of them. A rule fitted on restated values uses figures that arrived after the decision it is meant to describe.

This defect survives purging, embargoes and walk-forward splits, because it lives inside the values and not in the split. It is also directional: restatements move data toward what later turned out to be the case, so the flattering side is the usual one.

The alignment stage reads the vintage column when the file carries one, and that column dates the knowledge. With no vintage in the file, a numeric parameter named pit_lag_days has to be entered on the node. Absent both, the reading is returned as inconclusive and the missing key is named in the result.

The catalogue entry for that parameter carries no default and no candidate list. The desktop editor falls back to the first candidate when one exists, so a candidate list starting at zero would have written a publication delay on your behalf. The field stays empty until you fill it.

A negative value is rejected with its reason, since it would place a value in your hands before the period it describes had ended. Once the knowledge dates exist, each run bar takes the last value knowable at that bar.

The macro and alt-data family ships 4 nodes, of which 4 are validated by executable checks. The validated ones are four readers, one per mechanic. None of them names a series: the series is chosen on the node, from the list of the folder you point at, so a credit spread, a yield, a dollar index or a series of your own are read by the same node.

Each mechanic joins the same two objects: the dated P&L of your best configuration, and the external series selected on the node. Two copies of one reader on two series give two readings, each kept under its own key. A composite spread absent from your folder is derived from its two legs, and its knowledge date is the later of the two.

Lead-lag returns a correlation at each lag as a series, so the shape of the profile is what indicates a lead. The dominant lag is picked on the same data that produced it, and the result states that in the note attached to the figure.

Macro momentum splits your returns by the direction of the series over a lookback you enter, and returns both counts beside both means. A window in which the series held one direction throughout comes back as inconclusive, with the two counts quoted.

Mean reversion computes a rolling z-score whose window closes on the current bar, an AR(1) coefficient and a half-life. A coefficient at or above one is reported as an absent half-life with its reason, since a series without reversion has no half-life to return.

The stat-arb reading fits an Engle-Granger regression between your run equity and the macro level, then reads the residual. It uses equity and not returns, because a cointegration holds between integrated series.

A sweep is probed before the first configuration runs. The probe reruns the kernel on the price series truncated at a declared fraction, then compares the per-bar P&L over the overlap. A disagreement stops the run and names the kernel or the wired filters.

The probe takes 1 configuration from the sweep, so a dependency confined to another parameter set is outside what it measures. Its subject is the graph you wired, and the series it truncates is the bar file.

That probe operates on the bar files. An external series enters afterwards, joined to the P&L the run already produced, so a publication delay entered too short raises no disagreement there. The defence at that point is the alignment stage described above.

One consequence is worth carrying: a figure returned by a macro node is conditional on the lag you entered. Two different lags produce two different readings, which executable validation checks on each node.

The node takes a folder path and the identifier of the series inside it. No vendor is embedded and no default folder is assumed, so the files are the ones you placed there in CSV or Parquet. The outbound destinations the package declares serve a licence check and the assistant, and no bar is downloaded.

Preview the series before wiring it. The preview reports whether a vintage column was found, the median distance between vintage and observation, the number of repeated observations, the count of gaps, the run of unchanged values, and the rows whose date could not be read.

Two of those fields do the work here. A vintage column present means the file dates its own releases. A vintage column absent means the median publication lag is unavailable from the file, and the figure has to come from the publisher of the series.

Rows whose observation date could not be read are dropped, counted, and the count travels into the run result beside the verdict. A frequent cause is a date column mixing bare dates with dates carrying a timezone offset.

The loader counts the rows beyond the first that share an observation date, then reduces each date to a single row. The alignment stage handles a restatement history correctly when it receives one: a value published at one figure and restated later reads as the first figure between the two release dates. In the shipped run path, one row per observation date arrives.

The consequence is bounded and directional. A series whose file holds a full restatement history is read at its last stored figure per date, so a backtest over it uses values that arrived after the decision. That excess-row count is reported by the preview, and the run result carries the unreadable-row count without carrying this one.

The condition for closing it is a loader that keeps the cube of observation, vintage and value, so the run can reconstruct the series as of each date. That stage is the same one the roadmap entries below name.

What to do meanwhile, and it costs one preview: read that count before wiring the series. At zero, the file holds one row per observation date and the alignment carries the full promise. Above zero, treat the reading as fitted on restated values, and keep a file restricted to first releases when your source publishes one.

Four causes account for them, and each is a data shape the current loaders do not carry.

An event calendar is a list of timestamped instants with an importance and a surprise, and the series loader expects a date and a value. A volatility term structure is a surface with several maturities per date, plus a roll rule at expiry.

A yield-curve state, an insider-activity feed and a cross-sectional factor need several series aligned on a common index before any join to price. That panel stage is the one the panel-builder entry declares missing.

Search velocity, social attention and on-chain flows need a provider connector that the package does not carry, and each has a measurement trap of its own. Search interest is renormalised per request, attention is revised upward after the fact, and blocks reorganise, so a value read live differs from what the archive later holds.

Derivatives columns are a separate case, and the reason was rewritten once. Funding, open interest, basis, liquidations and taker-buy ratio are preserved by the bar loader when your file carries them, and they reach the calculation. What stays open is that no shipped source supplies open interest or basis.

Requiring a row to be complete across many columns intersects the gaps of each column at once. With columns independently present on a fraction of dates, the share of complete rows falls geometrically in the number of columns.

Real panels behave worse than that arithmetic. A weekly series is absent four days in five, a column starting later removes the early window, and a single long outage removes the period it spans. Dropping incomplete rows produces a truncated sample that downstream code treats as legitimate.

The practice that follows: constrain completeness on the target, impute the features explicitly after standardisation, and report the share of each feature that was filled. Print rows, columns and the first and last usable date at each stage.

The macro nodes apply a floor of their own. Below a declared number of aligned bars, a reading comes back inconclusive with the two date ranges quoted, so a run and a series barely overlapping produce a named refusal.

Real-time datasets show restatements large enough to reverse conclusions drawn on final data. A publication lag alone aligns the timing and keeps the final values, so the restatement leak remains after the shift.

The four mechanics report figures and profiles beside the verdict, and they exclude no configuration. A threshold applied to one of them is a decision you take outside the reading.

The delivered installer embeds the strategy engine and packaging stops when its source is missing. An explicit non-deliverable demo can use a labelled generic cross-sectional fallback. Its label determines whether these readings apply.