---
type: Contract
audience: [engineering, fund-ops]
status: stable
created: 2026-08-15
updated: 2026-08-15
generated:
by: process:viskan8n
at: 2026-08-15T08:15:00+00:00
references:
- viska-pm#223
- workflows/active/flex-pull-ADeTjBZiHOMIXzvr.descriptor.md
- workflows/active/viska-fastmail-ingest-transform.descriptor.md
- docs/workflows/substack-email-pipeline.md
- docs/superpowers/specs/2026-06-22-flex-multiday-normalize-apply-contract.md
---
# ViskaN8N daily-pipeline segment
The batch convention is `today - 1`; `report_date` is the ingest/report date, not a source document's publication date. Each stage below has one contract and names its refusal boundary.
## Stage 0 — IBKR Flex pull and freshness gate
### 1. THE ONE JOB
Pull the latest IBKR Flex statement, land the authoritative raw XML through the signed apply seam, and refuse downstream daily analysis unless the latest normalized `report_date` reaches the expected book date.
### 2. IN
| From | Exact input |
|---|---|
| n8n schedule | `0 30 13 * * 1-6`, timezone `UTC` (13:30 UTC Monday–Saturday) |
| authenticated on-demand caller | optional `body.report_date` recorded as `requested_date`; it does not override the IBKR query |
| IBKR `SendRequest` | `Status`, `ErrorCode`, `ErrorMessage`, `ReferenceCode`, `Url` |
| IBKR `GetStatement` | `FlexQueryResponse.FlexStatements.FlexStatement[]`, including per-statement report date, positions, NAV, cash and FX sections |
| gate caller | ISO `expected_book`; `scripts/prove-flex-freshness.mjs` defaults to the prior weekday and accepts `--expected=YYYY-MM-DD` |
### 3. OUT
| Artifact | Path / shape | Contract |
|---|---|---|
| immutable source drop | `flex-inbox/incoming/flex-<report_date>-<execution_id>.xml` | `docs/superpowers/specs/2026-06-22-flex-multiday-normalize-apply-contract.md` |
| authoritative store write | signed upload → `flex-apply-svc /apply` → ViskaDB `trading.*` relations | `docs/superpowers/specs/2026-06-28-flex-full-extract-contract-design.md` |
| operational heartbeat | `public.flex_sync_status`, key `workflow='ibkr_flex_eod'`; `last_run_at`, `last_status`, real `last_as_of`, next Mon–Sat 13:30 UTC `next_run_at` | `scripts/lib/flex-heartbeat.js` |
| freshness receipt | JSON `{execution_id,status,last_node,actual_book,expected_book,gate}` | exit `0=fresh`, `2=stale`, `3=pull_failed`, `4=indeterminate` |
### 4. REFUSALS
- A successful HTTP call is not freshness; only `actual_book >= expected_book` passes Gate 0.
- The gate does not fabricate an exchange-holiday calendar. Its default skips weekends only; callers supply `--expected` when an authoritative calendar says otherwise.
- An IBKR `Fail/1001` is `pull_failed`, never “did not fire” and never fresh.
- Slack alert delivery is a side effect. Both alert nodes use `onError: continueRegularOutput`, so a dead Slack credential cannot suppress `HB Failed` or `Write Heartbeat`.
- The step does not author analysis, repair IBKR data, or mint/rotate credentials.
### 5. THE CONTROL
| Control | Expected result |
|---|---|
| seeded stale twin: successful pull with `actual_book=2026-08-13`, `expected_book=2026-08-14` | exit `2`, `gate=stale` |
| passing twin: successful pull with both dates `2026-08-14` | exit `0`, `gate=fresh` |
| live control, execution `49324` | success through `Write Heartbeat`; `actual_book=expected_book=2026-08-14`; exit `0` |
| historical failure control, execution `47200` | IBKR `Fail/1001`; dead Slack aborted before heartbeat. The current alert topology must continue to `HB Failed`. |
### 6. STATUS
| built | proven |
|---|---|
| yes — active workflow `ADeTjBZiHOMIXzvr`, gate CLI and discriminating tests | yes — live pull `49324`; unit controls distinguish all four exit states |
### 7. THE SSOT
- ID/path: `workflows/manifest.json` → `workflows/active/flex-pull-ADeTjBZiHOMIXzvr.json`.
- Node map: `workflows/active/flex-pull-ADeTjBZiHOMIXzvr.descriptor.md`.
- Freshness executable: `scripts/prove-flex-freshness.mjs`.
- Pure cadence/heartbeat logic: `scripts/lib/flex-heartbeat.js` and `tests/flex/`.
- Open credential repair and IBKR application-level retry debt: n8n#59. Descriptor/cadence drift repair: n8n#107.
## Mailbox / Substack source-document lane
### 1. THE ONE JOB
Convert each qualifying Substack publication email received by the Fastmail `Substack` mailbox into one deterministic, contract-checked OKF source-document parcel and deliver it to immutable L1 storage for ViskaDB projection.
### 2. IN
| From | Exact input |
|---|---|
| Fastmail IMAP trigger | mailbox `Substack`; `LIST-ID: substack.com`; `X-MAILGUN-TAG: post`; resolved message; leave message unchanged |
| qualifying email | original Message-ID, sender/from, subject/title, publication and canonical article URLs, text/HTML body, publication timestamp, substantive HTTPS media |
### 3. OUT
| Artifact | Path / shape | Contract |
|---|---|---|
| OKF parcel | `parcels/substack/<source>-<published>-<hash>.md` in the parcels bucket | `type: source-document`; `source_meta.extraction_contract: okf-parcel-n8n-substack/6`; producer `process:okf-parcel-n8n-substack/6` |
| projection handoff | immutable L1 object consumed by ViskaDB's L2 projector | stable current revision in `okf_parcels`; older revisions retained as deprecated with `superseded_by` |
The older inactive Outlook workflow `wig8zrxkyoLLeO9G` writes `research_findings` and is not this stage.
### 4. REFUSALS
- The lane does not poll JMAP, mark mail read, or treat account/setup notices as publications.
- It does not write directly to L2 tables or to the retired `research_findings` path.
- It does not preserve unsafe URLs, tracking/capability parameters, tracking pixels, product chrome, or opaque Substack redirects.
- It does not reconstruct historical publisher structure from flattened L2 text; replay requires authoritative original MIME/HTML.
- It does not claim a reconnect or activation backfilled mail already present in the folder.
### 5. THE CONTROL
| Control | Expected result |
|---|---|
| seeded anchored substantive image plus tracking pixel | substantive image survives in Markdown and `media`; tracking pixel is absent |
| passing real twin, execution `49153` | 11 deduplicated Markdown images and 11 media entries; repeat reports `already_delivered` |
| malformed parcel twin | `Assert Contract` rejects absent identity/title/body, non-Substack object key, angle-bracket origin, or body under 50 characters before signing |
### 6. STATUS
| built | proven |
|---|---|
| yes — active workflow `37ZscHy7SyDYx1CC`, six-node contract `/6` | yes — executions `49087`/`49153`; extraction regressions green; ViskaDB projection invariant independently passed |
### 7. THE SSOT
- ID/path: `workflows/manifest.json` → `workflows/active/viska-fastmail-ingest-transform.json`.
- Node map: `workflows/active/viska-fastmail-ingest-transform.descriptor.md`.
- Full producer, recovery and serving contract: `docs/workflows/substack-email-pipeline.md`.
- Breaking controls: `tests/ingestion/substack-link-extraction.test.js` and `tests/ingestion/imap-raw-fetch.test.js`.
- Confirmed missed-email recovery: `scripts/recover-substack-email.js`.