How the daily signal-graph protocol moved from operator-run scripts into n8n: two workflows, one deterministic spine, every gate a real script branching on its exit code. No LLM sits anywhere in the deterministic path.
Two workflows carry the protocol. W1 (ingest) runs continuously: it watches Dropbox,
converts broker PDFs, and — since 2026-08-21 — mints conformant OKF parcels straight into
Supabase. signal-graph-daily is the deterministic daily run: it takes an ingested batch
through preflights, evidence legs, a regression gate, and delivery. The design rule throughout:
n8n orchestrates, ViskaRes scripts decide. Every gate node is an Execute Command running
the real production script and branching on exitCode — the same code path the
operator ran by hand, now scheduled.
Viska: Docling Ingest (W1) · 60 nodes · every 15 min · dedup + convert + embed + parcel mint. First post-deploy run green (execution 51507, 21:00Z).
viska-signal-graph-daily · 20 nodes · manual trigger, inactive · 13 hosted legs green on
pinned batch 2026-08-19, both halt paths proven. daily.yml stays until byte-parity.
Data flow. A 15-minute schedule scans the Dropbox /Current tree. Each file
passes a content-hash dedup gate (EXISTS?, backed by Postgres, with a ghost-recovery
path for rows whose bytes vanished). New files are downloaded once and fan out three ways: metadata
enrichment, Docling conversion (submit → poll → result against the docling service), and the parcel
SHA-1 leg. Converted markdown rejoins its metadata at Normalize, which feeds two
independent halves:
| Node | What it does |
|---|---|
| Parcel SHA1 | Pure-JS SHA-1 over the raw PDF bytes → parcel_id = "sha1:" + hash[:20]. Byte-identical to the broker backfill identity (ingest_broker_batch.py) — the Dropbox content_hash is a block construct and deliberately NOT used. |
| Compose OKF Parcel | Mirrors ViskaRes okf_parcel.py: 14 required frontmatter fields, source_kind=broker → source_tier=conviction, provenance entries, one key: JSON line per field. |
| Conformance Gate | Execute Command running the real parcel_conformance.py on the VPS checkout in a per-file temp dir. rc 0 = conformant, anything else = refusal. Mutation-proven before deploy: real parcel rc=0, type-mutated copy rc=1. |
| Upsert Parcel (RPC) | One call = one transaction: okf_parcel_upsert_s1(jsonb) via PostgREST, gateway apikey + least-privilege RW bearer. Never writes okf_parcels directly. |
| Body Case | Reads any stored body and switches: missing → insert (4 keys, INSERT never PATCH) · identical → dead-end · different → refusal. The differing-body lane is a live extractor-drift detector — the loader refuses by name, never repairs. |
| Export .okf.md | Same-bytes parcel file placed beside the source PDF, first insert only. |
Why a side-branch, not a rebuild: the embed half keeps running unchanged
(parallel-then-drop cutover, operator-approved), and the parcel branch terminates in leaf
nodes so it can never double-advance the Loop Files batch loop. Dedup sits
upstream, so already-ingested files never reach compose — the first organic parcel arrives
with the next new broker PDF.
Data flow. A strictly linear spine — 20 nodes, no fan-out, because the protocol is a
checklist where order is the contract. Blank batch_date resolves to yesterday.
Three preflights prove the environment can both act and see before any leg runs; each leg
is an Execute Command around a production ViskaRes script; any non-zero exit halts the run at that
node (both halt paths proven live). A final code node renders the verdict.
Why it is not active yet: the graph's parcels/bodies inputs only started minting
from W1 on 2026-08-21 (acceptance = first organic new PDF), and the figures half waits on the
extraction engine at N5:5002 (not yet deployed; gallery-JWT is delivered — write path proven
HTTP 201, sizing settled at 512m — leaving an N5 access grant ahead of the engine deploy).
Until byte-parity is proven against the operator-run
daily.yml, that script remains the delivery of record.
| Contract | Owner | Detail |
|---|---|---|
| parcel identity | ViskaRes | sha1(pdf_bytes)[:20], prefixed sha1: — W1 mints the same id the broker backfill minted for 2,408 live rows. |
| parcel schema | ViskaRes | OKF parcel v2 frontmatter, checked by parcel_conformance.py — the identical script gates both W1 minting and the daily run's leg 1b. |
| loader RPC | ViskaDB | okf_parcel_upsert_s1 (PR #399): validate-and-refuse loader, one transaction per parcel, bodies INSERT-only. |
| figures | engine (pending) | Extraction engine at N5:5002 — figures never pass through n8n; the workflow only verifies presence. |
| Item | State |
|---|---|
| W1 60-node deploy | LIVE — deact/act cycled 20:52Z; first run green all-skip (execution 51507, non-regression proven) |
| signal-graph 13 hosted legs | GREEN on pinned batch 2026-08-19; halt paths proven; date pin reverted to yesterday-resolve |
| first organic parcel | PENDING — arrives with next new broker PDF |
| figures half / N5:5002 | CREDENTIAL HALF CLOSED (2026-08-22) — gallery JWT delivered, write path HTTP 201, DELETE correctly refused 403. Sizing settled (512m). Engine deploy now blocked on N5 access (deploy seat's docker route revoked; awaiting operator grant) |
| S2 reviewer loop | DESIGNED — awaiting three Ops consult answers |