01
Framework
Hybrid triad — traditional automation + LLM + human. Scorecard for picking the right tool.
A Reading of the CFA Institute Series
Automation
Ahead
Practical GenAI for investment professionals, triangulated against the regulator and academic record.
01 · Setup · Why this series
Why this series, why now
- The CFA Institute published a four-part series on GenAI in finance — practical, code-grounded, fiduciary-credentialed.
- It is the kind of artifact an asset-management team will be asked about by clients, regulators, and counterparties through 2026.
- We read it cover to cover, then cross-checked the load-bearing claims against Anthropic, the IMF, Goldman, the FSB, and McKinsey. This is our reading.
02 · The Series at a Glance
Four installments, one practical thesis
02
LLM Guide
Open vs closed source. Finance-tuned (FinLLM, FinGPT) vs general-purpose (GPT-class). Benchmarks.
03
RAG for Documents
End-to-end pipeline on Apple / Amazon / Microsoft proxy statements. Hard numbers on what works.
04
Agentic AI
Five workflow patterns + three case studies. When to deploy an agent, when to stick with a workflow.
03 · Where the Series Aligns
Four points where CFA, regulators, and engineers agree
-
Workflow over agent in regulated finance— CFA, Anthropic, IMF
Deterministic, auditable paths beat autonomous direction for anything client money touches.
-
Human-in-the-loop is non-negotiable— CFA, IMF, FSB
High-stakes outputs route through a person. No exceptions in the regulated workflow.
-
Hallucination is the central practical risk— universal consensus
Not an edge case, a design constraint. Every production pattern in the series is shaped by it.
-
RAG beats pure pretrained LLM for current and proprietary data— universal consensus
Retrieval grounds the model in source documents. The pretrained-only path is the wrong path.
04 · Taxonomy
Workflows vs agents — the right primary axis
Workflows
Predefined paths
"Systems where LLMs and tools are orchestrated through predefined code paths." Anthropic — Building Effective Agents, Dec 2024
- Deterministic
- Auditable
- Composable from simple patterns
- The right default for regulated finance
Agents
Autonomous direction
"Systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." Anthropic — ibid.
- Higher capability ceiling
- Lower auditability
- Reserved for bounded, monitored deployments
- The exception, not the default
05 · Workflow Patterns
Five patterns cover most of what works
-
Prompt chaining Break a task into sequential LLM calls. Output of step N feeds step N+1.
-
Routing Classify the input, hand off to the specialized handler.
-
Parallelization Run independent subtasks concurrently, aggregate the results.
-
Orchestrator–workers A planner LLM decomposes the task, dispatches to worker LLMs, assembles the output.
-
Evaluator–optimizer One LLM produces, a second LLM critiques and asks for a revision.
06 · The RAG Case Study
What RAG actually delivers — Apple, Amazon, Microsoft proxy statements
62%
Multi-company, multi-variable extraction
Structured markdown + metadata. LLM-judge accuracy.
55%
Quantitative-data accuracy
Numbers degrade fastest. 11-point gap behind qualitative.
32%
Same task — naive chunking
Document structure ignored. The 30-point cost.
Reading the row — dimmed cell marks the baseline (what you get without document structuring). The 30-point gap from 62% to 32% is the headline.
- Document structuring matters. The 30-point gap between 62% (structured) and 32% (naive) is the headline finding.
- Numbers are harder than text. Quantitative fields trail qualitative by 11 points.
- Math without a calculator fails. The fix — give the agent a Python tool. See A5.
- Caveat. Current-state, improving. Not a ceiling.
07 · Adoption · Triangulated
How much is finance actually using this
McKinsey · State of AI 2024
~58% of financial institutions attribute revenue growth to AI. Trading and personalized client interaction are the named beneficiaries.
IMF · Bo Li speech, Sept 2024
"Large language models … are now enabling investors to process very large amounts of unstructured, often text-based, data."
Goldman · Top of Mind #129
Spend is real (~$1T projected). Payoff is uncertain.
08 · The Capex Hedge
$1 trillion of spend — does it pay off?
$1trillion
Projected AI capex
Goldman Sachs Research projection — the number that gets quoted. The skepticism inside the same report gets quoted less.
"Tech giants and beyond are set to spend over $1 trillion on AI capex in coming years … We're skeptical that current GenAI capability justifies the projected capex."
Jim Covello — Head of Global Equity Research, Goldman Sachs
"Limited US economic upside from AI over the next decade."
Daron Acemoglu — MIT (same Goldman piece)
- For a fiduciary audience, the honest framing pairs the spend figure with the payoff question.
- Not a contrarian take — Goldman's own house view on its own number.
- For an asset manager allocating to AI exposure, the payoff question is the question.
09 · Bottom Line
What to take from the series
-
The framework is sound
Hybrid (traditional + LLM + human), workflows first, agents on a leash. Use it as the structural backbone of any GenAI-in-finance conversation.
-
Anchor each claim in the primary source
Anthropic on taxonomy, IMF on regulator tone, Goldman on capex, FSB on system risk, McKinsey on adoption. CFA is the finance-specific translator, not the originator.
-
Use the RAG numbers honestly
62% / 55% / 32% — current state, improving. Document structure delivers a 30-point gain. Math needs tools.
-
Foreground the capex question
Industry consensus on benefit is not yet industry consensus on payoff.
Appendix · Operator Team
Technical Notes
Six sub-sections covering system-level risk, the autonomy-timeline tension, the tooling stack, math + tool-use, and source inventory.
A1 · System-Level Risk
Where the series underweights: system-level risk
CFA frames risk firm-level — hallucination, bias, compliance. The macroprudential bodies frame risk system-level. For an asset-management audience the system frame matters as much as the firm frame.
-
Foundation-model concentration
A small number of providers backs most production GenAI. A provider outage or model behavior change propagates to every downstream user simultaneously.
-
Model-driven herding
When many funds run the same foundation model with similar prompts on similar data, output correlation goes up. Correlated outputs produce correlated trades. Correlated trades thin liquidity in stress.
-
Third-party dependency
AI capability sits behind API contracts that few users have negotiating leverage over. Outsourcing inference is also outsourcing operational risk.
A2 · Autonomy Timeline Tension
Where the series runs hotter than the regulator
CFA · Installment 4 (Pisaneschi)
"Agentic AI is on track to influence every part of modern organizations."
"Today's agents are the least capable we'll ever use."
IMF · Bo Li, September 2024
"The much larger jump from AI-generated model inputs to very sophisticated autonomous AI-driven financial agents still seems far off."
Reading
CFA's tone, especially in Installment 4, runs more enthusiastic about near-term agentic deployment than the IMF's September 2024 view. Both can be right — agentic workflows (which CFA mostly describes) are deployable today; autonomous financial agents (what IMF sees as far off) are not the same thing. The briefing should not let the rhetorical overlap collapse the distinction.
A3 · CFA vs Peer Sources
Where the series runs hotter — the table
| Topic | CFA framing | Peer source | Reading |
|---|---|---|---|
| Adoption | Nvidia 55% "exploring" | McKinsey ~58% revenue-attribution; IMF productivity language | Swap vendor stat for McKinsey + IMF |
| Capex | $1T cited neutrally | Goldman (own source) is skeptical | Foreground skepticism |
| Autonomy timeline | "On track to influence every part …" | IMF: autonomous agents "far off" | Distinguish agentic workflows (now) from autonomous agents (later) |
| Risk frame | Firm-level (hallucination, compliance) | BIS / FSB add system-level (concentration, herding, third-party) | Add system-level frame |
| YC startup stat | CFA-original (73% agentic) | Not independently replicated by hermes | Flag as CFA-original; do not lead with it |
A4 · The Tooling Stack
What CFA used — concrete and reproducible
The RAG case-study stack from Installment 3:
-
Document ingestion
Mistral OCR→ structured markdown - Chunking Structured (preserve markdown hierarchy + metadata) vs naive — the 32% comparison
-
Vector store
Chroma -
Embeddings
OpenAI -
LLM
GPT-4o-mini - Evaluation LLM-judge methodology — a second LLM grades the extraction
-
Test set
Apple, Amazon, Microsoft
DEF 14Aproxy statements -
GitHub
CFA-Institute-RPC/The-Automation-Ahead— the Python is the actual asset
Reading
Mid-2025 default-modern. Nothing exotic. The 30-point structured-vs-naive gap is the lesson — not the model choice.
A5 · Math Hallucination + Tool-Use
The Nadella-compensation example — and the fix
The bug
99.7864%
LLM-returned. Last two digits hallucinated.
The correct answer
99.7854%
Same calculation with a Python tool. Traceable to the underlying number.
The reading
An LLM without a calculator is doing token-pattern arithmetic. It will get the order of magnitude right and the last digits wrong. This is not a model failure — it is a category error about what the model is.
The fix
Give the agent a Python tool.
The principle
Tools close capability gaps. Arithmetic → calculator. Database → SQL. Web fetch → browser. The "agent" pattern is mostly tool-use plus a loop.
A6 · Source Inventory + Next-Pass Options
Sources used and how to go deeper
| Source | Type | Used for |
|---|---|---|
| CFA Institute — Automation Ahead | Industry body | Series under review |
| Anthropic — Building Effective Agents | Engineering essay | Taxonomy + 5 workflow patterns |
| IMF — Bo Li (Sept 6 2024) + GFSR Ch 3 | Regulator | Risk + autonomy timeline |
| Goldman Sachs — Top of Mind #129 | Sell-side research | Capex skepticism |
| FSB — Financial Stability Implications of AI | Regulator | System-level risk |
| McKinsey — State of AI 2024 | Consultancy | Adoption stat |
| BIS — IFC Bulletin 67 + Bulletin 100 annex | Regulator | System-risk corroboration |
| ECB — FSR Special (May 2024) | Regulator | EU stability lens |