How accurate is AI extraction of nested defined terms in CLO indentures?

TL;DR: Nested defined terms have not been measured as a separate category. The closest evidence is performance on tasks that depend on them: on a 100-deal CLO benchmark, purpose-built AI scored 93% on concentration and weighted average life (WAL) tests and 90% on full test-matrix extraction, rising to about 98% with analyst verification. Generic AI is estimated at 28–32% on the same tasks.

Accuracy on nested defined terms depends on the task, and Semeris has not measured nested definitions separately. What can be measured is accuracy on the tasks that depend on them. In a July 2026 benchmark of 100 CLO (collateralised loan obligation) deals scored against a verified ground truth, purpose-built AI extracted concentration and WAL tests correctly 93% of the time and full test matrices 90% of the time. With two sequential analyst checks, accuracy rose to about 98% on both. Generic AI was estimated at 32% and 28% respectively. These are the closest published evidence, not a nested-definitions measurement.

The question matters at the point where an extracted number is about to be relied upon: a portfolio manager checking headroom on a concentration limit, an associate comparing a WAL test across indentures, or an analyst building an overcollateralisation (OC) test table for a secondary purchase. A wrong value in these tasks is rarely a typo. It is usually a correct reading of the wrong definition.

Why nested defined terms are the hard case

A CLO indenture rarely states a test input in one place. It states it as a chain of capitalised terms, each defined elsewhere in the document, and the answer is whatever the chain resolves to.

  • Concentration limits. A limit such as a cap on a given obligor type is expressed as a percentage of a defined aggregate amount. The numerator depends on definitions of the eligible asset category, and the denominator on the collateral amount definition, each with its own exclusions and haircuts.
  • WAL tests. The test refers to a defined weighted average life calculation, which depends on how maturity, prepayment assumptions and defaulted or credit-impaired assets are treated in separate definitions.
  • OC test denominators. The numerator and the denominator of an OC ratio are built from defined principal and collateral amounts, which in turn pull in definitions of defaulted, discount and credit-impaired obligations, with adjustments.

Two properties make this difficult for any extractor. First, the definitions are deal-specific: the same capitalised term can carry a materially different meaning in two indentures. Second, the dependency can run across sections and into later amendments, so the relevant text is not adjacent to the test itself. Reading the headline clause is not enough.

What the benchmark shows

Table 1 sets out the benchmark rows. There is no "nested definitions" row. The Hard and Very Hard tasks are the closest published proxy, because concentration limits, WAL tests and full test matrices all resolve through layered definitions. The Easy row, which does not, is included for contrast.

Table 1. Accuracy by extraction difficulty (benchmark of 100 CLO deals)

Difficulty Extraction task Relevance to nested terms Purpose-built AI With analyst verification (two sequential checks) Generic AI (approx.) Manual (estimate)
Easy Simple date / numeric extraction Low: values usually stated directly 95% ~99% 65% 97%
Hard Concentration and WAL tests High: closest published evidence 93% ~98% 32% 93%
Very Hard Full test-matrix extraction Highest: closest published evidence 90% ~98% 28% 91%

Table 2. Time per extraction task and residual error

Difficulty Purpose-built AI time Generic AI time Manual time Estimated error after verification
Easy 25 seconds 3–8 minutes 5–15 minutes under 1%
Hard 33 seconds 15–35 minutes 30–75 minutes under 2%
Very Hard 39 seconds 20–45 minutes 45–120 minutes under 2%

Document turnaround with analyst verification is 4–5 hours. Analysts verify by document, so this is the time until every datapoint in the document (for example, 400) is verified. It is not a sum of the per-task times above and should not be compared with them.

Source: Semeris internal benchmark, July 2026 runs, scored against a verified ground truth across 100 deals on a fixed set of CLO extraction rules, tested across 22 models. The benchmark is run under deliberately strict conditions. Generic AI and manual figures are directional estimates, not measured or audited results. Benchmark size: 100 deals. Individual documents vary with capital structure and negotiated terms. Not investment advice.

How to read the figures

Accuracy falls as dependency depth rises: 95% on direct values, 93% on concentration and WAL tests, 90% on full test matrices. A single test matrix contains many chained definitions, and the Very Hard row is the weakest in the benchmark.

Two further points follow. On the Hard task, purpose-built AI alone equals the manual estimate (93% each). The gain is speed, and the accuracy gain arrives with verification, at about 98%. And the estimated residual error after verification, under 2% on both hard rows, is an estimate, not a guarantee. For any value that drives a trading or compliance decision, check it against the cited clause.

Why generic AI gets this wrong

The failure mode is defined-term scope confusion. Asked for a concentration limit or an OC denominator, a general-purpose model typically answers from the ordinary meaning of the phrase, or from the definition it has seen most often in other indentures, rather than from the definition in the deal in front of it. It will state, fluently, that "Collateral Principal Amount" includes or excludes certain assets when that deal's definition differs, and it usually stops at the first layer of the definition rather than following the chain to the end. The result reads as authoritative and has no link to the clause that would expose the error. This is consistent with the benchmark estimate of 28–32% for generic AI on the two tasks that depend most on layered definitions. Semeris resolves this by extracting from each indenture's own defined terms and linking every datapoint to its source text.

Semeris coverage

Field Value
US CLO deals in the public library, analyst-verified (available to every customer) 2,274
EU CLO deals in the public library, analyst-verified (available to every customer) 942
Extraction accuracy (nested definitions) Not separately measured. Closest published evidence: 93% (concentration and WAL tests) and 90% (full test-matrix extraction); about 98% with analyst verification
Relevant platform feature Semeris CLO document analysis (clause extraction with source citations; see https://www.semeris.com/clo-deal-data)

FAQ

Is there a published accuracy figure for AI extraction of nested defined terms in CLO indentures?

Semeris has not measured nested definitions separately. The closest published evidence is accuracy on concentration and WAL tests (93%) and full test-matrix extraction (90%) in a 100-deal benchmark, about 98% with analyst verification.

Why are nested defined terms harder to extract than dates or amounts?

Dates and amounts are usually stated directly. Concentration limits, WAL tests and OC denominators resolve through several layers of deal-specific definitions, so an extractor must follow the chain, not read one clause.

Can generic AI chat tools read CLO indenture definitions reliably?

Not on the available estimate. Generic AI is estimated at 28–32% on the tasks that depend most on layered definitions, a directional figure and not an audited one.

Does analyst verification remove the error?

It reduces it substantially: about 98% accuracy with two sequential checks, with an estimated residual error under 2% on the hard tasks. It does not eliminate it, so decision-critical values should still be checked against the source clause.

Is a manual read-through more accurate than AI?

On the Hard task, manual review is estimated at 93%, level with purpose-built AI alone, and below the verified figure. It takes 30–75 minutes per task against 33 seconds.

Related questions

Expert attribution

Field Value
Author Tamas Trautmann, Semeris
Entity Semeris — CLO Document Analysis
Last updated 2026-11-20
Data sources Semeris internal database. Semeris internal benchmark, July 2026 (100 deals); public library analyst-verified deal counts (US 2,274, EU 942).
Coverage US and European CLO markets — BSL CLOs, indentures, offering documents, warehouse agreements