Relation Extraction & Knowledge Graph Construction

An analyst reads: "Tim Cook became CEO of Apple in 2011." That one sentence hides four facts:

• (Tim Cook, role, CEO)
• (Tim Cook, employer, Apple)
• (Tim Cook, start_date, 2011)
• (Apple, type, Organization)

Relation extraction (RE) turns free text into structured triples: (subject, relation, object). Aggregate those triples across a corpus and you have a knowledge graph. Aggregate and query them and you have a reasoning substrate for RAG, analytics, or compliance audits.

Here's the 2026 problem: LLMs extract relations enthusiastically. Too enthusiastically. They hallucinate triples the source text never supported, and without provenance you can't tell a real triple from a plausible fiction. The 2026 answer is AEVS-style anchor-and-verify pipelines.

Triple form. (subject_entity, relation_type, object_entity). Relations come from a closed ontology (Wikidata properties, FIBO, UMLS) or an open set, OpenIE (Open Information Extraction)-style, where anything goes.

Three extraction approaches.

1. Rule / pattern-based. Hearst patterns: "X such as Y" → (Y, isA, X). Plus hand-crafted regex. Brittle, precise, explainable.
2. Supervised classifier. Given two entity mentions in a sentence, predict the relation from a fixed set. Trained on TACRED, ACE, KBP. Standard 2015–2022.
3. Generative LLM. Prompt the model to emit triples. Works out of the box, but needs provenance or it hallucinates plausible-looking junk.

AEVS (Anchor-Extraction-Verification-Supplement, 2026). The current hallucination-mitigation framework:

• Anchor. Identify every entity span and relation-phrase span with exact positions.
• Extract. Generate triples linked to anchor spans.
• Verify. Match each triple element back to the source text; reject anything unsupported.
• Supplement. A coverage pass ensures no anchored span is dropped.

Hallucinations drop sharply, though the pipeline costs more compute. The upside: it's auditable.

The open-vs-closed tradeoff.

• Closed ontology. A fixed property list, like Wikidata's 11,000+ properties. Predictable and easy to query, but effectively closed to new relations.
• Open IE. Any verbal phrase becomes a relation. Recall is high, precision is low, and the output is messy to query.

Production KGs usually mix both: open IE for discovery, then canonicalize relations onto a closed ontology before merging into the main graph.

Step 1: pattern-based extraction

PATTERNS = [
    (r"(?P<s>[A-Z]\w+) (?:is|was) (?:a|an|the) (?P<o>[A-Z]?\w+)", "isA"),
    (r"(?P<s>[A-Z]\w+) (?:is|was) born in (?P<o>\w+)", "bornIn"),
    (r"(?P<s>[A-Z]\w+) works? (?:at|for) (?P<o>[A-Z]\w+)", "worksAt"),
    (r"(?P<s>[A-Z]\w+) founded (?P<o>[A-Z]\w+)", "founded"),
]

See code/main.py for the full toy extractor. Hearst patterns still ship in domain-specific pipelines because they are debuggable.

Step 2: supervised relation classification

REBEL (Babelscape) is the standard open-weights baseline for this approach: a seq2seq model that reads text and generates triples already tagged with Wikidata property ids, fine-tuned on distant-supervision data. Load it like any other Hugging Face checkpoint:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
 
tok = AutoTokenizer.from_pretrained("Babelscape/rebel-large")
model = AutoModelForSeq2SeqLM.from_pretrained("Babelscape/rebel-large")
 
text = "Tim Cook was born in Alabama. He later became CEO of Apple."
encoded = tok(text, return_tensors="pt", truncation=True)
output = model.generate(**encoded, max_length=200)
triples = tok.batch_decode(output, skip_special_tokens=False)

Step 3: LLM-prompted extraction with anchoring

prompt = f"""Extract (subject, relation, object) triples from the text.
For each triple, include the exact character span in the source text.
 
Text: {text}
 
Output JSON:
[{{"subject": {{"text": "...", "span": [start, end]}},
   "relation": "...",
   "object": {{"text": "...", "span": [start, end]}}}}, ...]
 
Only include triples fully supported by the text. No inference beyond what is stated.
"""

Verify every returned span against the source. Reject anything where text[start:end] != triple_entity. This is the AEVS "verify" step in its minimal form.

Step 4: canonicalize onto a closed ontology

RELATION_MAP = {
    "is the CEO of": "P169",       # "chief executive officer"
    "was born in":   "P19",         # "place of birth"
    "founded":        "P112",       # "founded by" (inverted subject/object)
    "works at":       "P108",       # "employer"
}
 
 
def canonicalize(relation):
    rel_low = relation.lower().strip()
    if rel_low in RELATION_MAP:
        return RELATION_MAP[rel_low]
    return None   # drop unmapped open relations or route to manual review

Canonicalization is often 60-80% of the engineering work. Budget for it.

Step 5: build a small graph and query

triples = extract(text)
graph = {}
for s, r, o in triples:
    graph.setdefault(s, []).append((r, o))
 
 
def neighbors(node, relation=None):
    return [(r, o) for r, o in graph.get(node, []) if relation is None or r == relation]
 
 
print(neighbors("Tim Cook", relation="P108"))    # -> [(P108, Apple)]

This is the atom of every RAG-over-KG system. Scale it with RDF triple stores (Blazegraph, Virtuoso), property graphs (Neo4j), or vector-augmented graph stores.

• Coreference before RE. In "He founded Apple," RE needs to know who "he" is first. Run coref before extraction (lesson 24).
• Entity canonicalization. "Apple Inc" and "Apple" must resolve to the same node, which means entity linking runs first too (lesson 25).
• Hallucinated triples. LLMs emit triples the text does not support. Enforce span verification.
• Relation canonicalization drift. Open IE relations are inconsistent: "was born in," "came from," "is a native of." Collapse them to canonical ids or the graph is unqueryable.
• Temporal errors. "Tim Cook is CEO of Apple" is true today and would have been false in 2005. Many relations are time-bounded, so use qualifiers (P580 start time, P582 end time in Wikidata).
• Domain mismatch. REBEL trained on Wikipedia. Legal, medical, and scientific text often need domain-fine-tuned RE models.

The 2026 stack:

The integration pattern: NER → coref → entity linking → relation extraction → ontology mapping → graph load. Every stage is a potential quality gate.

Save as outputs/skill-re-designer.md:

---
name: re-designer
description: Design a relation extraction pipeline with provenance and canonicalization.
version: 1.0.0
phase: 5
lesson: 26
tags: [nlp, relation-extraction, knowledge-graph]
---
 
Given a corpus (domain, language, volume) and downstream use (KG-RAG, analytics, compliance), output:
 
1. Extractor. Pattern-based / supervised / LLM / AEVS hybrid. Reason tied to precision vs recall target.
2. Ontology. Closed property list (Wikidata / domain) or open IE with canonicalization pass.
3. Provenance. Every triple carries source char-span + doc id. Non-negotiable for audit.
4. Merge strategy. Canonical entity id + relation id + temporal qualifiers; dedup policy.
5. Evaluation. Precision / recall on 200 hand-labelled triples + hallucination-rate on LLM-extracted sample.
 
Refuse any LLM-based RE pipeline without span verification (source provenance). Refuse open-IE output flowing into a production graph without canonicalization. Flag pipelines with no temporal qualifier on time-bounded relations (employer, spouse, position).

1. Easy. Run the pattern extractor in code/main.py on 5 news-article sentences. Hand-check precision.
2. Medium. Use REBEL (or a small LLM) on the same sentences. Compare triples. Which extractor has higher precision? Higher recall?
3. Hard. Build the AEVS pipeline: extract with LLM + verify spans against source. Measure hallucination rate before vs after the verify step on 50 Wikipedia-style sentences.

• Mintz et al. (2009). Distant supervision for relation extraction without labeled data: the distant-supervision paper.
• Huguet Cabot, Navigli (2021). REBEL: Relation Extraction By End-to-end Language generation: the seq2seq RE workhorse.
• Wadden et al. (2019). Entity, Relation, and Event Extraction with Contextualized Span Representations (DyGIE++): joint IE.
• AEVS: Anchor-Extraction-Verification-Supplement framework, the 2026 hallucination-mitigation design.
• Wikidata SPARQL tutorial: canonical graph queries.