Methodology

LLM Product Intelligence Knowledge Base (PIKB)

A pattern for building a **persistent, evolving Product Intelligence system** using LLMs.

A pattern for building a persistent, evolving Product Intelligence system using LLMs.

The Core Idea

This document defines a Product‑focused adaptation of the LLM Wiki pattern. Instead of transient retrieval or summarisation, an LLM incrementally builds and maintains a Product Intelligence Knowledge Base (PIKB) that compounds insight over time and supports evidence‑based product strategy and ideation.

The PIKB is designed to:

Accumulate insights from diverse sources (customers, sales, market, competitors)
Maintain an evolving, synthesised understanding of the product space
Anchor interpretation and ideation to explicit strategy and constraints
Surface opportunities and product ideas grounded in evidence

The LLM maintains the system. Humans provide judgement, strategy, and direction.

Architecture Overview

The PIKB consists of three layers:

1. Raw Sources (Immutable)

A curated collection of unmodified source data:

Customer interviews and feedback
Win/loss analysis
Usage and behavioural data
Market research and benchmarks
Competitor intelligence
Internal strategy and roadmap documents

Raw sources are never edited or summarised in place.

2. Product Intelligence Wiki (LLM‑Maintained)

A directory of Markdown files generated and continuously maintained by the LLM. This layer contains:

Synthesised insights
Customer and market understanding
Problem statements and opportunity areas
Competitive positioning
Product ideas and enhancement candidates

The wiki represents the current best understanding of the product reality and evolves as new evidence is ingested.

3. Schema (LLM Operating Manual)

A configuration document (e.g. AGENTS.md or CLAUDE.md) that defines:

Wiki structure and conventions
Ingestion workflows per insight type
Rules for synthesis and cross‑referencing
Guardrails for opportunity detection and idea generation

This schema transforms a general LLM into a disciplined Product Intelligence system.

Strategy as a First‑Class Constraint

Before ingesting insight data, the wiki must be seeded with baseline product reality. These foundations anchor interpretation and prevent unconstrained ideation.

Required Baseline Files

product_proposition.md
vision_and_strategy.md
lifecycle_model.md
target_markets.md
constraints_and_assumptions.md

These documents are treated as authoritative context and must be explicitly referenced when evaluating insights, opportunities, and ideas.

Example Wiki Structure

wiki/
  index.md
  log.md

  foundations/
    product_proposition.md
    vision_and_strategy.md
    lifecycle_model.md
    target_markets.md
    constraints_and_assumptions.md

  customers/
    personas/
    segments/
    jobs_to_be_done/
    unmet_needs/

  problems/
    problem_statements/
    pain_points/

  insights/
    customer_insights.md
    sales_insights.md
    usage_insights.md
    market_insights.md
    competitor_insights.md

  competitors/
    competitor_profiles/
    comparison_tables/

  opportunities/
    opportunity_backlog.md
    validated_opportunities/
    discarded_opportunities/

  experiments/
    hypotheses.md
    tests.md
    outcomes.md

  recommendations/
    product_ideas.md
    enhancement_candidates.md

Core Operations

Ingest

When new raw sources are added, the LLM:

Reads and interprets the source
Extracts key signals and patterns
Updates relevant wiki pages
Strengthens or challenges existing beliefs
Records changes in the log

Insights are never isolated; they are always integrated into the broader product understanding.

Query

Users query the wiki rather than raw data. Example questions:

Which customer problems recur most often?
Where does evidence conflict with our assumptions?
Which opportunities are best supported by insight?

High‑value analyses generated during queries are written back into the wiki so that learning compounds.

Lint (Health Checks)

Periodic maintenance passes identify:

Contradictory claims
Stale or superseded insights
Repeated pain points without opportunity framing
Opportunities lacking recent evidence
Orphaned or weakly connected pages

This keeps the PIKB current and decision‑ready.

Opportunity‑First Product Discovery

Ideas are not generated directly from insights. Instead, the LLM is trained to identify opportunities by detecting tension between:

Customer evidence
Strategic intent
Market dynamics
Current solution coverage

Opportunities are structured artifacts, each linked to supporting evidence and strategy.

Disciplined Idea Generation

Product ideas and enhancements are proposed only when:

A validated opportunity exists
Multiple independent evidence sources support it
The idea aligns with strategy and lifecycle stage
Constraints and risks are explicitly documented

Each idea includes:

Linked opportunity
Problem addressed
Solution hypothesis
Expected impact
Risks and unknowns
Evidence strength

Low‑confidence ideas are flagged rather than obscured.

Technology Innovation as a Constraint‑Relaxing Layer

The lifecycle above is rooted in evidence the team has authored — customer interviews, decisions, delivery outcomes. A peer evidence stream sits alongside it: technology shifts that change what's feasible. New capabilities in the world don't justify new opportunities, but they do change how an existing opportunity gets answered.

The implementation surface (Inc 30):

Raw evidence lands in raw/technologies/ (release notes, API launches, library / platform / hardware shifts).
Synthesis writes capability pages to wiki/capabilities/. Each page describes one capability in verb-imperative form, carries a maturity field (proposed / early / mature), and lists the existing opportunities/<slug> IDs it could relax a constraint for.
The recommend pipeline auto-loads mature capabilities tied to the candidate opportunity. Capabilities flow into synthesis as constraint-relaxers — never as primary justification. A recommendation whose only argument is "this technology now exists" is invalid by prompt rule and by runtime check; customer evidence remains mandatory.
The reconcile pipeline gains an opt-in --include-capabilities path: a mature capability can propose striking through a paragraph in constraints_and_assumptions.md it now relaxes. Capability-triggered baseline edits target ONLY constraints_and_assumptions.md; tech maturity does not get to rewrite vision, lifecycle, target markets, or proposition.

This keeps the methodology's core discipline intact — opportunity-first, evidence-cited, baseline-stable — while acknowledging that a stale constraints_and_assumptions.md line ("we can't do real-time voice", "PDFs need manual extraction") needs to age out when the world changes underneath it.

Hypothesis Lifecycle (Inc 32)

Most product teams operate primarily in the speculative space and only occasionally in the validated space. Roadmaps are largely built on speculation that has never been examined — what the team thinks it knows versus what it actually knows. The PIKB methodology gives speculation a first-class home so it can be made visible and walked through validation, rather than blurring it into evidence and contaminating downstream synthesis.

What a hypothesis is

A hypothesis is a first-party written-down working belief that has not yet been validated against evidence. It lives in wiki/hypotheses/<slug>.md and carries a locked three-state lifecycle:

speculative — the default. The team's working belief; nothing supports or contradicts it yet in the wiki.
corroborated — the wiki contains evidence (insights, decisions, delivery outcomes, research notes) that supports the claim.
refuted — the wiki contains evidence that contradicts the claim. Sometimes both states' evidence lists coexist; refuted wins, but the corroboration trail stays as part of the audit story.

Why hypotheses are first-party but aren't evidence

The harvest-object rule (the team's own writing about its own product) holds: speculation is still first-party, just not yet validated. Most teams already produce it as conversation, planning notes, slack threads — they just don't track it explicitly. PIKB's contribution is making it visible and giving it a path to validation or retirement.

But hypotheses are NOT evidence. Recommendations cannot stand on speculation alone. The recommend prompt rejects "this hypothesis says so" arguments the same way it rejects "this technology now exists" arguments (the Inc 30 capability rule, mirrored). Even corroborated hypotheses act only as constraint-relaxers — they shape the framing of a recommendation but the primary justification has to come from the corroborating evidence directly.

The validation flow

winnow validate <slug> runs the wiki against a hypothesis and produces three things:

A corroboration list — evidence pages that support the claim, with per-page rationale.
A refutation list — evidence pages that contradict the claim, with per-page rationale.
Validation proposals — concrete actions the team can take if neither list is conclusive ("review last quarter's win/loss notes for X mentions", "run a 5-user moderated session on Y", "check session-replay data for behaviour Z"). Quality bar: each proposal must be specific enough to act on.

Output lands as a sidecar at .winnow/proposals/<id>.{json,md} for operator review — same review-then-apply safety model as winnow reconcile. Winnow never auto-flips state. The proposed transition is computed but the operator approves it. This is methodologically deliberate: the boundary between "thinking" and "validated knowledge" is the operator's job to draw.

Boundaries

Hypotheses don't go through raw/. Raw is for first-party authored evidence (sales notes, support tickets, research transcripts) and for technology evidence (release notes, API launches). Hypotheses are written directly into wiki/hypotheses/, the same way decisions and recommendations are.
Hypotheses don't change foundations. Even corroborated ones don't qualify as decisions or delivery outcomes. Mode 6 (Baseline Reconciliation) only updates baselines from those two sources.
A corroborated hypothesis sometimes wants to be promoted to an insight. That's an operator-driven move, not an automatic transition — the insight has different schema (evidence: is required, state: doesn't apply) and different consumers. Promotion is a manual rewrite, not an LLM step.

Indexing and Traceability

`index.md`

A structured catalogue of all wiki content with short summaries. Used by the LLM for navigation and synthesis.

`log.md`

An append‑only chronological record of ingestions, analyses, belief changes, and lint passes. This provides historical context for decision‑making.

Why This Works

Product knowledge systems fail when maintenance effort exceeds value. In the PIKB model:

Bookkeeping is automated
Cross‑references are maintained continuously
Synthesis improves over time
Strategic memory is preserved

The LLM handles maintenance and synthesis. Humans focus on judgement, prioritisation, and leadership.

Final Note

This is a pattern, not a product.

When implemented with clear schema rules and strong strategic anchors, a Product Intelligence Knowledge Base becomes:

A living product memory
A disciplined opportunity engine
A guard‑railed product idea generator

The LLM maintains the system. Product leaders do the thinking.