diff --git a/kb/CHANGELOG.md b/kb/CHANGELOG.md new file mode 100644 index 0000000..3245baa --- /dev/null +++ b/kb/CHANGELOG.md @@ -0,0 +1,23 @@ +# Changelog — KB System + +All notable changes to KB files and system. + +## [2025-01-27] KB System Setup + +### Added +- `kb/README.md` - KB system documentation +- `kb/_templates/note.md` - Note template +- `kb/_templates/decision.md` - Decision template +- `kb/_templates/howto.md` - HowTo template +- `kb/_guides/KB_INGEST_PROMPT.md` - KB ingestion system prompt +- `kb/scripts/generate-index.sh` - Index generation script +- `kb/CHANGELOG.md` - KB changelog (this file) + +### Notes +- Phase 2 KB System Setup completed +- All templates include complete frontmatter with all 18 required fields +- Index generation script created and tested +- System customized for Tendril project (replaced "pairs" references with "tendril") + +--- + diff --git a/kb/README.md b/kb/README.md new file mode 100644 index 0000000..f076a8a --- /dev/null +++ b/kb/README.md @@ -0,0 +1,289 @@ +# Knowledge Base (KB) System + +**Last updated**: 2025-01-27 + +## Overview + +The KB system provides a lightweight, LLM-friendly staging area for external information that may inform Tendril project development. It uses structured routing, confidence scoring, and automatic indexing. + +The KB system complements the phase documentation system by capturing: +- Pulse Daily chats and ideas +- Informal notes and thoughts about the project +- External information that may inform the project +- Project-level decisions (complements phase-specific ADRs) +- How-to guides and playbooks +- Glossary entries +- Design ideas and UX concepts + +--- + +## Directory Structure + +``` +kb/ +├── 01_projects/ # Project-specific notes +│ ├── tendril/ # Tendril project notes +│ └── _shared/ # Cross-project notes +├── 02_systems/ # Infrastructure, DevOps, tooling +├── 03_research/ # Informal research, links +├── 04_design/ # Product specs, UX, copy +├── 05_decisions/ # ADR-style decisions +├── 06_glossary/ # Terms, acronyms +├── 07_playbooks/ # How-to guides, SOPs +├── 08_archive/ # Superseded content +├── _guides/ # KB system guidelines (read-only) +│ └── KB_INGEST_PROMPT.md # System prompt for ingestion +├── _templates/ # File templates (read-only) +│ ├── note.md +│ ├── decision.md +│ └── howto.md +├── _inbox/ # Unclassified content (rare) +├── _review_queue/ # Low-confidence routing (<0.60) +├── _index.md # Auto-generated searchable index +├── CHANGELOG.md # KB change tracking +├── README.md # KB system documentation (this file) +└── scripts/ + └── generate-index.sh # Index generation script +``` + +--- + +## File Naming Schema + +**Pattern**: `YYYY-MM-DD--slug--type.md` + +### Components + +- **Date (YYYY-MM-DD)**: Date content was created or captured +- **Slug**: 3-8 lowercase words, hyphen-joined, no stop-words (a, an, the, and, or, but, etc.) +- **Type**: One of: `idea`, `note`, `spec`, `decision`, `howto`, `retro`, `meeting` +- **Multi-part files**: Append `--p1`, `--p2`, etc. for long content + +### Examples + +- ✅ `2025-01-27--api-authentication-flow--spec.md` +- ✅ `2025-01-27--user-onboarding-decision--decision.md` +- ✅ `2025-01-27--comprehensive-api-design--spec--p1.md` (multi-part) +- ❌ `api-design.md` (missing date and type) +- ❌ `2025-01-27--the-api--note.md` (contains stop-word "the") + +### Regex Pattern + +``` +^\d{4}-\d{2}-\d{2}--[a-z0-9-]{3,}--(idea|note|spec|decision|howto|retro|meeting)(--p[0-9]+)?\.md$ +``` + +--- + +## Frontmatter Schema + +Every KB file MUST have complete YAML frontmatter with all 18 required fields. + +### Required Fields (18) + +```yaml +--- +title: "Descriptive title in sentence case" +date: "YYYY-MM-DD" # Must match filename date +captured_at: "YYYY-MM-DD" # Same as date (project compatibility) +author: ["author name"] # Array, default ["unknown author"] +source: { kind: pulse|chat|web|doc, ref: "" } +source_type: voice|web|pdf|image|personal_note|chat|pulse +project: ["tendril"] # Array, default ["tendril"] for Tendril +related_projects: ["tendril"] # Same as project (compatibility) +topics: ["topic1", "topic2"] # Array of 2-5 topic keywords +tags: ["tag1", "tag2"] # Array of 2-8 tags +type: idea|note|spec|decision|howto|retro|meeting # Must match filename type +status: draft|active|archived +phase_relevance: ["phase-01", "phase-02"] # Array of relevant phases or [] +routing_hint: "Brief explanation of routing decision" +proposed_path: "kb/XX_category/" # Where LLM intends to file it +routing_confidence: 0.87 # Numeric 0.00-1.00 +related: [] # Array of related file paths +summary: "One to three sentence summary of the content" +--- +``` + +### Optional Fields + +```yaml +key_takeaways: ["takeaway1", "takeaway2"] +action_candidates: ["action1", "action2"] +``` + +### Field Guidelines + +- **title**: Clear, descriptive, sentence case (not title case) +- **date**: Must match filename date prefix (YYYY-MM-DD) +- **author**: Array format, use `["unknown author"]` if unknown +- **source**: Object with `kind` (pulse, chat, web, doc) and `ref` (link, ID, or identifier) +- **project**: Array, default `["tendril"]` for Tendril project +- **topics**: Array of 2-5 topic keywords (e.g., `["authentication", "security", "api"]`) +- **tags**: Array of 2-8 tags for cross-cutting categorization (e.g., `["backend", "urgent", "review-needed"]`) +- **type**: Must match filename type suffix exactly +- **status**: `draft` for new content, `active` for current/relevant, `archived` for old +- **routing_hint**: Brief explanation (1-2 sentences) of routing decision +- **proposed_path**: The directory path where the LLM intends to file the content +- **routing_confidence**: Numeric value between 0.00 and 1.00 indicating confidence in the routing decision +- **related**: Array of related file paths or references +- **summary**: Concise 1-3 sentence summary of the content +- **phase_relevance**: Array of relevant phases (e.g., `["phase-01", "phase-02"]`) or empty array `[]` + +--- + +## Routing Decision Tree + +Content must be routed to one of these directories based on its primary purpose: + +1. **Is it project-specific?** → `/kb/01_projects/tendril/` or `_shared/` +2. **Is it a how-to or procedure?** → `/kb/07_playbooks/` +3. **Is it a decision with rationale?** → `/kb/05_decisions/` +4. **Is it about infrastructure/tooling?** → `/kb/02_systems/` +5. **Is it research or external reference?** → `/kb/03_research/` +6. **Is it design/UX/copy?** → `/kb/04_design/` +7. **Is it a definition or term?** → `/kb/06_glossary/` +8. **Is it obsolete?** → `/kb/08_archive/` (only if truly superseded) +9. **Unclear?** → `/kb/_inbox/` (should be rare) + +### Routing Destinations + +- **`/kb/01_projects/tendril/`** - Tendril project-specific notes, tasks, or updates +- **`/kb/01_projects/_shared/`** - Cross-project notes that apply to multiple projects +- **`/kb/02_systems/`** - Infrastructure, DevOps, tooling, technical systems, architecture decisions +- **`/kb/03_research/`** - Links, papers, competitor notes, external research, learning materials +- **`/kb/04_design/`** - Product specs, UX design, copy, user experience, interface design +- **`/kb/05_decisions/`** - ADR-style decisions with rationale, architectural decisions, strategic choices +- **`/kb/06_glossary/`** - Terms, acronyms, definitions, vocabulary +- **`/kb/07_playbooks/`** - How-to guides, SOPs, runbooks, procedures, operational guides +- **`/kb/08_archive/`** - Old or superseded content (use sparingly, prefer deletion if truly obsolete) +- **`/kb/_inbox/`** - Use only if content cannot be clearly classified (should be rare) +- **`/kb/_review_queue/`** - Low-confidence routing items requiring human review (routing_confidence < 0.60) + +--- + +## Routing Confidence System + +After determining the proposed routing destination, you MUST assess your confidence level: + +- **0.90–1.00**: Crystal clear placement - content clearly matches category +- **0.75–0.89**: Good guess - content fits well but might have minor ambiguity +- **0.60–0.74**: Uncertain - content could fit multiple categories, needs review +- **<0.60**: Low confidence - unclear routing, send to `_review_queue/` instead of proposed path + +### Routing Policy + +If `routing_confidence < 0.60`, the file MUST be placed in `kb/_review_queue/` instead of the proposed path, but still include `proposed_path` in frontmatter for human review. + +--- + +## Usage Guidelines + +### Creating KB Files + +1. **Use templates**: Copy from `kb/_templates/` and fill in all required fields +2. **Follow naming convention**: `YYYY-MM-DD--slug--type.md` +3. **Complete frontmatter**: All 18 required fields must be present +4. **Route appropriately**: Use routing decision tree to determine correct category +5. **Assess confidence**: Set `routing_confidence` honestly (0.00-1.00) + +### KB Ingestion + +For LLM-assisted KB ingestion: + +1. Paste `/kb/_guides/KB_INGEST_PROMPT.md` contents as the first message in Cursor chat +2. Provide raw content after the system prompt +3. LLM will analyze and route content appropriately +4. Review generated JSON output before creating file +5. Update index and changelog after creating KB file + +See `kb/_guides/KB_INGEST_PROMPT.md` for complete ingestion instructions. + +### Index and Changelog Updates + +**MANDATORY**: After creating, modifying, or deleting ANY file in `kb/` (except `_guides/`, `_templates/`, `README.md`, `_index.md`, and `CHANGELOG.md`): + +1. **Run index generation script:** + ```bash + kb/scripts/generate-index.sh + ``` + - This regenerates `kb/_index.md` with current KB content + - Verify the script completes successfully + - Check that changes appear in `kb/_index.md` + +2. **Update KB changelog:** + - Add entry to `kb/CHANGELOG.md` with date-based format: + ```markdown + ## [YYYY-MM-DD] KB File Added/Modified/Deleted + + ### Added/Changed/Removed + - `kb/XX_category/filename.md` - [Brief description] + ``` + - Use today's date (YYYY-MM-DD format) + - Include the full file path relative to repository root + - Provide a brief description (1-2 sentences) of what changed + +3. **Commit all changes together:** + - Commit KB file change, index update, and changelog entry together + - Use commit message: `docs(kb): [action] [filename-slug] and update index/changelog` + +**NEVER skip these steps** - Both index and changelog updates are mandatory for KB system integrity. + +--- + +## Integration with Phase Documentation + +The KB system complements the phase documentation system: + +- **KB decisions** (`kb/05_decisions/`) complement phase-specific ADRs (`tendril/phases/phase-XX-name/decisions.md`) +- **KB entries** can reference project phases via `phase_relevance[]` field +- **KB research** (`kb/03_research/`) may inform phase planning and development +- **KB playbooks** (`kb/07_playbooks/`) provide operational guides for phase work + +--- + +## Index Generation + +The KB index (`kb/_index.md`) is automatically generated and provides: + +- File listing by category +- Topics index (all unique topics with file references) +- Tags index (all unique tags with file references) +- Phase relevance index (files organized by phase) + +### Manual Index Generation + +```bash +kb/scripts/generate-index.sh +``` + +### Automatic Index Updates + +Gitea Actions workflow (`.github/workflows/kb-index-update.yml`) will auto-update the index on push to main when KB files change. (To be configured in Phase 3) + +--- + +## Templates + +Templates are available in `kb/_templates/`: + +- **`note.md`** - General notes template +- **`decision.md`** - ADR-style decision template +- **`howto.md`** - How-to guide template + +All templates include complete frontmatter with all 18 required fields and placeholder values. + +--- + +## Related Documentation + +- **KB Ingestion Prompt**: `kb/_guides/KB_INGEST_PROMPT.md` - Complete system prompt for LLM-assisted KB ingestion +- **KB Changelog**: `kb/CHANGELOG.md` - Change tracking for KB system +- **KB Index**: `kb/_index.md` - Auto-generated searchable index +- **Phase Documentation**: `tendril/phases/phase-XX-name/` - Phase-specific documentation + +--- + +**Location**: `kb/` +**Maintained by**: Tendril Project Team +**Repository**: https://git.parkingmeter.info/Mycelium/tendril + diff --git a/kb/_guides/KB_INGEST_PROMPT.md b/kb/_guides/KB_INGEST_PROMPT.md new file mode 100644 index 0000000..18b430d --- /dev/null +++ b/kb/_guides/KB_INGEST_PROMPT.md @@ -0,0 +1,309 @@ +# KB Ingestion System Prompt + +**Use this prompt in Cursor before pasting content to be ingested into the Knowledge Base.** + +--- + +## System Instructions + +You are a Knowledge Base ingestion assistant. Your role is to analyze incoming content (from "Pulse Daily" chats, ideas, notes, etc.) and route it to the appropriate location in the `/kb/` directory structure with proper naming, frontmatter, and formatting. + +## Classification and Routing Rules + +Content must be routed to one of these directories based on its primary purpose: + +- **`/kb/01_projects/tendril/`** - Tendril project-specific notes, tasks, or updates +- **`/kb/01_projects/_shared/`** - Cross-project notes that apply to multiple projects +- **`/kb/02_systems/`** - Infrastructure, DevOps, tooling, technical systems, architecture decisions +- **`/kb/03_research/`** - Links, papers, competitor notes, external research, learning materials +- **`/kb/04_design/`** - Product specs, UX design, copy, user experience, interface design +- **`/kb/05_decisions/`** - ADR-style decisions with rationale, architectural decisions, strategic choices +- **`/kb/06_glossary/`** - Terms, acronyms, definitions, vocabulary +- **`/kb/07_playbooks/`** - How-to guides, SOPs, runbooks, procedures, operational guides +- **`/kb/08_archive/`** - Old or superseded content (use sparingly, prefer deletion if truly obsolete) +- **`/kb/_inbox/`** - Use only if content cannot be clearly classified (should be rare) +- **`/kb/_review_queue/`** - Low-confidence routing items requiring human review (routing_confidence < 0.60) + +### Routing Decision Tree + +1. **Is it project-specific?** → `/kb/01_projects/tendril/` or `_shared/` +2. **Is it a how-to or procedure?** → `/kb/07_playbooks/` +3. **Is it a decision with rationale?** → `/kb/05_decisions/` +4. **Is it about infrastructure/tooling?** → `/kb/02_systems/` +5. **Is it research or external reference?** → `/kb/03_research/` +6. **Is it design/UX/copy?** → `/kb/04_design/` +7. **Is it a definition or term?** → `/kb/06_glossary/` +8. **Is it obsolete?** → `/kb/08_archive/` (only if truly superseded) +9. **Unclear?** → `/kb/_inbox/` (should be rare) + +### Routing Confidence Assessment + +After determining the proposed routing destination, you MUST assess your confidence level: + +- **0.90–1.00:** Crystal clear placement - content clearly matches category, type, and purpose +- **0.75–0.89:** Good guess - content fits well but might have minor ambiguity +- **0.60–0.74:** Uncertain - content could fit multiple categories, needs review +- **<0.60:** Low confidence - unclear routing, send to `_review_queue/` instead of proposed path + +**Routing Policy:** If `routing_confidence < 0.60`, the file MUST be placed in `kb/_review_queue/` instead of the proposed path, but still include `proposed_path` in frontmatter for human review. + +## File Naming Standards + +**Format:** `YYYY-MM-DD--slug--type.md` + +### Components + +- **Date (YYYY-MM-DD):** Use the date the content was created or captured (today's date if unknown) +- **Slug:** 3-8 lowercase words, hyphen-joined, no stop-words (a, an, the, and, or, but, etc.) + - Good: `user-authentication-flow`, `api-rate-limiting-strategy`, `cursor-workflow-setup` + - Bad: `the-api`, `a-user`, `and-authentication` +- **Type:** One of: `idea`, `note`, `spec`, `decision`, `howto`, `retro`, `meeting` +- **Multi-part files:** If content is too long, append `--p1`, `--p2`, etc. to the filename + - Example: `2025-01-27--comprehensive-api-design--spec--p1.md` + +### Type Selection + +- **`idea`** - New concepts, proposals, brainstorming +- **`note`** - General notes, observations, thoughts +- **`spec`** - Specifications, requirements, detailed plans +- **`decision`** - ADR-style decisions (should go in `/kb/05_decisions/`) +- **`howto`** - Step-by-step guides (should go in `/kb/07_playbooks/`) +- **`retro`** - Retrospectives, post-mortems, lessons learned +- **`meeting`** - Meeting notes, summaries + +## Frontmatter Requirements + +Every KB file MUST have complete YAML frontmatter with all required fields. The base system requires 14 fields, but projects may add additional fields (like `captured_at`, `source_type`, `related_projects`, `phase_relevance` for Tendril). + +**Base Required Fields (14):** +```yaml +--- +title: "Descriptive title in sentence case" +date: "YYYY-MM-DD" +author: ["author name"] # Array, default to ["unknown author"] if unknown +source: { kind: pulse|chat|web|doc, ref: "" } +project: ["tendril"] # Array, default ["tendril"] for Tendril project +topics: [] # Array of topic keywords +tags: [] # Array of tags for categorization +type: idea|note|spec|decision|howto|retro|meeting +status: draft|active|archived +routing_hint: "Brief explanation of why this was routed here" +proposed_path: "kb/XX_category/" # Where LLM intends to file it +routing_confidence: 0.87 # Numeric value 0.00-1.00 +related: [] # Array of related file paths or references +summary: "One to three sentence summary of the content" +--- +``` + +**Optional Project-Specific Fields** (add as needed): +- `captured_at` - Same as date (for compatibility) +- `source_type` - voice|web|pdf|image|personal_note|chat|pulse +- `related_projects` - Same as project (for compatibility) +- `phase_relevance` - Array of relevant phases (e.g., `["phase-01", "phase-02"]`) +- `key_takeaways` - Array of key insights +- `action_candidates` - Array of potential action items + +### Field Guidelines + +- **title:** Clear, descriptive, sentence case (not title case) +- **date:** Must match filename date prefix (YYYY-MM-DD) +- **author:** Array format, use `["unknown author"]` if unknown +- **source:** Object with `kind` (pulse, chat, web, doc) and `ref` (link, ID, or identifier) +- **project:** Array, default `["tendril"]` for Tendril project, expandable for future projects +- **topics:** Array of 2-5 topic keywords (e.g., `["authentication", "security", "api"]`) +- **tags:** Array of 2-8 tags for cross-cutting categorization (e.g., `["backend", "urgent", "review-needed"]`) +- **type:** Must match filename type suffix exactly +- **status:** `draft` for new content, `active` for current/relevant, `archived` for old +- **routing_hint:** Brief explanation (1-2 sentences) of routing decision +- **proposed_path:** The directory path where the LLM intends to file the content +- **routing_confidence:** Numeric value between 0.00 and 1.00 indicating confidence in the routing decision +- **related:** Array of related file paths or references +- **summary:** Concise 1-3 sentence summary of the content + +## Output Format + +**CRITICAL:** You MUST output your response in a JSON code block with this exact structure: + +```json +{ + "file_path": "kb/XX_category/YYYY-MM-DD--slug--type.md", + "frontmatter": { + "title": "...", + "date": "YYYY-MM-DD", + "author": ["..."], + "source": { "kind": "...", "ref": "..." }, + "project": ["tendril"], + "topics": ["..."], + "tags": ["..."], + "type": "...", + "status": "...", + "routing_hint": "...", + "proposed_path": "kb/XX_category/", + "routing_confidence": 0.87, + "related": [], + "summary": "..." + }, + "content": "# Title\n\n[Markdown content here]" +} +``` + +**IMPORTANT:** The `file_path` in the JSON output MUST reflect the actual destination: +- If `routing_confidence >= 0.60`: Use the `proposed_path` as the actual `file_path` +- If `routing_confidence < 0.60`: Use `kb/_review_queue/` as the actual `file_path` (but still include `proposed_path` in frontmatter) + +## Quality and Style Guidelines + +1. **Content Quality:** + - Preserve original meaning and intent + - Improve clarity and structure without changing substance + - Use proper Markdown formatting (headers, lists, code blocks) + - Break long content into logical sections + +2. **Writing Style:** + - Use clear, concise language + - Prefer active voice + - Use proper grammar and spelling + - Maintain professional tone + +3. **Structure:** + - Start with a clear title (H1) + - Use H2 for major sections + - Use H3 for subsections + - Include a summary section if helpful + - Add "Related" section for cross-references + +4. **Markdown Best Practices:** + - Use fenced code blocks with language tags + - Use proper list formatting + - Use links for references + - Use emphasis (bold/italic) sparingly + +## Safety Constraints + +**NEVER:** +- Modify files in `/kb/_guides/` or `/kb/_templates/` +- Overwrite existing files without explicit user permission +- Create files outside `/kb/` directory structure +- Skip frontmatter validation +- Use invalid filename patterns +- Create duplicate files (check existing files first) + +**ALWAYS:** +- Validate filename matches pattern: `YYYY-MM-DD--slug--type.md` +- Ensure all required frontmatter fields are present +- Verify date in frontmatter matches filename date +- Verify type in frontmatter matches filename type +- Check for existing similar files to avoid duplicates +- Use appropriate routing based on content purpose +- Assess routing confidence honestly (0.00-1.00) +- Route low-confidence items (<0.60) to `kb/_review_queue/` instead of proposed path + +## Example Workflow + +1. User pastes content into Cursor +2. You analyze the content to determine: + - Primary purpose and routing destination + - Appropriate type (idea, note, spec, etc.) + - Key topics and tags + - Related content + - **Routing confidence level (0.00-1.00)** +3. You generate: + - Proper filename following naming standards + - Complete frontmatter with all required fields + - Well-structured Markdown content +4. You determine actual file path: + - If `routing_confidence >= 0.60`: Use `proposed_path` as actual destination + - If `routing_confidence < 0.60`: Use `kb/_review_queue/` as actual destination +5. You output JSON code block with file_path, frontmatter, and content +6. User (or automation) creates the file from your JSON output + +## Validation Checklist + +Before outputting JSON, verify: + +- [ ] Filename matches pattern: `YYYY-MM-DD--slug--type.md` +- [ ] All required frontmatter fields are present +- [ ] Date in frontmatter matches filename date +- [ ] Type in frontmatter matches filename type +- [ ] Routing confidence is assessed honestly (0.00-1.00) +- [ ] `proposed_path` is set to intended destination directory +- [ ] Actual `file_path` reflects routing policy (if confidence < 0.60, use `kb/_review_queue/`) +- [ ] Routing destination is appropriate for content +- [ ] Content is well-structured Markdown +- [ ] Summary accurately describes the content +- [ ] Topics and tags are relevant and useful +- [ ] Related references are valid (if any) + +--- + +## Completion Summary and Reporting + +**CRITICAL:** After outputting the JSON code block, you MUST also provide a completion summary in markdown format with the following sections: + +### What Was Accomplished + +- **File Created**: `kb/XX_category/YYYY-MM-DD--slug--type.md` +- **Routing Decision**: [Brief explanation of why this content was routed to this category] +- **Content Type**: [type] in [category] +- **File Status**: [draft/active/archived] + +### KB File Details + +- **Title**: [title from frontmatter] +- **Type**: [type from frontmatter] +- **Topics**: [list of topics from frontmatter] +- **Tags**: [list of tags from frontmatter] +- **Summary**: [summary from frontmatter] +- **Author**: [author from frontmatter] +- **Source**: [source kind and ref from frontmatter] + +### Possible Uses in Project + +- [How this KB can be used in project planning/development] +- [Related workflows or decisions this informs] +- [Potential integrations with other systems or documentation] +- [Who might benefit from this knowledge] + +### Local Document Update Report + +**MANDATORY STEPS:** After creating the KB file, you must: + +1. **Update Index:** + - Run: `kb/scripts/generate-index.sh` + - This regenerates `kb/_index.md` with the new file's metadata + - Verify the script completes successfully + - Check that the new file appears in `kb/_index.md` + +2. **Update Changelog:** + - Add entry to `kb/CHANGELOG.md` with the following format: + ```markdown + ## [YYYY-MM-DD] KB File Added + + ### Added + - `kb/XX_category/YYYY-MM-DD--slug--type.md` - [Brief description of content] + ``` + - Use today's date (YYYY-MM-DD format) + - Include the full file path relative to repository root + - Provide a brief description (1-2 sentences) of what the KB file contains + +3. **Commit Changes:** + - Commit all three changes together: + - The new KB file + - The updated `kb/_index.md` + - The updated `kb/CHANGELOG.md` + - Use commit message: `docs(kb): add [filename-slug] and update index/changelog` + - Example: `docs(kb): add api-auth-decision and update index/changelog` + +**Status Checklist:** +- [ ] KB file created at correct path +- [ ] Index updated: `kb/scripts/generate-index.sh` run successfully +- [ ] Changelog updated: Entry added to `kb/CHANGELOG.md` +- [ ] All changes committed together +- [ ] Commit message follows format: `docs(kb): add [filename-slug] and update index/changelog` + +**Note:** If you cannot run the index generation script or update the changelog automatically, clearly indicate what needs to be done manually in the "Local Document Update Report" section. + +--- + +**Remember:** The goal is to create well-organized, discoverable knowledge that can be easily found and referenced later. Take time to classify correctly and provide complete metadata. Always update the index and changelog after creating KB files. + diff --git a/kb/_index.md b/kb/_index.md new file mode 100644 index 0000000..ed57fd0 --- /dev/null +++ b/kb/_index.md @@ -0,0 +1,20 @@ +# KB Index + +_Last updated: 2025-11-11_ + +This index is automatically generated from KB file metadata. It provides searchable access to all KB content organized by category, topic, tag, and phase relevance. + +--- + +## File Listing by Category + +--- + +## Summary + +- **Total KB Files**: 0 +- **Unique Topics**: 0 +- **Unique Tags**: 0 +- **Phases Referenced**: 0 + +_Index generated on 2025-11-11 11:41:41_ diff --git a/kb/_templates/decision.md b/kb/_templates/decision.md new file mode 100644 index 0000000..f307d17 --- /dev/null +++ b/kb/_templates/decision.md @@ -0,0 +1,39 @@ +--- +title: "${TITLE}" +date: "${DATE}" +captured_at: "${DATE}" +author: ["${AUTHOR}"] +source: { kind: ${SRC_KIND}, ref: "${SRC_REF}" } +source_type: ${SOURCE_TYPE} +project: ["tendril"] +related_projects: ["tendril"] +topics: [${TOPICS}] +tags: [decision, ${TAGS}] +type: decision +status: active +phase_relevance: [${PHASE_RELEVANCE}] +routing_hint: "${ROUTING_HINT}" +proposed_path: "kb/05_decisions/" +routing_confidence: 0.90 +related: [${RELATED}] +summary: "${SUMMARY}" +key_takeaways: [] +action_candidates: [] +--- + +## Decision + +## Context + +## Options Considered + +## Rationale + +## Consequences + +## Key Takeaways + +## Action Candidates + +## Revisit Date + diff --git a/kb/_templates/howto.md b/kb/_templates/howto.md new file mode 100644 index 0000000..cb2fa9a --- /dev/null +++ b/kb/_templates/howto.md @@ -0,0 +1,39 @@ +--- +title: "${TITLE}" +date: "${DATE}" +captured_at: "${DATE}" +author: ["${AUTHOR}"] +source: { kind: ${SRC_KIND}, ref: "${SRC_REF}" } +source_type: ${SOURCE_TYPE} +project: ["tendril"] +related_projects: ["tendril"] +topics: [${TOPICS}] +tags: [howto, ${TAGS}] +type: howto +status: active +phase_relevance: [${PHASE_RELEVANCE}] +routing_hint: "${ROUTING_HINT}" +proposed_path: "kb/07_playbooks/" +routing_confidence: 0.90 +related: [${RELATED}] +summary: "${SUMMARY}" +key_takeaways: [] +action_candidates: [] +--- + +## Steps + +1. + +2. + +3. + +## Verification + +## Rollback + +## Key Takeaways + +## Action Candidates + diff --git a/kb/_templates/note.md b/kb/_templates/note.md new file mode 100644 index 0000000..f2970a5 --- /dev/null +++ b/kb/_templates/note.md @@ -0,0 +1,35 @@ +--- +title: "${TITLE}" +date: "${DATE}" +captured_at: "${DATE}" +author: ["${AUTHOR}"] +source: { kind: ${SRC_KIND}, ref: "${SRC_REF}" } +source_type: ${SOURCE_TYPE} +project: ["tendril"] +related_projects: ["tendril"] +topics: [${TOPICS}] +tags: [${TAGS}] +type: note +status: draft +phase_relevance: [${PHASE_RELEVANCE}] +routing_hint: "${ROUTING_HINT}" +proposed_path: "kb/XX_category/" +routing_confidence: 0.75 +related: [${RELATED}] +summary: "${SUMMARY}" +key_takeaways: [] +action_candidates: [] +--- + +# Context + +# Key Points + +# Key Takeaways + +# Action Candidates + +# Actions / Next Steps + +# References + diff --git a/kb/scripts/generate-index.sh b/kb/scripts/generate-index.sh new file mode 100644 index 0000000..5147e5d --- /dev/null +++ b/kb/scripts/generate-index.sh @@ -0,0 +1,270 @@ +#!/bin/bash + +# KB Index Generation Script +# Generates kb/_index.md with searchable metadata from all KB files + +set -e + +# Get the script directory and KB root directory +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +KB_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +INDEX_FILE="$KB_ROOT/_index.md" + +echo "Generating KB index..." + +# Create temporary files for indexing +TMP_DIR=$(mktemp -d 2>/dev/null || mktemp -d -t 'kb-index') +trap "rm -rf '$TMP_DIR'" EXIT + +TOPICS_FILE="$TMP_DIR/topics.txt" +TAGS_FILE="$TMP_DIR/tags.txt" +PHASES_FILE="$TMP_DIR/phases.txt" +FILES_FILE="$TMP_DIR/files.txt" + +touch "$TOPICS_FILE" "$TAGS_FILE" "$PHASES_FILE" "$FILES_FILE" + +# Categories to scan +CATEGORIES=("01_projects" "02_systems" "03_research" "04_design" "05_decisions" "06_glossary" "07_playbooks" "08_archive") + +# Function to extract frontmatter from a file +extract_frontmatter() { + local file="$1" + if [[ ! -f "$file" ]]; then + return 1 + fi + + # Extract content between first --- and second --- + awk '/^---$/{if(++count==2)exit} count==1' "$file" 2>/dev/null || echo "" +} + +# Function to extract a YAML field value (simple fields) +extract_yaml_simple() { + local frontmatter="$1" + local field="$2" + + echo "$frontmatter" | grep "^${field}:" | sed "s/^${field}:[[:space:]]*//" | sed 's/^["'\'']//;s/["'\'']$//' | head -1 +} + +# Function to extract array values from YAML +extract_yaml_array() { + local frontmatter="$1" + local field="$2" + + # Try to extract array - handle both single-line and multi-line + local array_content=$(echo "$frontmatter" | awk -v field="$field:" ' + BEGIN { in_array=0; found=0 } + $0 ~ "^" field { + found=1 + sub("^" field "[[:space:]]*", "") + if ($0 ~ /\[.*\]/) { + print $0 + exit + } + in_array=1 + next + } + in_array { + if ($0 ~ /^[^[:space:]]/ && $0 !~ /^-/ && $0 !~ /^\[/) { + in_array=0 + exit + } + if ($0 ~ /^-/ || $0 ~ /^\[/) { + print $0 + } + } + ') + + # Extract values from array + echo "$array_content" | grep -oE '["'\''][^"'\'']+["'\'']|[^, \[\]]+' | sed 's/^["'\'']//;s/["'\'']$//;s/^[[:space:]]*//;s/[[:space:]]*$//' | grep -v '^$' | grep -v '^\[' | grep -v '^\]' +} + +# Function to process a KB file +process_kb_file() { + local file="$1" + local relative_path="${file#$KB_ROOT/}" + local category="" + + # Determine category from path + for cat in "${CATEGORIES[@]}"; do + if [[ "$relative_path" == "$cat"/* ]]; then + category="$cat" + break + fi + done + + if [[ -z "$category" ]]; then + return 0 # Skip files not in known categories + fi + + # Extract frontmatter + local frontmatter=$(extract_frontmatter "$file") + if [[ -z "$frontmatter" ]]; then + echo "Warning: No frontmatter found in $relative_path" >&2 + return 0 + fi + + # Extract metadata + local title=$(extract_yaml_simple "$frontmatter" "title") + local date=$(extract_yaml_simple "$frontmatter" "date") + local type=$(extract_yaml_simple "$frontmatter" "type") + local summary=$(extract_yaml_simple "$frontmatter" "summary") + + # Store file info + echo "$category|$relative_path|$title|$date|$type|$summary" >> "$FILES_FILE" + + # Extract and index topics + local topics=$(extract_yaml_array "$frontmatter" "topics") + if [[ -n "$topics" ]]; then + while IFS= read -r topic; do + topic=$(echo "$topic" | xargs) + if [[ -n "$topic" ]]; then + echo "$topic|$relative_path" >> "$TOPICS_FILE" + fi + done <<< "$topics" + fi + + # Extract and index tags + local tags=$(extract_yaml_array "$frontmatter" "tags") + if [[ -n "$tags" ]]; then + while IFS= read -r tag; do + tag=$(echo "$tag" | xargs) + if [[ -n "$tag" ]]; then + echo "$tag|$relative_path" >> "$TAGS_FILE" + fi + done <<< "$tags" + fi + + # Extract and index phase relevance + local phases=$(extract_yaml_array "$frontmatter" "phase_relevance") + if [[ -n "$phases" ]]; then + while IFS= read -r phase; do + phase=$(echo "$phase" | xargs) + if [[ -n "$phase" ]]; then + echo "$phase|$relative_path" >> "$PHASES_FILE" + fi + done <<< "$phases" + fi +} + +# Scan all KB files +echo "Scanning KB files..." + +for category in "${CATEGORIES[@]}"; do + category_dir="$KB_ROOT/$category" + if [[ ! -d "$category_dir" ]]; then + continue + fi + + # Find all .md files in category + find "$category_dir" -type f -name "*.md" | while read -r file; do + # Skip if in a special subdirectory + if [[ "$file" == *"/_guides/"* ]] || \ + [[ "$file" == *"/_templates/"* ]] || \ + [[ "$file" == *"/_inbox/"* ]] || \ + [[ "$file" == *"/_review_queue/"* ]]; then + continue + fi + + # Check if filename matches KB pattern + filename=$(basename "$file") + if [[ "$filename" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}--[a-z0-9-]+--(idea|note|spec|decision|howto|retro|meeting)(--p[0-9]+)?\.md$ ]]; then + process_kb_file "$file" + fi + done +done + +# Count files +FILE_COUNT=$(wc -l < "$FILES_FILE" 2>/dev/null || echo "0") + +# Generate index file +echo "Generating index file..." + +{ + cat << EOF +# KB Index + +_Last updated: $(date +%Y-%m-%d)_ + +This index is automatically generated from KB file metadata. It provides searchable access to all KB content organized by category, topic, tag, and phase relevance. + +--- + +## File Listing by Category + +EOF + + # Output files by category + for category in "${CATEGORIES[@]}"; do + category_files=$(grep "^$category|" "$FILES_FILE" 2>/dev/null || true) + if [[ -n "$category_files" ]]; then + echo "### $category" + echo "" + while IFS='|' read -r cat path title date type summary; do + echo "- [\`$path\`]($path) - $title ($date, $type)" + done <<< "$category_files" + echo "" + fi + done + + # Topics Index + if [[ -s "$TOPICS_FILE" ]]; then + echo "## Topics Index" + echo "" + sort -u "$TOPICS_FILE" | cut -d'|' -f1 | sort -u | while read -r topic; do + echo "### $topic" + grep "^$topic|" "$TOPICS_FILE" | cut -d'|' -f2 | sort -u | while read -r file; do + echo "- [\`$file\`]($file)" + done + echo "" + done + fi + + # Tags Index + if [[ -s "$TAGS_FILE" ]]; then + echo "## Tags Index" + echo "" + sort -u "$TAGS_FILE" | cut -d'|' -f1 | sort -u | while read -r tag; do + echo "### $tag" + grep "^$tag|" "$TAGS_FILE" | cut -d'|' -f2 | sort -u | while read -r file; do + echo "- [\`$file\`]($file)" + done + echo "" + done + fi + + # Phase Relevance Index + if [[ -s "$PHASES_FILE" ]]; then + echo "## Phase Relevance Index" + echo "" + sort -u "$PHASES_FILE" | cut -d'|' -f1 | sort -u | while read -r phase; do + echo "### $phase" + grep "^$phase|" "$PHASES_FILE" | cut -d'|' -f2 | sort -u | while read -r file; do + echo "- [\`$file\`]($file)" + done + echo "" + done + fi + + # Summary + TOPIC_COUNT=$(cut -d'|' -f1 "$TOPICS_FILE" 2>/dev/null | sort -u | wc -l || echo "0") + TAG_COUNT=$(cut -d'|' -f1 "$TAGS_FILE" 2>/dev/null | sort -u | wc -l || echo "0") + PHASE_COUNT=$(cut -d'|' -f1 "$PHASES_FILE" 2>/dev/null | sort -u | wc -l || echo "0") + + echo "---" + echo "" + echo "## Summary" + echo "" + echo "- **Total KB Files**: $FILE_COUNT" + echo "- **Unique Topics**: $TOPIC_COUNT" + echo "- **Unique Tags**: $TAG_COUNT" + echo "- **Phases Referenced**: $PHASE_COUNT" + echo "" + echo "_Index generated on $(date +%Y-%m-%d\ %H:%M:%S)_" + +} > "$INDEX_FILE" + +echo "Index generated successfully: $INDEX_FILE" +echo " - Files indexed: $FILE_COUNT" +echo " - Topics: $TOPIC_COUNT" +echo " - Tags: $TAG_COUNT" +echo " - Phases: $PHASE_COUNT"