知识库

好的，这是完整的 Markdown 文档，你可以直接复制粘贴保存。

---

PostgreSQL + 文件系统 混合知识库方案

一、整体架构

```
┌─────────────────────────────────────────────────────────────────────────┐
│ OpenCode / Agent │
│ （统一的检索与推理入口） │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ 统一知识库 API 层（MCP Server） │
│ 提供统一的检索接口：/search、/get_detail、/关联查询 │
└─────────────────────────────────────────────────────────────────────────┘
│
┌────────────────────────┼────────────────────────┐
▼ ▼ ▼
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Bug 数据源 │ │ Spec 数据源 │ │ 代码数据源 │
│ (MD + 图片) │ │ (PDF + 图片) │ │ (源码 + 二进制) │
└───────────────────┘ └───────────────────┘ └───────────────────┘
│ │ │
└────────────────────────┼────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ PostgreSQL（结构化元数据 + 索引层） │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ 统一知识条目表（knowledge_entries） │ │
│ │ - 所有数据源统一存储 │ │
│ │ - source_type 区分类型 │ │
│ │ - metadata JSONB 存储各类型特有字段 │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ 粗索引（倒排索引 + 全文索引 + 向量索引） │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ 关联关系表（知识图谱） │ │
│ └─────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ 文件系统（/data/kb/） │
│ ├── bugs/ ← Bug MD + 图片 │
│ ├── specs/ ← Spec PDF + 缓存 │
│ ├── firmware/ ← 固件源码/二进制 │
│ ├── cache/ ← 预解析缓存（PDF全文、图片缩略图、OCR结果） │
│ └── .metadata/ ← 全量索引缓存（加速启动） │
└─────────────────────────────────────────────────────────────────────────┘
```

---

二、数据库结构详解（完整DDL）

2.1 核心表：统一知识条目

这是整个知识库的唯一事实来源，所有数据源都写入这张表。

```sql
-- 核心统一知识条目表
CREATE TABLE knowledge_entries (
-- ====== 主键与类型 ======
id TEXT PRIMARY KEY, -- KB-2026-001
source_type TEXT NOT NULL, -- bug | spec | firmware | log | testcase
source_ref TEXT, -- 原始ID，如 BUG-2026-001

-- ====== 内容字段 ======
title TEXT NOT NULL,
content TEXT, -- 全文内容（用于向量化）
content_hash TEXT, -- MD5，用于检测变更
summary TEXT, -- 摘要（200字以内）

-- ====== 分类与版本 ======
module TEXT, -- 所属模块：auth-service
sub_module TEXT, -- 子模块：token-manager
version TEXT, -- 版本：v2.3.1
severity TEXT, -- P0/P1/Critical/High

-- ====== 检索关键字段 ======
error_codes TEXT[], -- ['500', 'ETIMEDOUT']
keywords TEXT[], -- ['timeout', 'redis']
log_pattern TEXT, -- 正则指纹
log_sample TEXT, -- 典型日志样本

-- ====== 解决方案与关联 ======
root_cause TEXT,
solution TEXT,
exclusion_rules TEXT, -- 误判排除条件
spec_ref TEXT[], -- 关联的Spec ID列表

-- ====== 存储路径 ======
md_path TEXT, -- MD文件路径
attachments_path TEXT[], -- 附件路径列表

-- ====== 元数据（JSON，存储各类型特有字段） ======
metadata JSONB DEFAULT '{}'::jsonb, -- 灵活存储特有信息

-- ====== 统计信息 ======
occurrence_count INT DEFAULT 1,
view_count INT DEFAULT 0,
useful_count INT DEFAULT 0, -- 点赞/有用数

-- ====== 时间戳 ======
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
resolved_at TIMESTAMP,
last_accessed_at TIMESTAMP
);

-- ====== 索引（加速查询） ======
CREATE INDEX idx_entries_source_type ON knowledge_entries(source_type);
CREATE INDEX idx_entries_module ON knowledge_entries(module);
CREATE INDEX idx_entries_severity ON knowledge_entries(severity);
CREATE INDEX idx_entries_version ON knowledge_entries(version);
CREATE INDEX idx_entries_error_codes ON knowledge_entries USING GIN (error_codes);
CREATE INDEX idx_entries_keywords ON knowledge_entries USING GIN (keywords);
CREATE INDEX idx_entries_created_at ON knowledge_entries(created_at DESC);

-- 全文检索索引
CREATE INDEX idx_entries_search ON knowledge_entries USING GIN (
setweight(to_tsvector('simple', COALESCE(title, '')), 'A') ||
setweight(to_tsvector('simple', COALESCE(content, '')), 'B') ||
setweight(to_tsvector('simple', COALESCE(root_cause, '')), 'C')
);
```

2.2 向量表（语义检索）

```sql
-- 启用pgvector
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE knowledge_embeddings (
entry_id TEXT REFERENCES knowledge_entries(id) ON DELETE CASCADE,
embedding_type TEXT CHECK (embedding_type IN ('title', 'full_text', 'log_sample', 'summary')),
embedding vector(768), -- bge-m3 维度
model_name TEXT DEFAULT 'BAAI/bge-m3',
created_at TIMESTAMP DEFAULT NOW(),
PRIMARY KEY (entry_id, embedding_type)
);

-- 向量索引（IVFFlat，加速检索）
CREATE INDEX idx_embeddings_vector ON knowledge_embeddings
USING ivfflat (embedding vector_cosine_ops) WITH (lists = 200);
```

2.3 附件表（统一管理图片、二进制等）

```sql
CREATE TABLE knowledge_attachments (
id SERIAL PRIMARY KEY,
entry_id TEXT REFERENCES knowledge_entries(id) ON DELETE CASCADE,

-- 文件信息
filename TEXT,
file_path TEXT, -- 相对路径
file_type TEXT, -- image | binary | source_code | pdf
mime_type TEXT,
file_size INT,
content_hash TEXT,

-- 图片特有（OCR/描述）
ocr_text TEXT,
image_description TEXT,

-- 版本信息
version TEXT,
uploaded_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_attachments_entry_id ON knowledge_attachments(entry_id);
CREATE INDEX idx_attachments_file_type ON knowledge_attachments(file_type);
```

2.4 关联关系表（知识图谱）

```sql
CREATE TABLE knowledge_relations (
id SERIAL PRIMARY KEY,
source_id TEXT REFERENCES knowledge_entries(id) ON DELETE CASCADE,
target_id TEXT REFERENCES knowledge_entries(id) ON DELETE CASCADE,
relation_type TEXT CHECK (relation_type IN (
'fixes', -- Bug修复了代码问题
'references', -- 引用
'contains', -- 包含
'depends_on', -- 依赖
'calls', -- 函数调用
'triggers', -- 触发
'related_to', -- 相关
'duplicates' -- 重复
)),
confidence FLOAT CHECK (confidence >= 0 AND confidence <= 1),
evidence TEXT, -- 关联证据
created_at TIMESTAMP DEFAULT NOW(),
UNIQUE (source_id, target_id, relation_type)
);

CREATE INDEX idx_relations_source ON knowledge_relations(source_id);
CREATE INDEX idx_relations_target ON knowledge_relations(target_id);
CREATE INDEX idx_relations_type ON knowledge_relations(relation_type);
```

2.5 数据源配置表（管理扩展）

```sql
CREATE TABLE data_sources (
source_type TEXT PRIMARY KEY,
display_name TEXT,
parser_module TEXT, -- Python模块路径
enabled BOOLEAN DEFAULT TRUE,
config JSONB DEFAULT '{}'::jsonb, -- Parser配置
last_sync TIMESTAMP,
sync_interval INT, -- 秒
total_entries INT DEFAULT 0,
created_at TIMESTAMP DEFAULT NOW()
);

-- 初始化数据源配置
INSERT INTO data_sources (source_type, display_name, parser_module) VALUES
('bug', 'Bug报告', 'parsers.bug_parser.BugParser'),
('spec', 'Spec文档', 'parsers.spec_parser.SpecParser'),
('firmware', '固件代码', 'parsers.firmware_parser.FirmwareParser');
```

---

三、不同数据源的扩展实现

3.1 统一Parser接口

每种数据源实现一个Parser，输出统一格式：

```python
# parsers/base_parser.py
from abc import ABC, abstractmethod
from typing import Dict, List, Optional

class BaseParser(ABC):
"""所有Parser的基类"""

@abstractmethod
def parse(self, file_path: str, **kwargs) -> Dict:
"""
解析文件，返回统一格式

返回格式：
{
"source_type": "bug|spec|firmware",
"source_ref": "原始ID",
"title": "标题",
"content": "全文内容",
"summary": "摘要（200字）",
"module": "模块",
"version": "版本",
"severity": "严重级别",
"error_codes": ["错误码列表"],
"keywords": ["关键词列表"],
"log_pattern": "正则指纹",
"log_sample": "典型日志",
"root_cause": "根因",
"solution": "解决方案",
"exclusion_rules": "排除条件",
"spec_ref": ["关联Spec"],
"metadata": {"额外字段": "值"},
"attachments": [
{"filename": "a.png", "data": b"..."}
]
}
"""
pass

@abstractmethod
def detect(self, file_path: str) -> bool:
"""检测是否能解析该文件"""
pass
```

3.2 Bug Parser实现

```python
# parsers/bug_parser.py
from parsers.base_parser import BaseParser
import frontmatter
import hashlib
from pathlib import Path

class BugParser(BaseParser):

def detect(self, file_path: str) -> bool:
return file_path.endswith('.md') and 'BUG-' in file_path

def parse(self, file_path: str, **kwargs) -> Dict:
path = Path(file_path)
post = frontmatter.load(path)
meta = post.metadata

# 处理图片
attachments = []
images_dir = path.parent / 'images'
if images_dir.exists():
for img_path in images_dir.glob('*.{png,jpg,jpeg}'):
attachments.append({
"filename": img_path.name,
"data": img_path.read_bytes(),
"ocr_text": self._ocr_image(img_path) # OCR识别
})

return {
"source_type": "bug",
"source_ref": meta.get('id', path.stem),
"title": meta.get('title', ''),
"content": post.content,
"summary": post.content[:200],
"module": meta.get('module'),
"version": meta.get('version', '1.0'),
"severity": meta.get('severity'),
"error_codes": meta.get('error_codes', []),
"keywords": meta.get('keywords', []),
"log_pattern": meta.get('log_pattern'),
"log_sample": meta.get('log_sample'),
"root_cause": meta.get('root_cause'),
"solution": meta.get('solution'),
"exclusion_rules": meta.get('exclusion_rules'),
"spec_ref": meta.get('spec_ref', []),
"metadata": {
"status": meta.get('status', 'open'),
"reporter": meta.get('reporter'),
"assignee": meta.get('assignee')
},
"attachments": attachments
}

def _ocr_image(self, img_path):
# OCR实现
import pytesseract
from PIL import Image
return pytesseract.image_to_string(Image.open(img_path))
```

3.3 Spec Parser实现

```python
# parsers/spec_parser.py
from parsers.base_parser import BaseParser
import hashlib
import json
from pathlib import Path
import fitz # PyMuPDF

class SpecParser(BaseParser):

def detect(self, file_path: str) -> bool:
return file_path.endswith('.pdf') or file_path.endswith('.docx')

def parse(self, file_path: str, **kwargs) -> Dict:
path = Path(file_path)

if path.suffix == '.pdf':
return self._parse_pdf(path)
else:
return self._parse_docx(path)

def _parse_pdf(self, pdf_path: Path) -> Dict:
doc = fitz.open(pdf_path)
full_text = []
attachments = []

for page_num in range(len(doc)):
page = doc[page_num]
text = page.get_text()
full_text.append(f"=== 第{page_num+1}页 ===\n{text}")

# 提取图片
for img_idx, img in enumerate(page.get_images()):
try:
xref = img[0]
base_image = doc.extract_image(xref)
attachments.append({
"filename": f"page{page_num+1}_img{img_idx}.png",
"data": base_image["image"],
"page": page_num + 1
})
except Exception:
pass

doc.close()

# 从内容提取spec_id
content = "\n".join(full_text)
spec_id = self._extract_spec_id(content, pdf_path.stem)

# 生成全文缓存（避免重复解析）
cache_path = Path("/data/kb/cache/specs") / f"{spec_id}.json"
cache_path.parent.mkdir(parents=True, exist_ok=True)
with open(cache_path, 'w') as f:
json.dump({
"spec_id": spec_id,
"full_text": content,
"page_count": len(doc)
}, f)

return {
"source_type": "spec",
"source_ref": spec_id,
"title": f"Spec {spec_id}",
"content": content,
"summary": content[:300] + "...",
"module": self._extract_module(content),
"version": self._extract_version(content),
"severity": "Info", # Spec通常无严重级别
"error_codes": [],
"keywords": self._extract_keywords(content),
"log_pattern": None,
"log_sample": None,
"root_cause": None,
"solution": None,
"exclusion_rules": None,
"spec_ref": [],
"metadata": {
"page_count": len(doc),
"pdf_path": str(pdf_path),
"cache_path": str(cache_path)
},
"attachments": attachments
}

def _extract_spec_id(self, content: str, fallback: str) -> str:
import re
# 匹配 "SPEC-XXX" 或 "文档编号：XXX"
patterns = [
r'(SPEC-[A-Z0-9]+)',
r'文档编号[:：]\s*([A-Z0-9-]+)',
r'ID[:：]\s*([A-Z0-9-]+)'
]
for pattern in patterns:
match = re.search(pattern, content, re.IGNORECASE)
if match:
return match.group(1)
return f"SPEC-{fallback}"

def _extract_module(self, content: str) -> str:
import re
match = re.search(r'模块[:：]\s*([^\n]+)', content)
return match.group(1).strip() if match else 'unknown'

def _extract_version(self, content: str) -> str:
import re
match = re.search(r'版本[:：]\s*v?([\d.]+)', content)
return match.group(1) if match else '1.0'

def _extract_keywords(self, content: str) -> list:
# 简单的关键词提取（可用TF-IDF或KeyBERT）
import re
# 提取所有大写缩写词
acronyms = re.findall(r'\b([A-Z]{2,})\b', content)
# 提取所有带引号的术语
terms = re.findall(r'["「]([^"」]+)["」"]', content)
return list(set(acronyms + terms))[:20]
```

3.4 Firmware Parser实现

```python
# parsers/firmware_parser.py
from parsers.base_parser import BaseParser
import hashlib
import re
from pathlib import Path

class FirmwareParser(BaseParser):

def detect(self, file_path: str) -> bool:
ext = Path(file_path).suffix
return ext in ['.c', '.cpp', '.h', '.hpp', '.bin', '.elf', '.hex']

def parse(self, file_path: str, **kwargs) -> Dict:
path = Path(file_path)

if path.suffix in ['.c', '.cpp', '.h', '.hpp']:
return self._parse_source(path)
else:
return self._parse_binary(path)

def _parse_source(self, src_path: Path) -> Dict:
with open(src_path, 'r', encoding='utf-8', errors='ignore') as f:
code = f.read()

# 提取函数
functions = self._extract_functions(code)

# 提取错误字符串
error_strings = self._extract_error_strings(code)

# 提取调用关系
call_graph = self._extract_call_graph(code, functions)

return {
"source_type": "firmware",
"source_ref": f"FW-{src_path.stem}",
"title": f"固件代码: {src_path.name}",
"content": code,
"summary": f"文件: {src_path.name}, 函数数: {len(functions)}",
"module": src_path.parent.name,
"version": kwargs.get('version', '1.0'),
"severity": "Medium",
"error_codes": [],
"keywords": list(set([f['name'] for f in functions[:20]])),
"log_pattern": None,
"log_sample": None,
"root_cause": None,
"solution": None,
"exclusion_rules": None,
"spec_ref": [],
"metadata": {
"file_type": "source_code",
"language": "C",
"functions": functions,
"call_graph": call_graph,
"error_strings": error_strings,
"total_lines": len(code.split('\n'))
},
"attachments": [{
"filename": src_path.name,
"data": code.encode('utf-8'),
"file_type": "source_code"
}]
}

def _extract_functions(self, code: str) -> list:
pattern = re.compile(
r'(?:static\s+)?(?:inline\s+)?([a-zA-Z_]\w*)\s+([a-zA-Z_]\w*)\s*\(([^)]*)\)\s*\{',
re.MULTILINE
)
functions = []
for match in pattern.finditer(code):
functions.append({
"return_type": match.group(1),
"name": match.group(2),
"params": match.group(3).split(',') if match.group(3) else []
})
return functions

def _extract_error_strings(self, code: str) -> list:
# 提取错误日志字符串
pattern = re.compile(r'(?:LOG_ERROR|log_error|printf)\s*\(\s*"([^"]+)"')
return pattern.findall(code)

def _extract_call_graph(self, code: str, functions: list) -> dict:
call_graph = {}
func_names = [f['name'] for f in functions]

for func in func_names:
# 在函数体中找到被调用的函数名（简化版）
pattern = re.compile(r'\b(' + '|'.join(func_names) + r')\s*\(')
# 这里需要找到该函数的位置，简化实现略
call_graph[func] = []

return call_graph

def _parse_binary(self, bin_path: Path) -> Dict:
"""解析二进制固件"""
import subprocess

# 使用strings提取可读字符串
try:
result = subprocess.run(
['strings', '-n', '4', str(bin_path)],
capture_output=True, text=True, timeout=10
)
strings = result.stdout.splitlines()
except Exception:
strings = []

return {
"source_type": "firmware",
"source_ref": f"FW-BIN-{bin_path.stem}",
"title": f"固件二进制: {bin_path.name}",
"content": "\n".join(strings[:1000]),
"summary": f"二进制固件: {bin_path.name}, 提取字符串数: {len(strings)}",
"module": bin_path.parent.name,
"version": "1.0",
"severity": "Medium",
"error_codes": [],
"keywords": list(set([s[:20] for s in strings[:50]])),
"log_pattern": None,
"log_sample": None,
"root_cause": None,
"solution": None,
"exclusion_rules": None,
"spec_ref": [],
"metadata": {
"file_type": "binary",
"strings_count": len(strings),
"sample_strings": strings[:20]
},
"attachments": []
}
```

---

四、粗索引的完整实现

粗索引由三部分组成：倒排索引、全文索引、向量索引。

4.1 倒排索引（关键词 → 条目ID）

```sql
-- 自动维护的倒排索引（通过触发器或应用层维护）
-- 方法：在knowledge_entries表中，keywords字段本身就是一个倒排索引（GIN）

-- 查询示例：查找包含任意关键词的条目
SELECT id, title, source_type, module, severity,
array_length(array_intersect(keywords, ARRAY['timeout', 'redis', 'connection']), 1) AS match_count
FROM knowledge_entries
WHERE keywords && ARRAY['timeout', 'redis', 'connection']
ORDER BY match_count DESC, occurrence_count DESC
LIMIT 20;

-- 如果需要更复杂的倒排索引，可以单独建表
CREATE TABLE inverted_index (
keyword TEXT,
entry_id TEXT REFERENCES knowledge_entries(id) ON DELETE CASCADE,
weight FLOAT DEFAULT 1.0,
position INT,
PRIMARY KEY (keyword, entry_id)
);

-- 插入示例
INSERT INTO inverted_index (keyword, entry_id, weight)
SELECT
unnest(keywords) AS keyword,
id,
1.0
FROM knowledge_entries;
```

4.2 全文检索索引（FTS）

```sql
-- 已在knowledge_entries表上创建了GIN全文索引
-- 查询示例

-- 简单全文检索
SELECT
id, title, source_type, module,
ts_rank(search_document, plainto_tsquery('simple', 'timeout connection redis')) AS rank
FROM knowledge_entries
WHERE search_document @@ plainto_tsquery('simple', 'timeout connection redis')
ORDER BY rank DESC
LIMIT 20;

-- 带权重过滤的全文检索（标题权重高）
SELECT
id, title, source_type,
ts_rank(
setweight(to_tsvector('simple', COALESCE(title, '')), 'A') ||
setweight(to_tsvector('simple', COALESCE(content, '')), 'B'),
plainto_tsquery('simple', 'timeout connection')
) AS rank
FROM knowledge_entries
WHERE
setweight(to_tsvector('simple', COALESCE(title, '')), 'A') ||
setweight(to_tsvector('simple', COALESCE(content, '')), 'B')
@@ plainto_tsquery('simple', 'timeout connection')
ORDER BY rank DESC
LIMIT 20;
```

4.3 向量索引（语义检索）

```sql
-- 查询最相似的条目
SELECT
e.id,
e.title,
e.source_type,
e.module,
1 - (emb.embedding <=> '[0.123, -0.456, ...]'::vector) AS similarity
FROM knowledge_embeddings emb
JOIN knowledge_entries e ON emb.entry_id = e.id
WHERE emb.embedding_type = 'full_text'
ORDER BY emb.embedding <=> '[0.123, -0.456, ...]'::vector
LIMIT 20;
```

4.4 粗索引的统一检索入口

```python
class CoarseIndex:
"""粗索引的统一检索接口"""

def __init__(self, db_conn):
self.db = db_conn

async def search(self, query: str, top_k: int = 20, filters: dict = None) -> list:
"""多路召回 + 融合排序"""
# 1. 提取关键词
keywords = self._extract_keywords(query)

# 2. 生成查询向量
query_vec = self._embed(query)

# 3. 并行执行三种检索
keyword_results = await self._keyword_search(keywords, filters)
fts_results = await self._fts_search(query, filters)
vector_results = await self._vector_search(query_vec, filters)

# 4. 融合排序（RRF - Reciprocal Rank Fusion）
scores = {}
for rank, (entry_id, score) in enumerate(keyword_results):
scores[entry_id] = scores.get(entry_id, 0) + 1 / (60 + rank + 1)
for rank, (entry_id, score) in enumerate(fts_results):
scores[entry_id] = scores.get(entry_id, 0) + 1 / (60 + rank + 1)
for rank, (entry_id, score) in enumerate(vector_results):
scores[entry_id] = scores.get(entry_id, 0) + 1 / (60 + rank + 1)

# 5. 返回TopK
sorted_results = sorted(scores.items(), key=lambda x: x[1], reverse=True)
return sorted_results[:top_k]

async def _keyword_search(self, keywords: list, filters: dict) -> list:
"""关键词检索（倒排索引）"""
conditions = ["keywords && $1::TEXT[]"]
params = [keywords]

if filters:
if 'module' in filters:
conditions.append(f"module = ${len(params) + 1}")
params.append(filters['module'])
if 'source_type' in filters:
conditions.append(f"source_type = ${len(params) + 1}")
params.append(filters['source_type'])
if 'severity' in filters:
conditions.append(f"severity = ${len(params) + 1}")
params.append(filters['severity'])

sql = f"""
SELECT id,
array_length(array_intersect(keywords, $1::TEXT[]), 1) AS match_count
FROM knowledge_entries
WHERE {' AND '.join(conditions)}
ORDER BY match_count DESC, occurrence_count DESC
LIMIT 30
"""
rows = await self.db.fetch(sql, *params)
return [(row['id'], row['match_count']) for row in rows]

async def _fts_search(self, query: str, filters: dict) -> list:
"""全文检索"""
conditions = ["search_document @@ plainto_tsquery('simple', $1)"]
params = [query]

if filters:
if 'module' in filters:
conditions.append(f"module = ${len(params) + 1}")
params.append(filters['module'])
# ... 其他过滤条件

sql = f"""
SELECT id, ts_rank(search_document, plainto_tsquery('simple', $1)) AS rank
FROM knowledge_entries
WHERE {' AND '.join(conditions)}
ORDER BY rank DESC
LIMIT 30
"""
rows = await self.db.fetch(sql, *params)
return [(row['id'], row['rank']) for row in rows]

async def _vector_search(self, query_vec: list, filters: dict) -> list:
"""向量检索"""
# 如果filters有过滤条件，需要JOIN knowledge_entries
sql = """
SELECT e.id, 1 - (emb.embedding <=> $1::vector) AS similarity
FROM knowledge_embeddings emb
JOIN knowledge_entries e ON emb.entry_id = e.id
WHERE emb.embedding_type = 'full_text'
"""
params = [query_vec]

if filters:
if 'module' in filters:
sql += f" AND e.module = ${len(params) + 1}"
params.append(filters['module'])
if 'source_type' in filters:
sql += f" AND e.source_type = ${len(params) + 1}"
params.append(filters['source_type'])

sql += " ORDER BY emb.embedding <=> $1::vector LIMIT 30"

rows = await self.db.fetch(sql, *params)
return [(row['id'], row['similarity']) for row in rows]

def _extract_keywords(self, text: str) -> list:
import re
# 提取错误码
codes = re.findall(r'\b(4\d{2}|5\d{2}|[A-Z_]{3,})\b', text)
# 提取关键词（名词/动词）
tokens = re.findall(r'\b([a-zA-Z]{3,})\b', text)
# 去重，取前10个
return list(set(codes + tokens))[:10]

def _embed(self, text: str) -> list:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('BAAI/bge-m3')
return model.encode(text).tolist()
```

---

五、文件系统目录结构

```
/data/kb/
│
├── bugs/ ← Bug数据源
│ ├── BUG-2026-001/
│ │ ├── report.md ← Bug正文
│ │ ├── images/
│ │ │ ├── vt_diagram.png ← VT图
│ │ │ └── screenshot.png
│ │ └── attachments/
│ │ └── trace.pcap
│ └── BUG-2026-002/
│ └── ...
│
├── specs/ ← Spec数据源
│ ├── auth/
│ │ ├── auth-spec.pdf
│ │ └── auth-spec-v2.pdf
│ └── payment/
│ └── payment-spec.pdf
│
├── firmware/ ← 固件代码数据源
│ ├── auth-service/
│ │ ├── token.c
│ │ ├── token.h
│ │ └── firmware.bin
│ └── network/
│ └── tcp_handler.c
│
├── cache/ ← 预解析缓存
│ ├── specs/
│ │ └── SPEC-AUTH-3.2.json ← PDF解析缓存（全文+图片）
│ ├── images/
│ │ └── thumbnails/ ← 图片缩略图
│ └── embeddings/
│ └── cache_meta.json ← 向量缓存元信息
│
└── .metadata/ ← 粗索引缓存（加速启动）
├── index.json ← 全量索引
├── embeddings.npy ← 向量矩阵
└── cache_meta.json ← 缓存版本信息
```

---

六、数据流总结

```
新数据文件（MD/PDF/源码）
▼
文件系统写入
▼
触发 Parser（根据文件类型自动选择）
▼
生成统一格式的条目
▼
写入 knowledge_entries（主表）
▼
生成向量 → 写入 knowledge_embeddings
▼
更新倒排索引（keywords字段自动索引）
▼
更新全文索引（search_document字段自动更新）
▼
（异步）更新 .metadata 缓存
▼
Agent可检索
```

---

七、核心优势总结

特性 实现方式
多数据源扩展 统一 knowledge_entries 表 + source_type 区分 + metadata JSONB
粗索引检索 倒排（GIN）+ 全文（FTS）+ 向量（pgvector）三路召回
文件管理 文件系统存大文件，数据库存元数据+路径
查询性能 三路索引并行 + RRF融合，毫秒级响应
缓存加速 .metadata/ 全量缓存，Agent启动即加载