通过litestream 进行sqlite-vec 数据备份以及恢复

实际上就是一个简单的测试，litestream 支持流式复制，比较适合对于sqlite进行备份，同时litestream 还支持对象存储的remote 模式，比较方便

环境准备

• minio

services: 
  s3:
     image: minio/minio
     command: server /data --console-address ":9001"
     environment:
       MINIO_ROOT_USER: minioadmin
       MINIO_ROOT_PASSWORD: minioadmin
     ports:
        - "9000:9000"
        - "9001:9001"

• litestream 配置

dbs:
  - path: ./examplev2.db
    replicas:
      - type: s3
        bucket: demo
        endpoint: http://localhost:9000
        path: db/
        access-key-id: minioadmin
        secret-access-key: minioadmin
        region: us-east-1
        force-path-style: true  # 对 MinIO 必须开启

使用

• 启动服务

./litestream replicate --config ./litestream.yml

• sqlite-vec db 操作

import sqlite3
import numpy as np
import sqlite_vec
from sqlite_vec import serialize_float32
db = sqlite3.connect("examplev2.db")
db.enable_load_extension(True)
sqlite_vec.load(db)
db.enable_load_extension(False)
vector = np.random.rand(2).astype(np.float32)
vector = [1,8]
db.execute('''
CREATE virtual TABLE IF NOT EXISTS vec_documents using vec0 (id INTEGER PRIMARY KEY,embedding FLOAT[2],name text)
''')
# 插入数据
db.execute(
    "INSERT INTO vec_documents (embedding,name) VALUES (?,?)",
    (serialize_float32(vector),"example_name")
)
result = db.execute('''
    select * from vec_documents where embedding=?;
''', (serialize_float32(vector),))

print(result.fetchall())
db.commit()
db.close()

• 复制效果

• 恢复操作

可以通过cli 使用s3的数据进行恢复

./litestream restore -config ./litestream.yml examplev2.db 

效果

说明

如果需要对于基于sqlite-vec 的数据进行备份以及恢复litestream是一个不错的选择，灵活方便，当然sqlite 备份以及恢复工具实际也有不少呢，litefs 也是一个不错的选择

参考资料

https://github.com/asg017/sqlite-vec

https://litestream.io/reference/config/

https://github.com/benbjohnson/litestream

https://litestream.io/reference/restore/

https://litestream.io/reference/replicate/