liteparse 的可视化引用

官方提供了一个示例，可以方便的进行显示，处理机制，先解析，转换格式（使用图片快照），搜索，基于搜索以及位置进行格式化显示

参考代码

import { LiteParse, searchItems } from "@llamaindex/liteparse";
import sharp from "sharp";

const DPI = 150;
const SCALE = DPI / 72;

async function main() {
  const parser = new LiteParse({ 
    outputFormat: "json",
    ocrServerUrl: "http://localhost:8000/ocr", // 使用自定义的ocr 服务
    dpi: DPI 
  });

  const result = await parser.parse("ch_en_num.jpg");
  // 图片快照
  const screenshots = await parser.screenshot("ch_en_num.jpg");
  const pages = ((result as any).json?.pages ?? (result as any).pages ?? []) as Array<{
    pageNum: number;
    textItems: any[];
  }>;

  // Search for a phrase, grouped by page
  const query = "符合国标";
  const hitsByPage = new Map<number, Array<{ x: number; y: number; width: number; height: number }>>();

  for (const page of pages) {
    // searchItems 搜索需要的内容，包含了坐标信息
    const matches = searchItems(page.textItems, { phrase: query });
    if (matches.length) hitsByPage.set(page.pageNum, matches);
  }
  console.log(`Found ${query} on ${hitsByPage.size} pages`);
  // Draw all highlights per page into a single image
  for (const [pageNum, rects] of hitsByPage) {
    const shot = screenshots.find((s) => s.pageNum === pageNum);
    if (!shot) continue;

    const composites = await Promise.all(
      rects.map(async (rect) => {
        const pixel = {
          left: Math.round(rect.x * SCALE),
          top: Math.round(rect.y * SCALE),
          width: Math.round(rect.width * SCALE),
          height: Math.round(rect.height * SCALE),
        };
        // 通过sharp 存储格式
        const overlay = await sharp({
          create: {
            width: pixel.width,
            height: pixel.height,
            channels: 4,
            background: { r: 255, g: 255, b: 0, alpha: 0.3 },
          },
        })
          .png()
          .toBuffer();

        return { input: overlay, left: pixel.left, top: pixel.top };
      })
    );

    const highlighted = await sharp(shot.imageBuffer)
      .composite(composites)
      .png()
      .toBuffer();
    await sharp(highlighted).toFile(`citation_page${pageNum}.png`);
    console.log(`Saved citation_page${pageNum}.png (${rects.length} highlights)`);
  }
}

main().catch(console.error);

说明

官方示例代码有定问题，以上是调整修改之后的，同时ocr 服务使用了基于rapidocr 包装的,官方也内置了ocr 协议服务

参考资料

https://developers.llamaindex.ai/liteparse/guides/visual-citations/