ragas 评估报错 {'message': 'Invalid n value (currently only n = 1 is supported)', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}

报错信息:{'message': 'Invalid n value (currently only n = 1 is supported)', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}

原因是在评估  answer_relevancy 指标的时候,调用了不支持 n>1(单次调用生成多个答案)的大模型。

解决办法:修改ragas的源码,路径为ragas.metrics._answer_relevance

源码中的self.strictness默认为3,默认调用为n=3

image

解决方案:

先将一次调用生成多个回答,改为多次进行调用,一次调用生成一个回答的方式,来兼容仅支持n=1的大模型(如deepseek)

    async def _ascore(self, row: t.Dict, callbacks: Callbacks) -> float:
        assert self.llm is not None, "LLM is not set"

        prompt_input = ResponseRelevanceInput(response=row["response"])
        
        responses = []
        # 给回答生成'strictness'个可能的问题,并评判答案是否有对问题进行解答
        for _ in range(self.strictness):
            response = await self.question_generation.generate(
                data=prompt_input,
                llm=self.llm,
                callbacks=callbacks
            )
            responses.append(response)

        return self._calculate_score(responses, row)

 

posted @ 2025-09-29 14:13  wangssd  阅读(18)  评论(0)    收藏  举报