Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用AI-Proxy插件代理 qwen 模型进行流式返回时因字段不全造成 golang sdk 解析失败 #1658

Open
1 task done
daixijun opened this issue Jan 9, 2025 · 0 comments
Labels
area/ai help wanted Extra attention is needed sig/wasm type/bug Something isn't working

Comments

@daixijun
Copy link
Contributor

daixijun commented Jan 9, 2025

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

  • I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

使用 Higress 配置 ai-proxy 插件来代理 dashscope / openai 等模型,在使用社区 golang sdk 进行流式返回的时候发现使用 qwen-plus 模型时, golang 侧解析出来的response结构体为空,但使用 gpt-4o-mini 模型时并没有问题

通过在终端使用 curl 测试两者区别发现使用 qwen 系列模型时返回结构中缺少部分值为 null 的字段

qwen系列返回数据
image

openai 系列返回数据
image

在 sdk 的仓库里看到结构体定义中, system_fingerprint 、finish_reason 等字段是必须存在的(没有添加标签omitempty),导致了结构体解析失败
https://github.com/sashabaranov/go-openai/blob/2a0ff5ac63e460cbe44cccd0d4199d51bf8682a4/chat_stream.go#L38
https://github.com/sashabaranov/go-openai/blob/2a0ff5ac63e460cbe44cccd0d4199d51bf8682a4/chat_stream.go#L53

Ⅱ. Describe what happened

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

希望在返回的结构中与 Openai 保持一致,补全缺失的字段信息

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

  • Higress version:
  • OS :
  • Others:
@johnlanni johnlanni added the help wanted Extra attention is needed label Jan 9, 2025
@github-project-automation github-project-automation bot moved this to Todo in Higress Jan 9, 2025
@johnlanni johnlanni added sig/wasm type/bug Something isn't working labels Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ai help wanted Extra attention is needed sig/wasm type/bug Something isn't working
Projects
Status: Todo
Development

No branches or pull requests

2 participants