Skip to content

Issues: modelscope/evalscope

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

23 Open 109 Closed
23 Open 109 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

数据混合测评 -没有按设置的权重采集数据 bug Something isn't working native
#287 opened Jan 20, 2025 by jackqdldd
5
main分支Arena Mode代码运行报错
#286 opened Jan 18, 2025 by VirPlayer
2
RAGAS 评估结果为空, 输入txt的正确格式是什么？ question Further information is requested rageval
#284 opened Jan 15, 2025 by xiguahuoshan
10 tasks
2
接入openai server评测时，system input 该如何自定义修改？
#281 opened Jan 14, 2025 by ethanyys
2
truthful_qa及ContinuationLogitsModelAdapter使用疑问
#280 opened Jan 12, 2025 by Haruka1307
3
能否对模型评估后的结果进行可视化操作？ enhancement New feature or request
#276 opened Jan 8, 2025 by JasonYuel
3
Meta-Llama-3.1-8B模型mmlu和ceval测试集评分为0 bug Something isn't working opencompass
#271 opened Jan 3, 2025 by yosure
@wangxingjun778
如何进行图像-文本通过CLIP嵌入在同一Embedding空间进行测试？ enhancement New feature or request
#270 opened Jan 2, 2025 by Nomothings
4
opencompass 支持的数据集比较多，但evalscope使用opencompass作为后端，有些数据集却不支持 enhancement New feature or request
#268 opened Dec 30, 2024 by weibingo
5 tasks
2
MMLU 测评结果与官方差距较大
#267 opened Dec 30, 2024 by Haruka1307
5
RAGAS generate testset task 生成问题时报错 Documents appears to be too short (ie 100 tokens or less) bug Something isn't working rageval
#255 opened Dec 19, 2024 by simonqian
2 of 6 tasks
@Yunnglin
4
使用opencompass评测后端结果和官网结构不一样 opencompass
#229 opened Dec 5, 2024 by yosure
4 tasks
@wangxingjun778
2
评估越来越慢，gpu利用率低 native
#227 opened Dec 3, 2024 by ChavesLiu
2 tasks done
@wangxingjun778
1
采用 Openai server 测试，能够产生 inference 结果，但是 eval 输出结果为空 opencompass
#208 opened Nov 22, 2024 by HaoWuSR
9 tasks
@wangxingjun778
2
使用python代码如何统一指定离线data目录？ documentation Improvements or additions to documentation
#201 opened Nov 13, 2024 by charliedream1
opencompass数据集中safety板块数据集支持 enhancement New feature or request
#197 opened Nov 12, 2024 by A-Zi-cong-xiao-jiu-hen-ke-ai
5
OpenCompass打印的测试集列表和网址给出的不一致，导致无法测试，比如cmmlu enhancement New feature or request
#191 opened Nov 9, 2024 by charliedream1
@wangxingjun778
11
能否给个longwriter和toolbench使用openai接口测试的例子 documentation Improvements or additions to documentation
#189 opened Nov 9, 2024 by charliedream1
2
输出结果没有分数 bug Something isn't working opencompass
#181 opened Nov 8, 2024 by Leo20100307
9 tasks
@wangxingjun778
6
请问，评估支持使用昇腾910NPU嘛？ good first issue Good for newcomers
#123 opened Sep 9, 2024 by yiyayieryo
4
Baseline模型对比模式结果错误
#118 opened Aug 29, 2024 by stay-leave
2
请教一下，如何使用openai兼容格式的大模型作为评估模型来进行两模型在自定义数据集上的评估任务呢？
#115 opened Aug 26, 2024 by EvilCalf
3
model_id为local ckpt路径时，infer template映射和解析出现问题
#24 opened Mar 26, 2024 by wangxingjun778
ProTip! What’s not been updated in a month: updated:<2024-12-24.