Skip to content

L
llm-coding-eval

CSDN 技术社区 / llm-coding-eval

通知 19
Star 3
Fork 1
L
llm-coding-eval
提交 cd872ef5 编写于 作者: CSDN-Ada助手's avatar CSDN-Ada助手
浏览文件
操作

add chatgpt+codegexx2

上级 ca18098a
隐藏空白更改
内联 并排
Showing with 10 addition and 9 deletion
README.md
浏览文件 @ cd872ef5
...@@ -88,15 +88,16 @@ python main.py --task_type evaluate ...@@ -88,15 +88,16 @@ python main.py --task_type evaluate
受限于模型推理速度，目前只测试了pass@1指标。 受限于模型推理速度，目前只测试了pass@1指标。
| | python | java | cpp | js | go | | | python | java | cpp | js | go |
|--------------|--------|--------|--------|--------|--------| |----------------------|--------|--------|--------|--------|--------|
| chatgpt | 64.02% | 15.85% | 26.22% | 47.00% | 31.70% | | chatgpt | 64.02% | 15.85% | 26.22% | 47.00% | 31.70% |
| bbt-7B | 0.61% | 1.83% | 1.22% | 1.83% | 0.00% | | bbt-7B | 0.61% | 1.83% | 1.22% | 1.83% | 0.00% |
| bbt-13B | 2.49% | 0.00% | 1.90% | 1.83% | 0.61% | | bbt-13B | 2.49% | 0.00% | 1.90% | 1.83% | 0.61% |
| chatglm2-6B | 7.93% | 5.45% | 0.61% | 6.70% | 1.83% | | chatglm2-6B | 7.93% | 5.45% | 0.61% | 6.70% | 1.83% |
| codegeex2-6B | 29.90% | 27.43% | 6.70% | 24.40% | 17.68% | | codegeex2-6B | 29.90% | 27.43% | 6.70% | 24.40% | 17.68% |
| llama2-7B | 5.49% | 8.54% | 1.22% | 3.66% | 6.10% | | llama2-7B | 5.49% | 8.54% | 1.22% | 3.66% | 6.10% |
| baichuan-7B | 7.93% | 1.83% | 0.00% | 6.71% | 6.71% | | baichuan-7B | 7.93% | 1.83% | 0.00% | 6.71% | 6.71% |
| chatgpt+codegeex2-6B | 60.02% | 40.85% | 30.49% | 50.66% | 44.51% |
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或