Skip to content

Add Jesse / Weightless (Zero-Weight) official HumanEval & HumanEval+ scores - #28

Open
m0at wants to merge 1 commit into
evalplus:mainfrom
m0at:jesse-zero-weight
Open

m0at wants to merge 1 commit into
evalplus:mainfrom
m0at:jesse-zero-weight

Conversation

@m0at

@m0at m0at commented Sep 18, 2026

Copy link
Copy Markdown

Summary

  • Add Jesse / Weightless (Zero-Weight) to results.json.
  • Benchmark: HumanEval (164 tasks) + HumanEval+ (extended edge cases).
  • Public Samples: https://github.com/m0at/jesse-humaneval-plus/blob/main/samples.jsonl
  • SHA256: ea97a04b4f5258959bae83d75d9a9de326d8a1f42100a9187cf8ec42113a0c0a
  • Pass@1 Scores:
    • HumanEval (Base): 100.0% (164/164)
    • HumanEval+ (Extended): 99.4% (163/164)
  • Size: null (Zero-Weight, 0 LLM API tokens, $0.00 cost)
  • prompted: false (autonomous zero-weight AST synthesis engine)
  • open-data: NONE

Test Plan

pip install evalplus==0.3.1
evalplus.evaluate --dataset humaneval --samples samples.jsonl

Confirmed results.json parses cleanly as valid JSON.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant