QuesmaOrg/BinaryAudit
View on GitHub ↗ ★ 107 Shell
An open-source benchmark for evaluating AI agents' ability to find backdoors hidden in compiled binaries.
7 mentions on Hacker News by 3 people: 0 stories, 7 comments · first 2026-02-10 · latest 2026-02-22
20082026
-
…likely to improve performance. So is any interactive feedback. Our example instruction is here: https://github.com/QuesmaOrg/BinaryAudit/blob/main/tasks/lig...
comment by stared · 2026-02-22 · in We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them -
…symbols. If you look at the actual backdoors many are already minimal and quite obfuscated, see: - https://github.com/QuesmaOrg/BinaryAudit/blob/main/tasks/dns... - https://github.com/QuesmaOrg/BinaryAudit/blob/main/tasks/dro...
comment by achille · 2026-02-22 · in We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them -
See direct benchmark link: https://quesma.com/benchmarks/binaryaudit/ Open-source GitHub: https://github.com/QuesmaOrg/BinaryAudit
comment by jakozaur · 2026-02-22 · in We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them -
Repo: https://github.com/quesmaOrg/BinaryAudit All charts: https://quesma.com/benchmarks/binaryaudit/
comment by stared · 2026-02-15 · in We hid backdoors in binaries – Opus 4.6 found 49% of them -
Repo: https://github.com/QuesmaOrg/BinaryAudit
comment by stared · 2026-02-13 · in BinaryAudit: Can AI find backdoors in raw machine code? -
…https://quesma.com/benchmarks/binaryaudit/ All tasks are open-source & we welcome contributions: https://github.com/QuesmaOrg/BinaryAudit Discussion on X: https://x.com/pmigdal/status/2021244382800760873
comment by stared · 2026-02-11 · in We hid backdoors in binaries – Opus 4.6 found 49% of them -
…AI benchmark tasks, using Ghidra and Radare2 to find backdoors in binaries of C and Rust servers: https://github.com/quesmaOrg/BinaryAudit
comment by stared · 2026-02-10 · in We hid backdoors in binaries – Opus 4.6 found 49% of them