Had also a look at Denuvo a while ago. Used LLVM to remove the x86 obfuscation and broke it down to VM-Opcodes. Atleast back then, Denuvo seemd to translate gamecode into a stackmachine.
This is how a VM push looks like:
temp[0]=add(mem[e268], fffffffffffffff8)
mem[temp[0]]]=mem[e560]
mem[e268]=temp[0]
But i quickly lost interest when it became MBA galore:
temp[7]=sub(add(add(and(mem[ebe8], b2f7), 3fd8), xor(lshr(mem[ebe8], 1), 2684)), lshr(add(mem[ebe8], b2f8), 1))
temp[d]=or(sub(sub(4ad, temp[7]), xor(and(shl(temp[7], 1), 95c), 95c)), 8000)
temp[e]=lshr(temp[d], 1)
temp[11]=lshr(add(temp[d], 8001), 1)
mem[ebe8]=sub(xor(xor(temp[e], 3fff), temp[11]), shl(and(and(temp[e], 3fff), temp[11]), 1))
BTW: anyone aware of LLVM optimizer passimplementations that can deal with MBAs ?
You can take a look at SiMBA++ -> https://github.com/pgarba/SiMBA-
It is a C++ implementation of SiMBA [1] - a tool to handle linear MBAs, made available by Denuvo itself. Denuvo have another tool - Gamba for handling some variety of non-linear MBAs. And then further improvisation by another researcher - MSiMBA [3].
SiMBA++ since written in C++, it is fast and it integrates well into the LLVM passes to automatically identify the MBAs and replace them in the LLVM IR with simplified expressions. So no additional work required.
Shameless plug - me and my colleague (author of SiMBA++) recently gave a talk about using LLVM for deobfuscation of WASM, where we talk about MBAs, SiMBA++ etc. The idea is not limited to WASM, it is language agnostic once you have a binary lifted to LLVM IR. https://www.youtube.com/watch?v=gKRdOcuXbYI
[1] SiMBA - https://github.com/DenuvoSoftwareSolutions/SiMBA [2] Gamba - https://github.com/DenuvoSoftwareSolutions/GAMBA [3] MSiMBA - https://github.com/mazeworks-security/MSiMBA
Xyntia was also just released to deal with MBA obfuscation, but I haven't had the chance to try it:
> BTW: anyone aware of LLVM optimizer passimplementations that can deal with MBAs ?
Your best bet is InstCombine, but likely most of the MBA patterns aren't going to be InstCombine patterns because who writes that kind of code?
In principle, you might see if you can tickle Alive2 (which can map LLVM IR to SMT logic) to see if you can get a peephole optimizer that's querying an SMT solver. But I'm not aware of anyone who's built a pass like that yet, and it's definitely not a regular pass in the compiler.
Wasn't aware of Alive2. Thx, I'll have look.
I had some success with https://github.com/mrphrazer/msynth But its hard to glue this to LLVM.