Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Microblogging: Synthesizing (obfuscated) expressions #1074

Closed
JonathanSalwan opened this issue Jan 24, 2022 · 6 comments
Closed

Microblogging: Synthesizing (obfuscated) expressions #1074

JonathanSalwan opened this issue Jan 24, 2022 · 6 comments

Comments

@JonathanSalwan
Copy link
Owner

JonathanSalwan commented Jan 24, 2022

Introduction

Software are getting more and more protected against reverse engineering. Several software protections exist [1]. In the past we already attacked some of these protections with Triton using symbolic and taint analysis [2, 3, 4, 5, 6]. Some software protections (like Mixed Boolean-Arithmetic expression: MBA [7]) transform simple operation into a more complicated one by mixing arithmetic operators (e.g: ADD, SUB, MUL) with bit-wise operators (e.g: AND, OR, XOR, ROL). For example:

  • x + y == (x | y) + y - (~x & y)
  • x ^ y == (x | y) - y + (~x & y)
  • x ^ y == (x & ~y) | (~x & y)
  • x | y == (x ^ y) + y - (~x & y)
  • x & y == -(x | y) + y + x

In this contribution we introduce a new engine which lay the foundation of expression synthesis and aims to attack such protections. Synthesis is a well known analysis when reversing obfuscated expressions [8, 9, 10] and tool like Qsynth [10] already relies on Triton for its analysis. That's why I thought it will be nice if we can add some program synthesis concepts in Triton.

The engine and its components

We have added the engine Synthesizer into a new namespace: triton::engines::synthesis:: and a new method has been added into the API to synthesize expressions. The engine is almost standalone, it only relies on the symbolic engine to create new symbolic variables but does not alters the symbolic state of your Triton context.

triton::engines::synthesis::SynthesisResult API::synthesize(const triton::ast::SharedAbstractNode& node, bool constant) {
  this->checkSymbolic();
  triton::engines::synthesis::Synthesizer synth(this->symbolic);
  return synth.synthesize(node, constant);
}

The SynthesisResult object contains the input AST (the obfuscated one), the output AST (the synthesized one), the time in milliseconds of the analysis and a flag success which is true if the input AST has been synthesized.

class SynthesisResult {
  [...]
  public:
      [...]
      const triton::ast::SharedAbstractNode getInput(void);
      const triton::ast::SharedAbstractNode getOutput(void);
      triton::usize getTime(void);
      bool successful(void);
}

Oracle table with two variables

Program synthesis uses an oracle table to determine the behavior of obfuscated expressions. We consider the target as a black box and by seeing the result of a given input we can determine what the black box does. Our oracle table is located in triton::engines::synthesis::oracles::binopTable and contains several entries. All these entries are used for analyzing an expression that contains two variables and one operator. For example, these following entries are used to synthesize a bvadd node.

/* bvadd synthesis */
{
  triton::ast::BVADD_NODE, {
    BinaryEntry(8, 0xab, 0xf4, 0x9f), BinaryEntry(16, 0xd732, 0x4aef, 0x2221), BinaryEntry(32, 0x469f7dd0, 0xb31b6eff, 0xf9baeccf), BinaryEntry(64, 0x4228671b9260d25a, 0x5434951c42b9e695, 0x965cfc37d51ab8ef),
    BinaryEntry(8, 0x20, 0x08, 0x28), BinaryEntry(16, 0x159a, 0x5ea1, 0x743b), BinaryEntry(32, 0x2f9557f0, 0x661e61f5, 0x95b3b9e5), BinaryEntry(64, 0x432e5bc7f37785fe, 0x98a0dea40c10db87, 0xdbcf3a6bff886185),
    BinaryEntry(8, 0x7a, 0x6b, 0xe5), BinaryEntry(16, 0x23ad, 0x14e1, 0x388e), BinaryEntry(32, 0xf9378199, 0x044faf6b, 0xfd873104), BinaryEntry(64, 0x72b52744d832920b, 0x93a68be90bbefa87, 0x065bb32de3f18c92),
    BinaryEntry(8, 0xef, 0xb4, 0xa3), BinaryEntry(16, 0x391a, 0x2b4f, 0x6469), BinaryEntry(32, 0x938c6494, 0x38918999, 0xcc1dee2d), BinaryEntry(64, 0x6f54a84153273cab, 0xa986855db460ed11, 0x18db2d9f078829bc),
    BinaryEntry(8, 0x40, 0x71, 0xb1), BinaryEntry(16, 0xda3b, 0x821b, 0x5c56), BinaryEntry(32, 0x8a99e28e, 0xa16daf8e, 0x2c07921c), BinaryEntry(64, 0xf0fd090236cd156f, 0xb2828dd6dffe38a5, 0xa37f96d916cb4e14),
    BinaryEntry(8, 0x7d, 0xaa, 0x27), BinaryEntry(16, 0x2cb0, 0xe80b, 0x14bb), BinaryEntry(32, 0x41491ca5, 0x10698ef9, 0x51b2ab9e), BinaryEntry(64, 0x0d7c0452a656cc4e, 0xf8916d8b06c25bd6, 0x060d71ddad192824),
    BinaryEntry(8, 0x04, 0xd9, 0xdd), BinaryEntry(16, 0xa94d, 0x4f26, 0xf873), BinaryEntry(32, 0xd43e3529, 0x50215e66, 0x245f938f), BinaryEntry(64, 0x6080204088c69b8c, 0x7dfa4f93888843e2, 0xde7a6fd4114edf6e),
    BinaryEntry(8, 0xd3, 0xac, 0x7f), BinaryEntry(16, 0x14ac, 0x4f23, 0x63cf), BinaryEntry(32, 0x2736690b, 0xd07ef240, 0xf7b55b4b), BinaryEntry(64, 0xe647b71edcd154ca, 0xeb318119771ced0d, 0xd179383853ee41d7),
    BinaryEntry(8, 0xda, 0xad, 0x87), BinaryEntry(16, 0xb6ee, 0x4008, 0xf6f6), BinaryEntry(32, 0x84559f32, 0xd2e90cde, 0x573eac10), BinaryEntry(64, 0x95fe7d45035c1641, 0x70586d899571ae0d, 0x0656eace98cdc44e),
    BinaryEntry(8, 0xd9, 0x47, 0x20), BinaryEntry(16, 0x8d36, 0x9497, 0x21cd), BinaryEntry(32, 0x4a8e0eb5, 0xaa9f8e8f, 0xf52d9d44), BinaryEntry(64, 0x9093d9c3bcc2f2fb, 0xa1a5835615c2033e, 0x32395d19d284f639),
    }
},

A BinaryEntry represents an entry in our oracle table. For example, the entry: BinaryEntry(8, 0xab, 0xf4, 0x9f) is structured as follows:

  • 8 is the size in bits of the expression
  • 0xab is the input value of the first variable (e.g: x)
  • 0xf4 is the input value of the second variable (e.g: y)
  • 0x9f is the result value of the bvadd operator computation on those input values

It means that if we inject into our black box (the obfuscated expression) 0xab and 0xfa into arguments and if the result of the executed black box is 0x9f, it means that the obfuscated expression might do an ADD operation. To be sure, we have to execute different entries. That's why in our oracle table we have multiple entries, with different values, on different sizes, and for all operators.

Constant synthesis

The engine also supports constant synthesizing. Let's consider, for example, the following expression: ((z << 8) >> 16) << 8, this expression contains only one variable and can be synthesized as follows: z & 0xffff00. To do that, the synthesizer uses the SMT quantifier forall and the constraint is expressed as follows: For all entries z, there exists a symbolic constant C such that ((z << 8) >> 16) << 8 == z <operator> C. We have to request this constraint to the solver with all operators to find the good one and its constant.

  • Query 1: ((z << 8) >> 16) << 8 == z & C
  • Query 2: ((z << 8) >> 16) << 8 == z ^ C
  • Query 3: ((z << 8) >> 16) << 8 == z | C
  • Query 4: ((z << 8) >> 16) << 8 == z * C
  • Query 5: ((z << 8) >> 16) << 8 == z + C
  • Query 6: ((z << 8) >> 16) << 8 == z - C
  • Query 7: ((z << 8) >> 16) << 8 == C - z

Constant synthesis is more relying on SMT reasoning than pure synthesis concepts as it uses SMT solver to solve constraint. However it seemed good to me to add this feature in the synthesis analysis because constant obfuscation is a real thing used in modern software (see examples below).

How to use the synthesizer

Ok let's talk about examples and how to use the synthesizer. Well, it's pretty simple, you just have to call ctx.synthesize on your obfuscated expression and if the synthesis succeed, you have a synthesized expression at return. You can work on standalone expressions as well as from an execution of obfuscated code.

Working on standalone expressions

The code below deals with standalone expressions. We extracted/crafted our own obfuscated expressions and for each expression we call ctx.synthesize.

import sys
from triton import *


# This following constant obfuscation comes from a modern software.
# This expression has been extracted from reverse engineering by ex
# colleagues at Quarkslab several years ago.
def x_xor_92_obfuscated(x):
    a = 229 * x + 247
    b = 237 * a + 214 + ((38 * a + 85) & 254)
    c = (b + ((-(2 * b) + 255) & 254)) * 3 + 77
    d = ((86 * c + 36) & 70) * 75 + 231 * c + 118
    e = ((58 * d + 175) & 244) + 99 * d + 46
    f = (e & 148)
    g = (f - (e & 255) + f) * 103 + 13
    R = (237 * (45 * g + (174 * g | 34) * 229 + 194 - 247) & 255)
    return R


def main():
    ctx = TritonContext(ARCH.X86_64)
    ast = ctx.getAstContext()

    ctx.setAstRepresentationMode(AST_REPRESENTATION.PYTHON)

    x = ast.variable(ctx.newSymbolicVariable(8, 'x'))
    y = ast.variable(ctx.newSymbolicVariable(8, 'y'))
    z = ast.variable(ctx.newSymbolicVariable(32, 'z'))

    # Some obfuscated expressions
    obf_exprs = [
        ~x & 0xff,
        -~x & 0xff,
        (x | y) + y - (~x & y),                             # x + y             (from http://archive.bar/pdfs/bar2020-preprint9.pdf)
        (x | y) - y + (~x & y),                             # x ^ y             (from http://archive.bar/pdfs/bar2020-preprint9.pdf)
        (x & ~y) | (~x & y),                                # x ^ y             (from ?)
        (x ^ y) + y - (~x & y),                             # x | y             (from http://archive.bar/pdfs/bar2020-preprint9.pdf)
        -(x | y) + y + x,                                   # x & y             (from http://archive.bar/pdfs/bar2020-preprint9.pdf)
        ((z << 8) >> 16) << 8,                              # z & 0xffff00      (from https://blog.regehr.org/archives/1636)
        (((x ^ y) + 2 * (x & y)) * 39 + 23) * 151 + 111,    # x + y             (from Ninon Eyrolle's thesis)
        x_xor_92_obfuscated(x),                             # x ^ 92            (from reverse engineering)
    ]

    for expr in obf_exprs:
        (print('In: %s' %(expr)) if len(str(expr)) < 100 else print('In: %s ...' %(str(expr)[0:100])))
        expr = ctx.synthesize(expr)
        print('Out: %s' %(expr))
        print()

    return 0


if __name__ == '__main__':
    sys.exit(main())

The result is the following:

$ time python ./example.py
In: ((~(x) & 0xff) & 0xff)
Out: ((0xff - x) & 0xff)

In: ((-((~(x) & 0xff)) & 0xff) & 0xff)
Out: ((x + 0x1) & 0xff)

In: (((((x | y) + y) & 0xff) - ((~(x) & 0xff) & y)) & 0xff)
Out: ((x + y) & 0xff)

In: (((((x | y) - y) & 0xff) + ((~(x) & 0xff) & y)) & 0xff)
Out: (x ^ y)

In: ((x & (~(y) & 0xff)) | ((~(x) & 0xff) & y))
Out: (x ^ y)

In: (((((x ^ y) + y) & 0xff) - ((~(x) & 0xff) & y)) & 0xff)
Out: (x | y)

In: (((((-((x | y)) & 0xff) + y) & 0xff) + x) & 0xff)
Out: (y & x)

In: (((((z << 0x8) & 0xffffffff) >> 0x10) << 0x8) & 0xffffffff)
Out: (z & 0xffff00)

In: (((((((((((x ^ y) + ((0x2 * (x & y)) & 0xff)) & 0xff) * 0x27) & 0xff) + 0x17) & 0xff) * 0x97) & 0xff ...
Out: ((x + y) & 0xff)

In: (((0xed * ((((((((0x2d * ((((((((((((((((((0x3a * (((((((((((0x56 * ((((((((((((0xed * ((((0xe5 * x) ...
Out: (x ^ 0x5c)

python ./example.py  0.22s user 0.01s system 99% cpu 0.234 total

Working on obfuscated trace (emulate the code)

I use Triton a lot of time when doing reverse engineering. I mainly start doing the reverse part and then when some parts of the code need better investment, I use Triton to emulate the piece of code that I'm working on. Thus, I can follow what instructions are executed, what data they use, taint data and follow them, define symbolic variables, asking for constraints, etc. It's mainly like a scriptable Python debugger but with a set of program analysis utilities. For example, let's go back with the constant obfuscation of the (x ^ 0x5c) operation from a modern software. You can easily extract the opcodes of the function to work on it. Then, you can define the input of the function as a symbolic variable, emulate the function and synthesize its return.

#!/usr/bin/env python
## -*- coding: utf-8 -*-

import sys
from triton import *

# Extract of the obfuscated function
CODE  = b"\x40\x88\xF8\x04\x81\xB1\xFE\x88\x44\x24\xFF\x40\x88\xF8\xF6\xE1"
CODE += b"\x04\xFF\x24\xFE\x8A\x4C\x24\xFF\x00\xC1\x88\xC8\xC0\xE0\x01\x34"
CODE += b"\xFE\x00\xC8\xB1\x03\xF6\xE1\x04\x4D\xB1\x56\x88\x44\x24\xFE\xF6"
CODE += b"\xE1\x04\x24\x24\x46\xB1\x4B\xF6\xE1\xB1\xE7\x8A\x54\x24\xFE\x88"
CODE += b"\x44\x24\xFD\x88\xD0\xF6\xE1\x04\x76\x8A\x4C\x24\xFD\x00\xC8\xB1"
CODE += b"\x3A\x88\x44\x24\xFC\xF6\xE1\x04\xAF\x24\xF4\xB1\x63\x40\x8A\x74"
CODE += b"\x24\xFC\x88\x44\x24\xFB\x40\x88\xF0\xF6\xE1\x04\x2E\x8A\x4C\x24"
CODE += b"\xFB\x00\xC8\x88\xC1\xC0\xE1\x01\x80\xE1\x28\x28\xC1\xB0\x67\x88"
CODE += b"\x44\x24\xFA\x88\xC8\x8A\x4C\x24\xFA\xF6\xE1\x04\x0D\xB1\x2D\x88"
CODE += b"\x44\x24\xF9\xF6\xE1\xB1\xAE\x40\x8A\x7C\x24\xF9\x88\x44\x24\xF8"
CODE += b"\x40\x88\xF8\xF6\xE1\x0C\x22\xB1\xE5\xF6\xE1\x8A\x4C\x24\xF8\x00"
CODE += b"\xC8\xB1\xED\xF6\xE1\x04\xEF\xC3"

# Emulate the code
def emulate(ctx, pc):
    while pc:
        opcode = ctx.getConcreteMemoryAreaValue(pc, 16)
        instruction = Instruction(pc, opcode)
        ctx.processing(instruction)
        print(instruction)
        pc = ctx.getConcreteRegisterValue(ctx.registers.rip)
    return

def main():
    ctx = TritonContext(ARCH.X86_64)
    ctx.setMode(MODE.ALIGNED_MEMORY, True)
    ctx.setMode(MODE.CONSTANT_FOLDING, True)
    ctx.setMode(MODE.AST_OPTIMIZATIONS, True)

    # By reversing the code we know that the function works on DIL
    ctx.symbolizeRegister(ctx.registers.dil, "x")

    ctx.setConcreteMemoryAreaValue(0x1000, CODE)
    emulate(ctx, 0x1000)

    # Get the AST at the return of the function
    al = ctx.getRegisterAst(ctx.registers.al)
    ast = ctx.getAstContext()

    # Print the obfuscated expression
    print(f"In: {ast.unroll(al)}")

    # Print the synthesized expression
    print(f"Out: {ctx.synthesize(al)}")

if __name__ == '__main__':
    sys.exit(main())

The result is the following:

$ python emu.py
0x1000: mov al, dil
0x1003: add al, 0x81
0x1005: mov cl, 0xfe
0x1007: mov byte ptr [rsp - 1], al
0x100b: mov al, dil
0x100e: mul cl
0x1010: add al, 0xff
0x1012: and al, 0xfe
0x1014: mov cl, byte ptr [rsp - 1]
0x1018: add cl, al
0x101a: mov al, cl
0x101c: shl al, 1
0x101f: xor al, 0xfe
0x1021: add al, cl
0x1023: mov cl, 3
0x1025: mul cl
0x1027: add al, 0x4d
0x1029: mov cl, 0x56
0x102b: mov byte ptr [rsp - 2], al
0x102f: mul cl
0x1031: add al, 0x24
0x1033: and al, 0x46
0x1035: mov cl, 0x4b
0x1037: mul cl
0x1039: mov cl, 0xe7
0x103b: mov dl, byte ptr [rsp - 2]
0x103f: mov byte ptr [rsp - 3], al
0x1043: mov al, dl
0x1045: mul cl
0x1047: add al, 0x76
0x1049: mov cl, byte ptr [rsp - 3]
0x104d: add al, cl
0x104f: mov cl, 0x3a
0x1051: mov byte ptr [rsp - 4], al
0x1055: mul cl
0x1057: add al, 0xaf
0x1059: and al, 0xf4
0x105b: mov cl, 0x63
0x105d: mov sil, byte ptr [rsp - 4]
0x1062: mov byte ptr [rsp - 5], al
0x1066: mov al, sil
0x1069: mul cl
0x106b: add al, 0x2e
0x106d: mov cl, byte ptr [rsp - 5]
0x1071: add al, cl
0x1073: mov cl, al
0x1075: shl cl, 1
0x1078: and cl, 0x28
0x107b: sub cl, al
0x107d: mov al, 0x67
0x107f: mov byte ptr [rsp - 6], al
0x1083: mov al, cl
0x1085: mov cl, byte ptr [rsp - 6]
0x1089: mul cl
0x108b: add al, 0xd
0x108d: mov cl, 0x2d
0x108f: mov byte ptr [rsp - 7], al
0x1093: mul cl
0x1095: mov cl, 0xae
0x1097: mov dil, byte ptr [rsp - 7]
0x109c: mov byte ptr [rsp - 8], al
0x10a0: mov al, dil
0x10a3: mul cl
0x10a5: or al, 0x22
0x10a7: mov cl, 0xe5
0x10a9: mul cl
0x10ab: mov cl, byte ptr [rsp - 8]
0x10af: add al, cl
0x10b1: mov cl, 0xed
0x10b3: mul cl
0x10b5: add al, 0xef
0x10b7: ret
In: (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvor ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvsub (b
vand (bvshl (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x
 (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ z
ero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 
8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 
8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8)
)) (_ bv75 16))))) (_ bv99 16))) (_ bv46 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bv
add (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bv
add ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ ext
ract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (
_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ 
bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv58 16))) (_ bv175 8)) (_ bv244 8))) (_ bv1 8)) (_ bv40 8)) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul (
(_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv2
54 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv1
18 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvan
d (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_
 bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv99 16))) (_ bv46 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd
 (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (
_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (
_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (
bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0
) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv58 16))) (_ bv175 8)) (_ bv244 8))))) (_ bv103 16))
) (_ bv13 8))) (_ bv174 16))) (_ bv34 8))) (_ bv229 16))) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvsub (bvand (bvshl (bvadd (bvadd ((_ extract 7 0) (bvmul ((
_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul 
((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv25
4 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8)
 (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand
 (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv99 16))) (_ bv46 8)) (bvand 
(bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvan
d (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_
 bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extra
ct 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (b
vadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) 
(_ bv58 16))) (_ bv175 8)) (_ bv244 8))) (_ bv1 8)) (_ bv40 8)) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_
 zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ b
v129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 7 0) (bvmul ((_ zero_extend 8) (b
vand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) 
x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16)
)) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv99 16))) (_ bv46 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (
bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) 
(_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv231 16))) (_ bv118 8)) ((_ extract 
7 0) (bvmul ((_ zero_extend 8) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) (bvadd (bvxor (bvshl (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extrac
t 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv255 8)) (_ bv254 8))) (_ bv1 8)) (_ bv254 8)) (bvadd (bvadd x (_ bv129 8)) (bvand (bvadd ((_ extract 7 0) (bvmul ((_ zero_extend 8) x) (_ bv254 16))) (_ bv2
55 8)) (_ bv254 8))))) (_ bv3 16))) (_ bv77 8))) (_ bv86 16))) (_ bv36 8)) (_ bv70 8))) (_ bv75 16))))) (_ bv58 16))) (_ bv175 8)) (_ bv244 8))))) (_ bv103 16))) (_ bv13 8))) (_ bv45 16))))) (_ bv237 16))) (_ bv2
39 8))


Out: (bvxor x (_ bv92 8))

As you can see, the result of the synthesis is: (bvxor x (_ bv92 8)) (SMT syntax). If you prefer, you can set the Python syntax like below:

>>> ctx.setAstRepresentationMode(AST_REPRESENTATION.PYTHON)
>>> print(f"Out: {ctx.synthesize(al)}")                    
Out: (x ^ 0x5c)

If you do not want to work on big expression at the end of the execution, you can also define a simplification callback that will be called each time an AST is assigned to a register or a memory cell during the execution (see below). Thus, you can perform a synthesis before each assignment. In other words, you synthesize the expression on the fly.

def my_cb(ctx, node):
    synth = ctx.synthesize(node, constant=True)
    if synth:
        return synth
    return node

ctx = TritonContext(ARCH.X86_64)
ctx.addCallback(CALLBACK.SYMBOLIC_SIMPLIFICATION, my_cb)

Into IDA

You can also use IDA and its API to extract opcodes and directly print results in the IDA console. For example, an IDA script could be like this:

#!/usr/bin/env python3
## -*- coding: utf-8 -*-

import ida_funcs
import ida_kernwin
import ida_gdl
import ida_bytes

from triton import *

if __name__ == '__main__':
    # Analyse all BB of the function
    func = ida_funcs.get_func(ida_kernwin.get_screen_ea())
    name = ida_funcs.get_func_name(func.start_ea)
    bb   = ida_gdl.FlowChart(func)[0]
    pc   = bb.start_ea

    ctx = TritonContext(ARCH.X86_64)
    ctx.symbolizeRegister(ctx.registers.dil, 'dil')
    ctx.setAstRepresentationMode(AST_REPRESENTATION.PYTHON)

    print(f"[+] Analyzing basic block at 0x{pc:x}")
    while pc != 0:
        opcode = idc.get_bytes(pc, 16)
        inst = Instruction(pc, opcode)
        ctx.processing(inst)
        pc = ctx.getSymbolicRegisterValue(ctx.registers.rip)

    al = ctx.getRegisterAst(ctx.registers.al)
    print(f'Synthesis: {ctx.synthesize(al)}')

It provides the following result:

2022-01-24-151025_1920x1080_scrot

Conclusion

We introduced a new engine which aims to synthesize expressions. It currently do constant synthesizing and two variables with one operator synthesizing. Synthesis is applied on root node and we do not yet go through the DAG to synthesize children branches. We let this job for the future. However, you can applied synthesis during the execution which synthesizes nodes at the construction which is pretty similar than going through the DAG when the full expression is built (edit: we do analyze children nodes now). In the future, oracle table may grow to support multiple operators and variables.

Bibliography

[01] Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection (book)
[02] https://github.com/JonathanSalwan/Tigress_protection
[03] http://shell-storm.org/talks/DIMVA2018-deobfuscation-salwan-bardin-potet.pdf
[04] http://shell-storm.org/talks/SSTIC2017_Deobfuscation_of_VM_based_software_protection.pdf
[05] http://shell-storm.org/talks/csaw2016-sos-rthomas-jsalwan.pdf
[06] http://shell-storm.org/talks/sthack2016-rthomas-jsalwan.pdf
[07] https://tel.archives-ouvertes.fr/tel-01623849/document
[08] https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/blazytko
[09] https://github.com/mrphrazer/msynth
[10] https://www.ndss-symposium.org/wp-content/uploads/2020/04/bar2020-23009.pdf

@JonathanSalwan JonathanSalwan added this to the v0.9 milestone Jan 24, 2022
@JonathanSalwan JonathanSalwan self-assigned this Jan 24, 2022
@JonathanSalwan JonathanSalwan changed the title Introduction to a new engine: Synthesizer of (obfuscated) expressions Introduction to a new engine: Synthesizing (obfuscated) expressions Jan 24, 2022
@mrexodia
Copy link
Contributor

Great news, I will have to check it out!

@cnheitman
Copy link
Collaborator

Awesome!! 🚀

@0xeb
Copy link

0xeb commented Jan 24, 2022

Good work.

@ohroy
Copy link

ohroy commented Jan 25, 2022

awesome

@JonathanSalwan
Copy link
Owner Author

JonathanSalwan commented Jan 27, 2022

Ok, we are now able to synthesize sub expressions of a tree. It leads to a synthesis of infinite operators with a very small oracle table (only ~700 entries). It also support constant synthesis.

Below an example. Let's consider the following function. Its original expression is: a - (~((b * a) * b)).

int f(unsigned a, unsigned b) {
  unsigned n = (a & ~(-((((b & a) * (b | a) + (b & ~a) * (~b & a)) & b) *                  \
               (((b&a)*(b|a) + (b& ~a)*(~b&a))|b) + (((b&a)*(b| a) + (b & ~a) *            \
               (~b & a)) & ~b) * (~((b & a) * (b | a) + (b & ~a) * (~b & a)) & b)) - 1)) - \
               (~a & (-((((b & a) * (b | a) + (b & ~a) * (~b & a)) & b) * (((b & a) *      \
               (b | a) + (b & ~a) * (~b & a)) | b) + (((b & a) * (b | a) + (b & ~a) *      \
               (~b & a)) & ~b) * (~((b & a) * (b | a) + (b & ~a) * (~b & a)) & b)) - 1));  \
  return n;
}

The Triton tool that attacks this function is the following:

#!/usr/bin/env python
## -*- coding: utf-8 -*-

import sys
from triton import *

CODE  = b"\x55\x48\x89\xE5\x89\x7D\xEC\x89\x75\xE8\x8B\x45\xE8\x23\x45\xEC"
CODE += b"\x89\xC2\x8B\x45\xE8\x0B\x45\xEC\x89\xD1\x0F\xAF\xC8\x8B\x45\xEC"
CODE += b"\xF7\xD0\x23\x45\xE8\x89\xC2\x8B\x45\xE8\xF7\xD0\x23\x45\xEC\x0F"
CODE += b"\xAF\xC2\x01\xC8\x23\x45\xE8\x89\xC2\x8B\x45\xE8\x23\x45\xEC\x89"
CODE += b"\xC1\x8B\x45\xE8\x0B\x45\xEC\x89\xCE\x0F\xAF\xF0\x8B\x45\xEC\xF7"
CODE += b"\xD0\x23\x45\xE8\x89\xC1\x8B\x45\xE8\xF7\xD0\x23\x45\xEC\x0F\xAF"
CODE += b"\xC1\x01\xF0\x0B\x45\xE8\x89\xD6\x0F\xAF\xF0\x8B\x45\xE8\x23\x45"
CODE += b"\xEC\x89\xC2\x8B\x45\xE8\x0B\x45\xEC\x89\xD1\x0F\xAF\xC8\x8B\x45"
CODE += b"\xEC\xF7\xD0\x23\x45\xE8\x89\xC2\x8B\x45\xE8\xF7\xD0\x23\x45\xEC"
CODE += b"\x0F\xAF\xC2\x8D\x14\x01\x8B\x45\xE8\xF7\xD0\x89\xD1\x21\xC1\x8B"
CODE += b"\x45\xE8\x23\x45\xEC\x89\xC2\x8B\x45\xE8\x0B\x45\xEC\x89\xD7\x0F"
CODE += b"\xAF\xF8\x8B\x45\xEC\xF7\xD0\x23\x45\xE8\x89\xC2\x8B\x45\xE8\xF7"
CODE += b"\xD0\x23\x45\xEC\x0F\xAF\xC2\x01\xF8\xF7\xD0\x23\x45\xE8\x0F\xAF"
CODE += b"\xC1\x8D\x14\x06\x8B\x45\xEC\x01\xD0\x83\xC0\x01\x89\x45\xFC\x8B"
CODE += b"\x45\xFC\x5D\xC3"

def emulate(ctx, pc):
    while pc:
        opcode = ctx.getConcreteMemoryAreaValue(pc, 16)
        instruction = Instruction(pc, opcode)
        ctx.processing(instruction)
        print(instruction)
        pc = ctx.getConcreteRegisterValue(ctx.registers.rip)
    return

def main():
    ctx = TritonContext(ARCH.X86_64)
    ast = ctx.getAstContext()

    ctx.setMode(MODE.CONSTANT_FOLDING, True)
    ctx.setMode(MODE.ALIGNED_MEMORY, True)
    ctx.setMode(MODE.AST_OPTIMIZATIONS, True)

    a = ast.variable(ctx.symbolizeRegister(ctx.registers.edi, "a"))
    b = ast.variable(ctx.symbolizeRegister(ctx.registers.esi, "b"))

    ctx.setConcreteMemoryAreaValue(0x1000, CODE)
    print('Emulate the code')
    emulate(ctx, 0x1000)

    eax = ctx.getRegisterAst(ctx.registers.eax)
    print()
    print('Synthesis IN:', ast.unroll(eax))
    print()
    eax = ctx.synthesize(eax, constant=False, subexpr=True)
    print('Synthesis OUT:', ast.unroll(eax))
    return 0

if __name__ == '__main__':
    sys.exit(main())

The output is the following:

$ time python test.py
Emulate the code
0x1000: push rbp
0x1001: mov rbp, rsp
0x1004: mov dword ptr [rbp - 0x14], edi
0x1007: mov dword ptr [rbp - 0x18], esi
0x100a: mov eax, dword ptr [rbp - 0x18]
0x100d: and eax, dword ptr [rbp - 0x14]
0x1010: mov edx, eax
0x1012: mov eax, dword ptr [rbp - 0x18]
[...]
0x10d4: mov eax, dword ptr [rbp - 0x14]
0x10d7: add eax, edx
0x10d9: add eax, 1
0x10dc: mov dword ptr [rbp - 4], eax
0x10df: mov eax, dword ptr [rbp - 4]
0x10e2: pop rbp
0x10e3: ret

Synthesis IN: (bvadd (bvadd a ((_ extract 31 0) (bvadd ((_ zero_extend 32) (bvmul (bvand (bvadd (bvmul (bvand (bvnot b) a) (bvand (bvnot a) b)) (bvmul (bvand b a) (bvor b a))) b) (bvor (bvadd (bvmul (bvand (bvnot b) a) (bvand (bvnot a) b)) (bvmul (bvand b a) (bvor b a))) b))) ((_ zero_extend 32) (bvmul (bvand (bvnot (bvadd (bvmul (bvand (bvnot b) a) (bvand (bvnot a) b)) (bvmul (bvand b a) (bvor b a)))) b) (bvand ((_ extract 31 0) (bvadd ((_ zero_extend 32) (bvmul (bvand b a) (bvor b a))) ((_ zero_extend 32) (bvmul (bvand (bvnot b) a) (bvand (bvnot a) b))))) (bvnot b))))))) (_ bv1 32))

Synthesis OUT: (bvadd (bvadd a (bvmul (bvmul a b) b)) (_ bv1 32))
python test.py  0.07s user 0.00s system 99% cpu 0.075 total

We found : ((a + ((a * b) * b)) + 0x1) which is equivalent to a - (~((b * a) * b)). As you can see there is more than only one operator. Thanks to Robin David algorithm [0].

To the moon 🚀 🚀

[0] https://www.ndss-symposium.org/wp-content/uploads/2020/04/bar2020-23009.pdf

@JonathanSalwan
Copy link
Owner Author

JonathanSalwan commented Jan 28, 2022

We now support opaque constant synthesis. It relies on SMT logic more than pure synthesis, so it's not used by default. To use it we have to set the opaque flag to true: ctx.synthesize(node, opaque=True).

It allows to synthesis expression like this:

op1 = (2 * (b & ~a) + ( -1 * (~ a | b) + ( -1 * ~( a & b) + (2 * ~( a | b) + 1 * a )))) # 0 (opaque constant)
op2 = (a | b) - (a + b) + (a & b)                                                       # 0 (opaque constant)
n = op1 + (1 << op2)                                                                    # 0 + (1 << 0) == 1
n = (op1 + 3) * (c ^ n)                                                                 # (0 + 3) * (c ^ 1) == 3 * (c ^ 1)

The input and output are the following:

IN:  ((((((((0x2 * (b & (~(a) & 0xff))) & 0xff) + ((((0xff * ((~(a) & 0xff) | b)) & 0xff) + ((((0xff * (~((a & b)) & 0xff)) & 0xff) + ((((0x2 * (~((a | b)) & 0xff)) & 0xff) + ((0x1 * a) & 0xff)) & 0xff)) & 0xff)) & 0xff)) & 0xff) + 0x3) & 0xff) * (c ^ ((((((0x2 * (b & (~(a) & 0xff))) & 0xff) + ((((0xff * ((~(a) & 0xff) | b)) & 0xff) + ((((0xff * (~((a & b)) & 0xff)) & 0xff) + ((((0x2 * (~((a | b)) & 0xff)) & 0xff) + ((0x1 * a) & 0xff)) & 0xff)) & 0xff)) & 0xff)) & 0xff) + ((0x1 << (((((a | b) - ((a + b) & 0xff)) & 0xff) + (a & b)) & 0xff)) & 0xff)) & 0xff))) & 0xff)

OUT: ((0x3 * (c ^ 0x1)) & 0xff)

By mixing SMT and pure synthesis we can do very great simplifications.

@JonathanSalwan JonathanSalwan changed the title Introduction to a new engine: Synthesizing (obfuscated) expressions Microblogging: Synthesizing (obfuscated) expressions Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants