4.6. Security and Performance

About Security

Pyarmor focuses on protecting Python scripts, through several irreversible obfuscation methods, Pyarmor makes sure the obfuscated scripts can’t be restored in any way.

Pyarmor provides rich options to obfuscate scripts to balance security and performance. If anyone announces they have broken pyarmor, please try a simple script with different security options, refer to Highest security and performance. If any irreversible obfuscation has been broken, report this security issue to pyarmor@163.com. Do not paste any hack link in pyarmor project.

However Pyarmor isn’t good at memory protection and anti-debug. Generally even debugger tracing binary extension pyarmor_runtime could not help to restore obfuscated scripts, but it may bypass runtime key verification.

If you care about runtime memory data protection and anti-debug, check Protecting Runtime Memory Data

About Performance

Though the highest security could protect Python scripts from any hack method, it may reduce performance. In most cases, we need to pick the right options to balance security and performance.

Here we test some options to understand their impact on performance. All the following tests use 2 scripts benchmark.py and testben.py. Note that the test results are different even run the same test script in the same machine twice, not speak of different test scripts in different machines. So the test data in these tables are only guidelines, not exact.

The content of benchmark.py

import sys


class BenTest(object):

    def __init__(self):
        self.a = 1
        self.b = "b"
        self.c = []
        self.d = {}


def foo():
    ret = []
    for i in range(100000):
        ret.extend(sys.version_info[:2])
        ret.append(BenTest())
    return len(ret)

The content of testben.py

import benchmark
import sys
import time


def metric(func):
    if not hasattr(time, 'process_time'):
        time.process_time = time.clock

    def wrap(*args, **kwargs):
        t1 = time.process_time()
        result = func(*args, **kwargs)
        t2 = time.process_time()
        print('%-16s: %10.3f ms' % (func.__name__, ((t2 - t1) * 1000)))
        return result
    return wrap


def test_import():
    t1 = time.process_time()
    import benchmark2 as m2
    t2 = time.process_time()
    print('%-16s: %10.3f ms' % ('test_import', ((t2 - t1) * 1000)))
    return m2


@metric
def test_foo():
    benchmark.foo()


if __name__ == '__main__':
    print('Python %s.%s' % sys.version_info[:2])
    test_import()
    test_foo()

Different Python Version Performance

First obfuscate the scripts with default options, run it in different Python version, and compare the elapsed time with original scripts.

In order to test the difference without and with __pycache__, run scripts twice.

There are 3 check points:

  1. Import fresh module without __pycache__

  2. Import module 2nd with __pycache__

  3. Run function "foo", an obfuscated class is called 10,000 times

Here are test steps:

$ rm -rf dist __pycache__

$ cp benchmark.py benchmark2.py
$ python testben.py

Python 3.7
test_import     :   1.303 ms
test_foo        : 250.360 ms

$ python testben.py

Python 3.7
test_import     :   0.290 ms
test_foo        : 252.273 ms

$ pyarmor gen testben.py benchmark.py benchmark2.py
$ python dist/testben.py

Python 3.7
test_import     :   0.907 ms
test_foo        : 311.076 ms

$ python dist/testben.py

Python 3.7
test_import     :   0.454 ms
test_foo        : 359.138 ms
Table-1. Pyarmor Performance with Python Version

Time (ms)

Import fresh module

Import module 2nd

Run function “foo”

Python

Origin

Pyarmor

Origin

Pyarmor

Origin

Pyarmor

3.7

1.303

0.907

0.290

0.454

252.2

311.0

3.8

1.305

0.790

0.286

0.338

272.232

295.973

3.9

1.198

1.681

0.265

0.449

267.561

331.668

3.10

1.070

1.026

0.408

0.300

281.603

322.608

3.11

1.510

0.832

0.464

0.616

164.104

289.866

RFT Mode Performance

RFT mode should be same fast as original scripts.

Here we compare RFT mode with default options, the test data is got by this way.

First obfuscate scripts with default options, then run it.

Then obfuscate scripts with RFT mode, and run it again:

$ rm -rf dist
$ pyarmor gen testben.py benchmark.py benchmark2.py
$ python dist/testben.py

$ rm -rf dist
$ pyarmor gen --enable-rft testben.py benchmark.py benchmark2.py
$ python dist/testben.py
Table-2. Performance of RFT Mode

Time (ms)

Import fresh module

Run function “foo”

Remark

Python

Pyarmor

RFT Mode

Pyarmor

RFT Mode

3.7

1.083

1.317

334.313

324.023

3.8

0.774

1.109

239.217

241.697

3.9

0.775

0.809

304.838

301.789

3.10

2.182

1.049

310.046

339.414

3.11

0.882

0.984

258.309

264.070

Next, we compare RFT mode and --obf-code 0 with original scripts by this way:

$ rm -rf dist __pycache__
$ python testben.py
...

$ pyarmor gen --enable-rft --obf-code=0 testben.py benchmark.py benchmark2.py
$ python testben.py
...
Table-2.1 Performance of RFT Mode and obf-code 0

Time (ms)

Import fresh module

Run function “foo”

Remark

Python

Pyarmor

RFT Mode

Pyarmor

RFT Mode

3.7

0.757

1.844

307.325

272.672

3.8

0.791

0.747

276.865

243.436

3.9

1.276

0.986

246.407

236.138

3.10

2.563

1.142

256.583

260.196

3.11

0.952

0.938

185.435

154.390

They’re almost the same.

BCC Mode Performance

BCC mode converts some code to C function, it needs extra time to load binary code, but the function may be faster. The following test data got by this way:

$ rm -rf dist __pycache__
$ python testben.py
...

$ python testben.py
...

$ pyarmor gen --enable-bcc testben.py benchmark.py benchmark2.py
$ python dist/testben.py
...

$ python dist/testben.py
...
Table-3. Performance of BCC Mode with Python Version

Time (ms)

Import fresh module

Import module 2nd

Run function “foo”

Python

Origin

BCC Mode

Origin

BCC Mode

Origin

BCC Mode

3.7

1.086

1.177

0.342

0.391

344.640

271.426

3.8

1.099

1.397

0.351

0.400

291.244

251.520

3.9

1.229

1.076

0.538

0.362

306.594

254.458

3.10

1.267

0.999

0.255

0.796

302.398

247.154

3.11

1.146

1.056

0.273

0.536

206.311

189.582

Impact of Different Options

In order to facilitate comparison, each option is used separately. For example, test --no-wrap by this way:

$ rm -rf dist __pycache__
$ pyarmor testben.py
...

$ pyarmor gen --no-wrap testben.py benchmark.py benchmark2.py
$ pyarmor dist/testben.py

Python 3.7
test_import     :      0.971 ms
test_foo        :    306.261 ms
Table-4. Impact of Different Options

Option

Performance

Security

--no-wrap

Increase

Reduce

--obf-module 0

Slightly increase

Slightly reduce

--obf-code 0

Remarkable increase

Remarkable reduce

--obf-code 2

Reduce

Increase

--enable-rft

Almost same

Remarkable increase

--enable-themida

Remarkable reduce

Remarkable increase

--mix-str

Reduce

Increase

--assert-call

Reduce

Increase

--assert-import

Slightly reduce

Increase

--private

Reduce

Increase

--restrict

Reduce

Increase