GMP performance guard Last modified: 2016-12-17 |

We're working on an automated performance regression avoidance mechanism for GMP. The goal is to make GMP improve monotonously, for all operand sizes.

- A weighted mean for an interval 1-n. The weight function should favour small sizes, but also perhaps common sizes like 512, 1024, 2048... The upper size limit n will vary with measured function; in particular it should never explore sizes unused due to standard algorithm choices.
- Polynomial fitting of functions' performance. Depending on measured function, fit either linear functions T(n) = A + Bn or quadratic functions T(n) = A + Bn + Cn^2, then apply least-squares fitting. If any of A, B, or C increases it is a performance regression.
- Check for anomalies like T(mpn_addmul_1(i)) < T(mpn_addlsh_n(i)), T(mpn_addmul_2(i)) < T(mpn_mul_1(i)) + T(mpn_addmul_1(i)). See table below.

function should be faster than... T(mpn_addlsh_n(i)) T(mpn_addmul_1(i)) T(mpn_mul_2(i)) T(mpn_mul_1(i)) + T(mpn_addmul_1(i)) T(mpn_addmul_2(i)) 2 × T(mpn_addmul_1(i)) T(mpn_lshift(i)) T(mpn_mul_1(i)) T(mpn_ANY(i)) T(mpn_ANY(i+1)) Normal speed checks. The expression in the left column should never be greater than the expression in the right column, for any i.

Performance data from previous measurements are kept in a database. To be stored: System identifier, system data, source repository identifier, source revision, and actual measurements.

To avoid data base poisoning, each test run are assigned an encrypted salted string, which is decrypted by the report reception system. If it doesn't decrypt to "This is a benevolent report XXX" the report is just discarded.

Large regressions for individual (functions × i). Threshold: ≈ 10% worse than the all-time-low.

Regressions of A, B, C as per the polynomial fitting.

Regressions of the weighted mean performance. Threshold: ≈ 2% worse than the all-time-low.