-
Notifications
You must be signed in to change notification settings - Fork 40
Performance
Raido Raagmaa edited this page Jun 20, 2024
·
61 revisions
In order to produce comparable test results all performance tests are done in the same server and with same methods. Tests are written as unit tests by using Contiperf2 and JUnit libraries and are located under the PerformanceTest class. Performance tests are run as Jenkins jobs in a Linux based environment and the latest digidoc4j release is compared against three earlier versions. After running the performance test job in Jenkins, Contiperf2 created a HTML-report under target/contiperf-report. Results are exported manually and combined in the table below.
Values in the test results are in milliseconds: 1: Average: Average response time for 1 sample 2: Max: Maximum response time for 1 sample
| Test (Test in Code) | Conditions | 5.3.1 19.06.2024 |
5.3.0 08.03.2024 |
5.2.0 11.08.2023 |
5.1.0* 13.03.2023 |
5.0.0* 23.05.2022 |
4.3.0* 18.11.2021 |
4.2.2* 18.11.2021 |
4.2.1* 25.08.2021 |
4.2.0* 18.05.2021 |
4.1.1* 18.05.2021 |
4.1.0* 18.05.2021 |
4.1.0-RC.1* 18.05.2021 |
4.0.0* 18.05.2021 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Validate Bdoc-TM (validateTestBDocTmSignatures) | 1000 samples in 1 thread | Average 48 ms (Max 4127 ms) | Average 46 ms (Max 4028 ms) | Average 63 ms (Max 4944 ms) | Average 69 ms (Max 3797 ms) | Average 50 ms (Max 3937 ms) | Average 39 ms (Max 3515 ms) | Average 40 ms (Max 3447 ms) | Average 40 ms (Max 4024 ms) | Average 39 ms (Max 3243 ms) | Average 40 ms (Max 3355 ms) | Average 38 ms (Max 3479 ms) | Average 38 ms (Max 3321 ms) | Average 31 ms (Max 1826 ms) |
| Validate ASiC-E (validateTestAsiceSignatures) | 1000 samples in 1 thread | Average 68 ms (Max 246 ms) | Average 74 ms (Max 304 ms) | Average 80 ms (Max 302 ms) | Average 88 ms (Max 533 ms) | Average 92 ms (Max 343 ms) | Average 96 ms (Max 671 ms) | Average 63 ms (Max 250 ms) | Average 63 ms (Max 319 ms) | Average 62 ms (Max 247 ms) | Average 76 ms (Max 500 ms) | Average 79 ms (Max 238 ms) | Average 64 ms (Max 807 ms) | Average 58 ms (Max 746 ms) |
| Validate ASiC-S (validateTestAsicsSignatures) | 1000 samples in 1 thread | Average 1 ms (Max 20 ms) | Average 1 ms (Max 16 ms) | Average 0 ms (Max 13 ms) | Average 0 ms (Max 16 ms) | Average 0 ms (Max 20 ms) | Average 0 ms (Max 12 ms) | Average 0 ms (Max 11 ms) | Average 0 ms (Max 16 ms) | Average 0 ms (Max 14 ms) | Average 0 ms (Max 13 ms) | Average 0 ms (Max 34 ms) | Average 0 ms (Max 11 ms) | Average 0 ms (Max 11 ms) |
| Validate Ddoc (validateTestDDocSignatures) | 1000 samples in 1 thread | Average 4 ms (Max 17 ms) | Average 4 ms (Max 14 ms) | Average 6 ms (Max 18 ms) | Average 7 ms (Max 32 ms) | Average 6 ms (Max 18 ms) | Average 7 ms (Max 49 ms) | Average 6 ms (Max 45 ms) | Average 6 ms (Max 38 ms) | Average 6 ms (Max 41 ms) | Average 6 ms (Max 37 ms) | Average 6 ms (Max 44 ms) | Average 6 ms (Max 33 ms) | Average 6 ms (Max 48 ms) |
| Validate Bdoc-TM in 20 threads (validateTestBDocTmSignaturesInThreads) | 1000 samples in 20 threads | Average 445 ms (Max 813 ms) | Average 434 ms (Max 759 ms) | Average 566 ms (Max 1049 ms) | Average 936 ms (Max 1331 ms) | Average 443 ms (Max 808 ms) | Average 364 ms (Max 533 ms) | Average 352 ms (Max 558 ms) | Average 333 ms (Max 556 ms) | Average 331 ms (Max 508 ms) | Average 327 ms (Max 600 ms) | Average 322 ms (Max 563 ms) | Average 340 ms (Max 526 ms) | Average 310 ms (Max 470 ms) |
| Validate ASiC-E in 20 threads (validateTestAsiceSignaturesInThreads) | 1000 samples in 20 threads | Average 704 ms (Max 1513 ms) | Average 729 ms (Max 1536 ms) | Average 895 ms (Max 1996 ms) | Average 1099 ms (Max 2902 ms) | Average 880 ms (Max 1939 ms) | Average 1044 ms (Max 2063 ms) | Average 748 ms (Max 1699 ms) | Average 743 ms (Max 1928 ms) | Average 720 ms (Max 1706 ms) | Average 839 ms (Max 1812 ms) | Average 728 ms (Max 1653 ms) | Average 837 ms (Max 1831 ms) | Average 648 ms (Max 1492 ms) |
| Validate ASiC-S in 20 threads (validateTestAsicsSignaturesInThreads) | 1000 samples in 20 threads | Average 10 ms (Max 126 ms) | Average 11 ms (Max 171 ms) | Average 9 ms (Max 131 ms) | Average 8 ms (Max 131 ms) | Average 7 ms (Max 114 ms) | Average 7 ms (Max 129 ms) | Average 8 ms (Max 127 ms) | Average 7 ms (Max 123 ms) | Average 7 ms (Max 137 ms) | Average 6 ms (Max 129 ms) | Average 6 ms (Max 123 ms) | Average 5 ms (Max 142 ms) | Average 8 ms (Max 158 ms) |
| Validate Ddoc in 20 threads (validateTestDDocSignaturesInThreads) | 1000 samples in 20 threads | Average 53 ms (Max 198 ms) | Average 53 ms (Max 199 ms) | Average 75 ms (Max 250 ms) | Average 98 ms (Max 323 ms) | Average 73 ms (Max 251 ms) | Average 78 ms (Max 224 ms) | Average 77 ms (Max 218 ms) | Average 74 ms (Max 248 ms) | Average 70 ms (Max 227 ms) | Average 72 ms (Max 211 ms) | Average 74 ms (Max 241 ms) | Average 73 ms (Max 219 ms) | Average 58 ms (Max 176 ms) |
| Validate large Bdoc-TM (validateTestBdocLargeContainer) | 1000 samples in 1 thread | Average 187 ms (Max 382 ms) | Average 216 ms (Max 404 ms) | Average 220 ms (Max 445 ms) | Average 241 ms (Max 983 ms) | Average 168 ms (Max 347 ms) | Average 186 ms (Max 958 ms) | Average 147 ms (Max 432 ms) | Average 148 ms (Max 595 ms) | Average 147 ms (Max 305 ms) | Average 150 ms (Max 612 ms) | Average 132 ms (Max 290 ms) | Average 159 ms (Max 280 ms) | Average 125 ms (Max 220 ms) |
| Validate large ASiC-E (validateTestAsiceLargeContainer) | 1000 samples in 1 thread | Average 190 ms (Max 711 ms) | Average 219 ms (Max 915 ms) | Average 226 ms (Max 445 ms) | Average 255 ms (Max 778 ms) | Average 178 ms (Max 304 ms) | Average 201 ms (Max 1155 ms) | Average 159 ms (Max 756 ms) | Average 155 ms (Max 422 ms) | Average 155 ms (Max 600 ms) | Average 168 ms (Max 945 ms) | Average 154 ms (Max 659 ms) | Average 136 ms (Max 339 ms) | Average 135 ms (Max 314 ms) |
| Validate large ASiC-S (validateTestAsicsLargeContainer) | 1000 samples in 1 thread | Average 64 ms (Max 77 ms) | Average 63 ms (Max 94 ms) | Average 70 ms (Max 104 ms) | Average 58 ms (Max 143 ms) | Average 64 ms (Max 132 ms) | Average 83 ms (Max 159 ms) | Average 83 ms (Max 148 ms) | Average 80 ms (Max 145 ms) | Average 79 ms (Max 150 ms) | Average 45 ms (Max 88 ms) | Average 45 ms (Max 87 ms) | Average 45 ms (Max 151 ms) | Average 43 ms (Max 95 ms) |
| Validate large Ddoc (validateTestDdocLargeContainer) | 1000 samples in 1 thread | Average 112 ms (Max 252 ms) | Average 108 ms (Max 278 ms) | Average 145 ms (Max 334 ms) | Average 158 ms (Max 512 ms) | Average 168 ms (Max 311 ms) | Average 181 ms (Max 278 ms) | Average 199 ms (Max 300 ms) | Average 191 ms (Max 252 ms) | Average 171 ms (Max 270 ms) | Average 192 ms (Max 323 ms) | Average 187 ms (Max 283 ms) | Average 194 ms (Max 304 ms) | Average 128 ms (Max 245 ms) |
| Validate Bdoc-TM with 1000 signatures (validateBDocWith1000Signatures) | 1 sample in 1 thread | Average 25876 ms (Max 25876 ms) | Average 24845 ms (Max 24845 ms) | Average 33699 ms (Max 33699 ms) | Average 44677 ms (Max 44677 ms) | Average 29579 ms (Max 29579 ms) | Average 23278 ms (Max 23278 ms) | Average 23767 ms (Max 23767 ms) | Average 21645 ms (Max 21545 ms) | Average 21486 ms (Max 21486 ms) | Average 21636 ms (Max 21636 ms) | Average 21197 ms (Max 21197 ms) | Average 20823 ms (Max 20823 ms) | Average 20496 ms (Max 20496 ms) |
| Validate ASiC-E with 1000 signatures (validateAsiceWith1000Signatures) | 1 sample in 1 thread | Average 48825 ms (Max 48825 ms) | Average 50732 ms (Max 50732 ms) | Average 58656 ms (Max 58656 ms) | Average 67151 ms (Max 67151 ms) | Average 60091 ms (Max 60091 ms) | Average 60628 ms (Max 60628 ms) | Average 41222 ms (Max 41222 ms) | Average 41184 ms (Max 41184 ms) | Average 40914 ms (Max 40914 ms) | Average 48280 ms (Max 48280 ms) | Average 49291 ms (Max 49291 ms) | Average 38216 ms (Max 38216 ms) | Average 37690 ms (Max 37690 ms) |
| Validate ASiC-S with 1000 signatures (validateAsicsWith1000Signatures) | 1 sample in 1 thread | Average 86 ms (Max 86 ms) | Average 86 ms (Max 86 ms) | Average 95 ms (Max 95 ms) | Average 65 ms (Max 65 ms) | Average 82 ms (Max 82 ms) | Average 111 ms (Max 111 ms) | Average 130 ms (Max 130 ms) | Average 125 ms (Max 125 ms) | Average 101 ms (Max 101 ms) | Average 103 ms (Max 103 ms) | Average 115 ms (Max 115 ms) | Average 106 ms (Max 106 ms) | Average 97 ms (Max 97 ms) |
| Validate Ddoc with 1000 signatures (validateDddocWith1000Signatures) | 1 sample in 1 thread | Average 6760 ms (Max 6760 ms) | Average 6418 ms (Max 6418 ms) | Average 9410 ms (Max 9410 ms) | Average 8329 ms (Max 8329 ms) | Average 8960 ms (Max 8960 ms) | Average 10072 ms (Max 10072 ms) | Average 9189 ms (Max 9189 ms) | Average 8794 ms (Max 8794 ms) | Average 8456 ms (Max 8456 ms) | Average 8709 ms (Max 8709 ms) | Average 8444 ms (Max 8444 ms) | Average 8573 ms (Max 8573 ms) | Average 8568 ms (Max 8568 ms) |
| Open Bdoc-TM details (openBDocTmContainerDetails) | 1000 samples in 1 thread | Average 16 ms (Max 192 ms) | Average 16 ms (Max 191 ms) | Average 22 ms (Max 223 ms) | Average 22 ms (Max 205 ms) | Average 14 ms (Max 166 ms) | Average 10 ms (Max 84 ms) | Average 10 ms (Max 76 ms) | Average 10 ms (Max 69 ms) | Average 9 ms (Max 71 ms) | Average 9 ms (Max 77 ms) | Average 9 ms (Max 71 ms) | Average 9 ms (Max 73 ms) | Average 8 ms (Max 55 ms) |
| Open ASiC-E details (openAsiceContainerDetails) | 1000 samples in 1 thread | Average 22 ms (Max 424 ms) | Average 21 ms (Max 402 ms) | Average 29 ms (Max 496 ms) | Average 31 ms (Max 515 ms) | Average 19 ms (Max 420 ms) | Average 13 ms (Max 281 ms) | Average 13 ms (Max 279 ms) | Average 13 ms (Max 261 ms) | Average 12 ms (Max 271 ms) | Average 12 ms (Max 241 ms) | Average 12 ms (Max 153 ms) | Average 12 ms (Max 308 ms) | Average 11 ms (Max 153 ms) |
| Open ASiC-S details (openAsicsContainerDetails) | 1000 samples in 1 thread | Average 0 ms (Max 20 ms) | Average 0 ms (Max 26 ms) | Average 0 ms (Max 49 ms) | Average 0 ms (Max 4 ms) | Average 0 ms (Max 2 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 5 ms) | Average 0 ms (Max 11 ms) | Average 0 ms (Max 9 ms) | Average 0 ms (Max 10 ms) | Average 0 ms (Max 8 ms) | Average 0 ms (Max 8 ms) | Average 0 ms (Max 16 ms) |
| Open Ddoc details (openDdocContainerDetails) | 1000 samples in 1 thread | Average 3 ms (Max 25 ms) | Average 3 ms (Max 15 ms) | Average 4 ms (Max 46 ms) | Average 4 ms (Max 18 ms) | Average 4 ms (Max 11 ms) | Average 4 ms (Max 38 ms) | Average 4 ms (Max 31 ms) | Average 3 ms (Max 26 ms) | Average 3 ms (Max 31 ms) | Average 3 ms (Max 28 ms) | Average 3 ms (Max 36 ms) | Average 3 ms (Max 28 ms) | Average 3 ms (Max 20 ms) |
| Save container on Disk (saveExistingContainerOnDisk) | 1000 samples in 20 threads | Average 231 ms (Max 919 ms) | Average 231 ms (Max 835 ms) | Average 292 ms (Max 797 ms) | Average 405 ms (Max 911 ms) | Average 216 ms (Max 633 ms) | Average 149 ms (Max 507 ms) | Average 147 ms (Max 465 ms) | Average 136 ms (Max 405 ms) | Average 139 ms (Max 502 ms) | Average 141 ms (Max 626 ms) | Average 133 ms (Max 380 ms) | Average 134 ms (Max 459 ms) | Average 118 ms (Max 389 ms) |
| TSL loading (loadingTSL) ** | 10 samples in 1 thread | Average 24874 ms (Max 35972 ms) | Average 24114 ms (Max 33281 ms) | Average 12004 ms (Max 16084 ms) | Average 12006 ms (Max 16450 ms) | Average 11596 ms (Max 23167 ms) | Average 9351 ms (Max 12626 ms) | Average 8681 ms (Max 11451 ms) | Average 8131 ms (Max 11572 ms) | Average 8478 ms (Max 11694 ms) | Average 7964 ms (Max 11268 ms) | Average 8066 ms (Max 10527 ms) | Average 6315 ms (Max 8826 ms) | Average 5028 ms (Max 7631 ms) |
| Create TM signature (createBDocTmSignature) | 50 samples in 1 thread | Discontinued | Discontinued | Discontinued | Average 150 ms (Max 804 ms) | Average 140 ms (Max 533 ms) | Average 161 ms (Max 632 ms) | Average 99 ms (Max 359 ms) | Average 100 ms (Max 398 ms) | Average 142 ms (Max 690 ms) | - | - | - | - |
| Create TS signature (createAsicSignature) | 50 samples in 1 thread | Average 210 ms (Max 672 ms) | Average 215 ms (Max 819 ms) | Average 221 ms (Max 601 ms) | Average 203 ms (Max 420 ms) | Average 201 ms (Max 318 ms) | Average 199 ms (Max 248 ms) | Average 205 ms (Max 294 ms) | Average 200 ms (Max 294 ms) | Average 375 ms (Max 2008 ms) | - | - | - | - |
Performance tests are run in an Jenkins Linux VM environment with the following parameters: Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz 2 virtual cores with 4GB RAM.
* Before version 5.2.0, the available RAM of the performance test environment was 2 GB.
** TSL loading times are strongly affected by availability of the requested resource.
Last run in 02.02.2021
| Test (Test in Code) | Conditions | 4.1.0 02.02.2021 |
4.1.0-RC.1 02.02.2021 |
|---|---|---|---|
| Validate Bdoc-TM (validateTestBDocTmSignatures) | 1000 samples in 1 thread | Average 33 ms (Max 2176 ms) | Average 25 ms (Max 1129 ms) |
| Validate ASiC-E (validateTestAsiceSignatures) | 1000 samples in 1 thread | Average 52 ms (Max 105 ms) | Average 49 ms (Max 154 ms) |
| Validate ASiC-S (validateTestAsicsSignatures) | 1000 samples in 1 thread | Average 0 ms (Max 7 ms) | Average 0 ms (Max 5 ms) |
| Validate Ddoc (validateTestDDocSignatures) | 1000 samples in 1 thread | Average 4 ms (Max 9 ms) | Average 4 ms (Max 25 ms) |
| Validate Bdoc-TM in 20 threads (validateTestBDocTmSignaturesInThreads) | 1000 samples in 20 threads | Average 107 ms (Max 299 ms) | Average 78 ms (Max 158 ms) |
| Validate ASiC-E in 20 threads (validateTestAsiceSignaturesInThreads) | 1000 samples in 20 threads | Average 185 ms (Max 823 ms) | Average 149 ms (Max 777 ms) |
| Validate ASiC-S in 20 threads (validateTestAsicsSignaturesInThreads) | 1000 samples in 20 threads | Average 3 ms (Max 26 ms) | Average 3 ms (Max 25 ms) |
| Validate Ddoc in 20 threads (validateTestDDocSignaturesInThreads) | 1000 samples in 20 threads | Average 31 ms (Max 107 ms) | Average 15 ms (Max 88 ms) |
| Validate large Bdoc-TM (validateTestBdocLargeContainer) | 1000 samples in 1 thread | Average 121 ms (Max 235 ms) | Average 119 ms (Max 185 ms) |
| Validate large ASiC-E (validateTestAsiceLargeContainer) | 1000 samples in 1 thread | Average 123 ms (Max 347 ms) | Average 122 ms (Max 196 ms) |
| Validate large ASiC-S (validateTestAsicsLargeContainer) | 1000 samples in 1 thread | Average 48 ms (Max 70 ms) | Average 48 ms (Max 62 ms) |
| Validate large Ddoc (validateTestDdocLargeContainer) | 1000 samples in 1 thread | Average 119 ms (Max 201 ms) | Average 108 ms (Max 182 ms) |
| Validate Bdoc-TM with 1000 signatures (validateBDocWith1000Signatures) | 1 sample in 1 thread | Average 10452 ms (Max 10452 ms) | Average 9410 ms (Max 9410 ms) |
| Validate ASiC-E with 1000 signatures (validateAsiceWith1000Signatures) | 1 sample in 1 thread | Average 15833 ms (Max 15833 ms) | Average 14669 ms (Max 14669 ms) |
| Validate ASiC-S with 1000 signatures (validateAsicsWith1000Signatures) | 1 sample in 1 thread | Average 87 ms (Max 87 ms) | Average 87 ms (Max 87 ms) |
| Validate Ddoc with 1000 signatures (validateDddocWith1000Signatures) | 1 sample in 1 thread | Average 6686 ms (Max 6686 ms) | Average 6692 ms (Max 6692 ms) |
| Open Bdoc-TM details (openBDocTmContainerDetails) | 1000 samples in 1 thread | Average 8 ms (Max 62 ms) | Average 7 ms (Max 56 ms) |
| Open ASiC-E details (openAsiceContainerDetails) | 1000 samples in 1 thread | Average 10 ms (Max 157 ms) | Average 9 ms (Max 125 ms) |
| Open ASiC-S details (openAsicsContainerDetails) | 1000 samples in 1 thread | Average 0 ms (Max 1 ms) | Average 0 ms (Max 1 ms) |
| Open Ddoc details (openDdocContainerDetails) | 1000 samples in 1 thread | Average 3 ms (Max 24 ms) | Average 2 ms (Max 24 ms) |
| Save container on Disk (saveExistingContainerOnDisk) | 1000 samples in 20 threads | Average 80 ms (Max 381 ms) | Average 69 ms (Max 486 ms) |
| TSL loading (loadingTSL) | 10 samples in 1 thread | Average 5325 ms (Max 6111 ms) | Average 4645 ms (Max 5341 ms) |
| Create TM signature (createBDocTmSignature) | 50 samples in 1 thread | Average 114 ms (Max 317 ms) | Average 89 ms (Max 296 ms) |
| Create TS signature (createAsicSignature) | 50 samples in 1 thread | Average 209 ms (Max 442 ms) | Average 241 ms (Max 1130 ms) |
Performance tests are run in an Jenkins Linux VM environment with the following parameters: Intel(R) Xeon(R) CPU E5-2650L v3 @ 1.80GHz 8 virtual cores with 8GB RAM.
Last run in 10.01.2020
| Test (Test in Code) | Conditions | 4.0.3 19.11.2020 |
4.0.0 10.02.2020 |
4.0.0-RC.1 11.12.2019 |
3.3.1 11.02.2020 |
3.3.0 21.08.2019 |
3.2.0 17.06.2019 |
3.1.1 11.04.2019 |
3.1.0 04.03.2019 |
3.0.0 18.11.2018 |
2.1.1 18.11.2018 |
2.1.0 18.11.2018 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Validate Bdoc-TM (validateTestBDocTmSignatures) | 1000 samples in 1 thread | Average 36 ms (Max 1739 ms) | Average 33 ms (Max 110 ms) | Average 28 ms (Max 101 ms) | Average 30 ms (Max 115 ms) | Average 28 ms (Max 95 ms) | Average 28 ms (Max 109 ms) | Average 29 ms (Max 92 ms) | Average 38 ms (Max 105 ms) | Average 28 ms (Max 1,431 ms) | Average 31 ms (Max 55 ms) | Average 38 ms (Max 1,374 ms) |
| Validate ASiC-E (validateTestAsiceSignatures) | 1000 samples in 1 thread | Average 81 ms (Max 155 ms) | Average 68 ms (Max 151 ms) | Average 38 ms (Max 110 ms) | Average 32 ms (Max 114 ms) | Average 29 ms (Max 84 ms) | Average 29 ms (Max 81 ms) | Average 30 ms (Max 104 ms) | Average 38 ms (Max 92 ms) | Average 21 ms (Max 73 ms) | Average 29 ms (Max 43 ms) | Average 31 ms (Max 48 ms) |
| Validate ASiC-S (validateTestAsicsSignatures) | 1000 samples in 1 thread | Average 1 ms (Max 8 ms) | Average 1 ms (Max 17 ms) | Average 0 ms (Max 9 ms) | Average 1 ms (Max 27 ms) | Average 1 ms (Max 8 ms) | Average 1 ms (Max 6 ms) | Average 1 ms (Max 12 ms) | Average 1 ms (Max 12 ms) | Average 1 ms (Max 15 ms) | Average 1 ms (Max 15 ms) | Average 1 ms (Max 9 ms) |
| Validate Ddoc (validateTestDDocSignatures) | 1000 samples in 1 thread | Average 7 ms (Max 28 ms) | Average 8 ms (Max 24 ms) | Average 6 ms (Max 32 ms) | Average 7 ms (Max 39 ms) | Average 7 ms (Max 12 ms) | Average 7 ms (Max 13 ms) | Average 7 ms (Max 15 ms) | Average 7 ms (Max 28 ms) | Average 6 ms (Max 430 ms) | Average 5 ms (Max 16 ms) | Average 6 ms (Max 425 ms) |
| Validate Bdoc-TM in 20 threads (validateTestBDocTmSignaturesInThreads) | Average 107 ms (Max 249 ms) | 1000 samples in 20 threads | Average 127 ms (Max 303 ms) | Average 122 ms (Max 362 ms) | Average 113 ms (Max 251 ms) | Average 104 ms (Max 269ms) | Average 104 ms (Max 282 ms) | Average 110 ms (Max 271 ms) | Average 139 ms (Max 393 ms) | Average 90 ms (Max 223 ms) | Average 259 ms (Max 975 ms) | Average 249 ms (Max 636 ms) |
| Validate ASiC-E in 20 threads (validateTestAsiceSignaturesInThreads) | 1000 samples in 20 threads | Average 222 ms (Max 992 ms) | Average 212 ms (Max 550 ms) | Average 134 ms (Max 330 ms) | Average 136 ms (Max 301 ms) | Average 114 ms (Max 362 ms) | Average 114 ms (Max 232 ms) | Average 121 ms (Max 312 ms) | Average 150 ms (Max 789 ms) | Average 103 ms (Max 264 ms) | Average 260 ms (Max 669 ms) | Average 249 ms (Max 669 ms) |
| Validate ASiC-S in 20 threads (validateTestAsicsSignaturesInThreads) | 1000 samples in 20 threads | Average 4 ms (Max 27 ms) | Average 6 ms (Max 59 ms) | Average 4 ms (Max 40 ms) | Average 6 ms (Max 58 ms) | Average 5 ms (Max 62 ms) | Average 5 ms (Max 67 ms) | Average 4 ms (Max 68 ms) | Average 5 ms (Max 72 ms) | Average 5 ms (Max 53 ms) | Average 4 ms (Max 61 ms) | Average 5 ms (Max 57 ms) |
| Validate Ddoc in 20 threads (validateTestDDocSignaturesInThreads) | 1000 samples in 20 threads | Average 23 ms (Max 136 ms) | Average 26 ms (Max 159 ms) | Average 21 ms (Max 125 ms) | Average 28 ms (Max 125 ms) | Average 24ms (Max 116 ms) | Average 26 ms (Max 143 ms) | Average 25 ms (Max 129 ms) | Average 26 ms (Max 135 ms) | Average 19 ms (Max 99 ms) | Average 16 ms (Max 100 ms) | Average 16 ms (Max 93 ms) |
| Validate large Bdoc-TM (validateTestBdocLargeContainer) | 1000 samples in 1 thread | Average 183 ms (Max 296 ms) | Average 178 ms (Max 295 ms) | Average 143 ms (Max 276 ms) | Average 124 ms (Max 250 ms) | Average 115 ms (Max 239 ms) | Average 115 ms (Max 167 ms) | Average 121 ms (Max 188 ms) | Average 128 ms (Max 179 ms) | Average 102 ms (Max 152 ms) | Average 110 ms (Max 158 ms) | Average 75 ms (Max 94 ms) |
| Validate large ASiC-E (validateTestAsiceLargeContainer) | 1000 samples in 1 thread | Average 183 ms (Max 296 ms) | Average 180 ms (Max 280 ms) | Average 143 ms (Max 226 ms) | Average 126 ms (Max 327 ms) | Average 115 ms (Max 168 ms) | Average 115 ms (Max 165 ms) | Average 121 ms (Max 175 ms) | Average 129 ms (Max 187 ms) | Average 107 ms (Max 161 ms) | Average 110 ms (Max 125 ms) | Average 76 ms (Max 289 ms) |
| Validate large ASiC-S (validateTestAsicsLargeContainer) | 1000 samples in 1 thread | Average 71 ms (Max 127 ms) | Average 74 ms (Max 117 ms) | Average 73 ms (Max 305 ms) | Average 79 ms (Max 106 ms) | Average 77 ms (Max 107 ms) | Average 77 ms (Max 101 ms) | Average 77 ms (Max 100 ms) | Average 77 ms (Max 101 ms) | Average 77 ms (Max 93 ms) | Average 77 ms (Max 110 ms) | Average 76 ms (Max 98 ms) |
| Validate large Ddoc (validateTestDdocLargeContainer) | 1000 samples in 1 thread | Average 146 ms (Max 230 ms) | Average 145 ms (Max 302 ms) | Average 147 ms (Max 306 ms) | Average 149 ms (Max 191 ms) | Average 145 ms (Max 239 ms) | Average 149 ms (Max 265 ms) | Average 141 ms (Max 279 ms) | Average 138 ms (Max 269 ms) | Average 146 ms (Max 264 ms) | Average 166 ms (Max 398 ms) | Average 145 ms (Max 230 ms) |
| Validate Bdoc-TM with 1000 signatures (validateBDocWith1000Signatures) | 1 sample in 1 thread | Average 15673 ms (Max 15673 ms) | Average 15,056 ms (Max 15,056 ms) | Average 12,383 ms (Max 12,383 ms) | Average 11,260 ms (Max 11,260 ms) | Average 10,516 ms (Max 10,516 ms) | Average 10,526 ms (Max 10,526 ms) | Average 10,729 ms (Max 10,729 ms) | Average 12,176 ms (Max 12,176 ms) | Average 9,228 ms (Max 9,228 ms) | Average 8,791 ms (Max 8,791 ms) | Average 10,454 ms (Max 10,454 ms) |
| Validate ASiC-E with 1000 signatures (validateAsiceWith1000Signatures) | 1 sample in 1 thread | Average 23842 ms (Max 23842 ms) | Average 21,423 ms (Max 21,423 ms) | Average 15,923 ms (Max 15,923 ms) | Average 14,159 ms (Max 14,159 ms) | Average 12,336 ms (Max 12,336 ms) | Average 12,842 ms (Max 12,842 ms) | Average 13,354 ms (Max 13,354 ms) | Average 14,254 ms (Max 14,254 ms) | Average 11,967 ms (Max 11,967 ms) | Average 11,049 ms (Max 11,049 ms) | Average 12,947 ms (Max 12,947 ms) |
| Validate ASiC-S with 1000 signatures (validateAsicsWith1000Signatures) | 1 sample in 1 thread | Average 128 ms (Max 128 ms) | Average 136 ms (Max 136 ms) | Average 135 ms (Max 135 ms) | Average 327 ms (Max 327 ms) | Average 143 ms (Max 143 ms) | Average 162 ms (Max 162 ms) | Average 144 ms (Max 144 ms) | Average 143 ms (Max 143 ms) | Average 154 ms (Max 154 ms) | Average 145 ms (Max 145 ms) | Average 152 ms (Max 152 ms) |
| Validate Ddoc with 1000 signatures (validateDddocWith1000Signatures) | 1 sample in 1 thread | Average 10247 ms (Max 10247 ms) | Average 10,070 ms (Max 10,070 ms) | Average 8,162 ms (Max 8,162 ms) | Average 11,454 ms (Max 11,454 ms) | Average 9,507 ms (Max 9,507 ms) | Average 10,319 ms (Max 10,319 ms) | Average 10,601 ms (Max 10,601 ms) | Average 9,720 ms (Max 9,720 ms) | Average 7,691 ms (Max 7,691 ms) | Average 7,237 ms (Max 7,237 ms) | Average 7,133 ms (Max 7,133 ms) |
| Open Bdoc-TM details (openBDocTmContainerDetails) | 1000 samples in 1 thread | Average 10 ms (Max 79 ms) | Average 10 ms (Max 79 ms) | Average 12 ms (Max 76 ms) | Average 10 ms (Max 76 ms) | Average 8 ms (Max 67 ms) | Average 7 ms (Max 68 ms) | Average 7 ms (Max 65 ms) | Average 7 ms (Max 59 ms) | Average 7 ms (Max 58 ms) | Average 6 ms (Max 64 ms) | Average 6 ms (Max 19 ms) |
| Open ASiC-E details (openAsiceContainerDetails) | 1000 samples in 1 thread | Average 14 ms (Max 155 ms) | Average 14 ms (Max 87 ms) | Average 12 ms (Max 76 ms) | Average 10 ms (Max 67 ms) | Average 9 ms (Max 67 ms) | Average 9 ms (Max 61 ms) | Average 10 ms (Max 69 ms) | Average 9 ms (Max 65 ms) | Average 8 ms (Max 137 ms) | Average 8 ms (Max 22 ms) | Average 9 ms (Max 85 ms) |
| Open ASiC-S details (openAsicsContainerDetails) | 1000 samples in 1 thread | Average 0 ms (Max 1 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 14 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 19 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 1 ms) | Average 0 ms (Max 7 ms) | Average 0 ms (Max 1 ms) |
| Open Ddoc details (openDdocContainerDetails) | 1000 samples in 1 thread | Average 4 ms (Max 29 ms) | Average 4 ms (Max 25 ms) | Average 3 ms (Max 15 ms) | Average 4 ms (Max 34 ms) | Average 4 ms (Max 11 ms) | Average 4 ms (Max 20 ms) | Average 4 ms (Max 11 ms) | Average 4 ms (Max 28 ms) | Average 3 ms (Max 24 ms) | Average 3 ms (Max 19 ms) | Average 3 ms (Max 19 ms) |
| Save container on Disk (saveExistingContainerOnDisk) | 1000 samples in 20 threads | Average 53 ms (Max 527 ms) | Average 57 ms (Max 576 ms) | Average 56 ms (Max 710 ms) | Average 41 ms (Max 247 ms) | Average 59 ms (Max 330 ms) | Average 60 ms (Max 363 ms) | Average 82 ms (Max 613 ms) | Average 50 ms (Max 247 ms) | Average 44 ms (Max 351 ms) | Average 53 ms (Max 1,119 ms) | Average 54 ms (Max 521 ms) |
| TSL loading (loadingTSL) | 10 samples in 1 thread | Average 8353 ms (Max 9809 ms) | Average 6,027 ms (Max 6,516 ms) | Average 5,563 ms (Max 5,843 ms) | Average 2,864 ms (Max 3,133 ms) | Average 3,234 ms (Max 5,081 ms) | Average 3,003 ms (Max 3,144 ms) | Average 3,801 ms (Max 5,960 ms) | Average 2,915 ms (Max 3,815 ms) | Average 2,908 ms (Max 3,985 ms) | Average 2,930 ms (Max 4,115 ms) | Average 3,007 ms (Max 4,145 ms) |
| Create TM signature (createBDocTmSignature) | 50 samples in 1 thread | Average 121 ms (Max 331 ms) | Average 502 ms (Max 1,410 ms) | Average 378 ms (Max 1,006 ms) | Average 364 ms (Max 895 ms) | Average 343 ms (Max 1,096 ms) | Average 309 ms (Max 541 ms) | Average 303 ms (Max 578 ms) | Average 332 ms (Max 2,298 ms) | Average 320 ms (Max 2,084 ms) | Average 293 ms (Max 731 ms) | Average 275 ms (Max 420 ms) |
| Create TS signature (createAsicSignature) | 50 samples in 1 thread | Average 217 ms (Max 485 ms) | Average 599 ms (Max 2,268 ms) | Average 442 ms (Max 819 ms) | Average 500 ms (Max 1,466 ms) | Average 447 ms (Max 1,198 ms) | Average 375 ms (Max 729 ms) | Average 354 ms (Max 730 ms) | Average 383 ms (Max 2,124 ms) | Average 330 ms (Max 399 ms) | Average 321 ms (Max 461 ms) | Average 322 ms (Max 377 ms) |
In versions 2.1.1 and prior logging level must be at least INFO, otherwise DDOC validation is much slower.
Performance tests are run in an Jenkins Linux VM environment with the following parameters: Intel(R) Xeon(R) CPU E5-2650L v3 @ 1.80GHz 8 virtual cores with 4GB RAM.
- Validate Bdoc-TM : Validation of one small BDOC container with one LT_TM signature.
- Validate ASiC-E : Validation of one small ASiC-E container with one LT signature.
- Validate ASiC-S : Validation of one small ASiC-S container.
- Validate Ddoc : Validation of one small DDOC container with one signature.
- Validate Bdoc-TM in 20 threads : Validation of one small BDOC container with one LT_TM signature.
- Validate ASiC-E in 20 threads : Validation of one small ASiC-E container with one LT signature.
- Validate ASiC-S in 20 threads : Validation of one small ASiC-S container.
- Validate Ddoc in 20 threads : Validation of one small DDOC container with one signature.
- Validate large Bdoc-TM : Validation of one large BDOC container (size is 2.8MB) with one LT_TM signature.
- Validate large ASiC-E : Validation of one large ASiC-E container (size is 2.8MB) with one LT signature.
- Validate large ASiC-S : Validation of one large ASiC-S container (size is 2.8MB).
- Validate large Ddoc : Validation of one large DDOC container (size is 2.8MB) with one signature.
- Save container on Disk : Saving of one BDOC container to test folder and checking the existence of the file on disk.
- TSL loading : Refreshing TSL data in context of PROD-environment.
- Create TM signature : Signing of one existing BDOC container with PKCS12 token, creating a signature with LT_TM profile.
- Create TS signature : Signing of one existing ASiC-E container with PKCS12 token, creating a signature with LT profile.
Official builds are provided through releases. If you want support, you need to be using official builds. For assistance, contact us by email [email protected]. Additional information can be found in wiki Q&A and on ID.ee portal.
For staying up to date with news impacting services and applications that use the DigiDoc4j library, join DigiDoc4j library newsletter.
Source code is provided on "as is" terms with no warranty (see license for more information). Do not file GitHub issues with generic support requests.