Accuracy and performance¶
Main operations of das-FaceBond (clustering and identification) are configurable by a given accuracy option. Such an option indicates how the system will proceed, and has the following accuracy values:
EXACT
: Requests to perform the operation exactly, being the more accurate option, it may be unfeasible for huge datasets (for instance, identifications over thousands of faces, or clustering over hundreds of faces) because of the time required to finish the operation.HIGH
: Reduces the accuracy of the system, but allows to perform identification over hundreds of thousands of faces, and clustering with thousands of faces.MEDIUM
: Reduces the accuracy of the system, but allows clustering with dozens of thousands of faces.LOW
: The less accurate one, but allowing clustering on hundreds of thousands of faces.
On identification, the system shows following accuracy and performance metrics:
- Accuracy is about 97.8% with MegaFace dataset, using a gallery with 10,000 distractors, and using the EXACT accuracy configuration.
- The system is able to search over 1,000,000 identification candidates per second when using HIGH accuracy (This speed is without considering the time to generate the embedding vector for the probe face image, and using HIGH accuracy on asynchronous identification. Using accuracy different than
EXACT
has a computational cost which is sublinear, so, when scaling down or up, computation time don’t changes linearly.). If using EXACT accuracy, the time needed is 5.8 seconds on average.
Following table shows the accuracy of the system depending on the number of elements in the gallery and the rank of the match. A rank-1 means that best match was found first in the list of candidates of the identification, and rank-10 means that best match was found up-to the first 10 candidates of the identification.
Gallery size | Rank-1 | Rank-10 |
---|---|---|
10 | 98.7% | 100.0% |
100 | 98.2% | 98.6% |
1,000 | 98.0% | 98.0% |
10,000 | 97.8% | 97.8% |
100,000 | 94.3% | 97.5% |
Regarding clustering operations, the system shows following accuracy and performance metrics:
- The clustering accuracy is about 97.6% with
setting=EXACT
and a minConfidence=0.97 on LFW benchmark dataset (Consider this minConfidence just orientative, it is task dependent, and in some cases it will be required to use a higher or lower value, usually in range [0.95, 0.99]). LFW dataset contains more than 13,000 images.
- Clustering performance is shown in the following table, depending on the gallery size and the indicated accuracy level (EXACT, HIGH, MEDIUM, LOW).
Gallery size | EXACT | HIGH | MEDIUM | LOW |
---|---|---|---|---|
≅ 100 | 9 seconds | - | - | - |
≅ 1,000 | 90 seconds | 4 minutes | 42 seconds | 14 seconds |
≅ 10,000 | 34 minutes | 21 minutes | 5 minutes | 80 seconds |
≅ 100,000 | 2 days | 6 hours | 1 hour | 15 minutes |
Operation considerations¶
Currently, the system is limited to work with galleries of up to 10.000.000 faces each.
Operation times¶
Biometric vectors generation time¶
Biometric vector generation can be accelerated by a factor x2 using GPUs. The biometric vector generation times with a NVidia Geforce RTX3090 GPU are as follows.
- 1M: 15h with 3090Ti
- 3M: 45h with 3090Ti