Neurotechnology has been ranked among 8 most accurate face recognition algorithm vendors out of 39, with tenth most accurate algorithm out of 78 in the FRVT leaderboard. The submission was also ranked as one of the best in two difficult scenarios, with second most accurate result on a complex dataset collected from operational photos related to ongoing criminal investigations, and fourth most accurate result with unconstrained, photojournalism-style photos.
NIST has started the FRVT Ongoing evaluation of face verification algorithms in February 2017. Since then, 78 algorithms from 39 vendors were tested on 6 datasets of different complexity. Each vendor can submit multiple algorithms once in every three months, but the report takes into account only 2 latest submissions per vendor. Thus, only 62 algorithms are analyzed in the current report from NIST.
Modern face recognition algorithms are based on convolutional neural networks, and the easiest way to improve classification accuracy is to use larger neural network models. The figure below shows the accuracy of all tested algorithms and their model size (in bytes) as reported by NIST.
VISA dataset
Although six different datasets are used in this evaluation, NIST ranks all algorithms by FRR @ FAR=0.01 % on VISA dataset.
According to the dataset description, the face images have geometry in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type and pose is generally excellent. The complexity of this dataset rises from the fact that part of the images are live capture and the other part is photographed from paper photos. Additionally, captured subjects are from more than 100 countries and have great diversity in ages, including children.
Our latest submission is ranked among 10 most accurate face recognition algorithms in the FRVT leaderboard and only 7 vendors have more accurate algorithms.
MUGSHOT dataset
On a larger but better quality dataset of frontal faces named MUGSHOT our latest submission is ranked 17th, but only 12 vendors have more accurate algorithms, and the most accurate one has only slightly better 1.7 % FRR compared to ours 2.7 % FRR at the same FAR level. MUGHOST dataset captures only adult subjects from the United States with geometry in reasonable conformance with the ISO/IEC 19794-5 Full Frontal image type.
WEBCAM and SELFIE datasets
WEBCAM and SELFIE datasets are much smaller and probably the least usable for comparing the most accurate face recognition algorithms, because there are 10 algorithms from 7 vendors (including Neurotechnology) which have 0 % FRR on the first of them, and there are less than 500 face images in the second one. Additionally, the image pairs had been collected on the same day, which, according to NIST, could result in over-optimistic statements of accuracy.
WILD dataset
Non-frontal face recognition is evaluated with a more difficult WILD dataset, which contained face images captured in photojournalism-style. According to the dataset description, the images are very unconstrained, with wide yaw and pitch pose variation. Moreover, faces can be occluded, including hair and hands.
Our most accurate submission is ranked 4th among other vendors.
EXPLOITATION dataset
Finally, the most difficult EXPLOITATION dataset also contained faces captured in unconstrained environment. The images are operational. They are taken from ongoing investigations of child exploitation crimes. The images are arbitrarily unconstrained. Poses vary considerably around all three axes, including subject lying down. Resolution varies very widely. Faces can be occluded by other objects, including hair and hands.
Our most accurate submission is ranked 2nd among other vendors in these most challenging conditions.
