Evaluating the Performance of Biometric Authentication Technology
The authentication performance of the biometric authentication technology is one of the indications that the user is most interested in introducing the biometric authentication.
If the authentication performance can be expressed as numerical values by several scales, the user can know the authentication performance by simply comparing them.
Although the numerical value relating to the performance of the product is clarified by the product developer even now, the user is not satisfied only with the value.
In this article, the difficulty of performance evaluation particular to biometric authentication technology is described.
*) Difficulty in Unifying Evaluation Data
When comparing different authentication technologies, it is necessary to use the same evaluation data for measuring the superiority or inferiority of technology accurately.
Regarding biometric authentication, the evaluation data are equivalent to biological information and it is difficult to duplicate data from the viewpoint of protection of personal information, so it is not easy to hold common evaluation data among evaluators.
As for vein data, since vein authentication uses infrared images, there is a problem that images are different for each product, which is different from fingerprints and faces, and it is difficult to define a common image.
For these reasons, it is actually the case that evaluation data is prepared for each evaluator and performance values are given out.
Since the evaluation data are different, it is impossible to simply compare the numerical values of the evaluation results and decide superiority or inferiority.
*) Increase in Required Evaluation Data Amount
Due to advances in biometric authentication technology, the current error rate has become very small as compared with the past.
As a result, the amount of data necessary to show the authentication performance becomes larger as compared with before, and the evaluation cost has become enormous.
Although methods that can measure performance with less data are being studied, there is no established one yet. It is getting hard to derive the performance value itself.
*) Difficulty of Unifying Evaluation Specifications
Since there is no provision as the evaluation specification of the biometric authentication technology, the evaluation specifications differ depending on the evaluator.
That is why it is impossible to simply compare the evaluation results.
As an activity on concerning the performance evaluation of biometric authentication in ISO, at WG5 in SC37, newly established in 2002, international standardization of the accuracy evaluation method is discussed, and ISO / IEC 19795-1 was created, which is describing the framework of the performance evaluation test.
In 19795-1, performance evaluation scales such as FMR, FNMR, FAR, FRR, FTE, etc. are defined and are still used as indicators for performance.
On the other hand, in 19795-1, only a framework of performance evaluation is described, and performance evaluation specifications are not described.
Evaluation in conformity with 19795-1 does not result in the same evaluation specification.
Regarding the performance evaluation for the forgery, there is no unified standard in terms of what type of forgery should be prepared for evaluation, and comparison with values is also difficult here.
*) Summary
Many researches are being carried out to solve these problems, but it is difficult to solve immediately.
Under such circumstances, I think that it will continue to be asked for a while about how the product development side will explain the technology and convince users.
Author of this article
| Yutaka Deguchi mofiria Corporation General Manager, Technology Promotion |
About the Author
Yutaka Deguchi is a General Manager of mofiria Corporation, Japanese company providing finger vein authentication technology.
Yutaka joined in 2013 from Toshiba corporation where he was responsible for the technology and product development in voice recognition and synthesis, and now is reponsible for the technological development including the management of intelectual property and collaboration with some research institutes.
