TU Darmstadt / ULB / TUprints

Template-Driven Knowledge Distillation for Compact and Accurate Periocular Biometrics Deep-Learning Models

Boutros, Fadi ; Damer, Naser ; Raja, Kiran ; Kirchbuchner, Florian ; Kuijper, Arjan (2022):
Template-Driven Knowledge Distillation for Compact and Accurate Periocular Biometrics Deep-Learning Models. (Publisher's Version)
In: Sensors, 22 (5), MDPI, e-ISSN 1424-8220,
DOI: 10.26083/tuprints-00021119,
[Article]

[img] Text
sensors-22-01921.pdf
Available under: CC BY 4.0 International - Creative Commons, Attribution.

Download (3MB)
Item Type: Article
Origin: Secondary publication DeepGreen
Status: Publisher's Version
Title: Template-Driven Knowledge Distillation for Compact and Accurate Periocular Biometrics Deep-Learning Models
Language: English
Abstract:

This work addresses the challenge of building an accurate and generalizable periocular recognition model with a small number of learnable parameters. Deeper (larger) models are typically more capable of learning complex information. For this reason, knowledge distillation (kd) was previously proposed to carry this knowledge from a large model (teacher) into a small model (student). Conventional KD optimizes the student output to be similar to the teacher output (commonly classification output). In biometrics, comparison (verification) and storage operations are conducted on biometric templates, extracted from pre-classification layers. In this work, we propose a novel template-driven KD approach that optimizes the distillation process so that the student model learns to produce templates similar to those produced by the teacher model. We demonstrate our approach on intra- and cross-device periocular verification. Our results demonstrate the superiority of our proposed approach over a network trained without KD and networks trained with conventional (vanilla) KD. For example, the targeted small model achieved an equal error rate (EER) value of 22.2% on cross-device verification without KD. The same model achieved an EER of 21.9% with the conventional KD, and only 14.7% EER when using our proposed template-driven KD.

Journal or Publication Title: Sensors
Volume of the journal: 22
Issue Number: 5
Publisher: MDPI
Collation: 14 Seiten
Uncontrolled Keywords: biometrics, knowledge distillation, periocular verification
Classification DDC: 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik
Divisions: 20 Department of Computer Science > Fraunhofer IGD
20 Department of Computer Science > Mathematical and Applied Visual Computing
Date Deposited: 11 Apr 2022 11:36
Last Modified: 11 Apr 2022 11:37
DOI: 10.26083/tuprints-00021119
Corresponding Links:
URN: urn:nbn:de:tuda-tuprints-211196
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/21119
PPN:
Export:
Actions (login required)
View Item View Item