TU Darmstadt / ULB / TUprints

A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients

Lyra, Simon ; Mayer, Leon ; Ou, Liyang ; Chen, David ; Timms, Paddy ; Tay, Andrew ; Chan, Peter Y. ; Ganse, Bergita ; Leonhardt, Steffen ; Hoog Antink, Christoph (2024)
A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients.
In: Sensors, 2021, 21 (4)
doi: 10.26083/tuprints-00017784
Article, Secondary publication, Publisher's Version

Copyright Information: CC BY 4.0 International - Creative Commons, Attribution.

Download (3MB) | Preview
Item Type: Article
Type of entry: Secondary publication
Title: A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients
Language: English
Date: 15 January 2024
Place of Publication: Darmstadt
Year of primary publication: 2021
Place of primary publication: Basel
Publisher: MDPI
Journal or Publication Title: Sensors
Volume of the journal: 21
Issue Number: 4
Collation: 18 Seiten
DOI: 10.26083/tuprints-00017784
Corresponding Links:
Origin: Secondary publication DeepGreen

Infrared thermography for camera-based skin temperature measurement is increasingly used in medical practice, e.g., to detect fevers and infections, such as recently in the COVID-19 pandemic. This contactless method is a promising technology to continuously monitor the vital signs of patients in clinical environments. In this study, we investigated both skin temperature trend measurement and the extraction of respiration-related chest movements to determine the respiratory rate using low-cost hardware in combination with advanced algorithms. In addition, the frequency of medical examinations or visits to the patients was extracted. We implemented a deep learning-based algorithm for real-time vital sign extraction from thermography images. A clinical trial was conducted to record data from patients on an intensive care unit. The YOLOv4-Tiny object detector was applied to extract image regions containing vital signs (head and chest). The infrared frames were manually labeled for evaluation. Validation was performed on a hold-out test dataset of 6 patients and revealed good detector performance (0.75 intersection over union, 0.94 mean average precision). An optical flow algorithm was used to extract the respiratory rate from the chest region. The results show a mean absolute error of 2.69 bpm. We observed a computational performance of 47 fps on an NVIDIA Jetson Xavier NX module for YOLOv4-Tiny, which proves real-time capability on an embedded GPU system. In conclusion, the proposed method can perform real-time vital sign extraction on a low-cost system-on-module and may thus be a useful method for future contactless vital sign measurements.

Uncontrolled Keywords: camera-based vital sign measurement, infrared thermography, IRT, object detection, deep learning, optical flow, ICU monitoring
Status: Publisher's Version
URN: urn:nbn:de:tuda-tuprints-177840
Additional Information:

This article belongs to the Special Issue Sensors and Methods for the Measurement of Cardiovascular and Respiratory Systems

Classification DDC: 600 Technology, medicine, applied sciences > 610 Medicine and health
600 Technology, medicine, applied sciences > 621.3 Electrical engineering, electronics
Divisions: 18 Department of Electrical Engineering and Information Technology > Artificial Intelligent Systems in Medicine (KISMED)
Date Deposited: 15 Jan 2024 13:41
Last Modified: 14 Mar 2024 10:28
SWORD Depositor: Deep Green
URI: https://tuprints.ulb.tu-darmstadt.de/id/eprint/17784
PPN: 516255258
Actions (login required)
View Item View Item