Publications

Alternative way of synthesis for high refractive index tellurides based glasses has been experimented, in addition to low temperature Spark Plasma Sintering. The composition tested, Ge25Se10Te65, has been chose in the Ge-Se-Te system and characterized. Its index refractive index of 3.12 and overall optical, thermal and mechanical properties makes it the perfect candidate for IR […]

Alternative way of synthesis for high refractive index tellurides based glasses has been experimented, in addition to low temperature Spark Plasma Sintering. The composition tested, Ge25Se10Te65, has been chose in the Ge-Se-Te system and characterized. Its index refractive index of 3.12 and overall optical, thermal and mechanical properties makes it the perfect candidate for IR application. However, due to its relative instability regarding crystallization, formation of GeTe crystals occurs during mechanical alloying using raw elements. Transparency has not been achieved in the sintered samples using this powder, as the crystallization rate is accelerated by the pressure during the process. In parallel, glass samples synthesized by melt-quenching have been used to determine optimal sintering parameters for this composition. The main issue met during those tests has been the carbon contamination, reducing overall transparency of the samples through scatterings. As such, it has been shown that the critical parameter to consider to limiting this pollution is the powder granulometry, needing to be above 100μm for optimal performance. This shows the potential for this method to produce high refractive index IR optics, using even unstable glasses.

This study is focused on evaluating the real-time performance of thermal object detection for smart and safe vehicular systems by deploying the trained networks on GPU & single-board EDGE-GPU computing platforms for onboard automotive sensor suite testing. A novel large-scale thermal dataset comprising of > 35,000 distinct frames is acquired, processed, and open-sourced in challenging […]

This study is focused on evaluating the real-time performance of thermal object detection for smart and safe vehicular systems by deploying the trained networks on GPU & single-board EDGE-GPU computing platforms for onboard automotive sensor suite testing. A novel large-scale thermal dataset comprising of > 35,000 distinct frames is acquired, processed, and open-sourced in challenging weather and environmental scenarios. The dataset is a recorded from lost-cost yet effective uncooled LWIR thermal camera, mounted stand-alone and on an electric vehicle to minimize mechanical vibrations. State-of-the-art YOLO-V5 networks variants are trained using four different public datasets as well newly acquired local dataset for optimal generalization of DNN by employing SGD optimizer. The effectiveness of trained networks is validated on extensive test data using various quantitative metrics which include precision, recall curve, mean average precision, and frames per second. The smaller network variant of YOLO is further optimized using TensorRT inference accelerator to explicitly boost the frames per second rate. Optimized network engine increases the frames per second rate by 3.5 times when testing on low power edge devices thus achieving 11 fps on Nvidia Jetson Nano and 60 fps on Nvidia Xavier NX development boards.

AI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framework for thermal vision with seven key object classes in order to provide superior thermal sensing […]

AI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framework for thermal vision with seven key object classes in order to provide superior thermal sensing and scene understanding input for advanced driver-assistance systems (ADAS). The networks are trained on public datasets and is validated on test data with three different test approaches which include test-time augmentation, test-time with no augmentation, and test-time with model ensembling. Additionally, a new model ensemble-based inference engine is proposed, and its efficacy is tested on locally gathered novel test data comprising of 20K thermal frames captured with an uncooled LWIR prototype thermal camera in challenging weather and environmental scenarios. The performance analysis of trained models is investigated by computing precision, recall, and mean average precision scores (mAP). Furthermore, the smaller network variant of thermal-YOLO architecture is optimized using TensorRT inference accelerator, which is then deployed on GPU and resource-constrained edge hardware Nvidia Jetson Nano. This is implemented to explicitly reduce the inference time on GPU as well as on Nvidia Jetson Nano to evaluate the feasibility for added real-time onboard installations.

Infrared cameras could serve automotive applications by delivering breakthrough perception systems for both in-cabin passengers monitoring and car surrounding. However, low-cost and high-throughput manufacturing methods are essential to sustain the growth in thermal imaging markets for automotive applications, and for other close-to-consumer applications which have a fast growth potential. Fast low cost infrared lenses suitable […]

Infrared cameras could serve automotive applications by delivering breakthrough perception systems for both in-cabin passengers monitoring and car surrounding. However, low-cost and high-throughput manufacturing methods are essential to sustain the growth in thermal imaging markets for automotive applications, and for other close-to-consumer applications which have a fast growth potential. Fast low cost infrared lenses suitable for microbolometers are currently already sold by companies like Umicore, Lightpath, FLIR… They are either made of a single inverse meniscus Chalcogenide glass or of two Silicon optics. In this paper, we explore hybrid systems with a large field of view around 40° combining Chalcogenide and Silicon in order to take advantage of both materials. Both are compatible with wafer-level process. Silicon optics can be manufactured by photolithography process and are expected to be more cost-effective than Chalcogenide ones. However they are constrained in shape and sag height. On the other hand, Chalcogenide optics can be collectively molded and could have more free shapes. They are thus more suitable to reach high-demanding performance. So hybrid designs could be seen as a compromise between cost and performance. In this paper, we show that fast lenses with diameter constraints to few millimeters to make affordable wafer-level process lead to small size detectors. As a consequence, the pixel pitch reduction of microbolometers is a key point to maintain a good resolution. Finally, strategies to improve the production yield of hybrid lenses are explored.

Driver’s drowsiness is one of the major causes of traffic accidents worldwide. An early detection of episodes of sleepiness becomes of fundamental importance for safety purposes. Several studies demonstrated that PERCLOS, that is the percentage of eyelid closure over the pupil across time, is one of the most accurate parameters for drowsiness state assessment. However, […]

Driver’s drowsiness is one of the major causes of traffic accidents worldwide. An early detection of episodes of sleepiness becomes of fundamental importance for safety purposes. Several studies demonstrated that PERCLOS, that is the percentage of eyelid closure over the pupil across time, is one of the most accurate parameters for drowsiness state assessment. However, since PERCLOS is typically computed from the visible video of the subjects, its evaluation is strictly dependent on the lighting conditions and it is not accessible if the driver wears sunglasses. The objective of this study is to overcome these limitations, evaluating drowsy states using a low-cost and high-resolution thermal infrared technology. Ten sleep-deprived subjects were recruited for the experiment, consisting in one-hour driving task on a driving static simulator. During the experiment, facial skin temperature was recorded by means of the thermal camera Device Alab SmartIr640, together with facial visible videos of the subjects. Relevant thermal features were estimated from facial regions of interest (i.e., nose tip, glabella) whereas PERCLOS was performed on visible videos. Features were extracted over a time window of 30 seconds. A data-driven multivariate machine learning approach based on a three-level Support Vector Classification of the drowsy state (AWAKE class: PERCLOS<0.15, FATIGUE class: 0.15<PERCLOS<0.23, and SLEEPY class: PERCLOS>0.15) was employed. The average classification accuracy was 0.65±0.09 (mean ± standard deviation). Although preliminary, these results indicate the possibility to assess driver's drowsiness based on facial thermal features, overcoming the limitation related to lighting condition and eyes detection, typical of standard methods.

In this work, spark plasma sintering is used to mold non conventional chalcogenide glasses of high refractive index at low temperature (<400°C). This equipment, usually used for sintering refractory materials, is presented as efficient for both densification and high precision molding of IR transparent bulks and lenses of telluride glasses. Thermo-mechanical and optical characteristics of […]

In this work, spark plasma sintering is used to mold non conventional chalcogenide glasses of high refractive index at low temperature (<400°C). This equipment, usually used for sintering refractory materials, is presented as efficient for both densification and high precision molding of IR transparent bulks and lenses of telluride glasses. Thermo-mechanical and optical characteristics of the selected Ge25Se10Te65 glass composition were investigated showing a refractive index of 3,12@10 µm and with however a limited resistance to crystallization. Mechanical milling of raw Ge, Se, Te elements leads to a major amorphous phase with the formation of a small proportion of GeTe crystals. Remaining GeTe crystals induce a fast crystallization rate during the sintering process leading to the opacity of the material. SPS flash moldings were then performed using melt quenched glass powders to produce complex lenses. It has been found that the critical parameter to reach optimal IR transparency is mainly the powder granulometry, which should be superior to 100 µm to prevent from MIE scatterings. The possibility of producing high refractive index infrared lenses has been demonstrated even with unstable glasses against crystallization.

Thermal imaging has played a dynamic role in the diversified field of consumer technology applications. To build artificially intelligent thermal imaging systems, large scale thermal datasets are required for successful convergence of complex deep learning models. In this study, we have highlighted various techniques for generating large scale synthetic facial thermal data using both public […]

Thermal imaging has played a dynamic role in the diversified field of consumer technology applications. To build artificially intelligent thermal imaging systems, large scale thermal datasets are required for successful convergence of complex deep learning models. In this study, we have highlighted various techniques for generating large scale synthetic facial thermal data using both public and locally gathered datasets. It includes data augmentation, synthetic data generation using StyleGAN network, and 2D to 3D image reconstruction using deep learning architectures. Training and validation accuracy of Wide ResNet CNN for binary gender recognition task is improved by 4.6% and 4.4% using original and newly generated synthetic data with an overall test accuracy of 83.33%.

Gender classification has found many useful applications in the broader domain of computer vision systems including in-cabin driver monitoring systems, human–computer interaction, video surveillance systems, crowd monitoring, data collection systems for the retail sector, and psychological analysis. In previous studies, researchers have established a gender classification system using visible spectrum images of the human face. […]

Gender classification has found many useful applications in the broader domain of computer vision systems including in-cabin driver monitoring systems, human–computer interaction, video surveillance systems, crowd monitoring, data collection systems for the retail sector, and psychological analysis. In previous studies, researchers have established a gender classification system using visible spectrum images of the human face. However, there are many factors affecting the performance of these systems including illumination conditions, shadow, occlusions, and time of day. Our study is focused on evaluating the use of thermal imaging to overcome these challenges by providing a reliable means of gender classification. As thermal images lack some of the facial definition of other imaging modalities, a range of state-of-the-art deep neural networks are trained to perform the classification task. For our study, the Tufts University thermal facial image dataset was used for training. This features thermal facial images from more than 100 subjects gathered in multiple poses and multiple modalities and provided a good gender balance to support the classification task. These facial samples of both male and female subjects are used to fine-tune a number of selected state-of-the-art convolution neural networks (CNN) using transfer learning. The robustness of these networks is evaluated through cross validation on the Carl thermal dataset along with an additional set of test samples acquired in a controlled lab environment using prototype uncooled thermal cameras. Finally, a new CNN architecture, optimized for the gender classification task, GENNet, is designed and evaluated with the pretrained networks.

Traffic accidents determine a large number of injuries, sometimes fatal, every year. Among other factors affecting a driver’s performance, an important role is played by stress which can decrease decision-making capabilities and situational awareness. In this perspective, it would be beneficial to develop a non-invasive driver stress monitoring system able to recognize the driver’s altered […]

Traffic accidents determine a large number of injuries, sometimes fatal, every year. Among other factors affecting a driver’s performance, an important role is played by stress which can decrease decision-making capabilities and situational awareness. In this perspective, it would be beneficial to develop a non-invasive driver stress monitoring system able to recognize the driver’s altered state. In this study, a contactless procedure for drivers’ stress state assessment by means of thermal infrared imaging was investigated. Thermal imaging was acquired during an experiment on a driving simulator, and thermal features of stress were investigated with comparison to a gold-standard metric (i.e., the stress index, SI) extracted from contact electrocardiography (ECG). A data-driven multivariate machine learning approach based on a non-linear support vector regression (SVR) was employed to estimate the SI through thermal features extracted from facial regions of interest (i.e., nose tip, nostrils, glabella). The predicted SI showed a good correlation with the real SI (r = 0.61, p = ~0). A two-level classification of the stress state (STRESS, SI ≥ 150, versus NO STRESS, SI < 150) was then performed based on the predicted SI. The ROC analysis showed a good classification performance with an AUC of 0.80, a sensitivity of 77%, and a specificity of 78%.

Human thermography is considered to be an integral medical diagnostic tool for detecting heat patterns and measuring quantitative temperature data of the human body. It can be used in conjunction with other medical diagnostic procedures for getting comprehensive medication results. In the proposed study we have highlighted the significance of Infrared Thermography (IRT) and the […]

Human thermography is considered to be an integral medical diagnostic tool for detecting heat patterns and measuring quantitative temperature data of the human body. It can be used in conjunction with other medical diagnostic procedures for getting comprehensive medication results. In the proposed study we have highlighted the significance of Infrared Thermography (IRT) and the role of machine learning in thermal medical image analysis for human health monitoring and various disease diagnosis in preliminary stages. The first part of the proposed study provides comprehensive information about the application of IRT in the diagnosis of various diseases such as skin and breast cancer detection in preliminary stages, dry eye syndromes, and ocular issues, liver disease, diabetes diagnosis and last but not least the novel COVID-19 virus. Whereas in the second phase we have proposed an autonomous breast tumor classification system using thermal breast images by employing state of the art Convolution Neural Network (CNN). The system achieves the overall accuracy of 80% and recall rate of 83.33%.

Richard Blythman virtually presented the team’s work on creating synthetic thermal image datasets for deep learning at an IEEE conference, Quality of Multimedia Experience (QoMEX 2020). Synthetic data holds promise for reducing the number of real images required to train deep learning algorithms, especially for thermal imaging applications where there are much fewer publicly-available datasets. […]

Richard Blythman virtually presented the team's work on creating synthetic thermal image datasets for deep learning at an IEEE conference, Quality of Multimedia Experience (QoMEX 2020). Synthetic data holds promise for reducing the number of real images required to train deep learning algorithms, especially for thermal imaging applications where there are much fewer publicly-available datasets. The team used photogrammetry techniques to reconstruct 3D thermographic models of humans, which were then rendered in various positions and poses to mimic in-cabin monitoring scenarios. Finally, deep learning algorithms for face detection and head pose estimation were trained and evaluated on a combination of synthetic and real data. In future, we hope to build a multi-thermal camera data acquisition setup to improve the data quality.

In this work, we extend existing methodologies to show how 2D thermal facial data can be mapped to provide 3D facial models. For the proposed research work we have used tufts datasets for generating 3D varying face poses by using a single frontal face pose. The system works by refining the existing image quality by […]

In this work, we extend existing methodologies to show how 2D thermal facial data can be mapped to provide 3D facial models. For the proposed research work we have used tufts datasets for generating 3D varying face poses by using a single frontal face pose. The system works by refining the existing image quality by performing fusion based image preprocessing operations. The refined outputs have better contrast adjustments, decreased noise level and higher exposedness of the dark regions. In the next phase, the refined version of images is used to create 3D facial geometry structures by using end to end Convolution Neural Networks (CNN). The same technique is also used on our local thermal face data acquired using uncooled prototype thermal camera (developed under Heliaus EU project) in an indoor lab environment which is then used for generating synthetic 3D face data along with varying yaw face angles and lastly facial depth map is generated.