Master's Thesis: Synthetic Faces for… at Fraunhofer-Ges… · Darmstadt

Background and Motivation Face-based age estimation is relevant for numerous applications, such as identity verification, youth protection or medical applications. However, training powerful neural networks requires large, well-balanced datasets with reliable age labels. Especially for rare age ranges or specific application scenarios, there are often not enough suitable real face images available.

Generative models such as StyleGANs or modern diffusion models make it possible to generate large numbers of synthetic face images in a targeted way and to enrich existing training datasets. However, our own initial experiments show that such data augmentation does not necessarily lead to a significant improvement in age estimation. While supplementing real training data with synthetic face images led to slight improvements in the mean absolute error (MAE) in some cases, the observed effects were small overall.

One likely cause is a pronounced domain shift between real and synthetic face images. In previous work, for example, a ResNet classifier was able to distinguish real from synthetic images with virtually perfect accuracy. This suggests that the generated images contain systematic features that allow neural networks to recognize their artificial origin. It is also known from the literature that generative models can leave characteristic statistical, spectral or structural artifacts, or “fingerprints”, in generated images. For age estimation, however, it is still largely unclear which of these artifacts are actually relevant and whether they impair the use of synthetic training data.

A second possible cause lies in the quality of the age labels of synthetic face images. With prompt-based diffusion models, for example, the perceived age of a generated person does not necessarily match the age specified in the prompt. Even with specialized GAN-based methods for face generation or age manipulation, characteristics and biases of the training dataset and of the model used can influence the generated age distribution. Synthetic face images may therefore contain not only a visual domain shift but also systematic or random label noise.

Goal of the Master’s Thesis The goal of this master’s thesis is to systematically investigate why synthetic face images have so far been of only limited use for training age estimation models. In particular, artifacts of different generative models are to be identified, their influence on age estimation models analyzed, and methods for reducing the domain shift developed. In addition, the thesis will examine how reliable the specified age labels of synthetically generated persons actually are.

To this end, the thesis will:

• Generate and compare face images using different generative methods, for example StyleGAN-based models and modern diffusion or text-to-image models.
• Systematically examine and quantify differences between real and synthetic face images. Possible approaches include classifiers for distinguishing real and synthetic images, feature space analyses and frequency domain analyses.
• Identify and investigate generator-specific and cross-generator artifacts. In particular, it will be analyzed which features neural networks use to distinguish real from synthetic images.
• Investigate the relationship between these artifacts and the performance of age estimation models. For this purpose, suitable ablation studies and controlled experiments are to be designed.
• Develop or evaluate methods for reducing relevant artifacts in synthetic face images. Possible approaches include classical image processing, frequency domain filtering, learning-based methods, or domain adaptation and image-to-image translation.
• Examine whether reducing the differences between real and synthetic images actually leads to better use of synthetic data when training age estimation models.
• Investigate the quality of synthetic age labels. In particular, it will be analyzed how well the perceived age of generated persons matches the desired or specified age, and whether certain generators or age ranges show systematic deviations.
• Examine the influence of incorrect age labels and visual artifacts as separately as possible, in order to determine which factor limits the use of synthetic data more strongly.
• Derive, develop and evaluate new strategies for generating, selecting, post-processing or weighting synthetic training data based on the findings.

Results The expected outcome is a systematic characterization of the differences between real and synthetic face images from various generative models. The thesis should show which artifacts or statistical properties make it possible to distinguish synthetic from real images, and to what extent these features are relevant for age estimation models.

Furthermore, methods for reducing the domain shift are to be developed or investigated, and their effects on training with mixed real and synthetic datasets evaluated. In particular, the thesis should clarify whether lower distinguishability between real and synthetic images also goes along with improved generalization of age estimation models.

Another focus is on the reliability of synthetic age labels. The thesis is expected to provide sound findings on how accurately different generative methods reproduce the desired ages, in which age ranges systematic errors occur, how these can be detected, and what influence this label noise has on training.

Based on this, the thesis should derive guidelines on when and under which conditions synthetic face images can add value for age estimation, and which requirements must be met. The developed methods will be evaluated on publicly available benchmark datasets such as UTKFace, APPA-REAL, AgeDB or IMDB-WIKI, as well as on suitable synthetically generated datasets such as HDA-SynChildFaces.

Be part of change

• Research and structure literature on current topics in machine learning, computer vision and generative models
• Generate and analyze synthetic face images using modern generative methods
• Research and implement machine learning, computer vision and image forensics approaches
• Develop suitable methods for analyzing and reducing artifacts in synthetic images
• Plan and conduct controlled experiments and ablation studies
• Train and evaluate neural networks for age estimation
• Critically evaluate and interpret your results
• Present your results
• Write up your results in a scientific thesis in the form of a master’s thesis

What you contribute

• Very good academic performance with an average grade of 2.2 or better (German grading scale)
• Good knowledge of machine learning, computer vision and training neural networks
• Ideally, knowledge of generative models or face analysis
• Good Python skills, ideally with experience in PyTorch
• Knowledge of GANs, diffusion models or image forensics is an advantage, but not a requirement
• Motivation to independently familiarize yourself with new and current research topics
• Interest in experimental research and the systematic analysis of neural networks
• Interest in scientific research

Resources

• MiVOLO

What we offer

• Flexible working hours
• Insights into the intersection of academic research and industrial application

We value and promote the diversity of our employees' skills and therefore welcome all applications – regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability. Our tasks are diverse and adaptable – for applicants with disabilities, we work together to find solutions that best promote their abilities.

With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.

Ready for a change? Then apply now and make a difference! Once we have received your online application, you will receive an automatic confirmation of receipt. We will then get back to you as soon as possible and let you know what happens next.

Fraunhofer Institute for Secure Information Technology SIT

<a href="http://www.sit.fraunhofer.de" target="_blank" rel="noopener">www.sit.fraunhofer.de

Requisition Number: 86077 Application Deadline:

[Not translated in selected language]