The vision of automated machine learning (AutoML) entails enabling non-expert users to use machine learning technology and expert users to support them in their daily business and relieve them from tedious tasks. However, AutoML systems tend to function as a black box with few to no means for interaction. I envision AutoML tools that allow for interaction with users, making the AutoML process more transparent to the user, providing information about the learning problem and the observations made, to synergize human experience and intuition with the efficient algorithms of AutoML.
Large language models and other foundation models have become central to modern AI, yet their training, fine-tuning, and deployment remain prohibitively expensive in terms of computational cost and energy consumption. My research addresses this by transferring AutoML methodology to the foundation model lifecycle. This includes automated model compression through principled pruning and quantization, parameter-efficient fine-tuning with methods such as LoRA where hyperparameter optimization and meta-learning can substantially reduce manual tuning effort, and hardware-aware deployment that leverages meta-learning to match model configurations to heterogeneous accelerator landscapes. By treating the post-training pipeline of foundation models as a structured optimization problem, I aim to make high-quality language technologies accessible beyond large-scale industrial labs while reducing their environmental footprint.
Natural language processing and automated machine learning are converging in two complementary directions that define a core part of my research agenda. On one hand, modern NLP systems present complex configuration spaces that are ideally suited for AutoML methods such as Bayesian optimization and pipeline search. On the other hand, large language models themselves are becoming powerful interfaces and tools within AutoML systems, enabling users to express domain knowledge, optimization priors, and task specifications in natural language. Going forward, I investigate how agentic AI systems driven by language models can orchestrate data science workflows and how AutoML can systematically optimize compound NLP systems such as RAG pipelines.
Multi-label classification denotes a supervised learning setting where not only a single class label but a subset of class labels is predicted. In my research, I heavily investigated automated machine learning methods for configuring multi-label classifiers. In this course, I have worked on assessing the quality of predictions and how the AutoML perspective may benefit the field of multi-label classification as a whole. For instance, I could demonstrate that selecting base learners individually for every label in binary relevance learning can yield significant performance improvements for label-wise averaged macro measures. This work was awarded the frontier prize at the intelligent data analysis conference for the most visionary contribution.
In algorithm selection, we aim to predict which algorithm should be used for what input which can result in immense speedups when considering computationally hard problems. For this, observations of algorithm runtimes need to be collected which are per definition very expensive to obtain and some algorithm executions may even take virtually forever. In such cases, algorithm runs are terminated early resulting in right-censored data which needs to be treated in a reasonable way.
Another fascinating topic to me is artificial evolution and how it can be interwoven with machine learning algorithms. In my research, I have worked on methods to synergize active learning and coevolution, devise (co-)evolutionary algorithms to optimize machine learning models, and evolutionary algorithms as optimizers for automated machine learning in more general.
Reliability is a more and more requested property for machine learning applications and one way of improving reliability is to ensure that machine learning models can express their uncertainty in a truthful way. In this sense, predictions made by machine learning models should neither be over- nor underconfident but well-calibrated. In my research, I investigate how a good calibration can be ensured through automated machine learning.
Furthermore, labels of data might underlie some uncertainty, and labeling processes allow for uncertainties to be expressed.