Research
My dissertation combines visual and graph-based representations for character and handwriting recognition in low-resource settings. A scenario where an expert can label a handful of examples per class, and everything else has to be earned from unlabelled data.
The work is funded by the DFG–SNSF project “Combining Image and Graph-based Neural Networks for Handwriting Recognition” (DFG 528122871 / SNSF 217594), a collaboration between the Pattern Recognition in Embedded Systems group at TU Dortmund University and the AIBEX group at the University of Fribourg.
Current threads
Prototype-anchored learning from a handful of exemplars
PAUL trains one autoencoder per class, anchored on one to five expert-selected exemplars, and expands pseudo-labels over an unlabelled pool. The selection criterion is the ratio between the best and the second-best reconstruction error — a sample is claimed only when one class model explains it decisively better than the rest. On EMNIST Balanced this reaches 66.9% accuracy against 53.3% for the strongest baseline, and it holds up on KMNIST and two Omniglot-derived benchmarks against SCAN, DEC, IDEC, JULE and centroid/GMM baselines across DINO, CLIP and convolutional features.
Vision-language models as document readers
Recognition pipelines have segmented pages into lines for decades. Benchmarking nine commercial and open-weight LVLMs across four scripts and three difficulty levels shows that assumption is worth revisiting: page-level input beat line segmentation on every handwritten script we tested, and cost less to run. For Cyrillic, character error rate dropped from 22.6% to 3.9%. The exception is heavily degraded material such as the Bentham collection, where segmentation still earns its place.
Writer retrieval at archival scale
FormWR is a 387,000-page benchmark for writer retrieval where handwriting appears only sparsely on printed forms; closer to what archives actually hold than page-length manuscripts. I contributed X-VLAD, a learned set-aggregation module that compresses a variable set of local descriptors into a 256-dimensional writer embedding. Transferred to HisFragIR20, it reaches 78.8% mAP and 97.9% Top-1, improving on prior published work by +21.6% mAP and +12.7% Top-1.
Information extraction from historical archives
Recognition is rarely the end goal — archives want structured, searchable records. This thread covers layout analysis, entity extraction and record linkage on historical archival material, in interdisciplinary work with historians.
Side interests
Privacy-preserving representations of wearable sensor data (my M.Sc. work, published at ICPR), knowledge graphs for reliable AI, and the ethics of LLM-based companionship in elder care. Joint work with researchers outside computer science.
Methods and interests
- Self-supervised learning
- Masked autoencoders
- Contrastive objectives
- Graph neural networks
- Hybrid image–graph architectures
- Autoencoders, VAEs and β-VAEs
- GANs
- Vision-language models
- Few-shot classification
- Deep clustering
- Open-set recognition
- Handwriting and character recognition
- Writer retrieval
- Knowledge representation
Collaborations
Ongoing work with Andreas Fischer, Anna Scius-Bertrand and Antônio F. S. Neto (University of Fribourg) on the DFG–SNSF handwriting recognition project, and interdisciplinary collaboration on the ethics of language-model companionship in elder care.
I co-supervise seven Bachelor's and Master's theses in these areas — see teaching and supervision.