Robert Jan Bruintjes

Much of the human experience is captured in the ever-increasing amounts of data that we humans generate. This data is used to build AI models that aid humans, e.g. by editing text, tracking traffic surrounding a self-driving car or detecting cancer in MRI scans. AI models typically get better when adding more training data. However, data cannot be sourced infinitely, and there are signs we are already running out of data to add to AI datasets. This is a concern, since AI models do not work well in settings where there is little data available (data-deficient settings), including important settings such as medical applications where gathering large datasets is costly and yields privacy concerns. We therefore desire data-efficient AI models: models that need less data, while still doing well.

This thesis investigates how design choices of neural network architectures (inductive biases) influence the data efficiency of vision models. In particular, we investigate the effects of modeling spatial biases apparent in training datasets. Such spatial biases are apparent through variability in positions and scales of imaged objects in the training data.

Vision models differ in their approach to modeling position information. Convolutional Neural Networks (CNNs) encode translation invariance through shared convolutional filters, enabling recognition of objects across positions with minimal data. However, this inductive bias removes positional information, which can hinder tasks requiring spatial reasoning. Vision Transformers (ViTs), in contrast, lack built-in spatial priors and rely on explicit positional embeddings to capture spatial structure, trading off data efficiency for global receptive fields.

Beyond position, scale variation presents a fundamental challenge: objects appearing at different sizes require models to generalize its learned recognition functions across scales. Conventional CNNs often learn separate representations for each scale, increasing data and compute demands. Scale-equivariant architectures address this by sharing representations across scales, and adaptive receptive field methods allow models to optimize kernel sizes for the observed distribution of object scales, improving robustness and efficiency.

To reconcile the challenges of modeling both types of spatial dataset biases, this thesis introduces flexible inductive biases: mechanisms that embed structural priors while allowing the model to modulate their influence based on dataset characteristics. Unlike fixed inductive biases, flexible inductive biases can be activated when beneficial and suppressed when irrelevant, reducing the risk of negative transfer and improving data-efficiency.

Overall, the thesis demonstrates that spatial biases should be modelled as adaptive, dataset-dependent mechanisms rather than fixed architectural constraints. By systematically modeling position and scale biases in datasets and enabling their flexible use, this work provides a principled framework for designing data-efficient vision models that generalize robustly across diverse visual domains.