First Look: New Ghost in the Shell Adaptation Set for 2026 Launch

Foundation models, which are often large neural networks trained on extensive text and image data, have dramatically transformed the way artificial intelligence systems manage tasks involving language and vision. Unlike models designed for specific tasks, these models generalize across a wide array using the knowledge gained during pretraining. Once developed, they can produce coherent responses, categorize images, or solve problems without needing additional task-specific training. Their scalability and adaptability across diverse domains are pivotal to AI advancement.

Despite their extensive capabilities, a key challenge is adapting these models to novel, unforeseen tasks. Generally, to achieve substantial performance, these models require carefully crafted prompts or labeled examples that instruct them on how to perform. This process can be cumbersome, as creating prompts involves trial and error, and gathering labeled examples can be both costly and time-consuming. Additionally, in practical applications, such supporting data is not always readily accessible, which limits the effectiveness of foundation models in zero-shot scenarios.

Multiple strategies have been employed to balance generality and task-specific performance. In-context learning allows models to replicate a task by presenting example input-output pairs during inference, while supervised fine-tuning modifies model weights with labeled data. Prompt engineering, another approach, involves designing prompts to direct the model toward desired outputs. Although these methods have successfully enhanced performance, each relies on external input—either human intervention or labeled data—making them less feasible in entirely unsupervised contexts.

Researchers at the Swiss Federal Institute of Technology Lausanne (EPFL) have introduced a joint inference framework to support unsupervised adaptation. This framework enables foundation models to coordinate predictions across multiple inputs without needing ground truth data or manual prompts. They proposed two specific techniques within this framework: unsupervised fine-tuning and unsupervised in-context learning. These techniques allow models, even those with fixed weights like GPT-4, to enhance accuracy independently of external guidance.

Unsupervised fine-tuning improves model predictions by focusing solely on its feedback. It creates an optimization goal where predictions for multiple inputs are generated collectively, and their joint probability is optimized. This process employs LoRA (Low-Rank Adaptation) for efficient weight updates and incorporates a regularization step to prevent trivial solutions, like providing the same answer for all inputs. For conditions where weight access isn’t possible, such as with GPT-4, the researchers developed unsupervised in-context learning. This approach recreates the benefits of labeled ICL by treating previously generated outputs as pseudo-labels, refining predictions over several iterations without human annotations. With each iteration, the model is conditioned on prior examples to produce more accurate responses, simulating a supervised learning loop using self-generated data.

The improvements from these unsupervised methods were significant. On the GSM8K dataset, meant for math reasoning, unsupervised ICL applied to the Qwen2.5-Math model resulted in a 39.2% absolute increase over the standard zero-shot baseline. Similarly, for the Llama-3.1-8B model evaluated over 13 natural language processing tasks, unsupervised fine-tuning produced a 23% average boost in accuracy, equaling the performance of fully supervised fine-tuning in six of the tasks. In vision-language tasks, unsupervised ICL also delivered robust gains—showing a 23% increase on the Food101