As Sweden's national center for applied AI, we're on a mission to accelerate the use of AI to benefit our society, our competitiveness, and everyone living in Sweden. We drive impactful initiatives in areas such as healthcare, energy, and public services while pushing the boundaries of AI research in fields such as natural language processing, machine learning and AI security. Join us in harnessing the untapped value of AI to drive innovation and create sustainable value for Sweden.

We are now looking for a master thesis student to join our team.

Introduction

A privacy audit is only as trustworthy as the choices behind it: whether a model is judged to leak depends on which attack is run and how it is configured, decisions that currently rest with a human expert. LeakPro (AI Sweden et al., 2024), an open-source privacy-auditing framework, provides a catalog of attacks with typed configuration schemas, a scheduler and typed results, but no guidance on which to run, so auditing remains limited to those who know the attack literature.

Gorilla (Patil et al., 2024) and ToolLLM (Qin et al., 2024) show that LLM agents can select and invoke tools from large API repositories, and AutoMIA (Tran et al., 2026) shows that agents- designed membership inference signals can outperform handcrafted baselines. These evaluations, however, score calls against a single reference answer. None provides expert ground truth for domain judgment under a compute budget, where several choices may be defensible and a wrong one silently under-reports leakage. This thesis proposes LeakPro-Agent, an LLM-based planning layer around the LeakPro core, to investigate whether LeakPro can be driven by a controlled auditing agent without modifying its existing attack implementations.

Project Background and Problem Statement

AI Sweden is leading a project to develop an open-source privacy auditing tool called LeakPro, designed to assess information leakage risks in machine learning models. This initiative, undertaken in collaboration with RISE, Sahlgrenska, Region Halland, Recorded Future, AstraZeneca, Region Västmanland, Syndata and Scaleout, aims to evaluate the risk of sensitive information disclosure when models trained on confidential data are made publicly available.

Research Question: This thesis focuses on whether an LLM agent can match expert attack selection and configuration under a fixed compute budget. In privacy auditing the choice is what matters: an audit that runs cleanly but selects the wrong attack under-reports leakage and is worse than no audit at all, because it gives false assurance that the model is safe to release. To our knowledge, no reference set records what an expert would have done on a given model under a given compute budget, so agent audit decisions cannot currently be scored. The problem is therefore twofold: to build that ground truth, and to measure the agent’s selection, configuration and compute cost against it.

Outline

The objectives are as follows:

• Tool exposure and groundwork: Build a Model Context Protocol (MCP) server (Anthropic, 2024) that derives tool definitions as automatically as possible from LeakPro’s attack schemas, following repository-to-tool standardization approaches (Di et al., 2026), and exposes the attack catalog without modifying individual attack implementations.

• An audit-scenario benchmark: Assemble 10–15 audit scenarios spanning model types, data modalities and threat models, each with a target model, a handler, a fixed compute budget and an expert-authored audit.yaml reference solution. Fix the scoring metrics, measure expert run-to-run variance, and release the suite as a benchmark for auditing agents.

• Auditor-agent implementation and evaluation: Develop an agent that receives a target model, handler, budget and plain-language goal, and score it on the suite for attack selection, configuration and outcome. Log GPU-hours, LLM latency and token cost per run, and plot risk estimate against compute for the agent and for LeakPro’s existing Optuna search (Akiba et al., 2019), showing whether agent-guided configuration reaches a given estimate more cheaply.

If time permits and the student has interest, an AutoMIA-style (Tran et al., 2026) propose-implement-evaluate loop can be explored for scenarios where the existing catalog underperforms. LeakPro’s dual-use nature constrains the work throughout. The agent audits only models the operator controls, and candidate attack code must be sandboxed and reviewed by a human before it is registered.

Who we’re looking for

• We are seeking a curious, independent, and self-driven MSc student who wants to work at the absolute frontier of AI research.

• Ongoing Master’s studies in Computer Science, Data Science, Engineering Physics, Complex Adaptive Systems, Machine Learning, or a related field.

• Comfortable with Python and deep learning, as well as with the reality that an experiment might yield unexpected results.

At AI Sweden, we are committed to building diverse and inclusive teams. Some positions may be subject to export control regulations, which means that specific requirements may apply.

Why should you do your thesis with AI Sweden?

Doing your thesis at AI Sweden means working alongside leading AI scientists and change leaders. AI Sweden is Sweden’s National Center for AI, we drive research questions that have both a long shelf-life and are widely applicable to Swedish industry and the public sector. We aim for publications at the most competitive venues and celebrate a culture of research excellence.

As an organization, we’re uniquely positioned at the sweet spot of governmental influence and startup agility. Small enough to stay adaptive and have fun but backed by and in close contact with both the government, academia and private and public sector.

Practical details
Location: Gothenburg (Hybrid). Remote work is fully supported for students based in Lund or Stockholm.

Application Deadline: 2026-10-25 (rolling selection – position may be filled earlier).
Start Date: January 2027

Contact

If you have any questions or thoughts, don’t hesitate to contact:

Fazeleh Hoseini, Research Scientist

Muhaddisa Ali, Research Scientist

AI Sweden does not accept unsolicited support and kindly ask not to be contacted by any advertisement agents, recruitment agencies or manning companies.