A membership inference attack (MIA) is a privacy attack that infers whether a record was included in the set of data used to produce a given output, such as a machine learning model or some released statistics. This information can be valuable in itself: for example, if the dataset was a medical study of patients with a certain disease and one can confirm they are part of it, then they have the disease.^[1][2]

There are two broad lines of research on MIAs. One focuses on published statistics about the dataset, asking whether a target individual is among the set of people over which the statistics were computed. For example, testing one's genotype against allele frequencies reported across single-nucleotide polymorphism sites to see if they contributed their DNA to the study. The other focuses on machine learning models, asking if a target example is among the records used to train the model, using either its prediction scores or its parameters. Attack performance is usually measured as true positive rate at a given false positive rate, or the area under the corresponding ROC curve, or the membership advantage, the true positive rate minus the false positive rate.^[3][4]

Many MIAs originated from the problem of determining individuals from aggregate genomic data. An early paper by Nils Homer et al. showed that one's genotype can be used, along with published allele frequency statistics across several SNPs, to determine whether they contributed their DNA to a study population with a similar genotype.^[2] This result had immediate practical consequences for storing genomic data, as the NIH and the Wellcome Trust subsequently decided to transition many datasets from public repositories to controlled access, such as dbGaP.^[5] Later papers generalized the result: for example, Dwork et al. showed that any number of statistics published about a database could be used to infer membership, and connected differential privacy and MIAs.^[6]

Similar techniques apply to data beyond SNPs and other genomics statistics. Knock Knock, Who's There?, a 2018 paper, formulated the problem as a binary classification task: train a classifier to distinguish between individuals that are members from those that are not and then apply it to a person that may be represented in an aggregate trace of locations published, for example, as a heat map. The paper demonstrated that such an attack can successfully determine individuals in trace data, even when no single trace is published.^[7] This principle can be generalized to any other type of aggregates, including simple counts over a sensitive population.^[4]

The attack by Shokri et al. was the first successful membership inference attack on an ML model.^[1] It was based on the observation that, for many models, the confidence score on a test record is higher when the record was part of the training set. To develop an attack, the authors used shadow models, ML models trained on similar data as the target one. Since the adversary knows which records were used to train them, they can be used to train another classifier to predict whether a record belongs to the target model's training set. Shadow models allow the adversary to approximate the target model's internal state without direct access, making membership inference possible without querying the model itself. In 2019, ML-Leaks showed that, for some models, a single shadow model is enough to launch an attack, and thus the danger was not limited to the specific architecture.^[8]

Many subsequent attacks used similar ideas, but with modifications to reduce complexity or assumptions. The most important change was relaxing the requirements for the shadow model. In particular, there are two types of membership inference attacks: score-based and label-based. Score-based attacks such as those described in Yeom et al.^[9] or Carlini et al.'s LiRA^[3] use the model's confidence scores, while label-based attack such as Choquette-Choo et al.'s Label Only Membership Inference^[10] use the predicted class directly. Label-based attacks are generally more powerful, especially when the adversary can influence the model input; for example, in natural language processing, a label-based attack can score the target record and several variations that differ by a few words to identify if it is in the training set based on whether these variations change the prediction. Another approach, model distillation, uses a simpler student model to replicate the behavior of a complex teacher model, allowing extraction of additional information about the training data.^[4]

The effectiveness of such attacks varies depending on the dataset and the model. Overfitting is the most common source of model leakage, and standard techniques generally reduce it.^[9] For neural networks, techniques such as dropout and weight decay and early stopping mitigate overfitting. However, membership inference attacks are generally more effective when the model is large, the data is complex, and both the training and test sets are large, which is often the case for modern ML models.^[4]

Membership inference attacks are less effective against large language models, for both technical and practical reasons. An evaluation from 2024 suggested that on many tasks, these attacks perform only slightly better than random guessing, mostly due to the distribution of training data. Since the model typically encounters any particular piece of data only once, the overlap between the training and test sets is minimal.^[11] A related finding is that for many attacks, a simple baseline that had nothing to do with the model performed similarly to sophisticated membership inference attacks.^[12] This suggests that, if the train and test sets were sampled differently, for example, if the latter was taken from a different time period than the former, the attack might erroneously claim that the record comes from the training set simply because it is old. A similar issue affected the original genomic study: since the study samples were from homogeneous populations, testing a member of a different ethnic group might falsely identify them as part of the sample.^[2]

Differential privacy is the most obvious defense, as it limits the information any one record can reveal and provides theoretical guarantees. For neural networks, it is commonly implemented via DP-SGD.^[13] In practice, defending against MIAs usually means improving generalization, reducing overfitting; thus, the standard techniques such as dropout, weight decay, early stopping are all valid, if indirect, ways of improving privacy.^[9] Alternatively, the model can be modified to limit the amount of information it leaks: for example, the confidence scores can be replaced with approximate values, or the top k classes can be used to limit the resolution of the outputs. More research has explored model-specific techniques: knowledge distillation or ensemble methods can improve generalization and reduce information about the training data without explicit changes. These techniques typically improve privacy without theoretical guarantees and often require adaptation to the model; if the attack can adapt to the defense, it remains effective.^[8][4]

Membership inference is part of a larger group of privacy attacks, with several subtypes. Attribute inference attacks attempt to determine attributes of a record not revealed by the model, for example, an individual's race given their age and sex. Similarly, model inversion attempts to retrieve an input sample given the model's output. Training-data reconstruction attempts to recover any training sample directly, which may be possible for certain models, such as large language models. Finally, model extraction attacks attempt to obtain the model weights directly, which is a separate type of attack. Membership inference is a core component of some of these attacks, such as training-data reconstruction, and can often be used to evaluate how much information a model leaks about its training data. The attacks also often share components: for example, some model inversion techniques can be used to launch membership inference attacks, and vice versa.^[4]

In particular, it has implications for data protection law, as it limits the ability to anonymize data by claiming the model contains only aggregates. In the GDPR, data is personal whenever it can be used to identify someone, directly or indirectly, including by inferring membership in a given database. If the model or statistics can be used to do that, it is personal data; therefore, its processing requires the same protections as any other personal data, for example, the right to be forgotten. Similar logic applies to other data privacy laws and regulations. Membership inference is also used in the reverse direction: to evaluate whether a model still contains personal information after attempting to delete it, for example, via machine unlearning. Moreover, the original application of MIAs in genomics strongly influenced regulation: many datasets of this type are now only available behind authorization systems.^[5]

• Adversarial machine learning
• Differential privacy
• Inference attack
• Data re-identification
• De-anonymization