PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
Abstract
We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members". This scheme, which we call PANORAMIA, quantifies the privacy leakage for large-scale ML models without control of the training process or model re-training and only requires access to a subset of the training data. To demonstrate its applicability, we evaluate our auditing scheme across multiple ML domains, ranging from image and tabular data classification to large-scale language models.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2024
- DOI:
- 10.48550/arXiv.2402.09477
- arXiv:
- arXiv:2402.09477
- Bibcode:
- 2024arXiv240209477K
- Keywords:
-
- Computer Science - Cryptography and Security;
- Computer Science - Machine Learning
- E-Print:
- 19 pages