Differential Privacy for Mental Health Datasets with PyTorch: From Theory to DP-SGD

Updated on December 06, 2025 15 minutes read

Mental health data is both extremely valuable and extremely sensitive.
Crisis events, diagnoses, therapy notes, and substance use histories are powerful features for machine learning, but they can also expose people to serious harms if mishandled.

Traditional de-identification, which involves removing names or IDs, is no longer sufficient to guarantee privacy.
Modern linkage and membership‑inference attacks can often recover individuals from seemingly anonymised datasets, especially in high‑dimensional clinical data.

Differential privacy (DP) gives us a principled way to limit how much a model reveals about any single person in the training data.
In deep learning, the main workhorse algorithm is Differentially Private Stochastic Gradient Descent (DP‑SGD), a noisy variant of standard SGD.

This article is for ML engineers, data scientists, and technically minded clinicians or researchers who work with mental health data.

Background and Prerequisites

What you should already know

You should be comfortable with basic Python and PyTorch.
If you can read and write simple nn.Module classes and training loops, you are in good shape.

Some familiarity with probability and linear algebra is helpful.
You should know what gradients, loss functions, and mini‑batches are in standard supervised learning.

On the domain side, a basic mental health context is useful.
Knowing what PHQ‑9, GAD‑7, crisis services, and therapy sessions are will help the example feel realistic.

Why mental health data is especially sensitive

Mental health datasets often include diagnostic histories, crisis admissions, self‑harm events, and medication changes.
These may be combined with social factors like employment, housing instability, or substance use.

Under many regulations, health data is treated as a special category requiring strong safeguards.
Ethically, mental health information is particularly sensitive due to stigma and the personal nature of the content.

Differential privacy does not replace legal and ethical governance, but it strengthens technical protections.
It provides guarantees even against attackers with significant background knowledge.

Quick reminder: standard SGD with mini-batches

Standard mini‑batch SGD trains a model with parameters $ theta $ as follows.
At each step, you sample a batch $B$, compute a gradient, and update the parameters.

Formally, on a batch $B$ you compute g = (1 / |B|) · Σ_{(xᵢ, yᵢ) ∈ B} ∇_θ ℓ(θ; xᵢ, yᵢ)

, And then update θ ← θ − η g, where η is the learning rate.

In DP‑SGD, we will keep the same high‑level structure but modify the gradient computation.
We will clip individual gradients and add noise before applying the update.

Core Theory: Differential Privacy, Epsilon, Delta, and DP-SGD

Intuition: neighbouring datasets and indistinguishability

Differential privacy is a property of a randomised algorithm that acts on a dataset.
Think of an algorithm $ (M) $ that takes a database of mental health records and outputs a trained model.

Two datasets are called neighbouring if they differ by exactly one person.
In a mental health context, that might mean adding or removing one patient’s entire record from the training data.

The core DP idea is that an attacker should not be able to tell, with high confidence, whether a specific individual is included.
The distribution of outputs of $ (M) $ should look almost the same on both neighbouring datasets.

Formal definition of ε, δ, and differential privacy

Formally, a randomised mechanism M satisfies (ε, δ)-differential privacy if, for all neighbouring datasets D and D′ and for all sets of outputs S:

Pr[M(D) ∈ S] ≤ e^ε · Pr[M(D′) ∈ S] + δ.

Here ε ≥ 0 is the privacy budget or privacy loss parameter. The smaller ε is, the closer the two output distributions are, and the stronger the privacy guarantee.

The parameter δ is a small failure probability, often chosen much smaller than 1/N for a dataset of size N. It captures a tiny probability that the guarantee might not hold, for example, because of the tails of the noise distribution.

Sensitivity and the Gaussian mechanism

Many DP mechanisms work by adding noise scaled to the sensitivity of a function.
Sensitivity measures how much the output can change when one record changes.

For a function f(D) that maps datasets to vectors, the L2-sensitivity is

Δf = max over neighbouring datasets D and D′ of ‖f(D) − f(D′)‖₂,

where the maximum is taken over all neighbouring datasets (datasets that differ in exactly one individual).

In the Gaussian mechanism, we release

M(D) = f(D) + N(0, σ² I),

where σ is chosen based on Δf, ε, and δ.

Higher sensitivity or stronger privacy demands more noise.
The key trick in DP‑SGD is to bound the sensitivity of each gradient step by clipping.

Composition and the privacy budget

Real training loops apply many DP operations to the same dataset.
Every DP‑SGD step consumes some privacy budget, and these costs accumulate.

Simple composition says that if you run k mechanisms, each with (εᵢ, δᵢ)-DP, then the total mechanism is (Σᵢ εᵢ, Σᵢ δᵢ)-DP. This bound is often pessimistic when you have many small steps.

More advanced accounting methods, such as Rényi Differential Privacy and moments accountants, track privacy loss more tightly across iterations. Libraries like Opacus implement these accountants and can report a final ε given your training setup.

From SGD to DP-SGD: clipping and noise

Standard mini‑batch SGD uses the average gradient of the batch.
DP‑SGD instead works with per‑example gradients.

On each batch:

1. For each example $ i $, compute the per-example gradient gᵢ.
2. Clip each gradient to a maximum L₂ norm C: ṡ gᵢ = gᵢ / max(1, ‖gᵢ‖₂ / C).
3. Average the clipped gradients:

ḡ = (1 / |B|) · Σ_{i ∈ B} ṡ gᵢ. 4. Add Gaussian noise to the average:

ĝ = ḡ + N(0, σ² C² I). 5. Update the model parameters using ĝ instead of the true average gradient.

Clipping ensures that no single patient’s record can push the parameters too far in one step.
Noise ensures that even these bounded contributions are hidden inside random fluctuations.

In mental health models, that means the network learns overall patterns like “prior crisis increases risk” without memorising any individual’s combination of symptoms and events.

Hands-On DP-SGD Implementation with PyTorch

Overview of the example pipeline

We will now build a complete example in PyTorch.
The goal is to train a binary classifier predicting 30‑day crisis risk from synthetic tabular mental health data.

The pipeline includes:

Generating a synthetic clinical dataset, building data loaders, defining a small MLP model, training a non‑private baseline, and training a DP‑SGD version with gradient clipping and noise.

In a real project, you would use high‑quality clinical data under strict governance.
Here, we focus on the mechanics in a safe synthetic setting.

Environment setup

We assume you have Python 3 and PyTorch installed.
You will also use scikit‑learn for evaluation metrics.

Install dependencies with:

pip install torch torchvision torchaudio scikit-learn

Then import the core libraries and set up your device:

import torch
import torch.nn as nn
from torc utils.data import Dataset, DataLoader, random_split

From sklearn.metrics import accuracy_score, roc_auc_score, recall_score

torch.manual_seed(42)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

If you have a GPU, DP‑SGD will benefit from it, especially for larger models or datasets.
For this small example, the CPU is sufficient.

Synthetic mental health dataset

We construct a dataset with features such as age, depression scores, functioning, sessions, substance use, and prior crisis.
The label is a synthetic binary indicator for a crisis in the next 30 days.

class MentalHealthDataset(Dataset):
    def __init__(self, num_samples=8000):
        super().__init__()
        n = num_samples

        age = torch.randint(18, 80, (n, 1)).float()
        phq9 = torch.randint(0, 28, (n, 1)).float()      # depressive symptoms
        gaf = torch.randint(20, 91, (n, 1)).float()      # functioning score
        num_sessions = torch.poisson(torch.full((n, 1), 5.0))
        substance = torch.bernoulli(torch.full((n, 1), 0.3))
        prior_crisis = torch.bernoulli(torch.full((n, 1), 0.15))

        features = torch.cat(
            [age, phq9, gaf, num_sessions, substance, prior_crisis],
            dim=1
        )

        logits = (
            -6.0
            + 0.12 * phq9
            - 0.03 * gaf
            + 0.15 * num_sessions
            + 1.0 * substance
            + 1.8 * prior_crisis
        )

        probs = torch.sigmoid(logits)
        labels = torch.bernoulli(probs).view(-1)

        self.features = features
        self.labels = labels

    def __len__(self):
        return self.labels.shape[0]

    def __getitem__(self, idx):
        return self.features[idx], self.labels[idx]

We then create train, validation, and test splits, and wrap them in data loaders:

dataset = MentalHealthDataset(num_samples=8000)

train_size = int(0.7 * len(dataset))
val_size = int(0.15 * len(dataset))
test_size = len(dataset) - train_size - val_size

train_ds, val_ds, test_ds = random_split(
    dataset,
    [train_size, val_size, test_size]
)

train_loader = DataLoader(train_ds, batch_size=256, shuffle=True)
val_loader = DataLoader(val_ds, batch_size=1024, shuffle=False)
test_loader = DataLoader(test_ds, batch_size=1024, shuffle=False)

This structure mirrors typical clinical modelling workflows.
You hold out test data to evaluate generalisation and tune hyperparameters on the validation set.

Crisis risk MLP model and evaluation metrics

Our model is a small multi‑layer perceptron with one hidden layer.
It outputs a single logit representing crisis risk.

class CrisisRiskMLP(nn.Module):
    def __init__(self, input_dim):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, 16),
            nn.ReLU(),
            nn.Linear(16, 1)
        )

    def forward(self, x):
        return self.net(x).squeeze(1)

We also define an evaluation function that computes accuracy, ROC‑AUC, and positive‑class recall.
These are natural metrics for risk prediction tasks.

def evaluate_model(model, data_loader, device):
    model.eval()
    all_probs = []
    all_labels = []

    with torch.no_grad():
        for x_batch, y_batch in data_loader:
            x_batch = x_batch.to(device)
            y_batch = y_batch.to(device)

            logits = model(x_batch)
            probs = torch.sigmoid(logits)

            all_probs.append(probs.cpu())
            all_labels.append(y_batch.cpu())

    all_probs = torch.cat(all_probs).numpy()
    all_labels = torch.cat(all_labels).numpy()

    preds = (all_probs >= 0.5).astype("int32")

    acc = accuracy_score(all_labels, preds)
    auc = roc_auc_score(all_labels, all_probs)
    recall_pos = recall_score(all_labels, preds, pos_label=1)

    return {
        "accuracy": acc,
        "roc_auc": auc,
        "recall_positive": recall_pos
    }

In mental health settings, high recall for the positive class (crisis) is often important.
Missing a high‑risk patient can be more harmful than generating some false alarms.

Baseline non-private training

We first train a non‑private model for comparison.
This gives us a performance reference before adding DP.

def train_non_private(model, train_loader, val_loader, epochs=8, lr=5e-3):
    model.to(device)
    criterion = nn.BCEWithLogitsLoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=lr)

    for epoch in range(1, epochs + 1):
        model.train()
        running_loss = 0.0

        for x_batch, y_batch in train_loader:
            x_batch = x_batch.to(device)
            y_batch = y_batch.to(device)

            optimizer.zero_grad()
            logits = model(x_batch)
            loss = criterion(logits, y_batch)

            loss.backward()
            optimizer.step()

            running_loss += loss.item() * x_batch.size(0)

        train_loss = running_loss / len(train_loader.dataset)
        val_metrics = evaluate_model(model, val_loader, device)

        print(
            f"[Non-DP] Epoch {epoch:02d} | "
            f"Train loss: {train_loss:.4f} | "
            f"Val AUC: {val_metrics['roc_auc']:.3f} | "
            f"Val Recall+: {val_metrics['recall_positive']:.3f}"
        )

Use the function like this:

input_dim = dataset[0][0].shape[0]
base_model = CrisisRiskMLP(input_dim=input_dim)

train_non_private(base_model, train_loader, val_loader)
test_metrics = evaluate_model(base_model, test_loader, device)
print("Non-private test metrics:", test_metrics)

On this synthetic data, you should see a reasonably high ROC‑AUC.
This confirms that the features containanaigsignal orr crisis risk.

Implementing DP-SGD in PyTorch (naive version)

Now we implement a simple DP‑SGD training loop.
This version is not optimised, but makes the algorithm transparent.

The main idea is to compute per‑sample gradients, clip them, average them, and add Gaussian noise:

def train_dp_sgd(
    model,
    train_loader,
    val_loader,
    epochs=5,
    lr=5e-3,
    max_grad_norm=1.0,
    noise_multiplier=1.0,
):
    model.to(device)
    optimizer = torch.optim.SGD(model.parameters(), lr=lr)
    criterion = nn.BCEWithLogitsLoss(reduction="none")  # per-sample loss

    for epoch in range(1, epochs + 1):
        model.train()
        total_loss = 0.0

        for x_batch, y_batch in train_loader:
            x_batch = x_batch.to(device)
            y_batch = y_batch.to(device)
            batch_size = y_batch.size(0)

            logits = model(x_batch)
            losses = criterion(logits, y_batch)  # shape: (batch_size,)

            per_sample_grads = {
                name: [] for name, p in model.named_parameters()
                if p.requires_grad
            }

            for i in range(batch_size):
                optimizer.zero_grad()
                losses[i].backward(retain_graph=True)

                For name, p in model.named_parameters():
                    If not p.requires_grad or p.grad is None:
                        continue
                    per_sample_grads[name].append(p.grad.detach().clone())

            optimizer.zero_grad()

            For name, p in model.named_parameters():
                If not p.requires_grad:
                    continue

                grads = torch.stack(per_sample_grads[name], dim=0)  # (B, *)
                grads_flat = grads.view(batch_size, -1)
                grad_norms = grads_flat.norm(2, dim=1)

                clip_factors = (max_grad_norm / (grad_norms + 1e-6)).clamp(max=1.0)
                clip_factors = clip_factors.view(
                    batch_size, *([1] * (grads.dim() - 1))
                )

                grads_clipped = grads * clip_factors
                grad_mean = grads_clipped.mean(dim=0)

                noise_std = noise_multiplier * max_grad_norm / batch_size
                noise = torch.randn_like(grad_mean) * noise_std

                p.grad = grad_mean + noise

            optimizer.step()

            batch_loss = losses.mean().item()
            total_loss += batch_loss * batch_size

        train_loss = total_loss / len(train_loader.dataset)
        val_metrics = evaluate_model(model, val_loader, device)

        print(
            f"[DP-SGD] Epoch {epoch:02d} | "
            f"Train loss: {train_loss:.4f} | "
            f"Val AUC: {val_metrics['roc_auc']:.3f} | "
            f"Val Recall+: {val_metrics['recall_positive']:.3f}"
        )

We can now train a DP‑SGD model and compare it with the baseline:

dp_model = CrisisRiskMLP(input_dim=input_dim)

train_dp_sgd(
    dp_model,
    train_loader,
    val_loader,
    epochs=5,
    lr=5e-3,
    max_grad_norm=1.0,
    noise_multiplier=1.0,  # try 0.5, 1.0, 1.5
)

dp_test_metrics = evaluate_model(dp_model, test_loader, device)
print("DP-SGD test metrics:", dp_test_metrics)

As you increase the noise multiplier, you should see test performance degrade.
That visible gap between DP and non‑DP models is the privacy–utility trade‑off in action.

Using Opacus for practical DP training

For a research prototype, this naive loop is fine.
For realistic projects, you should use a dedicated DP library such as Opacus for PyTorch.

With Opac, you attach a PrivacyEngine to your optimiser and data loader.
It handles per‑sample gradients, clipping, noise, and privacy accounting:

from opacus import PrivacyEngine

model = CrisisRiskMLP(input_dim=input_dim).to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=5e-3)
criterion = nn.BCEWithLogitsLoss()

privacy_engine = PrivacyEngine()

model, optimizer, private_train_loader = privacy_engine.make_private(
    module=model,
    optimizer=optimizer,
    data_loader=train_loader,
    noise_multiplier=1.0,
    max_grad_norm=1.0,
)

for epoch in range(5):
    model.train()
    for x_batch, y_batch in private_train_loader:
        x_batch = x_batch.to(device)
        y_batch = y_batch.to(device)

        optimizer.zero_grad()
        logits = model(x_batch)
        loss = criterion(logits, y_batch)
        loss.backward()
        optimizer.step()

    epsilon = privacy_engine.get_epsilon(delta=1e-5)
    print(f"Epoch {epoch+1}: ε = {epsilon:.2f} for δ = 1e-5")

Opacus is far more efficient than the naive loop and provides explicit (\varepsilon) estimates.
This makes it much easier to communicate privacy guarantees to stakeholders.

Systems and Production Considerations

Data pipelines from EHR to DP training

In a real mental health system, your DP‑SGD training loop is only one part of the pipeline.
You need reliable and secure data flows from clinical systems into your training environment.

A typical pipeline might:

Extract structured data from the EHR and digital mental health apps.
Pseudonymise or de‑identify records where possible before ML use.
Build feature tables with reproducible code or a feature store.
Train models under DP‑SGD in a restricted environment.
Register and deploy models behind internal APIs.

The DP aspects primarily affect the training environment.
At inference time, you serve the model like any other, but you document that it was trained under DP.

Infrastructure and performance trade-offs

DP‑SGD is typically slower and more resource‑intensive than standard SGD.
Per‑sample gradients, clipping, and noise add overhead, especially for large models.

You should plan for more compute, or for smaller models, or both.
For many tabular mental health tasks, a small MLP or even logistic regression can be effective and easier to train privately.

Using GPUs can help mitigate DP overhead, particularly with libraries optimised for vectorised per‑sample gradients.
Batch size, learning rate, and noise multiplier all interact with privacy accounting and optimisation dynamics.

Observability, monitoring, and model lifecycle

DP‑SGD does not eliminate the need for monitoring.
You still need to track performance metrics and data drift, as in any ML system.

In a mental health application, you should pay special attention to:

Calibration of predicted risks, subgroup performance by age, sex, or other relevant attributes and changes in data distribution when services or documentation practices change.

You should also log and version all privacy‑related parameters.
That includes clip norms, noise multipliers, sampling rates, and resulting (\varepsilon,\delta).

Risk, Ethics, Safety, and Governance

Privacy and security beyond DP-SGD

Differential privacy limits what can be inferred from models and statistics.
It does not, by itself, secure your infrastructure or guarantee legal compliance.

You still need strong access controls, encryption at rest and in transit, and proper logging and auditing.
Breach response plans, data retention policies, and user consent management remain essential.

In mental health, building trust with patients and clinicians is crucial.
Being transparent about data uses and safeguards, including DP, helps support that trust.

Bias, fairness, and representativeness

Mental health datasets are often skewed and incomplete.
Some groups may be underdiagnosed, pathologised, or under‑recorded.

DP noise can interact with these patterns in non‑trivial ways.
It may further degrade performance for already underrepresented subpopulations if not monitored carefully.

You should evaluate models across important subgroups and consider fairness metrics alongside global performance.
A human‑in‑the‑loop design is essential when decisions affect care pathways.

Misuse and over-interpretation of risk scores

Even well‑calibrated risk models can be misused.
Scores might be interpreted as deterministic truths rather than probabilistic estimates.

In mental health settings, this can lead to over‑reliance on algorithms or to inappropriate restrictions.
DP does not prevent misinterpretation; it only reduces leakage of training data.

Clear documentation, conservative deployment, and ongoing stakeholder training are key.
Models should be treated as decision-support tools, not decision-makers.

Domain-Specific Case Study: DP-SGD for Crisis Readmission Risk

Problem setup

Consider a network of community clinics that wants to predict which patients are at high risk of crisis readmission within 30 days.
They hope to allocate limited outreach and community resources more effectively.

The available data includes demographics, symptom scores, functioning scales, therapy engagement, and past crises.
Ethics review boards and data protection officers insist on strong privacy protections.

Model and DP design choices

The team decides to start with a small MLP trained via DP‑SGD.
They use a DP library such as Opacus to handle clipping, noise, and privacy accounting.

They choose a batch size and number of epochs that are compatible with their compute budget.
They tune the noise multiplier to reach a target privacy budget, say (\varepsilon) between 3 and 8 for a small (\delta).

The non‑private model might achieve an AUC around 0.85 on held‑out data.
The DP‑trained model might reach an AUC around 0.78–0.80, with somewhat lower recall.

Interpreting results and using them in practice

Clinicians see risk bands rather than raw probabilities, such as low, medium, and high risk.
For high‑risk patients, the system suggests additional outreach or safety planning.

Each risk band is accompanied by key contributing features, for example, high PHQ‑9, prior crisis, or poor functioning.
This helps clinicians make sense of the scores and integrate them with their own judgment.

Because the model is trained with DP‑SGD, the organisation can more confidently discuss privacy protections with patients and regulators.
The DP guarantee becomes one pillar of a broader responsible‑AI and data‑governance strategy.

Skills Mapping and Learning Path

Programming and tooling skills

Working through this material builds practical Python and PyTorch skills.
You learn to implement custom datasets, data loaders, neural networks, and evaluation loops.

You also practice using scikit‑learn metrics for clinical risk tasks.
Building synthetic tabular datasets strengthens your understanding of feature engineering and label generation.

ML, AI, and privacy skills

From an ML perspective, you deepen your understanding of supervised learning on tabular health data.
You see how model capacity, batch size, and optimisation choices affect performance.

From a privacy perspective, you learn the formal ((\varepsilon,\delta)) definition and the intuition behind sensitivity and noise.
You also learn how DP‑SGD implements clipping and noise in gradient‑based training.

Understanding how to navigate the privacy–utility trade‑off is a valuable skill in health‑tech roles.
It connects theory to concrete hyperparameters and evaluation procedures.

Domain, ethics, and governance skills

On the mental health side, you become more familiar with typical clinical variables and risk outcomes.
You see how they can be translated into ML features and labels.

You also see how technical choices must align with legal and ethical frameworks.
DP‑SGD is one piece of a compliance and governance story that also includes consent, access control, and oversight.

Suggested next steps

Good next steps include:

Applying DP‑SGD to a public health dataset, switching from the naive implementation to a library like Opacus, experimenting with different noise multipliers and clip norms, exploring fairness metrics under varying privacy budgets.

You can also explore federated learning combined with DP for multi‑institution modelling.
This is especially relevant when mental health services cannot centralise data.

• Explore Programs

Conclusion

Differential privacy gives a principled way to limit information leakage from models trained on sensitive mental health data.
In deep learning, DP‑SGD brings this guarantee into gradient‑based optimisation.

By clipping per‑example gradients and adding noise, DP‑SGD trades some accuracy for formal privacy guarantees.
With tools like PyTorch and Opacus, these techniques are now accessible to practitioners, not just theoreticians.

In production mental health systems, DP‑SGD sits inside a larger ecosystem of secure data pipelines, MLOps, monitoring, and governance.
When combined with clinical expertise and ethical oversight, it can support safer, more trustworthy AI tools in mental health care.