ProxyFL - Decentralized federated learning through proxy model sharing

Authors: Shivam Kalra*, Junfeng Wen*, Jesse C. Cresswell*, Maksims Volkovs, Hamid R. Tizhoosh†

• * Denotes equal contribution
• † University of Waterloo / Vector Institute

Abstract

Institutions in highly regulated domains such as finance and healthcare often have restrictive rules around data sharing. Federated learning is a distributed learning framework that enables multi-institutional collaborations on decentralized data with improved protection for each collaborator’s data privacy. In this paper, we propose a communication-efficient scheme for decentralized federated learning called ProxyFL, or proxy-based federated learning. Each participant in ProxyFL maintains two models, a private model, and a publicly shared proxy model designed to protect the participant’s privacy. Proxy models allow efficient information exchange among participants without the need of a centralized server. The proposed method eliminates a significant limitation of canonical federated learning by allowing model heterogeneity; each participant can have a private model with any architecture. Furthermore, our protocol for communication by proxy leads to stronger privacy guarantees using differential privacy analysis. Experiments on popular image datasets, and a cancer diagnostic problem using high-quality gigapixel histology whole slide images, show that ProxyFL can outperform existing alternatives with much less communication overhead and stronger privacy.

Graphic Overall view of ProxyFL

See image source at Nature Communications

Prerequisite

• Python 3.9

conda create -n ProxyFL python=3.9
conda activate ProxyFL

• PyTorch 1.9.0

conda install pytorch=1.9.0 torchvision=0.10.0 numpy=1.21.2 -c pytorch

• mpi4py 3.1.2

conda install -c conda-forge mpi4py=3.1.2

• opacus 0.14.0

pip install 'opacus==0.14.0'

• matplotlib 3.4.3

conda install -c conda-forge matplotlib=3.4.3

Run experiment

Download data via

bash download_data.sh

Then run the script

bash run_exp.sh

Citation

If you find this code useful in your research, please cite the following paper:

@article{kalra2021proxyfl,
    author={Kalra, Shivam and Wen, Junfeng and Cresswell, Jesse C. and Volkovs, Maksims and Tizhoosh, H. R.},
    title={Decentralized federated learning through proxy model sharing},
    journal={Nature Communications},
    year={2023},
    month={May},
    day={22},
    volume={14},
    number={1},
    pages={2899},
    issn={2041-1723},
    doi={10.1038/s41467-023-38569-4}
}

Useful links

• ProxyFL Paper
• layer6 Website
• Patent
• University of Waterloo

Disclaimers

This technology is patneted (see patent). The code is provided for research purposes only and without any warranty. Any commercial use is prohibited.

NOTE: This repository is a clone of ProxyFL repository from layer6, with additions made by Kimia Lab at Mayo Clinic