Example Applications

NVIDIA FLARE has several tutorials and examples to help you get started with federated learning and to explore certain features in the examples directory.

1. Hello World Examples

Can be run from hello_world notebook.

1.1. Workflows

• Hello NumPy - Example using the FedAvg workflow with a NumPy trainer

• Hello Cross-Site Validation - Example using the Cross Site Eval workflow, also demonstrates running cross site validation using the previous training results.

• Hello Cyclic Weight Transfer (GitHub) - Example using the CyclicController workflow to implement Cyclic Weight Transfer with TensorFlow as the deep learning training framework

• Swarm Learning - Example using Swarm Learning and Client-Controlled Cross-site Evaluation workflows.

1.2. Deep Learning

• Hello PyTorch - Example image classifier using FedAvg and PyTorch as the deep learning training framework

• Hello TensorFlow - Example image classifier using FedAvg and TensorFlow as the deep learning training frameworks

2. Tutorial Notebooks

• Intro to the FL Simulator - Shows how to use the NVIDIA FLARE FL Simulator to run a local simulation of an NVFLARE deployment to test and debug an application without provisioning a real FL project.

• Hello FLARE API - Goes through the different commands of the FLARE API to show the syntax and usage of each.

• NVFLARE in POC Mode - Shows how to use POC mode to test the features of a full FLARE deployment on a single machine.

• NVFlare CLI Tutorial - Walks through the current nvflare command groups for local setup, recipes, jobs, systems, studies, provisioning, and deployment.

• Job Recipe - Introduces Job Recipes to simplify federated learning job creation and execution with a high-level API.

• FLARE Logging - Covers how to configure logging in FLARE for different use cases and modes.

3. Federated Learning Algorithms

• Federated Learning with CIFAR-10 (GitHub) - Includes examples of using FedAvg, FedProx, FedOpt, SCAFFOLD, homomorphic encryption, and streaming of TensorBoard metrics to the server during training

• FL Algorithms
  • Federated Averaging
  • FedProx
  • FedOpt
  • SCAFFOLD
  • Federated XGBoost
  • Federated Analytics

4. Privacy Preserving Algorithms

Privacy preserving algorithms in NVIDIA FLARE are implemented as filters that can be applied as data is sent or received between peers.

• Federated Learning with CIFAR-10 (GitHub) - Includes examples of using FedAvg, FedProx, FedOpt, SCAFFOLD, homomorphic encryption, and streaming of TensorBoard metrics to the server during training

5. Traditional ML examples

• Federated Linear Model with Scikit-learn (GitHub) - For an example of using NVIDIA FLARE with scikit-learn, a widely used open-source machine learning library that supports supervised and unsupervised learning.

• Federated K-Means Clustering with Scikit-learn (GitHub) - NVIDIA FLARE with scikit-learn and k-Means.

• Federated SVM with Scikit-learn (GitHub) - NVIDIA FLARE with scikit-learn and SVM.

• Federated XGBoost (GitHub) - Includes examples of histogram-based and tree-based algorithms. Tree-based algorithms also includes bagging and cyclic approaches. Also includes an example of vertical federated XGBoost.

6. Medical Image Analysis

• MONAI Integration (GitHub) - For an example of using NVIDIA FLARE to train a 3D medical image analysis model using federated averaging (FedAvg) and MONAI Bundle MONAI

7. Federated Statistics

• Federated Statistic Overview - Discuss the overall federated statistics features.

• Federated Statistics for medical imaging (Github) - Example of gathering local image histogram to compute the global dataset histograms.

• Federated Statistics for tabular data with DataFrame (Github) - Example of gathering local statistics summary from Pandas DataFrame to compute the global dataset statistics.

• Federated Statistics with Monai Statistics integration for Spleen CT Image (Github) - Example demonstrated Monai statistics integration and few other features in federated statistics

8. Federated Site Policies

• Federated Policies (Github) - Discuss the federated site policies for authorization, resource and data privacy management

• Custom Authentication (Github) - Show the custom authentication policy and secure mode.

• Job-Level Authorization (Github) - Show the job-level authorization policy and secure mode.

• KeyCloak Site Authentication Integration (Github) - Demonstrate KeyCloak integration for supporting site-specific authentication.

9. Experiment Tracking

• FL Experiment Tracking with TensorBoard Streaming - (documentation) - Example building on Hello PyTorch with TensorBoard streaming from clients to server

• FL Experiment Tracking with MLflow - (documentation)- Example integrating Hello PyTorch with MLflow with streaming from clients to server

• FL Experiment Tracking with Weights and Biases - Example integrating Hello PyTorch with Weights and Biases streaming capability from clients to server.

10. Natural Language Processing (NLP)

• NLP-NER (Github) - Illustrates both BERT and GPT-2 models from Hugging Face (BERT-base-uncased, GPT-2) on a Named Entity Recognition (NER) task using the NCBI disease dataset.

11. Federated Large Language Model (LLM)

• Parameter Efficient Fine Turning - Example utilizing NeMo’s PEFT methods to adapt a LLM to a downstream task.

• Prompt-Tuning Example - Example for using FLARE with NeMo for prompt learning.

• Supervised Fine Tuning (SFT) - Example to fine-tune all parameters of a LLM on supervised data.

• LLM Tuning via HuggingFace SFT Trainer - Example for using FLARE with a HuggingFace trainer for LLM tuning tasks.

12. Graph Neural Network (GNN)

• Protein Classification - Example using GNNs for Protein Classification using PPI dataset using GraphSAGE.

• Financial Transaction Classification - Example using GNNs for Financial Transaction Classification with Elliptic++ dataset using GraphSAGE.

13. Financial Applications

• Financial Application with Federated XGBoost Methods Example using XGBoost in various ways to train a federated model to perform fraud detection with a finance dataset.

• Financial Transaction Classification - Example using GNNs for Financial Transaction Classification with Elliptic++ dataset using GraphSAGE.

Setting up a virtual environment for examples and notebooks

It is recommended to set up a virtual environment before installing the dependencies for the examples. Install dependencies for a virtual environment with:

python3 -m pip install --user --upgrade pip
python3 -m pip install --user virtualenv

Once venv is installed, you can use it to create a virtual environment with:

$ python3 -m venv nvflare_example

This will create the nvflare_example directory in current working directory if it doesn’t exist, and also create directories inside it containing a copy of the Python interpreter, the standard library, and various supporting files.

Activate the virtualenv by running the following command:

$ source nvflare_example/bin/activate

Installing required packages

In each example folder, install required packages for training:

pip install --upgrade pip
pip install -r requirements.txt

(optional) some examples contain scripts for plotting the TensorBoard event files, if needed, please also install the additional requirements in the example folder:

pip install -r plot-requirements.txt

JupyterLab with your virtual environment for Notebooks

To run examples including notebooks, we recommend using JupyterLab.

After activating your virtual environment, install JupyterLab.

pip install jupyterlab

If you need to register the virtual environment you created so it is usable in JupyterLab, you can register the kernel with:

python -m ipykernel install --user --name="nvflare_example"

Start a Jupyter Lab:

jupyter lab .

When you open a notebook, select the kernel you registered, “nvflare_example”, using the dropdown menu at the top right.

Custom Code in Example Apps

There are several ways to make custom code available to clients when using NVIDIA FLARE. Most hello-* examples use a custom folder within the FL application. Note that using a custom folder in the app needs to be allowed when using secure provisioning. By default, this option is disabled in the secure mode. POC mode, however, will work with custom code by default.

In contrast, the CIFAR-10 example assumes that the learner code is already installed on the client’s system and available in the PYTHONPATH. Hence, the app folders do not include the custom code there. The PYTHONPATH is set in the run_poc.sh or run_secure.sh scripts of the example. Running these scripts as described in the README will make the learner code available to the clients.