Architecture & Developer Guide
This guide is for developers who need to understand FLARE internals, build custom workflows, or extend the platform. For higher-level usage, see the User Guide.
System Architecture
Core Concepts
Workflows & Controllers
Advanced Topics
Large Models & LLM
Techniques for federated training and fine-tuning of large models, including LLMs.
Deployment & Optimization:
Notes on Large Models – Deployment considerations for large model training
Message Quantization – Reducing message size via quantization
File Streaming – Streaming large files between participants
Tensor Downloader – Efficient model parameter transfer
Memory Management – Controlling memory usage during training
Decomposer for Large Objects – Serializing large objects efficiently
LLM Fine-Tuning:
Federated SFT (Supervised Fine-Tuning) – Federated SFT with HuggingFace
Federated PEFT (Parameter-Efficient Fine-Tuning) – LoRA and other PEFT methods
NeMo SFT Integration – Federated SFT with NeMo
NeMo PEFT Integration – Federated PEFT with NeMo
NeMo Prompt Learning – Federated prompt tuning with NeMo
Hierarchical Architecture
3rd-Party Integration
Low-Level APIs
These are foundational APIs that higher-level abstractions (Client API, FLARE API) are built on top of. Most users do not need these directly, but they are available for advanced customization.