nvflare.app_common.np.recipes.cross_site_eval module

class NumpyCrossSiteEvalRecipe(name: str = 'numpy_cross_site_eval', min_clients: int = 2, eval_script: str | None = None, eval_args: str = '', launch_external_process: bool = False, command: str = 'python3 -u', initial_ckpt: str | None = None, model_dir: str | None = None, model_name: dict | None = None, submit_model_timeout: int = 600, validation_timeout: int = 6000, client_memory_gc_rounds: int = 0, cuda_empty_cache: bool = False)[source]

Bases: Recipe

Recipe for standalone cross-site evaluation with pre-trained NumPy models.

Creates a cross-site evaluation workflow that loads pre-trained models and evaluates them across all client sites without performing any training.

Parameters:

  • name – Name of the federated job. Defaults to “numpy_cross_site_eval”.

  • min_clients – Minimum number of clients required to start the job. Defaults to 2.

  • eval_script – Path to the evaluation script that will be executed on each client. If not provided, uses a built-in dummy validator (for testing only).

  • eval_args – Command line arguments to pass to the evaluation script. Defaults to “”.

  • launch_external_process – Whether to launch the script in external process. Defaults to False.

  • command – If launch_external_process=True, command to run script (prepended to script). Defaults to “python3 -u”.

  • initial_ckpt – Absolute path to a pre-trained model file (.npy) on the server. If provided, this takes precedence over model_dir/model_name. The file may not exist locally (server-side path).

  • model_dir – Directory containing pre-trained models (relative to run directory). Defaults to “models”. Only used when initial_ckpt is not provided.

  • model_name – Dictionary mapping model identifiers to filenames, e.g., {“model_1”: “model_1.npy”, “model_2”: “model_2.npy”}. If None, defaults to {“server”: “server.npy”}. Only used when initial_ckpt is not provided.

  • submit_model_timeout – Timeout (seconds) for submitting models to clients. Defaults to 600.

  • validation_timeout – Timeout (seconds) for validation tasks on clients. Defaults to 6000.

Example

Using eval_script with initial_ckpt:

```python recipe = NumpyCrossSiteEvalRecipe(

eval_script=”evaluate.py”, eval_args=”–data_root /path/to/data”, initial_ckpt=”/path/to/pretrained_model.npy”, min_clients=2,

)

Using model_dir/model_name (models from training run):

```python recipe = NumpyCrossSiteEvalRecipe(

eval_script=”evaluate.py”, model_dir=”models”, model_name={“server”: “server.npy”}, min_clients=2,

)

This is base class of a recipe. Recipes are implemented by jobs. A concrete recipe must provide the job for recipe implementation.

param job:

the job that implements the recipe.