nvflare.app_common.np.recipes.cross_site_eval module

class NumpyCrossSiteEvalRecipe(name: str = 'numpy_cross_site_eval', min_clients: int = 2, eval_script: str | None = None, eval_args: str = '', launch_external_process: bool = False, command: str = 'python3 -u', initial_ckpt: str | None = None, model_dir: str | None = None, model_name: dict | None = None, submit_model_timeout: int = 600, validation_timeout: int = 6000, client_memory_gc_rounds: int = 0, cuda_empty_cache: bool = False)

Bases: Recipe

Recipe for standalone cross-site evaluation with pre-trained NumPy models.

Recipe parameters become part of the generated job definition and must never contain actual secret values. Read secrets from site environment variables or mounted files; references are supported only where documented in nvflare.recipe.secrets.

Creates a cross-site evaluation workflow that loads pre-trained models and evaluates them across all client sites without performing any training.

Parameters:

• name – Name of the federated job. Defaults to “numpy_cross_site_eval”.

• min_clients – Minimum number of clients required to start the job. Defaults to 2.

• eval_script – Path to the evaluation script that will be executed on each client. If not provided, uses a built-in dummy validator (for testing only).

• eval_args – Command line arguments to pass to the evaluation script. Defaults to “”.

• launch_external_process – Whether to launch the script in external process. Defaults to False.

• command – If launch_external_process=True, command to run script (prepended to script). Defaults to “python3 -u”.

• initial_ckpt – Absolute path to a pre-trained model file (.npy) on the server. If provided, this takes precedence over model_dir/model_name. The file may not exist locally (server-side path).

• model_dir – Directory containing pre-trained models (relative to run directory). Defaults to “models”. Only used when initial_ckpt is not provided.

• model_name – Dictionary mapping model identifiers to filenames, e.g., {“model_1”: “model_1.npy”, “model_2”: “model_2.npy”}. If None, defaults to {“server”: “server.npy”}. Only used when initial_ckpt is not provided.

• submit_model_timeout – Timeout (seconds) for submitting models to clients. Defaults to 600.

• validation_timeout – Timeout (seconds) for validation tasks on clients. Defaults to 6000.

Example

Using eval_script with initial_ckpt:

```python recipe = NumpyCrossSiteEvalRecipe(

eval_script=”evaluate.py”, eval_args=”–data_root /path/to/data”, initial_ckpt=”/path/to/pretrained_model.npy”, min_clients=2,

)

Using model_dir/model_name (models from training run):

```python recipe = NumpyCrossSiteEvalRecipe(

eval_script=”evaluate.py”, model_dir=”models”, model_name={“server”: “server.npy”}, min_clients=2,

)

This is base class of a recipe. Recipes are implemented by jobs. A concrete recipe must provide the job for recipe implementation.

Security contract – no secrets in recipe parameters:

Recipe parameters (train_args, task_args, eval_args, per_site_config, config overrides, dicts passed to add_client_config/add_server_config, exec params, etc.) can be written in clear text into generated job configuration. These parameters and their nested values must never contain actual passwords, API keys, tokens, private keys, or other credentials. Instead, read secrets from site environment variables or mounted secret files inside your code, or pass a placeholder created with nvflare.recipe.secrets.secret_ref() or nvflare.recipe.secrets.secret_file_ref() at a supported runtime boundary. See nvflare.recipe.secrets for the supported parameter locations.

Before export or run, recipes scan their parameters with heuristics and emit nvflare.recipe.secrets.PotentialSecretWarning when a value looks like an actual secret. The scan is best-effort: absence of a warning does not prove a parameter is safe to share.

param job:

the job that implements the recipe.