nvflare.fuel.hci.client.fl_admin_api_spec module

exception APISyntaxError

Bases: Exception

class FLAdminAPIResponse(status: APIStatus, details: dict | None = None, raw: dict | None = None)

Bases: dict

Structure containing the response of calls to the api as key value pairs.

The status key is the primary indicator of the success of a call and can contain APIStatus.SUCCESS or another APIStatus. Most calls will return additional information in the details key, which is also a dictionary of key value pairs. The raw key can optionally have the underlying response from AdminAPI when relevant, particularly when data is received from the server and the status of a call is APIStatus.ERROR_RUNTIME to provide additional information.

Note that the status in this response primarily indicates that the command submitted successfully. Depending on the command and especially for calls to multiple clients, the contents of details or the raw response should be examined to determine if the execution of the command was successful for each specific client.

Parameters:

• status – APIStatus for primary indicator of the success of a call

• details – response details

• raw – raw response from server

class FLAdminAPISpec

Bases: ABC

abstract abort(job_id: str, target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Issue a command to abort training.

Parameters:

• job_id (str) – job id

• target_type – server | client

• targets – if target_type is client, targets can optionally be a list of client names

Returns: FLAdminAPIResponse

abstract abort_job(job_id: str) → FLAdminAPIResponse

Abort a job that is running.

Parameters:

job_id (str) – the job id to abort

Returns: FLAdminAPIResponse

abstract cat_target(target: str, options: str | None = None, file: str | None = None) → FLAdminAPIResponse

Issue cat command.

Sends the shell command to get the contents of the target’s specified file allowing for options that the cat command of admin client allows. The target can be “server” or a specific client name for example “site2”. The file is required and should contain the relative path to the file from the working directory of the target. The allowed options are “-n” to number all output lines, “-b” to number nonempty output lines, “-s” to suppress repeated empty output lines, and “-T” to display TAB characters as ^I.

Parameters:

• target (str) – either server or single client’s client name.

• options (str) – the options string as provided to the ls command for admin client.

• file (str) – the path to the file to return the contents of

Returns: FLAdminAPIResponse

abstract check_status(target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Checks and returns the FL status.

If target_type is server, the call does not wait for the server to retrieve information on the clients but returns the last information the server had at the time this call is made.

If target_type is client, specific clients can be specified in targets, and this call generally takes longer than the function to just check the FL server status because this one waits for communication from the server to client then back.

Note that this is still the previous training check_status, and there will be a new call to get status through InfoCollector, which will be able to get information from components.

Returns: FLAdminAPIResponse

abstract clone_job(job_id: str) → FLAdminAPIResponse

Clone a job that exists by copying the job contents and providing a new job_id.

Parameters:

job_id (str) – job id of the job to clone

Returns: FLAdminAPIResponse

abstract delete_job(job_id: str) → FLAdminAPIResponse

Delete the specified job and workspace from the permanent store.

Parameters:

job_id (str) – the job id to delete

Returns: FLAdminAPIResponse

abstract download_job(job_id: str) → FLAdminAPIResponse

Download the specified job in the system.

Parameters:

job_id (str) – Job id for the job to download

Returns: FLAdminAPIResponse

abstract get_active_sp() → FLAdminAPIResponse

Gets the active server (service provider).

Returns: FLAdminAPIResponse

abstract get_available_apps_to_upload()

abstract get_connected_client_list() → FLAdminAPIResponse

A convenience function to get a list of the clients currently connected to the FL server.

Operates through the check status server call. Note that this returns the client list based on the last known statuses on the server, so it can be possible for a client to be disconnected and not yet removed from the list of connected clients.

Returns: FLAdminAPIResponse

abstract get_working_directory(target: str) → FLAdminAPIResponse

Gets the workspace root directory of the specified target.

Parameters:

target (str) – either server or single client’s client name.

Returns: FLAdminAPIResponse

abstract grep_target(target: str, options: str | None = None, pattern: str | None = None, file: str | None = None) → FLAdminAPIResponse

Issue grep command.

Sends the shell command to grep the contents of the target’s specified file allowing for options that the grep command of admin client allows. The target can be “server” or a specific client name for example “site2”. The file is required and should contain the relative path to the file from the working directory of the target. The pattern is also required. The allowed options are “-n” to print line number with output lines, “-i” to ignore case distinctions, and “-b” to print the byte offset with output lines.

Parameters:

• target (str) – either server or single client’s client name.

• options (str) – the options string as provided to the grep command for admin client.

• pattern (str) – the pattern to search for

• file (str) – the path to the file to grep

Returns: FLAdminAPIResponse

abstract list_jobs(options: str | None = None) → FLAdminAPIResponse

List the jobs in the system.

Parameters:

options (str) – the options string as provided to the list_jobs command for admin client.

Returns: FLAdminAPIResponse

abstract list_sp() → FLAdminAPIResponse

Gets the information on the available servers (service providers).

Returns: FLAdminAPIResponse

abstract ls_target(target: str, options: str | None = None, path: str | None = None) → FLAdminAPIResponse

Issue ls command to retrieve the contents of the path.

Sends the shell command to get the directory listing of the target allowing for options that the ls command of admin client allows. If no path is specified, the contents of the working directory are returned. The target can be “server” or a specific client name for example “site2”. The allowed options are: “-a” for all, “-l” to use a long listing format, “-t” to sort by modification time newest first, “-S” to sort by file size largest first, “-R” to list subdirectories recursively, “-u” with -l to show access time otherwise sort by access time.

Parameters:

• target (str) – either server or single client’s client name.

• options (str) – the options string as provided to the ls command for admin client.

• path (str) – optionally, the path to specify (relative to the working directory of the specified target)

Returns: FLAdminAPIResponse

abstract promote_sp(sp_end_point: str) → FLAdminAPIResponse

Sends command through overseer_agent to promote the specified sp_end_point to become the active server.

Parameters:

sp_end_point – service provider end point to promote to active in the form of server:fl_port:admin_port like example.com:8002:8003

Returns: FLAdminAPIResponse

abstract remove_client(targets: List[str]) → FLAdminAPIResponse

Issue a command to remove a specific FL client or FL clients.

Note that the targets will not be able to start with an API command after shutting down. Also, you will not be able to issue admin commands through the server to that client until the client is restarted (this includes being able to issue the restart command through the API).

Parameters:

targets – a list of client names

Returns: FLAdminAPIResponse

abstract reset_errors(job_id: str) → FLAdminAPIResponse

Resets the collector errors.

Parameters:

job_id (str) – job id

Returns: FLAdminAPIResponse

abstract restart(target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Issue a command to restart the specified target.

If the target is server, all FL clients will be restarted as well.

Parameters:

• target_type – server | client

• targets – if target_type is client, targets can optionally be a list of client names

Returns: FLAdminAPIResponse

abstract set_timeout(timeout: float) → FLAdminAPIResponse

Sets the timeout for admin commands on the server in seconds.

This timeout is the maximum amount of time the server will wait for replies from clients. If the timeout is too short, the server may not receive a response because clients may not have a chance to reply.

Parameters:

timeout – timeout in seconds of admin commands to set on the server

Returns: FLAdminAPIResponse

abstract show_errors(job_id: str, target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Gets and shows errors from the Info Collector.

Parameters:

• job_id (str) – job id

• target_type – server | client

• targets – if target_type is client, targets can optionally be a list of client names

Returns: FLAdminAPIResponse

abstract show_stats(job_id: str, target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Gets and shows stats from the Info Collector.

Parameters:

• job_id (str) – job id

• target_type – server | client

• targets – if target_type is client, targets can optionally be a list of client names

Returns: FLAdminAPIResponse

abstract shutdown(target_type: TargetType, targets: List[str] | None = None) → FLAdminAPIResponse

Issue a command to stop FL entirely for a specific FL client or specific FL clients.

Note that the targets will not be able to start with an API command after shutting down.

Parameters:

• target_type – server | client

• targets – if target_type is client, targets can optionally be a list of client names

Returns: FLAdminAPIResponse

abstract submit_job(job_folder: str) → FLAdminAPIResponse

Submit a job.

Assumes job folder is in the upload_dir set in API init.

Parameters:

job_folder (str) – name of the job folder in upload_dir to submit

Returns: FLAdminAPIResponse

abstract tail_target_log(target: str, options: str | None = None) → FLAdminAPIResponse

Returns the end of target’s log allowing for options that the tail of admin client allows.

The option “-n” can be used to specify the number of lines for example “-n 100”, or “-c” can specify the number of bytes.

Parameters:

• target (str) – either server or single client’s client name.

• options (str) – the options string as provided to the tail command for admin client. For this command, “-n” can be used to specify the number of lines for example “-n 100”, or “-c” can specify the number of bytes.

Returns: FLAdminAPIResponse

abstract wait_until_server_status(interval: int = 20, timeout: int | None = None, callback: Callable[[FLAdminAPIResponse], bool] | None = None, fail_attempts: int = 3) → FLAdminAPIResponse

Wait until provided callback returns True.

There is the option to specify a timeout and interval to check the server status. If no callback function is provided, the default callback returns True when the server status is “training stopped”. A custom callback can be provided to add logic to handle checking for other conditions. A timeout should be set in case there are any error conditions that result in the system being stuck in a state where the callback never returns True.

Parameters:

• interval (int) – in seconds, the time between consecutive checks of the server

• timeout (int) – if set, the amount of time this function will run until before returning a response message

• callback – the reply from check_status_server() will be passed to the callback, along with any additional kwargs

• logic. (which can go on to perform additional) –

• fail_attempts (int) – number of consecutive failed attempts of getting the server status before returning with ERROR_RUNTIME.

Returns: FLAdminAPIResponse

class TargetType(value)

Bases: str, Enum

An enumeration.

ALL = 'all'

CLIENT = 'client'

SERVER = 'server'