nvflare.fuel.hci.client.fl_admin_api_spec module¶
- class FLAdminAPIResponse(status: APIStatus, details: dict | None = None, raw: dict | None = None)[source]¶
Bases:
dict
Structure containing the response of calls to the api as key value pairs.
The status key is the primary indicator of the success of a call and can contain APIStatus.SUCCESS or another APIStatus. Most calls will return additional information in the details key, which is also a dictionary of key value pairs. The raw key can optionally have the underlying response from AdminAPI when relevant, particularly when data is received from the server and the status of a call is APIStatus.ERROR_RUNTIME to provide additional information.
Note that the status in this response primarily indicates that the command submitted successfully. Depending on the command and especially for calls to multiple clients, the contents of details or the raw response should be examined to determine if the execution of the command was successful for each specific client.
- Parameters:
status – APIStatus for primary indicator of the success of a call
details – response details
raw – raw response from server
- class FLAdminAPISpec[source]¶
Bases:
ABC
- abstract abort(job_id: str, target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Issue a command to abort training.
- Parameters:
job_id (str) – job id
target_type – server | client
targets – if target_type is client, targets can optionally be a list of client names
Returns: FLAdminAPIResponse
- abstract abort_job(job_id: str) FLAdminAPIResponse [source]¶
Abort a job that is running.
- Parameters:
job_id (str) – the job id to abort
Returns: FLAdminAPIResponse
- abstract cat_target(target: str, options: str | None = None, file: str | None = None) FLAdminAPIResponse [source]¶
Issue cat command.
Sends the shell command to get the contents of the target’s specified file allowing for options that the cat command of admin client allows. The target can be “server” or a specific client name for example “site2”. The file is required and should contain the relative path to the file from the working directory of the target. The allowed options are “-n” to number all output lines, “-b” to number nonempty output lines, “-s” to suppress repeated empty output lines, and “-T” to display TAB characters as ^I.
- Parameters:
target (str) – either server or single client’s client name.
options (str) – the options string as provided to the ls command for admin client.
file (str) – the path to the file to return the contents of
Returns: FLAdminAPIResponse
- abstract check_status(target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Checks and returns the FL status.
If target_type is server, the call does not wait for the server to retrieve information on the clients but returns the last information the server had at the time this call is made.
If target_type is client, specific clients can be specified in targets, and this call generally takes longer than the function to just check the FL server status because this one waits for communication from the server to client then back.
Note that this is still the previous training check_status, and there will be a new call to get status through InfoCollector, which will be able to get information from components.
Returns: FLAdminAPIResponse
- abstract clone_job(job_id: str) FLAdminAPIResponse [source]¶
Clone a job that exists by copying the job contents and providing a new job_id.
- Parameters:
job_id (str) – job id of the job to clone
Returns: FLAdminAPIResponse
- abstract delete_job(job_id: str) FLAdminAPIResponse [source]¶
Delete the specified job and workspace from the permanent store.
- Parameters:
job_id (str) – the job id to delete
Returns: FLAdminAPIResponse
- abstract download_job(job_id: str) FLAdminAPIResponse [source]¶
Download the specified job in the system.
- Parameters:
job_id (str) – Job id for the job to download
Returns: FLAdminAPIResponse
- abstract get_active_sp() FLAdminAPIResponse [source]¶
Gets the active server (service provider).
Returns: FLAdminAPIResponse
- abstract get_connected_client_list() FLAdminAPIResponse [source]¶
A convenience function to get a list of the clients currently connected to the FL server.
Operates through the check status server call. Note that this returns the client list based on the last known statuses on the server, so it can be possible for a client to be disconnected and not yet removed from the list of connected clients.
Returns: FLAdminAPIResponse
- abstract get_working_directory(target: str) FLAdminAPIResponse [source]¶
Gets the workspace root directory of the specified target.
- Parameters:
target (str) – either server or single client’s client name.
Returns: FLAdminAPIResponse
- abstract grep_target(target: str, options: str | None = None, pattern: str | None = None, file: str | None = None) FLAdminAPIResponse [source]¶
Issue grep command.
Sends the shell command to grep the contents of the target’s specified file allowing for options that the grep command of admin client allows. The target can be “server” or a specific client name for example “site2”. The file is required and should contain the relative path to the file from the working directory of the target. The pattern is also required. The allowed options are “-n” to print line number with output lines, “-i” to ignore case distinctions, and “-b” to print the byte offset with output lines.
- Parameters:
target (str) – either server or single client’s client name.
options (str) – the options string as provided to the grep command for admin client.
pattern (str) – the pattern to search for
file (str) – the path to the file to grep
Returns: FLAdminAPIResponse
- abstract list_jobs(options: str | None = None) FLAdminAPIResponse [source]¶
List the jobs in the system.
- Parameters:
options (str) – the options string as provided to the list_jobs command for admin client.
Returns: FLAdminAPIResponse
- abstract list_sp() FLAdminAPIResponse [source]¶
Gets the information on the available servers (service providers).
Returns: FLAdminAPIResponse
- abstract ls_target(target: str, options: str | None = None, path: str | None = None) FLAdminAPIResponse [source]¶
Issue ls command to retrieve the contents of the path.
Sends the shell command to get the directory listing of the target allowing for options that the ls command of admin client allows. If no path is specified, the contents of the working directory are returned. The target can be “server” or a specific client name for example “site2”. The allowed options are: “-a” for all, “-l” to use a long listing format, “-t” to sort by modification time newest first, “-S” to sort by file size largest first, “-R” to list subdirectories recursively, “-u” with -l to show access time otherwise sort by access time.
- Parameters:
target (str) – either server or single client’s client name.
options (str) – the options string as provided to the ls command for admin client.
path (str) – optionally, the path to specify (relative to the working directory of the specified target)
Returns: FLAdminAPIResponse
- abstract promote_sp(sp_end_point: str) FLAdminAPIResponse [source]¶
Sends command through overseer_agent to promote the specified sp_end_point to become the active server.
- Parameters:
sp_end_point – service provider end point to promote to active in the form of server:fl_port:admin_port like example.com:8002:8003
Returns: FLAdminAPIResponse
- abstract remove_client(targets: List[str]) FLAdminAPIResponse [source]¶
Issue a command to remove a specific FL client or FL clients.
Note that the targets will not be able to start with an API command after shutting down. Also, you will not be able to issue admin commands through the server to that client until the client is restarted (this includes being able to issue the restart command through the API).
- Parameters:
targets – a list of client names
Returns: FLAdminAPIResponse
- abstract reset_errors(job_id: str) FLAdminAPIResponse [source]¶
Resets the collector errors.
- Parameters:
job_id (str) – job id
Returns: FLAdminAPIResponse
- abstract restart(target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Issue a command to restart the specified target.
If the target is server, all FL clients will be restarted as well.
- Parameters:
target_type – server | client
targets – if target_type is client, targets can optionally be a list of client names
Returns: FLAdminAPIResponse
- abstract set_timeout(timeout: float) FLAdminAPIResponse [source]¶
Sets the timeout for admin commands on the server in seconds.
This timeout is the maximum amount of time the server will wait for replies from clients. If the timeout is too short, the server may not receive a response because clients may not have a chance to reply.
- Parameters:
timeout – timeout in seconds of admin commands to set on the server
Returns: FLAdminAPIResponse
- abstract show_errors(job_id: str, target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Gets and shows errors from the Info Collector.
- Parameters:
job_id (str) – job id
target_type – server | client
targets – if target_type is client, targets can optionally be a list of client names
Returns: FLAdminAPIResponse
- abstract show_stats(job_id: str, target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Gets and shows stats from the Info Collector.
- Parameters:
job_id (str) – job id
target_type – server | client
targets – if target_type is client, targets can optionally be a list of client names
Returns: FLAdminAPIResponse
- abstract shutdown(target_type: TargetType, targets: List[str] | None = None) FLAdminAPIResponse [source]¶
Issue a command to stop FL entirely for a specific FL client or specific FL clients.
Note that the targets will not be able to start with an API command after shutting down.
- Parameters:
target_type – server | client
targets – if target_type is client, targets can optionally be a list of client names
Returns: FLAdminAPIResponse
- abstract submit_job(job_folder: str) FLAdminAPIResponse [source]¶
Submit a job.
Assumes job folder is in the upload_dir set in API init.
- Parameters:
job_folder (str) – name of the job folder in upload_dir to submit
Returns: FLAdminAPIResponse
- abstract tail_target_log(target: str, options: str | None = None) FLAdminAPIResponse [source]¶
Returns the end of target’s log allowing for options that the tail of admin client allows.
The option “-n” can be used to specify the number of lines for example “-n 100”, or “-c” can specify the number of bytes.
- Parameters:
target (str) – either server or single client’s client name.
options (str) – the options string as provided to the tail command for admin client. For this command, “-n” can be used to specify the number of lines for example “-n 100”, or “-c” can specify the number of bytes.
Returns: FLAdminAPIResponse
- abstract wait_until_server_status(interval: int = 20, timeout: int | None = None, callback: Callable[[FLAdminAPIResponse], bool] | None = None, fail_attempts: int = 3) FLAdminAPIResponse [source]¶
Wait until provided callback returns True.
There is the option to specify a timeout and interval to check the server status. If no callback function is provided, the default callback returns True when the server status is “training stopped”. A custom callback can be provided to add logic to handle checking for other conditions. A timeout should be set in case there are any error conditions that result in the system being stuck in a state where the callback never returns True.
- Parameters:
interval (int) – in seconds, the time between consecutive checks of the server
timeout (int) – if set, the amount of time this function will run until before returning a response message
callback – the reply from check_status_server() will be passed to the callback, along with any additional kwargs
logic. (which can go on to perform additional)
fail_attempts (int) – number of consecutive failed attempts of getting the server status before returning with ERROR_RUNTIME.
Returns: FLAdminAPIResponse