Azure Confidential Virtual Machine Deployment Guide

Overview

Creating Azure confidential virtual machine (CVM) is quite similar to creating regular VM. This guide will utilize Azure CLI (az) to create one Azure CVM and verify it by performing attestation operations. After that, you can install NVFlare and transfer the startup kits to start NVFlare inside the CVM.

Note

Launching Azure CVM may require your Azure account to have certain permissions. Please consult your Azure account and Azure for more information.

Steps for Launching Azure CVM

Login and create one resource group
Create the Azure CVM
Retrieve attestation reports

Login and create one resource group

First, you have to login to Azure with az cli. Then you can create one resource group to host all resources generated by the following operations.

You can choose another name for the resource group and another location.

#!/usr/bin/env bash

resource_group=cc-cvm-rg
location=northeurope

az login

az group create --name $resource_group --location $location

Create the Azure CVM

With the resource group created, we can go directly to create the CVM.

#!/usr/bin/env bash

resource_group=cc-cvm-rg
cvm_name=cc_prep_cvm
cvm_size=Standard_DC4as_v5
user_name=azureuser
user_password=<YOUR_OWN_PASSWORD>
image_name=Canonical:0001-com-ubuntu-confidential-vm-jammy:22_04-lts-cvm:latest

az vm create --resource-group $resource_group \
   --name $cvm_name \
   --size $cvm_size \
   --admin-username $user_name \
   --admin-password $user_password \
   --enable-vtpm true \
   --image $image_name \
   --public-ip-sku Standard --security-type ConfidentialVM \
   --os-disk-security-encryption-type VMGuestStateOnly \
   --enable-secure-boot true

This cvm_size is based on AMD SEV-SNP. Therefore, the attestation token retrieved at the next step will contain snp-related fields. Remember to change the user_password to your own password. Your subscription may have policies to ensure higher security, please examine all properties, network securities and permissions for compliances.

Retrieve attestation reports

You will find the public IPv4 address of the above CVM. Please log in to it with the credential defined in the above script.

To retrieve the attestation reports inside the CVM, we first need to prepare the environment. Run the following commands to install necessary tools and download source codes to perform attestation.

#!/usr/bin/env bash

sudo apt-get update && \
   sudo apt-get install -y build-essential cmake unzip jq \
   libcurl4-openssl-dev libjsoncpp-dev libboost-all-dev nlohmann-json3-dev

wget https://packages.microsoft.com/repos/azurecore/pool/main/a/azguestattestation1/azguestattestation1_1.1.2_amd64.deb
sudo dpkg -i azguestattestation1_1.1.2_amd64.deb

wget https://github.com/Azure/confidential-computing-cvm-guest-attestation/archive/refs/heads/main.zip
unzip main.zip

pushd confidential-computing-cvm-guest-attestation-main/cvm-attestation-sample-app
cmake . && make

sudo install -D -m0755 AttestationClient /usr/local/bin

popd

Now the attestation tool is built and installed. We can retrieve the attestation token and examine it.

#!/usr/bin/env bash

sudo AttestationClient -o token > token.b64
jwt=$(cat token.b64)
echo "Showing attestation token in base64-encoded format"
echo $jwt

echo "Showing the header of attestation token"
echo -n $jwt | cut -d "." -f 1 | base64 -d 2>/dev/null | jq .

echo "Showing the payload of attestation token"
echo -n $jwt | cut -d "." -f 2 | base64 -d 2>/dev/null | jq .

Next Steps

Now you can install NVFlare and transfer your startup kit into this CVM instance and start the NVFlare.

The following is a sample cc_site-1.yml file, which is used with project.yml for cc provision. A sample project.yml is also shown in the following. Note this project.yml includes the server’s cc configuration yaml file, which is described in the Confidential Azure Container Instances Deployment Guide - Secure Aggregation on FLARE Server with Azure ACI (Azure Container Instance)

The AZCVMAuthorizer uses sharedeus2.eus2.attest.azure.net as the default Microsoft Azure Attestation endpoint.

compute_env: azure_cvm
cc_cpu_mechanism: amd_sev_snp
role: client
cc_issuers:
  - id: az_cvm_authorizer
    path: nvflare.app_opt.confidential_computing.az_cvm_authorizer.AZCVMAuthorizer
    token_expiration: 100 # seconds, needs to be less than check_frequency

The following is the sample project.yml file.

api_version: 3
name: example_project
description: NVIDIA FLARE sample project yaml file
participants:
  # Change the name of the server (server1) to the Fully Qualified Domain Name
  # (FQDN) of the server, for example: server1.example.com.
  # Ensure that the FQDN is correctly mapped in the /etc/hosts file.
  - name: server1
    type: server
    org: nvidia
    fed_learn_port: 8002
    cc_config: cc_server.yml
  - name: site-1
    type: client
    org: nvidia
    cc_config: cc_site-1.yml
    # Specifying listening_host will enable the creation of one pair of
    # certificate/private key for this client, allowing the client to function
    # as a server for 3rd-party integration.
    # The value must be a hostname that the external trainer can reach via the network.
    # listening_host: site-1-lh
  - name: admin@nvidia.com
    type: admin
    org: nvidia
    role: project_admin
# The same methods in all builders are called in their order defined in builders section
builders:
  - path: nvflare.lighter.impl.workspace.WorkspaceBuilder
  - path: nvflare.lighter.impl.static_file.StaticFileBuilder
    args:
      # config_folder can be set to inform NVIDIA FLARE where to get configuration
      config_folder: config
      # scheme for communication driver (currently supporting the default, grpc, only).
      # scheme: grpc

      # app_validator is used to verify if uploaded app has proper structures
      # if not set, no app_validator is included in fed_server.json
      # app_validator: PATH_TO_YOUR_OWN_APP_VALIDATOR
  - path: nvflare.lighter.impl.cert.CertBuilder
  - path: nvflare.lighter.cc_provision.impl.cc.CCBuilder
  - path: nvflare.lighter.impl.signature.SignatureBuilder