Perform safe rollout of new deployments for real-time inference

Before following the steps in this article, make sure you have the following prerequisites:

• The Azure CLI and the ml extension to the Azure CLI. For more information, see Install, set up, and use the CLI (v2).

  Important

  The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.

• An Azure Machine Learning workspace. If you don't have one, use the steps in the Install, set up, and use the CLI (v2) to create one.

• A user account that has at least one of the following Azure role-based access control (Azure RBAC) roles:

  • An Owner role for the Azure Machine Learning workspace
  • A Contributor role for the Azure Machine Learning workspace
  • A custom role that has Microsoft.MachineLearningServices/workspaces/onlineEndpoints/* permissions

  For more information, see Manage access to Azure Machine Learning workspaces.

• Optionally, Docker Engine, installed and running locally. This prerequisite is highly recommended. You need it to deploy a model locally, and it's helpful for debugging.

APPLIES TO: Python SDK azure-ai-ml v2 (current)

Before following the steps in this article, make sure you have the following prerequisites:

• An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.

• To install the Python SDK v2, use the following command:

  pip install azure-ai-ml azure-identity
  

  To update an existing installation of the SDK to the latest version, use the following command:

  pip install --upgrade azure-ai-ml azure-identity
  

  For more information, see Install the Python SDK v2 for Azure Machine Learning.

• A user account that has at least one of the following Azure role-based access control (Azure RBAC) roles:

  • An Owner role for the Azure Machine Learning workspace
  • A Contributor role for the Azure Machine Learning workspace
  • A custom role that has Microsoft.MachineLearningServices/workspaces/onlineEndpoints/* permissions

  For more information, see Manage access to Azure Machine Learning workspaces.

• Optionally, Docker Engine, installed and running locally. This prerequisite is highly recommended. You need it to deploy a model locally, and it's helpful for debugging.

• An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.

• An Azure Machine Learning workspace. For instructions for creating a workspace, see Create the workspace.

• A user account that has at least one of the following Azure role-based access control (Azure RBAC) roles:

  • An Owner role for the Azure Machine Learning workspace
  • A Contributor role for the Azure Machine Learning workspace
  • A custom role that has Microsoft.MachineLearningServices/workspaces/onlineEndpoints/* permissions

  For more information, see Manage access to Azure Machine Learning workspaces.

Set environment variables

You can configure default values to use with the Azure CLI. To avoid passing in values for your subscription, workspace, and resource group multiple times, run the following code:

az account set --subscription <subscription-ID>
az configure --defaults workspace=<Azure-Machine-Learning-workspace-name> group=<resource-group-name>

Clone the examples repository

To follow along with this article, first clone the examples repository (azureml-examples). Then go to the repository's cli/ directory:

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples
cd cli

The commands in this tutorial are in the deploy-safe-rollout-online-endpoints.sh file in the cli directory, and the YAML configuration files are in the endpoints/online/managed/sample/ subdirectory.

Clone the examples repository

To run the training examples, first clone the examples repository (azureml-examples). Then go to the azureml-examples/sdk/python/endpoints/online/managed directory:

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples/sdk/python/endpoints/online/managed

The information in this article is based on the online-endpoints-safe-rollout.ipynb notebook. This article contains the same content as the notebook, but the order of the code blocks differs slightly between the two documents.

Connect to an Azure Machine Learning workspace

The workspace is the top-level resource for Azure Machine Learning. A workspace provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, you connect to your workspace, where you perform deployment tasks. To follow along, open your online-endpoints-safe-rollout.ipynb notebook.

1. Import the required libraries:

   # import required libraries
   from azure.ai.ml import MLClient
   from azure.ai.ml.entities import (
       ManagedOnlineEndpoint,
       ManagedOnlineDeployment,
       Model,
       Environment,
       CodeConfiguration,
   )
   from azure.identity import DefaultAzureCredential

   Note

   If you're using a Kubernetes online endpoint, import the KubernetesOnlineEndpoint and KubernetesOnlineDeployment class from the azure.ai.ml.entities library.

2. Configure workspace settings and get a handle to the workspace:

   To connect to a workspace, you need identifier parameters—a subscription, a resource group, and a workspace name. You use this information in the MLClient class from the azure.ai.ml module to get a handle to the required Azure Machine Learning workspace. This example uses the default Azure authentication.

   # enter details of your AML workspace
   subscription_id = "<SUBSCRIPTION_ID>"
   resource_group = "<RESOURCE_GROUP>"
   workspace = "<AML_WORKSPACE_NAME>"

   # get a handle to the workspace
   ml_client = MLClient(
       DefaultAzureCredential(), subscription_id, resource_group, workspace
   )

If you have Git installed on your local machine, you can follow the instructions to clone the examples repository. Otherwise, follow the instructions to download files from the examples repository.

Clone the examples repository

To follow along with this article, clone the azureml-examples repository, and then go to the azureml-examples/cli/endpoints/online/model-1 folder.

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples/cli/endpoints/online/model-1

Download files from the examples repository

Instead of cloning the examples repository, you can download the repository to your local machine:

1. Go to https://github.com/Azure/azureml-examples/.
2. Select <> Code, and then go to the Local tab and select Download ZIP.

Create an online endpoint

You first set the endpoint name and then configure it. In this article, you use the endpoints/online/managed/sample/endpoint.yml file to configure the endpoint. That file contains the following lines:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endpoint
auth_mode: key

The following table describes keys that the endpoint YAML format uses. To see how to specify these attributes, see CLI (v2) online endpoint YAML schema. For information about limits related to managed online endpoints, see Azure Machine Learning online endpoints and batch endpoints.

To create an online endpoint:

1. Set your endpoint name by running the following Unix command. Replace YOUR_ENDPOINT_NAME with a unique name.

   export ENDPOINT_NAME="<YOUR_ENDPOINT_NAME>"
   

   Important

   Endpoint names must be unique within an Azure region. For example, in the Azure westus2 region, there can be only one endpoint with the name my-endpoint.

2. Create the endpoint in the cloud by running the following code. This code uses the endpoint.yml file to configure the endpoint:

   az ml online-endpoint create --name $ENDPOINT_NAME -f endpoints/online/managed/sample/endpoint.yml
   

Create the blue deployment

You can use the endpoints/online/managed/sample/blue-deployment.yml file to configure the key aspects of a deployment named blue. That file contains the following lines:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: my-endpoint
model:
  path: ../../model-1/model/
code_configuration:
  code: ../../model-1/onlinescoring/
  scoring_script: score.py
environment: 
  conda_file: ../../model-1/environment/conda.yaml
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04:latest
instance_type: Standard_DS3_v2
instance_count: 1

To use the blue-deployment.yml file to create the blue deployment for your endpoint, run the following command:

az ml online-deployment create --name blue --endpoint-name $ENDPOINT_NAME -f endpoints/online/managed/sample/blue-deployment.yml --all-traffic

In the blue-deployment.yaml file, the path line specifies where to upload files from. The Azure Machine Learning CLI uses this information to upload the files and register the model and environment. As a best practice for production, you should register the model and environment and specify the registered name and version separately in the YAML code. Use the format model: azureml:<model-name>:<model-version> for the model, for example, model: azureml:my-model:1. For the environment, use the format environment: azureml:<environment-name>:<environment-version>, for example, environment: azureml:my-env:1.

For registration, you can extract the YAML definitions of model and environment into separate YAML files and use the commands az ml model create and az ml environment create. To find out more about these commands, run az ml model create -h and az ml environment create -h.

For more information about registering your model as an asset, see Register a model by using the Azure CLI or Python SDK. For more information about creating an environment, see Create a custom environment.

Create an online endpoint

To create a managed online endpoint, use the ManagedOnlineEndpoint class. This class provides a way for you to configure the key aspects of the endpoint.

1. Configure the endpoint:

   # Creating a unique endpoint name with current datetime to avoid conflicts
   import random
   
   online_endpoint_name = "endpt-moe-" + str(random.randint(0, 10000))
   
   # create an online endpoint
   endpoint = ManagedOnlineEndpoint(
       name=online_endpoint_name,
       description="this is a sample online endpoint",
       auth_mode="key",
       tags={"foo": "bar"},
   )

   Note

   To create a Kubernetes online endpoint, use the KubernetesOnlineEndpoint class.

2. Create the endpoint:

   ml_client.online_endpoints.begin_create_or_update(endpoint).result()

Create the blue deployment

To create a deployment for your managed online endpoint, use the ManagedOnlineDeployment class. This class provides a way for you to configure the key aspects of the deployment.

1. Configure the blue deployment:

   # create blue deployment
   model = Model(path="../model-1/model/sklearn_regression_model.pkl")
   env = Environment(
       conda_file="../model-1/environment/conda.yaml",
       image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
   )
   
   blue_deployment = ManagedOnlineDeployment(
       name="blue",
       endpoint_name=online_endpoint_name,
       model=model,
       environment=env,
       code_configuration=CodeConfiguration(
           code="../model-1/onlinescoring", scoring_script="score.py"
       ),
       instance_type="Standard_DS3_v2",
       instance_count=1,
   )

   In this example, the path parameter specifies where to upload files from. The Python SDK uses this information to upload the files and register the model and environment. As a best practice for production, you should register the model and environment and specify the registered name and version separately in the code.

   For more information about registering your model as an asset, see Register a model by using the Azure CLI or Python SDK.

   For more information about creating an environment, see Create a custom environment.

   Note

   To create a deployment for a Kubernetes online endpoint, use the KubernetesOnlineDeployment class.

2. Create the deployment:

   ml_client.online_deployments.begin_create_or_update(blue_deployment).result()

   # blue deployment takes 100 traffic
   endpoint.traffic = {"blue": 100}
   ml_client.online_endpoints.begin_create_or_update(endpoint).result()

When you create a managed online endpoint in Azure Machine Learning studio, you must define an initial deployment for the endpoint. Before you can define a deployment, you must have a registered model in your workspace. The following section shows you how to register a model to use for the deployment.

Register your model

A model registration is a logical entity in the workspace. This entity can contain a single model file or a directory of multiple files. As a best practice for production, you should register your model and environment.

To register the example model, take the steps in the following sections.

Upload the model files

1. Go to Azure Machine Learning studio.

2. Select Models.

3. Select Register, and then select From local files.

4. Under Model type, select Unspecified type.

5. Select Browse, and then select Browse folder.

   

6. Go to the local copy of the repository you cloned or downloaded earlier, and then select \azureml-examples\cli\endpoints\online\model-1\model. When prompted, select Upload and wait for the upload to finish.

7. Select Next.

Configure and register the model

1. On the Model settings page, under Name, enter a friendly name for the model. The steps in this article assume the model is named model-1.

2. Select Next, and then select Register to complete the registration.

For later examples in this article, you also need to register a model from the \azureml-examples\cli\endpoints\online\model-2\model folder in your local copy of the repository. To register that model, repeat the steps in the previous two sections, but name the model model-2.

For more information about working with registered models, see Work with registered models in Azure Machine Learning.

For information about creating an environment in the studio, see Create an environment.

Create a managed online endpoint and the blue deployment

You can use Azure Machine Learning studio to create a managed online endpoint directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You can't create an empty managed online endpoint.

One way to create a managed online endpoint in the studio is from the Models page. This method also provides an easy way to add a model to an existing managed online deployment. To deploy the model named model-1 that you registered previously in the Register your model section, take the steps in the following sections.

Select a model

1. Go to Azure Machine Learning studio, and then select Models.

2. In the list, select the model-1 model.

3. Select Deploy > Real-time endpoint.

   

   A window opens that you can use to specify detailed information about your endpoint.

   

Configure initial settings

1. Under Endpoint name, enter a name for your endpoint.

2. Under Compute type, keep the default value of Managed.

3. Under Authentication type, keep the default value of key-based authentication.

4. Select Next, and then on the Model page, select Next.

Configure remaining settings and create the deployment

1. On the Deployment page, take the following steps:

   1. Under Deployment name, enter blue.
   2. If you want to view graphs of your endpoint activities in the studio later:
      1. Under Inferencing data collection, turn on the toggle.
      2. Under Application Insights diagnostics, turn on the toggle.
   3. Select Next.

2. On the Code and environment for inferencing page, take the following steps:

   1. Under Select a scoring script for inferencing, select Browse, and then select the \azureml-examples\cli\endpoints\online\model-1\onlinescoring\score.py file from the repository you cloned or downloaded earlier.
   2. In the search box above the list of environments, start entering sklearn, and then select the sklearn-1.5:19 curated environment.
   3. Select Next.

3. On the Compute page, take the following steps:

   1. Under Virtual machine, keep the default value.
   2. Under Instance count, replace the default value with 1.
   3. Select Next.

4. On the Live Traffic page, select Next to accept the default traffic allocation of 100 percent to the blue deployment.

5. On the Review page, review your deployment settings, and then select Create.

   

Create an endpoint from the Endpoints page

Alternatively, you can create a managed online endpoint from the Endpoints page in the studio.

1. Go to Azure Machine Learning studio.

2. Select Endpoints.

3. Select Create.

   

   A window opens that you can use to specify detailed information about your endpoint and deployment.

4. Select a model, and then select Select.

5. Enter settings for your endpoint and deployment as described in the previous two sections. In each step, use the default values, and in the last step, select Create to create the deployment.

To view the status of your existing endpoint and deployment, run the following commands:

az ml online-endpoint show --name $ENDPOINT_NAME 

az ml online-deployment show --name blue --endpoint $ENDPOINT_NAME 

The output lists information about the $ENDPOINT_NAME endpoint and the blue deployment.

Test the endpoint by using sample data

You can invoke the endpoint by using the invoke command. The following command uses the sample-request.json JSON file to send a sample request:

az ml online-endpoint invoke --name $ENDPOINT_NAME --request-file endpoints/online/model-1/sample-request.json

Use the following code to check the status of the model deployment:

ml_client.online_endpoints.get(name=online_endpoint_name)

Test the endpoint by using sample data

You can use the instance of MLClient that you created earlier to get a handle to the endpoint. To invoke the endpoint, you can use the invoke command with the following parameters:

• endpoint_name: The name of the endpoint
• request_file: A file that contains request data
• deployment_name: The name of a deployment to test in the endpoint

The following code uses the sample-request.json JSON file to send a sample request.

# test the blue deployment with some sample data
ml_client.online_endpoints.invoke(
    endpoint_name=online_endpoint_name,
    deployment_name="blue",
    request_file="../model-1/sample-request.json",
)

View managed online endpoints

You can view all your managed online endpoints in the studio endpoints page. The Details tab of each endpoint's page displays critical information, such as the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code. To see this information, take the following steps:

1. In the studio, select Endpoints. A list of all the endpoints in the workspace is displayed.

2. Optionally, create a filter on the compute instance type to show only managed types.

3. Select an endpoint name to view the endpoint's Details page.

Test the endpoint by using sample data

On the endpoint page, you can use the Test tab to test your managed online deployment. To enter sample input and view the results, take the following steps:

1. On the endpoint page, go to the Test tab. In the Deployment list, the blue deployment is already selected.

2. Go to the sample-request.json file and copy its sample input.

3. In the studio, paste the sample input into the Input box.

4. Select Test.

In the deployment described in Deploy and score a machine learning model by using an online endpoint, you set the instance_count value to 1 in the deployment YAML file. You can scale out by using the update command:

az ml online-deployment update --name blue --endpoint-name $ENDPOINT_NAME --set instance_count=2

You can use the instance of MLClient that you created earlier to get a handle to the deployment. To scale the deployment, you can increase or decrease the value of instance_count.

# scale the deployment
blue_deployment = ml_client.online_deployments.get(
    name="blue", endpoint_name=online_endpoint_name
)
blue_deployment.instance_count = 2
ml_client.online_deployments.begin_create_or_update(blue_deployment).result()

Get detailed information about the endpoint

# Get the details for online endpoint
endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)

# existing traffic details
print(endpoint.traffic)

# Get the scoring URI
print(endpoint.scoring_uri)

To scale the deployment up or down by adjusting the number of instances, take the following steps:

1. On the endpoint page, go to the Details tab, and find the card for the blue deployment.

2. On the header of the blue deployment card, select the edit icon.

3. Under Instance count, enter 2.

4. Select Update.

Create a new deployment named green:

az ml online-deployment create --name green --endpoint-name $ENDPOINT_NAME -f endpoints/online/managed/sample/green-deployment.yml

Because you don't explicitly allocate any traffic to the green deployment, it has zero traffic allocated to it. You can verify that fact by using the following command:

az ml online-endpoint show -n $ENDPOINT_NAME --query traffic

Test the new deployment

Even though the green deployment has 0 percent of traffic allocated to it, you can invoke it directly by using the --deployment option:

az ml online-endpoint invoke --name $ENDPOINT_NAME --deployment-name green --request-file endpoints/online/model-2/sample-request.json

If you want to use a REST client to invoke the deployment directly without going through traffic rules, set the following HTTP header: azureml-model-deployment: <deployment-name>. The following code uses Client for URL (cURL) to invoke the deployment directly. You can run the code in a Unix or Windows Subsystem for Linux (WSL) environment. For instructions for retrieving the $ENDPOINT_KEY value, see Get the key or token for data plane operations.

# get the scoring uri
SCORING_URI=$(az ml online-endpoint show -n $ENDPOINT_NAME -o tsv --query scoring_uri)
# use curl to invoke the endpoint
curl --request POST "$SCORING_URI" --header "Authorization: Bearer $ENDPOINT_KEY" --header 'Content-Type: application/json' --header "azureml-model-deployment: green" --data @endpoints/online/model-2/sample-request.json

Create a new deployment for your managed online endpoint, and name the deployment green:

# create green deployment
model2 = Model(path="../model-2/model/sklearn_regression_model.pkl")
env2 = Environment(
    conda_file="../model-2/environment/conda.yaml",
    image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
)

green_deployment = ManagedOnlineDeployment(
    name="green",
    endpoint_name=online_endpoint_name,
    model=model2,
    environment=env2,
    code_configuration=CodeConfiguration(
        code="../model-2/onlinescoring", scoring_script="score.py"
    ),
    instance_type="Standard_DS3_v2",
    instance_count=1,
)

# use MLClient to create green deployment
ml_client.online_deployments.begin_create_or_update(green_deployment).result()

Test the new deployment

Even though the green deployment has 0 percent of the traffic allocated to it, you can still invoke the endpoint and deployment. The following code uses the sample-request.json JSON file to send a sample request.

ml_client.online_endpoints.invoke(
    endpoint_name=online_endpoint_name,
    deployment_name="green",
    request_file="../model-2/sample-request.json",
)

You can create a new deployment to add to your managed online endpoint. To create a deployment named green, take the steps in the following sections.

Configure initial settings

1. On the endpoint page, go to the Details tab, and then select Add Deployment.

2. On the Select a model page, select model-2, and then select Select.

3. On the Endpoint page and on the Model page, select Next.

4. On the Deployment page, take the following steps:

   1. Under Deployment name, enter green.
   2. Under Inferencing data collection, turn on the toggle.
   3. Under Application Insights diagnostics, turn on the toggle.
   4. Select Next.

5. On the Code and environment for inferencing page, take the following steps:

   1. Under Select a scoring script for inferencing, select Browse, and then select the \azureml-examples\cli\endpoints\online\model-2\onlinescoring\score.py file from the repository you cloned or downloaded earlier.
   2. In the search box above the list of environments, start entering sklearn, and then select the sklearn-1.5:19 curated environment.
   3. Select Next.

6. On the Compute page, take the following steps:

   1. Under Virtual machine, keep the default value.
   2. Under Instance count, replace the default value with 1.
   3. Select Next.

Configure remaining settings and create the deployment

1. On the Live Traffic page, select Next to accept the default traffic allocation of 100 percent to the blue deployment and 0 percent to green.

2. On the Review page, review your deployment settings, and then select Create.

   

Add a deployment from the Models page

Alternatively, you can use the Models page to add a deployment:

1. In the studio, select Models.

2. Select a model in the list.

3. Select Deploy > Real-time endpoint.

4. Under Endpoint, select Existing.

5. In the list of endpoints, select the managed online endpoint that you want to deploy the model to, and then select Next.

   

6. On the Model page, select Next.

7. To finish creating the green deployment, follow steps 4 through 6 in the Configure initial settings section and all the steps in the Configure remaining settings and create the deployment section.

Test the new deployment

Even though 0 percent of the traffic goes to the green deployment, you can still invoke the endpoint and that deployment. On the endpoint page, you can use the Test tab to test your managed online deployment. To enter sample input and view the results, take the following steps:

1. On the endpoint page, go to the Test tab.

2. In the Deployment list, select green.

3. Go to the sample-request.json file and copy its sample input.

4. In the studio, paste the sample input into the Input box.

5. Select Test.

Use the following command to mirror 10 percent of the traffic and send it to the green deployment:

az ml online-endpoint update --name $ENDPOINT_NAME --mirror-traffic "green=10"

You can test mirrored traffic by invoking the endpoint several times without specifying a deployment to receive the incoming traffic:

for i in {1..20} ; do
    az ml online-endpoint invoke --name $ENDPOINT_NAME --request-file endpoints/online/model-1/sample-request.json
done

You can confirm that the specified percentage of the traffic is sent to the green deployment by checking the logs from the deployment:

az ml online-deployment get-logs --name green --endpoint $ENDPOINT_NAME

After testing, you can set the mirror traffic to zero to disable mirroring:

az ml online-endpoint update --name $ENDPOINT_NAME --mirror-traffic "green=0"

Use the following code to mirror 10 percent of the traffic and send it to the green deployment:

endpoint.mirror_traffic = {"green": 10}
ml_client.begin_create_or_update(endpoint).result()

You can test mirrored traffic by invoking the endpoint several times without specifying a deployment to receive the incoming traffic:

# You can test mirror traffic by invoking the endpoint several times
for i in range(20):
    ml_client.online_endpoints.invoke(
        endpoint_name=online_endpoint_name,
        request_file="../model-1/sample-request.json",
    )

You can confirm that the specified percentage of the traffic is sent to the green deployment by checking the logs from the deployment:

ml_client.online_deployments.get_logs(
    name="green", endpoint_name=online_endpoint_name, lines=50
)

After testing, you can set the mirror traffic to zero to disable mirroring:

endpoint.mirror_traffic = {"green": 0}
ml_client.begin_create_or_update(endpoint).result()

To mirror 10 percent of the traffic and send it to the green deployment, take the following steps:

1. On the endpoint page, go to the Details tab, and then select Update traffic.

2. Turn on the Enable mirrored traffic toggle.

3. In the Deployment name list, select green.

4. Under Traffic allocation %, keep the default value of 10 percent.

5. Select Update.

The endpoint details page now shows a mirrored traffic allocation of 10 percent to the green deployment.

To test mirrored traffic, see the Azure CLI or Python tabs to invoke the endpoint several times. Confirm that the specified percentage of traffic is sent to the green deployment by checking the logs from the deployment. You can access the deployment logs on the endpoint page by going to the Logs tab.

You can also use metrics and logs to monitor the performance of the mirrored traffic. For more information, see Monitor online endpoints.

After testing, you can disable mirroring by taking the following steps:

1. On the endpoint page, go to the Details tab, and then select Update traffic.

2. Turn off the Enable mirrored traffic toggle.

3. Select Update.

az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=90 green=10"

endpoint.traffic = {"blue": 90, "green": 10}
ml_client.begin_create_or_update(endpoint).result()

1. On the endpoint page, go to the Details tab, and then select Update traffic.

2. Adjust the deployment traffic by allocating 10 percent to the green deployment and 90 percent to the blue deployment.

3. Select Update.

az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=0 green=100"

endpoint.traffic = {"blue": 0, "green": 100}
ml_client.begin_create_or_update(endpoint).result()

1. On the endpoint page, go to the Details tab, and then select Update traffic.

2. Adjust the deployment traffic by allocating 100 percent to the green deployment and 0 percent to the blue deployment.

3. Select Update.

az ml online-deployment delete --name blue --endpoint $ENDPOINT_NAME --yes --no-wait

ml_client.online_deployments.begin_delete(
    name="blue", endpoint_name=online_endpoint_name
).wait()

1. On the endpoint page, go to the Details tab, and then go to the blue deployment card.

2. Next to the deployment name, select the delete icon.

az ml online-endpoint delete --name $ENDPOINT_NAME --yes --no-wait

ml_client.online_endpoints.begin_delete(name=online_endpoint_name)

1. Go to Azure Machine Learning studio.

2. Select Endpoints.

3. Select an endpoint in the list.

4. Select Delete.

Alternatively, you can delete a managed online endpoint directly in the endpoint page by going to the Details tabs and selecting the delete icon.