Hi Sayak Banerjee,
Thanks for reaching out to Microsoft Q&A platform.
Please check current/latest Status of cluster.
First, head over to the Azure Portal, navigate to HDInsight, and locate your cluster named "sayakban-cc-spark-cluster." If it’s stuck in the "Deleting" state, let's move on to the next steps.
Sometimes, dependencies might prevent the deletion. You can try deleting them manually. Go to the Resource Group where the cluster is deployed, then delete the Storage Account, Virtual Network, and any Public IP associated with the cluster. After removing these dependencies, retry deleting the cluster.
If the manual deletion doesn’t work, you can use the Azure CLI to force delete the cluster. Run the following command in Azure Cloud Shell or your local terminal with Azure CLI installed:
az hdinsight delete --name sayakban-cc-spark-cluster --resource-group <resource-group-name> --yes
Don't forget to replace <resource-group-name>
with the actual resource group name of your cluster.
If the cluster is still not budging, try stopping it before deletion using PowerShell. Use these commands:
Stop-AzHDInsightCluster -ResourceGroupName "<resource-group-name>" -ClusterName "sayakban-cc-spark-cluster" -Force
Remove-AzHDInsightCluster -ResourceGroupName "<resource-group-name>" -ClusterName "sayakban-cc-spark-cluster" -Force
I hope these steps help you resolve the issue. If you have any questions or need further assistance, feel free to reach out.
If you find the above information helpful, please do not forget to accept this answer.