More Search Latency from AKS with Azure Postgres Flexible Server

When connecting to an Azure Postgres Flexible Server with the Pgvector extension enabled, I am experiencing higher search latency when inferencing from AKS compared to Azure Databricks. The setup includes a HNSW index on text embeddings for semantic search results. Despite separating the connection time from the search time, the latency from AKS remains significant and more than that of Azure Databricks. What could be the reasons for this difference in performance?