How can I enable the 'SHAREPOINT' type connection in Azure Databricks?

Question

How can I enable the 'SHAREPOINT' type connection in Azure Databricks?

Power BI Test User 2 0

I tried to connect SharePoint using Azure Databricks Lakehouse Federation.

First, I select Connection type "Microsoft SharePoint" to create a Lakehouse Federation. User's image Next, I input authentication info and sign in with Microsoft SharePoint. Then, I am successfully authorized but fail to create connection

How can I enable the 'SHAREPOINT' type connection in Azure Databricks?

Any settings needed?

Ganesh Gurram 4,965 Reputation points Microsoft External Staff

2025-03-05T05:29:03.24+00:00
Hi @Power BI Test User 2

Thank you for posting your query!

Currently, SharePoint is not natively supported as a data source by Lakehouse Federation. Supported data sources are shown below:

However, you can still integrate SharePoint data into Azure Databricks using the following workaround:

Ensure that your Azure Databricks workspace can establish outbound connections, specifically:

Login authentication → login.microsoftonline.com (port 443) for Azure AD authentication.

Storage access → Ensure that your Databricks clusters can access Azure Blob Storage or ADLS, which will act as an intermediate storage layer.

Note: Since direct connectivity is unavailable, we recommend moving SharePoint data to a supported storage service using either Azure Data Factory (ADF) or Azure Logic Apps:

Using Azure Data Factory (ADF):

Create a data pipeline using the SharePoint connector in ADF.

Set Azure Blob Storage / ADLS as the destination.

Schedule and monitor the pipeline to ensure timely data transfers.

Using Azure Logic Apps:

Create a Logic App that listens for SharePoint file changes.

Use the SharePoint connector to extract files and save them to Azure Blob Storage or ADLS.

Once your data is in Azure Blob Storage or ADLS, set up a connection:

Navigate to the Catalog in your Azure Databricks workspace. Click "Add a Connection". Select Azure Blob Storage or ADLS as the Connection Type. Provide the necessary connection details (e.g., storage account credentials). Test the connection to ensure it's working correctly. Click "Create Connection" to complete the setup. Assign Proper Permissions

Ensure that users have the right access to manage the data:

Assign USE CONNECTION and CREATE FOREIGN CATALOG permissions in Databricks.

Ensure that Azure RBAC roles (e.g., "Storage Blob Data Reader") are assigned for users accessing the data.

For more details refer: https://learn.microsoft.com/en-us/azure/databricks/query-federation/

https://learn.microsoft.com/en-us/azure/databricks/connect/

Similar issues: https://learn.microsoft.com/en-us/answers/questions/990518/connect-azure-databricks-service-to-sharepoint?utm_source=chatgpt.com

https://learn.microsoft.com/en-us/answers/questions/643110/reading-data-from-sharepoint-using-databricks?utm_source=chatgpt.com

Hope this helps. Do let us know if you have any further queries.
Power BI Test User 2 0 Reputation points

2025-03-05T07:05:55.72+00:00

@Ganesh Gurram

Thank you for your answer!

Verify Feature Availability in Azure Databricks - Navigate to Databricks Admin Console > Feature Enablement. Ensure that the Microsoft SharePoint connection type is enabled for your workspace.

I'm not sure there is no option to enable Microsoft SharePoint connection type ...

Are there any version differences?
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ganesh Gurram 4,965 Reputation points Microsoft External Staff

2025-03-06T03:52:07.71+00:00

@Power BI Test User 2 - Apologize for the inconvenience caused!

As I mentioned in the previous response, Currently, SharePoint is not natively supported as a data source by Lakehouse Federation.

I tried to reproduce the issue from my end.

SharePoint is not available as a connection type.

I hope this information helps!
AnnuKumari-MSFT 34,351 Reputation points Microsoft Employee

2025-03-10T09:49:00.3633333+00:00

Hello Power BI Test User 2 ,

Just checking if you got a chance to go through the previous response and try the suggested approach? Kindly let us know if the response helped in answering your query. Thankyou

1 answer

Your answer

Power BI Test User 2 0 Reputation points

2025-03-05T07:05:55.72+00:00

@Ganesh Gurram

Thank you for your answer!

Verify Feature Availability in Azure Databricks - Navigate to Databricks Admin Console > Feature Enablement. Ensure that the Microsoft SharePoint connection type is enabled for your workspace.

I'm not sure there is no option to enable Microsoft SharePoint connection type ...

Are there any version differences?
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ganesh Gurram 4,965 Reputation points Microsoft External Staff

2025-03-06T03:52:07.71+00:00

@Power BI Test User 2 - Apologize for the inconvenience caused!

As I mentioned in the previous response, Currently, SharePoint is not natively supported as a data source by Lakehouse Federation.

I tried to reproduce the issue from my end.

SharePoint is not available as a connection type.

I hope this information helps!
AnnuKumari-MSFT 34,351 Reputation points Microsoft Employee

2025-03-10T09:49:00.3633333+00:00

Hello Power BI Test User 2 ,

Just checking if you got a chance to go through the previous response and try the suggested approach? Kindly let us know if the response helped in answering your query. Thankyou

Answer 1

Hello @Power BI Test User 2,

As per this MS Documentation,

Currently, Sharepoint is not support as a Data source in the Lakehouse Federation of Azure Databricks.

You can try the below workaround if there is a need to achieve this using Databricks.

Using Microsoft App registration and the Microsoft Graph API:

First create an App registration and create a secret. Store the details like client id, tenant id and secret.

Grant Files.Read.All api permission to the application with Application permission type.

enter image description here

Generate the access token using the below code:


auth_url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"

auth_data = {

    "grant_type": "client_credentials",

    "client_id": client_id,

    "client_secret": client_secret,

    "scope": "https://graph.microsoft.com/.default"

}

auth_response = requests.post(auth_url, data=auth_data)

access_token = auth_response.json().get("access_token")

Next get your site id using below code.


headers = {"Authorization": f"Bearer {access_token}"}

site_res = requests.get("https://graph.microsoft.com/v1.0/sites/root:/sites/<site_name>",headers=headers)

site_id = site_res.json()['id']

Now, use the below code to get the file stored in dbfs.


download_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drive/root:/filename.xlsx:/content"

file_response = requests.get(download_url, headers=headers)

print(file_response.status_code)

if file_response.status_code == 200:

    file_path = "/dbfs/filename.xlsx"

    with open(file_path, "wb") as f:

        f.write(file_response.content)

    print(f"file was created")

else:

    print(file_response.status_code)

Now, you can read this file from DBFS as per your requirement.

Hope this helps. Do let us know if you have any further queries.

Share via

How can I enable the 'SHAREPOINT' type connection in Azure Databricks?

1 answer

Your answer