Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,235 questions
0 answers
Azure Synapse Link for Dataverse - How to delete audit entries after a specified period of time
We have a Synapse Link for Dataverse that sends audit table data into Synapse. When setting up the Synapse Link for Dataverse, Audit and Audit_Partitioned tables were automatically created. This setup works well, and all audit entries appear as…
asked 2025-03-13T12:14:43.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 38%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUA%3C/text%3E%3C/svg%3E)
Umar Ali
0
Reputation points
commented 2025-03-13T18:12:09.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
9,985
Reputation points Microsoft External Staff
1 answer
How to access secrets from Key Vault from Synapse Spark Job Definition PySpark file
Hello Everyone I'm trying to read the data source credentials as a secrets from the Key Vault using the Python SDK on a PySpark Job. I've gone through many articles and used different ways to read the secrets from Key Vault, but nothing worked out.…
asked 2025-03-04T07:32:09.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
manigandan
0
Reputation points
commented 2025-03-13T12:52:50.1933333+00:00
manigandan
0
Reputation points
1 answer
Restoring synapselink parquet files
Hi. We accidentally unlinked our audit table in synapselink for dataverse. Now we lost all data stored by synapselink from over 6 months and have been deleting data from d365. Is there anyway to recover the parquet files automatically created by…
asked 2025-03-13T07:44:51.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 7.000000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jani Hämäläinen
105
Reputation points
answered 2025-03-13T11:01:20.86+00:00
Amira Bedhiafi
29,711
Reputation points
1 answer
Update/Transform Headerless CSV Source
I am attempting to create a new dataflow to sink into a SQL table from a CSV source. My csv is missing it's header row though, so I'm stumbling on how to map the columns. It is delimited via pipe ('|'). Previous answers I have found reference a Headers…
asked 2025-03-10T23:28:23.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 33%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENF%3C/text%3E%3C/svg%3E)
Nestor Fregoso
0
Reputation points
commented 2025-03-13T08:57:17.9666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
14,560
Reputation points Microsoft External Staff
0 answers
Recommended CI/CD Setup for Azure Synapse
Most tutorials and recommendations for how to set up CICD in azure synapse suggest creating multiple workspaces (e.g. -dev, -qa, -uat, -prod) and using a devops pipeline to move artifacts between the environments. Including this video tutorial from the…
asked 2025-03-11T09:28:23.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 28.999999999999996%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFS%3C/text%3E%3C/svg%3E)
Finn Schmidt
86
Reputation points
commented 2025-03-13T08:55:32.26+00:00
Finn Schmidt
86
Reputation points
3 answers
I am trying to use the default sql storage of synapse ws as the sink for my data, but it throws logon failure
Here is the error I am getting Cannot connect to SQL database: 'jdbc:sqlserver://<synapse workspace name>.sql.azuresynapse.net:1433;database={Built-in}', 'Managed Identity (factory name): <synapse workspace name>'.[SQL Exception]Error…
asked 2025-02-07T12:14:52.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 26%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESN%3C/text%3E%3C/svg%3E)
Seema Nair
0
Reputation points
commented 2025-03-13T05:02:23.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
1 answer
Azure Synapse Link Roles
Hi Team I have a Synapse Link With F&O now we are transitioning to Production, but all the UAT process has been made with SysAdmin permissions on the Power Platform Environment, I tried to find documentation about the roles but can't find anything,…
asked 2025-03-10T18:50:37.9866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 99%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Manuel Alejandro Estrada Doñes
20
Reputation points
commented 2025-03-13T01:03:09.57+00:00
Ganesh Gurram
4,965
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Is It Possible to Parse JSON Data in an External Table with Serverless SQL Pool?
Hello, I’m currently working on creating an external table in Azure Synapse Serverless SQL Pool using a JSON Lines formatted file. However, I’ve encountered a limitation: Serverless SQL Pool does not natively support JSON file formats for external table…
asked 2025-03-11T20:37:10.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 98%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESF%3C/text%3E%3C/svg%3E)
Saul Figueroa
40
Reputation points
accepted 2025-03-12T21:03:22.7033333+00:00
Saul Figueroa
40
Reputation points
1 answer
Deterministic encryption algorithm for encrypting a data column
Hi, I need to encrypt only a table column of data possibly with a deterministic algorithm and a fixed key. I've tried to use the simple simmetric encryption, but for each execution changing the related encrypted column. Any suggests to me, please? Thanks
asked 2024-03-03T17:51:27.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
pmscorca
1,032
Reputation points
commented 2025-03-12T16:42:08.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vasili Seniuts
0
Reputation points
1 answer
One of the answers was accepted by the question author.
What’s the best approach to create external tables from JSON logs using Serverless SQL Pool?.
Hello, I am currently using Azure Logging to monitor logs in real-time via Log Stream. The goal is to store these logs in a Storage Account Gen 2 container and create an external table in Azure Synapse Analytics for data analysis. However, I…
asked 2025-03-10T21:16:18.3+00:00
Saul Figueroa
40
Reputation points
accepted 2025-03-12T13:12:00.8933333+00:00
Saul Figueroa
40
Reputation points
1 answer
How to remove/unset Spark configs that are pre-set by Microsoft
Background Synapse Apache Spark configurations allow users to create a blueprint for a Spark configuration that can be used for customizing the Spark behavior. This blueprint can be assigned to a Spark Pool (and will be used as new default for new…
asked 2025-03-08T12:09:21.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 42%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Martin B
126
Reputation points
commented 2025-03-12T09:12:30.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula
95
Reputation points Microsoft External Staff
2 answers
Unable to update datetime to sink via data flow
Hi, I want to pass in a parameter to the data flow. This variable is current UTC value. @formatDateTime(utcNow(), 'yyyy-MM-dd HH:mm:ss.fff') In my data flow, I have a derived column. My sink column is timestamp data…
asked 2025-03-05T06:01:45.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 14.000000000000002%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lotus88
116
Reputation points
commented 2025-03-11T20:22:54+00:00
Chandra Boorla
9,985
Reputation points Microsoft External Staff
1 answer
Unable to read content of file despite able to see FileInfo. urlopen error [Errno 5] Input/output error
I have 2 notebooks in different accounts, Staging & Production. Both use Managed Identity, linkedService -> System-assigned managed identity, and mounted drive. Both notebooks use the exact same code: Both can see the FileInfo, name, size,…
asked 2025-03-07T05:28:18.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 85%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDW%3C/text%3E%3C/svg%3E)
Daniel Wang
5
Reputation points
commented 2025-03-11T17:57:46.8366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
865
Reputation points Microsoft External Staff
1 answer
One of the answers was accepted by the question author.
Synapselink Audit table source files
Hi. We have Synapselink for dataverse that send audit table data. The link creates audit and audit_partitioned tables automatically. Because of need to update spark pool there was pause in automatic table syncing. Now Audit_partitioned is missing some…
asked 2025-03-07T10:45:39.65+00:00
Jani Hämäläinen
105
Reputation points
commented 2025-03-11T15:55:24.2966667+00:00
Vinodh247
29,781
Reputation points MVP
3 answers
Connect Local Machine to Synapse Lake Database using Python
Hi, I am trying to develop tooling for my use case, where I am using PySpark/Spark SQL on Synapse Analytics to create tables in Lake database. Sample code below: spark.sql("CREATE TABLE abc AS SELECT * FROM view") The table abc is created in…
asked 2025-02-17T15:02:43.2566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 88%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHN%3C/text%3E%3C/svg%3E)
Haria, Neel
0
Reputation points
commented 2025-03-11T09:14:40.2633333+00:00
AnnuKumari-MSFT
34,351
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Synapse stuck in provisioning state
I deployed a Synapse Workspace a long time ago. It never finished deploying, and now I want to get rid of it, but I am unable to delete it. The deletion fails with message: "Workspace is in non-terminal state Provisioning and cannot be…
asked 2025-03-06T13:39:11.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 28.000000000000004%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rasmus Kromann
20
Reputation points
accepted 2025-03-11T07:13:02.38+00:00
Rasmus Kromann
20
Reputation points
0 answers
Unable to use Linked Service Connection Using "To File Path": The configuration is overwritten after Publishing and during usage
Configuring Access to Amazon S3 with Limited Permissions I am working with an Amazon S3 storage system where my access is restricted to a specific bucket/ directory, rather than having root-level access. This limited permission setup has led me to…
asked 2025-03-07T08:37:29.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
sr24
0
Reputation points
commented 2025-03-11T03:22:20.6466667+00:00
AnnuKumari-MSFT
34,351
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Process streaming data with Synapse Pipeline or Data Flow
Hi Team, May I know if I can use Synapse pipeline or Data Flow to connect Eventhub and process Streaming data? Pipeline: Data Flow: I try to use this visual programming to create a process to read streaming data from eventhub and join cosmos db etc.…
asked 2024-10-18T13:18:52.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 71%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJZ%3C/text%3E%3C/svg%3E)
Jia Zhang
60
Reputation points
commented 2025-03-10T20:49:37.55+00:00
Chandra Boorla
9,985
Reputation points Microsoft External Staff
3 answers
One of the answers was accepted by the question author.
Azure ML Workspace - Unable to get access token for ADLS Gen2
Hello Microsoft Q&A, when running azure ml pipelines I got the following error: " permission denied when access stream. Reason: Some(This request is not authorized to perform this operation using this permission.) " When I checked the data…
asked 2023-04-11T15:48:31.0466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 35%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
Lukas
151
Reputation points
commented 2025-03-10T17:04:21.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 55.00000000000001%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Riyaz Ahmed Khaja Hussain
0
Reputation points
1 answer
Azure Synapse Analytics is not getting registered when working with Lab " Explore Azure Synapse Analytics"
getting error while trying the lab Explore Azure Synapse Analytics Registering resource providers... Microsoft.Synapse is not yet registered. Waiting for 30 seconds before rechecking... Microsoft.Synapse is not yet registered. Waiting for 30 seconds…
asked 2024-12-14T10:38:47.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 66%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Monish Bhanushali
15
Reputation points
commented 2025-03-08T07:26:15.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 86%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Shubham Dubey
0
Reputation points