Custom Text Log DCR not Ingesting

Question

Custom Text Log DCR not Ingesting

Conradie, Francois F 0

Hi,

I am struggling to determine why my Custom text DCR is not ingesting from a Linux box. I have tried just about every guide and troubleshooting site and still am not able to determine why this is not working. When ingesting the desired log file from a local linux server I manage it works, but when trying to ingest from the actual source server it just does.... nothing. No errors.

Here are the details:

Custom table in Sentinel created with RawData : string and TimeGenerated: DateTime
DCR created in same location as Custom table, same Resource Group and Subscription.
DCR has source server/resource with logs added which is an ARC / AMA enabled Linux resource in AWS.
DCE created in same Location, resource group and subscription as the Custom table and associated with DCR
No transformation used, left as source.

User's image

When looking at DCR metrics it is constantly moving up/down from 0 to 5 giving the impression something is being picked up. No dropped rows or Transform errors based on metrics.
Source Server is sending heartbeat via AMA.

User's image

No errors in DCRLogErrors

I am starting to think it may be connectivity issues or even permissions to read the log file?

i) Does AMA need specific file permissions on the folder it is picking the log files up from?

Thanks

Naveena Patlolla 975 Reputation points Microsoft External Staff

2025-03-12T18:08:18.23+00:00
Hi Conradie, Francois F

Network and Firewall Checks

1.Outbound connectivity:

Ensure the Linux server can reach the DCE endpoint over the internet.

Test connectivity to the DCE endpoint:

curl -v https://<DCE-endpoint>

Verify there are no firewall or proxy restrictions blocking traffic.

Since this is an AWS-based Linux machine, verify connectivity to Azure Monitor endpoints:

Test connectivity to Azure Monitor ingestion endpoints:

nc -zv global.ingest.monitor.azure.com 443

If the connection fails, ensure outbound connectivity to the required Azure Monitor ingestion URLs.

Check Azure Arc and AMA Status

Azure Arc status:

Go to the Azure portal > Azure Arc > Servers.

Verify the Linux server is listed and shows as "Connected."

Check the server’s resource in Azure to ensure there are no errors.

Validate Data Flow Using AMA Diagnostic Logs

Run the following command to check AMA logs:

sudo tail -f /var/opt/microsoft/azuremonitoragent/log/mdmtraces.log

Look for any errors related to log ingestion.

Please "upvote" if the comment is helpful
Conradie, Francois F 0 Reputation points

2025-03-12T18:38:39.86+00:00

Naveena Patlolla Thanks so much for these, I will test them and advise.

Fyi I do not see a mdmtraces.log, but will tail the mssd.err / info / warn.
Conradie, Francois F 0 Reputation points

2025-03-13T06:16:03.5733333+00:00

Update,

1.Outbound connectivity: I can curl from the source server to the DCE endpoint. It does give a 404, but I assume this is expected. This would also mean there are no firewall/proxies in place that is preventing connection?
2. Check Azure Arc and AMA Status: Both ARC and AMA show connected and "success" via Portal and Heartbeat is coming through.
3. Test connectivity to Azure Monitor ingestion endpoints: TBC
4. Validate Data Flow Using AMA Diagnostic Logs: TBC

I am wondering if file permissions may be an issue? How do I confirm the agent has permission to read the files? I saw this in an another post but was for windows "Make sure the Azure Monitor Agent can read the log files. Right-click the file, go to Properties > Security, and check that the agent has permission to read the file."

Thanks again for the help.
Madugula Jahnavi 10 Reputation points Microsoft External Staff

2025-03-13T09:29:26.36+00:00

Conradie, Francois F, Does the monitor agent user have read access to the log file and its directories!
Conradie, Francois F 0 Reputation points

2025-03-13T11:02:59.27+00:00

Hi @Madugula Jahnavi ,

Yes it does seem so. Is there a way to confirm/test this?

What I do know is that these are the permissions as seen on the folder structure:
Parent Directory: drwxr-xr-x
Log directory: -rw-r--r--
Madugula Jahnavi 10 Reputation points Microsoft External Staff

2025-03-13T12:31:01.86+00:00

Conradie, Francois F. You can check access control with getfacl command and verify if the reader access existed.

Your answer

Naveena Patlolla 975 Reputation points Microsoft External Staff

2025-03-12T18:08:18.23+00:00

Hi Conradie, Francois F

Network and Firewall Checks

1.Outbound connectivity:

Ensure the Linux server can reach the DCE endpoint over the internet.

Test connectivity to the DCE endpoint:

curl -v https://<DCE-endpoint>

Verify there are no firewall or proxy restrictions blocking traffic.

Since this is an AWS-based Linux machine, verify connectivity to Azure Monitor endpoints:

Test connectivity to Azure Monitor ingestion endpoints:

nc -zv global.ingest.monitor.azure.com 443

If the connection fails, ensure outbound connectivity to the required Azure Monitor ingestion URLs.

Check Azure Arc and AMA Status

Azure Arc status:

Go to the Azure portal > Azure Arc > Servers.

Verify the Linux server is listed and shows as "Connected."

Check the server’s resource in Azure to ensure there are no errors.

Validate Data Flow Using AMA Diagnostic Logs

Run the following command to check AMA logs:

sudo tail -f /var/opt/microsoft/azuremonitoragent/log/mdmtraces.log

Look for any errors related to log ingestion.

Please "upvote" if the comment is helpful
Conradie, Francois F 0 Reputation points

2025-03-12T18:38:39.86+00:00

Naveena Patlolla Thanks so much for these, I will test them and advise.

Fyi I do not see a mdmtraces.log, but will tail the mssd.err / info / warn.
Conradie, Francois F 0 Reputation points

2025-03-13T06:16:03.5733333+00:00

Update,

1.Outbound connectivity: I can curl from the source server to the DCE endpoint. It does give a 404, but I assume this is expected. This would also mean there are no firewall/proxies in place that is preventing connection?
2. Check Azure Arc and AMA Status: Both ARC and AMA show connected and "success" via Portal and Heartbeat is coming through.
3. Test connectivity to Azure Monitor ingestion endpoints: TBC
4. Validate Data Flow Using AMA Diagnostic Logs: TBC

I am wondering if file permissions may be an issue? How do I confirm the agent has permission to read the files? I saw this in an another post but was for windows "Make sure the Azure Monitor Agent can read the log files. Right-click the file, go to Properties > Security, and check that the agent has permission to read the file."

Thanks again for the help.
Madugula Jahnavi 10 Reputation points Microsoft External Staff

2025-03-13T09:29:26.36+00:00

Conradie, Francois F, Does the monitor agent user have read access to the log file and its directories!
Conradie, Francois F 0 Reputation points

2025-03-13T11:02:59.27+00:00

Hi @Madugula Jahnavi ,

Yes it does seem so. Is there a way to confirm/test this?

What I do know is that these are the permissions as seen on the folder structure:
Parent Directory: drwxr-xr-x
Log directory: -rw-r--r--
Madugula Jahnavi 10 Reputation points Microsoft External Staff

2025-03-13T12:31:01.86+00:00

Conradie, Francois F. You can check access control with getfacl command and verify if the reader access existed.

Share via

Custom Text Log DCR not Ingesting

Your answer