Random MessageBox when executing tests

Question

Random MessageBox when executing tests

Lorenzo Zambelli 65

Hello,

we are running vstest.console in Visual Studio 2022 Enterprise Edition on a Hyper-V Windows 11 24H2 virtual machine, and we randomly get the blocking messagebox you see in the screenshot below, does anyone know the cause of that and how to fix it?

Thank you in advance for your support. image_2025_03_04T12_43_27_197Z

Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-05T09:24:24.74+00:00

Hi @Lorenzo Zambelli ,

The window seems doesn’t provide any error message. If you click the OK button or close it, will the test continue running? Will the tests pass?

Do you use a custom .runsettings file? If so, can you confirm that this issue is not caused by one of your custom configurations in the .runsettings file?

Is this issue replicated on a newly created test project?

Besides, I see many messages say "The test attribute has a null or empty value: Owner". Do you consider fixing this warning and testing again?
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-10T13:11:07.0766667+00:00
Hi @Lorenzo Zambelli ,

Does this issue persist? If you try to run the PowerShell as administrator, will it work?

I suggest you try following to troubleshoot this issue

Try to use vstest.console.exe to run a newly created simple project, see if the same issue appears. It's helpful to confirm if this issue is caused by specific configurations or tests.

Try to test your project on another machine if possible. It's helpful to confirm if this issue is caused by the environment(system).
Lorenzo Zambelli 65 Reputation points

2025-03-11T07:32:14.0366667+00:00

Hello @Tianyu Sun-MSFT ,

thank you for your replies and sorry for my delay.

The window seems doesn’t provide any error message. If you click the OK button or close it, will the test continue running? Will the tests pass?

When I click ok, the tests will continue running, but it won't always fail.

Do you use a custom .runsettings file? If so, can you confirm that this issue is not caused by one of your custom configurations in the .runsettings file?

We don't use a .runsettings file

Is this issue replicated on a newly created test project?

Actually all our tests are new because we migrating to this platform from a previous custom one.

Besides, I see many messages say "The test attribute has a null or empty value: Owner". Do you consider fixing this warning and testing again?

How can we fix that?

Does this issue persist? If you try to run the PowerShell as administrator, will it work?

This issue is persisting, at the moment running PowerShell as administrator the messagebox has not yet appeared, but because it's random I need to run some more tests to confirm that admin role will fix it, but if that's the case how could I run my Azure DevOps pipeline process as administrator?
Lorenzo Zambelli 65 Reputation points

2025-03-11T07:40:03.7933333+00:00

Removed duplicated comment.
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-11T13:58:24.1433333+00:00

Hi @Lorenzo Zambelli,

...but if that's the case how could I run my Azure DevOps pipeline process as administrator?

In that case, I suggest you try to add Azure DevOps tag for further help.

I'm not very sure if there are some settings need to be set there, but in that case, at least the cause is confirmed, and it's related to permissions.

Besides, from your description, I doubt that this window may be a warning window that some files or resources cannot be accessed.

Is there any special permission or rule on your machine? Are you running other processes at the same time?
Lorenzo Zambelli 65 Reputation points

2025-03-11T15:09:05.5066667+00:00

Hello @Tianyu Sun-MSFT ,

after several runs I can confirm that the PowerShell as administrator won't solve the issue.

So our best hope is to fix the warnings you saw in my previous screenshot... Do you have any idea how to fix them?
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-12T10:02:14.85+00:00

Hi @Lorenzo Zambelli ,

I saw that the warning message mentioned test attribute. Did you check the test attribute in your test file? Are there any special or custom settings for the test attribute in your test file?

I also saw that the vstest.console.exe searched and read the YAML file, is there any corresponding setting in the YAML file?

You can try to test with a newly created simple test file(I knew that your tests are also new, maybe try a simple one), I believe that these warnings won’t appear. If so, try to match and test the test attributes, see if one of them triggers the warning.

A simple test for example:
Lorenzo Zambelli 65 Reputation points

2025-03-12T13:24:05.74+00:00

Hello @Tianyu Sun-MSFT ,

thank you for your suggestions.

We fixed the "empty owner" warning, but nothing changed, we still have the MessageBox.

The values from .yaml files are correctly read, otherwise our tests won't run correctly.

I can try a brand new test like you suggested that won't require a .yaml file, but I realized that I forgot to say that we are using C++, not C#... I don't know if this could help you...

Your answer

Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-05T09:24:24.74+00:00

Hi @Lorenzo Zambelli ,

The window seems doesn’t provide any error message. If you click the OK button or close it, will the test continue running? Will the tests pass?

Do you use a custom .runsettings file? If so, can you confirm that this issue is not caused by one of your custom configurations in the .runsettings file?

Is this issue replicated on a newly created test project?

Besides, I see many messages say "The test attribute has a null or empty value: Owner". Do you consider fixing this warning and testing again?
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-10T13:11:07.0766667+00:00

Hi @Lorenzo Zambelli ,

Does this issue persist? If you try to run the PowerShell as administrator, will it work?

I suggest you try following to troubleshoot this issue

Try to use vstest.console.exe to run a newly created simple project, see if the same issue appears. It's helpful to confirm if this issue is caused by specific configurations or tests.

Try to test your project on another machine if possible. It's helpful to confirm if this issue is caused by the environment(system).
Lorenzo Zambelli 65 Reputation points

2025-03-11T07:32:14.0366667+00:00

Hello @Tianyu Sun-MSFT ,

thank you for your replies and sorry for my delay.

The window seems doesn’t provide any error message. If you click the OK button or close it, will the test continue running? Will the tests pass?

When I click ok, the tests will continue running, but it won't always fail.

Do you use a custom .runsettings file? If so, can you confirm that this issue is not caused by one of your custom configurations in the .runsettings file?

We don't use a .runsettings file

Is this issue replicated on a newly created test project?

Actually all our tests are new because we migrating to this platform from a previous custom one.

Besides, I see many messages say "The test attribute has a null or empty value: Owner". Do you consider fixing this warning and testing again?

How can we fix that?

Does this issue persist? If you try to run the PowerShell as administrator, will it work?

This issue is persisting, at the moment running PowerShell as administrator the messagebox has not yet appeared, but because it's random I need to run some more tests to confirm that admin role will fix it, but if that's the case how could I run my Azure DevOps pipeline process as administrator?
Lorenzo Zambelli 65 Reputation points

2025-03-11T07:40:03.7933333+00:00

Removed duplicated comment.
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-11T13:58:24.1433333+00:00

Hi @Lorenzo Zambelli,

...but if that's the case how could I run my Azure DevOps pipeline process as administrator?

In that case, I suggest you try to add Azure DevOps tag for further help.

I'm not very sure if there are some settings need to be set there, but in that case, at least the cause is confirmed, and it's related to permissions.

Besides, from your description, I doubt that this window may be a warning window that some files or resources cannot be accessed.

Is there any special permission or rule on your machine? Are you running other processes at the same time?
Lorenzo Zambelli 65 Reputation points

2025-03-11T15:09:05.5066667+00:00

Hello @Tianyu Sun-MSFT ,

after several runs I can confirm that the PowerShell as administrator won't solve the issue.

So our best hope is to fix the warnings you saw in my previous screenshot... Do you have any idea how to fix them?
Tianyu Sun-MSFT 33,551 Reputation points Microsoft External Staff

2025-03-12T10:02:14.85+00:00

Hi @Lorenzo Zambelli ,

I saw that the warning message mentioned test attribute. Did you check the test attribute in your test file? Are there any special or custom settings for the test attribute in your test file?

I also saw that the vstest.console.exe searched and read the YAML file, is there any corresponding setting in the YAML file?

You can try to test with a newly created simple test file(I knew that your tests are also new, maybe try a simple one), I believe that these warnings won’t appear. If so, try to match and test the test attributes, see if one of them triggers the warning.

A simple test for example:
Lorenzo Zambelli 65 Reputation points

2025-03-12T13:24:05.74+00:00

Hello @Tianyu Sun-MSFT ,

thank you for your suggestions.

We fixed the "empty owner" warning, but nothing changed, we still have the MessageBox.

The values from .yaml files are correctly read, otherwise our tests won't run correctly.

I can try a brand new test like you suggested that won't require a .yaml file, but I realized that I forgot to say that we are using C++, not C#... I don't know if this could help you...

Share via

Random MessageBox when executing tests

Your answer