Identifying Deadlocks using Windows Debugging Tools (WinDbg)

0.00 avg. rating (0% score) - 0 votes

If you had ever worked with multi-threaded applications, you will surely have faced situations where you have to troubleshoot deadlocks and it could be a nightmare to do so. You might have also worked with WinDbg. I came across a similar scenario and used WinDbg to identify deadlocks. So I thought its worth sharing. So here we go:

When we install Windows Debugging Tools and start using it, we came across number of issues which held us from following along. I was able to proceed by following following these steps:

  • My process got hang up at times. I used the following command to take dump of the process. The following command will create .dmp file of the process which will give us complete memory dump of the process that we can analyze later on:
    • adplus -p 6912 -hang -y c:\symbols  -o d:\Loogs (6912 is the PID of the process which is getting hung up)
  • From within WinDbg, we can either connect to a process directly  or open a dump file that can be created by following command mentioned above
  • Now came the task of loading debugging  symbols. Its better to provide the online symbol path with following command OR download symbols and give the proper path. For online symbols, give this command:
    • .symfix c:\smbols
  • Once symbols are loaded, some assemblies are to be loaded in order to figure out what has caused a deadlock. Load the following assembly:
    • .loadby sos mscorwks
  • There could be an error that assemblies could not be loaded or at least I had got this problem. SOS assemblies should be in GAC or on the root of the Windows Debugging Tools installation directory to get loaded. In case of any issues, follow this URL  http://blogs.msdn.com/b/dougste/archive/2009/02/18/failed-to-load-data-access-dll-0×80004005-or-what-is-mscordacwks-dll.aspx
  • If the assemblies are loaded successfully. Type ‘!SyncBlk’ to display all resources which were locked at the time we generated the .dmp file. Please note:
    • If ‘Monitor Held’ column shows 1, it means no thread is waiting
    • If ‘Monitor Held’ column shows 11, it means 5 thread are waiting for this resource
    • In case you had connected to a running process, waiting thread numbers will also be listed. Else they won’t
  • Till now we know which threads were waiting for a resource. In order to figure out deadlocks, we will need some more assemblies. Definitely you would have to do some research to figure it out but in this case, you are lucky by reading this blog :)
  • Now, download SOSEx dll files by visiting http://www.stevestechspot.com/SOSEXUpdatedV11Available.aspx which will help us figuring out deadlock
  • Place the dll file in the root folder of Windows Debugging Tools
  • Now execute command ‘.load sosex’ to load the DLL withing WinDbg
  • Once successfully loaded, give the command to figure out deadlocks. Command is ‘!dlk’. This command takes sometime

This is it and you can get to the bottlenecks but in other cases, you might also have to look at thread current instruction position and/or CLR. See below a screen shot showing a sample crash dump analysis. This analysis has been done on one of the applications I was doing troubleshooting with. SOSEx assemblies were really helpful, you can see that we came to know the thread numbers too causing the deadlock that we can investigate further what went wrong there.

0.00 avg. rating (0% score) - 0 votes

W@rfi

Owner of this blog site. Have expertise on Microsoft technologies.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *