[转]A Word of Windbg

Before I came to work at Microsoft I worked as a professional developer on the Microsoft platform and I used to work with the Visual Studio (6 latterly) debugger which I thought was a pretty cool debugger.

 

When I joined Microsoft I found that a lot of people worked with WinDbg as their primary debugger and, for a while, I just didn't get it as it seemed like an unnecessarily grungy tool for achieving things that Visual Studio could already do.

 

However, over a period of time I came to see the light and I'm now of the view that WinDbg is about the best debugger I've ever used (including those ones I used to work with on Unix platforms). WinDbg is just the most powerful debugger out there for user-mode debugging and I don’t do any other kind of debugging J

 

So, there is a lot of information out there already on WinDbg but I just wanted to try and highlight it here and provide a single-pager on it as it seems to be grossly overlooked as a debugger for managed code and, yet, it’s capable of a whole tonne of stuff that no single other debugger can do for you right now.

 

People are often put off WinDbg because of its interface and the vast array of commands so in this little series of postings I’ll try and demystify a little bit and show how you can debug a few common scenarios in managed code with it.

 

In this post, let’s just get to the point where we can use WinDbg to debug a regular process and do some work with it then in the next post(s) I’ll look at more stuff that you can do.

 

Installation

 

So, first things first, get hold of a copy of WinDbg from here and install it. If you want to follow along with this post then you can go and download it right now - the install takes about 2 minutes once you've got the 10MB download down from the site.

 

The first thing you'll realise is how lightweight an installation WinDbg is and that (sometimes) means that you can install it into places that you'd never install a copy of Visual Studio or the Visual Studio remote debug set up. Indeed, often if you raise a support call with Microsoft you'll get asked to install WinDbg and it's automated helper (the "autodumper") in order to get a crash dump of your application to send back to Microsoft.

 

Before we get into debugging something I need to have a quick word or two about Symbols and Commands.

Symbols

 

The first issue that we need to address in order to progress is that of symbols. If you want to get decent stack traces and dumps of variable values and so on out of the debugger then you need symbols for the modules that you’re debugging whether those modules are yours or someone else’s and whether those modules are managed or unmanaged (although there’s quite a lot more that you can do with managed code without symbols).

 

Symbols are typically either private symbols (include variable information), retail symbols (function information but not variable information) or export symbols (generally not so useful for debugging purposes). If you’re debugging with “export” symbols it’s not usually enough to actually work out what’s going on.

 

For your own code you’re responsible for building the symbol files (usually .PDB program databases) by setting the right flags on the compiler or choosing the right configuration in a project (this is true for VC++, VB6 and all the .NET languages). This will give you full private symbols.

 

When the debugger (and lots of other tools) want to find symbols they typically check the contents of an environment variable named _NT_SYMBOL_PATH which is used (just like PATH – i.e. a list of folder separated by semi-colons) to find symbol files. You can set _NT_SYMBOL_PATH prior to running WinDbg or you can use the debugger’s own symbol path settings to find symbol files. These settings are controlled via the File->Symbol File Path menu option or using the .sympath command.

 

For other people’s code you need to get symbols from them. For Microsoft code you’re in luck because the symbols are available from a public symbol server on the internet. In order to make use of the symbol server you should set your symbol path to be something like;

 

SRV*C:\MyLocalSymbols*http://msdl.microsoft.com/download/symbols

 

What this says is that the debugger should first check a local cache named C:\MyLocalSymbols and if symbols are not found there then go out to the Microsoft symbol server for symbols and, if found, download them and cache them in that folder. Note that symbols are downloaded based upon name, version and checksum information from the modules being debugged so the debugger usually knows if the symbols match the code properly.

 

If you wanted to combine this with a local location for your own code then you’d use something like;

 

            C:\MyCodesSymbols; SRV*C:\MyLocalSymbols*http://msdl.microsoft.com/download/symbols

 

so that the debugger would check your folder first and would then proceed to look in the cache and finally to the symbol server for symbols.

Commands

 

A core set of commands would be as below. I rarely venture beyond these as I’m not an advanced user of WinDbg by any means and this gets me by.

 

.hh - bring up the help file :-) The help file is fantastic so don't skip on it - everything's documented in there.

 

g - "GO!". That is, continue running.

 

bp (Address) - set a break point. Note that (address) can take many formats here including just a memory location but the most common format specifier here is to use syntax such as;

 

bp KERNEL32!CreateFileA

 

(remember that most Windows functions have an ASCII and a Unicode variant so we have KERNEL32!CreateFileA and KERNEL32!CreateFileW).

 

bl - list all breakpoints. Each breakpoint listed has a number in the list which you need for...

 

bc (number), be (number), bd (number) - these respectively clear, enable and disable breakpoints from the list.

 

kb, kp, kd etc. The commands beginning with "k" show the stack for the thread that the debugger is looking at. There are a few variants so check out the help for those. The simplest variant that I use is;

 

kb 200

 

(200 is the maximum depth of stack trace that I'm looking for)

 

dw, db, ds, etc. The commands beginning with "d" show the contents of memory. One of the most common is "dt" which shows you the contents of memory laid out according to the specification of a type as long as the debugger can see the definition of a type.

 

sxe, sxd, sxi, sxn. The commands beginning with "sx" set the behaviour for what the debugger should do when an exception occurs. A good example here would be;

 

sxe 0xc0000005

 

which is saying "I want to break into the debugger if there's an access violation".

 

lm - lists the modules loaded by the program and what kind of symbols are loaded for the modules. Note that export symbols are not really symbols at all but are really the debugger guessing which function you're in based upon the export table of the DLL. For COM servers in particular (which have only 4 exported functions or so) you'll often find yourself appearing in DllRegisterServer if you only have export symbols. Good examples here would be;

 

lm v (verbose mode)

lm v mUSER* (verbose mode, matc any modules that begin with USER*)

 

x - Examine symbols. This is very, very useful as it shows you the set of symbols loaded for particular modules. A good example here would be;

 

x USER32!* (show me all the symbols loaded from the USER32 module)

x USER32!Create* (show me all the symbols loaded from the USER32 module that begin with "Create")

 

.cls – clear the screen. You’ll be needing this one!

 

.reload – this causes the symbol information to be reloaded for a particular module.  The most common form of this would be to do something like;

 

.reload /f user32.dll

 

Where the /f overrides the debugger’s naturally lazy mode of working whereby it wouldn’t actually do the reload right there and then. Normally, this is a useful parameter to use.

 

~ - this is the tilde character and it lists all the threads in the process and shows their status. This can be combined with wildcards in order to execute particular commands on all threads in the process. The single most common one is probably to combine it with a “stack” command to do something like;

 

~* kb 200

 

which will show you the stack frames for all the threads in the process.

 

~N s – this changes the debugger’s focus to another thread. For example;

 

~7 s

 

Will switch the debugger so that it’s focused on thread 7. Note that 7 here is the debugger’s thread number rather than the real thread ID and it comes from the list given by the ~ command.

 

Along with the core set of commands, WinDbg is capable of loading up debugger extensions to extend the core functionality of the debugger. These are “bang commands” in that they begin with an exclamation mark. You can manipulate the extensions that the debugger has loaded using the following commands;

 

.chain – this shows the extensions that the debugger has loaded

.load – this loads an extension DLL (e.g. .load SOS.DLL)

.unload – unloads an extension DLL

.unloadall

.setdll – this sets the “default” extension DLL.

 

A quick word about .setdll and how command processing works here. If you have an extension namedUEXT.DLL which contains a command named help (and most extensions will have a help command) then you can run that command using;

 

!UEXT.help

 

And that works fine. If you find yourself using UEXT more than any other extension then you can use;

 

.setdll UEXT.DLL

 

To make UEXT.DLL the default extension and then you can just use;

 

!help

 

And that will now default to mean !UEXT.help because of the default extension setting.

 

A First Debugging Session

 

So, let’s finish up this post with a quick debugging session and next time around I’ll talk about debugging managed code rather than just any old process.

 

1. Run up WinDbg and also run up a copy of NotePad.
2. Within WinDbg attach to the NotePad process that you just ran by either using the File->Attach to Process menu or the F6 shortcut. This will give you a dialog where you can select a process. Note that the “Non Invasive” option would allow you to detach from the debugged process without killing it (if you’re on Windows XP or 2003) but it limits the commands that you can use so leave it unchecked.
3. Don’t worry about the workspace information, let it go.
4. The debugger should present some diagnostics as it loads information and will then stop with the command prompt. The debuggee (NotePad) is now halted.
5. Let’s take a look around. Firstly, let’s look at the modules that are loaded by notepad. Execute a “lm v”to list all the modules and follow it up with a simple “lm”. On my system I get this;

 

0:001> lm

start    end        module name

01000000 01014000   notepad    (no symbols)          

4ce00000 4cf02000   COMCTL32   (export symbols)       C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.2149_x-ww_a84b1f06\COMCTL32.dll

5ad70000 5ada7000   uxtheme    (export symbols)       C:\WINDOWS\system32\uxtheme.dll

605d0000 605d9000   mslbui     (export symbols)       C:\WINDOWS\system32\mslbui.dll

61220000 61232000   MSH_ZWF    (export symbols)       C:\Program Files\Microsoft Hardware\Mouse\MSH_ZWF.dll

73000000 73026000   WINSPOOL   (export symbols)       C:\WINDOWS\system32\WINSPOOL.DRV

74720000 7476b000   MSCTF      (export symbols)       C:\WINDOWS\system32\MSCTF.dll

763b0000 763f9000   comdlg32   (export symbols)       C:\WINDOWS\system32\comdlg32.dll

77120000 771ac000   OLEAUT32   (export symbols)       C:\WINDOWS\system32\OLEAUT32.DLL

771b0000 779c4000   SHELL32    (export symbols)       C:\WINDOWS\system32\SHELL32.dll

779d0000 77a46000   SHLWAPI    (export symbols)       C:\WINDOWS\system32\SHLWAPI.dll

77a50000 77b8d000   ole32      (export symbols)       C:\WINDOWS\system32\ole32.dll

77c10000 77c68000   msvcrt     (export symbols)       C:\WINDOWS\system32\msvcrt.dll

77cc0000 77d5b000   ADVAPI32   (export symbols)       C:\WINDOWS\system32\ADVAPI32.dll

77d60000 77df1000   USER32     (export symbols)       C:\WINDOWS\system32\USER32.dll

77e00000 77e93000   RPCRT4     (export symbols)       C:\WINDOWS\system32\RPCRT4.dll

77ea0000 77ee5000   GDI32      (export symbols)       C:\WINDOWS\system32\GDI32.dll

7c800000 7c8f2000   kernel32   (export symbols)       C:\WINDOWS\system32\kernel32.dll

7c900000 7c9ae000   ntdll      (export symbols)       C:\WINDOWS\system32\ntdll.dll

 

 

6. Let’s get some symbols because we can’t really work with export symbols. Set up your symbol path (either through the File->Symbol Path menu or through the .sympath command) so look something like this;

 

SRV*e:\LocalSymbolCache*http://msdl.microsoft.com/download/symbols

 

7. Reload symbols for the loaded modules by using the “.reload /f” command – this may well take a little time as the symbols for all those modules that NotePad loaded trickle down from the internet to your local cache. This gets better with time as you pull down common symbols from the net.
8. After the reload completes do another lm and make sure you got some symbols. My session now gives me;

 

0:001> lm

start    end        module name

01000000 01014000   notepad    (pdb symbols)          e:\LocalSymbols\notepad.pdb\15800B8231AF4FDE85232D42B267D3E51\notepad.pdb

4ce00000 4cf02000   COMCTL32   (pdb symbols)          e:\LocalSymbols\MicrosoftWindowsCommon-Controls-6.0.2600.2149-comctl32.pdb\4133208CAF9A4B93982A1F873FBC79261\MicrosoftWindowsCommon-Controls-6.0.2600.2149-comctl32.pdb

5ad70000 5ada7000   uxtheme    (pdb symbols)          e:\LocalSymbols\uxtheme.pdb\3A4D37B97BA34837AC589BDB15CC265D2\uxtheme.pdb

605d0000 605d9000   mslbui     (pdb symbols)          e:\LocalSymbols\MSLBUI.pdb\ED5CCEFBD83C4F929AAB9A1029C54DDD1\MSLBUI.pdb

61220000 61232000   MSH_ZWF    (export symbols)       C:\Program Files\Microsoft Hardware\Mouse\MSH_ZWF.dll

73000000 73026000   WINSPOOL   (pdb symbols)          e:\LocalSymbols\winspool.pdb\20EE1501116B4C4180ABE40B209DEC962\winspool.pdb

74720000 7476b000   MSCTF      (pdb symbols)          e:\LocalSymbols\msctf.pdb\F8F1C6E0E1CC4D1CB4936C12A6A6D4C92\msctf.pdb

763b0000 763f9000   comdlg32   (pdb symbols)          e:\LocalSymbols\comdlg32.pdb\2BC57F8BE63A429FB4B439EA2854387A2\comdlg32.pdb

77120000 771ac000   OLEAUT32   (pdb symbols)          e:\LocalSymbols\oleaut32.pdb\96D8A9A6CB704201A365F345B2A9F72D2\oleaut32.pdb

771b0000 779c4000   SHELL32    (pdb symbols)          e:\LocalSymbols\shell32.pdb\FE0DC899DD78422AB29561FE8255E1C92\shell32.pdb

779d0000 77a46000   SHLWAPI    (pdb symbols)          e:\LocalSymbols\shlwapi.pdb\231AB6F0A19F4DBB9837440BCC167BEC2\shlwapi.pdb

77a50000 77b8d000   ole32      (pdb symbols)          e:\LocalSymbols\ole32.pdb\656B303C367B4E679CA249E32CE572D92\ole32.pdb

77c10000 77c68000   msvcrt     (pdb symbols)          e:\LocalSymbols\msvcrt.pdb\8E17C9B9A55C4118A7D35025C5BE51871\msvcrt.pdb

77cc0000 77d5b000   ADVAPI32   (pdb symbols)          e:\LocalSymbols\advapi32.pdb\57E2A801C610451AA66B7CD94F0910F72\advapi32.pdb

77d60000 77df1000   USER32     (pdb symbols)          e:\LocalSymbols\user32.pdb\35948255FAC04CA99F5CF61959B45DC42\user32.pdb

77e00000 77e93000   RPCRT4     (pdb symbols)          e:\LocalSymbols\rpcrt4.pdb\79C7047CB77B435DB01EB365C3AF17152\rpcrt4.pdb

77ea0000 77ee5000   GDI32      (pdb symbols)          e:\LocalSymbols\gdi32.pdb\E7D5B033EFE5466E97DB048BE4C88FA62\gdi32.pdb

7c800000 7c8f2000   kernel32   (pdb symbols)          e:\LocalSymbols\kernel32.pdb\5091FA1742544F2EA3D425642634F78E2\kernel32.pdb

7c900000 7c9ae000   ntdll      (pdb symbols)          e:\LocalSymbols\ntdll.pdb\7B2A2EF0E63F4848A8D309E84E47322C2\ntdll.pdb

 

9. So, that’s modules and symbols. Let’s see if we can set some breakpoints. Suppose that I want to hit a breakpoint anytime that NotePad opens up a file. Let’s do that. Firstly, take a look at symbols in KERNEL32 to see if we can find the CreateFile function that NotePad uses.
10. Issue a “x KERNEL32!CreateF*” and see what results we get back you should find the ASCIICreateFileA and the Unicode CreateFileW functions in there.
11. Set a breakpoint on a function. Issue a “bp kernel32!CreateFileW” to set a breakpoint on that function.
12. Issue a “GO” command “g” to continue the debuggee running.
13. Note that when the debuggee is running you can always get back to the debugger by issuing aCTRL+BREAK to the debugger window when it will try and halt the debuggee for you and drop you back to the command prompt.
14. Open a file with notepad. You’ll spot NotePad loading some more modules in order to do this operation and get that dialog onto the screen. Your breakpoint will more than likely hit before the file dialog comes up.
15. Check-out where you are with your breakpoint – i.e. what the stack frame looks like. Issue a “kb 200”command. You should get results as below;

 

0:000> kb

ChildEBP RetAddr  Args to Child             

0006be18 76671b2a 76671b74 00000020 00000003 kernel32!CreateFileW

0006be94 766724e2 000a6b40 0006bf4c 00000000 cscui!IsCSCEnabled+0x38

0006bea8 77a68b49 000a7084 77a51a60 0006bf44 cscui!DllGetClassObject+0x72

0006bec4 77a80f5e 000a7084 77a51a60 0006bf44 ole32!CClassCache::CDllPathEntry::DllGetClassObject+0x2d

0006bedc 77a80e9a 0006bef0 77a51a60 0006bf44 ole32!CClassCache::CDllFnPtrMoniker::BindToObjectNoSwitch+0x1f

0006bf08 77a81cc6 0006bf4c 00000000 0006c540 ole32!CClassCache::GetClassObject+0x38

0006bf84 77a806aa 77b76ca4 00000000 0006c540 ole32!CServerContextActivator::CreateInstance+0x106

0006bfc4 77a81e19 0006c540 00000000 0006ca8c ole32!ActivationPropertiesIn::DelegateCreateInstance+0xf7

0006c018 77a81d90 77b76ca8 00000000 0006c540 ole32!CApartmentActivator::CreateInstance+0x110

0006c038 77a8101e 77b76ca8 00000001 00000000 ole32!CProcessActivator::CCICallback+0x6d

0006c058 77a80fd5 77b76ca0 0006c39c 00000000 ole32!CProcessActivator::AttemptActivation+0x2c

0006c090 77a81e7a 77b76ca0 0006c39c 00000000 ole32!CProcessActivator::ActivateByContext+0x42

0006c0b8 77a806aa 77b76ca0 00000000 0006c540 ole32!CProcessActivator::CreateInstance+0x49

0006c0f8 77a81bc4 0006c540 00000000 0006ca8c ole32!ActivationPropertiesIn::DelegateCreateInstance+0xf7

0006c348 77a806aa 77b765d4 00000000 0006c540 ole32!CClientContextActivator::CreateInstance+0x8f

0006c388 77a805dc 0006c540 00000000 0006ca8c ole32!ActivationPropertiesIn::DelegateCreateInstance+0xf7

0006cb38 77a64eb1 000a2f08 00000000 00000001 ole32!ICoCreateInstanceEx+0x3c9

0006cb60 77a64e80 000a2f08 00000000 00000001 ole32!CComActivator::DoCreateInstance+0x28

0006cb84 77a65102 000a2f08 00000000 00000001 ole32!CoCreateInstanceEx+0x1e

0006cbb4 779d69a5 000a2f08 00000000 00000001 ole32!CoCreateInstance+0x37

 

16. So, we can see the direct call-stack that led to our call to CreateFileW. If we had full symbols for these modules we could use a KP rather than a KB command and we’d get a list of parameters to the functions. As it is with what we have here we’d need to resort to disassembly to get the parameter information here and it’s beyond this posting.
17. Continue the debuggee by issuing a “g” command and continue issuing “g” commands until NotePad finally puts its dialog on the screen – note how many times we hit this function here J If you’re interested, issue a few KB’s on some of these call chains and see what exactly it is that Notepad is doing.
18. Break the debuggee by hitting CTRL+BREAK on the WinDbg window.
19. Issue a “~” command to get a picture of the threads in the process. In my session I get 3 as below;

 

0:002> ~

   0  Id: 9bc.b58 Suspend: 1 Teb: 7ffdf000 Unfrozen

   1  Id: 9bc.8a4 Suspend: 1 Teb: 7ffde000 Unfrozen

.  2  Id: 9bc.d4 Suspend: 1 Teb: 7ffdd000 Unfrozen

 

20. Note that this display indicates that the debugger is focused on thread “2” which is really OS thread 8A4 in process 9BC (all hex I’m afraid).
21. Check out where all the threads are by issuing a “~* kb 200”. You might not get anything too exciting here but it’s illustrative of how to use the technique.
22. Finally, close off this session by having a look at some extension commands.
    1. List the set of extensions that you’ve got by issuing the “.chain” command.
    2. Have a look at the commands available in these extensions by issuing: !exts.help, !uext.help,  !ntdsexts.help,  !ext.help and see what commands are available to you.
    3. Try an extension command. For example, “!exts.cs” will display all the Critical Sections that the process has and their status as to whether they are locked or not. This can be a useful command in many server-side scenarios.
23. End the debugging session with a “q” command.

 

That’s it. I’ll post again in the near future around how you can build on what I’ve just walked through here in order to debug managed code with WinDbg and take advantage of features within the debugger that don’t (yet) appear elsewhere.