Advanced Windows Debugging -- 1

This is a short tutorial for debugging C/C++ code on
Windows with VC++ (and windbg,ntsd,cdb) on x86.

You can download the free debugger windbg from msdn.
It is more powerful than VC++ in terms of scripting,
but less convinient GUI.

We will not cover gdb(linux), dbx(solaris), wdb(hp),
in this article, they also offer equivalent functionality.
If you use .Net or C# or Java or VB, this tutorial is not
for you.

The source was converted to html for the blog, by vim :TOHtml

> cat main.c

 #include <stdio.h>
 extern int errno;
 int fx(int a, int b,int c){
     int d;
     d = a + b; // stop on 5th call.
     printf("a=%d,b=%d,c=%d,d=%d\n",a,b,c,d);
     return d+c;
 }
 void bad_open(void){
     FILE *f,*g;
     f=fopen("/","r");
     g=fopen("/dev/null","w");
     printf("f=%p g=%p, errno=%d\n",f,g,errno);
 }
 int main(){
    int a=1,b=2,c=3,i,e;
    for(i=0;i<1000;i++){
        if( i*i % 100 == 1 ) e = fx(a,b,c), e++, printf("fx=%d\n",e);
        else a++, b++, c++;
    }
    bad_open();
 }
 

When you have a cryptic error message, you can look at the
preprocessor output with:

> cl /E main.c | vim -

Now let's examine the assembly listing main.cod generated with:

> cl.exe /Fc main.c

Now is a good time to brush up your x86 assembly,
a good reference is "Art of Assembly" by Randall Hyde at UC Riverside.

> vim main.cod

Start with compiling your C program with Debugging
information on. In the next posting we will dissect and
debug it.

> cl.exe  /ZI main.c

The /ZI switch creates main.pdb (program database).

To be continued.