2005/06/14
2005/06/12
Advanced Debugging -- Prologue and Epilog
Compile the code with 'cl /ZI main.cpp'
or by pressing F5.
Press F10 to begin debugging.
Press F9 to on fx() to set breakpoint.
Press F5 to run till the breakpoint.
Press A-8 to switch to Assembly mode.
You are now in the prologue of the function:
The frames form a linked list on the stack, with ebp pointing to
the previous frame above it. Local variables are created on the
stack, usually with a stack guard and a signature fence to detect
stack overruns (have you ever called sprintf on a locally declared string
or declared a local array inside a function?).
VC++ is very good at finding stack thrashing in debug mode.
Press F10 to single step through the frame setup (prologue)
till the end of the function (epilog), where the registers are
restored:
In C (cdecl) functions, the caller cleans up the stack. That is, the caller
will pop out the parameters after the call.
This allows varargs (printf( fmt, i,...)), the callee
doesn't have to know how many parameters are
on the stack.
The default return type of a C function is int,
because the return value comes for free in register eax.
In VC, you can watch '@eax' to see the implicit
return values.
Earlier languages like Pascal and lisp from 1970s
with heavy emphasis on tail recursion optimization
let the callee clean the stack, so the stack would not
grow (but was reused) in recursive calls.
To be continued.
or by pressing F5.
Press F10 to begin debugging.
Press F9 to on fx() to set breakpoint.
Press F5 to run till the breakpoint.
Press A-8 to switch to Assembly mode.
You are now in the prologue of the function:
int fx(int a, int b,int c){
-> push ebp // save the base pointer
mov ebp,esp // point ebp to stack frame.
sub esp,44h // make space for new frame.
push ebx // save original registers on stack.
push esi
push edi
The frames form a linked list on the stack, with ebp pointing to
the previous frame above it. Local variables are created on the
stack, usually with a stack guard and a signature fence to detect
stack overruns (have you ever called sprintf on a locally declared string
or declared a local array inside a function?).
VC++ is very good at finding stack thrashing in debug mode.
Press F10 to single step through the frame setup (prologue)
till the end of the function (epilog), where the registers are
restored:
pop edi
pop esi
pop ebx
add esp,44h // destroy the frame.
pop ebp // restore the frame pointer.
ret
In C (cdecl) functions, the caller cleans up the stack. That is, the caller
will pop out the parameters after the call.
This allows varargs (printf( fmt, i,...)), the callee
doesn't have to know how many parameters are
on the stack.
The default return type of a C function is int,
because the return value comes for free in register eax.
In VC, you can watch '@eax' to see the implicit
return values.
Earlier languages like Pascal and lisp from 1970s
with heavy emphasis on tail recursion optimization
let the callee clean the stack, so the stack would not
grow (but was reused) in recursive calls.
To be continued.
2005/06/11
Advanced Windows Debugging -- 1
This is a short tutorial for debugging C/C++ code on
Windows with VC++ (and windbg,ntsd,cdb) on x86.
You can download the free debugger windbg from msdn.
It is more powerful than VC++ in terms of scripting,
but less convinient GUI.
We will not cover gdb(linux), dbx(solaris), wdb(hp),
in this article, they also offer equivalent functionality.
If you use .Net or C# or Java or VB, this tutorial is not
for you.
The source was converted to html for the blog, by vim :TOHtml
> cat main.c
When you have a cryptic error message, you can look at the
preprocessor output with:
> cl /E main.c | vim -
Now let's examine the assembly listing main.cod generated with:
> cl.exe /Fc main.c
Now is a good time to brush up your x86 assembly,
a good reference is "Art of Assembly" by Randall Hyde at UC Riverside.
> vim main.cod
Start with compiling your C program with Debugging
information on. In the next posting we will dissect and
debug it.
The /ZI switch creates main.pdb (program database).
To be continued.
Windows with VC++ (and windbg,ntsd,cdb) on x86.
You can download the free debugger windbg from msdn.
It is more powerful than VC++ in terms of scripting,
but less convinient GUI.
We will not cover gdb(linux), dbx(solaris), wdb(hp),
in this article, they also offer equivalent functionality.
If you use .Net or C# or Java or VB, this tutorial is not
for you.
The source was converted to html for the blog, by vim :TOHtml
> cat main.c
#include <stdio.h>
extern int errno;
int fx(int a, int b,int c){
int d;
d = a + b; // stop on 5th call.
printf("a=%d,b=%d,c=%d,d=%d\n",a,b,c,d);
return d+c;
}
void bad_open(void){
FILE *f,*g;
f=fopen("/","r");
g=fopen("/dev/null","w");
printf("f=%p g=%p, errno=%d\n",f,g,errno);
}
int main(){
int a=1,b=2,c=3,i,e;
for(i=0;i<1000;i++){
if( i*i % 100 == 1 ) e = fx(a,b,c), e++, printf("fx=%d\n",e);
else a++, b++, c++;
}
bad_open();
}
When you have a cryptic error message, you can look at the
preprocessor output with:
> cl /E main.c | vim -
Now let's examine the assembly listing main.cod generated with:
> cl.exe /Fc main.c
Now is a good time to brush up your x86 assembly,
a good reference is "Art of Assembly" by Randall Hyde at UC Riverside.
> vim main.cod
Start with compiling your C program with Debugging
information on. In the next posting we will dissect and
debug it.
> cl.exe /ZI main.c
The /ZI switch creates main.pdb (program database).
To be continued.