* ASSERT::
* Function and Parameter Attributes::
* Backtraces::
-* gdb::
-* Debugging by Infinite Loop::
+* GDB::
+* Triple Faults::
* Modifying Bochs::
* Debugging Tips::
@end menu
points, and so on. Based on what you learn, you can then insert more
@func{printf} calls in the new, smaller region of code you suspect.
Eventually you can narrow the problem down to a single statement.
-@xref{Debugging by Infinite Loop}, for a related technique.
+@xref{Triple Faults}, for a related technique.
@node ASSERT
@section @code{ASSERT}
Assertions are useful because they can catch problems early, before
-they'd otherwise be noticed. Pintos provides the
-@code{ASSERT}, defined in @file{<debug.h>}, for assertions.
-Ideally, each function should begin with a set of
-assertions that check its arguments for validity. (Initializers for
-functions' local variables are evaluated before assertions are
+they'd otherwise be noticed. Ideally, each function should begin with a
+set of assertions that check its arguments for validity. (Initializers
+for functions' local variables are evaluated before assertions are
checked, so be careful not to assume that an argument is valid in an
initializer.) You can also sprinkle assertions throughout the body of
functions in places where you suspect things are likely to go wrong.
They are especially useful for checking loop invariants.
-When an assertion proves untrue, the kernel panics. The panic message
-should help you to find the problem. See the description of
-backtraces below for more information.
+Pintos provides the @code{ASSERT} macro, defined in @file{<debug.h>},
+for checking assertions.
+
+@defmac ASSERT (expression)
+Tests the value of @var{expression}. If it evaluates to zero (false),
+the kernel panics. The panic message includes the expression that
+failed, its file and line number, and a backtrace, which should help you
+to find the problem. @xref{Backtraces}, for more information.
+@end defmac
@node Function and Parameter Attributes
@section Function and Parameter Attributes
@file{<debug.h>}, to print a backtrace at any point in your code.
The addresses in a backtrace are listed as raw hexadecimal numbers,
-which are meaningless by themselves. You can translate them into
-function names and source file line numbers using a tool called
-@command{addr2line} (80@var{x}86) or @command{i386-elf-addr2line}
-(SPARC).
-
-The output format of @command{addr2line} is not ideal, so
-we've supplied a wrapper for it simply called @command{backtrace}.
+which are difficult to interpret. We provide a tool called
+@command{backtrace} to translate these into function names and source
+file line numbers.
Give it the name of your @file{kernel.o} as the first argument and the
hexadecimal numbers composing the backtrace (including the @samp{0x}
prefixes) as the remaining arguments. It outputs the function name
0xc010cf67 0xc0102319 0xc010325a 0x804812c 0x8048a96 0x8048ac8.
@end example
-@node gdb
-@section @command{gdb}
-
-You can run the Pintos kernel under the supervision of the
-@command{gdb} (80@var{x}86) or @command{i386-elf-gdb} (SPARC)
-debugger. First,
-start Pintos with the @option{--gdb} option, e.g.@: @command{pintos
---gdb -- run mytest}. Second, in a separate terminal, invoke @command{gdb} (or
-@command{i386-elf-gdb}) on
-@file{kernel.o}:
+@node GDB
+@section GDB
+
+You can run Pintos under the supervision of the GDB debugger.
+First, start Pintos with the @option{--gdb} option, e.g.@:
+@command{pintos --gdb -- run mytest}. Second, open a second terminal on
+the same machine and
+use @command{pintos-gdb} to invoke GDB on
+@file{kernel.o}:@footnote{@command{pintos-gdb} is a wrapper around
+@command{gdb} (80@var{x}86) or @command{i386-elf-gdb} (SPARC) that loads
+the Pintos macros at startup.}
@example
-gdb kernel.o
+pintos-gdb kernel.o
@end example
-@noindent and issue the following @command{gdb} command:
+@noindent and issue the following GDB command:
@example
target remote localhost:1234
@end example
-(If the @command{target remote} command fails, then make sure that both
-@command{gdb} and @command{pintos} are running on the same machine by
-running @command{hostname} in each terminal. If the names printed
-differ, then you need to open a new terminal for @command{gdb} on the
-machine running @command{pintos}.)
-
-Now @command{gdb} is connected to the simulator over a local
-network connection. You can now issue any normal @command{gdb}
+Now GDB is connected to the simulator over a local
+network connection. You can now issue any normal GDB
commands. If you issue the @samp{c} command, the simulated BIOS will take
control, load Pintos, and then Pintos will run in the usual way. You
-can pause the process at any point with @key{Ctrl+C}. If you want
-@command{gdb} to stop when Pintos starts running, set a breakpoint on
-@func{main} with the command @code{break main} before @samp{c}.
+can pause the process at any point with @key{Ctrl+C}.
-You can read the @command{gdb} manual by typing @code{info gdb} at a
-terminal command prompt, or you can view it in Emacs with the command
-@kbd{C-h i}. Here's a few commonly useful @command{gdb} commands:
+@menu
+* Using GDB::
+* Example GDB Session::
+* Debugging User Programs::
+* GDB FAQ::
+@end menu
-@table @code
-@item c
+@node Using GDB
+@subsection Using GDB
+
+You can read the GDB manual by typing @code{info gdb} at a
+terminal command prompt. Here's a few commonly useful GDB commands:
+
+@deffn {GDB Command} c
Continues execution until @key{Ctrl+C} or the next breakpoint.
+@end deffn
-@item break @var{function}
-@itemx break @var{filename}:@var{linenum}
-@itemx break *@var{address}
-Sets a breakpoint at the given function, line number, or address.
+@deffn {GDB Command} break function
+@deffnx {GDB Command} break file:line
+@deffnx {GDB Command} break *address
+Sets a breakpoint at @var{function}, at @var{line} within @var{file}, or
+@var{address}.
(Use a @samp{0x} prefix to specify an address in hex.)
-@item p @var{expression}
-Evaluates the given C expression and prints its value.
+Use @code{break main} to make GDB stop when Pintos starts running.
+@end deffn
+
+@deffn {GDB Command} p expression
+Evaluates the given @var{expression} and prints its value.
If the expression contains a function call, that function will actually
be executed.
+@end deffn
-@item l *@var{address}
-Lists a few lines of code around the given address.
+@deffn {GDB Command} l *address
+Lists a few lines of code around @var{address}.
(Use a @samp{0x} prefix to specify an address in hex.)
+@end deffn
-@item bt
+@deffn {GDB Command} bt
Prints a stack backtrace similar to that output by the
@command{backtrace} program described above.
+@end deffn
-@item p/a @var{address}
-Prints the name of the function or variable that occupies the given
-address.
+@deffn {GDB Command} p/a address
+Prints the name of the function or variable that occupies @var{address}.
(Use a @samp{0x} prefix to specify an address in hex.)
+@end deffn
+
+@deffn {GDB Command} diassemble function
+Disassembles @var{function}.
+@end deffn
+
+We also provide a set of macros specialized for debugging Pintos,
+written by Godmar Back @email{gback@@cs.vt.edu}. You can type
+@code{help user-defined} for basic help with the macros. Here is an
+overview of their functionality, based on Godmar's documentation:
+
+@deffn {GDB Macro} debugpintos
+Attach debugger to a waiting pintos process on the same machine.
+Shorthand for @code{target remote localhost:1234}.
+@end deffn
+
+@deffn {GDB Macro} dumplist list type element
+Prints the elements of @var{list}, which should be a @code{struct} list
+that contains elements of the given @var{type} (without the word
+@code{struct}) in which @var{element} is the @struct{list_elem} member
+that links the elements.
+
+Example: @code{dumplist all_list thread all_elem} prints all elements of
+@struct{thread} that are linked in @code{struct list all_list} using the
+@code{struct list_elem all_elem} which is part of @struct{thread}.
+@end deffn
+
+@deffn {GDB Macro} btthread thread
+Shows the backtrace of @var{thread}, which is a pointer to the
+@struct{thread} of the thread whose backtrace it should show. For the
+current thread, this is identical to the @code{bt} (backtrace) command.
+It also works for threads that are suspended in @func{schedule},
+provided you know where their kernel stack page is located.
+@end deffn
+
+@deffn {GDB Macro} btthreadlist list element
+Shows the backtraces of all threads in @var{list}, the @struct{list} in
+which the threads are kept. Specify @var{element} as the
+@struct{list_elem} field used inside @struct{thread} to link the threads
+together.
+
+Example: @code{btthreadlist all_list all_elem} shows the backtraces of
+all threads contained in @code{struct list all_list}, linked together by
+@code{all_elem}. This command is useful to determine where your threads
+are stuck when a deadlock occurs. Please see the example scenario below.
+@end deffn
+
+@deffn {GDB Macro} btpagefault
+Print a backtrace of the current thread after a page fault exception.
+Normally, when a page fault exception occurs, GDB will stop
+with a message that might say:
-@item diassemble @var{function}
-Disassembles the specified @var{function}.
-@end table
+@example
+Program received signal 0, Signal 0.
+0xc0102320 in intr0e_stub ()
+@end example
-If you notice other strange behavior while using @command{gdb}, there
-are three possibilities: a bug in your
-modified Pintos, a bug in Bochs's
-interface to @command{gdb} or in @command{gdb} itself, or
-a bug in the original Pintos code. The first and second
-are quite likely, and you should seriously consider both. We hope
-that the third is less likely, but it is also possible.
+In that case, the @code{bt} command might not give a useful
+backtrace. Use @code{btpagefault} instead.
-@menu
-* Debugging User Programs::
-@end menu
+You may also use @code{btpagefault} for page faults that occur in a user
+process. In this case, you may also wish to load the user program's
+symbol table (@pxref{Debugging User Programs}).
+@end deffn
+
+@deffn {GDB Macro} hook-stop
+GDB invokes this macro every time the simulation stops, which Bochs will
+do for every processor exception, among other reasons. If the
+simulation stops due to a page fault, @code{hook-stop} will print a
+message that says and explains further whether the page fault occurred
+in the kernel or in user code.
+
+If the exception occurred from user code, @code{hook-stop} will say:
+@example
+pintos-debug: a page fault exception occurred in user mode
+pintos-debug: hit 'c' to continue, or 's' to step to intr_handler
+@end example
+
+In Project 2, a page fault in a user process leads to the termination of
+the process. You should expect those page faults to occur in the
+robustness tests where we test that your kernel properly terminates
+processes that try to access invalid addresses. To debug those, set a
+break point in @func{page_fault} in @file{exception.c}, which you will
+need to modify accordingly.
+
+In Project 3, a page fault in a user process no longer automatically
+leads to the termination of a process. Rather, you may have to page in
+the page containing the address the process was trying to access, either
+because it was swapped out or because this is the first time it's
+accessed. In either case, you will reach @func{page_fault} and need to
+take the appropriate action there.
+
+If the page fault did not occur in user mode while executing a user
+process, then it occurred in kernel mode while executing kernel code.
+In this case, @code{hook-stop} will print this message:
+@example
+pintos-debug: a page fault occurred in kernel mode
+@end example
+followed by the output of the @code{btpagefault} command.
+
+Before Project 3, a page fault exception in kernel code is always a bug
+in your kernel, because your kernel should never crash. Starting with
+Project 3, the situation will change if you use @func{get_user} and
+@func{put_user} strategy to verify user memory accesses
+(@pxref{Accessing User Memory}).
+@end deffn
+
+@node Example GDB Session
+@subsection Example GDB Session
+
+This section narrates a sample GDB session, provided by Godmar Back.
+This example illustrates how one might debug a Project 1 solution in
+which occasionally a thread that calls @func{timer_sleep} is not woken
+up. With this bug, tests such as @code{mlfqs_load_1} get stuck.
+
+This session was captured with a slightly older version of Bochs and the
+GDB macros for Pintos, so it looks slightly different than it would now.
+Program output is shown in normal type, user input in @strong{strong}
+type.
+
+First, I start Pintos:
+
+@smallexample
+$ @strong{pintos -v --gdb -- -q -mlfqs run mlfqs-load-1}
+Writing command line to /tmp/gDAlqTB5Uf.dsk...
+bochs -q
+========================================================================
+ Bochs x86 Emulator 2.2.5
+ Build from CVS snapshot on December 30, 2005
+========================================================================
+00000000000i[ ] reading configuration from bochsrc.txt
+00000000000i[ ] Enabled gdbstub
+00000000000i[ ] installing nogui module as the Bochs GUI
+00000000000i[ ] using log file bochsout.txt
+Waiting for gdb connection on localhost:1234
+@end smallexample
+
+@noindent Then, I open a second window on the same machine and start GDB:
+
+@smallexample
+$ @strong{pintos-gdb kernel.o}
+GNU gdb Red Hat Linux (6.3.0.0-1.84rh)
+Copyright 2004 Free Software Foundation, Inc.
+GDB is free software, covered by the GNU General Public License, and you are
+welcome to change it and/or distribute copies of it under certain conditions.
+Type "show copying" to see the conditions.
+There is absolutely no warranty for GDB. Type "show warranty" for details.
+This GDB was configured as "i386-redhat-linux-gnu"...
+Using host libthread_db library "/lib/libthread_db.so.1".
+@end smallexample
+
+@noindent Then, I tell GDB to attach to the waiting Pintos emulator:
+
+@smallexample
+(gdb) @strong{debugpintos}
+Remote debugging using localhost:1234
+0x0000fff0 in ?? ()
+Reply contains invalid hex digit 78
+@end smallexample
+
+@noindent Now I tell Pintos to run by executing @code{c} (short for
+@code{continue}) twice:
+
+@smallexample
+(gdb) @strong{c}
+Continuing.
+Reply contains invalid hex digit 78
+(gdb) @strong{c}
+Continuing.
+@end smallexample
+
+@noindent Now Pintos will continue and output:
+
+@smallexample
+Pintos booting with 4,096 kB RAM...
+Kernel command line: -q -mlfqs run mlfqs-load-1
+374 pages available in kernel pool.
+373 pages available in user pool.
+Calibrating timer... 102,400 loops/s.
+Boot complete.
+Executing 'mlfqs-load-1':
+(mlfqs-load-1) begin
+(mlfqs-load-1) spinning for up to 45 seconds, please wait...
+(mlfqs-load-1) load average rose to 0.5 after 42 seconds
+(mlfqs-load-1) sleeping for another 10 seconds, please wait...
+@end smallexample
+
+@noindent
+@dots{}until it gets stuck because of the bug I had introduced. I hit
+@key{Ctrl+C} in the debugger window:
+
+@smallexample
+Program received signal 0, Signal 0.
+0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
+649 while (i <= PRI_MAX && list_empty (&ready_list[i]))
+(gdb)
+@end smallexample
+
+@noindent
+The thread that was running when I interrupted Pintos was the idle
+thread. If I run @code{backtrace}, it shows this backtrace:
+
+@smallexample
+(gdb) @strong{bt}
+#0 0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
+#1 0xc0101778 in schedule () at ../../threads/thread.c:714
+#2 0xc0100f8f in thread_block () at ../../threads/thread.c:324
+#3 0xc0101419 in idle (aux=0x0) at ../../threads/thread.c:551
+#4 0xc010145a in kernel_thread (function=0xc01013ff , aux=0x0)
+ at ../../threads/thread.c:575
+#5 0x00000000 in ?? ()
+@end smallexample
+
+@noindent
+Not terribly useful. What I really like to know is what's up with the
+other thread (or threads). Since I keep all threads in a linked list
+called @code{all_list}, linked together by a @struct{list_elem} member
+named @code{all_elem}, I can use the @code{btthreadlist} macro from the
+macro library I wrote. @code{btthreadlist} iterates through the list of
+threads and prints the backtrace for each thread:
+
+@smallexample
+(gdb) @strong{btthreadlist all_list all_elem}
+pintos-debug: dumping backtrace of thread 'main' @@0xc002f000
+#0 0xc0101820 in schedule () at ../../threads/thread.c:722
+#1 0xc0100f8f in thread_block () at ../../threads/thread.c:324
+#2 0xc0104755 in timer_sleep (ticks=1000) at ../../devices/timer.c:141
+#3 0xc010bf7c in test_mlfqs_load_1 () at ../../tests/threads/mlfqs-load-1.c:49
+#4 0xc010aabb in run_test (name=0xc0007d8c "mlfqs-load-1")
+ at ../../tests/threads/tests.c:50
+#5 0xc0100647 in run_task (argv=0xc0110d28) at ../../threads/init.c:281
+#6 0xc0100721 in run_actions (argv=0xc0110d28) at ../../threads/init.c:331
+#7 0xc01000c7 in main () at ../../threads/init.c:140
+
+pintos-debug: dumping backtrace of thread 'idle' @@0xc0116000
+#0 0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
+#1 0xc0101778 in schedule () at ../../threads/thread.c:714
+#2 0xc0100f8f in thread_block () at ../../threads/thread.c:324
+#3 0xc0101419 in idle (aux=0x0) at ../../threads/thread.c:551
+#4 0xc010145a in kernel_thread (function=0xc01013ff , aux=0x0)
+ at ../../threads/thread.c:575
+#5 0x00000000 in ?? ()
+@end smallexample
+
+@noindent
+In this case, there are only two threads, the idle thread and the main
+thread. The kernel stack pages (to which the @struct{thread} points)
+are at @t{0xc0116000} and @t{0xc002f000}, respectively. The main thread
+is stuck in @func{timer_sleep}, called from @code{test_mlfqs_load_1}.
+
+Knowing where threads are stuck can be tremendously useful, for instance
+when diagnosing deadlocks or unexplained hangs.
@node Debugging User Programs
@subsection Debugging User Programs
-You can also use @command{gdb} to debug a user program running under
-Pintos. Start by issuing this @command{gdb} command to load the
+You can also use GDB to debug a user program running under
+Pintos. Start by issuing this GDB command to load the
program's symbol table:
@example
add-symbol-file @var{program}
so be careful in interpreting the results. Also, a name that appears in
both the kernel and the user program will actually refer to the kernel
name. (The latter problem can be avoided by giving the user executable
-name on the @command{gdb} command line, instead of @file{kernel.o}.)
+name on the GDB command line, instead of @file{kernel.o}, and then using
+@code{add-symbol-file} to load @file{kernel.o}.)
+
+@node GDB FAQ
+@subsection FAQ
+
+@table @asis
+@item GDB can't connect to Bochs.
+
+If the @command{target remote} command fails, then make sure that both
+GDB and @command{pintos} are running on the same machine by
+running @command{hostname} in each terminal. If the names printed
+differ, then you need to open a new terminal for GDB on the
+machine running @command{pintos}.
+
+@item GDB doesn't recognize any of the macros.
+
+If you start GDB with @command{pintos-gdb}, it should load the Pintos
+macros automatically. If you start GDB some other way, then you must
+issue the command @code{source @var{pintosdir}/src/misc/gdb-macros},
+where @var{pintosdir} is the root of your Pintos directory, before you
+can use them.
-@node Debugging by Infinite Loop
-@section Debugging by Infinite Loop
+@item Can I debug Pintos with DDD?
-If you get yourself into a situation where the machine reboots in a
-loop, that's probably a ``triple fault.'' In such a situation you
-might not be able to use @func{printf} for debugging, because the
-reboots might be happening even before everything needed for
-@func{printf} is initialized. In such a situation, you might want to
-try what I call ``debugging by infinite loop.''
+Yes, you can. DDD invokes GDB as a subprocess, so you'll need to tell
+it to invokes @command{pintos-gdb} instead:
+@example
+ddd --gdb --debugger pintos-gdb
+@end example
+
+@item Can I use GDB inside Emacs?
+
+Yes, you can. Emacs has special support for running GDB as a
+subprocess. Type @kbd{M-x gdb} and enter your @command{pintos-gdb}
+command at the prompt. The Emacs manual has information on how to use
+its debugging features in a section titled ``Debuggers.''
+
+@item GDB is doing something weird.
+
+If you notice strange behavior while using GDB, there
+are three possibilities: a bug in your
+modified Pintos, a bug in Bochs's
+interface to GDB or in GDB itself, or
+a bug in the original Pintos code. The first and second
+are quite likely, and you should seriously consider both. We hope
+that the third is less likely, but it is also possible.
+@end table
-What you do is pick a place in the Pintos code, insert the statement
+@node Triple Faults
+@section Triple Faults
+
+When a CPU exception handler, such as a page fault handler, cannot be
+invoked because it is missing or defective, the CPU will try to invoke
+the ``double fault'' handler. If the double fault handler is itself
+missing or defective, that's called a ``triple fault.'' A triple fault
+causes an immediate CPU reset.
+
+Thus, if you get yourself into a situation where the machine reboots in
+a loop, that's probably a ``triple fault.'' In a triple fault
+situation, you might not be able to use @func{printf} for debugging,
+because the reboots might be happening even before everything needed for
+@func{printf} is initialized.
+
+There are at least two ways to debug triple faults. First, you can run
+Pintos in Bochs under GDB (@pxref{GDB}). If Bochs has been built
+properly for Pintos, a triple fault under GDB will cause it to print the
+message ``Triple fault: stopping for gdb'' on the console and break into
+the debugger. (If Bochs is not running under GDB, a triple fault will
+still cause it to reboot.) You can then inspect where Pintos stopped,
+which is where the triple fault occurred.
+
+Another option is what I call ``debugging by infinite loop.''
+Pick a place in the Pintos code, insert the infinite loop
@code{for (;;);} there, and recompile and run. There are two likely
possibilities:
In a case like this, you might appreciate being able to make Bochs
print out more debug information, such as the exact type of fault that
occurred. It's not very hard. You start by retrieving the source
-code for Bochs 2.1.1 from @uref{http://bochs.sourceforge.net} and
+code for Bochs 2.2.6 from @uref{http://bochs.sourceforge.net} and
extracting it into a directory. Then read
-@file{pintos/src/misc/bochs-2.1.1.patch} and apply the patches needed.
+@file{pintos/src/misc/bochs-2.2.6.README} and apply the patches needed.
Then run @file{./configure}, supplying the options you want (some
suggestions are in the patch file). Finally, run @command{make}.
This will compile Bochs and eventually produce a new binary
projects / pintos-anon / commitdiff