2 @appendix Debugging Tools
4 Many tools lie at your disposal for debugging Pintos. This appendix
5 introduces you to a few of them.
10 * Function and Parameter Attributes::
19 @section @code{printf()}
21 Don't underestimate the value of @func{printf}. The way
22 @func{printf} is implemented in Pintos, you can call it from
23 practically anywhere in the kernel, whether it's in a kernel thread or
24 an interrupt handler, almost regardless of what locks are held.
26 @func{printf} is useful for more than just examining data.
27 It can also help figure out when and where something goes wrong, even
28 when the kernel crashes or panics without a useful error message. The
29 strategy is to sprinkle calls to @func{printf} with different strings
30 (e.g.@: @code{"<1>"}, @code{"<2>"}, @dots{}) throughout the pieces of
31 code you suspect are failing. If you don't even see @code{<1>} printed,
32 then something bad happened before that point, if you see @code{<1>}
33 but not @code{<2>}, then something bad happened between those two
34 points, and so on. Based on what you learn, you can then insert more
35 @func{printf} calls in the new, smaller region of code you suspect.
36 Eventually you can narrow the problem down to a single statement.
37 @xref{Triple Faults}, for a related technique.
40 @section @code{ASSERT}
42 Assertions are useful because they can catch problems early, before
43 they'd otherwise be noticed. Ideally, each function should begin with a
44 set of assertions that check its arguments for validity. (Initializers
45 for functions' local variables are evaluated before assertions are
46 checked, so be careful not to assume that an argument is valid in an
47 initializer.) You can also sprinkle assertions throughout the body of
48 functions in places where you suspect things are likely to go wrong.
49 They are especially useful for checking loop invariants.
51 Pintos provides the @code{ASSERT} macro, defined in @file{<debug.h>},
52 for checking assertions.
54 @defmac ASSERT (expression)
55 Tests the value of @var{expression}. If it evaluates to zero (false),
56 the kernel panics. The panic message includes the expression that
57 failed, its file and line number, and a backtrace, which should help you
58 to find the problem. @xref{Backtraces}, for more information.
61 @node Function and Parameter Attributes
62 @section Function and Parameter Attributes
64 These macros defined in @file{<debug.h>} tell the compiler special
65 attributes of a function or function parameter. Their expansions are
69 Appended to a function parameter to tell the compiler that the
70 parameter might not be used within the function. It suppresses the
71 warning that would otherwise appear.
75 Appended to a function prototype to tell the compiler that the
76 function never returns. It allows the compiler to fine-tune its
77 warnings and its code generation.
81 Appended to a function prototype to tell the compiler to never emit
82 the function in-line. Occasionally useful to improve the quality of
83 backtraces (see below).
86 @defmac PRINTF_FORMAT (@var{format}, @var{first})
87 Appended to a function prototype to tell the compiler that the function
88 takes a @func{printf}-like format string as the argument numbered
89 @var{format} (starting from 1) and that the corresponding value
90 arguments start at the argument numbered @var{first}. This lets the
91 compiler tell you if you pass the wrong argument types.
97 When the kernel panics, it prints a ``backtrace,'' that is, a summary
98 of how your program got where it is, as a list of addresses inside the
99 functions that were running at the time of the panic. You can also
100 insert a call to @func{debug_backtrace}, prototyped in
101 @file{<debug.h>}, to print a backtrace at any point in your code.
102 @func{debug_backtrace_all}, also declared in @file{<debug.h>},
103 prints backtraces of all threads.
105 The addresses in a backtrace are listed as raw hexadecimal numbers,
106 which are difficult to interpret. We provide a tool called
107 @command{backtrace} to translate these into function names and source
109 Give it the name of your @file{kernel.o} as the first argument and the
110 hexadecimal numbers composing the backtrace (including the @samp{0x}
111 prefixes) as the remaining arguments. It outputs the function name
112 and source file line numbers that correspond to each address.
114 If the translated form of a backtrace is garbled, or doesn't make
115 sense (e.g.@: function A is listed above function B, but B doesn't
116 call A), then it's a good sign that you're corrupting a kernel
117 thread's stack, because the backtrace is extracted from the stack.
118 Alternatively, it could be that the @file{kernel.o} you passed to
119 @command{backtrace} is not the same kernel that produced
122 Sometimes backtraces can be confusing without any corruption.
123 Compiler optimizations can cause surprising behavior. When a function
124 has called another function as its final action (a @dfn{tail call}), the
125 calling function may not appear in a backtrace at all. Similarly, when
126 function A calls another function B that never returns, the compiler may
127 optimize such that an unrelated function C appears in the backtrace
128 instead of A. Function C is simply the function that happens to be in
129 memory just after A. In the threads project, this is commonly seen in
130 backtraces for test failures; see @ref{The pass function fails, ,
131 @func{pass} Fails}, for more information.
134 * Backtrace Example::
137 @node Backtrace Example
140 Here's an example. Suppose that Pintos printed out this following call
141 stack, which is taken from an actual Pintos submission for the file
145 Call stack: 0xc0106eff 0xc01102fb 0xc010dc22 0xc010cf67 0xc0102319
146 0xc010325a 0x804812c 0x8048a96 0x8048ac8.
149 You would then invoke the @command{backtrace} utility like shown below,
150 cutting and pasting the backtrace information into the command line.
151 This assumes that @file{kernel.o} is in the current directory. You
152 would of course enter all of the following on a single shell command
153 line, even though that would overflow our margins here:
156 backtrace kernel.o 0xc0106eff 0xc01102fb 0xc010dc22 0xc010cf67
157 0xc0102319 0xc010325a 0x804812c 0x8048a96 0x8048ac8
160 The backtrace output would then look something like this:
163 0xc0106eff: debug_panic (lib/debug.c:86)
164 0xc01102fb: file_seek (filesys/file.c:405)
165 0xc010dc22: seek (userprog/syscall.c:744)
166 0xc010cf67: syscall_handler (userprog/syscall.c:444)
167 0xc0102319: intr_handler (threads/interrupt.c:334)
168 0xc010325a: intr_entry (threads/intr-stubs.S:38)
169 0x0804812c: (unknown)
170 0x08048a96: (unknown)
171 0x08048ac8: (unknown)
174 (You will probably not see exactly the same addresses if you run the
175 command above on your own kernel binary, because the source code you
176 compiled and the compiler you used are probably different.)
178 The first line in the backtrace refers to @func{debug_panic}, the
179 function that implements kernel panics. Because backtraces commonly
180 result from kernel panics, @func{debug_panic} will often be the first
181 function shown in a backtrace.
183 The second line shows @func{file_seek} as the function that panicked,
184 in this case as the result of an assertion failure. In the source code
185 tree used for this example, line 405 of @file{filesys/file.c} is the
189 ASSERT (file_ofs >= 0);
193 (This line was also cited in the assertion failure message.)
194 Thus, @func{file_seek} panicked because it passed a negative file offset
197 The third line indicates that @func{seek} called @func{file_seek},
198 presumably without validating the offset argument. In this submission,
199 @func{seek} implements the @code{seek} system call.
201 The fourth line shows that @func{syscall_handler}, the system call
202 handler, invoked @func{seek}.
204 The fifth and sixth lines are the interrupt handler entry path.
206 The remaining lines are for addresses below @code{PHYS_BASE}. This
207 means that they refer to addresses in the user program, not in the
208 kernel. If you know what user program was running when the kernel
209 panicked, you can re-run @command{backtrace} on the user program, like
210 so: (typing the command on a single line, of course):
213 backtrace tests/filesys/extended/grow-too-big 0xc0106eff 0xc01102fb
214 0xc010dc22 0xc010cf67 0xc0102319 0xc010325a 0x804812c 0x8048a96
218 The results look like this:
221 0xc0106eff: (unknown)
222 0xc01102fb: (unknown)
223 0xc010dc22: (unknown)
224 0xc010cf67: (unknown)
225 0xc0102319: (unknown)
226 0xc010325a: (unknown)
227 0x0804812c: test_main (...xtended/grow-too-big.c:20)
228 0x08048a96: main (tests/main.c:10)
229 0x08048ac8: _start (lib/user/entry.c:9)
232 You can even specify both the kernel and the user program names on
233 the command line, like so:
236 backtrace kernel.o tests/filesys/extended/grow-too-big 0xc0106eff
237 0xc01102fb 0xc010dc22 0xc010cf67 0xc0102319 0xc010325a 0x804812c
241 The result is a combined backtrace:
245 0xc0106eff: debug_panic (lib/debug.c:86)
246 0xc01102fb: file_seek (filesys/file.c:405)
247 0xc010dc22: seek (userprog/syscall.c:744)
248 0xc010cf67: syscall_handler (userprog/syscall.c:444)
249 0xc0102319: intr_handler (threads/interrupt.c:334)
250 0xc010325a: intr_entry (threads/intr-stubs.S:38)
251 In tests/filesys/extended/grow-too-big:
252 0x0804812c: test_main (...xtended/grow-too-big.c:20)
253 0x08048a96: main (tests/main.c:10)
254 0x08048ac8: _start (lib/user/entry.c:9)
257 Here's an extra tip for anyone who read this far: @command{backtrace}
258 is smart enough to strip the @code{Call stack:} header and @samp{.}
259 trailer from the command line if you include them. This can save you
260 a little bit of trouble in cutting and pasting. Thus, the following
261 command prints the same output as the first one we used:
264 backtrace kernel.o Call stack: 0xc0106eff 0xc01102fb 0xc010dc22
265 0xc010cf67 0xc0102319 0xc010325a 0x804812c 0x8048a96 0x8048ac8.
271 You can run Pintos under the supervision of the GDB debugger.
272 First, start Pintos with the @option{--gdb} option, e.g.@:
273 @command{pintos --gdb -- run mytest}. Second, open a second terminal on
275 use @command{pintos-gdb} to invoke GDB on
276 @file{kernel.o}:@footnote{@command{pintos-gdb} is a wrapper around
277 @command{gdb} (80@var{x}86) or @command{i386-elf-gdb} (SPARC) that loads
278 the Pintos macros at startup.}
282 @noindent and issue the following GDB command:
284 target remote localhost:1234
287 Now GDB is connected to the simulator over a local
288 network connection. You can now issue any normal GDB
289 commands. If you issue the @samp{c} command, the simulated BIOS will take
290 control, load Pintos, and then Pintos will run in the usual way. You
291 can pause the process at any point with @key{Ctrl+C}.
295 * Example GDB Session::
300 @subsection Using GDB
302 You can read the GDB manual by typing @code{info gdb} at a
303 terminal command prompt. Here's a few commonly useful GDB commands:
305 @deffn {GDB Command} c
306 Continues execution until @key{Ctrl+C} or the next breakpoint.
309 @deffn {GDB Command} break function
310 @deffnx {GDB Command} break file:line
311 @deffnx {GDB Command} break *address
312 Sets a breakpoint at @var{function}, at @var{line} within @var{file}, or
314 (Use a @samp{0x} prefix to specify an address in hex.)
316 Use @code{break main} to make GDB stop when Pintos starts running.
319 @deffn {GDB Command} p expression
320 Evaluates the given @var{expression} and prints its value.
321 If the expression contains a function call, that function will actually
325 @deffn {GDB Command} l *address
326 Lists a few lines of code around @var{address}.
327 (Use a @samp{0x} prefix to specify an address in hex.)
330 @deffn {GDB Command} bt
331 Prints a stack backtrace similar to that output by the
332 @command{backtrace} program described above.
335 @deffn {GDB Command} p/a address
336 Prints the name of the function or variable that occupies @var{address}.
337 (Use a @samp{0x} prefix to specify an address in hex.)
340 @deffn {GDB Command} diassemble function
341 Disassembles @var{function}.
344 We also provide a set of macros specialized for debugging Pintos,
345 written by Godmar Back @email{gback@@cs.vt.edu}. You can type
346 @code{help user-defined} for basic help with the macros. Here is an
347 overview of their functionality, based on Godmar's documentation:
349 @deffn {GDB Macro} debugpintos
350 Attach debugger to a waiting pintos process on the same machine.
351 Shorthand for @code{target remote localhost:1234}.
354 @deffn {GDB Macro} dumplist list type element
355 Prints the elements of @var{list}, which should be a @code{struct} list
356 that contains elements of the given @var{type} (without the word
357 @code{struct}) in which @var{element} is the @struct{list_elem} member
358 that links the elements.
360 Example: @code{dumplist all_list thread allelem} prints all elements of
361 @struct{thread} that are linked in @code{struct list all_list} using the
362 @code{struct list_elem allelem} which is part of @struct{thread}.
365 @deffn {GDB Macro} btthread thread
366 Shows the backtrace of @var{thread}, which is a pointer to the
367 @struct{thread} of the thread whose backtrace it should show. For the
368 current thread, this is identical to the @code{bt} (backtrace) command.
369 It also works for any thread suspended in @func{schedule},
370 provided you know where its kernel stack page is located.
373 @deffn {GDB Macro} btthreadlist list element
374 Shows the backtraces of all threads in @var{list}, the @struct{list} in
375 which the threads are kept. Specify @var{element} as the
376 @struct{list_elem} field used inside @struct{thread} to link the threads
379 Example: @code{btthreadlist all_list allelem} shows the backtraces of
380 all threads contained in @code{struct list all_list}, linked together by
381 @code{allelem}. This command is useful to determine where your threads
382 are stuck when a deadlock occurs. Please see the example scenario below.
385 @deffn {GDB Macro} btthreadall
386 Short-hand for @code{btthreadlist all_list allelem}.
389 @deffn {GDB Macro} btpagefault
390 Print a backtrace of the current thread after a page fault exception.
391 Normally, when a page fault exception occurs, GDB will stop
392 with a message that might say:@footnote{To be precise, GDB will stop
393 only when running under Bochs. When running under QEMU, you must
394 set a breakpoint in the @code{page_fault} function to stop execution
395 when a page fault occurs. In that case, the @code{btpagefault} macro is
399 Program received signal 0, Signal 0.
400 0xc0102320 in intr0e_stub ()
403 In that case, the @code{bt} command might not give a useful
404 backtrace. Use @code{btpagefault} instead.
406 You may also use @code{btpagefault} for page faults that occur in a user
407 process. In this case, you may wish to also load the user program's
408 symbol table using the @code{loadusersymbols} macro, as described above.
411 @deffn {GDB Macro} hook-stop
412 GDB invokes this macro every time the simulation stops, which Bochs will
413 do for every processor exception, among other reasons. If the
414 simulation stops due to a page fault, @code{hook-stop} will print a
415 message that says and explains further whether the page fault occurred
416 in the kernel or in user code.
418 If the exception occurred from user code, @code{hook-stop} will say:
420 pintos-debug: a page fault exception occurred in user mode
421 pintos-debug: hit 'c' to continue, or 's' to step to intr_handler
424 In Project 2, a page fault in a user process leads to the termination of
425 the process. You should expect those page faults to occur in the
426 robustness tests where we test that your kernel properly terminates
427 processes that try to access invalid addresses. To debug those, set a
428 break point in @func{page_fault} in @file{exception.c}, which you will
429 need to modify accordingly.
431 In Project 3, a page fault in a user process no longer automatically
432 leads to the termination of a process. Instead, it may require reading in
433 data for the page the process was trying to access, either
434 because it was swapped out or because this is the first time it's
435 accessed. In either case, you will reach @func{page_fault} and need to
436 take the appropriate action there.
438 If the page fault did not occur in user mode while executing a user
439 process, then it occurred in kernel mode while executing kernel code.
440 In this case, @code{hook-stop} will print this message:
442 pintos-debug: a page fault occurred in kernel mode
444 followed by the output of the @code{btpagefault} command.
446 Before Project 3, a page fault exception in kernel code is always a bug
447 in your kernel, because your kernel should never crash. Starting with
448 Project 3, the situation will change if you use the @func{get_user} and
449 @func{put_user} strategy to verify user memory accesses
450 (@pxref{Accessing User Memory}).
453 @c Unfortunately, this does not work with Bochs's gdb stub.
455 @c If you don't want GDB to stop for page faults, then issue the command
456 @c @code{handle SIGSEGV nostop}. GDB will still print a message for
457 @c every page fault, but it will not come back to a command prompt.
460 @node Example GDB Session
461 @subsection Example GDB Session
463 This section narrates a sample GDB session, provided by Godmar Back.
464 This example illustrates how one might debug a Project 1 solution in
465 which occasionally a thread that calls @func{timer_sleep} is not woken
466 up. With this bug, tests such as @code{mlfqs_load_1} get stuck.
468 This session was captured with a slightly older version of Bochs and the
469 GDB macros for Pintos, so it looks slightly different than it would now.
470 Program output is shown in normal type, user input in @strong{strong}
473 First, I start Pintos:
476 $ @strong{pintos -v --gdb -- -q -mlfqs run mlfqs-load-1}
477 Writing command line to /tmp/gDAlqTB5Uf.dsk...
479 ========================================================================
480 Bochs x86 Emulator 2.2.5
481 Build from CVS snapshot on December 30, 2005
482 ========================================================================
483 00000000000i[ ] reading configuration from bochsrc.txt
484 00000000000i[ ] Enabled gdbstub
485 00000000000i[ ] installing nogui module as the Bochs GUI
486 00000000000i[ ] using log file bochsout.txt
487 Waiting for gdb connection on localhost:1234
490 @noindent Then, I open a second window on the same machine and start GDB:
493 $ @strong{pintos-gdb kernel.o}
494 GNU gdb Red Hat Linux (6.3.0.0-1.84rh)
495 Copyright 2004 Free Software Foundation, Inc.
496 GDB is free software, covered by the GNU General Public License, and you are
497 welcome to change it and/or distribute copies of it under certain conditions.
498 Type "show copying" to see the conditions.
499 There is absolutely no warranty for GDB. Type "show warranty" for details.
500 This GDB was configured as "i386-redhat-linux-gnu"...
501 Using host libthread_db library "/lib/libthread_db.so.1".
504 @noindent Then, I tell GDB to attach to the waiting Pintos emulator:
507 (gdb) @strong{debugpintos}
508 Remote debugging using localhost:1234
510 Reply contains invalid hex digit 78
513 @noindent Now I tell Pintos to run by executing @code{c} (short for
514 @code{continue}) twice:
519 Reply contains invalid hex digit 78
524 @noindent Now Pintos will continue and output:
527 Pintos booting with 4,096 kB RAM...
528 Kernel command line: -q -mlfqs run mlfqs-load-1
529 374 pages available in kernel pool.
530 373 pages available in user pool.
531 Calibrating timer... 102,400 loops/s.
533 Executing 'mlfqs-load-1':
535 (mlfqs-load-1) spinning for up to 45 seconds, please wait...
536 (mlfqs-load-1) load average rose to 0.5 after 42 seconds
537 (mlfqs-load-1) sleeping for another 10 seconds, please wait...
541 @dots{}until it gets stuck because of the bug I had introduced. I hit
542 @key{Ctrl+C} in the debugger window:
545 Program received signal 0, Signal 0.
546 0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
547 649 while (i <= PRI_MAX && list_empty (&ready_list[i]))
552 The thread that was running when I interrupted Pintos was the idle
553 thread. If I run @code{backtrace}, it shows this backtrace:
557 #0 0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
558 #1 0xc0101778 in schedule () at ../../threads/thread.c:714
559 #2 0xc0100f8f in thread_block () at ../../threads/thread.c:324
560 #3 0xc0101419 in idle (aux=0x0) at ../../threads/thread.c:551
561 #4 0xc010145a in kernel_thread (function=0xc01013ff , aux=0x0)
562 at ../../threads/thread.c:575
563 #5 0x00000000 in ?? ()
567 Not terribly useful. What I really like to know is what's up with the
568 other thread (or threads). Since I keep all threads in a linked list
569 called @code{all_list}, linked together by a @struct{list_elem} member
570 named @code{allelem}, I can use the @code{btthreadlist} macro from the
571 macro library I wrote. @code{btthreadlist} iterates through the list of
572 threads and prints the backtrace for each thread:
575 (gdb) @strong{btthreadlist all_list allelem}
576 pintos-debug: dumping backtrace of thread 'main' @@0xc002f000
577 #0 0xc0101820 in schedule () at ../../threads/thread.c:722
578 #1 0xc0100f8f in thread_block () at ../../threads/thread.c:324
579 #2 0xc0104755 in timer_sleep (ticks=1000) at ../../devices/timer.c:141
580 #3 0xc010bf7c in test_mlfqs_load_1 () at ../../tests/threads/mlfqs-load-1.c:49
581 #4 0xc010aabb in run_test (name=0xc0007d8c "mlfqs-load-1")
582 at ../../tests/threads/tests.c:50
583 #5 0xc0100647 in run_task (argv=0xc0110d28) at ../../threads/init.c:281
584 #6 0xc0100721 in run_actions (argv=0xc0110d28) at ../../threads/init.c:331
585 #7 0xc01000c7 in main () at ../../threads/init.c:140
587 pintos-debug: dumping backtrace of thread 'idle' @@0xc0116000
588 #0 0xc010168c in next_thread_to_run () at ../../threads/thread.c:649
589 #1 0xc0101778 in schedule () at ../../threads/thread.c:714
590 #2 0xc0100f8f in thread_block () at ../../threads/thread.c:324
591 #3 0xc0101419 in idle (aux=0x0) at ../../threads/thread.c:551
592 #4 0xc010145a in kernel_thread (function=0xc01013ff , aux=0x0)
593 at ../../threads/thread.c:575
594 #5 0x00000000 in ?? ()
598 In this case, there are only two threads, the idle thread and the main
599 thread. The kernel stack pages (to which the @struct{thread} points)
600 are at @t{0xc0116000} and @t{0xc002f000}, respectively. The main thread
601 is stuck in @func{timer_sleep}, called from @code{test_mlfqs_load_1}.
603 Knowing where threads are stuck can be tremendously useful, for instance
604 when diagnosing deadlocks or unexplained hangs.
606 @deffn {GDB Macro} loadusersymbols
608 You can also use GDB to debug a user program running under Pintos.
609 To do that, use the @code{loadusersymbols} macro to load the program's
612 loadusersymbols @var{program}
615 where @var{program} is the name of the program's executable (in the host
616 file system, not in the Pintos file system). For example, you may issue:
618 (gdb) @strong{loadusersymbols tests/userprog/exec-multiple}
619 add symbol table from file "tests/userprog/exec-multiple" at
620 .text_addr = 0x80480a0
624 After this, you should be
625 able to debug the user program the same way you would the kernel, by
626 placing breakpoints, inspecting data, etc. Your actions apply to every
627 user program running in Pintos, not just to the one you want to debug,
628 so be careful in interpreting the results: GDB does not know
629 which process is currently active (because that is an abstraction
630 the Pintos kernel creates). Also, a name that appears in
631 both the kernel and the user program will actually refer to the kernel
632 name. (The latter problem can be avoided by giving the user executable
633 name on the GDB command line, instead of @file{kernel.o}, and then using
634 @code{loadusersymbols} to load @file{kernel.o}.)
635 @code{loadusersymbols} is implemented via GDB's @code{add-symbol-file}
644 @item GDB can't connect to Bochs.
646 If the @command{target remote} command fails, then make sure that both
647 GDB and @command{pintos} are running on the same machine by
648 running @command{hostname} in each terminal. If the names printed
649 differ, then you need to open a new terminal for GDB on the
650 machine running @command{pintos}.
652 @item GDB doesn't recognize any of the macros.
654 If you start GDB with @command{pintos-gdb}, it should load the Pintos
655 macros automatically. If you start GDB some other way, then you must
656 issue the command @code{source @var{pintosdir}/src/misc/gdb-macros},
657 where @var{pintosdir} is the root of your Pintos directory, before you
660 @item Can I debug Pintos with DDD?
662 Yes, you can. DDD invokes GDB as a subprocess, so you'll need to tell
663 it to invokes @command{pintos-gdb} instead:
665 ddd --gdb --debugger pintos-gdb
668 @item Can I use GDB inside Emacs?
670 Yes, you can. Emacs has special support for running GDB as a
671 subprocess. Type @kbd{M-x gdb} and enter your @command{pintos-gdb}
672 command at the prompt. The Emacs manual has information on how to use
673 its debugging features in a section titled ``Debuggers.''
675 @item GDB is doing something weird.
677 If you notice strange behavior while using GDB, there
678 are three possibilities: a bug in your
679 modified Pintos, a bug in Bochs's
680 interface to GDB or in GDB itself, or
681 a bug in the original Pintos code. The first and second
682 are quite likely, and you should seriously consider both. We hope
683 that the third is less likely, but it is also possible.
687 @section Triple Faults
689 When a CPU exception handler, such as a page fault handler, cannot be
690 invoked because it is missing or defective, the CPU will try to invoke
691 the ``double fault'' handler. If the double fault handler is itself
692 missing or defective, that's called a ``triple fault.'' A triple fault
693 causes an immediate CPU reset.
695 Thus, if you get yourself into a situation where the machine reboots in
696 a loop, that's probably a ``triple fault.'' In a triple fault
697 situation, you might not be able to use @func{printf} for debugging,
698 because the reboots might be happening even before everything needed for
699 @func{printf} is initialized.
701 There are at least two ways to debug triple faults. First, you can run
702 Pintos in Bochs under GDB (@pxref{GDB}). If Bochs has been built
703 properly for Pintos, a triple fault under GDB will cause it to print the
704 message ``Triple fault: stopping for gdb'' on the console and break into
705 the debugger. (If Bochs is not running under GDB, a triple fault will
706 still cause it to reboot.) You can then inspect where Pintos stopped,
707 which is where the triple fault occurred.
709 Another option is what I call ``debugging by infinite loop.''
710 Pick a place in the Pintos code, insert the infinite loop
711 @code{for (;;);} there, and recompile and run. There are two likely
716 The machine hangs without rebooting. If this happens, you know that
717 the infinite loop is running. That means that whatever caused the
718 reboot must be @emph{after} the place you inserted the infinite loop.
719 Now move the infinite loop later in the code sequence.
722 The machine reboots in a loop. If this happens, you know that the
723 machine didn't make it to the infinite loop. Thus, whatever caused the
724 reboot must be @emph{before} the place you inserted the infinite loop.
725 Now move the infinite loop earlier in the code sequence.
728 If you move around the infinite loop in a ``binary search'' fashion, you
729 can use this technique to pin down the exact spot that everything goes
730 wrong. It should only take a few minutes at most.
732 @node Modifying Bochs
733 @section Modifying Bochs
735 An advanced debugging technique is to modify and recompile the
736 simulator. This proves useful when the simulated hardware has more
737 information than it makes available to the OS. For example, page
738 faults have a long list of potential causes, but the hardware does not
739 report to the OS exactly which one is the particular cause.
740 Furthermore, a bug in the kernel's handling of page faults can easily
741 lead to recursive faults, but a ``triple fault'' will cause the CPU to
742 reset itself, which is hardly conducive to debugging.
744 In a case like this, you might appreciate being able to make Bochs
745 print out more debug information, such as the exact type of fault that
746 occurred. It's not very hard. You start by retrieving the source
747 code for Bochs 2.2.6 from @uref{http://bochs.sourceforge.net} and
748 saving the file @file{bochs-2.2.6.tar.gz} into a directory.
749 The script @file{pintos/src/misc/bochs-2.2.6-build.sh}
750 applies a number of patches contained in @file{pintos/src/misc}
751 to the Bochs tree, then builds Bochs and installs it in a directory
753 Run this script without arguments to learn usage instructions.
754 To use your @file{bochs} binary with @command{pintos},
755 put it in your @env{PATH}, and make sure that it is earlier than
756 @file{@value{localpintosbindir}/bochs}.
758 Of course, to get any good out of this you'll have to actually modify
759 Bochs. Instructions for doing this are firmly out of the scope of
760 this document. However, if you want to debug page faults as suggested
761 above, a good place to start adding @func{printf}s is
762 @func{BX_CPU_C::dtranslate_linear} in @file{cpu/paging.cc}.
767 The page allocator in @file{threads/palloc.c} and the block allocator in
768 @file{threads/malloc.c} clear all the bytes in memory to
769 @t{0xcc} at time of free. Thus, if you see an attempt to
770 dereference a pointer like @t{0xcccccccc}, or some other reference to
771 @t{0xcc}, there's a good chance you're trying to reuse a page that's
772 already been freed. Also, byte @t{0xcc} is the CPU opcode for ``invoke
773 interrupt 3,'' so if you see an error like @code{Interrupt 0x03 (#BP
774 Breakpoint Exception)}, then Pintos tried to execute code in a freed page or
777 An assertion failure on the expression @code{sec_no < d->capacity}
778 indicates that Pintos tried to access a file through an inode that has
779 been closed and freed. Freeing an inode clears its starting sector
780 number to @t{0xcccccccc}, which is not a valid sector number for disks
781 smaller than about 1.6 TB.