Add some clarifications to address question from "Kevin Hart"

[pintos-anon] / doc / threads.texi

@node Project 1--Threads, Project 2--User Programs, Pintos Tour, Top
@chapter Project 1: Threads

In this assignment, we give you a minimally functional thread system.
Your job is to extend the functionality of this system to gain a
better understanding of synchronization problems.

You will be working primarily in the @file{threads} directory for
this assignment, with some work in the @file{devices} directory on the
side.  Compilation should be done in the @file{threads} directory.

Before you read the description of this project, you should read all of
the following sections: @ref{Introduction}, @ref{Coding Standards},
@ref{Debugging Tools}, and @ref{Development Tools}.  You should at least
skim the material in @ref{Threads Tour} and especially
@ref{Synchronization}.  To complete this project you will also need to
read @ref{4.4BSD Scheduler}.

@menu
* Project 1 Background::        
* Project 1 Requirements::      
* Project 1 FAQ::               
@end menu

@node Project 1 Background
@section Background


@menu
* Understanding Threads::       
* Project 1 Source Files::      
* Project 1 Synchronization::   
* Development Suggestions::     
@end menu

@node Understanding Threads
@subsection Understanding Threads

The first step is to read and understand the code for the initial thread
system.
Pintos already implements thread creation and thread completion,
a simple scheduler to switch between threads, and synchronization
primitives (semaphores, locks, condition variables, and memory
barriers).

Some of this code might seem slightly mysterious.  If
you haven't already compiled and run the base system, as described in
the introduction (@pxref{Introduction}), you should do so now.  You
can read through parts of the source code to see what's going
on.  If you like, you can add calls to @func{printf} almost
anywhere, then recompile and run to see what happens and in what
order.  You can also run the kernel in a debugger and set breakpoints
at interesting spots, single-step through code and examine data, and
so on.

When a thread is created, you are creating a new context to be
scheduled.  You provide a function to be run in this context as an
argument to @func{thread_create}.  The first time the thread is
scheduled and runs, it starts from the beginning of that function
and executes in that context.  When the function returns, the thread
terminates.  Each thread, therefore, acts like a mini-program running
inside Pintos, with the function passed to @func{thread_create}
acting like @func{main}.

At any given time, exactly one thread runs and the rest, if any,
become inactive.  The scheduler decides which thread to
run next.  (If no thread is ready to run
at any given time, then the special ``idle'' thread, implemented in
@func{idle}, runs.)
Synchronization primitives can force context switches when one
thread needs to wait for another thread to do something.

The mechanics of a context switch are
in @file{threads/switch.S}, which is 80@var{x}86
assembly code.  (You don't have to understand it.)  It saves the
state of the currently running thread and restores the state of the
thread we're switching to.

Using the @command{gdb} debugger, slowly trace through a context
switch to see what happens (@pxref{gdb}).  You can set a
breakpoint on @func{schedule} to start out, and then
single-step from there.@footnote{@command{gdb} might tell you that
@func{schedule} doesn't exist, which is arguably a @command{gdb} bug.
You can work around this by setting the breakpoint by filename and
line number, e.g.@: @code{break thread.c:@var{ln}} where @var{ln} is
the line number of the first declaration in @func{schedule}.}  Be sure
to keep track of each thread's address
and state, and what procedures are on the call stack for each thread.
You will notice that when one thread calls @func{switch_threads},
another thread starts running, and the first thing the new thread does
is to return from @func{switch_threads}.  You will understand the thread
system once you understand why and how the @func{switch_threads} that
gets called is different from the @func{switch_threads} that returns.
@xref{Thread Switching}, for more information.

@strong{Warning}: In Pintos, each thread is assigned a small,
fixed-size execution stack just under @w{4 kB} in size.  The kernel
tries to detect stack overflow, but it cannot do so perfectly.  You
may cause bizarre problems, such as mysterious kernel panics, if you
declare large data structures as non-static local variables,
e.g. @samp{int buf[1000];}.  Alternatives to stack allocation include
the page allocator and the block allocator (@pxref{Memory Allocation}).

@node Project 1 Source Files
@subsection Source Files

Here is a brief overview of the files in the @file{threads}
directory.  You will not need to modify most of this code, but the
hope is that presenting this overview will give you a start on what
code to look at.

@table @file
@item loader.S
@itemx loader.h
The kernel loader.  Assembles to 512 bytes of code and data that the
PC BIOS loads into memory and which in turn loads the kernel into
memory, does basic processor initialization, and jumps to the
beginning of the kernel.  @xref{Pintos Loader}, for details. You should
not need to look at this code or modify it.

@item kernel.lds.S
The linker script used to link the kernel.  Sets the load address of
the kernel and arranges for @file{start.S} to be at the very beginning
of the kernel image.  @xref{Pintos Loader}, for details. Again, you
should not need to look at this code
or modify it, but it's here in case you're curious.

@item start.S
Jumps to @func{main}.

@item init.c
@itemx init.h
Kernel initialization, including @func{main}, the kernel's ``main
program.''  You should look over @func{main} at least to see what
gets initialized.  You might want to add your own initialization code
here.  @xref{Kernel Initialization}, for details.

@item thread.c
@itemx thread.h
Basic thread support.  Much of your work will take place in these
files.  @file{thread.h} defines @struct{thread}, which you are likely
to modify in all four projects.  See @ref{struct thread} and @ref{Thread
Support} for more information.

@item switch.S
@itemx switch.h
Assembly language routine for switching threads.  Already discussed
above.  @xref{Thread Functions}, for more information.

@item palloc.c
@itemx palloc.h
Page allocator, which hands out system memory in multiples of 4 kB
pages.  @xref{Page Allocator}, for more information.

@item malloc.c
@itemx malloc.h
A simple implementation of @func{malloc} and @func{free} for
the kernel.  @xref{Block Allocator}, for more information.

@item interrupt.c
@itemx interrupt.h
Basic interrupt handling and functions for turning interrupts on and
off.  @xref{Interrupt Handling}, for more information.

@item intr-stubs.S
@itemx intr-stubs.h
Assembly code for low-level interrupt handling.  @xref{Interrupt
Infrastructure}, for more information.

@item synch.c
@itemx synch.h
Basic synchronization primitives: semaphores, locks, condition
variables, and memory barriers.  You will need to use these for
synchronization in all
four projects.  @xref{Synchronization}, for more information.

@item io.h
Functions for I/O port access.  This is mostly used by source code in
the @file{devices} directory that you won't have to touch.

@item mmu.h
Functions and macros related to memory management, including page
directories and page tables.  This will be more important to you in
project 3.  For now, you can ignore it.

@item flags.h
Macros that define a few bits in the 80@var{x}86 ``flags'' register.
Probably of no interest.  See @bibref{IA32-v1}, section 3.4.3, for more
information.
@end table

@menu
* devices code::                
* lib files::                   
@end menu

@node devices code
@subsubsection @file{devices} code

The basic threaded kernel also includes these files in the
@file{devices} directory:

@table @file
@item timer.c
@itemx timer.h
System timer that ticks, by default, 100 times per second.  You will
modify this code in this project.

@item vga.c
@itemx vga.h
VGA display driver.  Responsible for writing text to the screen.
You should have no need to look at this code.  @func{printf}
calls into the VGA display driver for you, so there's little reason to
call this code yourself.

@item serial.c
@itemx serial.h
Serial port driver.  Again, @func{printf} calls this code for you,
so you don't need to do so yourself.  Feel free to look through it if
you're curious.

@item disk.c
@itemx disk.h
Supports reading and writing sectors on up to 4 IDE disks.  This won't
actually be used until project 2.

@item intq.c
@itemx intq.h
Interrupt queue, for managing a circular queue that both kernel
threads and interrupt handlers want to access.  Used by the keyboard
and serial drivers.
@end table

@node lib files
@subsubsection @file{lib} files

Finally, @file{lib} and @file{lib/kernel} contain useful library
routines.  (@file{lib/user} will be used by user programs, starting in
project 2, but it is not part of the kernel.)  Here's a few more
details:

@table @file
@item ctype.h
@itemx inttypes.h
@itemx limits.h
@itemx stdarg.h
@itemx stdbool.h
@itemx stddef.h
@itemx stdint.h
@itemx stdio.c
@itemx stdio.h
@itemx stdlib.c
@itemx stdlib.h
@itemx string.c
@itemx string.h
A subset of the standard C library.  @xref{C99}, for
information
on a few recently introduced pieces of the C library that you might
not have encountered before.  @xref{Unsafe String Functions}, for
information on what's been intentionally left out for safety.

@item debug.c
@itemx debug.h
Functions and macros to aid debugging.  @xref{Debugging Tools}, for
more information.

@item random.c
@itemx random.h
Pseudo-random number generator.

@item round.h
Macros for rounding.

@item syscall-nr.h
System call numbers.  Not used until project 2.

@item kernel/list.c
@itemx kernel/list.h
Doubly linked list implementation.  Used all over the Pintos code, and
you'll probably want to use it a few places yourself in project 1.

@item kernel/bitmap.c
@itemx kernel/bitmap.h
Bitmap implementation.  You can use this in your code if you like, but
you probably won't have any need for it in project 1.

@item kernel/hash.c
@itemx kernel/hash.h
Hash table implementation.  Likely to come in handy for project 3.

@item kernel/console.c
@itemx kernel/console.h
@item kernel/stdio.h
Implements @func{printf} and a few other functions.
@end table

@node Project 1 Synchronization
@subsection Synchronization

Proper synchronization is an important part of the solutions to these
problems.  Any synchronization problem can be easily solved by turning
interrupts off: while interrupts are off, there is no concurrency, so
there's no possibility for race conditions.  Therefore, it's tempting to
solve all synchronization problems this way, but @strong{don't}.
Instead, use semaphores, locks, and condition variables to solve the
bulk of your synchronization problems.  Read the tour section on
synchronization (@pxref{Synchronization}) or the comments in
@file{threads/synch.c} if you're unsure what synchronization primitives
may be used in what situations.

In the Pintos projects, the only class of problem best solved by
disabling interrupts is coordinating data shared between a kernel thread
and an interrupt handler.  Because interrupt handlers can't sleep, they
can't acquire locks.  This means that data shared between kernel threads
and an interrupt handler must be protected within a kernel thread by
turning off interrupts.

This project only requires accessing a little bit of thread state from
interrupt handlers.  For the alarm clock, the timer interrupt needs to
wake up sleeping threads.  In the advanced scheduler, the timer
interrupt needs to access a few global and per-thread variables.  When
you access these variables from kernel threads, you will need to disable
interrupts to prevent the timer interrupt from interfering.

When you do turn off interrupts, take care to do so for the least amount
of code possible, or you can end up losing important things such as
timer ticks or input events.  Turning off interrupts also increases the
interrupt handling latency, which can make a machine feel sluggish if
taken too far.

Disabling interrupts can be useful for debugging, if you want to make
sure that a section of code is not interrupted.  You should remove
debugging code before turning in your project.

There should be no busy waiting in your submission.  A tight loop that
calls @func{thread_yield} is one form of busy waiting.

@node Development Suggestions
@subsection Development Suggestions

In the past, many groups divided the assignment into pieces, then each
group member worked on his or her piece until just before the
deadline, at which time the group reconvened to combine their code and
submit.  @strong{This is a bad idea.  We do not recommend this
approach.}  Groups that do this often find that two changes conflict
with each other, requiring lots of last-minute debugging.  Some groups
who have done this have turned in code that did not even compile or
boot, much less pass any tests.

Instead, we recommend integrating your team's changes early and often,
using a source code control system such as CVS (@pxref{CVS}) or a
group collaboration site such as SourceForge (@pxref{SourceForge}).
This is less likely to produce surprises, because everyone can see
everyone else's code as it is written, instead of just when it is
finished.  These systems also make it possible to review changes and,
when a change introduces a bug, drop back to working versions of code.

You should expect to run into bugs that you simply don't understand
while working on this and subsequent projects.  When you do,
reread the appendix on debugging tools, which is filled with
useful debugging tips that should help you to get back up to speed
(@pxref{Debugging Tools}).  Be sure to read the section on backtraces
(@pxref{Backtraces}), which will help you to get the most out of every
kernel panic or assertion failure.

@node Project 1 Requirements
@section Requirements

@menu
* Project 1 Design Document::   
* Alarm Clock::                 
* Priority Scheduling::         
* Advanced Scheduler::          
@end menu

@node Project 1 Design Document
@subsection Design Document

Before you turn in your project, you must copy @uref{threads.tmpl, , the
project 1 design document template} into your source tree under the name
@file{pintos/src/threads/DESIGNDOC} and fill it in.  We recommend that
you read the design document template before you start working on the
project.  @xref{Project Documentation}, for a sample design document
that goes along with a fictitious project.

@node Alarm Clock
@subsection Alarm Clock

Reimplement @func{timer_sleep}, defined in @file{devices/timer.c}.
Although a working implementation is provided, it ``busy waits,'' that
is, it spins in a loop checking the current time and calling
@func{thread_yield} until enough time has gone by.  Reimplement it to
avoid busy waiting.

@deftypefun void timer_sleep (int64_t @var{ticks})
Suspends execution of the calling thread until time has advanced by at
least @w{@var{x} timer ticks}.  Unless the system is otherwise idle, the
thread need not wake up after exactly @var{x} ticks.  Just put it on
the ready queue after they have waited for the right amount of time.

@func{timer_sleep} is useful for threads that operate in real-time,
e.g.@: for blinking the cursor once per second.

The argument to @func{timer_sleep} is expressed in timer ticks, not in
milliseconds or any another unit.  There are @code{TIMER_FREQ} timer
ticks per second, where @code{TIMER_FREQ} is a macro defined in
@code{devices/timer.h}.  The default value is 100.  We don't recommend
changing this value, because any change is likely to cause many of
the tests to fail.
@end deftypefun

Separate functions @func{timer_msleep}, @func{timer_usleep}, and
@func{timer_nsleep} do exist for sleeping a specific number of
milliseconds, microseconds, or nanoseconds, respectively, but these will
call @func{timer_sleep} automatically when necessary.  You do not need
to modify them.

If your delays seem too short or too long, reread the explanation of the
@option{-r} option to @command{pintos} (@pxref{Debugging versus
Testing}).

The alarm clock implementation is not needed for later projects,
although it could be useful for project 4.

@node Priority Scheduling
@subsection Priority Scheduling

Implement priority scheduling in Pintos.
When a thread is added to the ready list that has a higher priority
than the currently running thread, the current thread should
immediately yield the processor to the new thread.  Similarly, when
threads are waiting for a lock, semaphore, or condition variable, the
highest priority waiting thread should be woken up first.  A thread
may raise or lower its own priority at any time, but lowering its
priority such that it no longer has the highest priority must cause it
to immediately yield the CPU.

Thread priorities range from @code{PRI_MIN} (0) to @code{PRI_MAX} (63).
Lower numbers correspond to @emph{higher} priorities, so that priority 0
is the highest priority and priority 63 is the lowest.
The initial thread priority is passed as an argument to
@func{thread_create}.  If there's no reason to choose another
priority, use @code{PRI_DEFAULT} (31).  The @code{PRI_} macros are
defined in @file{threads/thread.h}, and you should not change their
values.

One issue with priority scheduling is ``priority inversion''.  Consider
high, medium, and low priority threads @var{H}, @var{M}, and @var{L},
respectively.  If @var{H} needs to wait for @var{L} (for instance, for a
lock held by @var{L}), and @var{M} is on the ready list, then @var{H}
will never get the CPU because the low priority thread will not get any
CPU time.  A partial fix for this problem is for @var{H} to ``donate''
its priority to @var{L} while @var{L} is holding the lock, then recall
the donation once @var{L} releases (and thus @var{H} acquires) the lock.

Implement priority donation.  You will need to account for all different
situations in which priority donation is required.  Be sure to handle
multiple donations, in which multiple priorities are donated to a single
thread.  You must also handle nested donation: if @var{H} is waiting on
a lock that @var{M} holds and @var{M} is waiting on a lock that @var{L}
holds, then both @var{M} and @var{L} should be boosted to @var{H}'s
priority.

You must implement priority donation for locks.  You need not
implement priority donation for semaphores or condition variables
(but you are welcome to do so).  You do need to implement
priority scheduling in all cases.

Finally, implement the following functions that allow a thread to
examine and modify its own priority.  Skeletons for these functions are
provided in @file{threads/thread.c}.

@deftypefun void thread_set_priority (int @var{new_priority})
Sets the current thread's priority to @var{new_priority}.  If the
current thread no longer has the highest priority, yields.
@end deftypefun

@deftypefun int thread_get_priority (void)
Returns the current thread's priority.  In the presence of priority
donation, returns the higher (donated) priority.
@end deftypefun

You need not provide any interface to allow a thread to directly modify
other threads' priorities.

The priority scheduler is not used in any later project.

@node Advanced Scheduler
@subsection Advanced Scheduler

Implement a multilevel feedback queue scheduler similar to the
4.4@acronym{BSD} scheduler to
reduce the average response time for running jobs on your system.
@xref{4.4BSD Scheduler}, for a detailed description of
the MLFQS requirements.

The advanced scheduler builds on the priority scheduler.  You should
have the priority scheduler working, except possibly for priority
donation, before you start work on the advanced scheduler.

You must write your code so that we can choose a scheduling algorithm
policy at Pintos startup time.  By default, the round-robin scheduler
must be active, but we must be able to choose the 4.4@acronym{BSD}
scheduler
with the @option{-mlfqs} kernel option.  Passing this
option sets @code{enable_mlfqs}, declared in @file{threads/init.h}, to
true.

When the 4.4@acronym{BSD} scheduler is enabled, threads no longer
directly control their own priorities.  The @var{priority} argument to
@func{thread_create} should be ignored, as well as any calls to
@func{thread_set_priority}, and @func{thread_get_priority} should return
the thread's current priority as set by the scheduler.

The advanced scheduler is not used in any later project.

@node Project 1 FAQ
@section FAQ

@table @b
@item How much code will I need to write?

Here's a summary of our reference solution, produced by the
@command{diffstat} program.  The final row gives total lines inserted
and deleted; a changed line counts as both an insertion and a deletion.

@verbatim
 devices/timer.c       |   42 +++++-
 threads/fixed-point.h |  120 ++++++++++++++++++
 threads/synch.c       |   88 ++++++++++++-
 threads/thread.c      |  196 ++++++++++++++++++++++++++----
 threads/thread.h      |   23 +++
 5 files changed, 440 insertions(+), 29 deletions(-)
@end verbatim

@item How do I update the @file{Makefile}s when I add a new source file?

@anchor{Adding Source Files}
To add a @file{.c} file, edit the top-level @file{Makefile.build}.
Add the new file to variable @samp{@var{dir}_SRC}, where
@var{dir} is the directory where you added the file.  For this
project, that means you should add it to @code{threads_SRC} or
@code{devices_SRC}.  Then run @code{make}.  If your new file
doesn't get
compiled, run @code{make clean} and then try again.

When you modify the top-level @file{Makefile.build} and re-run
@command{make}, the modified
version should be automatically copied to
@file{threads/build/Makefile}.  The converse is
not true, so any changes will be lost the next time you run @code{make
clean} from the @file{threads} directory.  Unless your changes are
truly temporary, you should prefer to edit @file{Makefile.build}.

A new @file{.h} file does not require editing the @file{Makefile}s.

@item What does @code{warning: no previous prototype for `@var{func}'} mean?

It means that you defined a non-@code{static} function without
preceding it by a prototype.  Because non-@code{static} functions are
intended for use by other @file{.c} files, for safety they should be
prototyped in a header file included before their definition.  To fix
the problem, add a prototype in a header file that you include, or, if
the function isn't actually used by other @file{.c} files, make it
@code{static}.

@item What is the interval between timer interrupts?

Timer interrupts occur @code{TIMER_FREQ} times per second.  You can
adjust this value by editing @file{devices/timer.h}.  The default is
100 Hz.

We don't recommend changing this value, because any changes are likely
to cause many of the tests to fail.

@item How long is a time slice?

There are @code{TIME_SLICE} ticks per time slice.  This macro is
declared in @file{threads/thread.c}.  The default is 4 ticks.

We don't recommend changing this values, because any changes are likely
to cause many of the tests to fail.
@end table

@menu
* Alarm Clock FAQ::             
* Priority Scheduling FAQ::     
* Advanced Scheduler FAQ::      
@end menu

@node Alarm Clock FAQ
@subsection Alarm Clock FAQ

@table @b
@item Do I need to account for timer values overflowing?

Don't worry about the possibility of timer values overflowing.  Timer
values are expressed as signed 64-bit numbers, which at 100 ticks per
second should be good for almost 2,924,712,087 years.  By then, we
expect Pintos to have been phased out of the CS 140 curriculum.
@end table

@node Priority Scheduling FAQ
@subsection Priority Scheduling FAQ

@table @b
@item Doesn't priority scheduling lead to starvation?

Yes, strict priority scheduling can lead to starvation
because a thread will not run if any higher-priority thread is runnable.
The advanced scheduler introduces a mechanism for dynamically
changing thread priorities.

Strict priority scheduling is valuable in real-time systems because it
offers the programmer more control over which jobs get processing
time.  High priorities are generally reserved for time-critical
tasks. It's not ``fair,'' but it addresses other concerns not
applicable to a general-purpose operating system.

@item What thread should run after a lock has been released?

When a lock is released, the highest priority thread waiting for that
lock should be unblocked and put on the list of ready threads.  The
scheduler should then run the highest priority thread on the ready
list.

@item If the highest-priority thread yields, does it continue running?

Yes.  As long as there is a single highest-priority thread, it continues
running until it blocks or finishes, even if it calls
@func{thread_yield}.
If multiple threads have the same highest priority,
@func{thread_yield} should switch among them in ``round robin'' order.

@item What happens to the priority of a donating thread?

Priority donation only changes the priority of the donee
thread.  The donor thread's priority is unchanged.  
Priority donation is not additive: if thread @var{A} (with priority 5) donates
to thread @var{B} (with priority 3), then @var{B}'s new priority is 5, not 8.

@item Can a thread's priority change while it is on the ready queue?

Yes.  Consider this case: low-priority thread @var{L} holds a
lock that high-priority thread @var{H} wants, so @var{H} donates its
priority to @var{L}.  @var{L} releases the lock and
thus loses the CPU and is moved to the ready queue.  Now @var{L}'s
old priority is restored while it is in the ready queue.

@item Can a thread added to the ready list preempt the processor?

Yes.  If a thread added to the ready list has higher priority than the
running thread, the correct behavior is to immediately yield the
processor.  It is not acceptable to wait for the next timer interrupt.
The highest priority thread should run as soon as it is runnable,
preempting whatever thread is currently running.

@item How does @func{thread_set_priority} affect a thread receiving donations?

It should do something sensible, but no particular behavior is
required.  None of the test cases call @func{thread_set_priority} from a
thread while it is receiving a priority donation.

@item Calling @func{printf} in @func{sema_up} or @func{sema_down} reboots!

@anchor{printf Reboots}
Yes.  These functions are called before @func{printf} is ready to go.
You could add a global flag initialized to false and set it to true
just before the first @func{printf} in @func{main}.  Then modify
@func{printf} itself to return immediately if the flag isn't set.
@end table

@node Advanced Scheduler FAQ
@subsection Advanced Scheduler FAQ

@table @b
@item How does priority donation interact with the advanced scheduler?

It doesn't have to.  We won't test priority donation and the advanced
scheduler at the same time.
@end table