Make tests public. Rewrite most tests. Add tests.

[pintos-anon] / doc / intro.texi

@node Introduction, Pintos Tour, Top, Top
@chapter Introduction

Welcome to Pintos.  Pintos is a simple operating system framework for
the 80@var{x}86 architecture.  It supports kernel threads, loading and
running user programs, and a file system, but it implements all of
these in a very simple way.  In the Pintos projects, you and your
project team will strengthen its support in all three of these areas.
You will also add a virtual memory implementation.

Pintos could, theoretically, run on a regular IBM-compatible PC.
Unfortunately, it is impractical to supply every CS 140 student
a dedicated PC for use with Pintos.  Therefore, we will run Pintos projects
in a system simulator, that is, a program that simulates an 80@var{x}86
CPU and its peripheral devices accurately enough that unmodified operating
systems and software can run under it.  In class we will use the
@uref{http://bochs.sourceforge.net, , Bochs} and 
@uref{http://fabrice.bellard.free.fr/qemu/, ,
qemu} simulators.  Pintos has also been tested with
@uref{http://www.vmware.com/products/server/gsx_features.html, ,
VMware GSX Server}.

These projects are hard.  CS 140 has a reputation of taking a lot of
time, and deservedly so.  We will do what we can to reduce the workload, such
as providing a lot of support material, but there is plenty of
hard work that needs to be done.  We welcome your
feedback.  If you have suggestions on how we can reduce the unnecessary
overhead of assignments, cutting them down to the important underlying
issues, please let us know.

This chapter explains how to get started working with Pintos.  You
should read the entire chapter before you start work on any of the
projects.

@menu
* Getting Started::             
* Grading::                     
* License::                     
* Acknowledgements::            
* Trivia::                      
@end menu

@node Getting Started
@section Getting Started

To get started, you'll have to log into a machine that Pintos can be
built on.  The CS140 ``officially supported''
Pintos development machines are the machines in Sweet Hall managed by
Stanford ITSS, as described on the
@uref{http://www.stanford.edu/services/cluster/environs/sweet/, , ITSS
webpage}.  You may use the Solaris or Linux machines.  We will test your
code on these machines, and the instructions given here assume this
environment.  However, Pintos and its supporting tools are portable
enough that it should build ``out of the box'' in other environments.

Once you've logged into one of these machines, either locally or
remotely, start out by adding our binaries directory to your @env{PATH}
environment.  Under @command{csh}, Stanford's login shell, you can do so
with this command:@footnote{The term @samp{`uname -m`} expands to either
@file{sun4u} or @file{i686} according to the type of computer you're
logged into.}
@example
set path = ( $path /usr/class/cs140/`uname -m`/bin )
@end example
@noindent
(Notice that both @samp{`} are left single quotes or ``backticks.'')
It is a good idea to add this line to the @file{.cshrc} file
in your home directory.  Otherwise, you'll have to type it every time
you log in.

@menu
* Source Tree Overview::        
* Building Pintos::             
* Running Pintos::              
* Debugging versus Testing::    
@end menu

@node Source Tree Overview
@subsection Source Tree Overview

Now you can extract the source for Pintos into a directory named
@file{pintos/src}, by executing
@example
tar xzf /usr/class/cs140/pintos/pintos.tar.gz
@end example
Alternatively, fetch
@uref{http://@/www.stanford.edu/@/class/@/cs140/@/pintos/@/pintos.@/tar.gz}
and extract it in a similar way.

Let's take a look at what's inside.  Here's the directory structure
that you should see in @file{pintos/src}:

@table @file
@item threads/
Source code for the base kernel, which you will modify starting in
project 1.

@item userprog/
Source code for the user program loader, which you will modify
starting with project 2.

@item vm/
An almost empty directory.  You will implement virtual memory here in
project 3.

@item filesys/
Source code for a basic file system.  You will use this file system
starting with project 2, but you will not modify it until project 4.

@item devices/
Source code for I/O device interfacing: keyboard, timer, disk, etc.
You will modify the timer implementation in project 1.  Otherwise
you should have no need to change this code.

@item lib/
An implementation of a subset of the standard C library.  The code in
this directory is compiled into both the Pintos kernel and, starting
from project 2, user programs that run under it.  In both kernel code
and user programs, headers in this directory can be included using the
@code{#include <@dots{}>} notation.  You should have little need to
modify this code.

@item lib/kernel/
Parts of the C library that are included only in the Pintos kernel.
This also includes implementations of some data types that you are
free to use in your kernel code: bitmaps, doubly linked lists, and
hash tables.  In the kernel, headers in this
directory can be included using the @code{#include <@dots{}>}
notation.

@item lib/user/
Parts of the C library that are included only in Pintos user programs.
In user programs, headers in this directory can be included using the
@code{#include <@dots{}>} notation.

@item tests/
Tests for each project.  You can modify this code if it helps you test
your submission, but we will replace it with the originals before we run
the tests.

@item examples/
Example user programs for use starting with project 2.

@item misc/
@itemx utils/
These files may come in handy if you decide to try working with Pintos
away from the ITSS machines.  Otherwise, you can ignore them.
@end table

@node Building Pintos
@subsection Building Pintos

As the next step, build the source code supplied for
the first project.  First, @command{cd} into the @file{threads}
directory.  Then, issue the @samp{make} command.  This will create a
@file{build} directory under @file{threads}, populate it with a
@file{Makefile} and a few subdirectories, and then build the kernel
inside.  The entire build should take less than 30 seconds.

Watch the commands executed during the build.  On the Linux machines,
the ordinary system tools are used.  On a SPARC machine, special build
tools are used, whose names begin with @samp{i386-elf-}, e.g.@:
@code{i386-elf-gcc}, @code{i386-elf-ld}.  These are ``cross-compiler''
tools.  That is, the build is running on a SPARC machine (called the
@dfn{host}), but the result will run on a simulated 80@var{x}86 machine
(called the @dfn{target}).  The @samp{i386-elf-@var{program}} tools are
specially built for this configuration.

Following the build, the following are the interesting files in the
@file{build} directory:

@table @file
@item Makefile
A copy of @file{pintos/src/Makefile.build}.  It describes how to build
the kernel.  @xref{Adding Source Files}, for more information.

@item kernel.o
Object file for the entire kernel.  This is the result of linking
object files compiled from each individual kernel source file into a
single object file.  It contains debug information, so you can run
@command{gdb} or @command{backtrace} (@pxref{Backtraces}) on it.

@item kernel.bin
Memory image of the kernel.  These are the exact bytes loaded into
memory to run the Pintos kernel.  To simplify loading, it is always
padded out with zero bytes up to an exact multiple of 4 kB in
size.

@item loader.bin
Memory image for the kernel loader, a small chunk of code written in
assembly language that reads the kernel from disk into memory and
starts it up.  It is exactly 512 bytes long, a size fixed by the
PC BIOS.

@item os.dsk
Disk image for the kernel, which is just @file{loader.bin} followed by
@file{kernel.bin}.  This file is used as a ``virtual disk'' by the
simulator.
@end table

Subdirectories of @file{build} contain object files (@file{.o}) and
dependency files (@file{.d}), both produced by the compiler.  The
dependency files tell @command{make} which source files need to be
recompiled when other source or header files are changed.

@node Running Pintos
@subsection Running Pintos

We've supplied a program for conveniently running Pintos in a simulator,
called @command{pintos}.  In the simplest case, you can invoke
@command{pintos} as @code{pintos @var{argument}@dots{}}.  Each
@var{argument} is passed to the Pintos kernel for it to act on.

Try it out.  First @command{cd} into the newly created @file{build}
directory.  Then issue the command @code{pintos run alarm-multiple},
which passes the arguments @code{run alarm-multiple} to the Pintos
kernel.  In these arguments, @command{run} instructs the kernel to run a
test and @code{alarm-multiple} is the test to run.

This command creates a @file{bochsrc.txt} file, which is needed for
running Bochs, and then invoke Bochs.  Bochs opens a new window that
represents the simulated machine's display, and a BIOS message briefly
flashes.  Then Pintos boots and runs the @code{alarm-multiple} test
program, which outputs a few screenfuls of text.  When it's done, you
can close Bochs by clicking on the ``Power'' button in the window's top
right corner, or rerun the whole process by clicking on the ``Reset''
button just to its left.  The other buttons are not very useful for our
purposes.

(If no window appeared at all, and you just got a terminal full of
corrupt-looking text, then you're probably logged in remotely and X
forwarding is not set up correctly.  In this case, you can fix your X
setup, or you can use the @option{-v} option to disable X output:
@code{pintos -v -- run alarm-multiple}.)

The text printed by Pintos inside Bochs probably went by too quickly to
read.  However, you've probably noticed by now that the same text was
displayed in the terminal you used to run @command{pintos}.  This is
because Pintos sends all output both to the VGA display and to the first
serial port, and by default the serial port is connected to Bochs's
@code{stdout}.  You can log this output to a file by redirecting at the
command line, e.g.@: @code{pintos run alarm-multiple > logfile}.

The @command{pintos} program offers several options for configuring the
simulator or the virtual hardware.  If you specify any options, they
must precede the commands passed to the Pintos kernel and be separated
from them by @option{--}, so that the whole command looks like
@code{pintos @var{option}@dots{} -- @var{argument}@dots{}}.  Invoke
@code{pintos} without any arguments to see a list of available options.
Options can select a simulator to use: the default is Bochs, but on the
Linux machines @option{--qemu} selects qemu.  You can run the simulator
with a debugger (@pxref{gdb}).  You can set the amount of memory to give
the VM.  Finally, you can select how you want VM output to be displayed:
use @option{-v} to turn off the VGA display, @option{-t} to use your
terminal window as the VGA display instead of opening a new window
(Bochs only), or @option{-s} to suppress the serial output to
@code{stdout}.

The Pintos kernel has commands and options other than @command{run}.
These are not very interesting for now, but you can see a list of them
using @option{-h}, e.g.@: @code{pintos -h}.

@node Debugging versus Testing
@subsection Debugging versus Testing

When you're debugging code, it's useful to be able to be able to run a
program twice and have it do exactly the same thing.  On second and
later runs, you can make new observations without having to discard or
verify your old observations.  This property is called
``reproducibility.''  The simulator we use by default, Bochs, can be set
up for
reproducibility, and that's the way that @command{pintos} invokes it
by default.

Of course, a simulation can only be reproducible from one run to the
next if its input is the same each time.  For simulating an entire
computer, as we do, this means that every part of the computer must be
the same.  For example, you must use the same command-line argument, the
same disks, the same version
of Bochs, and you must not hit any keys on the keyboard (because you
could not be sure to hit them at exactly the same point each time)
during the runs.

While reproducibility is useful for debugging, it is a problem for
testing thread synchronization, an important part of most of the projects.  In
particular, when Bochs is set up for reproducibility, timer interrupts
will come at perfectly reproducible points, and therefore so will
thread switches.  That means that running the same test several times
doesn't give you any greater confidence in your code's correctness
than does running it only once.

So, to make your code easier to test, we've added a feature, called
``jitter,'' to Bochs, that makes timer interrupts come at random
intervals, but in a perfectly predictable way.  In particular, if you
invoke @command{pintos} with the option @option{-j @var{seed}}, timer
interrupts will come at irregularly spaced intervals.  Within a single
@var{seed} value, execution will still be reproducible, but timer
behavior will change as @var{seed} is varied.  Thus, for the highest
degree of confidence you should test your code with many seed values.

On the other hand, when Bochs runs in reproducible mode, timings are not
realistic, meaning that a ``one-second'' delay may be much shorter or
even much longer than one second.  You can invoke @command{pintos} with
a different option, @option{-r}, to set up Bochs for realistic
timings, in which a one-second delay should take approximately one
second of real time.  Simulation in real-time mode is not reproducible,
and options @option{-j} and @option{-r} are mutually exclusive.

On the Linux machines only, the qemu simulator is available as an
alternative to Bochs (use @option{--qemu} when invoking
@command{pintos}).  The qemu simulator is much faster than Bochs, but it
only supports real-time simulation and does not have a reproducible
mode.

@node Grading
@section Grading

We will grade your assignments based on test results and design quality,
each of which comprises 50% of your grade.

@menu
* Testing::                     
* Design::                      
@end menu

@node Testing
@subsection Testing

Your test result grade will be based on our tests.  Each project has
several tests, each of which has a name beginning with @file{tests}.
To completely test your submission, invoke @code{make check} from the
project @file{build} directory.  This will build and run each test and
print a ``pass'' or ``fail'' message for each one.  When a test fails,
@command{make check} also prints some details of the reason for failure.
After running all the tests, @command{make check} also prints a summary
of the test results.

For project 1, the tests will probably run faster in Bochs.  For the
rest of the projects, they will probably run faster in qemu.

You can also run individual tests one at a time.  A given test @var{t}
writes its output to @file{@var{t}.output}, then a script scores the
output as ``pass'' or ``fail'' and writes the verdict to
@file{@var{t}.result}.  To run and grade a single test, @command{make}
the @file{.result} file explicitly from the @file{build} directory, e.g.@:
@code{make tests/threads/alarm-multiple.result}.  If @command{make} says
that the test result is up-to-date, but you want to re-run it anyway,
either run @code{make clean} or delete the @file{.output} file by hand.

By default, each test provides feedback only at completion, not during
its run.  If you prefer, you can observe the progress of each test by
specifying @option{VERBOSE=1} on the @command{make} command line, as in
@code{make check VERBOSE=1}.  You can also provide arbitrary options to the
@command{pintos} run by the tests with @option{PINTOSOPTS='@dots{}'},
e.g.@: @code{make check PINTOSOPTS='--qemu'} to run the tests under
qemu.

All of the tests and related files are in @file{pintos/src/tests}.
Before we test your submission, we will replace the contents of that
directory by a pristine, unmodified copy, to ensure that the correct
tests are used.  Thus, you can modify some of the tests if that helps in
debugging, but we will run the originals.

All software has bugs, so some of our tests may be flawed.  If you think
a test failure is a bug in the test, not a bug in your code,
please point it out.  We will look at it and fix it if necessary.

Please don't try to take advantage of our generosity in giving out our
test suite.  Your code has to work properly in the general case, not
just for the test cases we supply.  For example, it would be unacceptable
to explicitly base the kernel's behavior on the name of the running
test case.  Such attempts to side-step the test cases will receive no
credit.  If you think your solution may be in a gray area here, please
ask us about it.

@node Design
@subsection Design

We will judge your design based on the design document and the source
code that you submit.  We will read your entire design document and much
of your source code.  

We provide a design document template for each project.  For each
significant part of a project, the template asks questions in four
areas: data structures, algorithms, synchronization, and rationale.  An
incomplete design document or one that strays from the template without
good reason may be penalized.  Incorrect capitalization, punctuation,
spelling, or grammar can also cost points.  @xref{Project
Documentation}, for a sample design document for a fictitious project.

Design quality will also be judged based on your source code.  We will
typically look at the differences between the original Pintos source
tree and your submission, based on the output of a command like
@code{diff -urpb pintos.orig pintos.submitted}.  We will try to match up your
description of the design with the code submitted.  Important
discrepancies between the description and the actual code will be
penalized, as will be any bugs we find by spot checks.

The most important aspects of design quality are those that specifically
relate to the operating system issues at stake in the project.  For
example, the organization of an inode is an important part of file
system design, so in the file system project a poorly designed inode
would lose points.  Other issues are much less important.  For
example, multiple Pintos design problems call for a ``priority
queue,'' that is, a dynamic collection from which the minimum (or
maximum) item can quickly be extracted.  Fast priority queues can be
implemented many ways, but we do not expect you to build a fancy data
structure even if it might improve performance.  Instead, you are
welcome to use a linked list (and Pintos even provides one with
convenient functions for sorting and finding minimums and maximums).

Pintos is written in a consistent style.  Make your additions and
modifications in existing Pintos source files blend in, not stick out.
In new source files, adopt the existing Pintos style by preference, but
make the self-consistent at the very least.  Use horizontal and vertical
white space to make code readable.  Add a comment to every structure,
structure member, global or static variable, and function definition.
Update existing comments as you modify code.  Don't comment out or use
the preprocessor to ignore blocks of code.  Use assertions to document
key invariants.  Decompose code into functions for clarity.  Code that
is difficult to understand because it violates these or other ``common
sense'' software engineering practices will be penalized.

In the end, remember your audience.  Code is written primarily to be
read by humans.  It has to be acceptable to the compiler too, but the
compiler doesn't care about how it looks or how well it is written.

@node License
@section License

Pintos is distributed under a liberal license that allows free use,
modification, and distribution.  Students and others who work on Pintos
own the code that they write and may use it for any purpose.

In the context of Stanford's CS 140 course, please respect the spirit
and the letter of the honor code by refraining from reading any homework
solutions available online or elsewhere.  Reading the source code for
other operating system kernels, such as Linux or FreeBSD, is allowed,
but do not copy code from them literally.  Please cite the code that
inspired your own in your design documentation.

Pintos comes with NO WARRANTY, not even for MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.

The @file{LICENSE} file at the top level of the Pintos source
distribution has full details of the license and lack of warranty.

@node Acknowledgements
@section Acknowledgements

Pintos and this documentation were written by Ben Pfaff
@email{blp@@cs.stanford.edu}.

The original structure and form of Pintos was inspired by the Nachos
instructional operating system from the University of California,
Berkeley.  A few of the source files were originally more-or-less
literal translations of the Nachos C++ code into C.  These files bear
the original UCB license notice.

A few of the Pintos source files are derived from code used in the
Massachusetts Institute of Technology's 6.828 advanced operating systems
course.  These files bear the original MIT license notice.

The Pintos projects and documentation originated with those designed for
Nachos by current and former CS140 teaching assistants at Stanford
University, including at least Yu Ping, Greg Hutchins, Kelly Shaw, Paul
Twohey, Sameer Qureshi, and John Rector.  If you're not on this list but
should be, please let me know.

Example code for condition variables (@pxref{Condition Variables}) is
from classroom slides originally by Dawson Engler and updated by Mendel
Roseblum.

@node Trivia
@section Trivia

Pintos originated as a replacement for Nachos with a similar design.
Since then Pintos has greatly diverged from the Nachos design.  Pintos
differs from Nachos in two important ways.  First, Pintos runs on real
or simulated 80@var{x}86 hardware, whereas Nachos runs as a process on a
host operating system.  Second, like most real-world operating systems,
Pintos is written in C, whereas Nachos is written in C++.

Why the name ``Pintos''?  First, like nachos, pinto beans are a common
Mexican food.  Second, Pintos is small and a ``pint'' is a small amount.
Third, like drivers of the eponymous car, students are likely to have
trouble with blow-ups.