X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?p=pintos-anon;a=blobdiff_plain;f=doc%2Fintro.texi;h=e1eefab50bbad3ac2217f8a5cc870bf80448bea0;hp=1c6584f4d9d0d4e8700f5a7651cf8b0ea56f1db8;hb=615bf3b3d2a8573ed6fb9ddc0055745e163ac999;hpb=5780c9f434cca090f88463b7f0199d49b4ded288 diff --git a/doc/intro.texi b/doc/intro.texi index 1c6584f..e1eefab 100644 --- a/doc/intro.texi +++ b/doc/intro.texi @@ -8,61 +8,63 @@ these in a very simple way. In the Pintos projects, you and your project team will strengthen its support in all three of these areas. You will also add a virtual memory implementation. -Pintos could, theoretically, run on a regular IBM-compatible PC. As -much fun as it might be, it is impractical to supply every student in -CS 140 with his or her own PC. Therefore, we will run Pintos projects -in a PC simulator, that is, a program that simulates an 80@var{x}86 -CPU and its peripheral devices well enough that unmodified operating +Pintos could, theoretically, run on a regular IBM-compatible PC. +Unfortunately, it is impractical to supply every CS 140 student +a dedicated PC for use with Pintos. Therefore, we will run Pintos projects +in a system simulator, that is, a program that simulates an 80@var{x}86 +CPU and its peripheral devices accurately enough that unmodified operating systems and software can run under it. In class we will use the -@uref{http://bochs.sourceforge.net, , Bochs} simulator. Pintos has -also been tested within @uref{http://fabrice.bellard.free.fr/qemu/, , -qemu} and +@uref{http://bochs.sourceforge.net, , Bochs} and +@uref{http://fabrice.bellard.free.fr/qemu/, , +qemu} simulators. Pintos has also been tested with @uref{http://www.vmware.com/products/server/gsx_features.html, , VMware GSX Server}. These projects are hard. CS 140 has a reputation of taking a lot of -time, and deservedly so. We will do what we can to ease the pain, -such as providing a lot of support material, but to some extent there -is simply some amount of hard work that needs to be done. Because -Pintos is a new system, it is also possible that some parts of the -projects are not only difficult, but more difficult than they should -be. We welcome your feedback. If you have suggestions on how we can -reduce the unnecessary overhead of assignments, cutting them down to -the important underlying issues, please let us know. +time, and deservedly so. We will do what we can to reduce the workload, such +as providing a lot of support material, but there is plenty of +hard work that needs to be done. We welcome your +feedback. If you have suggestions on how we can reduce the unnecessary +overhead of assignments, cutting them down to the important underlying +issues, please let us know. This chapter explains how to get started working with Pintos. You -should read the entire chapter before you proceed to any of the +should read the entire chapter before you start work on any of the projects. @menu * Getting Started:: -* Pintos License:: -* Pintos Trivia:: +* Grading:: +* License:: +* Acknowledgements:: +* Trivia:: @end menu @node Getting Started @section Getting Started To get started, you'll have to log into a machine that Pintos can be -built on. For the purposes of CS 140, our ``officially supported'' -Pintos development machines are the Sun Solaris machines managed by +built on. The CS140 ``officially supported'' +Pintos development machines are the machines in Sweet Hall managed by Stanford ITSS, as described on the -@uref{http://www.stanford.edu/dept/itss/services/cluster/environs/sweet/, -, ITSS webpage}. We will test your code on these machines, and the -instructions given here assume this environment. However, Pintos and -its supporting tools are portable enough that it should build ``out of -the box'' in other environments, including the Linux machines managed -by ITSS. +@uref{http://www.stanford.edu/services/cluster/environs/sweet/, , ITSS +webpage}. You may use the Solaris or Linux machines. We will test your +code on these machines, and the instructions given here assume this +environment. However, Pintos and its supporting tools are portable +enough that it should build ``out of the box'' in other environments. Once you've logged into one of these machines, either locally or -remotely, start out by adding our binaries directory to your -@env{PATH} environment. Under @command{csh}, the Stanford default -shell, you can do so with this command: +remotely, start out by adding our binaries directory to your @env{PATH} +environment. Under @command{csh}, Stanford's login shell, you can do so +with this command:@footnote{The term @samp{`uname -m`} expands to either +@file{sun4u} or @file{i686} according to the type of computer you're +logged into.} @example -set path = ( $path /usr/class/cs140/i386/bin ) +set path = ( $path /usr/class/cs140/`uname -m`/bin ) @end example @noindent -It might be a good idea to add this line into the @file{.cshrc} file +(Notice that both @samp{`} are left single quotes or ``backticks.'') +It is a good idea to add this line to the @file{.cshrc} file in your home directory. Otherwise, you'll have to type it every time you log in. @@ -70,26 +72,27 @@ you log in. * Source Tree Overview:: * Building Pintos:: * Running Pintos:: +* Debugging versus Testing:: @end menu @node Source Tree Overview @subsection Source Tree Overview Now you can extract the source for Pintos into a directory named -@file{pintos/src} by executing +@file{pintos/src}, by executing @example tar xzf /usr/class/cs140/pintos/pintos.tar.gz @end example -Alternatively, retrieve -@uref{http://www.stanford.edu/class/cs140/pintos/pintos.tar.gz} and -extract it in a similar way. +Alternatively, fetch +@uref{http://@/www.stanford.edu/@/class/@/cs140/@/pintos/@/pintos.@/tar.gz} +and extract it in a similar way. Let's take a look at what's inside. Here's the directory structure that you should see in @file{pintos/src}: @table @file @item threads/ -Source code for the base kernel, which you will modify starting with +Source code for the base kernel, which you will modify starting in project 1. @item userprog/ @@ -97,62 +100,71 @@ Source code for the user program loader, which you will modify starting with project 2. @item vm/ -An almost empty directory, where you will implement virtual memory in +An almost empty directory. You will implement virtual memory here in project 3. @item filesys/ -Source code for a basic file system, which you will use starting with -project 2 but which you should not modify until project 4. +Source code for a basic file system. You will use this file system +starting with project 2, but you will not modify it until project 4. @item devices/ Source code for I/O device interfacing: keyboard, timer, disk, etc. -You will improve the timer implementation in project 1, but otherwise +You will modify the timer implementation in project 1. Otherwise you should have no need to change this code. @item lib/ An implementation of a subset of the standard C library. The code in this directory is compiled into both the Pintos kernel and, starting -from project 2, user programs that run under it. Headers in this -directory can be included using the @code{#include <@dots{}>} -notation. You should have little need to modify this code. +from project 2, user programs that run under it. In both kernel code +and user programs, headers in this directory can be included using the +@code{#include <@dots{}>} notation. You should have little need to +modify this code. @item lib/kernel/ Parts of the C library that are included only in the Pintos kernel. This also includes implementations of some data types that you are free to use in your kernel code: bitmaps, doubly linked lists, and -hash tables. +hash tables. In the kernel, headers in this +directory can be included using the @code{#include <@dots{}>} +notation. @item lib/user/ Parts of the C library that are included only in Pintos user programs. +In user programs, headers in this directory can be included using the +@code{#include <@dots{}>} notation. @item tests/ -Code for testing each project. +Tests for each project. You can modify this code if it helps you test +your submission, but we will replace it with the originals before we run +the tests. + +@item examples/ +Example user programs for use starting with project 2. @item misc/ @itemx utils/ These files may come in handy if you decide to try working with Pintos -away from ITSS's Sun Solaris machines. Otherwise, you can ignore -them. +away from the ITSS machines. Otherwise, you can ignore them. @end table @node Building Pintos @subsection Building Pintos -The next thing to do is to try building the source code supplied for +As the next step, build the source code supplied for the first project. First, @command{cd} into the @file{threads} directory. Then, issue the @samp{make} command. This will create a @file{build} directory under @file{threads}, populate it with a @file{Makefile} and a few subdirectories, and then build the kernel inside. The entire build should take less than 30 seconds. -Watch the commands executed during the build. You should notice that -the build tools' names begin with @samp{i386-elf-}, e.g.@: +Watch the commands executed during the build. On the Linux machines, +the ordinary system tools are used. On a SPARC machine, special build +tools are used, whose names begin with @samp{i386-elf-}, e.g.@: @code{i386-elf-gcc}, @code{i386-elf-ld}. These are ``cross-compiler'' -tools. That is, the build is running on a Sparc machine (called the -@dfn{host}), but the result will run on an 80@var{x}86 machine (called -the @dfn{target}). The @samp{i386-elf-@var{program}} tools, which -reside in @file{/usr/class/cs140/i386/bin}, are specially built for -this configuration. +tools. That is, the build is running on a SPARC machine (called the +@dfn{host}), but the result will run on a simulated 80@var{x}86 machine +(called the @dfn{target}). The @samp{i386-elf-@var{program}} tools are +specially built for this configuration. Following the build, the following are the interesting files in the @file{build} directory: @@ -160,7 +172,7 @@ Following the build, the following are the interesting files in the @table @file @item Makefile A copy of @file{pintos/src/Makefile.build}. It describes how to build -the kernel. @xref{Adding c or h Files}, for more information. +the kernel. @xref{Adding Source Files}, for more information. @item kernel.o Object file for the entire kernel. This is the result of linking @@ -171,17 +183,17 @@ single object file. It contains debug information, so you can run @item kernel.bin Memory image of the kernel. These are the exact bytes loaded into memory to run the Pintos kernel. To simplify loading, it is always -padded out with zero bytes so that it is an exact multiple of 4 kB in +padded out with zero bytes up to an exact multiple of 4 kB in size. @item loader.bin Memory image for the kernel loader, a small chunk of code written in assembly language that reads the kernel from disk into memory and -starts it up. It is exactly 512 bytes long. Its size is fixed by the +starts it up. It is exactly 512 bytes long, a size fixed by the PC BIOS. @item os.dsk -Disk image for the kernel, simply @file{loader.bin} followed by +Disk image for the kernel, which is just @file{loader.bin} followed by @file{kernel.bin}. This file is used as a ``virtual disk'' by the simulator. @end table @@ -194,84 +206,282 @@ recompiled when other source or header files are changed. @node Running Pintos @subsection Running Pintos -To start the kernel that you just built in the Bochs simulator, first -@command{cd} into the newly created @file{build} directory. Then -issue the command @code{pintos run}. This command will create a -@file{bochsrc.txt} file, which is needed for running Bochs, and then -invoke Bochs. - -Bochs opens a new window that represents the simulated machine's -display, and a BIOS message briefly flashes. Then Pintos boots and -runs a simple test program that outputs a few screenfuls of text. -When it's done, you can close Bochs by clicking on the ``Power'' -button in the window's top right corner, or rerun the whole process by -clicking on the ``Reset'' button just to its left. The other buttons -are not very useful for our purposes. +We've supplied a program for conveniently running Pintos in a simulator, +called @command{pintos}. In the simplest case, you can invoke +@command{pintos} as @code{pintos @var{argument}@dots{}}. Each +@var{argument} is passed to the Pintos kernel for it to act on. + +Try it out. First @command{cd} into the newly created @file{build} +directory. Then issue the command @code{pintos run alarm-multiple}, +which passes the arguments @code{run alarm-multiple} to the Pintos +kernel. In these arguments, @command{run} instructs the kernel to run a +test and @code{alarm-multiple} is the test to run. + +This command creates a @file{bochsrc.txt} file, which is needed for +running Bochs, and then invoke Bochs. Bochs opens a new window that +represents the simulated machine's display, and a BIOS message briefly +flashes. Then Pintos boots and runs the @code{alarm-multiple} test +program, which outputs a few screenfuls of text. When it's done, you +can close Bochs by clicking on the ``Power'' button in the window's top +right corner, or rerun the whole process by clicking on the ``Reset'' +button just to its left. The other buttons are not very useful for our +purposes. (If no window appeared at all, and you just got a terminal full of corrupt-looking text, then you're probably logged in remotely and X forwarding is not set up correctly. In this case, you can fix your X -setup, or you can use the @option{-v} option.) - -The text printed by Pintos inside Bochs probably went by too quickly -to read. However, you've probably noticed by now that the same text -was displayed in the terminal you used to run @command{pintos}. This -is because Pintos sends all output both to the VGA display and to the -first serial port, and by default the serial port is connected to -Bochs's @code{stdout}. You can log this output to a file by -redirecting at the command line, e.g.@: @code{pintos run > logfile}. - -The @command{pintos} program offers multiple options for running -Pintos. Specify these options on the command line @emph{before} the -@option{run} command. Use @code{pintos help} to see a list of the -options. You can select a simulator other than Bochs, although the -Leland systems only have Bochs installed. You can start the simulator -running a debugger (@pxref{i386-elf-gdb}). You can set the amount of -memory to give the VM. Finally, you can set up how you want VM output -to be displayed: use @option{-v} to turn off the VGA display, -@option{-t} to use your terminal window as the VGA display instead of -opening a new window, or @option{-s} to suppress the serial output to +setup, or you can use the @option{-v} option to disable X output: +@code{pintos -v -- run alarm-multiple}.) + +The text printed by Pintos inside Bochs probably went by too quickly to +read. However, you've probably noticed by now that the same text was +displayed in the terminal you used to run @command{pintos}. This is +because Pintos sends all output both to the VGA display and to the first +serial port, and by default the serial port is connected to Bochs's +@code{stdout}. You can log this output to a file by redirecting at the +command line, e.g.@: @code{pintos run alarm-multiple > logfile}. + +The @command{pintos} program offers several options for configuring the +simulator or the virtual hardware. If you specify any options, they +must precede the commands passed to the Pintos kernel and be separated +from them by @option{--}, so that the whole command looks like +@code{pintos @var{option}@dots{} -- @var{argument}@dots{}}. Invoke +@code{pintos} without any arguments to see a list of available options. +Options can select a simulator to use: the default is Bochs, but on the +Linux machines @option{--qemu} selects qemu. You can run the simulator +with a debugger (@pxref{gdb}). You can set the amount of memory to give +the VM. Finally, you can select how you want VM output to be displayed: +use @option{-v} to turn off the VGA display, @option{-t} to use your +terminal window as the VGA display instead of opening a new window +(Bochs only), or @option{-s} to suppress the serial output to @code{stdout}. -The @command{pintos} program offers commands other than @samp{run} and -@samp{help}, but we won't have any need for them until project 2. - -The Pintos kernel has its own command line that you can use to pass -options. These options must be specified @emph{after} the -@option{run} command. These options are not very interesting for now, -but you can see a list of them using the @option{-u} option, e.g.@: -@code{pintos run -u}. - -@node Pintos License -@section Pintos License - -Pintos is distributed under a liberal license that allows it to be -freely used, modified, and distributed. Students and others own their -own code and may use it for any purpose. In the context of Stanford's -CS 140 course, please respect the spirit and the letter of the honor -code by refraining from reading any homework solutions available -online or elsewhere. (Source code for other operating system kernels, -such as Linux or FreeBSD, is of course fair game.) - -There is NO WARRANTY for Pintos, not even for MERCHANTABILITY or -FITNESS FOR A PARTICULAR PURPOSE. - -Please refer to the @file{LICENSE} file at the top level of the Pintos -source distribution for details of license and lack of warranty. - -@node Pintos Trivia -@section Pintos Trivia - -The design of Pintos is inspired by Nachos, an instructional operating -system originally from UC Berkeley, and even uses a few pieces of -Nachos code. Pintos is different from Nachos in two important ways. -First, Nachos requires a host operating system such as Solaris, -whereas Pintos runs on real or simulated 80@var{x}86 hardware. -Second, Nachos is written in C++, whereas, like most real-world -operating systems, Pintos is written in C. - -The name ``Pintos'' was chosen for multiple reasons. First, it was -named for a Mexican food to reflect that it was inspired by Nachos. -Second, Pintos is small and a ``pint'' is a small quantity. Third, -like drivers of the eponymous car, students are likely to have trouble -with blow-ups. +The Pintos kernel has commands and options other than @command{run}. +These are not very interesting for now, but you can see a list of them +using @option{-h}, e.g.@: @code{pintos -h}. + +@node Debugging versus Testing +@subsection Debugging versus Testing + +When you're debugging code, it's useful to be able to be able to run a +program twice and have it do exactly the same thing. On second and +later runs, you can make new observations without having to discard or +verify your old observations. This property is called +``reproducibility.'' The simulator we use by default, Bochs, can be set +up for +reproducibility, and that's the way that @command{pintos} invokes it +by default. + +Of course, a simulation can only be reproducible from one run to the +next if its input is the same each time. For simulating an entire +computer, as we do, this means that every part of the computer must be +the same. For example, you must use the same command-line argument, the +same disks, the same version +of Bochs, and you must not hit any keys on the keyboard (because you +could not be sure to hit them at exactly the same point each time) +during the runs. + +While reproducibility is useful for debugging, it is a problem for +testing thread synchronization, an important part of most of the projects. In +particular, when Bochs is set up for reproducibility, timer interrupts +will come at perfectly reproducible points, and therefore so will +thread switches. That means that running the same test several times +doesn't give you any greater confidence in your code's correctness +than does running it only once. + +So, to make your code easier to test, we've added a feature, called +``jitter,'' to Bochs, that makes timer interrupts come at random +intervals, but in a perfectly predictable way. In particular, if you +invoke @command{pintos} with the option @option{-j @var{seed}}, timer +interrupts will come at irregularly spaced intervals. Within a single +@var{seed} value, execution will still be reproducible, but timer +behavior will change as @var{seed} is varied. Thus, for the highest +degree of confidence you should test your code with many seed values. + +On the other hand, when Bochs runs in reproducible mode, timings are not +realistic, meaning that a ``one-second'' delay may be much shorter or +even much longer than one second. You can invoke @command{pintos} with +a different option, @option{-r}, to set up Bochs for realistic +timings, in which a one-second delay should take approximately one +second of real time. Simulation in real-time mode is not reproducible, +and options @option{-j} and @option{-r} are mutually exclusive. + +On the Linux machines only, the qemu simulator is available as an +alternative to Bochs (use @option{--qemu} when invoking +@command{pintos}). The qemu simulator is much faster than Bochs, but it +only supports real-time simulation and does not have a reproducible +mode. + +@node Grading +@section Grading + +We will grade your assignments based on test results and design quality, +each of which comprises 50% of your grade. + +@menu +* Testing:: +* Design:: +@end menu + +@node Testing +@subsection Testing + +Your test result grade will be based on our tests. Each project has +several tests, each of which has a name beginning with @file{tests}. +To completely test your submission, invoke @code{make check} from the +project @file{build} directory. This will build and run each test and +print a ``pass'' or ``fail'' message for each one. When a test fails, +@command{make check} also prints some details of the reason for failure. +After running all the tests, @command{make check} also prints a summary +of the test results. + +For project 1, the tests will probably run faster in Bochs. For the +rest of the projects, they will probably run faster in qemu. + +You can also run individual tests one at a time. A given test @var{t} +writes its output to @file{@var{t}.output}, then a script scores the +output as ``pass'' or ``fail'' and writes the verdict to +@file{@var{t}.result}. To run and grade a single test, @command{make} +the @file{.result} file explicitly from the @file{build} directory, e.g.@: +@code{make tests/threads/alarm-multiple.result}. If @command{make} says +that the test result is up-to-date, but you want to re-run it anyway, +either run @code{make clean} or delete the @file{.output} file by hand. + +By default, each test provides feedback only at completion, not during +its run. If you prefer, you can observe the progress of each test by +specifying @option{VERBOSE=1} on the @command{make} command line, as in +@code{make check VERBOSE=1}. You can also provide arbitrary options to the +@command{pintos} run by the tests with @option{PINTOSOPTS='@dots{}'}, +e.g.@: @code{make check PINTOSOPTS='--qemu'} to run the tests under +qemu. + +All of the tests and related files are in @file{pintos/src/tests}. +Before we test your submission, we will replace the contents of that +directory by a pristine, unmodified copy, to ensure that the correct +tests are used. Thus, you can modify some of the tests if that helps in +debugging, but we will run the originals. + +All software has bugs, so some of our tests may be flawed. If you think +a test failure is a bug in the test, not a bug in your code, +please point it out. We will look at it and fix it if necessary. + +Please don't try to take advantage of our generosity in giving out our +test suite. Your code has to work properly in the general case, not +just for the test cases we supply. For example, it would be unacceptable +to explicitly base the kernel's behavior on the name of the running +test case. Such attempts to side-step the test cases will receive no +credit. If you think your solution may be in a gray area here, please +ask us about it. + +@node Design +@subsection Design + +We will judge your design based on the design document and the source +code that you submit. We will read your entire design document and much +of your source code. + +We provide a design document template for each project. For each +significant part of a project, the template asks questions in four +areas: data structures, algorithms, synchronization, and rationale. An +incomplete design document or one that strays from the template without +good reason may be penalized. Incorrect capitalization, punctuation, +spelling, or grammar can also cost points. @xref{Project +Documentation}, for a sample design document for a fictitious project. + +Design quality will also be judged based on your source code. We will +typically look at the differences between the original Pintos source +tree and your submission, based on the output of a command like +@code{diff -urpb pintos.orig pintos.submitted}. We will try to match up your +description of the design with the code submitted. Important +discrepancies between the description and the actual code will be +penalized, as will be any bugs we find by spot checks. + +The most important aspects of design quality are those that specifically +relate to the operating system issues at stake in the project. For +example, the organization of an inode is an important part of file +system design, so in the file system project a poorly designed inode +would lose points. Other issues are much less important. For +example, multiple Pintos design problems call for a ``priority +queue,'' that is, a dynamic collection from which the minimum (or +maximum) item can quickly be extracted. Fast priority queues can be +implemented many ways, but we do not expect you to build a fancy data +structure even if it might improve performance. Instead, you are +welcome to use a linked list (and Pintos even provides one with +convenient functions for sorting and finding minimums and maximums). + +Pintos is written in a consistent style. Make your additions and +modifications in existing Pintos source files blend in, not stick out. +In new source files, adopt the existing Pintos style by preference, but +make the self-consistent at the very least. Use horizontal and vertical +white space to make code readable. Add a comment to every structure, +structure member, global or static variable, and function definition. +Update existing comments as you modify code. Don't comment out or use +the preprocessor to ignore blocks of code. Use assertions to document +key invariants. Decompose code into functions for clarity. Code that +is difficult to understand because it violates these or other ``common +sense'' software engineering practices will be penalized. + +In the end, remember your audience. Code is written primarily to be +read by humans. It has to be acceptable to the compiler too, but the +compiler doesn't care about how it looks or how well it is written. + +@node License +@section License + +Pintos is distributed under a liberal license that allows free use, +modification, and distribution. Students and others who work on Pintos +own the code that they write and may use it for any purpose. + +In the context of Stanford's CS 140 course, please respect the spirit +and the letter of the honor code by refraining from reading any homework +solutions available online or elsewhere. Reading the source code for +other operating system kernels, such as Linux or FreeBSD, is allowed, +but do not copy code from them literally. Please cite the code that +inspired your own in your design documentation. + +Pintos comes with NO WARRANTY, not even for MERCHANTABILITY or FITNESS +FOR A PARTICULAR PURPOSE. + +The @file{LICENSE} file at the top level of the Pintos source +distribution has full details of the license and lack of warranty. + +@node Acknowledgements +@section Acknowledgements + +Pintos and this documentation were written by Ben Pfaff +@email{blp@@cs.stanford.edu}. + +The original structure and form of Pintos was inspired by the Nachos +instructional operating system from the University of California, +Berkeley. A few of the source files were originally more-or-less +literal translations of the Nachos C++ code into C. These files bear +the original UCB license notice. + +A few of the Pintos source files are derived from code used in the +Massachusetts Institute of Technology's 6.828 advanced operating systems +course. These files bear the original MIT license notice. + +The Pintos projects and documentation originated with those designed for +Nachos by current and former CS140 teaching assistants at Stanford +University, including at least Yu Ping, Greg Hutchins, Kelly Shaw, Paul +Twohey, Sameer Qureshi, and John Rector. If you're not on this list but +should be, please let me know. + +Example code for condition variables (@pxref{Condition Variables}) is +from classroom slides originally by Dawson Engler and updated by Mendel +Roseblum. + +@node Trivia +@section Trivia + +Pintos originated as a replacement for Nachos with a similar design. +Since then Pintos has greatly diverged from the Nachos design. Pintos +differs from Nachos in two important ways. First, Pintos runs on real +or simulated 80@var{x}86 hardware, whereas Nachos runs as a process on a +host operating system. Second, like most real-world operating systems, +Pintos is written in C, whereas Nachos is written in C++. + +Why the name ``Pintos''? First, like nachos, pinto beans are a common +Mexican food. Second, Pintos is small and a ``pint'' is a small amount. +Third, like drivers of the eponymous car, students are likely to have +trouble with blow-ups.