Boot Camp part of Teaching by Philip Greenspun

• You can try this at home, with our home study program. If you join ArsDigita after completing it, you'll earn a $10,000 signing bonus.
• You can try this at home connecting to our server via our ongoing remote boot camp: We will provide an account on one of our Oracle-backed machines and guide you through our problem sets. Most of the work will be done remotely, but we can schedule a time for you to visit and work with us for a few days.
• To check the schedule and apply, visit http://arsdigita.com/events/.

Background: This material is generally taught to computer science seniors at MIT. You should have at least some programming background before attempting the boot camp. It is also helpful to have some mechanical skills (driving a Unix machine, editing files in Emacs; you can get these by installing Linux on a PC and playing around).

Objectives

• enough about Unix that they don't impose too great a load on the professional sysadmins
• enough about Oracle that they don't impose too great a load on the professional database administrators
• (in the glorious long version) enough about our server architecture and procedures that they can keep a service up and running 24x7

Gross Schedule

1. short: three weeks; three problem sets on development systems that are set up in advance by professionals (i.e., the students don't learn about Unix and Oracle installation)
2. medium: five weeks; setting up a development system and then doing three problem sets
3. long: twelve weeks

• Week 1: Unix sysadmin. Install Solaris on a bare Ultra 5.
• Week 2: Oracle dba. Install Oracle 8i on the Ultra 5.
• Week 3: Problem set 1 from 6.916: learning the tools and Web fundamentals.
• Week 4: Problem set 2 from 6.916: getting familiar with the ACS (our toolkit). Billing credit cards.
• Week 5: Problem set 3 from 6.916: understanding content management.
• Week 6: A new problem set designed to teach students about online community extensions.
• Week 7: A new problem set entirely about ecommerce (touched on in Week 4).
• Week 8: ArsDigita Server Architecture and troubleshooting down Web services.
• Weeks 9-12: building a site for a real client (e.g., a non-profit group).

Textbooks

• Visual Explanations : Images and Quantities, Evidence and Narrative (Edward Tufte 1997; Graphics Press; order directly by calling (800) 822-2454)
• Philip and Alex's Guide to Web Publishing
• Practical Programming in Tcl and Tk (Brent Welch 1997; Prentice-Hall)
• Oracle 8: The Complete Reference
• Unix Power Tools (1 copy for every 4 students)

  Web-only

  (must be printed with Hewlett-Packard Web PrintSmart)
  
  
• SQL for Web Nerds
• ArsDigita Server Architecture

• red book and Essential Sysadmin; http://docs.sun.com has answerbook online (very very slow, though). Good chunk of the stuff is available in .html and we can put it up.
• Unix Power Tools (Peek 1997; O'Reilly)
• Oracle DBA Handbook (Loney 1998; Osborne McGraw-Hill).

Week 1

• install Solaris right up through CDE
• install Netscape either from CD-ROM or from FTP
• 1 hour: adding and removing accounts (useradd, usermod; also show AdminTool)
• install pre-built gcc
• build fresh gcc
• build GNU Emacs 20.x
• install pre-built tclsh 7.4
• install pre-built bash, tcsh, zsh
• install pre-built Perl
• build ssh
• build ImageMagick and then add JPEG, PNG, TIFF libraries
• install AOLserver
• install restart-aolserver (*** Scott is going to look at this to see if we need setuid Perl)
• build qmail configure and restart machine to make sure it comes back with qmail up
• install keepalive
• install rollover (to roll error log)
• install watchdog
• mirror disks
• backups: (1) create a tar backup to disk of just the .tcl files in a /web dir, then restore one file, (2) create a UFSDUMP of a partition to a Unix file, (3) restore one file from that UFSDUMP
• security: what to worry about, scripts to help deal with

Note: Scott Blomquist is available Monday, Tuesday, and Friday afternoons.

Weekend

• Chapter 12 of Philip and Alex's Guide to Web Publishing, available at http://www.arsdigita.com/books/panda/databases-choosing.html ("Database Management Systems").
• Oracle8 DBA Handbook: Chapters 1-5,7, 9, 10
• as needed: Oracle 8i Concepts (official Oracle doc)

1. the basic structure of Oracle
2. how Oracle looks to Unix: N processes and what each one does
3. how to order disk drives to support Oracle and then how to use them
4. how a table is stored in Oracle: segments and extents
5. what the rollback segment is and how it is used
6. users, roles, and permissions
7. preparing for the worst: backup strategies
8. steps to making sure that you can do online backups

Week 2

Day 1 (afternoon): install Oracle 8i with Intermedia and SQL*Net, create a user called "web" in its own "web" tablespace (with one datafile called "web.dbf"). Tweak the Oracle installation until the acceptance tests in http://www.photo.net/wtr/oracle-tips.html will run.

Day 1 (at home): read chapters 13 and 14 of Oracle8 DBA Handbook

Day 2 (morning):

• reboot machine; make sure that Oracle shuts down automatically and starts up automatically; check Oracle error log to verify clean startup
• create a new user, "mrgreed", with the Solaris shell command useradd
• log out and log back in as "mrgreed" and make sure that "sqlplus" is in this user's path and that this user can successfully connect to Oracle upon typing the web username and password; if not, fix the machine config, drop the mrgreed user and start over from the "useradd" command.

Day 2 (afternoon):

• create a new Oracle user called "mrgreed" whose data goes into the USERS tablespace
• create a tab-separated file in Emacs containing five lines, each line to contain your favorite stock symbol, an integer number of shares owned, and a date acquired (in the form MM/DD/YYYY)
• create an Oracle table, owned by the mrgreed user, to hold these data:

  create table my_stocks (
         symbol	       varchar(20) not null,
         n_shares	       integer not null,
         date_acquired   date not null       
  );
  

• use the sqlload shell command to invoke SQL*Loader to slurp up your tab-separated file into the my_stocks table (see page 1183 of Oracle8: The Complete Reference and the official Oracle docs at http://philip.greenspun.com/sql/ref/utilities)
• with the help of the Oracle Net8 Administrator's Guide at http://philip.greenspun.com/sql/ref/net8, configure your server to talk to another student's and vice versa
• using the SQL*Plus copy command, documented on page 533 of Oracle8: The Complete Reference and at http://philip.greenspun.com/sql/ref/copycommand, copy the my_stocks table from your fellow student's server into a table named "stupid_stock_picks" on your own machine
• Use an SQL INTERSECT command to print a list of stocks that both of you picked

• create a table called "foobar" owned by your "web" user and fill it with a few rows of data
• start two SQL*Plus sessions, both connected as the mrgreed user to your local database
• type "delete from my_stocks;" in one session
• type "select * from my_stocks;" in both sessions
• type "rollback;" in the first session
• in a separate Unix shell, using the exp command, invoke the Oracle Export utility, documented at http://philip.greenspun.com/sql/ref/export, to create two consistent export files, one for the entire database (entire-db.dmp) and one for just those objects owned by the mrgreed user (mrgreed.dmp)
• type "delete from my_stocks where symbol = '**one of your symbols**';" and then "commit;"
• type "select * from my_stocks;" in both sessions
• start third SQL*Plus session, connected as the web user, and type "drop table foobar;"
• use the Oracle Import utility, documented at http://philip.greenspun.com/sql/ref/import, to recover the lost row from the my_stocks table using the mrgreed.dmp file and the dropped foobar table using the entire-db.dmp file.

Day 3 (afternoon): get Intermedia text search to work

Day 4 (morning, afternoon):

• develop a set of scripts to do online backups with archived redo logs that are point-in-time recoverable
• create a table owned by your web user:

  create table frequently_updated (
         time_of_update		date not null,
         freespace_for_web	integer not null
  );
  

• create a PL/SQL procedure, record_freespace that will (1) query the blocks column in the dba_free_space view to figure out how many free blocks are in the Web user's data file (you might have to connect as dba and grant select on this view to the web user; (2) insert a row into frequently_updated; (3) commit the update.
• set up your Oracle instance to have one SNP background processes (see http://philip.greenspun.com/sql/ref/jobs), with a job queue interval of 10 seconds
• restart Oracle
• schedule a DBMS Job to run record_freespace every 30 seconds
• verify that record_freespace is growing by one row every 30 seconds
• make an online backup of your database (i.e., while record_freespace is being filled)
• run dbverify on the backup files that you've created
• recover the database to any point in time during the middle of that online backup and verify that record_freespace in fact contains entries right up to the recovered time and none after.

Day 5 (morning): Tuning. Make sure that the web user can run "set autotrace on". Require use of analyze table compute statistics.

Day 5 (afternoon): deadlocks, integrity constraint violations (finding and dropping integrity constraints), Helpful: Chapter 32 of Oracle8: The Complete Reference

Day 5 (late afternoon): compile AOLserver Oracle driver and connect your AOLserver to Oracle as the web user

Weekend

• Philip and Alex's Guide to Web Publishing, Chapters 1, 4, 5, 10, 11
• Practical Programming in Tcl and Tk (remember that we only use Tcl, not Tk, so limit yourself to chapters 1-11 and 13; remember also that this book is about Tcl 8 and AOLserver has Tcl 7.4 compiled in, so you can't use the clock command or event loops or some other things).
• SQL for Web Nerds, chapters 1-5, 8

Week 3

• 9:00-10:30: lecture
• 10:30-2:30: supervised work on Problem Set 1
• 2:30-4:00: lecture
• 4:00-5:00: supervised work on Problem Set 1

Tuesday morning and afternoon lectures: Common errors will be user to further introduce the programming environment.


Wednesday morning lecture: Structured approach to building SQL queries
Wednesday evening reading:

Reading from Visual Explanations:

• Space Shuttle Challenger example: Pages 38-53
• Smallest effective difference: Page 74
• Multiples in Space and Time: Pages 105-119, esp. the Tufte medical chart on p. 111
• Everything important about Web design: Pages 146-196

Friday: Full day lecture at MIT

Week 4

Week 5

Doing a short boot camp on-site somewhere

Week -2: professional system administrators configure student development machines with the Unix operating system and Oracle8i. There should be one development machine for every 8 students. This is partly to work around a bug in Oracle 8.1.5 (won't run on a machine with more than 10 IP addresses). Once the basic stuff in the LCS computing facility document is set up, you need to

• get 10 IP addresses per machine
• define a bunch of hostnames like "st01.yourorg.com" through "st40.yourorg.com"
• config each Unix box to listen on either 9 or all 10 (depending on whether Oracle will start) of the allocated IP addresses
• create 10 Unix shell accounts on each machine, named "st01" through "st10", "st11" through "st20", etc.
• create 10 Oracle tablespaces on each machine, each named "stN" with one 40 MB autoextending data file "stN.dbf"

  
  create tablespace st13 datafile '/opt/oracle/u02/oradata/ora8/st13.dbf' size 40m autoextend on;
  

• create 10 Oracle users on each machine, each named "st01" up...

  
  create user st13 identified by st13 
  default tablespace st13
  quota unlimited on st13;
  

• grant permissions to all the Oracle users:

  
  grant connect, resource to st13;
  

• test:
  1. Log in as st13
  2. type "sqlplus st13/st13"
  3. type "create table foobar (yow integer);"
  4. type "drop table foobar;"
• create 10 Web server roots, one for each student: /web/stN/www/ and /web/stN/tcl/ (/web is typically a symlink over to /home)
• create 10 AOLserver .ini files, stN.ini, each one binding to only the IP address and hostname corresponding to that student's host and connecting to Oracle as the right user. Each one should have its page root at /web/stN/www and its private Tcl directory at /web/stN/tcl/
• make 10 entries in /etc/inittab so that these servers start automatically
• test "restart-aolserver st13" (script available from http://arsdigita.com/free-tools/)

Week -2: boot camp sponsor orders textbooks and prints Web-based materials.

Week -2: boot camp sponsor verifies that ssh connections can be made through their firewall to baden.ai.mit.edu, trying to log in as "foo" password "bar" and getting "login incorrect".