|
Posted-By: auto-faq 3.1.1.2
Archive-name: 386bsd-faq/part4 See reader questions & answers on this topic! - Help others by sharing your knowledge Section 3. (Kernel Building and Maintenance) 3.0 System Internals One of the interesting aspects of *BSD is the fact that it comes with the complete source. This allows you to make changes to the system, recompile, and test out your new ideas. This section of the FAQ describes many of the different aspects of this endeavor and common problems and pitfalls that are encountered. Kevin Lahey provided the substantial portion of this section. You can contact him via E-Mail at (kml@rokkaku.atl.ga.us) or contact Dave Burgess (burgess@cynjut.neonramp.com). 3.1 Kernel 3.1.1 How do I build a kernel? The kernel can be compiled in a variety of ways to support different devices and configurations. Compilation is controlled by a config file that specifies the characteristics of the kernel. A set of different config files is located in /sys/i386/conf or /sys/arch/i386/conf. The configuration file names are in upper case. To build a particular kernel (in this example, we use the GENERICISA configuration file in NetBSD or FreeBSD): % cd /sys/i386/conf % config GENERICISA % cd /sys/compile/GENERICISA % make depend % make In NetBSD, since there are multiple architectures supported, there is an architecture line in the middle of the path to these files. See the build.kernel script in section 3.8 for more information. Remember, when structures in the kernel change, there are some programs (ps, top, etc.) that will cease to work correctly. You will need to recompile these programs as well as the new kernel. You need to do the following to make sure that your programs get updated as well as the kernel: cd /usr/src/include make install cd /usr/src/lib/libkvm make clean && make && make install cd /usr/src/bin/ps make clean && make && make install cd /usr/src/... 3.1.1.1 Why does the kernel code for NetBSD still use the old K&R style declarations when the ANSI declarations are SO much better? 3.1.1.2 How do I port NetBSD to another platform? We still use the old style K&R definitions easier, because bringing up a new port on a foreign machine with a brain-damaged compiler can be impossible, or at least very difficult, if you don't do it this way. Remember, NetBSD is multi-platform, and tries to make it as easy as possible to port. Which means building pieces of the system with someone else's compiler. 3.1.2 I want to do one of the following things: * add a device not in the distributed kernel (third com port, additional disk or tape, line printer driver, etc). * use a patch from the net or the patchkit to fix a kernel bug. * add another swap device. * recompile the kernel to remove extraneous devices so that it takes up less space. * configure more pseudo-terminals to allow for more xterms or network logins. You're going to have to recompile the kernel after you modify the config file. See section 3.2 below for more information about the config file in general. 3.1.3 I want to build and profile a kernel. What do I need to do? Step 1 is to build a profiled kernel: cd /sys/arch/<X>/conf config -p CONFIG.FILE.OF.YOUR.CHOICE cd /sys/arch/<X>/compile/CONFIG.FILE.OF.YOUR.CHOICE make depend make <install the kernel and reboot> Step 2 is to start the profiling process: log in su to root kgmon -r -h kgmon -b Step 3 is to run the application in question Step 4 is to stop the profile: kgmon -h kgmon -p (this produces the gmon.out file, which is the profiles kernel information) Step 5 is to analyze the output: gprof /{kernel name} gmon.out > profile which produces a hierarchical call graph as well as flat profile. The flat profile is easiest to use for beginners, although both are good information sources for kernel code performance. 3.1.4 Now that I have a kernel, how do I install it? Your kernel is called /kernel or /netbsd. Copy the new kernel from /sys/compile/GENERICISA/(whatever) to /, assuming that it is in that directory. If you really screw up the new kernel, you want to have something to fall back on, so be sure to save /kernel to /kernel.old before copying in a new kernel. 3.1.5 My system is complaining about stray interrupt 7. Is my machine going to explode or anything? No. They are caused by lots of things. They are, as far as anyone that should be expected to know about this stuff, harmless. There are ramifications on them being there, but for MOST users they do not pose a real threat to your operations. For those of you that are doing REALLY interrupt intensive stuff, you may want to grab a technical reference and figure out why the 8259 is not getting reset correctly. In spite of the number of times this has come up (and people have even referenced this section) there are still at least two questions on the net about this. A memorable one was a guy who was just vehement that the stray int 7 was what was keeping his system from booting. In fact, he went so far as to say that this document was practically worthless because I didn't tell him how to fix it. Of course, once he configured his hard drive controller so that it was on the right interrupt, his booting problem went away. I have said it before and I will say it again. For MOST users they do not pose a real threat to your operations. I have heard of three people (out of at least 2000) that have actually have problems so bad that they couldn't proceed. They bought new computers, and the problem went away. These stray interrupts are caused by something in the PC. I have yet to see a convincing explanation of precisely what, but they are definitely caused by something. There are four ways to deal with this problem. 1) Ignore them. They are spurious and do not effect the operation of your computer. 2) Implement the lpt driver. This way, the driver traps (the lpt driver expects IRQ 7) and then quietly discards them. That is why when folks implement the lpt driver the 'problem' goes away. The computer is taught how to ignore them. 3) Do what the original 386bsd code did. Comment out the diagnostic and associated code that tries to deal with them so you don't see the error message. 4) Buy a new computer that doesn't cause this problem. It is a known hardware problem with the 8259 being reset incorrectly in hardware. Kalevi Suominen (jks@geom.helsinki.fi) offers this technical explanation of the 'stray interrupt 7' phenomenon. In the section of the Intel Peripheral Handbook dealing with the 8259A there is a description of the 6-step interrupt sequence for an 80x86 system (and 7-step for an MCS-80/85), and then the following paragraph: "If no interrupt request is present at step 4 of either sequence (i.e., the request was too short in duration) the 8259A will issue an interrupt level 7. Both the vectoring bytes and the CAS lines will look like an interrupt level 7 was requested." This explains how some transient disturbances or improperly functioning adapter cards could trigger a stray interrupt 7 in a system operating in the *level* interrupt mode (such as a PS/2 with MCA): An interrupt request will disappear as soon as the interrupt line goes inactive. That such interrupts may also occur in a system operating in the *edge* triggered mode (such as an ordinary PC/AT with ISA) is a little harder to see. Yet it is possible - even without malfunctioning hardware - because masking an interrupt request will hide its presence from the 8259A as well: 1. The interrupt flag (IF) of the 80x86 is reset either directly (e.g., by a "cli" instruction) or because an interrupt handler is entered. In the latter case the corresponding in-service (IS) bit of the 8259A is set (effectively blocking interrupts of lower priority). 2. The 8259A receives an unmasked interrupt request (IRQn), and, in case an interrupt is being served and has higher priority than IRQn, the IS bit of the 8259A is reset by an end of interrupt (EOI) command. (These steps may occur in either order.) If IRQn has higher priority (e.g. IRQ0), no EOI is necessary. 3. The 8259A activates the interrupt (INT) line to the 80x86 (which will ignore it - for the moment). 4. The interrupt mask (IM) bit of the 8259A for IRQn is set. (A little late, though. The sequence has already started.) 5. The interrupt flag (IF) of the 80x86 is set (either directly, or as a consequence of e.g. an "iret" instruction). 6. The 80x86 will now acknowledge the INT request by activating the INTA line of the 8259A. 7. The 8259A will not see the masked IRQn and will continue by issuing a spurious interrupt of level 7 instead. The original interrupt request (IRQn) will not be lost, however. It is preserved by the associated edge sense latch of the 8259A, and will be acted on after the IM bit has been reset again. The net result is that a single interrupt request will be delivered *twice* to the 80x86: first as a stray interrupt 7 and later as the proper interrupt. In particular, it is perfectly safe to ignore the stray interrupt (other than sending an EOI). It is just the ghost of an aborted interrupt sequence: the system was not prepared for it yet. Bill Roman provides this update, which is so technical I can't even figure out what to cut out or how to repackage it so it makes sense to people like me. As an interesting aside, he is also a Linux user; thereby proving that I'll accept help the FAQ from everyone: First of all, Kalevi Suominen's explanation is correct, but requires just a little more explanation. Step 4 in the data book concerns the 8259's detection of INTA (interrupt acknowledge) asserted by the processor. This means that the processor has detected INT (interrupt request) asserted by the 8259, the previous instruction has ended, and the interrupt enable flag is true. The processor has begun its interrupt processing, and the 8259 *must* supply a vector; there is no way for it to tell the processor "never mind". INTA causes the 8259 to examine the currently asserted interrupt requests and return the vector corresponding to the highest current request. If there is no longer any interrupt request asserted, it supplies vector 7 as a default. (Intel calls this "DEFAULT IR7" later in the data book.) There is thus a race condition between deassertion of an interrupt request and interrupt servicing by the processor. An interrupt request which is removed will not always cause DEFAULT IR7 -- only if the request is deasserted after the processor has detected INT and irrevocably decided to act on it, but before the 8259 has detected INTA and prioritized its interrupts. It is easy to see how this could happen when the 8259 is in level triggered mode, but it is counterintuitive that it should happen in edge triggered mode. To understand this, it is necessary to look at Intel's "Priority Cell--Simplified Logic Diagram" (figure 9 in a 1991 databook I have at hand). The edge sense latch output is gated by the request; if the request is latched, then deasserted, the 8259 no longer sees it. There must be a reason Intel did it this way, but it's sure not evident to me. The data book provides a little more information in a later section titled "Edge and Level Triggered Modes". The most important point is that the corresponding bit in the In Service Register is *not* set when the 8259 generates a DEFAULT IR7. If the IRQ 7 interrupt service routine sees this condition (i.e., is entered and ISR bit 7 is zero) it should simply execute an interrupt return instruction without sending an End of Interrupt (EOI) command to the 8259. Also, the IRQ 7 interrupt service routine can be reentered due to this condition. Intel recommends that the interrupt service routine keep track of whether it is currently active so this can be detected. I don't think that changing the interrupt mask bit can cause the problem, especially if it is changed while the interrupt flag is clear. As I mentioned, the problem occurs only when the actual interrupt acknowledge process has begun, which can't happen while IF is clear. What can generate DEFAULT IR7 is a device driver that doesn't mask off its device's interrupt (either in the 8259 or in the device itself) while it is performing operations which can cause the device to deassert its interrupt request. For example: imagine a hypothetical device with an interrupt status register. This register latches all the device's status which could cause an interrupt, and clears this status when it is read. If the processor begins executing the instruction which reads this register just as a status bit is set, the device will assert and deassert the request within the space of that instruction. On an x86 architecture processor I have worked with, this did indeed generate a DEFAULT IR7. All device drivers should make sure that the device's interrupt request is disabled while the device is being serviced. A well-behaved device will never deassert its request without explicit software action. This leaves only the poor folks who have had to buy new computers to get rid of the problem. My suspicion in this case is that crosstalk on the mainboard is causing glitches on interrupt request lines. Remember that the f**king wizards at IBM made the request lines active high instead of active low with a pull-up (which would have allowed wire-or interrupt sharing). Some devices (some serial port cards, I believe) use a tri-state driver to drive the request high, but disable the driver entirely when the request is deasserted, counting on a weak pull-down. This leaves interrupt request lines floating, although the 8259 has the inputs enabled. This is all conjecture, though. Provided by: Bill Roman (roman@songdog.eskimo.com) 3.1.6 I keep getting "wd0c: extra interrupt". What does it mean? It means that the drive was already processing a command (active) when it received an interrupt from the system telling it to see if it had anything to do. This is mostly harmless but could indicate that the drive/controller is having problems if the message appears often. 3.1.7 I keep getting silo overflow messages, but the system doesn't seem to mind. Is there a problem? Not exactly. This simply means that the First in first out buffer is getting too full. I markedly reduced the incidence of silo overflows on my system by editing dev/isa/com.c to change the FIFO threshold from 8 to 4 characters. This way, the buffer gets more attention and reduces the chance of overflowing the buffer. Another possibility is caused when you are using internet over a telephone line via SLIP or PPP. You might have an unbuffered UART on your serial port, like the 8250 or the 16450. With the introduction of the IBM PS/2 systems the first 16550 UART's were shipped to provide a 16 byte buffer. But unfortunately the buffering of the original 16550 did not work. The problem was solved with the improved 16550A UART. If you run MSD (under MSDOS!), an application that comes with MS-Windows, you can determine the UART type of your serial ports (by choosing COMS). The UART type is also showed when your UNIX boots up. To solve the `silo overflow' problem you can by a Multi/IO card with a `real' 16550A, or a card with an extra serial port, like the HAYES ESP card. The Hayes card has a DOSSETUP utility to configure its port address and IRQ. The port address can be chosen between 180H and 380H, the IRQ address 3, 4, 5 or 9. Normally COM1 (tty00) and COM2 (tty01) claim IRQ 3 and 4. So you can choose (for example) IRQ 9 for the Hayes ESP card. Then you have to add the appropriate lines in your kernel configuration: options COM_HAYESP device com0 at isa? port "IO_COM1" irq 4 device com1 at isa? port "IO_COM2" irq 3 # com2: Hayes ESP serial port device com2 at isa? port "IO_COM3" irq 9 For more information on com ports in general, try this URL: http://comminfo.com/pages/ctsfaq01.htm#q6. 3.1.8 I found a bug in the kernel. How do I report it? Both NetBSD and FreeBSD include a facility called 'bugfiler'. While the instructions are included in both system, there is still some apparent confusion about when to use (and when to NOT use) bugfiler. Jordan K. Hubbard (jkh@whisker.lotus.ie) provides us with this short article for FreeBSD. To send bug reports, you want to use the sendbug(1) command. The entire package for sending and filing these bugs is known as "the bugfiler", which is where the confusion stepped in, but sendbug is definitely the command you want to use. Second, it doesn't take a "net connection" to use sendbug, since all it does is package up your "bug report form" and mail it to us; no direct Internet connectivity is required, just mail. So if you can send Internet mail you can use sendbug, or you can also send mail to the `FreeBSD-bugs@freefall.cdrom.com' address (do NOT send it to FreeBSD.cdrom.com since it will BOUNCE, this is not the place to send bugs to, just to ftp stuff from!). NetBSD has a similar facility, but has a different program and host for bug reports. The program for NetBSD is called send-pr and is slightly different in several respects. It is part of the GNATS system, which the NetBSD core developers started using in February of 1994. It is recommended that NetBSD users see the man page on send-pr before filing bug reports. For getting information from GNATS, see the file doc/BUGS. There is a email interface to the NetBSD PR database. To query the database send mail to "query-pr@gnats.netbsd.org". The mail server will send a bug database listing back to the user. There are several flags that are useful to send to the mail server. The flags are entered in the "Subject" line: --summary Display an one-line entry for each bug --full Display the full entry for each bug --help Display a help string containing the rest of the flags. PR The Problem Report number of a particular bug For example, to send a query about all the bugs: $ Mail -s "--summary" query-pr@gnats.netbsd.org < /dev/null If you want to know more about a particular bug, let's say bug 40: $ Mail -s "--full 40" query-pr@gnats.netbsd.org < /dev/null John Conklin is trying to get a page set up at the NetBSD WWW site (www.netbsd.org) that will allow people to interactively query the bug database. It should be operational soon. 3.2 What exactly is this config file, anyway? What are all of these cryptic notations? The config file is the list of all of the optional (and settings for the mandatory) parts of the kernel. If the system is made up of static object files which don't change, then all you should ever need to do is modify the config file, reconfigure the kernel objects, and relink. Since both NetBSD and FreeBSD are distributed with source, these files are recompiled and a kernel is constructed. Some of these have been deprecated, and may not be in use for a particular version of the system (i.e. ISO9660 and CD9660 are the same, CD9660 being the newer version). 3.2.1 Okay, fine. Why shouldn't I just add every device I can find to the kernel, so I'll never have to recompile this again? Because it takes up space. The kernel is wired into memory, so every byte it uses comes out of the pool of memory for everything else. It can't page out sections that aren't in use. If your kernel is larger than 640K, then it can't be loaded. You'll need to use Julian Elischer's bootblocks to put it in high memory, which seem to be fairly complex. Installing them (once they are compiled) is as easy as using disklabel. Newer versions of the *BSD kith provide the capability to build a kernel that is larger than 640K. Complete instructions are provided in the appropriate systems. 3.2.2 What should I remove from the kernel? What do you need? If you only have an SCSI controller, you don't need the wd0 device; if you have another kind of disk controller, you don't need sd0. Unless you actually HAVE more than one Ethernet controller, you should comment out all but one of them. If you don't have an ethernet controller, you don't need any of the controllers or NFS compiled in. Without a CD-ROM, ISOFS is kind of pointless. Just look at what you have and think about what you really need. 3.2.3 I can't get enough remote login sessions or xterm sessions. I also can only get four sessions working at a time. What can I do? Increase the count of pseudo-terminals -- pseudo-device pty 12 # or whatever Every pseudo terminal should have a /dev/pty* entry. If you have 12 pseudo terminals, you should also have at least 12 pty devices in the /dev directory. The MAKEDEV script in /dev will create as many pseudo- terminals as you tell it to. 3.2.4 How do I get ddb, the kernel debugger, compiled into the kernel and running? If you are using older versions of the 386BSD family, you will need to add a line in your config file that looks like this: device-pseudo ddb If you are using a more recent version (the division is pretty unclear about when the switch was made) and do not have any device-pseudo entries, you will need to add the line: options DDB to your config file. Build the kernel, then run dbsym on it: % dbsym ./kernel Install it and go for it. Ctl-Alt-Esc drops you into the debugger. Note: DDB as shipped originally is a memory hog, and it is very difficult to get a kernel small enough with enough fun things in it to debug in 640K On the NetBSD-sparc system, the L1-A is used by the the DDB routines to drop you into the debugger. 3.2.5 I'm getting all kinds of errors when I try to build a new version of GCC. How can I upgrade GCC to the most current version? Yes, this will happen on most architectures on the first compile of src/gnu/usr.bin/gcc/libgcc. As was stated in the mailing list before, when you get to this error: 1) run a 'make' in the gcc directory. It will error out (most likely) on libgcc. 1) Do a 'make install' at this point to install at least gcc, cpp, and cc1. 2) go back and compile in src/gnu/usr.bin/gcc/ WITHOUT doing a "make clean" 3) install everything in src/gnu/usr.bin/gcc 3.2.6 Can I patch the current running OS image? In general, I think, the answer is no. The prevailing philosophy seems to be that one should use sysctl for such things, but that requires that one has already compiled in the ability to change the specific variable in question. (I discovered this when I wanted to patch tickadj at runtime; I added it to kernfs, and when I offered the patches (which are quite small) I was told sysctl was the `correct' way. What's incorrect about /kern was never quite explained; the closest anyone came was to invoke internationalization concerns. Of course, using /kern also requires having compiled in support for tweaking the variable you want to change.) Besides, unless you've patched securelevel, I don't think there is any good way to twiddle variables in the running kernel. /dev/{,k}mem are, I believe, read-only once init sets securelevel to 1. If you need to know more about the sysctl command, try "sysctl -a | more" to get a list of the parameters that can be modified while the system is running. 3.2.7 Can I have more than one config file? Should I rename it to something else? Any other hints? You can create as many (or as few) config files as you desire. The system, once the patchkit is applied, will have between 10 and 15, each of which implements certain functions or features. In addition, the normal place for the patchkit to make changes to the config files is in the GENERICISA file. Since this file should remain unchanged and available, it is always a good idea to copy this file to a meaningful name and modify that file. In other words, change every reference in 3.1.1 from GENERICISA to HAL (or whatever you call your system). One final note. Every /sys/compile directory takes up 800K or so; you might want to watch to see how big these all get. 3.2.8 I have been getting a lot of "virtual memory exhausted" errors when I am compiling a program with a really big static array. I have 128Meg of memory and 8Gig of swap. How can this be happening? If you are using Csh, you can simply unlimit your processes in your system level /etc/csh.cshrc file or your personal .cshrc file. You can also modify your kernel so that the amount of memory available is less limiting. J"org Wunsch (j@tcd-dresden.de) provides us with this brief description: From a recent posting regarding the problem how much virtual memory one could get. | On the other hand, i've also changed the definitions you | mentioned. But i didn't like to modify the header files, and | actually, modifying the values is as easy as: | | options "DFLDSIZ='(16 * 1024 * 1024)'" | options "MAXDSIZ='(64 * 1024 * 1024)'" | | Include the above lines into your kernel's config file, reconfig | and rebuild it. | This has been reported to work well for NetBSD, but causes problems in the mkdep step of the kernel compile in FreeBSD. Check the FreeBSD Web Site for a more definitive answer. <Insert address of pointer here>. This is just a hint for those people poking around with compiling large source files. Especially, due to some gcc problems with large static arrays, compiling X applications with huge bitmaps would cause virtual memory trouble. Increasing the limits (o'course, as long as the h/w resources suffice) could help there. The default definitions for the above parameters are found in /usr/include/machine/vmparam.h. 3.2.8.1 I am running NetBSD and really DO have 128 Meg of memory; but the generic kernel only sees the first 64Meg. How can I fix this? The EXTMEM_SIZE entry needs to be set for your system. I'm completely clear on the concept, but it goes something like this: The value of EXTMEM_SIZE needs to be set to the number of bytes in the extended memory range for your system. Since this excludes the first Meg of memory, the value for the machine above would be 127 * 1024, for a grand total of 130048. Once you reset that, you will need to either get the -current sources (which will do the following for you) or you need to set NKPDE to something larger than 12. For 128Meg, 16 seems to be a reasonable number; for 256Meg, 20 would probably work, with 24 being adequate for 512Meg. Both of these can be set in the source, or you can use the "options" method listed in the last paragraph. 3.2.8.2 How do I dedicate 16Meg of memory to nothing but disk buffers? Set BUFPAGES to the number of 4K buffers you want to allocate. For 16Meg, you would use "options BUFPAGES=4096". 3.2.9 Where can I learn more about all this? We've skipped over a lot of details here; the straight dope comes from "Building Berkeley UNIX Kernels with Config", by Samuel J. Leffler and Michael J. Karels. 3.3 Other kernel related kind of questions. 3.3.1 Has the method for system call changed in NetBSD? Q. Is there something special with the order I need to update binaries and libraries in? If I drop in the new libc, everything gives me a bus error. Both shared and static do this. A. On the port-i386 list, Charles Hannum discussed changing the system call mechanism (doing it via an INT instead of a call gate). Looking at src/lib/libc/arch/i386/sys/syscall.S, it looks like this change is in. Your binaries are (if you are using an old kernel) probably crashing at each system call now. So.. first compiling a new kernel with COMPAT_10 in it should make your newly linked binaries work, I guess (have not recompiled since the update myself yet). Also don't forget that you need to use config.new now. So, the answer is Yes, the mechanism for system calls has changed, but the old method (using a call gate) is still available by specifying COMPAT_10 in your configuration file. 3.3.2 Does anyone have a system building script that takes things like building a new config and multiple config files into account? The program I use to rebuild my kernel is available from http://cynjut.neonramp.com/build-kernel 3.3.3 How do I upgrade from my release version of NetBSD (and probably FreeBSD) to the '-current' development sources? Your best method _by far_ would to to pull down a snapshot (assuming one exists for your arch) & install all except the etc tarfile, then diff the etc tarfile contents with your setup to check for any updates you should make. If you really want to do it the hard way, here is your map. # Remember to make yourself a new config (not config.old) kernel # config file. # # This means any old config file you may have had from # V1.1 needs to have the mainbus changes made. See GENERICADP and look # for the isa0,eisa0,pci0 at root add the mainbus0 stuff and attached # your buses to it. For updates to V1.2, you will need to make # certain the COMPAT_* options are correct. # # Make sure you have COMPAT_11 as part of your kernel config # options. # # This assumes that the -current source is in /usr/src (cd /usr/src/usr.sbin/config ; make && make install && make cleandir) # if you don't do this, config of your kernel config file will # fail with errors in files.i386 (cd /usr/src/gnu/usr.bin/gas ; make && make install && make cleandir) # if you don't do this, you won't be able to build locore.s, with # errors about cpuid instruction not found (cd /sys/arch/i386/conf ; config BCC13) (cd /sys/arch/i386/compile/BCC13 ; make depend && make) # copy new kernel to /, and boot off it (cd /usr/src/share/mk ; make install) # if you don't do this, you'll get errors building gcc, when it # doesn't know how to make the manual pages (don't know how to # make gcc.0) # # Rebuild yacc first or the yacc from v1.1 dies on Error Code 1 when # it hits %expect 34 in /usr/src/gnu/gcc/common/c-parse.y. The # new yacc included with V1.1B handles it, build and install it # before it gets used in the gcc building. # # *** # (cd /usr/src/usr.bin/yacc; make && make install && make cleandir) # # The build now uses a new tsort option (-q) when it gets to the # point of 'building standard cc1 library'. At this point if using # the tsort that comes with V1.1 it dies on a Error 1 and no # obvious explanation. Going to the common sub-directory under # gcc (ie. /usr/src/gnu/usr.bin/gcc/common) # and typing 'make' will reveal the full error. # (cd /usr/src/usr.bin/tsort; make && make install && make cleandir) # # cd /usr/src/gnu/usr.bin/gcc make # # Build will die here on libgcc # (cd cc; make install) (cd cc1; make install) (cd cc1obj; make install) (cd cc1plus; make install) (cd cpp; make install) (cd g++; make install) (cd libgcc; make; make install) (cd libobjc; make; make install) (cd /usr/src/gnu/usr.bin/gcc ; make && make install && make cleandir) # # Bernd Wiserner says that the ld.so that will be built next will # work only with libc.so.12.0, not with libc.so.12.3 # His instructions to make a working ld.so follow: # Do NOT run ldconfig while doing the folowing 5 lines ... # (cd /usr/src/include ; make && make includes) # cp -p /usr/libexec/ld.so /usr/libexec/ld.so.good (cd /usr/src/gnu/usr.bin/ld ; make && make install && make cleandir) cp -p /usr/libexec/ld.so.good /usr/libexec/ld.so # # Then build ld.so again ... (cd /usr/src/gnu/usr.bin/ld ; make && make install && make cleandir) # Thanks, Bernd... # And now the libraries (cd /usr/src/lib ; make && make install && make cleandir) (cd /usr/src/bin/sh ; make && make install && make cleandir) # and now back to the beginning and make the world # (cd /usr/src/bin ; make && make install && make cleandir) (cd /usr/src/sbin ; make && make install && make cleandir) mkdir /usr/share/doc/usd/13.viref # otherwise "make install" in /usr/src/usr.bin will fail because # the destination directory doesn't exist - from Tom Thai # if you're using the obj directory hierarchy, use the # initscan.c from the obj directory, otherwise use the initscan.c # that was created here. cd /usr/src/usr.bin/lex if test -d /usr/src/usr.bin/lex/obj ; then cp initscan.c obj/scan.c else cp initscan.c scan.c fi # if you don't, then lex won't be built (cd /usr/src/usr.bin ; make && make install && make cleandir) (cd /usr/src/usr.sbin ; make && make install && make cleandir) (cd /usr/src/libexec ; make && make install && make cleandir) (cd /usr/src/gnu ; make && make install && make cleandir) (cd /usr/src/share ; make && make install && make cleandir) mkdir /usr/share/doc/usd/30.rogue /usr/share/doc/usd/31.trek # otherwise "make install" in /usr/src/games might fail # if the dirs weren't already there (cd /usr/src/games ; make && make install && make cleandir) 3.3.4 Is there a Makefile that does all that happy world-building stuff? There is one in the /usr/src directory. Unfortunately, it seldom gets sent down unless you are using sup to get -current. If you need it for NetBSD, you can FTP to ftp.netbsd.org and get it from the src archive. ftp://ftp.netbsd.org/pub/NetBSD-current/src/Makefile The same can be said for FreeBSD and OpenBSD, but obviously you will need to get it from their FTP sites; not NetBSD's. 3.3.5 Can NetBSD do cross compilation? Sure. Check out the cross-compiling howto for MacBSD for an example. All of the NetBSD ports should be capable of doing 'pretty much' the same thing. You can find Colin Wood's MacBSD cross compilation howto at the following URL: http://www.macbsd.com/macbsd/howto/cc-HOWTO 3.3.6 My network memory seems to be leaking. The numbers just keep increasing slowly over time. Is there a problem I need to worry about? Probably not. When the system starts, the kernel malloc pool is not backed by real memory. As these pages are allocated, real memory is assigned to them, increasing your memory usage. As these pages are released, they are returned to the malloc pool, but the memory is never returned to the system. Eventually, your malloc area will reach a point of statis, where new pages are not needed from real memory; the releases to the malloc area should cover the new pages needed. 3.4 X11/XFree86/XS3 3.4.1 What options should I define to get the X extensions included? Once you have applied the patch kit, the only thing left to do is to modify the config file to include the following line: options XSERVER, UCONSOLE recompile the kernel and the kernel should support X. 3.4.2 Where can I get the FAQ for 'X'? Steve Kotsopoulos' general 'X on Intel-based Unix' FAQ available by anonymous ftp from export.lcs.mit.edu in /contrib/faq/Intel-Unix-X-faq. 3.4.3 Why does X drop characters when using xdm? When I run xdm from the console, it keeps losing keystrokes and the shift keys don't always work. Why? You need to run xdm with the -nodaemon flag. The reason is xdm normally detaches from the keyboard. This allows other processes (like getty) to return to reading from the keyboard. A race condition results, where some keystrokes are sent to xdm and others are sent to other processes. Using the -nodaemon flag causes xdm to stay attached to the keyboard so no other process can use it. This answer comes from Michael C. Newell (root@wanderer.nsi.nasa.gov) This bit of trivia is also covered in detail in the X FAQ and the README that accompanies XFree86. 3.4.4 What can I do to figure out why 'X' doesn't work with NetBSD? The best way to debug problems with starting 'X' is to redirect the output of the 'startx' program to a file: % startx >& startx.log # csh % startx 2>&1 > startx.log # sh With the output from this, the problems should be fairly easy to identify and (hopefully) fix. 3.4.5 Under NetBSD and FreeBSD, xlock (or any other program that uses passwords) fails to validate user passwords. Anyone know why? OK, first off, make sure you're using the latest version of xlock. If you've pulled it out of the /ports/ distribution then you've got version 1.14. This is woefully ancient, so get the latest, which at the time of writing is 2.7 (just do an archie search for 'xlockmore-2.7' to find it). Get this, compile it up and *make sure it's setuid root*. So, after you've copied it into /usr/X11R6/bin or wherever, do the following: # chown root.wheel ./xlock # chmod 4755 ./xlock After that, it should work fine. The problem is that without being setuid root xlock cannot read the real system passwords. If you look in /etc/passwd you'll see that the passwords are all '*'d out, because FreeBSD and NetBSD use shadow passwords. That's what worked for me. A couple of other suggestions were raised last time this problem cropped up: o If you're using a pre-compiled xlock then it might have been linked against the USA encryption libraries. If you're outside the States then the encryption libraries are different, and a US xlock will not be able to read the passwords. The fix is to get the sources and recompile. o If you've compiled it from source, made it setuid root, and it still doesn't work, someone suggested changing the size of the constant PASSLENGTH in xlock.c from '20' to '40'. I haven't had to do this, and in v2.7 it's '64' anyway, so it shouldn't be a problem. 3.5 I want to run 'XYZA' which is dynamically linked and from 'some other operating system'. What special things do I have to do to get it working? For NetBSD: Assuming you are trying to simulate a SVR4 system, you want to create the '/emul/svr4' directory. You will also want to ensure the "COMPAT_SVR4" option is in your kernel config file. This option will change in 1.3-current and later, so be sure to check out the config files included with those versions to ensure you have the right options. Note that there are known problems when trying to run a staically linked Linux ELF executable and you have SVR4 emulation built into your kernel. THe problem is the order in which ELF executables are processed through the kernel tables. The SVR4 emulation gets processed first, thus causing the LInux ELF executable to be improperly processed. This may cause certain Linux static ELF executables to fail under the *BSD systems. The way to fix this is to remove SVR4 emulation from the kernel or switch to a real SVR4 executable. Also note that newer versions of NetBSD do not make a distinction between Linux ELF executables and SVR4 ELF executables. The only difference (from the 1.3 kernel's viewpoint) is whether they are 32 or 64 bit ELF executables. With this accomplished, you will need to copy several files into the emulation directoy. A live example might be best at this point. Most of this information is include in the 'compat_linux(8)' manpage. First, set up the '/emul/linux' directory. Within this directory (and virtually all of the emulation directories) you will need the following: etc/ (the emulated /etc directory) lib/ (the emulated /lib directory) usr/ (the emulated /usr directory) From there, the simplest way to populate these directories is to use a program like 'cpio' or 'tar' to build an archive. Having a linux machine available will greatly simplify this. Copy (basically everything from the Linux (or whatever) /etc and /lib directories. Any executables that you need from the Linux system should then be copied into an appropriate place in the usr/ directory. For example, when creating the Linux Doom system on NetBSD, you need to have the following files (which should all come from the Linux Doom distribution). usr/X11R6/lib libX11.so.6.0 usr/bin: as86 ld86 usr/games/doom README.linuxx doom1.wad linuxxdoom sndserver.old With Doom specifically, you may need to set 'DOOMWAD' (or whatever it is) to usr/games/doom for it to work correctly. In addition to the 'X' version, the native version should also work (with recent versions of NetBSD (1.1+) It is generally accepted that NetBSD's emulation of the other systems is pretty close *except for* sound. The audio drivers in NetBSD are not a robust (yet) as they probably should be. Don't be surprised if sound emulation works badly, if at all, for any of the other operating systems emulation works for. The good news is that work under way (early 1997) in the -current version of NetBSD (pre 1.3) should fix most of this. The sound subsystem has been modified so that machine independent components are seperated from the machine dependent stuff, and each of the machine dependent parts is getting a facelist. Ultimately, the NetBSD sounds system should be 100% compatible with the one from FreeBSD (that was the design goal) and probably Linux as well. 3.6 You promised to talk about timezones below. Are you going to? >I've seen lots of stuff about timezone's being a bit dodgy, >especially with most European timezones changing over to DST on >the 27th March. I must say that that was NOT the case for me - >pumpy (the author's system) is running off the >/usr/share/zoneinfo/GB-Eire timezone file, (symbolically) linked >to /etc/localtime, the CMOS clock is running off GMT, and the >kernel is compiled with "timezone 0". For a full discussion of this problem, check out the 'options(4)' manpage. It describes the DST and TIMEZONE options in detail. In newer kernels, the problems are far less dramatic than they used to be (see below): The problems with timezones and real-time clocks is that most 386 hardware is just stupid. It doesn't recognize DST, even when told to. It can't use network system time during DST because it makes all the PC clocks off by an hour. For machines that dual boot DOS and *BSD, one of the simplest solutions involves using a program called 'clock372' from Simtelnet (the exact site is not available). After you install it, you add a couple of lines to your DOS CONFIG.SYS and set the hardware clock to DST. From there, you build a kernel with DST and TIMEZONE set to 0 and your whole system "just works". In older kernels the following works: I use /usr/share/zoneinfo/MET as /etc/localtime and have the kernel configured as timezone -1 dst 4 (My wife is running DOS on this machine for doom sometimes ;-) I set this strange dst value after diging in some old ultrix(?) man pages. There were several dst-changing-method listed and 4 was the code for the central europe one. This gave me an idea... I use an Ultrix box every day, so why not... Now, I don't know how closely this applies to NetBSD since Ultrix is based on a much older version of BSD, and this isn't for the kernel config, but for an envar of timezone values, but it's at least somewhat enlightening on possible meanings for these things. Could someone in the know shed light on how accurately this models the timezone stuff in the kernel config? When I did "man timezone" this is what I got (portion of this quoted from the DEC MIPS Ultrix 4.3a timezone(3) manpage, slightly hacked by me (Michael L. VanLoon (michaelv@iastate.edu)) STD offset [DST [offset][,start[/time],end[/time]]] the components of the string have the following meaning: STD and DST Three or more characters that are the designation for the standard (STD) or summer (DST) time zone. Only STD is required; if DST is missing, then summer time does not apply in this locale. Upper- and lowercase letters are explicitly allowed. Any characters except a leading colon (:), digits, comma (,), minus (-), plus (+), and ASCII NUL are allowed. offset Indicates the value to be added to the local time to arrive at Coordinated Universal Time. The offset has the form: hh[:mm[:ss]] The minutes (mm) and seconds (ss) are optional. The hour (hh) is required and may be a single digit. The offset following STD is required. If no offset follows DST, summer time is assumed to be one hour ahead of standard time. One or more digits may be used; the value is always interpreted as a decimal number. The hour must be between 0 and 24, and the minutes (and seconds) - if present - between zero and 59. If preceded by a "-", the time zone is east of the Prime Meridian; otherwise it is west (which may be indicated by an optional preceding "+"). start and end Indicates when to change to and back from summer time. Start describes the date when the change from standard to summer time occurs and end describes the date when the change back happens. The format of start and end must be one of the following: Jn The Julian day n (1 < n < 365). Leap days are not counted. That is, in all years, including leap years, February 28 is day 59 and March 1 is day 60. It is impossible to explicitly refer to the occasional February 29. n The zero-based Julian day (0 < n < 365). Leap days are counted, and it is possible to refer to February 29. Mm.n.d The nth d day of month m (1 < n < 5, 0 < d < 6, 1 < m < 12). When n is 5 it refers to the last d day of month m. Day 0 is Sunday. time The time field describes the time when, in current time, the change to or from summer time occurs. Time has the same format as offset except that no leading sign (a minus (-) or a plus (+) sign) is allowed. The default, if time is not given, is 02:00:00. As an example of the previous format, if the TZ environment variable had the value EST5EDT4,M4.1.0,M10.5.0 it would describe the rule, which went into effect in 1987, for the Eastern time zone in the USA. Specifically, EST would be the designation for standard time, which is 5 hours behind GMT. EDT would be the designation for DST, which is 4 hours behind GMT. DST starts on the first Sunday in April and ends on the last Sunday in October. In both cases, since the time was not specified, the change to and from DST would occur at the default time of 2:00 AM. The timezone call remains for compatibility reasons only; it is impossible to reliably map timezone's arguments (zone, a `minutes west of GMT' value and DST, a `daylight saving time in effect' flag) to a time zone abbreviation. 3.6.1 How do you change the timezone on NetBSD (FreeBSD also?)? Relink /etc/localtime. This will correct the difference from GMT (or its trendy equivelant) to your local timezone. In addition, the kernel needs to be modified to take the clock time in your CMOS into account. Since most folks that run DOS prefer to have their clocks set to local time, the timezone hack was introduced to allow the kernel to adjust the CMOS clock time to GMT. Once GMT has been computed, the /etc/localtime file can be referenced to determine the corrected local time. All generic kernels are built using the offset from California (why is anyone's guess:-) so just about everyone's clock will be off by their timezone offset from Berkeley. Also, it may pay to actually copy the correct timezone file rather than link it. That way, you clock will be correct even in single users mode (when the /usr partition is not even mounted. The disadvantage of this is that anytime the timezone file gets updated, you will need to make certain that the file is copied into the /etc directory. 3.6.2 The translation between seconds-since-the-epoch and date differs by about 18 seconds between BSD and other Unixes when running ntp; why? See ntp FAQ. Apparently, the time correction takes leap seconds into account twice. The timezone files in our system take the leap seconds into account in the kernel, and nntp takes the same 18 leap seconds into account when syncing the time. Because of that, the time will appear to be off by eighteen seconds. (Henning Schulzrinne) 3.7 How can I implement CVS to track MY changes to the kernel source tree, yet still follow the -current development tree? I'll append the scripts I use, but be warned, they may bite you if you are careless... The main reason I use cvs import is to handle updates from the ``vendor''. The best way I've found is to import _exactly_ what was shipped. This means unconfigured, and I put config.h, etc, in .cvsignore. If I have to modify configure.in then obviously I commit them :-) #!/bin/sh # This is a shell archive. # remove everything above the "#!/bin/sh" line # and feed to /bin/sh # Use -c option to overwrite existing files # # Contents: # README.import # import.sh # prune.sh # # packed by: <sjg@zen.void.oz.au> on Sat Jun 17 20:00:34 EST 1995 # PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test -f README.import -a x$1 != x-c ; then echo shar: Will not over-write existing file \"README.import\" else echo shar: Extracting \"README.import\" \(2902 characters\) sed 's/^X//' >README.import << '!EOF' XThe following may be of use to others wanting to use CVS to merge XNetBSD sources with local changes but are not confident that they have Xinterpreted the documentation accurately. X XMuch thanks to Chris Demetriou (cgd) for taking the time to spell out Xthe steps he used when merging NetBSD sources without which I might Xnot have taken the plunge myself :-) The following is based on Chris' Xtips, though of course any errors are mine. X XOk. My NetBSD sources are kept in usr.src, if NetBSD is all Xyou use CVS for you might want to simply call it src. X XI unpack tar files and/or sup into a directory /d2/current. X XTo import the entire tree I: X Xcd /d2/current/src X Xcvs import "-I! " -m "from netbsd-current as of 950508" usr.src NetBSD \ XNetBSD-950508 > /tmp/cvs.out 2>&1 X XWhere: Xusr.src is the repository where the imported data goes (so set it X according to your own needs), XNetBSD is a vendor tag. XNetBSD-950508 is a release tag (there can be multiple release tags given). X XI use "-I! " as otherise some files that you need (like Xbin/csh/USD.doc/csh.a) will be ignored.. The space after the ! is Xneeded. X XIt takes quite a while. It is a good idea to save the output to a file. X XAt the end you may well get a message like: X X cvs checkout -jNetBSD:yesterday -jNetBSD usr.src X XThis means there were some conflicts between your local sources and Xthe import. So I do what it says - but not in my working tree. X Xcd /d2/tmp Xcvs checkout -jNetBSD:yesterday -jNetBSD usr.src > /tmp/merge.out 2>&1 X XYou can then go find all the files with conflicts. XEither grep '^C' /tmp/merge.out or find usr.src -name '.#*' -print XGo edit them to resolce the conflicts. This is usually obvious. X XWhen happy. X Xcd /d2/tmp/usr.src Xcvs commit -m"merged local changes with NetBSD-950508" Xcd .. Xrm -rf usr.src X XOk. Now if you are brave you can: X Xcd /usr.src (or whereever your working sources are) Xcvs update X XFinally, you should occasionally make sure you remove old files. X XI use a script to do this. It does a diff between files on the NetBSD Xbranch looking for the latest release tag (eg. NetBSD-950508). XIf cvs diff remports that a file does not have that tag, it is because Xit was not present in the import (ie removed). X XThe command sequence is: X Xcvs diff -s -r NetBSD -r NetBSD-950508 > /tmp/prune.out 2>&1 X X# check that all went well... Xcat /tmp/prune.out | awk '/Diffing/ { dir=$NF } X/NetBSD-/ { file=$NF; print dir "/" file }' > /tmp/pruned X Xcat /tmp/pruned | xargs cvs tag -d NetBSD Xcat /tmp/pruned | xargs rm -f Xcat /tmp/pruned | xargs cvs delete X XNote that this is a slow process on a 486DX33! So don't plan on Xmerging everything very often. Folk who mainly hack the kernel can do Xsrc/sys more frequently. The sequence is analogous eg. X Xcd /d2/current/src/sys X Xcvs import "-I! " -m "from netbsd-current as of 950508" usr.src/sys NetBSD \ XNetBSD-950508 > /tmp/cvs.out 2>&1 X Xetc. X XHope this helps. X X--sjg !EOF if test 2902 -ne `wc -c < README.import`; then echo shar: \"README.import\" unpacked with wrong size! fi fi if test -f import.sh -a x$1 != x-c ; then echo shar: Will not over-write existing file \"import.sh\" else echo shar: Extracting \"import.sh\" \(290 characters\) sed 's/^X//' >import.sh << '!EOF' X: Xtoday=`date '+%y%m%d'` X Xrep=${1:-usr.src} X X# -I! doesn't work, it needs a space after the ! Xcvs import "-I! " -m "from netbsd-current as of $today" $rep NetBSD NetBSD-$today X X# cd somewhere X# cvs checkout -jNetBSD:yesterday -jNetBSD usr.src > /tmp/cvs.out 2>&1 X# merge changes and commit !EOF if test 290 -ne `wc -c < import.sh`; then echo shar: \"import.sh\" unpacked with wrong size! fi chmod +x import.sh fi if test -f prune.sh -a x$1 != x-c ; then echo shar: Will not over-write existing file \"prune.sh\" else echo shar: Extracting \"prune.sh\" \(491 characters\) sed 's/^X//' >prune.sh << '!EOF' X: Xthen=${1:-`date '+%y%m%d'`} XTF=/tmp/prune.$$ XTF2=/tmp/prune2.$$ X#S=-s XS= X Xcase `echo -n .` in -n*) N=; C="\c";; *) N=-n; C=;; esac X Xask () { echo $N "${2:-$1?} "$C; read $1; } X Xcvs diff $S -r NetBSD -r NetBSD-$then > $TF 2>&1 || cat $TF >&2 X Xcat $TF | awk '/Diffing/ { dir=$NF } /NetBSD-/ { file=$NF; print dir "/" file }' > $TF2 X Xcat $TF2 Xask proceed Xcase "$proceed" in X[yY]*) Xcat $TF2 | xargs cvs tag -d NetBSD Xcat $TF2 | xargs rm -f Xcat $TF2 | xargs cvs delete X;; Xesac Xrm -f $TF $TF2 !EOF if test 491 -ne `wc -c < prune.sh`; then echo shar: \"prune.sh\" unpacked with wrong size! fi chmod +x prune.sh fi exit 0 3.8 Optional Op-codes for NetBSD, FreeBSD, and other systems. MNEMONIC INSTRUCTION ---------------------------------- AAC Alter All Commands AAR Alter At Random AB Add Backwards AFVC Add Finagle's Variable Constant AIB Attack Innocent Bystander AWTT Assemble With Tinker Toys BAC Branch to Alpha Centauri BAF Blow All Fuses BAFL Branch And Flush BBIL Branch on Blown Indicator Light BBT Branch on Binary Tree BBW Branch Both Ways BCF Branch and Catch Fire BCIL Branch Creating Infinite Loop BDC Break Down and Cry BDT Burn Data Tree BEW Branch Either Way BF Belch Fire BH Branch and Hang BOB Branch On Bug BOD Beat On the Disk BOI Bite Operator Immediately BPDI Be Polite, Don't Interrupt BPO Branch on Power Off BRSS Branch on Sunspot BST Backspace and Stretch Tape BW Branch on Whim CDC Close Disk Cover CDIOOAZ Calm Down, It's Only Ones and Zeros CEMU Close Eyes and Monkey with User space CH Create Havoc CLBR Clobber Register CM Circulate Memory CML Compute Meaning of Life COLB Crash for Operators Lunch Break CPPR Crumple Printer Paper and Rip CRASH Continue Running After Stop or Halt CRB Crash and Burn CRN Convert to Roman Numerals CS Crash System CSL Curse and Swear Loudly CU Convert to Unary CVG Convert to Garbage CWOM Complement Write-Only Memory CZZC Convert Zone to Zip Code DBZ Divide By Zero DC Divide and Conquer DMNS Do what I Mean, Not what I Say DMPK Destroy Memory Protect Key DPMI Declare Programmer Mentally Incompetent DPR Destroy Program DTC Destroy This Command DTE Decrement Telephone Extension DTVFL Destroy Third Variable From Left DW Destroy World ECO Electrocute Computer Operator EFD Emulate Frisbee Using Disk Pack EIAO Execute In Any Order EIOC Execute Invalid Opcode ENF Emit Noxious Fumes EO Execute Operator EROS Erase Read-Only Storage FLI Flash Lights Impressively FSM Fold, Spindle and Mutilate GCAR Get Correct Answer Regardless GDP Grin Defiantly at Programmer GFM Go Forth and Multiply IAE Ignore All Exceptions IBP Insert Bug and Proceed ISC Insert Sarcastic Comments JTZ Jump to Twilight Zone LCC Load and Clear Core MAZ Multiply Answer by Zero MLR Move and Lose Record MWAG Make Wild-Assed Guess MWT Malfunction Without Telling OML Obey Murphy's Laws PD Play Dead PDSK Punch Disk PEHC Punch Extra Holes on Cards POCL Punch Out Console Lights POPI Punch Operator Immediately RA Randomize Answer RASC Read And Shred Card RCB Read Command Backwards RD Reverse Directions RDA Refuse to Disclose Answer RDB Run Disk Backwards RIRG Read Inter-Record Gap RLI Rotate Left Indefinitely ROC Randomize Opcodes RPB Read, Print and Blush RPM Read Programmer's Mind RSD On Read Error Self-Destruct RWCR Rewind Card Reader SAI Skip All Instructions SAS Sit and Spin SCCA Short Circuit on Correct Answer SFH Set Flags to Half mast SLMTU=x SLIP MTU size SLP Sharpen Light Pen SPS Set Panel Switches SPSW Scramble Program Status Word SQPC Sit Quietly and Play with your Crayons SRDR Shift Right Double Ridiculous STA Store Anywhere TARC Take Arithmetic Review Course TPF Turn Power Off TPN Turn Power On UCB Uncouple CPU and Branch ULDA Unload Accumulator UP Understand Program WBT Water Binary Tree WHFO Wait Until Hell Freezes Over WI Write Illegibly WSWW Work in Strange and Wondrous Ways XSP Execute Systems Programmer ZAR Zero Any Register If you have gotten this far, you deserved some humor. -- Dave Burgess Network Engineer - Nebraska On-Ramp, Inc. *bsd FAQ Maintainer / SysAdmin for the NetBSD system in my spare bedroom "Just because something is stupid doesn't mean there isn't someone that doesn't want to do it...." User Contributions:Comment about this article, ask questions, or add new information about this topic: |
Let's keep it forever!