So far, everything we've done has been with the standard GENERIC kernel distributed with FreeBSD. You may find it an advantage to install a custom kernel:
-CURRENT version of FreeBSD.In older releases of BSD, you needed to build a new kernel for just about anything you wanted to change, even things as simple as a different IRQ for a device. FreeBSD has evolved quite a bit since then, and it's becoming increasingly less necessary to build a custom kernel. You will certainly need to do so if you want to enable kernel-wide options, such as extra
Configuring a kernel has changed a lot since the early days of FreeBSD, but it's not done yet. The information in this chapter represents a snapshot in the
FreeBSD is distributed in source, and building a kernel primarily involves compiling the source files needed for the kernel. To build a kernel, you perform the following steps:
/usr/src and run make kernel. This builds and installs the kernel and all modules. We'll look at it in more detail on page 617, where we'll also see alternatives that give more control over the process.A lot of the configuration file relates to the I/O devices that you may connect to your machine. In older releases of FreeBSD, it was often necessary to specify some of the IRQ,
The kernel installation does not install /boot/device.hints automatically. If it doesn't exist, copy it from the configuration directory:
# cp -p /usr/src/sys/i386/conf/GENERIC.h±nts /boot/device.hints
The kernel sources are kept in the directory /usr/src/sys. The
If your system doesn't have the directory /usr/src/sys, the kernel source has not been installed. If you have a CD-ROM, the sources are on the first CD-ROM in the directory /src. To install from the CD-ROM, perform the following steps:
# mkdir -p /usr/src/sys # ln -s /usr/src/sys /sys # cd / # cat /cdrom/src/ssys.[a-h]* | tar xzvf -
The
By definition, the files on CD-ROM are out of date. See Chapter 31 for details of how to get the current, up-to-date sources.
Next, move to the directory i386/conf and copy the GENERIC configuration file to the name you want to give your kernel. For example:
# cd /usr/src/sys/i386/conf # cp GENERIC FREEBIE
Traditionally, this name is in all capital letters and, if you are maintaining multiple FreeBSD machines with different hardware, it's a good idea to name it after your machine's hostname. In this example we call it FREEBIE.
Now, edit FREEBIE with your favourite
# # FREEBIE -- My personal configuration file # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # # http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # ifyou've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # Anexhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # Ifyou are in doubt as to the purpose or necessity of a line, check first # inNOTES. # # $FreeBSD: src/sys/i386/conf/FREEBIE,v 1.369 2002/10/19 16:54:07 rwatson Exp $ machine "i386" cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE maxusers 0
The directory /sys/i386/conf contains a number of configuration files:
$ make LIN
The general format of a configuration file is quite simple. Each line contains a keyword and one or more arguments. Anything following a # is considered a comment and ignored. Keywords that contain numbers used as text must be
One of the results of this
options APPLE_MAC_COMPATIBILITY
You can build a kernel with this option. It will make no difference whatsoever. Now it's unlikely that you'll think up a non-existent option like this, but it's much more possible that you'll SYSVSHM, with the result that you don't compile in the option you wanted. The config program warns if you use unknown options, so take these warnings seriously.
Kernel options change from release to release, so there's no point in describing them all here. In the following sections we'll look at some of the more interesting ones; for a complete list, read LINT or the online
Every kernel you build requires the keywords machine, cpu, and ident. For example,
machine "i386" For 386 architecture machine "alpha" For alpha architecture machine "sparc64" For PARC 64 architecture cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE
The keyword machine describes the machine architecture for which the kernel is to be built. Currently it should be i386 for the Intel architecture, and alpha for the AXP architecture. Don't confuse this with the processor: for example, the i386 architecture refers to the
cpu describes which CPU chip or chips this kernel should support. For the i386 architecture, the possible values are I386_CPU, I486_CPU, I586_CPU and I686_CPU, and you can specify any combination of these values. For a custom kernel, it is best to specify only the CPU you have. If, for example, you have an Intel Pentium, use I586_CPU for cpu_type.
If you're not sure what dmesg command when running the GENERIC kernel. For example:
CPU: AMD Athlon(tm) XP processor 1700+ (1462.51-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x662 Stepping = 2 Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,^OV,PAT,PS E36,MMX,FXSR,SSE> AMD Features=0xc0480000<<b19>,AMIE,DSP,3DNcw!>
This shows that the processor, an AMD Athlon XP, is a "686 class" CPU, so to run a kernel on this processor, you must set I686_CPU in the config file.
Since Release 5 of FreeBSD, it is no longer possible to build a single kernel with support for both the 80386 processor and later processors: the code for the later processors is optimized to use instructions that the 80386 processor does not have. Choose either I386_CPU or any combination of the others.
ident specifies a name used to identify the kernel. In the file GENERIC it is GENERIC. Change this to whatever you named your kernel, in this example, FREEBIE. The value you put in ident will print when you boot up the kernel, so it's useful to give a kernel a different name if you want to keep it separate from your usual kernel (if you want to build an machine and cpu,
This name is passed to the C compiler as a variable, so don't use names like DEBUG, or something that could be confused with another machine or CPU name, like vax.
There are a number of global kernel options, most of which you don't need to change. In the following section, we'll look at some of the few exceptions.
There are some devices that the GENERIC kernel does not support. In older releases of FreeBSD, you needed to build a new kernel to support them. This is seldom the case any more: most devices can be supported by klds. Work is under way to support the
This value sets the size of a number of important
maxusers="64"
See 532 for more details of /boot/loader.conf.
maxusers is intended to be roughly equal to the number of 0 tells the kernel to autosize the tables depending on the amount of memory on the system. If the autosizing
doesn' t work to your satisfaction, change this value. Even if you are the only person to use the machine, you shouldn't set maxusers lower than the default
value 32, especially if you're using X or compiling software. The reason is that the most important table set by maxusers
is the maximum number of processes, which is set to 20 + 16 * maxusers, so if you set maxusers to one, you can only have 36 maxusers to 32 will allow you to have up to 532
maxusers does not limit the number of users who can log into the machine. It simply sets various table sizes to reasonable values considering the maximum number of users you will likely have on your system and how many processes each of them will be running. It's probable that this parameter will go away in future.
FreeBSD 5.0 supports most modern SMP and the device in the configuration file if you're interested.
FreeBSD is a very stable operating system. No software is perfect, however, and sometimes it crashes. When it does, it provides a number of facilities to help fix the problem. Some of these are dependent on kernel build options.
Even if you have no intention of debugging a kernel problem yourself, you should set debug symbols when you build a kernel. They cost nothing except disk space, and if you are short on disk space, you can remove most of the files after the build.
To set the debug symbols, remove the leading # mark from this line in the configuration file:
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
Under normal circumstances this makes no difference: the build process still installs the kernel without the debug symbols, and it has no effect on performance. If, however, you have a crash, the kernel with debug symbols is available in the kernel build directory, in this case /usr/src/sys/i386/compile/FREEBIE/kernel.debug, to assist analysis of the problem. Without this file it will be very difficult to find it.
So why is it commented out? Without debug symbols, your build directory will take about 50 MB of disk space. With debug symbols, it will be about 250 MB. The FreeBSD Project couldn't agree to change it.
If you run into trouble with your system, there are a number of other debugging options that you can use. The following are the more important ones:
options DDB options BREAK_TO_DEBUGGER options DDB_UNATTENDED # Don't drop into DDB for a panic options GDB_REMOTE_CHAT #Use gdb remote debugging protocol options KTRACE options DIAGNOSTIC options INVARIANTS options INVARIANT_SUPPORT options WITNESS #Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN #Don't run witness on spinlocks for speed These options provide support for various debugging features.
Specify to include the kernel debugger, . If you set this option, you might also want to set the BREAK_TO_DEBUGGER option,
Use the option BREAK_TO_DEBUGGER if you have installed the kernel debugger and you have the system console on a serial line.
If you have a panic on a system with , it will not reboot automatically. Instead, it will enter and give you a chance to examine the remains of the system before rebooting. This can be a disadvantage on systems that run
supports remote debugging from another FreeBSD machine via a serial connection. See the online GDB_REMOTE_CHAT.
Set KTRACE if you want to use the ktrace .
A number of source files use the DIAGNOSTIC option to enable extra
INVARIANTS is used in a number of source files to enable extra
INVARIANT_SUPPORT option compiles in support for verifying some of the internal structures. It is a prerequisite for INVARIANTS. The intent is that you can set INVARIANTS for single source files (by changing the source file or specifying it on the command line) if you have INVARIANT_SUPPORT enabled.
One of the big changes in FreeBSD Release 5 relates to the manner in which the kernel performs resource locking. As a result, the danger exists of deadlocks, locks that can'tbe undone without rebooting the machine. WITNESS checks for the danger of deadlocks and warns if it finds a potential deadlock ("lock order reversal"). This is a very expensive debugging option: it can slow the machine down by an order of
A WITNESS with WITNESS_SKIPSPIN, which avoids
When changing the configuration file, consider that it probably won't be the only time you make these changes. At some time in the future, you'll upgrade the system, and you'll have to build your custom kernel all over again. But the GENERIC kernel configuration file will have changed as well. You have two choices:
To prepare for this approach, try to change as little as possible in the body of the configuration file. Instead, add all your changes to the end, along with a comment so that you can readily recognize the changes. For example, you might append:
# Added by grog, 24 October 2002 # Comment out WITNESS, WITNESS_SKIPSPIN and SCSI_DELAY above options BREAK_TO_DEBUGGER options SCSI_DELAY=3000 #Delay (in ms) before probing SCSI options CAMDEBUG options MSGBUF_SIZE=81920 #options TIMEREQUESTS #watch for delays device snp #Snoop device - to look at pty/vty/etc.
That won't be all, of course. Look at that option SCSI_DELAY.That option already exists in the configuration file (with a value of 15 seconds instead of 3). If you leave both, config will issue a warning. You need to comment out the first
The traditional way to build a BSD kernel was, assuming a configuration file called FREEBIE:
# cd /usr/src/sys/i386/conf # config FREEBIE # cd ../compile/FREEBIE # make depend all install
At the time of writing, this still works, but it will go away at some time in the future. It has the disadvantage that you need to know your architecture, and the kernel is built in the source tree, which is not a good idea. The new kernel build method starts from /usr/src, the same directory as all other build operations, and it builds the kernel in the /usr/obj hierarchy.
You'll need about 250 MB of free space on /usr/obj to build a kernel. If you're really tight on space, you can reduce this value to about 50 MB by omitting the makeoptions DEBUG=-g specification, but if you have problems with the system at some later stage, it will be much more difficult to find what is causing them.
There are a number of alternative commands for building the kernel:
# cd /usr/src # make kernel KERNCONF=FREEBIE build and install kernel and klds # make buildkernel KERNCONF=FREEBIE build kernel and klds #make installkernel KERNCONF=FREEBIE install prebuilt kernel and klds #make reinstallkernel KERNCONF=FREEBIE reinstall kernel and klds # make kernel KERNCONF=FREEBIE -DNO_MODULES build and install kernel only
The easiest way is make kernel, which is subdivided into the buildkernel and installkernel steps. You can perform these steps individually if you want. If you use the reinstallkernel target instead of the installkernel target, the /boot/kernel.old hierarchy remains unchanged. This is useful when a previous kernel build failed to boot.
If you know the klds are not going to change, you can speed things up by not building them again. Use the -DNO_MODULES flag in combination with any of the other targets to
The first step in building the kernel is to run the config program. You no longer have to do this yourself; the buildkernel and kernel targets do it for you. config creates a directory in which to build the kernel and fills it with the necessary infrastructure, notably the kernel Makefile and a number of
It's possible to get error messages at this stage if you have made a mistake in the config file. If the config command fails when you give it your kernel description, you've probably made a simple error somewhere. Fortunately, config will print the line number that it had trouble with, so you can quickly find it with an editor. For example:
config: line 17: syntax error
One possibility is that you have GENERIC or LINT kernel definitions.
The next step is to compile all the source files and create a kernel and a set of matching klds. It can take some time, up to an hour on a slow machine. It's also possible to have errors, and unfortunately they are usually not self-da), you require the scbus device as well, and if you have just about any kind of Ethernet card, you require the miibus device as well:
device scbus # SCSI bus (required) device da # Direct Access (disks) device miibus # MII bus support device fxp # Intel EtherExpress PRO/100B (82557, 82558)
If you leave scbus or miibus out of the configuration, config will not complain, but the kernel link phase will fail with lots of
If you can't resolve the problem after comparing your configuration file with GENERIC, send mail to FreeBSD-questions@FreeBSD.org with your kernel configuration, and it should be
Next, shutdown the system and reboot load the new kernel:
# shutdown -r now
If the new kernel does not boot, or fails to recognize your devices, don't panic. Reset the machine, and when the boot prompt appears, press the space bar to interrupt the boot. Then boot the old kernel:
Ok unload remove the kernel that the loader has loaded Ok load /boot/kernel.old/kernel load the previous kernel Ok boot
When reconfiguring a kernel, it is always a good idea to keep on hand a kernel that is known to work. There are two points here:
reinstallkernel target mentioned above:
# make buildkernel KERNCONF=FREEBIE build kernel and klds # make reinstallkernel KERNCONF=FREEBIE reinstall kernel and klds
kernel.save, which the installation procedure does not touch.After booting with a good kernel you can check over your configuration file and try to build it again. One helpful resource is the /var/log/messages file which records, among other things, all of the kernel messages from every successful boot. Also, the dmesg command prints the most recent kernel messages from the current boot. After some time the original messages are overwritten, so the system startup procedure saves the messages at
FreeBSD Release 5 comes with devfs, the device file system. One great advantage of devfs is that it automatically creates device nodes for the hardware it finds, so you no longer need to run the /dev/MAKEDEV script supplied with older releases of FreeBSD.
As we saw at the beginning of the chapter, you may not have to build a new kernel to implement the functionality you want. Instead, just load it into the running kernel with a Kernel Loadable Module (kld). The directory /boot/kernel/modules contains a number of klds. To load them, use kldload. For example, if you wanted to load SCO UNIX compatibility, you would enter:
# kldload ibcs2
This loads the module /boot/kernel/modules/ibcs2.ko. Note that you don't need to specify the directory name, nor the .ko extension.
To find what modules are loaded, use kldstat:
# kldstat Id Refs Address Size Name 1 5 0xc0100000 1d08b0 kernel 2 2 0xc120d000 a000 ibcs2.ko 3 1 0xc121b000 3000 ibcs2_coff.ko 4 1 0xc1771000 e000 linux.ko 5 1 0xc177f000 bf000 vinum.ko
You can also unload some klds, but not all of them. Use kldunload for this purpose:
# kldunload vinum
You can't unload a kld which has active resources. In the case of vinum, for example, you can only unload it when none of its volumes are mounted.
sysctl is a relatively new kernel interface that allows access to specific variables in the kernel. Some of these variables are read-only: you can look, but not touch. Others are
sysctl variables are usually referred to simply as sysctls. Each sysctl has a name in "Management Information Base"(MIB) form, consisting of a .). The first component of the name indicates the part of the kernel to which it relates. The following examples give you an idea of how to use the sysctl program:
$ sysctl kern.ostype FreeBSD $ sysctl kern list all sysctls starting with kern $ sysctl -a list all sysctls # sysctl net.inet.ip.forwarding=1 turn IP forwarding on net.inet.ip.forwarding: 0 -> 1
Some of the more interesting sysctls are:
kern.ostype: FreeBSD kern.osrelease: 5.0-RELEASE
kern.version: FreeBSD 5.0-RELEASE #0: Thu Jan 16 15:03:31 CST 2003
grog@freebie.example.org:/usr/src/sys/GENERIC kern.hostname: freebie.example.org
kern.boottime: { sec = 1007165073, usec = 570637 } Fri Jan 17 10:34:33 2003
kern.bootfile: /boot/kernel/kernel
kern.init_path: /sbin/init:/sbin/oinit:/sbin/init.bak:/stand/sysinstall
kern.module_path: /boot/kernel;/boot/kernel;/boot/modules;/modules
kern.coredump: 1
kern.corefile: /var/tmp/%N.core
kern.msgbuf: nreach TCP 213.46.243.23:25370 139.130.136.138:25 in via ppp0
net.inet.ip.fw.enable: 1 hw.machine: i386
hw.model: Pentium II/Pentium II Xeon/Celeron
hw.ncpu: 1
hw.byteorder: 1234
hw.physmem: 129949696
hw.usermem: 100556800
hw.pagesize: 4096
hw.floatingpoint: 1
hw.machine_arch: i386
hw.ata.ata_dma: 1
hw.ata.wc: 1
hw.ata.tags: 0
hw.ata.atapi_dma: 0
compat.linux.osname: Linux
compat.linux.osrelease: 2.2.12
compat.linux.oss_version: 198144
Many of these need no description, but some are less obvious:
kern.msgbuf shows the contents of the kernel kern.corefile specifies a template for the name of the %N is replaced by the name of the program.Keeping up with FreeBSD-CURRENT requires work on your part. You should be on the FreeBSD-current@FreeBSD.org
FreeBSD-CURRENT is not as stable as the released releases. To prepare yourself for possible problems, you should build kernels that include debug symbols. The resultant kernel is about 30 MB in size, but it will make debugging with ddb (the kernel debugger) or gdb much easier. Even if you don't intend to do this yourself, the information will be of great use to anybody you may call in to help. We looked at how to build a debug kernel on page 614.
You will run into problems with FreeBSD-CURRENT. When it happens, please first read the
If you experience a panic, please don't just send a message to FreeBSD-current saying "My kernel panics when I type foo." Remember that you're asking somebody to use their
If you don't do at least this, there isn't much chance that a
When the kernel panics, and you have dumping enabled, you'll usually see something like this on the console:
\
Fatal trap 9: general protection fault while in kernel mode
instruction pointer = 0x8:0xc01c434b
stack pointer = 0x10:0xc99f8d0c
frame pointer = 0x10:0xc99f8d28
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 2638 (find)
interrupt mask = net tty bio cam
trap number = 9
panic: general protection fault
syncing disks... 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7
giving up on 6 buffers
Uptime: 17h53m13s
dumping to dev #ad/1, offset 786560
dump ata0: resetting devices .. done
You don't need to write this information down: it is saved in the dump.
When you reboot, the system startup scripts find that you have a dump in the designated dump device (see above) and copy it and the current kernel to /var/crash, assuming the directory exists and there's enough space for the dump. You'll see something like this in the directory:
# cd /var/crash # ls -l -rw-r--r-- 1 root wheel 3 Dec 29 10:09 bounds -rw-r--r-- 1 root wheel 4333000 Dec 29 10:10 kernel.22 -rw-r--r-- 1 root wheel 5 Sep 17 1999 minfree -rw------- 1 root wheel 268369920 Dec 29 10:09 vmcore.22
The important files here are kernel.22, which contains a copy of the kernel running when the crash occurred, and vmcore.22, which contains the contents of memory. The number 22 indicates that the sequence number of the dump. It's possible to have multiple dumps in /var/crash. Note that you can waste a lot of space like that.
The file bounds contains the number of the next dump (23 in this case), and minfree specifies the minimum amount of free space (in
savecore copies the kernel from which you booted. As we've seen, it typically isn't a debug kernel. In the example above, we installed /usr/src/sys/i386/conf/FREEBIE/kernel, but the
# ln -s /usr/src/sys/i386/conf/FREEBIE/kernel.debug . # ls -lL -rw-r--r-- 1 root wheel 3 Dec 29 10:09 bounds -rwxr-xr-x 1 grog lemis 16796546 Dec 18 14:21 kernel.debug -rw-r--r-- 1 root wheel 4333000 Dec 29 10:10 kernel.22 -rw-r--r-- 1 root wheel 5 Sep 17 1999 minfree -rw------- 1 root wheel 268369920 Dec 29 10:09 vmcore.22
As you can see, it's much larger.
Next, run gdb against the kernel and the dump:
# gdb -k kernel.debug vmcore.22
The first thing you see is a political message from the
#0 dumpsys () at ../../kern/kern_shutdown.c:473
473 if (dumping++) {
(kgdb)
Due to the way C, gd and FreeBSD work, the real information you're looking for is further down the stack. The first thing you need to do is to find out exactly where it happens. Do that with the command:
(kgdb) bt
#0 dumpsys () at ../../kern/kern_shutdown.c:473
#1 0xc01c88bf in boot (howto=256) at ../../kern/kern_shutdown.c:313
#2 0xc01c8ca5 in panic (fmt=0xc03a8cac "%s") at ../../kern/kern_shutdown.c:581
#3 0xc033ab03 in trap_fatal (frame=0xc99f8ccc, eva=0)
at ../../i386/i386/trap.c:956
#4 0xc033a4ba in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16,
tf_edi = -1069794208, tf_esi = -1069630360, tf_ebp = -912290520,
tf_isp = -912290568, tf_ebx = -1069794208, tf_edx = 10, tf_ecx = 10,
tf_eax = -1, tf_trapno = 9, tf_err = 0, tf_eip = -1071889589, tf_cs = 8,
tf_eflags = 66182, tf_esp = 1024, tf_ss = 6864992})
at ../../i386/i386/trap.c:618
#5 0xc01c434b in malloc (size=1024, type=0xc03c3c60, flags=0)
at ../../kern/kern_malloc.c:233
#6 0xc01f015c in allocbuf (bp=0xc3a6f7cc, size=1024)
at ../../kern/vfs_bio.c:2380
#7 0xc01effa6 in getblk (vp=0xc9642f00, blkno=0, size=1024, slpflag=0,
slptimeo=0) at ../../kern/vfs_bio.c:2271
#8 0xc01eded2 in bread (vp=0xc9642f00, blkno=0, size=1024, cred=0x0,
bpp=0xc99f8e3c) at ../../kern/vfs_bio.c:504
#9 0xc02d0634 in ffs_read (ap=0xc99f8ea0) at ../../ufs/ufs/ufs_readwrite.c:273
#10 0xc02d734e in ufs_readdir (ap=0xc99f8ef0) at vnode_if.h:334
#11 0xc02d7cd1 in ufs_vnoperate (ap=0xc99f8ef0)
at ../../ufs/ufs/ufs_vnops.c:2382
#12 0xc01fbc3b in getdirentries (p=0xc9a53ac0, uap=0xc99f8f80)
at vnode_if.h:769
#13 0xc033adb5 in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds = 47,
tf_edi = 134567680, tf_esi = 134554336, tf_ebp = -1077937404,
tf_isp = -912289836, tf_ebx = 672064612, tf_edx = 134554336,
tf_ecx = 672137600, tf_eax = 196, tf_trapno = 7, tf_err = 2,
tf_eip = 671767876, tf_cs = 31, tf_eflags = 582, tf_esp = -1077937448,
tf_ss = 47}) at ../../i386/i386/trap.c:1155
#14 0xc032b825 in Xint0x80_syscall ()
#15 0x280a1eee in ?? ()
#16 0x280a173a in ?? () #17 0x804969e in ?? () #18 0x804b550 in ?? () #19 0x804935d in ?? () (kgdb)
The rest of this chapter is only of interest to programmers with a good understanding of C. If you're not a programmer, this is about as far as you can go. Save this information and supply it to whomever you ask for help. It's usually not enough to solve the problem, but it's a good start, and your helper will be able to tell you what to do next.
The #0, which is seldom the one that interests us. In general, we've had a panic, the most important frame is the function that calls panic:
#3 0xc033ab03 in trap_fatal (frame=0xc99f8ccc, eva=0)
at ../../i386/i386/trap.c:956
The information here is:
#3 is the frame number. This is a number allocated by gdb. You can use it to reference the frame in a number of commands.0xc033ab03 is the trap_fatal is the name of the function.(frame=0xc99f8ccc, eva=0) are the parameter values supplied to trap_fatal.../../i386/i386/trap.c:956 gives the name of the source file and the line number in the file. The path names are relative to the kernel build directory, so they usually start with ../../.In this example, the panic comes from a user process. Starting at the bottom, depending on the processor platform, you may see the user process stack. You can recognize them on an Intel platform by the addresses below the kernel 0xc0000000.On other platforms, the address might be different. In general, you won't get any symbolic information for these frames, since the kernel
Climbing up the stack, you'll find the getdirentries.To perform its function, getdirentries ffs_read, the function that reads from a UFS file. ffs_read calls bread, which reads into the buffer cache. To do so, it allocates a buffer with getblk and allocbuf, which calls malloc to allocate memory for buffer cache. The next thing we see is a stack frame for trap: something has gone wrong inside malloc. trap determines that the trap in unrecoverable and calls trap_fatal, which in turn calls panic. The
In general, you start analyzing a panic dump in the stack frame that called panic, but in the case of the fatal trap that we have here, the most important stack frame is the one below trap, in this case frame 5. That's where things went wrong. Select it with the frame command, abbreviated to f, and list the code with list (or l):
(kgdb) f 5
#5 0xc01c434b in malloc (size=1024, type=0xc03c3c60, flags=0)
at ../../kern/kern_malloc.c:233
233 va = kbp->kb_next;
(kgdb) l
228 }
229 freep->next = savedlist;
230 if(kbp->kb_last == NULL)
231 kbp->kb_last = (caddr_t)freep;
232 }
233 va = kbp->kb_next;
234 kbp->kb_next = ((struct freelist *)va)->next;
235 #ifdef INVARIANTS
236 freep = (struct freelist *)va;
237 savedtype = (const char *) freep->type->ks_shortdesc;
(kgdb)
You might want to look at the local (info local, which you can I loc:
(kgdb) I loc type = (struct malloc_type *) 0xc03c3c60 kbp = (struct kmembuckets *) 0xc03ebc68 kup = (struct kmemusage *) 0x0 freep = (struct freelist *) 0x0 indx = 10 npg = -1071714292 allocsize = -1069794208 s=6864992 va = 0xffffffff <Address 0xffffffff out of bounds> cp = 0x0 savedlist = 0x0 ksp = (struct malloc_type *) 0xffffffff (kgdb)
The line where the problem occurs is 233:
233 va =kbp->kb_next;
Look at the structure :
(kgdb) p*kbp $2 = {
kb_next = 0xffffffff <Address 0xffffffff out of bounds>,
kb_last = 0xc1a31000 "",
kb_calls = 83299,
kb_total = 1164,
kb_elmpercl = 4,
kb_totalfree = 178,
kb_highwat = 20,
kb couldfree = 3812
}
The problem here is that the pointer kb_next is set to Oxffffffff . It should contain a valid address, but as gdb observes, this isn't not valid.
So far we have found that the crash is in malloc, and that it's caused by an invalid pointer in an internal data structure. malloc is a function that is used many times a second by all computers. It's unlikely that the bug is in malloc. In fact, the most likely cause is that a function that has used memory allocated by malloc has overwritten its bounds and hit malloc's data structures.
What do we do now? To quote fortune:
The seven eyes of Ningauble the Wizard floated back to his hood as he reported to Fafhrd: "I have seen much, yet cannot explain all. The Gray Mouser is exactly twenty-five feet below the deepest cellar in the palace of Gilpkerio Kistomerces. Even though twenty-four parts in twenty-five of him are dead, he is alive.
"Now about Lankhmar. She's been invaded, her walls breached everywhere and desperate fighting is going on in the streets, by a fierce host which out-numbers Lankhmar's inhabitants by fifty to one -and equipped with all modern weapons. Yet you can save the city."
"How?" demanded Fafhrd.
Ningauble shrugged. "You're a hero. You should know."
-- Fritz Leiber, from "The Swords of Lankhmar"
From here on, you're on your own. If you get this far, the FreeBSD-hackers
So far, everything we've done has been with the standard GENERIC kernel distributed with FreeBSD. You may find it an advantage to install a custom kernel:
-CURRENT version of FreeBSD.In older releases of BSD, you needed to build a new kernel for just about anything you wanted to change, even things as simple as a different IRQ for a device. FreeBSD has evolved quite a bit since then, and it's becoming increasingly less necessary to build a custom kernel. You will certainly need to do so if you want to enable kernel-wide options, such as extra
Configuring a kernel has changed a lot since the early days of FreeBSD, but it's not done yet. The information in this chapter represents a snapshot in the
FreeBSD is distributed in source, and building a kernel primarily involves compiling the source files needed for the kernel. To build a kernel, you perform the following steps:
/usr/src and run make kernel. This builds and installs the kernel and all modules. We'll look at it in more detail on page 617, where we'll also see alternatives that give more control over the process.A lot of the configuration file relates to the I/O devices that you may connect to your machine. In older releases of FreeBSD, it was often necessary to specify some of the IRQ,
The kernel installation does not install /boot/device.hints automatically. If it doesn't exist, copy it from the configuration directory:
# cp -p /usr/src/sys/i386/conf/GENERIC.h±nts /boot/device.hints
The kernel sources are kept in the directory /usr/src/sys. The
If your system doesn't have the directory /usr/src/sys, the kernel source has not been installed. If you have a CD-ROM, the sources are on the first CD-ROM in the directory /src. To install from the CD-ROM, perform the following steps:
# mkdir -p /usr/src/sys # ln -s /usr/src/sys /sys # cd / # cat /cdrom/src/ssys.[a-h]* | tar xzvf -
The
By definition, the files on CD-ROM are out of date. See Chapter 31 for details of how to get the current, up-to-date sources.
Next, move to the directory i386/conf and copy the GENERIC configuration file to the name you want to give your kernel. For example:
# cd /usr/src/sys/i386/conf # cp GENERIC FREEBIE
Traditionally, this name is in all capital letters and, if you are maintaining multiple FreeBSD machines with different hardware, it's a good idea to name it after your machine's hostname. In this example we call it FREEBIE.
Now, edit FREEBIE with your favourite
# # FREEBIE -- My personal configuration file # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # # http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # ifyou've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # Anexhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # Ifyou are in doubt as to the purpose or necessity of a line, check first # inNOTES. # # $FreeBSD: src/sys/i386/conf/FREEBIE,v 1.369 2002/10/19 16:54:07 rwatson Exp $ machine "i386" cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE maxusers 0
The directory /sys/i386/conf contains a number of configuration files:
$ make LIN
The general format of a configuration file is quite simple. Each line contains a keyword and one or more arguments. Anything following a # is considered a comment and ignored. Keywords that contain numbers used as text must be
One of the results of this
options APPLE_MAC_COMPATIBILITY
You can build a kernel with this option. It will make no difference whatsoever. Now it's unlikely that you'll think up a non-existent option like this, but it's much more possible that you'll SYSVSHM, with the result that you don't compile in the option you wanted. The config program warns if you use unknown options, so take these warnings seriously.
Kernel options change from release to release, so there's no point in describing them all here. In the following sections we'll look at some of the more interesting ones; for a complete list, read LINT or the online
Every kernel you build requires the keywords machine, cpu, and ident. For example,
machine "i386" For 386 architecture machine "alpha" For alpha architecture machine "sparc64" For PARC 64 architecture cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE
The keyword machine describes the machine architecture for which the kernel is to be built. Currently it should be i386 for the Intel architecture, and alpha for the AXP architecture. Don't confuse this with the processor: for example, the i386 architecture refers to the
cpu describes which CPU chip or chips this kernel should support. For the i386 architecture, the possible values are I386_CPU, I486_CPU, I586_CPU and I686_CPU, and you can specify any combination of these values. For a custom kernel, it is best to specify only the CPU you have. If, for example, you have an Intel Pentium, use I586_CPU for cpu_type.
If you're not sure what dmesg command when running the GENERIC kernel. For example:
CPU: AMD Athlon(tm) XP processor 1700+ (1462.51-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x662 Stepping = 2 Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,^OV,PAT,PS E36,MMX,FXSR,SSE> AMD Features=0xc0480000<<b19>,AMIE,DSP,3DNcw!>
This shows that the processor, an AMD Athlon XP, is a "686 class" CPU, so to run a kernel on this processor, you must set I686_CPU in the config file.
Since Release 5 of FreeBSD, it is no longer possible to build a single kernel with support for both the 80386 processor and later processors: the code for the later processors is optimized to use instructions that the 80386 processor does not have. Choose either I386_CPU or any combination of the others.
ident specifies a name used to identify the kernel. In the file GENERIC it is GENERIC. Change this to whatever you named your kernel, in this example, FREEBIE. The value you put in ident will print when you boot up the kernel, so it's useful to give a kernel a different name if you want to keep it separate from your usual kernel (if you want to build an machine and cpu,
This name is passed to the C compiler as a variable, so don't use names like DEBUG, or something that could be confused with another machine or CPU name, like vax.
There are a number of global kernel options, most of which you don't need to change. In the following section, we'll look at some of the few exceptions.
There are some devices that the GENERIC kernel does not support. In older releases of FreeBSD, you needed to build a new kernel to support them. This is seldom the case any more: most devices can be supported by klds. Work is under way to support the
This value sets the size of a number of important
maxusers="64"
See 532 for more details of /boot/loader.conf.
maxusers is intended to be roughly equal to the number of 0 tells the kernel to autosize the tables depending on the amount of memory on the system. If the autosizing
doesn' t work to your satisfaction, change this value. Even if you are the only person to use the machine, you shouldn't set maxusers lower than the default
value 32, especially if you're using X or compiling software. The reason is that the most important table set by maxusers
is the maximum number of processes, which is set to 20 + 16 * maxusers, so if you set maxusers to one, you can only have 36 maxusers to 32 will allow you to have up to 532
maxusers does not limit the number of users who can log into the machine. It simply sets various table sizes to reasonable values considering the maximum number of users you will likely have on your system and how many processes each of them will be running. It's probable that this parameter will go away in future.
FreeBSD 5.0 supports most modern SMP and the device in the configuration file if you're interested.
FreeBSD is a very stable operating system. No software is perfect, however, and sometimes it crashes. When it does, it provides a number of facilities to help fix the problem. Some of these are dependent on kernel build options.
Even if you have no intention of debugging a kernel problem yourself, you should set debug symbols when you build a kernel. They cost nothing except disk space, and if you are short on disk space, you can remove most of the files after the build.
To set the debug symbols, remove the leading # mark from this line in the configuration file:
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
Under normal circumstances this makes no difference: the build process still installs the kernel without the debug symbols, and it has no effect on performance. If, however, you have a crash, the kernel with debug symbols is available in the kernel build directory, in this case /usr/src/sys/i386/compile/FREEBIE/kernel.debug, to assist analysis of the problem. Without this file it will be very difficult to find it.
So why is it commented out? Without debug symbols, your build directory will take about 50 MB of disk space. With debug symbols, it will be about 250 MB. The FreeBSD Project couldn't agree to change it.
If you run into trouble with your system, there are a number of other debugging options that you can use. The following are the more important ones:
options DDB options BREAK_TO_DEBUGGER options DDB_UNATTENDED # Don't drop into DDB for a panic options GDB_REMOTE_CHAT #Use gdb remote debugging protocol options KTRACE options DIAGNOSTIC options INVARIANTS options INVARIANT_SUPPORT options WITNESS #Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN #Don't run witness on spinlocks for speed These options provide support for various debugging features.
Specify to include the kernel debugger, . If you set this option, you might also want to set the BREAK_TO_DEBUGGER option,
Use the option BREAK_TO_DEBUGGER if you have installed the kernel debugger and you have the system console on a serial line.
If you have a panic on a system with , it will not reboot automatically. Instead, it will enter and give you a chance to examine the remains of the system before rebooting. This can be a disadvantage on systems that run
supports remote debugging from another FreeBSD machine via a serial connection. See the online GDB_REMOTE_CHAT.
Set KTRACE if you want to use the ktrace .
A number of source files use the DIAGNOSTIC option to enable extra
INVARIANTS is used in a number of source files to enable extra
INVARIANT_SUPPORT option compiles in support for verifying some of the internal structures. It is a prerequisite for INVARIANTS. The intent is that you can set INVARIANTS for single source files (by changing the source file or specifying it on the command line) if you have INVARIANT_SUPPORT enabled.
One of the big changes in FreeBSD Release 5 relates to the manner in which the kernel performs resource locking. As a result, the danger exists of deadlocks, locks that can'tbe undone without rebooting the machine. WITNESS checks for the danger of deadlocks and warns if it finds a potential deadlock ("lock order reversal"). This is a very expensive debugging option: it can slow the machine down by an order of
A WITNESS with WITNESS_SKIPSPIN, which avoids
When changing the configuration file, consider that it probably won't be the only time you make these changes. At some time in the future, you'll upgrade the system, and you'll have to build your custom kernel all over again. But the GENERIC kernel configuration file will have changed as well. You have two choices:
To prepare for this approach, try to change as little as possible in the body of the configuration file. Instead, add all your changes to the end, along with a comment so that you can readily recognize the changes. For example, you might append:
# Added by grog, 24 October 2002 # Comment out WITNESS, WITNESS_SKIPSPIN and SCSI_DELAY above options BREAK_TO_DEBUGGER options SCSI_DELAY=3000 #Delay (in ms) before probing SCSI options CAMDEBUG options MSGBUF_SIZE=81920 #options TIMEREQUESTS #watch for delays device snp #Snoop device - to look at pty/vty/etc.
That won't be all, of course. Look at that option SCSI_DELAY.That option already exists in the configuration file (with a value of 15 seconds instead of 3). If you leave both, config will issue a warning. You need to comment out the first
The traditional way to build a BSD kernel was, assuming a configuration file called FREEBIE:
# cd /usr/src/sys/i386/conf # config FREEBIE # cd ../compile/FREEBIE # make depend all install
At the time of writing, this still works, but it will go away at some time in the future. It has the disadvantage that you need to know your architecture, and the kernel is built in the source tree, which is not a good idea. The new kernel build method starts from /usr/src, the same directory as all other build operations, and it builds the kernel in the /usr/obj hierarchy.
You'll need about 250 MB of free space on /usr/obj to build a kernel. If you're really tight on space, you can reduce this value to about 50 MB by omitting the makeoptions DEBUG=-g specification, but if you have problems with the system at some later stage, it will be much more difficult to find what is causing them.
There are a number of alternative commands for building the kernel:
# cd /usr/src # make kernel KERNCONF=FREEBIE build and install kernel and klds # make buildkernel KERNCONF=FREEBIE build kernel and klds #make installkernel KERNCONF=FREEBIE install prebuilt kernel and klds #make reinstallkernel KERNCONF=FREEBIE reinstall kernel and klds # make kernel KERNCONF=FREEBIE -DNO_MODULES build and install kernel only
The easiest way is make kernel, which is subdivided into the buildkernel and installkernel steps. You can perform these steps individually if you want. If you use the reinstallkernel target instead of the installkernel target, the /boot/kernel.old hierarchy remains unchanged. This is useful when a previous kernel build failed to boot.
If you know the klds are not going to change, you can speed things up by not building them again. Use the -DNO_MODULES flag in combination with any of the other targets to
The first step in building the kernel is to run the config program. You no longer have to do this yourself; the buildkernel and kernel targets do it for you. config creates a directory in which to build the kernel and fills it with the necessary infrastructure, notably the kernel Makefile and a number of
It's possible to get error messages at this stage if you have made a mistake in the config file. If the config command fails when you give it your kernel description, you've probably made a simple error somewhere. Fortunately, config will print the line number that it had trouble with, so you can quickly find it with an editor. For example:
config: line 17: syntax error
One possibility is that you have GENERIC or LINT kernel definitions.
The next step is to compile all the source files and create a kernel and a set of matching klds. It can take some time, up to an hour on a slow machine. It's also possible to have errors, and unfortunately they are usually not self-da), you require the scbus device as well, and if you have just about any kind of Ethernet card, you require the miibus device as well:
device scbus # SCSI bus (required) device da # Direct Access (disks) device miibus # MII bus support device fxp # Intel EtherExpress PRO/100B (82557, 82558)
If you leave scbus or miibus out of the configuration, config will not complain, but the kernel link phase will fail with lots of
If you can't resolve the problem after comparing your configuration file with GENERIC, send mail to FreeBSD-questions@FreeBSD.org with your kernel configuration, and it should be
Next, shutdown the system and reboot load the new kernel:
# shutdown -r now
If the new kernel does not boot, or fails to recognize your devices, don't panic. Reset the machine, and when the boot prompt appears, press the space bar to interrupt the boot. Then boot the old kernel:
Ok unload remove the kernel that the loader has loaded Ok load /boot/kernel.old/kernel load the previous kernel Ok boot
When reconfiguring a kernel, it is always a good idea to keep on hand a kernel that is known to work. There are two points here:
reinstallkernel target mentioned above:
# make buildkernel KERNCONF=FREEBIE build kernel and klds # make reinstallkernel KERNCONF=FREEBIE reinstall kernel and klds
kernel.save, which the installation procedure does not touch.After booting with a good kernel you can check over your configuration file and try to build it again. One helpful resource is the /var/log/messages file which records, among other things, all of the kernel messages from every successful boot. Also, the dmesg command prints the most recent kernel messages from the current boot. After some time the original messages are overwritten, so the system startup procedure saves the messages at
FreeBSD Release 5 comes with devfs, the device file system. One great advantage of devfs is that it automatically creates device nodes for the hardware it finds, so you no longer need to run the /dev/MAKEDEV script supplied with older releases of FreeBSD.
As we saw at the beginning of the chapter, you may not have to build a new kernel to implement the functionality you want. Instead, just load it into the running kernel with a Kernel Loadable Module (kld). The directory /boot/kernel/modules contains a number of klds. To load them, use kldload. For example, if you wanted to load SCO UNIX compatibility, you would enter:
# kldload ibcs2
This loads the module /boot/kernel/modules/ibcs2.ko. Note that you don't need to specify the directory name, nor the .ko extension.
To find what modules are loaded, use kldstat:
# kldstat Id Refs Address Size Name 1 5 0xc0100000 1d08b0 kernel 2 2 0xc120d000 a000 ibcs2.ko 3 1 0xc121b000 3000 ibcs2_coff.ko 4 1 0xc1771000 e000 linux.ko 5 1 0xc177f000 bf000 vinum.ko
You can also unload some klds, but not all of them. Use kldunload for this purpose:
# kldunload vinum
You can't unload a kld which has active resources. In the case of vinum, for example, you can only unload it when none of its volumes are mounted.
sysctl is a relatively new kernel interface that allows access to specific variables in the kernel. Some of these variables are read-only: you can look, but not touch. Others are
sysctl variables are usually referred to simply as sysctls. Each sysctl has a name in "Management Information Base"(MIB) form, consisting of a .). The first component of the name indicates the part of the kernel to which it relates. The following examples give you an idea of how to use the sysctl program:
$ sysctl kern.ostype FreeBSD $ sysctl kern list all sysctls starting with kern $ sysctl -a list all sysctls # sysctl net.inet.ip.forwarding=1 turn IP forwarding on net.inet.ip.forwarding: 0 -> 1
Some of the more interesting sysctls are:
kern.ostype: FreeBSD kern.osrelease: 5.0-RELEASE
kern.version: FreeBSD 5.0-RELEASE #0: Thu Jan 16 15:03:31 CST 2003
grog@freebie.example.org:/usr/src/sys/GENERIC kern.hostname: freebie.example.org
kern.boottime: { sec = 1007165073, usec = 570637 } Fri Jan 17 10:34:33 2003
kern.bootfile: /boot/kernel/kernel
kern.init_path: /sbin/init:/sbin/oinit:/sbin/init.bak:/stand/sysinstall
kern.module_path: /boot/kernel;/boot/kernel;/boot/modules;/modules
kern.coredump: 1
kern.corefile: /var/tmp/%N.core
kern.msgbuf: nreach TCP 213.46.243.23:25370 139.130.136.138:25 in via ppp0
net.inet.ip.fw.enable: 1 hw.machine: i386
hw.model: Pentium II/Pentium II Xeon/Celeron
hw.ncpu: 1
hw.byteorder: 1234
hw.physmem: 129949696
hw.usermem: 100556800
hw.pagesize: 4096
hw.floatingpoint: 1
hw.machine_arch: i386
hw.ata.ata_dma: 1
hw.ata.wc: 1
hw.ata.tags: 0
hw.ata.atapi_dma: 0
compat.linux.osname: Linux
compat.linux.osrelease: 2.2.12
compat.linux.oss_version: 198144
Many of these need no description, but some are less obvious:
kern.msgbuf shows the contents of the kernel kern.corefile specifies a template for the name of the %N is replaced by the name of the program.Keeping up with FreeBSD-CURRENT requires work on your part. You should be on the FreeBSD-current@FreeBSD.org
FreeBSD-CURRENT is not as stable as the released releases. To prepare yourself for possible problems, you should build kernels that include debug symbols. The resultant kernel is about 30 MB in size, but it will make debugging with ddb (the kernel debugger) or gdb much easier. Even if you don't intend to do this yourself, the information will be of great use to anybody you may call in to help. We looked at how to build a debug kernel on page 614.
You will run into problems with FreeBSD-CURRENT. When it happens, please first read the
If you experience a panic, please don't just send a message to FreeBSD-current saying "My kernel panics when I type foo." Remember that you're asking somebody to use their
If you don't do at least this, there isn't much chance that a
When the kernel panics, and you have dumping enabled, you'll usually see something like this on the console:
\
Fatal trap 9: general protection fault while in kernel mode
instruction pointer = 0x8:0xc01c434b
stack pointer = 0x10:0xc99f8d0c
frame pointer = 0x10:0xc99f8d28
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 2638 (find)
interrupt mask = net tty bio cam
trap number = 9
panic: general protection fault
syncing disks... 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7
giving up on 6 buffers
Uptime: 17h53m13s
dumping to dev #ad/1, offset 786560
dump ata0: resetting devices .. done
You don't need to write this information down: it is saved in the dump.
When you reboot, the system startup scripts find that you have a dump in the designated dump device (see above) and copy it and the current kernel to /var/crash, assuming the directory exists and there's enough space for the dump. You'll see something like this in the directory:
# cd /var/crash # ls -l -rw-r--r-- 1 root wheel 3 Dec 29 10:09 bounds -rw-r--r-- 1 root wheel 4333000 Dec 29 10:10 kernel.22 -rw-r--r-- 1 root wheel 5 Sep 17 1999 minfree -rw------- 1 root wheel 268369920 Dec 29 10:09 vmcore.22
The important files here are kernel.22, which contains a copy of the kernel running when the crash occurred, and vmcore.22, which contains the contents of memory. The number 22 indicates that the sequence number of the dump. It's possible to have multiple dumps in /var/crash. Note that you can waste a lot of space like that.
The file bounds contains the number of the next dump (23 in this case), and minfree specifies the minimum amount of free space (in
savecore copies the kernel from which you booted. As we've seen, it typically isn't a debug kernel. In the example above, we installed /usr/src/sys/i386/conf/FREEBIE/kernel, but the
# ln -s /usr/src/sys/i386/conf/FREEBIE/kernel.debug . # ls -lL -rw-r--r-- 1 root wheel 3 Dec 29 10:09 bounds -rwxr-xr-x 1 grog lemis 16796546 Dec 18 14:21 kernel.debug -rw-r--r-- 1 root wheel 4333000 Dec 29 10:10 kernel.22 -rw-r--r-- 1 root wheel 5 Sep 17 1999 minfree -rw------- 1 root wheel 268369920 Dec 29 10:09 vmcore.22
As you can see, it's much larger.
Next, run gdb against the kernel and the dump:
# gdb -k kernel.debug vmcore.22
The first thing you see is a political message from the
#0 dumpsys () at ../../kern/kern_shutdown.c:473
473 if (dumping++) {
(kgdb)
Due to the way C, gd and FreeBSD work, the real information you're looking for is further down the stack. The first thing you need to do is to find out exactly where it happens. Do that with the command:
(kgdb) bt
#0 dumpsys () at ../../kern/kern_shutdown.c:473
#1 0xc01c88bf in boot (howto=256) at ../../kern/kern_shutdown.c:313
#2 0xc01c8ca5 in panic (fmt=0xc03a8cac "%s") at ../../kern/kern_shutdown.c:581
#3 0xc033ab03 in trap_fatal (frame=0xc99f8ccc, eva=0)
at ../../i386/i386/trap.c:956
#4 0xc033a4ba in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16,
tf_edi = -1069794208, tf_esi = -1069630360, tf_ebp = -912290520,
tf_isp = -912290568, tf_ebx = -1069794208, tf_edx = 10, tf_ecx = 10,
tf_eax = -1, tf_trapno = 9, tf_err = 0, tf_eip = -1071889589, tf_cs = 8,
tf_eflags = 66182, tf_esp = 1024, tf_ss = 6864992})
at ../../i386/i386/trap.c:618
#5 0xc01c434b in malloc (size=1024, type=0xc03c3c60, flags=0)
at ../../kern/kern_malloc.c:233
#6 0xc01f015c in allocbuf (bp=0xc3a6f7cc, size=1024)
at ../../kern/vfs_bio.c:2380
#7 0xc01effa6 in getblk (vp=0xc9642f00, blkno=0, size=1024, slpflag=0,
slptimeo=0) at ../../kern/vfs_bio.c:2271
#8 0xc01eded2 in bread (vp=0xc9642f00, blkno=0, size=1024, cred=0x0,
bpp=0xc99f8e3c) at ../../kern/vfs_bio.c:504
#9 0xc02d0634 in ffs_read (ap=0xc99f8ea0) at ../../ufs/ufs/ufs_readwrite.c:273
#10 0xc02d734e in ufs_readdir (ap=0xc99f8ef0) at vnode_if.h:334
#11 0xc02d7cd1 in ufs_vnoperate (ap=0xc99f8ef0)
at ../../ufs/ufs/ufs_vnops.c:2382
#12 0xc01fbc3b in getdirentries (p=0xc9a53ac0, uap=0xc99f8f80)
at vnode_if.h:769
#13 0xc033adb5 in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds = 47,
tf_edi = 134567680, tf_esi = 134554336, tf_ebp = -1077937404,
tf_isp = -912289836, tf_ebx = 672064612, tf_edx = 134554336,
tf_ecx = 672137600, tf_eax = 196, tf_trapno = 7, tf_err = 2,
tf_eip = 671767876, tf_cs = 31, tf_eflags = 582, tf_esp = -1077937448,
tf_ss = 47}) at ../../i386/i386/trap.c:1155
#14 0xc032b825 in Xint0x80_syscall ()
#15 0x280a1eee in ?? ()
#16 0x280a173a in ?? () #17 0x804969e in ?? () #18 0x804b550 in ?? () #19 0x804935d in ?? () (kgdb)
The rest of this chapter is only of interest to programmers with a good understanding of C. If you're not a programmer, this is about as far as you can go. Save this information and supply it to whomever you ask for help. It's usually not enough to solve the problem, but it's a good start, and your helper will be able to tell you what to do next.
The #0, which is seldom the one that interests us. In general, we've had a panic, the most important frame is the function that calls panic:
#3 0xc033ab03 in trap_fatal (frame=0xc99f8ccc, eva=0)
at ../../i386/i386/trap.c:956
The information here is:
#3 is the frame number. This is a number allocated by gdb. You can use it to reference the frame in a number of commands.0xc033ab03 is the trap_fatal is the name of the function.(frame=0xc99f8ccc, eva=0) are the parameter values supplied to trap_fatal.../../i386/i386/trap.c:956 gives the name of the source file and the line number in the file. The path names are relative to the kernel build directory, so they usually start with ../../.In this example, the panic comes from a user process. Starting at the bottom, depending on the processor platform, you may see the user process stack. You can recognize them on an Intel platform by the addresses below the kernel 0xc0000000.On other platforms, the address might be different. In general, you won't get any symbolic information for these frames, since the kernel
Climbing up the stack, you'll find the getdirentries.To perform its function, getdirentries ffs_read, the function that reads from a UFS file. ffs_read calls bread, which reads into the buffer cache. To do so, it allocates a buffer with getblk and allocbuf, which calls malloc to allocate memory for buffer cache. The next thing we see is a stack frame for trap: something has gone wrong inside malloc. trap determines that the trap in unrecoverable and calls trap_fatal, which in turn calls panic. The
In general, you start analyzing a panic dump in the stack frame that called panic, but in the case of the fatal trap that we have here, the most important stack frame is the one below trap, in this case frame 5. That's where things went wrong. Select it with the frame command, abbreviated to f, and list the code with list (or l):
(kgdb) f 5
#5 0xc01c434b in malloc (size=1024, type=0xc03c3c60, flags=0)
at ../../kern/kern_malloc.c:233
233 va = kbp->kb_next;
(kgdb) l
228 }
229 freep->next = savedlist;
230 if(kbp->kb_last == NULL)
231 kbp->kb_last = (caddr_t)freep;
232 }
233 va = kbp->kb_next;
234 kbp->kb_next = ((struct freelist *)va)->next;
235 #ifdef INVARIANTS
236 freep = (struct freelist *)va;
237 savedtype = (const char *) freep->type->ks_shortdesc;
(kgdb)
You might want to look at the local (info local, which you can I loc:
(kgdb) I loc type = (struct malloc_type *) 0xc03c3c60 kbp = (struct kmembuckets *) 0xc03ebc68 kup = (struct kmemusage *) 0x0 freep = (struct freelist *) 0x0 indx = 10 npg = -1071714292 allocsize = -1069794208 s=6864992 va = 0xffffffff <Address 0xffffffff out of bounds> cp = 0x0 savedlist = 0x0 ksp = (struct malloc_type *) 0xffffffff (kgdb)
The line where the problem occurs is 233:
233 va =kbp->kb_next;
Look at the structure :
(kgdb) p*kbp $2 = {
kb_next = 0xffffffff <Address 0xffffffff out of bounds>,
kb_last = 0xc1a31000 "",
kb_calls = 83299,
kb_total = 1164,
kb_elmpercl = 4,
kb_totalfree = 178,
kb_highwat = 20,
kb couldfree = 3812
}
The problem here is that the pointer kb_next is set to Oxffffffff . It should contain a valid address, but as gdb observes, this isn't not valid.
So far we have found that the crash is in malloc, and that it's caused by an invalid pointer in an internal data structure. malloc is a function that is used many times a second by all computers. It's unlikely that the bug is in malloc. In fact, the most likely cause is that a function that has used memory allocated by malloc has overwritten its bounds and hit malloc's data structures.
What do we do now? To quote fortune:
The seven eyes of Ningauble the Wizard floated back to his hood as he reported to Fafhrd: "I have seen much, yet cannot explain all. The Gray Mouser is exactly twenty-five feet below the deepest cellar in the palace of Gilpkerio Kistomerces. Even though twenty-four parts in twenty-five of him are dead, he is alive.
"Now about Lankhmar. She's been invaded, her walls breached everywhere and desperate fighting is going on in the streets, by a fierce host which out-numbers Lankhmar's inhabitants by fifty to one -and equipped with all modern weapons. Yet you can save the city."
"How?" demanded Fafhrd.
Ningauble shrugged. "You're a hero. You should know."
-- Fritz Leiber, from "The Swords of Lankhmar"
From here on, you're on your own. If you get this far, the FreeBSD-hackers
Для получения официальных документов о завершении программы дополнительного профессионального образования (удостоверения о повышении квалификации, дипломов о профессиональной переподготовке и MBA) необходимо предоставить:
Внимание! Вы можете не заказывать доставку бумажной версии официального документы, а скачать его в электронном виде и распечатать самостоятельно. Информация о выданном документе в течение 1 месяца загружается в Федеральную информационную систему «Федеральный реестр сведений о документах об образовании и (или) о квалификации, документах об обучении» - ФИС ФРДО.
Доступ на новый сайт осуществляется с использованием адреса электронной почты, который был указан вами при регистрации на "старом". Мы постарались перенести все ваши данные с прежнего ресурса, однако не исключена вероятность потери части информации.
При возникновении проблемы со входом, воспользуйтесь функцией сброса пароля
Если вы обнаружите несоответствия, пожалуйста, сообщите нам.