Corefiles
Linux
To enable corefiles on a Linux system, we have to enable it via sysctl(8), create the directory to hold the corefiles and also set the ulimit accordingly:
mkdir -m1733 /var/crash || chmod 1733 /var/crash echo 'kernel.core_pattern=/var/crash/core.%e.%p.sig-%s' >> /etc/sysctl.d/local.conf sysctl -p /etc/sysctl.d/local.conf
This will set the corefile format to core.%processname.%pid.sig-%signal
Be sure to raise the corefile size limit, preferably via /etc/security/limits.conf:
ulimit -c 1048576 # 1024 MB max corefile size
Note: /proc/sys/kernel/core_uses_pid has been obsoleted by /proc/sys/kernel/core_pattern.
- Application Core Files in SUSE Linux (Aaron Gresko, 27 Oct 2005)
- core(5)
Linux kernel
To enable kernel crashdumps[1]:
sudo apt-get install kexec-tools kdump-tools crash # Debian sudo apt-get install linux-crashdump # Ubuntu[2]
At a minimum, the following should be set:
$ grep ^[A-Z] /etc/default/kexec LOAD_KEXEC=true KERNEL_IMAGE="/vmlinuz" INITRD="/initrd.img" APPEND="" USE_GRUB_CONFIG=false $ grep ^[A-Z] /etc/default/kdump-tools USE_KDUMP=1 KDUMP_COREDIR="/var/crash" MAKEDUMP_ARGS="-c -d 31"
The kernel needs to support crashdumps too:
CONFIG_KEXEC=y CONFIG_CRASH_DUMP=y CONFIG_DEBUG_INFO=y CONFIG_DEBUG_KERNEL=y ...?
If all goes well, we should[3] have something like this:
$ kdump-config status current state : ready to kdump
$ kdump-config show USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0x2f000000 current state: ready to kdump kernel link: /usr/lib/debug/boot/vmlinux-3.16.0-4-amd64 kexec command: /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64 root=UUID=3222f73c-deb2-49d1-a74f-3c9438aebc91 ro console=ttyS0,115200 console=tty0 irqpoll maxcpus=1 nousb systemd.unit=kdump-tools.service" --initrd=/boot/initrd.img-3.16.0-4-amd64 /boot/vmlinuz-3.16.0-4-amd64
The debug kernel will be assumed to be installed in /usr/lib/debug/vmlinux-$(uname -r)
$ sudo apt-get install linux-image-amd64-dbg $ ls -gohL /usr/lib/debug/boot/vmlinux-$(uname -r) -rw-r--r-- 1 117M Jul 17 17:17 /usr/lib/debug/boot/vmlinux-3.16.0-4-amd64
With all that in place, we should be ready to dump:
sysctl vm.drop_caches=3 echo s > /proc/sysrq-trigger echo c > /proc/sysrq-trigger
After the system is up again, a crashdump should exist in KDUMP_COREDIR and we can use crash to analyze the dump:
$ sudo crash /usr/lib/debug/vmlinux-3.16.0-4-amd64 /var/crash/201508030242/dump.201508030242
KERNEL: /usr/lib/debug/vmlinux-3.16.0-4-amd64
DUMPFILE: /var/crash/201508030242/dump.201508030242 [PARTIAL DUMP]
CPUS: 4
DATE: Mon Aug 3 02:41:48 2015
UPTIME: 00:06:45
LOAD AVERAGE: 0.00, 0.04, 0.04
TASKS: 130
NODENAME: foobar
RELEASE: 3.16.0-4-amd64
VERSION: #1 SMP Debian 3.16.7-ckt11-1+deb8u2 (2015-07-17)
MACHINE: x86_64 (2195 Mhz)
MEMORY: 1 GB
PANIC: "Oops: 0002 [#1] SMP " (check log for details)
PID: 1278
COMMAND: "bash"
TASK: ffff88003cd78050 [THREAD_INFO: ffff88003bb0c000]
CPU: 0
STATE: TASK_RUNNING (PANIC)
crash>
Solaris
To enable corefiles on a Solaris system, we have to enable it via coreadm(1M), create the directory to hold the corefiles and also set the ulimit accordingly:
mkdir -m1733 /var/core # Should be already in place
coreadm -g '/var/core/core.%f.%p' -i '/var/core/core.%f.%p' \
-e global -e process -e global-setid -e proc-setid -e log
ulimit -c 524288
This will modify /etc/coreadm.conf as well, so it'll persistent across reboots.
MacOS
In MacOS 10.7, corefiles are usually enabled from a kernel POV, but a few things have to be prepared:
$ sysctl -a | grep kern.core kern.corefile = /cores/core.%P kern.coredump = 1 $ chmod 1733 /cores # /cores should be already in place, but only root could write to it $ ulimit -c 524288
Note: corefiles on MacOS X seem to be very large. Dumping core from a freshly started bash-process produces a ~360MB corefile.