Analyzing Kernel Modules Data in OProfile in Fedora Core 3/4 and RHEL4

William Cohen
[e-mail]
6/20/2005

OProfile collects data on a system-wide basis for kernel- and
user-space code running on the machine. However, once a module is
loaded into the kernel, the information about the origin of the kernel
module is lost. The module could have come from the initrd file on
boot up, the directory with the various kernel modules, or a locally
created kernel module. As a result when OProfile records sample for a
module, it just lists the samples for the modules for an executable in
the root directory, but this is unlikley to the place with the actual
code for the module. You will need to take some steps to make sure
that analysis tools get the executable.

For example on a AMD64 machine the sampling is set up to record "Data
cache accesses" and "Data cache misses" and you would like to see the
data for the ext3 module:


$ opreport /ext3
CPU: AMD64 processors, speed 797.948 MHz (estimated)
Counted DATA_CACHE_ACCESSES events (Data cache accesses) with a unit mask of 0x00 (No unit mask) count 500000
Counted DATA_CACHE_MISSES events (Data cache misses) with a unit mask of 0x00 (No unit mask) count 500000
DATA_CACHE_ACC...|DATA_CACHE_MIS...|
samples| %| samples| %|
------------------------------------
148721 100.000 1493 100.000 ext3

To get a more detailed view of the actions of the module, you will
need to ether have the module be unstripped (e.g. installed from a
custom build) or have the debuginfo RPM installed for the kernel.

Find out which kernel is running, "uname -a", get the appropriate
debuginfo rpm, and install on the machine.

Then make a symbolic link so oprofile finds the code for the module in the
correct place:

# ln -s /lib/modules/`uname -r`/kernel/fs/ext3/ext3.ko /ext3

Then the detailed information can be obtained with:

# opreport image:/ext3 -l|more
warning: could not check that the binary file /ext3 has not been modified since the profile was taken. Results may be inaccurate.
CPU: AMD64 processors, speed 797.948 MHz (estimated)
Counted DATA_CACHE_ACCESSES events (Data cache accesses) with a unit mask of 0x00 (No unit mask) count 500000
Counted DATA_CACHE_MISSES events (Data cache misses) with a unit mask of 0x00 (No unit mask) count 500000
samples % samples % symbol name
16728 11.2479 7 0.4689 ext3_group_sparse
16454 11.0637 4 0.2679 ext3_count_free_blocks
14583 9.8056 51 3.4159 ext3_fill_super
8281 5.5681 129 8.6403 ext3_ioctl
7810 5.2514 62 4.1527 ext3_write_info
7286 4.8991 67 4.4876 ext3_ordered_writepage
6509 4.3767 130 8.7073 ext3_new_inode
6378 4.2886 156 10.4488 ext3_new_block
5932 3.9887 87 5.8272 ext3_xattr_block_list
...

#!/bin/bash
#
# opreport_module is a script to generate profiling information
# for a module. It should automate the process.
#
# NOTE that his needs to be run as root to create and remove the required
# symlinks
#
# Copyright 2005
# Read the file COPYING
#
# Authors: Will Cohen

# print help message
do_help()
{
echo -en "Usage:\t`basename $0`"
echo -e " module_name [arguments for opreport]"
exit 1
}

# Figure out which version of oprofile is being used.
OP_VERSION=`opcontrol -v|awk '{print $3}'`

# need to warn that make sure that data taken for the kernel running
# check arguments
# there should be one
MODULE_NAME=`basename $1`

# find module path
MODULE_DIRECTORY="/lib/modules/`uname -r`/"
MODULE_PATH=`find $MODULE_DIRECTORY -name "$MODULE_NAME.ko"`

if test ! -f $MODULE_PATH; then
#try .o files before giving up
MODULE_PATH=`find $MODULE_DIRECTORY -name "$MODULE_NAME.o"`
if test ! -f $MODULE_PATH; then
echo "Unable to find a module for $MODULE_NAME" >&2
exit 1
fi
fi

# check where oprofile saved data
# /module
# /lib/module/uname/...
# see if the module file exists
OPROF_MOD_LOCATION="/$MODULE_NAME"
# if not make the link
if test ! -f $OPROF_MOD_LOCATION; then
#make the link
ln -s $MODULE_PATH $OPROF_MOD_LOCATION
fi
# do the analsysis
shift 1
case "$OP_VERSION" in
0.5*)oprofpp $OPROF_MOD_LOCATION $@ ;;
0.6*)opreport image:$OPROF_MOD_LOCATION $@ ;;
0.7*)opreport image:$OPROF_MOD_LOCATION $@ ;;
0.8*)opreport image:$OPROF_MOD_LOCATION $@ ;;
0.9*)opreport image:$OPROF_MOD_LOCATION $@ ;;
esac
# undo any symlink
if test -L $OPROF_MOD_LOCATION; then
#make the link
rm $OPROF_MOD_LOCATION
fi