Sunday, February 21, 2021

Building a Linux profile for Volatility 2 and 3

I needed to analyze a memory dump with Volatility 2/3. These are the steps I followed to build a Linux profile based on Red Hat Enterprise Linux (RHEL) 5.11 x64 (kernel version 2.6.18-398). At the time of writing, the Volatility repository doesn't have a profile for this OS (1)

I installed RHEL511x64 on VMware Workstation v14 using default settings during setup (always press Next). No updates once installed the OS. 

Note: if you're following this procedure and you need to upgrade the kernel to a more recent version, first download these packages from :
  • kernel-<KernelVersion>.x86_64.rpm
  • kernel-headers-<KernelVersion>.x86_64.rpm
  • kernel-debug-<KernelVersion>.x86_64.rpm
  • kernel-debug-devel-<KernelVersion>.x86_64.rpm
  • kernel-devel-<KernelVersion>.x86_64.rpm
Install them as root, reboot, on the boot menu choose the new kernel to load and then proceed to install the packages listed right below (skip the kernel* ones).

Since the VM was already running the kernel version I needed, from the mounted ISO I installed these packages that would be needed later (2):

cd /media/RHEL_5.11\ x86_64\ DVD/Server/
rpm -iv --force kernel-headers-$(uname -r)*.rpm kernel-debug-$(uname -r)*.rpm kernel-debug-devel-$(uname -r)*.rpm kernel-devel-$(uname -r)*.rpm glibc-headers-*.x86_64.rpm glibc-devel-*.x86_64.rpm cpp-*.x86_64.rpm gcc-[0-9.-]*.x86_64.rpm zlib-devel-*.x86_64.rpm elfutils-libelf-devel-*.x86_64.rpm libstdc++-devel-*.x86_64.rpm gcc-c++-*.x86_64.rpm
I then installed the VMware Tools for convenience so that I could copy and paste between the host OS and this VM.

From this repository:
I downloaded the appropriate kernel-debuginfo packages (3):
  • kernel-debuginfo-common-2.6.18-398.el5.x86_64.rpm
  • kernel-debug-debuginfo-2.6.18-398.el5.x86_64.rpm
  • kernel-debuginfo-2.6.18-398.el5.x86_64.rpm
which I pasted for convenience in the /tmp path.
cd /tmp
rpm -iv kernel-debuginfo-common*-$(uname -r)*.rpm kernel-debug-debuginfo-$(uname -r)*.rpm kernel-debuginfo-$(uname -r)*.rpm
Next step was to download libdwarf/dwarfdump from After trying to compile different versions, I eventually succeeded with libdwarf-20180129.tar.gz.
cd /tmp
tar -zxvf libdwarf-20180129.tar.gz
cd dwarf-20180129/
cp dwarfdump/dwarfdump /usr/local/sbin/
From the master branch on GitHub, I pulled the latest version available of Volatility 2.
cd /tmp
mv volatility-master/ volatility/
But make failed with this error when I tried to generate the dwarf module (4).

# cd volatility/tools/linux/
# make

make -C //lib/modules/2.6.18-398.el5/build CONFIG_DEBUG_INFO=y M="/tmp/volatility/tools/linux" modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-398.el5-x86_64'
  CC [M]  /tmp/volatility/tools/linux/module.o
/tmp/volatility/tools/linux/module.c:218: error: redefinition of ‘struct module_sect_attr’
/tmp/volatility/tools/linux/module.c:225: error: redefinition of ‘struct module_sect_attrs’
/tmp/volatility/tools/linux/module.c:379:5: warning: "STATS" is not defined
/tmp/volatility/tools/linux/module.c:395:5: warning: "DEBUG" is not defined
make[2]: *** [/tmp/volatility/tools/linux/module.o] Error 1
make[1]: *** [_module_/tmp/volatility/tools/linux] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.18-398.el5-x86_64'
make: *** [dwarf] Error 2

The workaround sed -i 's/PWD/shell pwd/g' Makefile  was of no help (5).

I looked at the source code of module.c (volatility/tools/linux/) and found a section starting with #if LINUX_VERSION_CODE == KERNEL_VERSION(2,6,18). As an attempt to bypass the issue, I deleted the whole section, specifically from line 212 to 239 (6).

I tried make again and this time it worked!

make -C //lib/modules/2.6.18-398.el5/build CONFIG_DEBUG_INFO=y M="/tmp/volatility/tools/linux" modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-398.el5-x86_64'
  CC [M]  /tmp/volatility/tools/linux/module.o
/tmp/volatility/tools/linux/module.c:350:5: warning: "STATS" is not defined
/tmp/volatility/tools/linux/module.c:366:5: warning: "DEBUG" is not defined
  Building modules, stage 2.
  CC      /tmp/volatility/tools/linux/module.mod.o
  LD [M]  /tmp/volatility/tools/linux/module.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-398.el5-x86_64'
dwarfdump -di module.ko > module.dwarf
make -C //lib/modules/2.6.18-398.el5/build M="/tmp/volatility/tools/linux" clean
make[1]: Entering directory `/usr/src/kernels/2.6.18-398.el5-x86_64'
  CLEAN   /tmp/volatility/tools/linux/.tmp_versions
make[1]: Leaving directory `/usr/src/kernels/2.6.18-398.el5-x86_64'

And the last step for Volatility 2 was to create the zip archive for the profile (7).
cd /tmp
zip volatility/tools/linux/module.dwarf /boot/$(uname -r)
Regarding Volatility 3, I made a copy of these two files:
cd /tmp
cp /boot/$(uname -r) .
cp /usr/lib/debug/lib/modules/$(uname -r)/vmlinux vmlinuz-$(uname -r)
I suspended the VM and ran vmss2core (included in the installation of VMware Workstation or downloadable from vmware Flings) on my host to create a vmss.core file from the .vmem and .vmss files listed in the picture below. (The presence or not of both files depends on the hypervisor you're using and the conversion to vmss.core is not always needed).

.vmem and .vmss

This is a general description of vmss2core.

"C:\Program Files (x86)\VMware\VMware Workstation\vmss2core.exe"
vmss2core version 14921873 Copyright (C) 1998-2019 VMware, Inc. All rights reserved.
Missing .vmss filename

   A tool to convert VMware checkpoint state files into formats
   that third party debugger tools understand. It can handle both
   suspend (.vmss) and snapshot (.vmsn) checkpoint state files
   (hereafter referred to as a 'vmss file') as well as both
   monolithic and non-monolithic (separate .vmem file) encapsulation
   of checkpoint state data.


      GENERAL:  vmss2core [[options] | [-l linuxoffsets options]] \
                  <vmss file> [<vmem file>]
   output can be produced with these options:

   -N6     Red Hat crash core file for Linux 2.6 (vmss.core).

This is how I ran the tool (8).

C:>"C:\Program Files (x86)\VMware\VMware Workstation\vmss2core.exe" -N6 RHEL511x64-328dacd0.vmss RHEL511x64-328dacd0.vmem
vmss2core version 14921873 Copyright (C) 1998-2019 VMware, Inc. All rights reserved.
Started core writing.
Writing note section header.
Writing 1 memory section headers.
Writing notes.
... 10 MBs written.
... 20 MBs written.
... 30 MBs written.
... 1020 MBs written.
Finished writing core.

The output is a vmss.core file. For its analysis with Volatility 2/3, I used two different Ubuntu VMs to avoid any problem with different versions of Python, virtualenv, dependencies of other tools and so no.

Volatility 2.6.1 (tested with: Ubuntu 16.04.7 LTS, Python 2.7.12) 

These are the steps I followed to install Volatility 2.6.1:
git clone
sudo apt-get install libpcre++-dev pcregrep python-dev python-pip subversion yara
pip install distorm3 PyCrypto yara-python
cd volatility
python build
sudo python build install
python -h
I copied the zipped profile for Volatility 2 to volatility/volatility/plugins/overlays/linux and ran python --plugins=list --info to check if the profile was detected.

Volatility Foundation Volatility Framework 2.6.1

LinuxRedHat511x64     - A Profile for Linux RedHat511 x64

These are some of the plugins I tested using the newly created profile:

python -f ~/Desktop/vmss.core --profile=LinuxRedHat511x64 <linux plugin>




Volatility 3 v2.3.0 (tested with: Ubuntu 20.04.4, Python 3.8.10)

Volatility 3 installation (9)
sudo apt install git python3-pip
sudo pip3 install capstone jsonschema leechcorepyc pefile pycryptodome python-snappy==0.6.0 yara-python
git clone
cd volatility3
python3 build
sudo python3 build install
python3 -h
dwarf2json (10)
sudo snap install go --classic
git clone
cd dwarf2json/
go build
./dwarf2json linux
To create a "profile" for Volatility 3 (11), I ran dwarf2json against the files vmlinux and that I had exported before from the VM (12):
./dwarf2json linux --elf vmlinuz-2.6.18-398.el5 --system-map | xz -c > RHEL511x64_2.6.18-398.json.xz
If the dwarf2json step fails, try to increase the amount of memory assigned to the VM (8GB+).

Then I:
  • went to ~/volatility3/volatility3/symbols
  • created a subdirectory named linux
  • copied in here the .json.xz archive created with dwarf2json
I ran Volatility 3 (with the verbosity option) against the vmss.core file to check if the Linux banner was successfully detected.

python3 -vvvv -f ~/Desktop/vmss.core linux.pslist.PsList
Volatility 3 Framework 1.0.1
INFO     root        : Volatility plugins path: ['/home/user/volatility3/volatility3/plugins', '/home/user/volatility3/volatility3/framework/plugins']
INFO     root        : Volatility symbols path: ['/home/user/volatility3/volatility3/symbols', '/home/user/volatility3/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a linux category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 2.6.18-398.el5 ( (gcc version 4.1.2 20080704 (Red Hat 4.1.2-55)) #1 SMP Tue Aug 12 06:26:17 EDT 2014\n\x00'

The banner appears if it matches the banner stored inside the memory dump.

strings and Linux banner

And this was the ouput of the linux.pstree.PsTree plugin.

python3 -f ~/Desktop/vmss.core linux.pstree.PsTree

I put the two profiles on my GitHub:

  3. Linux plugins issues:
  4. compilation error module.c against CentOS 5.11 kernel 2.6.18-398.el5:
  5. fail to create linux profile:
  8. Converting a snapshot file to memory dump using the vmss2core tool: