DensityScout

This tool calculates density (like entropy) for files of any file-system-path to finally output an accordingly descending ordered list. This makes it possible to quickly find (even unknown) malware on a potentially infected Microsoft Windows driven machine.

Author

Christian Wojner

Language

English

License

ISCL

Releases

Changes
Build 45 Unicode, new options -g -P, bugfixes     x
Build 43 Bugfixes     x
Build 42 -     x

Description

DensityScout is a tool that has been written for one purpose: finding (possibly unknown) malware on a potentially infected system. Therefore it takes advantage of the typical approach of malware authors to protect their "products" with obfuscation like run-time-packing and -encryption. The tool itself is based on the concept of our Bytehist tool, btw.

So what does DensityScout do?

DensityScout's main focus is to scan a desired file-system-path by calculating the density of each file to finally print out an accordingly descending list. Usually most Microsoft Windows executables are not packed or encrypted in any way which throws the hits of malicious executables to the top of the list where one can easily focus on.

What's Density?

Density can also be understood as "entropy". However, the algorithm behind density is not 100% equal to the one which entropy is based on. So we decided to choose a different name.

Usage / Syntax

DensityScout is a command-line based tool ...

densityscout [options] file_or_directory

options: -a .............. Show errors and empties, too
         -d .............. Just output data (Format: density|path)
         -l density ...... Just files with density lower than the given value
         -g density ...... Just files with density greater than the given value
         -n number ....... Maximum number of lines to print
         -m mode ......... Mode ABS (default) or CHI (for filesize > 100 Kb)
         -o file ......... File to write output to
         -p density ...... Immediately print if lower than the given density
         -P density ...... Immediately print if greater than the given density
         -r .............. Walk recursively
         -s suffix(es) ... Filetype(s) (i.e.: dll or dll,exe,...)
         -S suffix(es) ... Filetype(s) to ignore (i.e.: dll or dll,exe)
         -pe ............. Include all portable executables by magic number
         -PE ............. Ignore all portable executables by magic number

Note:    Packed and/or encrypted data usually has a much higher density than
         normal data (like text or executable binaries).

Modes:   ABS ... Computes the average distance from the ideal quantity for each
                 byte-state according to the overall byte-quantity of the
                 evaluated file.
                 Typical ABS-density for a packed file: < 0.1
                 Typical ABS-density for a normal file: > 0.9

         CHI ... Just the same as ABS but actually squaring each distance.
                 Typical CHI-density for a packed file: < 100.0
               Typical CHI-density for a normal file: > 1000.0

Why so many options?

Most of the options are focused on performance optimizations and better quality of the results in equal measure. Without any restrictions/filters/understanding DensityScout can easily take hours to finish if you fire it at a whole drive/mount-point as such a mis-configuration easily leads to an unwanted computation of many gigabytes. Also it might not be a good idea to include file-types which are packed/encrypted per se.

 

Having said this it doesn't really make sense to come up with all possible combinations of DensityScout's options so we will show a typical example for finding malware on a potentially infected system using DensityScout.

 

Here is one of the fastest ways to get a quick glance of if there's anything "suspicious" of a specific Microsoft Windows installation's System32 folder:

 

densityscout -pe -p 0.1 -o results.txt c:\Windows\System32

 

The option 

-pe
tells DensityScout to only select PE (Portable Executable) files by checking them against the magic number ("MZ") of portable executables. This approach also matches portable executables with extensions one wouldn't expect.

 

The option 

-p 0.1
instructs DensityScout to throw out a hint on the command-line for each file it found with a density below of what is put next to it as soon as a match is found. If you do not use this option you have to wait until DensityScout is finished and puts out the resulting descending list. Though this option can indeed provide the wanted information quite fast the downside on it is obviously that those hints can never be descending. However, the value 
0.1
in the shown example is a fairly good threshold to get what we want keeping the hints visually manageable.

 

The option 

-o results.txt
defines the output file which will be used for the result list. If this option is not provided the final list is sent directly to SYSOUT which you might not want because of the quantity.

 

The last option specifies the path to start from. In the current example it's only 

C:\Windows\System32
excluding any sub-directories. As mentioned this is one of the fastest approaches but a very common one. To do a recursive run you just have to add the 
-r
option.

What you get ...

On a hypothetically infected Windows 7 Professional installation during the run-time of DensityScout you might see something similar to the following:

DensityScout (Build 45)

by Christian Wojner


Calculating density for file ...
(0.03763) | c:\Windows\System32\bootres.dll
(0.05963) | c:\Windows\System32\network.exe
(0.05214) | c:\Windows\System32\WdfCoinstaller01009.dll

 

While 

bootres.dll
and 
WdfCoinstaller01009.dll
are typically legitimate this promptly reveals a suspicious file named 
network.exe
.

The first 20 lines of the final result list may look like this:

(0.03763) | c:\Windows\System32\bootres.dll
(0.05214) | c:\Windows\System32\WdfCoinstaller01009.dll
(0.05963) | c:\Windows\System32\network.exe
(0.11521) | c:\Windows\System32\LkmdfCoInst.dll
(0.12726) | c:\Windows\System32\mcupdate_GenuineIntel.dll
(0.20664) | c:\Windows\System32\iglhsip64.dll
(0.27113) | c:\Windows\System32\pegibbfc.rs
(0.27516) | c:\Windows\System32\usk.rs
(0.27633) | c:\Windows\System32\cero.rs
(0.28895) | c:\Windows\System32\pegi.rs
(0.30524) | c:\Windows\System32\AuthFWGP.dll
(0.30681) | c:\Windows\System32\iscsicpl.exe
(0.32147) | c:\Windows\System32\msshavmsg.dll
(0.32388) | c:\Windows\System32\SrpUxNativeSnapIn.dll
(0.32859) | c:\Windows\System32\qedwipes.dll
(0.34056) | c:\Windows\System32\imagesp1.dll
(0.34697) | c:\Windows\System32\oflc.rs
(0.36592) | c:\Windows\System32\auditpolmsg.dll
(0.36870) | c:\Windows\System32\onexui.dll
(0.38369) | c:\Windows\System32\resmon.exe

 

The next step would be to take a closer look at 

network.exe
trying to find out if it is legitimate or not (actually it is NOT).

 

However, as you can see there won't be a lot of legitimate packed portable executables with a density less than 0.1 (

bootres.dll
and 
WdfCoinstaller01009.dll
in this case) on a healthy Microsoft Windows installation.

Further thinking ...

DensityScout isn't only good for finding malicious executables - it can also be used to find packed or encrypted data-containers and the like!

Be aware!

For the ones that are already aware of our investigations regarding "The WOW Effect" be warned on doing live-forensics and analysis on 64-Bit Microsoft Windows systems using the 32-Bit version of DensityScout (or/and any other 32-Bit based tool). Use the 64-Bit version instead! The ones of you who do not know what this means exactly, please do read our according paper.