pcapFS is a FUSE module allowing it to mount captured network data as a virtual file system. This makes it especially convenient to analyze the payload (and to some extend the metadata) of your captured network traffic.
While there are already several tools out there which are able to extract data from your PCAPs, pcapFS has some features that make it different from these tools—most notably:
Instead of extracting the payload (i.e. copying the data to disk), pcapFS provides direct access into the PCAP files. To speed the access up, an index is created when a PCAP is mounted for the first time. This takes almost the same time as opening a PCAP with Wireshark. After the index is created, we can use it for all further operations. Moreover, the index can be used to mount the PCAP any time later making the data available almost instantly.
In pcapFS each protocol and decoder is implemented as a virtual file. These virtual files store references into other virtual files or directly into the PCAP, which are used to read their data. Currently the following protocols and decoders are supported:
We do not provide any precompiled packages yet. This is mainly because a lot of the dependencies of pcapFS are also not available as packages in most of the Linux distribution around. So, for the moment you have to build pcapFS from source.
Building pcapFS works best on rather modern Linux distribution. See the corresponding section of this README for further details.
As already mentioned, there are several dependencies which are not packaged for most Linux distributions. Moreover, you need a reasonably modern C++ compiler supporting at least C++14. Depending on your Linux distribution there are different steps required to compile pcapFS. Have a look at the scripts here.
Afterwards you can build pcapFS like:
$ mkdir build $ cd build $ cmake .. $ make
$ ./scripts/dependencies/install-all-dependencies.sh #optional, if you don't want to install everything yourself (use at your own risk) $ ./scripts/dependencies/install-catch2.sh #optional, if you don't want to install everything yourself (use at your own risk) $ mkdir build $ cd build $ cmake -DBUILD_TESTING=on .. $ make
The general way to mount a network capture looks like this:
$ pcapfs [options] <pcap> <mountpoint>
So, just mounting a single PCAP is as simple as:
$ pcapfs /path/to/some/test.pcap /mount/point
To unmount a previously mounted network capture use
fusermount3 with the
$ fusermount3 -u /mount/point
Since the example above did not specify any index file, pcapFS automatically creates an index file for you. This file
will be in the current working directory and will be named somthing like
20181130-125450_pcapfs.index (the first
component is the date when the index was created, the second the time, and the last one is just a fixed string). You can
use this index if you want to mount the PCAP again using the
$ pcapfs -i 20181130-125450_pcapfs.index /path/to/some/pcap /mount/point
If you provide a path to a non-existing index file on the command line, an index with this name will be created for you.
If you don't want your index to be written to disk, use the
--in-memory options. This skips the writing of
the index which, of course, means that the index has to be rebuilt the next time you want to mount the PCAP.
pcapFS lets you mount multiple PCAPs at the same time. The mount point will contain the payload of all PCAPs as if only one PCAP would have been mounted. It makes no difference if the PCAPs you mount are completely unrelated or if you are providing a very long network capture split into several PCAPs. Note that conversations spanning over two or more PCAPs are entirely supported by pcapFS, i.e. no prior merging of PCAPs is required in order to extract your long lasting download from multiple PCAPs!
For this purpose, you can specify a directory instead of a regular PCAP file:
$ pcapfs /path/to/some/pcaps/ /mount/point
In the example above pcapFS would try to mount all regular files contained in the
/path/to/some/pcaps folder. If you
want to limit the files to be mounted, you can provide a file name suffix to only include files ending with this
$ pcapfs --pcap-suffix=.pcap /path/to/some/pcaps/ /mount/point
This would tell pcapFS to only mount files ending with
.pcap from the directory
If nothing else is specified, pcapFS will create a directory structure looking something like this:
$ pcapfs /path/to/some/test.pcap /mnt/point $ tree -r -L 1 /mnt/point /mnt/point/ ├── udp ├── tcp ├── ssl ├── http ├── ftp └── dns 6 directories, 0 files
That is, the first directory level contains the protocols detected and parsed by pcapFS. Within these directories you will find the payload of the corresponding conversations as files.
$ tree -r -L 2 /mnt/point/ | grep -A 3 -E ' (udp|tcp|ssl|http|dns)' ├── udp │ ├── 0-9_UDPFILE3 │ ├── 0-99816_UDPFILE1522 │ ├── 0-99773_UDPFILE1521 -- ├── tcp │ ├── 0-99886_tcp3927 │ ├── 0-9977_tcp687 │ ├── 0-99112_tcp3922 -- ├── ssl │ ├── 9997-656_SSL │ ├── 999-5_SSL │ ├── 9984-3081_SSL -- ├── http │ ├── 998-811 │ ├── 9986-93333_icons-16x16.png │ ├── 9986-81178_header-desk-logo.png -- └── dns ├── 998-0_RES-18314 ├── 997-0_REQ-18314 ├── 99-0_RES-63051
pcapFS is, however, not limited to this directory layout. Instead, it lets you choose the layout that is most suitable for your current analysis. For instance, assume that you are interested in the ports a particular host has send packets to. In this case you could call pcapFS like this:
$ pcapfs --sortby=/srcIP/dstPort/dstIP /path/to/some/test.pcap /mount/point
After that your directory hierarchy should look like the following:
$ tree -rd -L 3 /mnt/point/ /mnt/point/ ... ├── 172.16.139.241 │ └── 53 │ └── 172.16.128.202 ├── 172.16.133.99 │ ├── 8200 │ │ └── 22.214.171.124 │ ├── 5500 │ │ └── 172.16.139.250 │ ├── 443 │ │ ├── 126.96.36.199 │ │ ├── 188.8.131.52 │ │ ├── 184.108.40.206 │ │ ├── 220.127.116.11 │ │ ├── 18.104.22.168 │ │ ├── 22.214.171.124 │ │ ├── 126.96.36.199 │ │ ├── 188.8.131.52 │ │ ├── 184.108.40.206 │ │ ├── 220.127.116.11 │ │ ├── 18.104.22.168 │ │ ├── 22.214.171.124 │ │ └── 126.96.36.199 │ ├── 1900 │ │ └── 188.8.131.52 │ ├── 1853 │ │ └── 184.108.40.206 │ ├── 138 │ │ └── 172.16.133.255 │ └── 137 │ └── 172.16.133.255 ├── 172.16.133.97 │ ├── 8014 │ │ └── 172.16.128.169 │ ├── 5500 │ │ └── 172.16.139.250 │ ├── 5462 │ │ └── 172.16.139.250 │ ├── 5447 │ │ └── 172.16.139.250 │ ├── 443 │ │ ├── 220.127.116.11 │ │ ├── 18.104.22.168 │ │ └── 22.214.171.124 │ ├── 1900 ...
--sortby argument used above defines the layout of the virtual directory hierarchy created for you. pcapFS
provides what we call properties for this. The following table lists the properties which are currently available
along with the protocol they origin from:
|protocol||n/a||A protocol implemented in pcapFS|
|srcIP||ip||Source IP address|
|dstIP||ip||Destination IP address|
|srcPort||tcp, udp||Source port|
|dstPort||tcp, udp||Destination port|
|srcPort||tcp, udp||Source port|
|domain||http||The domain parsed from the HTTP Host header|
|path||http||The path parsed from a HTTP request|
A protocol implemented in pcapFS can define its own properties based on values it parsed. Therefore, as more and more protocols are added to pcapFS, you will have very fine grained possibilities to build your directory hierarchy.
Note that the current implementation does not check whether a property you specified actually exists. That is, you
could also provide the following
$ pcapfs --sortby=/foo/protocol/domain/path /path/to/some/test.pcap /mount/point /mount/point └── PCAPFS_PROP_NOT_AVAIL ├── tcp │ └── PCAPFS_PROP_NOT_AVAIL │ ├── 0-139_tcp10 │ └── 0-131_tcp9 ├── ssl │ └── PCAPFS_PROP_NOT_AVAIL │ └── 0-1838_SSL └── http └── server.test ├── image │ ├── 8-308_png │ └── 7-311_jpeg ├── 6-309_json ├── 5-333_gzip ├── 4-339_deflate ├── 3-318_html ├── 2-312_headers └── 1-306_ip 8 directories, 11 files
As you can see, the
foo component lead to the creation of the
PCAPFS_PROP_NOT_AVAIL folder containing the
directories for the protocols. There are additional
PCAPFS_PROP_NOT_AVAIL folders in
ssl. This is
because the parsers for TCP and SSL do not provide the
path properties. The HTTP parser on the other
hand provides these properties leading to the
It is possible for pcapFS to decrypt and decode certain protocols on the fly if you provide it with the corresponding
key material. Right now, we have prototypical support for SSL (just the
SSL_RSA_WITH_RC4_128_MD5 cipher suite) and
XOR. Both need a key file containing the key material which can be provided either via the command line (
--keys) or via the configuration file. The argument can be a single file or a directory
containing multiple key files. Example key files can be found in the tests folder. Note that
we are still in the process of deciding on an adequate file format, so be prepared for changes here.
pcapFS uses TOML as the format for its configuration file. A sample config file looks like this:
[general] sortby = "/dstIP/dstPort/srcIP" [keys] keyfiles = [ "/path/to/some/key.file", "relative/path/to/other/key.file", ] [[decode.xor.properties]] srcIP = "126.96.36.199" dstIP = "188.8.131.52" dstPort = 2345 [[decode.xor.properties]] srcPort = 1111 dstPort = 2222 protocol = "udp" [[decode.ssl.properties]] srcIP = "184.108.40.206" srcPort = 8080
[general] section allows setting the
sortby option described above.
[keys] section allows you to define a list of paths to key files. Note that relative paths are interpreted as
relative to the config file. Just as with the
-k command line option, you are free to use files or directories here.
[decode] section can be used to provide custom protocol parsing and decoding rules. That is, you can tell pcapFS
which parser to use for connections meeting given criteria. The example config above defines three rules, two for XOR
decoding and one for SSL. As the
properties key implies, you can use pcapFS properties to define your decoding rules.
In case of the SSL example above, all connections from source IP 220.127.116.11 and source Port 8080 would be parsed with the
SSL protocol parser. For XOR we defined two rules both stating that connection meeting the criteria should be parsed
with the XOR parser: the first one matches all connections from source IP 18.104.22.168 to destination IP 22.214.171.124 and
destination port 2345, the second one matches all UDP "connections" from source port 1111 to destination port 2222.
Note that decoding options are independent from an implemented protocol detection. E.g. you can specify a certain port for HTTP decoding, but the HTTP parser still checks if the transferred data over this port is valid HTTP.