Moved fuzz dir to test/fuzz

Extended contributing document to testing and licensing
This commit is contained in:
Olof hagsand 2021-10-10 18:03:33 +02:00
parent edbbb43e1f
commit 8ede8a87b2
29 changed files with 62 additions and 1053 deletions

44
test/fuzz/README.md Normal file
View file

@ -0,0 +1,44 @@
# Fuzzing with AFL
Clixon can be fuzzed with [american fuzzy lop](https://github.com/google/AFL/releases) but not without pain.
Some issues are as follows:
- Static linking. Fuzzing requires static linking. You can statically link clixon using: `LINKAGE=static ./configure` but that does not work with Clixon plugins (at least yet). Therefore fuzzing has been made with no plugins using the hello example only.
- Multiple processes. Only the backend can run stand-alone, cli/netconf/restconf requires a backend. When you fuzz eg clixon_cli, the backend must be running and it will be slow due to IPC. Possibly one could link them together and run as a monolith by making a threaded image.
Restconf also has the extra problem of running TLS sockets.
## Prereqs
See [AFL docs](https://afl-1.readthedocs.io/en/latest) for installing afl.
On ubuntu this may be enough:
```
sudo apt install afl
```
Or get source: `https://github.com/google/AFL`
You may have to change cpu frequency:
```
cd /sys/devices/system/cpu
echo performance | tee cpu?/cpufreq/scaling_governor
```
And possibly change core behaviour:
```
echo core >/proc/sys/kernel/core_pattern
```
### backend/restconf
Backend and restconf requires the preeny package to change sockets to stdio.
Preeny has a "desocketizing" module necessary to map stdio to the internal sockets that the backend uses. Install preeny example:
```
sudo apt install libini-config-dev # debian/ubuntu
sudo apt install libseccomp-dev # debian/ubuntu
git clone https://github.com/zardus/preeny.git
cd preeny
make
sudo cp x86_64-linux-gnu/desock.so /usr/local/lib/ # install
```

View file

@ -0,0 +1,37 @@
# Clixon fuzzing
This dir contains code for fuzzing clixon backend. (NOTE DOES NOT WORK)
It requires the preeny package to change sockets to stdio.
Plugins do not work
## Prereqs
Install AFL and preeny, see [..](..)
## Build
Make a modification to how Clixon sends internal messages in `include/clixon_custom.h`:
```
#define CLIXON_PROTO_PLAIN
```
(Note this is obsolete)
Build clixon statically with the afl-clang compiler:
```
CC=/usr/bin/afl-clang-fast LINKAGE=static ./configure --with-restconf=evhtp
make clean
make
sudo make install
```
## Run tests
Populate the input/ dir with input usecases, there are two examples already in this dir that can be modified.
Use the script `runfuzz.sh` to run one test:
```
./runfuzz.sh
```
After (or during) the test, investigate results in the output dir.

View file

@ -0,0 +1 @@
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="42" username="olof"><edit-config><target><candidate/></target><default-operation>none</default-operation><test-option>test-then-set</test-option><error-option>stop-on-error</error-option><config><interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"><interface><name>eth/0/0</name></interface></interfaces></config></edit-config></rpc>]]>]]>

View file

@ -0,0 +1 @@
<hello username="olof" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="42"><capabilities><capability>urn:ietf:params:netconf:base:1.0</capability></capabilities></hello>]]>]]>

54
test/fuzz/backend/runfuzz.sh Executable file
View file

@ -0,0 +1,54 @@
#!/usr/bin/env bash
# Run a fuzzing test using american fuzzy lop
# Add input strings in input
set -eux
if [ $# -ne 0 ]; then
echo "usage: $0 \n"
exit 255
fi
if [ ! -x /usr/local/lib/desock.so ] ; then
echo "preeny desock.so not found"
exit 255
fi
MEGS=500 # memory limit for child process (50 MB)
# remove input and input dirs
#test ! -d input || rm -rf input
test ! -d output || sudo rm -rf output
# create if dirs dont exists
#test -d input || mkdir input
test -d output || mkdir output
APPNAME=example
cfg=conf.xml
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_YANG_MODULE_MAIN>clixon-example</CLICON_YANG_MODULE_MAIN>
<CLICON_SOCK_FAMILY>IPv4</CLICON_SOCK_FAMILY>
<CLICON_SOCK_PORT>4535</CLICON_SOCK_PORT>
<CLICON_SOCK>127.0.0.1</CLICON_SOCK>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>/usr/local/var/$APPNAME</CLICON_XMLDB_DIR>
<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
<CLICON_MODULE_LIBRARY_RFC7895>false</CLICON_MODULE_LIBRARY_RFC7895>
</clixon-config>
EOF
# Run script
# CC=/usr/bin/afl-clang
sudo LD_PRELOAD="/usr/local/lib/desock.so" afl-fuzz -i input -o output -d -m $MEGS -- /usr/local/sbin/clixon_backend -Fs init -f $cfg
# Dryrun without afl:
#echo "sudo LD_PRELOAD=\"/usr/local/lib/desock.so\" /usr/local/sbin/clixon_backend -Fs init -f ./conf.xml < input/1.xml"
#sudo LD_PRELOAD="/usr/local/lib/desock.so" /usr/local/sbin/clixon_backend -Fs init -f ./conf.xml < input/1.xml

61
test/fuzz/cli/README.md Normal file
View file

@ -0,0 +1,61 @@
# Clixon fuzzing
This dir contains code for fuzzing clixon cli.
## Prereqs
Install AFL, see [..](..)
Build and install a clixon system (in particular the backend, the CLI will be replaced)
Build and install CLIgen statically:
```
./configure LINKAGE=static INSTALLFLAGS="" CC=/usr/bin/afl-clang-fast
```
## Build
Build clixon cli statically with the afl-clang compiler:
```
CC=/usr/bin/afl-clang-fast LINKAGE=static INSTALLFLAGS="" ./configure # Dont care about restconf
make clean
cd apps/cli
make clixon_cli
sudo make install
```
To link an example plugin properly it gets a little more complex::
- First, you need to identify which example plugins you want to link. Add these to `EXTRAS` variable
- Configure and compile those plugins, where the `clixon_plugin_init()` function is removed.
- Configure and compile the cli WITH the `EXTRAS` variable set.
Below is an example of how to do this for the main example. You can replace the main example plugins with another application:
```
CC=/usr/bin/afl-clang-fast CFLAGS="-O2 -Wall -DCLIXON_STATIC_PLUGINS" LINKAGE=static INSTALLFLAGS="" ./configure
make clean
make
sudo make install
cd example # Compile and install application plugins (here main example)
make clean
make
sudo make install
cd ..
cd apps/cli # Compile and install clixon_cli with pre-compiled plugins
rm clixon_cli
EXTRAS="../../example/main/example_cli.o" make clixon_cli
sudo make install
```
## Run tests
Run the script `runfuzz.sh` to run one test with a cli spec and an input string, eg:
```
./runfuzz.sh
```
After (or during) the test, investigate results in the output dir.

View file

@ -0,0 +1,3 @@
set table parameter a value 42
table
example 42

View file

@ -0,0 +1 @@
show configuration

View file

@ -0,0 +1,2 @@
validate
commit

52
test/fuzz/cli/runfuzz.sh Executable file
View file

@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Run a fuzzing test using american fuzzy lop
set -eux
if [ $# -ne 0 ]; then
echo "usage: $0\n"
exit 255
fi
APPNAME=example
cfg=conf.xml
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_YANG_MODULE_MAIN>clixon-example</CLICON_YANG_MODULE_MAIN>
<CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>/usr/local/var/$APPNAME</CLICON_XMLDB_DIR>
<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
<CLICON_MODULE_LIBRARY_RFC7895>false</CLICON_MODULE_LIBRARY_RFC7895>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
<CLICON_CLI_GENMODEL_COMPLETION>1</CLICON_CLI_GENMODEL_COMPLETION>
<CLICON_CLI_GENMODEL_TYPE>VARS</CLICON_CLI_GENMODEL_TYPE>
<CLICON_CLI_AUTOCLI_EXCLUDE>clixon-restconf</CLICON_CLI_AUTOCLI_EXCLUDE>
<CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
<CLICON_CLI_TAB_MODE>0</CLICON_CLI_TAB_MODE>
</clixon-config>
EOF
# Kill previous
sudo clixon_backend -z -f $cfg -s init
# Start backend
sudo clixon_backend -f $cfg -s init
MEGS=500 # memory limit for child process (50 MB)
# remove input and input dirs
#test ! -d input || rm -rf input
test ! -d output || rm -rf output
# create if dirs dont exists
#test -d input || mkdir input
test -d output || mkdir output
# Run script
afl-fuzz -i input -o output -m $MEGS -- clixon_cli -f $cfg

View file

@ -0,0 +1,30 @@
# Clixon fuzzing
This dir contains code for fuzzing clixon netconf.
## Prereqs
Install AFL, see [..](..)
Build and install a clixon system (in particular the backend, the netconf will be replaced)
## Build
Build clixon netconf statically with the afl-clang compiler:
```
CC=/usr/bin/afl-clang-fast LINKAGE=static ./configure # Dont care about restconf
make clean
cd apps/netconf
make clixon_netconf
sudo make install
```
## Run tests
Run the script `runfuzz.sh` to run one test with a cli spec and an input string, eg:
```
./runfuzz.sh
```
After (or during) the test, investigate results in the output dir.

View file

@ -0,0 +1 @@
<rpc message-id="42" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config><target><candidate/></target><config><table xmlns="urn:example:clixon"><parameter><name>a</name></parameter></table></config></edit-config></rpc>]]>]]>

View file

@ -0,0 +1 @@
<rpc message-id="99" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">><commit/></rpc>]]>]]>

View file

@ -0,0 +1 @@
<rpc message-id="238" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">><get-config><source><running/></source></get-config></rpc>]]>]]>

52
test/fuzz/netconf/runfuzz.sh Executable file
View file

@ -0,0 +1,52 @@
#!/usr/bin/env bash
# Run a fuzzing test using american fuzzy lop
set -eux
if [ $# -ne 0 ]; then
echo "usage: $0\n"
exit 255
fi
APPNAME=example
cfg=conf.xml
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_YANG_MODULE_MAIN>clixon-example</CLICON_YANG_MODULE_MAIN>
<CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>/usr/local/var/$APPNAME</CLICON_XMLDB_DIR>
<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
<CLICON_MODULE_LIBRARY_RFC7895>false</CLICON_MODULE_LIBRARY_RFC7895>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
<CLICON_CLI_GENMODEL_COMPLETION>1</CLICON_CLI_GENMODEL_COMPLETION>
<CLICON_CLI_GENMODEL_TYPE>VARS</CLICON_CLI_GENMODEL_TYPE>
<CLICON_CLI_AUTOCLI_EXCLUDE>clixon-restconf</CLICON_CLI_AUTOCLI_EXCLUDE>
<CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
<CLICON_CLI_TAB_MODE>0</CLICON_CLI_TAB_MODE>
</clixon-config>
EOF
# Kill previous
sudo clixon_backend -z -f $cfg -s init
# Start backend
sudo clixon_backend -f $cfg -s init
MEGS=500 # memory limit for child process (50 MB)
# remove input and input dirs
#test ! -d input || rm -rf input
test ! -d output || rm -rf output
# create if dirs dont exists
#test -d input || mkdir input
test -d output || mkdir output
# Run script
afl-fuzz -i input -o output -m $MEGS -x xml.dict -- clixon_netconf -qf $cfg -o CLICON_NETCONF_HELLO_OPTIONAL=true

View file

@ -0,0 +1,47 @@
#
# AFL dictionary for XML
# ----------------------
#
# Several basic syntax elements and attributes, modeled on libxml2.
#
# Created by Michal Zalewski <lcamtuf@google.com>
#
attr_encoding=" encoding=\"1\""
attr_generic=" a=\"1\""
attr_href=" href=\"1\""
attr_standalone=" standalone=\"no\""
attr_version=" version=\"1\""
attr_xmlns=" xmlns=\"1\""
attr_xmlns=" xmlns:ns=\"1\""
entity_builtin="&lt;"
entity_decimal="&#1;"
entity_external="&a;"
entity_hex="&#x1;"
string_any="ANY"
string_brackets="[]"
string_cdata="CDATA"
string_dashes="--"
string_empty_dblquotes="\"\""
string_empty_quotes="''"
string_parentheses="()"
string_pcdata="#PCDATA"
string_percent="%a"
string_public="PUBLIC"
string_ucs4="UCS-4"
string_utf16="UTF-16"
string_utf8="UTF-8"
string_xmlns="xmlns:"
tag_attlist="<!ATTLIST"
tag_cdata="<![CDATA["
tag_close="</a>"
tag_doctype="<!DOCTYPE"
tag_open="<a>"
tag_open_close="<a />"
tag_open_exclamation="<!"
tag_open_q="<?"
tag_sq2_close="]]>]]>"
tag_xml_q="<?xml?>"

View file

@ -0,0 +1,31 @@
# Clixon fuzzing
This dir contains code for fuzzing clixon restconf
It requires the preeny package to change sockets to stdio.
## Prereqs
Install AFL and preeny, see [..](..)
Build and install a clixon system (in particular the backend, RESTCONF binary will be replaced)
## Build
Build clixon restconf statically with the afl-clang compiler:
```
CC=/usr/bin/afl-clang-fast LINKAGE=static ./configure --with-restconf=native
make clean
cd apps/restconf
make clixon_restconf
sudo make install
```
## Run tests
Use the script `runfuzz.sh` to run one test:
```
./runfuzz.sh
```
After (or during) the test, investigate results in the output dir.

View file

@ -0,0 +1,10 @@
PUT /restconf/data/ietf-interfaces:interfaces/interface=eth%2f0%2f0 HTTP/1.1
Host: 127.0.0.1
Accept: */*
Content-Type: application/yang-data+json
Content-Length: 91
{"ietf-interfaces:interface":{"name":"eth/0/0","type":"clixon-example:eth","enabled":true}}

View file

@ -0,0 +1,5 @@
GET /restconf/data/ HTTP/1.1
Host: localhost
Accept: application/yang-data+xml

View file

@ -0,0 +1,8 @@
POST /restconf/data/ietf-interfaces:interfaces HTTP/1.1
Host: 127.0.0.1
Accept: application/yang-data+xml
Content-Type: application/yang-data+xml
Content-Length: 138
<interface xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"><name>eth/0/42</name><type>ex:eth</type><enabled>true</enabled></interface>

View file

@ -0,0 +1,4 @@
DELETE /restconf/data/ietf-interfaces:interfaces HTTP/1.1
Host: 127.0.0.1

65
test/fuzz/restconf/runfuzz.sh Executable file
View file

@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Run a fuzzing test using american fuzzy lop
# Add input strings in input
set -eux
if [ $# -ne 0 ]; then
echo "usage: $0"
exit 255
fi
if [ ! -x /usr/local/lib/desock.so ] ; then
echo "preeny desock.so not found"
exit 255
fi
MEGS=500 # memory limit for child process (50 MB)
# remove input and input dirs
#test ! -d input || rm -rf input
test ! -d output || sudo rm -rf output
# create if dirs dont exists
#test -d input || mkdir input
test -d output || mkdir output
APPNAME=example
cfg=conf.xml
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_YANG_MODULE_MAIN>clixon-example</CLICON_YANG_MODULE_MAIN>
<CLICON_SOCK>/usr/local/var/hello.sock</CLICON_SOCK>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>/usr/local/var/$APPNAME</CLICON_XMLDB_DIR>
<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
<CLICON_MODULE_LIBRARY_RFC7895>false</CLICON_MODULE_LIBRARY_RFC7895>
<restconf><enable>true</enable><auth-type>none</auth-type><socket><namespace>default</namespace><address>0.0.0.0</address><port>80</port><ssl>false</ssl></socket></restconf>
</clixon-config>
EOF
# Kill previous
echo "cfg: $cfg"
sudo clixon_backend -z -f $cfg -s init
# Start backend
sudo clixon_backend -f $cfg -s init
# Dryrun without afl (comment this if you run for real)
sudo LD_PRELOAD="/usr/local/lib/desock.so" clixon_restconf -rf $cfg < input/1.http || true
sudo LD_PRELOAD="/usr/local/lib/desock.so" clixon_restconf -rf $cfg < input/2.http || true
sudo LD_PRELOAD="/usr/local/lib/desock.so" clixon_restconf -rf $cfg < input/3.http || true
sudo LD_PRELOAD="/usr/local/lib/desock.so" clixon_restconf -rf $cfg < input/4.http || true
exit
# Run script
# CC=/usr/bin/afl-clang
sudo LD_PRELOAD="/usr/local/lib/desock.so" afl-fuzz -i input -o output -d -m $MEGS -- /usr/local/sbin/clixon_restconf -rf $cfg