Scaling large lists report
This commit is contained in:
Olof hagsand
parent
14d319dd9b
commit
c3d9b392dd
12 changed files with 354 additions and 230 deletions
|
|
@ -102,6 +102,7 @@
|
||||||
|
|
||||||
### Minor changes
|
### Minor changes
|
||||||
|
|
||||||
|
* A scaling of [large lists](doc/scaling) report is added
|
||||||
* A new "hello world" example is added
|
* A new "hello world" example is added
|
||||||
* Optimized validation of large lists
|
* Optimized validation of large lists
|
||||||
* New xmldb_get1() returning actual cache - not a copy. This has lead to some householding instead of just deleting the copy
|
* New xmldb_get1() returning actual cache - not a copy. This has lead to some householding instead of just deleting the copy
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,7 @@ support.
|
||||||
|
|
||||||
* [Background](#background)
|
* [Background](#background)
|
||||||
* [Frequently asked questions (FAQ)](doc/FAQ.md)
|
* [Frequently asked questions (FAQ)](doc/FAQ.md)
|
||||||
|
* [Hello world](example/hello/README.md)
|
||||||
* [Changelog](CHANGELOG.md)
|
* [Changelog](CHANGELOG.md)
|
||||||
* [Installation](#installation)
|
* [Installation](#installation)
|
||||||
* [Licenses](#licenses)
|
* [Licenses](#licenses)
|
||||||
|
|
@ -26,7 +27,7 @@ support.
|
||||||
* [Runtime](#runtime)
|
* [Runtime](#runtime)
|
||||||
* [Clixon project page](http://www.clicon.org)
|
* [Clixon project page](http://www.clicon.org)
|
||||||
* [Tests and CI](test/README.md)
|
* [Tests and CI](test/README.md)
|
||||||
* [Scaling: large lists](doc/large-lists.md)
|
* [Scaling: large lists](doc/scaling/large-lists.md)
|
||||||
* [Containers](docker/README.md)
|
* [Containers](docker/README.md)
|
||||||
* [Roadmap](doc/ROADMAP.md)
|
* [Roadmap](doc/ROADMAP.md)
|
||||||
* [Reference manual](#reference)
|
* [Reference manual](#reference)
|
||||||
|
|
|
||||||
|
|
@ -1,134 +0,0 @@
|
||||||
# Large lists in Clixon
|
|
||||||
|
|
||||||
* [Background](#background)
|
|
||||||
* [Overview](#overview)
|
|
||||||
* [Test descriptions]#test-descriptions)
|
|
||||||
|
|
||||||
## Background
|
|
||||||
|
|
||||||
Clixon is a configuration management tool. In this paper the case of
|
|
||||||
a large number of "flat" list and leaf-list entries are investigated.
|
|
||||||
There may be other scaling usecases, such as large configuratin
|
|
||||||
"depth", large number of requesting clients, etc. However, these are
|
|
||||||
not investigated here.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
The basic case is a large list, according to the following Yang specification:
|
|
||||||
```
|
|
||||||
list y {
|
|
||||||
key "a";
|
|
||||||
leaf a {
|
|
||||||
type int32;
|
|
||||||
}
|
|
||||||
leaf b {
|
|
||||||
type string;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
where `a` is a unique key and `b` is a payload, useful in replace operations.
|
|
||||||
|
|
||||||
There is also a leaf-list as follows:
|
|
||||||
```
|
|
||||||
leaf-list c {
|
|
||||||
type string;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
XML lists with `N` elements are generated based on
|
|
||||||
this configuration, eg for `N=10`:
|
|
||||||
```
|
|
||||||
<y><a>0</a><b>0</b></y>
|
|
||||||
<y><a>1</a><b>1</b></y>
|
|
||||||
<y><a>2</a><b>2</b></y>
|
|
||||||
<y><a>3</a><b>3</b></y>
|
|
||||||
<y><a>4</a><b>4</b></y>
|
|
||||||
<y><a>5</a><b>5</b></y>
|
|
||||||
<y><a>6</a><b>6</b></y>
|
|
||||||
<y><a>7</a><b>7</b></y>
|
|
||||||
<y><a>8</a><b>8</b></y>
|
|
||||||
<y><a>9</a><b>9</b></y>
|
|
||||||
```
|
|
||||||
|
|
||||||
Requests are made using a random function, a request on the list above will on the form:
|
|
||||||
```
|
|
||||||
curl -G http://localhost/restconf/data/y=(rnd%$N)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Test descriptions
|
|
||||||
|
|
||||||
### Limitations
|
|
||||||
|
|
||||||
Test were not made using CLI interaction.
|
|
||||||
|
|
||||||
### Setup
|
|
||||||
|
|
||||||
The setup consisted of the following components running on the same machine:
|
|
||||||
* A clixon backend daemon
|
|
||||||
* A clixon restconf daemon
|
|
||||||
* An nginx daemon daemon
|
|
||||||
* A netconf client program
|
|
||||||
* curl client
|
|
||||||
* A bash terminal and test script [plot_perf.sh](../test/plot_perf.sh)
|
|
||||||
* Gnuplot for generating plots
|
|
||||||
|
|
||||||
### Config file
|
|
||||||
The following Clixon config file was used:
|
|
||||||
```
|
|
||||||
<clixon-config xmlns="http://clicon.org/config">
|
|
||||||
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
|
|
||||||
<CLICON_YANG_DIR>$dir</CLICON_YANG_DIR>
|
|
||||||
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
|
|
||||||
<CLICON_YANG_MODULE_MAIN>scaling</CLICON_YANG_MODULE_MAIN>
|
|
||||||
<CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
|
|
||||||
<CLICON_BACKEND_PIDFILE>/usr/local/var/example/example.pidfile</CLICON_BACKEND_PIDFILE>
|
|
||||||
<CLICON_RESTCONF_PRETTY>false</CLICON_RESTCONF_PRETTY>
|
|
||||||
<CLICON_XMLDB_DIR>$dir</CLICON_XMLDB_DIR>
|
|
||||||
<CLICON_XMLDB_PRETTY>false</CLICON_XMLDB_PRETTY>
|
|
||||||
</clixon-config>
|
|
||||||
```
|
|
||||||
where `$dir` and `$cfg`are local files. For more info see [plot_perf.sh].
|
|
||||||
|
|
||||||
### Testcases
|
|
||||||
|
|
||||||
All tests measure the "real" time of a command on a lightly loaded
|
|
||||||
machine using the Linux command `time(1)`.
|
|
||||||
|
|
||||||
The following tests were made (for each architecture and protocol):
|
|
||||||
* Write `N` entries in one single operation. (With an empty datastore)
|
|
||||||
* Read `N` entries in one single operation. (With a datastore of `N` entries)
|
|
||||||
* Commit `N` entries (With a candidate of `N` entries and empty running)
|
|
||||||
* Read 1 entry (In a datastore of `N` entries)
|
|
||||||
* Write/Replace 1 entry (In a datastore of `N` entries)
|
|
||||||
* Delete 1 entry (In a datastore of `N` entries)
|
|
||||||
|
|
||||||
### Protocols
|
|
||||||
|
|
||||||
The tests are made using:
|
|
||||||
* Netconf[RFC6241] and
|
|
||||||
* Restconf[RFC8040].
|
|
||||||
Notably, CLI tests are for future study.
|
|
||||||
|
|
||||||
### Architectures
|
|
||||||
|
|
||||||
The tests were made on the following hardware, all running Ubuntu Linux:
|
|
||||||
* [i686] dual Intel Core Duo processor (IBM Thinkpad X60), 3GB memory
|
|
||||||
* arm 32-bit (Raspberry PI 3)
|
|
||||||
* x86 64-bit (Intel NUC)
|
|
||||||
|
|
||||||
### Operating systems
|
|
||||||
|
|
||||||
On i686:
|
|
||||||
```
|
|
||||||
Linux version 4.4.0-143-generic (buildd@lgw01-amd64-037) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) ) #169-Ubuntu SMP Thu Feb 7 07:56:51 UTC 2019
|
|
||||||
```
|
|
||||||
|
|
||||||
## Results
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
[RFC6241](https://tools.ietf.org/html/rfc6241) "Network Configuration Protocol (NETCONF)"
|
|
||||||
[RFC8040](https://tools.ietf.org/html/rfc8040) "RESTCONF Protocol"
|
|
||||||
[i686](https://ark.intel.com/content/www/us/en/ark/products/27235/intel-core-duo-processor-t2400-2m-cache-1-83-ghz-667-mhz-fsb.html)
|
|
||||||
[plot_perf.sh](../test/plot_perf.sh) Test script
|
|
||||||
|
|
||||||
|
|
||||||
BIN
doc/scaling/clixon-commit-0.png
Normal file
BIN
doc/scaling/clixon-commit-0.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 6.7 KiB |
BIN
doc/scaling/clixon-delete-100.png
Normal file
BIN
doc/scaling/clixon-delete-100.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.5 KiB |
BIN
doc/scaling/clixon-get-0.png
Normal file
BIN
doc/scaling/clixon-get-0.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8 KiB |
BIN
doc/scaling/clixon-get-100.png
Normal file
BIN
doc/scaling/clixon-get-100.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.4 KiB |
BIN
doc/scaling/clixon-put-0.png
Normal file
BIN
doc/scaling/clixon-put-0.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 7.7 KiB |
BIN
doc/scaling/clixon-put-100.png
Normal file
BIN
doc/scaling/clixon-put-100.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.1 KiB |
131
doc/scaling/large-lists.md
Normal file
131
doc/scaling/large-lists.md
Normal file
|
|
@ -0,0 +1,131 @@
|
||||||
|
# Large lists in Clixon
|
||||||
|
|
||||||
|
* [Background](#background)
|
||||||
|
* [Overview](#overview)
|
||||||
|
* [Test descriptions](#test-descriptions)
|
||||||
|
* [Results](#results)
|
||||||
|
* [References](#references)
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
CIixon can handle large configurations. Here, large number of elements
|
||||||
|
in a "flat" list is presented. There are other scaling usecases,
|
||||||
|
such as large configuratin "depth", large number of requesting
|
||||||
|
clients, etc.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The basic case is a large list, according to the following Yang specification:
|
||||||
|
```
|
||||||
|
container x {
|
||||||
|
description "top-level container";
|
||||||
|
list y {
|
||||||
|
description "List with potential large number of elements";
|
||||||
|
key "a";
|
||||||
|
leaf a {
|
||||||
|
description "key in list";
|
||||||
|
type int32;
|
||||||
|
}
|
||||||
|
leaf b {
|
||||||
|
description "payload data";
|
||||||
|
type string;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
where `a` is a unique key and `b` is a payload, useful in replace operations.
|
||||||
|
|
||||||
|
XML lists with `N` elements are generated based on
|
||||||
|
this configuration, eg for `N=10`:
|
||||||
|
```
|
||||||
|
<y><a>0</a><b>0</b></y>
|
||||||
|
<y><a>1</a><b>1</b></y>
|
||||||
|
<y><a>2</a><b>2</b></y>
|
||||||
|
<y><a>3</a><b>3</b></y>
|
||||||
|
<y><a>4</a><b>4</b></y>
|
||||||
|
<y><a>5</a><b>5</b></y>
|
||||||
|
<y><a>6</a><b>6</b></y>
|
||||||
|
<y><a>7</a><b>7</b></y>
|
||||||
|
<y><a>8</a><b>8</b></y>
|
||||||
|
<y><a>9</a><b>9</b></y>
|
||||||
|
```
|
||||||
|
|
||||||
|
Requests are either made over the _whole_ dataset, or for one specific element. The following example shows a Restconf GET operation of a single element:
|
||||||
|
```
|
||||||
|
curl -X GET http://localhost/restconf/data/scaling:x/y=3
|
||||||
|
{"scaling:y": [{"a": 3,"b": "3"}]}
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Operations of single elements (transactions) are made in a burst of
|
||||||
|
random elements, typically 100.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
All details of the setup are in the [test script](../../test/plot_perf.sh).
|
||||||
|
|
||||||
|
### Testcases
|
||||||
|
|
||||||
|
All tests measure the "real" time of a command on a lightly loaded
|
||||||
|
machine using the Linux command `time(1)`.
|
||||||
|
|
||||||
|
The following tests were made (for each architecture and protocol):
|
||||||
|
* Write `N` entries in one single operation. (With an empty datastore)
|
||||||
|
* Read `N` entries in one single operation. (With a datastore of `N` entries)
|
||||||
|
* Commit `N` entries (With a candidate of `N` entries and empty running)
|
||||||
|
* Read 1 entry (In a datastore of `N` entries)
|
||||||
|
* Write/Replace 1 entry (In a datastore of `N` entries)
|
||||||
|
* Delete 1 entry (In a datastore of `N` entries)
|
||||||
|
|
||||||
|
The tests are made using Netconf and Restconf, except commit which is made only for Netconf.
|
||||||
|
|
||||||
|
### Architecture and OS
|
||||||
|
|
||||||
|
The tests were made on the following hardware, all running Ubuntu Linux:
|
||||||
|
* i686: dual Intel Core Duo processor (IBM Thinkpad X60)
|
||||||
|
* arm: ARMv7 Processor rev 5 (v7l) (Raspberry PI 2 Model B)
|
||||||
|
* x86-64: Intel Quad-core I5-8259U (Intel NUC Coffee Lake)
|
||||||
|
|
||||||
|
i686: Ubuntu 16.04.6 LTS
|
||||||
|
```
|
||||||
|
Linux version 4.4.0-143-generic (buildd@lgw01-amd64-037) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) ) #169-Ubuntu SMP Thu Feb 7 07:56:51 UTC 2019
|
||||||
|
```
|
||||||
|
|
||||||
|
Arm : Raspbian GNU/Linux 9
|
||||||
|
```
|
||||||
|
|
||||||
|
Linux version 4.14.79-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1159 SMP Sun Nov 4 17:50:20 GMT 2018
|
||||||
|
```
|
||||||
|
|
||||||
|
x86_64: Ubuntu 18.04.1 LTS
|
||||||
|
```
|
||||||
|
inux version 4.15.0-47-generic (buildd@lgw01-amd64-001) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019
|
||||||
|
```
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Discussion
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
[RFC6241](https://tools.ietf.org/html/rfc6241) "Network Configuration Protocol (NETCONF)"
|
||||||
|
[RFC8040](https://tools.ietf.org/html/rfc8040) "RESTCONF Protocol"
|
||||||
|
[i686](https://ark.intel.com/content/www/us/en/ark/products/27235/intel-core-duo-processor-t2400-2m-cache-1-83-ghz-667-mhz-fsb.html)
|
||||||
|
[plot_perf.sh](../test/plot_perf.sh) Test script
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1,6 +1,5 @@
|
||||||
# Clixon examples
|
# Clixon examples
|
||||||
|
|
||||||
Clixon have the following examples:
|
Clixon have the following examples:
|
||||||
* [Main example](main/README.md)
|
|
||||||
* [Hello world](hello/README.md)
|
* [Hello world](hello/README.md)
|
||||||
|
* [Main example](main/README.md)
|
||||||
|
|
|
||||||
|
|
@ -1,39 +1,34 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Transactions per second for large lists read/write plotter using gnuplot
|
# Performance of large lists. See large-lists.md
|
||||||
# What do I want to plot?
|
# The parameters are shown below (under Default values)
|
||||||
# First: on i32, i64, arm32
|
# Examples
|
||||||
# PART 1: Basic load
|
# 1. run all measurements up to 10000 entris collect all results in /tmp/plots
|
||||||
# 1. How long to write 100K entries?
|
# run=true plot=false to=10000 resdir=/tmp/plots ./plot_perf.sh
|
||||||
# - netconf / restconf
|
# 2. Use existing data plot and show on X11
|
||||||
# - list / leaf-list
|
# run=false plot=true resdir=/tmp/plots term=x11 ./plot_perf.sh
|
||||||
# 2. How long to read 100K entries?
|
# 3. Use existing data plot i686 and armv7l data as png
|
||||||
# - netconf/ restconf
|
# archs="i686 armv7l" run=false plot=true resdir=/tmp/plots term=png ./plot_perf.sh
|
||||||
# - list / leaf-list
|
|
||||||
# 3. How long to commit 100K entries? (netconf)
|
|
||||||
# - list / leaf-list
|
|
||||||
#
|
|
||||||
# PART 2: Load 100K entries. Commit.
|
|
||||||
# 4. How many read operations per second?
|
|
||||||
# - netconf/ restconf
|
|
||||||
# - list / leaf-list
|
|
||||||
# 5. How many write operations per second?
|
|
||||||
# - netconf / restconf
|
|
||||||
# - list / leaf-list
|
|
||||||
# 6. How may delete operations per second?
|
|
||||||
# - netconf / restconf
|
|
||||||
# - list / leaf-list
|
|
||||||
# The script uses bash builtin "time" command which is somewhat difficult to
|
|
||||||
# understand. See: https://linux.die.net/man/1/bash # pipelines
|
|
||||||
# You essentially have to do: { time stuff; } 2>&1
|
|
||||||
# See: https://stackoverflow.com/questions/26784870/parsing-the-output-of-bashs-time-builtin
|
|
||||||
|
|
||||||
# Magic line must be first in script (see README.md)
|
# Magic line must be first in script (see README.md)
|
||||||
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi
|
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi
|
||||||
|
|
||||||
# op from step to reqs
|
arch=$(arch)
|
||||||
to=1000
|
# Default values
|
||||||
step=100
|
: ${to:=5000} # Max N
|
||||||
reqs=100
|
: ${step=1000} # Iterate in steps (also starting point)
|
||||||
|
: ${reqs=100} # Number of requests in each burst
|
||||||
|
: ${run:=true} # run tests (or skip them). If false just plot
|
||||||
|
: ${term:=x11} # x11 interactive, alt: png
|
||||||
|
: ${resdir=$dir} # Result dir (both data and gnuplot)
|
||||||
|
: ${plot=false} # Result dir (both data and gnuplot)
|
||||||
|
: ${archs=$arch} # Plotting can be made for many architectures (not run)
|
||||||
|
|
||||||
|
# 0 prefix to protect against shell dynamic binding)
|
||||||
|
to0=$to
|
||||||
|
step0=$step
|
||||||
|
reqs0=$reqs
|
||||||
|
|
||||||
|
ext=$term # gnuplot output file extenstion
|
||||||
|
|
||||||
# Global variables
|
# Global variables
|
||||||
APPNAME=example
|
APPNAME=example
|
||||||
|
|
@ -42,6 +37,13 @@ fyang=$dir/plot.yang
|
||||||
fxml=$dir/data.xml
|
fxml=$dir/data.xml
|
||||||
fjson=$dir/data.json
|
fjson=$dir/data.json
|
||||||
|
|
||||||
|
# Resultdir - if different from $dir that gets erased
|
||||||
|
#resdir=$dir
|
||||||
|
|
||||||
|
if [ ! -d $resdir ]; then
|
||||||
|
mkdir $resdir
|
||||||
|
fi
|
||||||
|
|
||||||
# For memcheck
|
# For memcheck
|
||||||
# clixon_netconf="valgrind --leak-check=full --show-leak-kinds=all clixon_netconf"
|
# clixon_netconf="valgrind --leak-check=full --show-leak-kinds=all clixon_netconf"
|
||||||
# clixon_netconf="valgrind --tool=callgrind clixon_netconf
|
# clixon_netconf="valgrind --tool=callgrind clixon_netconf
|
||||||
|
|
@ -53,19 +55,20 @@ module scaling{
|
||||||
namespace "urn:example:clixon";
|
namespace "urn:example:clixon";
|
||||||
prefix sc;
|
prefix sc;
|
||||||
container x {
|
container x {
|
||||||
list y {
|
description "top-level container";
|
||||||
key "a";
|
list y {
|
||||||
leaf a {
|
description "List with potential large number of elements";
|
||||||
type uint32;
|
key "a";
|
||||||
|
leaf a {
|
||||||
|
description "key in list";
|
||||||
|
type int32;
|
||||||
|
}
|
||||||
|
leaf b {
|
||||||
|
description "payload data";
|
||||||
|
type string;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
leaf b {
|
}
|
||||||
type string;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
leaf-list c {
|
|
||||||
type string;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
|
|
@ -109,48 +112,49 @@ genfile(){
|
||||||
# where proto is one of:
|
# where proto is one of:
|
||||||
# netconf, restconf
|
# netconf, restconf
|
||||||
# where op is one of:
|
# where op is one of:
|
||||||
# writeall readall commitall read write
|
# get put delete commit
|
||||||
runnet(){
|
runnet(){
|
||||||
op=$1
|
op=$1
|
||||||
n=$2 # Number of entries in DB
|
nr=$2 # Number of entries in DB (keep diff from n due to shell dynamic binding)
|
||||||
reqs=$3
|
reqs=$3
|
||||||
|
|
||||||
echo -n "$n " >> $dir/$op-netconf-$reqs
|
file=$resdir/$op-netconf-$reqs-$arch
|
||||||
|
echo -n "$nr " >> $file
|
||||||
case $op in
|
case $op in
|
||||||
write)
|
put)
|
||||||
if [ $reqs = 0 ]; then # Write all in one go
|
if [ $reqs = 0 ]; then # Write all in one go
|
||||||
genfile $n netconf;
|
genfile $nr netconf;
|
||||||
{ time -p cat $fxml | $clixon_netconf -qf $cfg -y $fyang ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
{ time -p cat $fxml | $clixon_netconf -qf $cfg -y $fyang ; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
else # reqs != 0
|
else # reqs != 0
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $n ) ));
|
rnd=$(( ( RANDOM % $nr ) ));
|
||||||
echo "<rpc><edit-config><target><candidate/></target><config><x xmlns=\"urn:example:clixon\"><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>";
|
echo "<rpc><edit-config><target><candidate/></target><config><x xmlns=\"urn:example:clixon\"><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>";
|
||||||
done | $clixon_netconf -qf $cfg -y $fyang ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
done | $clixon_netconf -qf $cfg -y $fyang > /dev/null; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
read)
|
get)
|
||||||
if [ $reqs = 0 ]; then # Read all in one go
|
if [ $reqs = 0 ]; then # Read all in one go
|
||||||
{ time -p echo "<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>" | $clixon_netconf -qf $cfg -y $fyang > /dev/null ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
{ time -p echo "<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>" | $clixon_netconf -qf $cfg -y $fyang > /dev/null ; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
else # reqs != 0
|
else # reqs != 0
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $nr ) ))
|
rnd=$(( ( RANDOM % $nr ) ))
|
||||||
echo "<rpc><edit-config><target><candidate/></target><config><x><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>"
|
echo "<rpc><edit-config><target><candidate/></target><config><x xmlns=\"urn:example:clixon\"><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>"
|
||||||
done | $clixon_netconf -qf $cfg -y $fyang; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
done | $clixon_netconf -qf $cfg -y $fyang > /dev/null; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
delete)
|
delete)
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $nr ) ))
|
rnd=$(( ( RANDOM % $nr ) ))
|
||||||
echo "<rpc><edit-config><target><candidate/></target><config><x><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>"
|
echo "<rpc><edit-config><target><candidate/></target><config><x xmlns=\"urn:example:clixon\"><y><a>$rnd</a><b>$rnd</b></y></x></config></edit-config></rpc>]]>]]>"
|
||||||
done | $clixon_netconf -qf $cfg -y $fyang; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
done | $clixon_netconf -qf $cfg -y $fyang; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
;;
|
;;
|
||||||
commit)
|
commit)
|
||||||
{ time -p echo "<rpc><commit/></rpc>]]>]]>" | $clixon_netconf -qf $cfg -y $fyang > /dev/null ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-netconf-$reqs
|
{ time -p echo "<rpc><commit/></rpc>]]>]]>" | $clixon_netconf -qf $cfg -y $fyang > /dev/null ; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
err "Operation not supported" "$op"
|
err "Operation not supported" "$op"
|
||||||
exit
|
exit
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -159,43 +163,43 @@ done | $clixon_netconf -qf $cfg -y $fyang; } 2>&1 | awk '/real/ {print $2}' >> $
|
||||||
# where proto is one of:
|
# where proto is one of:
|
||||||
# netconf, restconf
|
# netconf, restconf
|
||||||
# where op is one of:
|
# where op is one of:
|
||||||
# writeall readall commitall read write
|
# get put delete
|
||||||
runrest(){
|
runrest(){
|
||||||
op=$1
|
op=$1
|
||||||
n=$2 # Number of entries in DB
|
nr=$2 # Number of entries in DB
|
||||||
reqs=$3
|
reqs=$3
|
||||||
|
|
||||||
echo -n "$n " >> $dir/$op-restconf-$reqs
|
file=$resdir/$op-restconf-$reqs-$arch
|
||||||
|
echo -n "$nr " >> $file
|
||||||
case $op in
|
case $op in
|
||||||
write)
|
put)
|
||||||
if [ $reqs = 0 ]; then # Write all in one go
|
if [ $reqs = 0 ]; then # Write all in one go
|
||||||
genfile $n restconf
|
genfile $nr restconf
|
||||||
# restconf @- means from stdin
|
# restconf @- means from stdin
|
||||||
{ time -p curl -sS -X PUT -d @$fjson http://localhost/restconf/data/scaling:x ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-restconf-$reqs
|
{ time -p curl -sS -X PUT -d @$fjson http://localhost/restconf/data/scaling:x ; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
else # Small requests
|
else # Small requests
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $n ) ));
|
rnd=$(( ( RANDOM % $nr ) ));
|
||||||
curl -sS -X PUT http://localhost/restconf/data/scaling:x/y=$rnd -d "{\"scaling:y\":{\"a\":$rnd,\"b\":\"$rnd\"}}"
|
curl -sS -X PUT http://localhost/restconf/data/scaling:x/y=$rnd -d "{\"scaling:y\":{\"a\":$rnd,\"b\":\"$rnd\"}}"
|
||||||
done ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-restconf-$reqs
|
done ; } 2>&1 | awk '/real/ {print $2}' | tr , .>> $file
|
||||||
#
|
#
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
read)
|
get)
|
||||||
if [ $reqs = 0 ]; then # Read all in one go
|
if [ $reqs = 0 ]; then # Read all in one go
|
||||||
{ time -p curl -sS -X GET http://localhost/restconf/data/scaling:x > /dev/null; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-restconf-$reqs
|
{ time -p curl -sS -X GET http://localhost/restconf/data/scaling:x > /dev/null; } 2>&1 | awk '/real/ {print $2}' | tr , . >> $file
|
||||||
else # Small requests
|
else # Small requests
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $n ) ));
|
rnd=$(( ( RANDOM % $nr ) ));
|
||||||
curl -sS -X GET http://localhost/restconf/data/scaling:x/y=$rnd
|
curl -sS -X GET http://localhost/restconf/data/scaling:x/y=$rnd
|
||||||
done ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-restconf-$reqs
|
done ; } 2>&1 | awk '/real/ {print $2}' | tr , .>> $file
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
delete)
|
delete)
|
||||||
{ time -p for (( i=0; i<$reqs; i++ )); do
|
{ time -p for (( i=0; i<$reqs; i++ )); do
|
||||||
rnd=$(( ( RANDOM % $n ) ));
|
rnd=$(( ( RANDOM % $nr ) ));
|
||||||
curl -sS -X GET http://localhost/restconf/data/scaling:x/y=$rnd
|
curl -sS -X GET http://localhost/restconf/data/scaling:x/y=$rnd
|
||||||
done ; } 2>&1 | awk '/real/ {print $2}' >> $dir/$op-restconf-$reqs
|
done ; } 2>&1 | awk '/real/ {print $2}' | tr , .>> $file
|
||||||
|
|
||||||
;;
|
;;
|
||||||
*)
|
*)
|
||||||
err "Operation not supported" "$op"
|
err "Operation not supported" "$op"
|
||||||
|
|
@ -246,8 +250,8 @@ plot(){
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# reset file
|
# reset file
|
||||||
new "Create file $dir/$op-$proto-$reqs"
|
new "Create file $resdir/$op-$proto-$reqs-$arch"
|
||||||
echo "" > $dir/$op-$proto-$reqs
|
echo -n "" > $resdir/$op-$proto-$reqs-$arch
|
||||||
for (( n=$from; n<=$to; n=$n+$step )); do
|
for (( n=$from; n<=$to; n=$n+$step )); do
|
||||||
reset
|
reset
|
||||||
if [ $can = n ]; then
|
if [ $can = n ]; then
|
||||||
|
|
@ -256,7 +260,7 @@ plot(){
|
||||||
commit
|
commit
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
new "$op-$proto-$reqs $n"
|
new "$op-$proto-$reqs-$arch $n"
|
||||||
if [ $proto = netconf ]; then
|
if [ $proto = netconf ]; then
|
||||||
runnet $op $n $reqs
|
runnet $op $n $reqs
|
||||||
else
|
else
|
||||||
|
|
@ -266,6 +270,7 @@ plot(){
|
||||||
echo # newline
|
echo # newline
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if $run; then
|
||||||
new "test params: -f $cfg -y $fyang"
|
new "test params: -f $cfg -y $fyang"
|
||||||
if [ $BE -ne 0 ]; then
|
if [ $BE -ne 0 ]; then
|
||||||
new "kill old backend"
|
new "kill old backend"
|
||||||
|
|
@ -286,26 +291,35 @@ start_restconf -f $cfg -y $fyang
|
||||||
new "waiting"
|
new "waiting"
|
||||||
sleep $RCWAIT
|
sleep $RCWAIT
|
||||||
|
|
||||||
|
|
||||||
|
to=$to0
|
||||||
|
step=$step0
|
||||||
|
reqs=$reqs0
|
||||||
|
|
||||||
|
# Put all tests
|
||||||
for proto in netconf restconf; do
|
for proto in netconf restconf; do
|
||||||
new "$proto write all entries to candidate (restconf:running)"
|
new "$proto put all entries to candidate (restconf:running)"
|
||||||
plot write $proto $step $step $to 0 0 0 # all candidate 0 running 0
|
plot put $proto $step $step $to 0 0 0 # all candidate 0 running 0
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Get all tests
|
||||||
for proto in netconf restconf; do
|
for proto in netconf restconf; do
|
||||||
new "$proto read all entries from running"
|
new "$proto get all entries from running"
|
||||||
plot read netconf $step $step $to 0 n n # start w full datastore
|
plot get $proto $step $step $to 0 n n # start w full datastore
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Netconf commit all
|
||||||
new "Netconf commit all entries from candidate to running"
|
new "Netconf commit all entries from candidate to running"
|
||||||
plot commit netconf $step $step $to 0 n 0 # candidate full running empty
|
plot commit netconf $step $step $to 0 n 0 # candidate full running empty
|
||||||
|
|
||||||
reqs=100
|
# Transactions get/put/delete
|
||||||
|
reqs=$reqs0
|
||||||
for proto in netconf restconf; do
|
for proto in netconf restconf; do
|
||||||
new "$proto read $reqs from full database"
|
new "$proto get $reqs from full database"
|
||||||
plot read $proto $step $step $to $reqs n n
|
plot get $proto $step $step $to $reqs n n
|
||||||
|
|
||||||
new "$proto Write $reqs to full database(replace / alter values)"
|
new "$proto put $reqs to full database(replace / alter values)"
|
||||||
plot write $proto $step $step $to $reqs n n
|
plot put $proto $step $step $to $reqs n n
|
||||||
|
|
||||||
new "$proto delete $reqs from full database(replace / alter values)"
|
new "$proto delete $reqs from full database(replace / alter values)"
|
||||||
plot delete $proto $step $step $to $reqs n n
|
plot delete $proto $step $step $to $reqs n n
|
||||||
|
|
@ -324,15 +338,127 @@ if [ $BE -ne 0 ]; then
|
||||||
# kill backend
|
# kill backend
|
||||||
stop_backend -f $cfg
|
stop_backend -f $cfg
|
||||||
fi
|
fi
|
||||||
|
fi # if run
|
||||||
|
|
||||||
|
if $plot; then
|
||||||
|
|
||||||
|
# 1. Get config
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/get-restconf-0-$a\" title \"rc-$a\", \"$resdir/get-netconf-0-$a\" title \"nc-$a\","
|
||||||
|
done
|
||||||
|
|
||||||
arch=$(arch)
|
|
||||||
gnuplot -persist <<EOF
|
gnuplot -persist <<EOF
|
||||||
set title "Clixon transactions per second r/w large lists" font ",14" textcolor rgbcolor "royalblue"
|
set title "Clixon get config"
|
||||||
set xlabel "entries"
|
set style data linespoint
|
||||||
set ylabel "transactions per second"
|
set xlabel "Entries"
|
||||||
set terminal x11 enhanced title "Clixon transactions " persist raise
|
set ylabel "Time[s]"
|
||||||
plot "$dir/readlist" with linespoints title "read list", "$dir/writelist" with linespoints title "write list", "$dir/writeleaflist" with linespoints title "write leaf-list" , "$dir/restreadlist" with linespoints title "rest get list"
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-get-0.$term"
|
||||||
|
plot $gplot
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
|
|
||||||
|
# 2. Put config
|
||||||
|
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/put-restconf-0-$a\" title \"rc-$a\", \"$resdir/put-netconf-0-$a\" title \"nc-$a\","
|
||||||
|
done
|
||||||
|
|
||||||
|
gnuplot -persist <<EOF
|
||||||
|
set title "Clixon put config"
|
||||||
|
set style data linespoint
|
||||||
|
set xlabel "Entries"
|
||||||
|
set ylabel "Time[s]"
|
||||||
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-put-0.$term"
|
||||||
|
plot $gplot
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 3. Commit config
|
||||||
|
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/commit-netconf-0-$a\" title \"nc-$a\","
|
||||||
|
done
|
||||||
|
|
||||||
|
gnuplot -persist <<EOF
|
||||||
|
set title "Clixon commit config"
|
||||||
|
set style data linespoint
|
||||||
|
set xlabel "Entries"
|
||||||
|
set ylabel "Time[s]"
|
||||||
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-commit-0.$term"
|
||||||
|
plot $gplot
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 4. Get single entry
|
||||||
|
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/get-restconf-100-$a\" using 1:(\$2/$reqs0) title \"rc-$a\", \"$resdir/get-netconf-100-$a\" using 1:(\$2/$reqs0) title \"nc-$a\","
|
||||||
|
done
|
||||||
|
|
||||||
|
gnuplot -persist <<EOF
|
||||||
|
set title "Clixon get single entry"
|
||||||
|
set style data linespoint
|
||||||
|
set xlabel "Entries"
|
||||||
|
set ylabel "Time[s]"
|
||||||
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-get-100.$term"
|
||||||
|
plot $gplot
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 5. Put single entry
|
||||||
|
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/put-restconf-100-$a\" using 1:(\$2/$reqs0) title \"rc-$a\", \"$resdir/put-netconf-100-$a\" using 1:(\$2/$reqs0) title \"nc-$a\","
|
||||||
|
done
|
||||||
|
|
||||||
|
gnuplot -persist <<EOF
|
||||||
|
set title "Clixon put single entry"
|
||||||
|
set style data linespoint
|
||||||
|
set xlabel "Entries"
|
||||||
|
set ylabel "Time[s]"
|
||||||
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-put-100.$term"
|
||||||
|
plot $gplot
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# 6. Delete single entry
|
||||||
|
|
||||||
|
gplot=""
|
||||||
|
for a in $archs; do
|
||||||
|
gplot="$gplot \"$resdir/delete-restconf-100-$a\" using 1:(\$2/$reqs0) title \"rc-$a\", \"$resdir/delete-netconf-100-$a\" using 1:(\$2/$reqs0) title \"nc-$a\","
|
||||||
|
|
||||||
|
done
|
||||||
|
|
||||||
|
|
||||||
|
gnuplot -persist <<EOF
|
||||||
|
set title "Clixon delete single entry"
|
||||||
|
set style data linespoint
|
||||||
|
set xlabel "Entries"
|
||||||
|
set ylabel "Time[s]"
|
||||||
|
set grid
|
||||||
|
set terminal $term
|
||||||
|
set yrange [*:*]
|
||||||
|
set output "$resdir/clixon-delete-100.$term"
|
||||||
|
plot $gplot
|
||||||
|
EOF
|
||||||
|
|
||||||
|
fi # if plot
|
||||||
|
|
||||||
#rm -rf $dir
|
#rm -rf $dir
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue