From c4a8f98512ab6373af34dda00960f4777eccc848 Mon Sep 17 00:00:00 2001 From: Olof hagsand Date: Wed, 17 Apr 2019 17:41:51 +0200 Subject: [PATCH] scaling doc update --- doc/scaling/large-lists.md | 129 +++++++++++++++++++++++++++++-------- 1 file changed, 101 insertions(+), 28 deletions(-) diff --git a/doc/scaling/large-lists.md b/doc/scaling/large-lists.md index b36f9f2c..333e4ac1 100644 --- a/doc/scaling/large-lists.md +++ b/doc/scaling/large-lists.md @@ -1,19 +1,25 @@ # Large lists in Clixon - * [Background](#background) - * [Overview](#overview) - * [Test descriptions](#test-descriptions) - * [Results](#results) - * [References](#references) +Olof Hagsand, 2019-04-17 -## Background + * [1. Background](#1-background) + * [2. Overview](#2-overview) + * [3. Tests](#3-tests) + * [4. Results](#4-results) + * [5. Discussion](#5-discussion) + * [6. Future work](#6-future-work) + * [7. References](#7-references) + +## 1. Background CIixon can handle large configurations. Here, large number of elements in a "flat" list is presented. There are other scaling usecases, such as large configuratin "depth", large number of requesting clients, etc. -## Overview +Thanks to [Netgate](www.netgate.com) for supporting this work. + +## 2.Overview The basic case is a large list, according to the following Yang specification: ``` @@ -60,7 +66,7 @@ Requests are either made over the _whole_ dataset, or for one specific element. Operations of single elements (transactions) are made in a burst of random elements, typically 100. -## Tests +## 3. Tests All details of the setup are in the [test script](../../test/plot_perf.sh). @@ -82,27 +88,39 @@ The tests are made using Netconf and Restconf, except commit which is made only ### Architecture and OS The tests were made on the following hardware, all running Ubuntu Linux: -* i686: dual Intel Core Duo processor (IBM Thinkpad X60) -* arm: ARMv7 Processor rev 5 (v7l) (Raspberry PI 2 Model B) -* x86-64: Intel Quad-core I5-8259U (Intel NUC Coffee Lake) -i686: Ubuntu 16.04.6 LTS -``` -Linux version 4.4.0-143-generic (buildd@lgw01-amd64-037) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) ) #169-Ubuntu SMP Thu Feb 7 07:56:51 UTC 2019 -``` +#### i686 -Arm : Raspbian GNU/Linux 9 -``` +* IBM Thinkpad X60 +* Dual Intel Core Duo processor +* Ubuntu 16.04.6 LTS +* Linux version 4.4.0-143-generic (buildd@lgw01-amd64-037) + * gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) + * #169-Ubuntu SMP Thu Feb 7 07:56:51 UTC 2019 -Linux version 4.14.79-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1159 SMP Sun Nov 4 17:50:20 GMT 2018 -``` +#### ARM -x86_64: Ubuntu 18.04.1 LTS -``` -inux version 4.15.0-47-generic (buildd@lgw01-amd64-001) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 -``` +* Raspberry PI 2 Model B +* ARMv7 Processor rev 5 (v7l) +* Raspbian GNU/Linux 9 +* Linux version 4.14.79-v7+ (dc4@dc4-XPS13-9333) + * gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) + * #1159 SMP Sun Nov 4 17:50:20 GMT 2018 -## Results +#### x86_64 + +* Intel NUC Coffee Lake +* Intel Quad-core I5-8259U +* Ubuntu 18.04.1 LTS +* Linux version 4.15.0-47-generic (buildd@lgw01-amd64-001) + * gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3) + * #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 + +## 4. Results + +### Access of the whole datastore + +This section shows the results of the measurements as defined in [Tests](#tests). ![Get config](clixon-get-0.png "Get config") @@ -110,22 +128,77 @@ inux version 4.15.0-47-generic (buildd@lgw01-amd64-001) (gcc version 7.3.0 (Ubun ![Commit config](clixon-commit-0.png "Commit config") +### Access of single entries + ![Get single entry](clixon-get-100.png "Get single entry") ![Put single entry](clixon-put-100.png "Put single entry") ![Delete single entry](clixon-delete-100.png "Delete single entry") -## Discussion +### Profiling +An example profiling of the most demanding case was made: Put single restconf for with 5000 existing entries. +The tool used is valgrind/callgrind with the following approximate result: +* from_rpc_callback 100% + * from_client_commit 65% + * candidate_commit 65% + * from_validate_common 30% + * xml_diff 13% + * xml_yang_validate_all 10% + * xmldb_copy 29% + * xml_copy 22% + * from_client_edit_config 35% + * xmldb_put 35% + * clicon_xml2file 30% + * fprintf 13% + * xml_chardata_encode 12% +It can be seen that most cycles are spend in file copying and writing +to file which explains the linear behaviour. +## 5. Discussion -## References +All measurements show clear performance differences between the +architectures, which was expected. + +By looking at top and other tools, it seems clear that the main +bootleneck is CPU for the `clixon_backend`. The clients, eg `nginx`, +clixon_restconf` and `clixon_netconf` seem negligable. Memory +footprint is also not limiting. + +Accessing the whole configuration is similar between protocols and +linear in time. This is to be expected since the tranfer is dependent +on the size of the database. + +Accessing a single entry is also similar in all cases and shows large +differences. As expected, the CPU architecture is significant, but +there is also a large difference between Netconf and Restconf. + +The Netconf and restconf setups differ somewhat which may explain +differences in performance. Primarily, the Netconf input is piped to a +single Netconf client while a curl is re-started for each Restconf +call. Also, the Restconf PUT includes a commit. + +Further, the single entry access is linear wrt number of entries. This +means it is possible to run large scale applications on high +performance CPUs, such as 100K entries on a x86_64 in the results, but +it also means that very large lists may not be supported, and that the +system degrades with the size of the lists. + +Examining the profiling of the most demanding Restconf PUT case + +Concluding, the + +## 6. Future work + +It is recommended that the following items should be studied: +* Try to improve the access of single list elements, to sub-linear perforamnce. +* CLI access on large lists (not included in this study) + +## 7. References [RFC6241](https://tools.ietf.org/html/rfc6241) "Network Configuration Protocol (NETCONF)" [RFC8040](https://tools.ietf.org/html/rfc8040) "RESTCONF Protocol" [i686](https://ark.intel.com/content/www/us/en/ark/products/27235/intel-core-duo-processor-t2400-2m-cache-1-83-ghz-667-mhz-fsb.html) [plot_perf.sh](../test/plot_perf.sh) Test script - -