Very large commit for upcoming 4.4 release

Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.

test/cicd/Makefile.in

@@ -0,0 +1,74 @@
+#
+# ***** BEGIN LICENSE BLOCK *****
+# 
+# Copyright (C) 2009-2020 Olof Hagsand
+1#
+# This file is part of CLIXON
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Alternatively, the contents of this file may be used under the terms of
+# the GNU General Public License Version 3 or later (the "GPL"),
+# in which case the provisions of the GPL are applicable instead
+# of those above. If you wish to allow use of your version of this file only
+# under the terms of the GPL, and not to allow others to
+# use your version of this file under the terms of Apache License version 2, 
+# indicate your decision by deleting the provisions above and replace them with
+# the notice and other provisions required by the GPL. If you do not delete
+# the provisions above, a recipient may use your version of this file under
+# the terms of any one of the Apache License version 2 or the GPL.
+#
+# ***** END LICENSE BLOCK *****
+#
+
+VPATH       	= @srcdir@
+srcdir  	= @srcdir@
+top_srcdir  	= @top_srcdir@
+
+SHELL	= /bin/sh
+
+.PHONY: all clean distclean depend install uninstall
+
+HOSTS  = vandal.hagsand.com # i86_32 ubuntu
+HOSTS += clixon.dogwood.com # FreeBSD x86_64
+#HOSTS += nuc1.hagsand.com   # x86_64 ubuntu
+#HOSTS += pi2.hagsand.com    # arm Raspian
+
+SCRIPTS  = cligen-mk.sh
+SCRIPTS += clixon-mk.sh
+SCRIPTS += clixon-config.sh
+
+.PHONY: all clean distclean depend install uninstall $(HOSTS)
+
+all: $(HOSTS)
+
+$(HOSTS): 
+	for s in $(SCRIPTS); do \
+	(scp $$s $@:/tmp/ ; ssh $@ chmod 750 /tmp/$$s || exit 1) \
+	done; 
+	./cicd.sh $@ 2>&1 | tee $@.log
+
+clean:
+	rm -f *.log
+
+distclean: clean
+	rm -f Makefile *~ .depend
+
+depend:
+
+install-include:
+
+install:
+
+uninstall:
+

test/cicd/README.md

@@ -0,0 +1,14 @@
+CICD scripts
+============
+Manual scripts for running committed code on a set of hosts.
+
+The script then uses a Makefile and logs in to each host, pulls from
+git, configure, makes and runs through the tests. Make is used to get
+concurrency - non-trivial with bash, eg with `make -j 10`
+
+Note there are other cicd scripts than this, such as the the "travis" scrips.
+
+The Makefile contains a configurable HOSTS variable, please edit.
+
+Logs appear in : <hostname>.log.
+

test/cicd/cicd.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# CI/CD script complementing trevor github
+# Login in to a number of hosts and fo the following:
+# 0. Create and transfer sub-scripts used in main script: cligen-mk.sh clixon-mk.sh clixon-config.sh
+# 1. pull latest version
+# 2. Run configure
+# 3. Compile and install (assume mk.sh)
+# 4. Run tests
+# Assume:
+# - subscripts SCRIPTS exists locally where this script is executed
+# - A test/site.sh file is handmade on each host
+# - some commands are passwordless using
+#    sudo visudo -f /etc/sudoers.d/clixonci
+#       <user> ALL = (root)NOPASSWD : ALL
+#       <user> ALL = (www-data)NOPASSWD : ALL
+#       <user> ALL = (clicon)NOPASSWD : /usr/local/sbin/clixon_backend
+# Experiment in identifying all commands: /usr/bin/make,/usr/local/sbin/clixon_backend,/usr/bin/pkill,/usr/local/bin/clixon_util_socket,/usr/bin/tee,/bin/rm,/usr/bin/touch,/bin/chmod
+#
+# Typical run:  ./cicd.sh 2>&1 | tee cilog
+
+set -eux # x
+
+if [ $# -ne 1 ]; then 
+    echo "usage: $0 <host>"
+    exit -1
+fi
+
+h=$1
+
+ssh -t $h "test -d src/cligen || (cd src;git clone https://github.com/olofhagsand/cligen.git)"
+ssh -t $h "(cd src/cligen;git pull)"
+ssh -t $h "(cd src/cligen;./configure)"
+ssh -t $h "(cd src/cligen; /tmp/cligen-mk.sh)"
+ssh -t $h "test -d src/clixon || (cd src;git clone https://github.com/clicon/clixon.git)"
+ssh -t $h "(cd src/clixon;git pull)"
+ssh -t $h "(cd src/clixon; /tmp/clixon-config.sh)"
+ssh -t $h "(cd src/clixon; /tmp/clixon-mk.sh)"
+ssh -t $h sudo ldconfig
+ssh -t $h "(cd src/clixon/test; ./sum.sh)"

test/cicd/cligen-mk.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+# A top-level maker for cligen
+set -eux
+if [ $(uname) = "FreeBSD" ]; then
+    MAKE=$(which gmake)
+else
+    MAKE=$(which make)
+fi
+$MAKE clean
+$MAKE -j10
+sudo $MAKE install

test/cicd/clixon-config.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+# A top-level configurer for clixon
+set -eux
+if [ $(uname) = "FreeBSD" ]; then
+    ./configure  --with-cligen=/usr/local --with-wwwuser=www --enable-optyangs
+else
+   ./configure --enable-optyangs
+fi

test/cicd/clixon-mk.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+# A top-level maker for clixon
+set -eux
+if [ $(uname) = "FreeBSD" ]; then
+    MAKE=$(which gmake)
+else
+    MAKE=$(which make)
+fi
+$MAKE clean
+$MAKE -j10
+sudo $MAKE install
+sudo $MAKE install-include
+(cd example; $MAKE)
+(cd util; $MAKE)
+(cd example; sudo $MAKE install)
+(cd util; sudo $MAKE install)