Merge branch 'develop'

CHANGELOG.md

@@ -1,9 +1,126 @@
 # Clixon Changelog
 
+## 3.6.0 (Upcoming)
+
+### Major changes:
+* Experimental NACM RFC8341 Network Configuration Access Control Model, see [NACM](README_NACM.md).
+  * New CLICON_NACM_MODE config option, default is disabled.
+  * New CLICON_NACM_FILE config option, if CLICON_NACM_MODE is "external"
+  * Added username attribute to all internal RPC:s from frontend to backend
+  * Added NACM backend module in example
+* Restructure and more generic plugin API for cli, backend, restconf, and netconf. See example further down and the [example](example/README.md)
+  * Changed `plugin_init()` to `clixon_plugin_init()` returning an api struct with function pointers. There are no other hardcoded plugin functions.
+  * Master plugins have been removed. Plugins are loaded alphabetically. You can ensure plugin load order by prefixing them with an ordering number, for example.
+  * Plugin RPC callback interface have been unified between backend, netconf and restconf.
+    * Backend RPC register callback function (Netconf RPC or restconf operation POST) has been changed from:
+           `backend_rpc_cb_register()` to `rpc_callback_register()`
+    * Backend RPC callback signature has been changed from:
+           `int cb(clicon_handle h, cxobj *xe, struct client_entry *ce, cbuf *cbret, void *arg)`
+       to:
+            `int cb(clicon_handle h, cxobj *xe, struct client_entry *ce, cbuf *cbret, void *arg)`
+    * Frontend netconf and restconf plugins can register callbacks as well with same API as backends.
+  * Moved specific plugin functions from apps/ to generic functions in lib/
+  * New config option CLICON_BACKEND_REGEXP to match backend plugins (if you do not want to load all).
+  * Added authentication plugin callback (ca_auth)
+    * Added clicon_username_get() / clicon_username_set()
+  * Removed some obscure plugin code that seem not to be used (please report if needed!)
+    * CLI parse hook
+    * CLICON_FIND_PLUGIN
+    * clicon_valcb()
+    * CLIXON_BACKEND_SYSDIR
+    * CLIXON_CLI_SYSDIR	
+    * CLICON_MASTER_PLUGIN config variable
+  * Example of migrating a backend plugin module:
+    * Add all callbacks in a clixon_plugin_api struct
+    * Rename plugin_init() -> clixon_plugin_init() and return api as function value
+    * Rename backend_rpc_cb_register() -> rpc_callback_register() for any RPC/restconf operation POST calls
+```
+/* This is old style with hardcoded function names (eg plugin_start) */
+int plugin_start(clicon_handle h, int argc, char **argv)
+{
+    return 0;
+}
+int
+plugin_init(clicon_handle h)
+{
+   return 0;
+}
+
+/* This is new style with all function names in api struct */
+clixon_plugin_api *clixon_plugin_init(clicon_handle h);
+
+static clixon_plugin_api api = {
+    "example",           /* name */
+    clixon_plugin_init,  /* init */
+    NULL,                /* start */
+    NULL,                /* exit */
+    .ca_auth=plugin_credentials   /* restconf specific: auth */
+};
+
+clixon_plugin_api *clixon_plugin_init(clicon_handle h)
+{
+    return &api; /* Return NULL on error */
+}
+```
+
+* Builds and installs a new restconf library: `libclixon_restconf.so` and clixon_restconf.h
+  * The restconf library can be included by a restconf plugin.
+  * Example code in example/Makefile.in and example/restconf_lib.c
+* Restconf error handling for get, put and post. (thanks Stephen Jones, Netgate)
+  * Available both as xml and json (set accept header).
+* Proper RFC 6241 Netconf error handling
+  * New functions added in clixon_netconf_lib.[ch]
+  * Datastore code modified for RFC 6241
+
+### Minor changes:
+
+* INSTALLFLAGS added with default value -s(strip).
+  * For debug do: CFLAGS=-g INSTALLFLAGS= ./configure
+* plugin_start() callbacks added for restconf
+* Authentication
+  * Example extended with http basic authentication for restconf
+  * Documentation in FAQ.md
+* Updated ietf-netconf-acm to ietf-netconf-acm@2018-02-14.yang from RFC 8341
+* The Clixon example has changed name from "routing" to "example" affecting all config files, plugins, tests, etc.
+  * Secondary backend plugin added
+* Removed username to rpc calls (added below)
+* README.md extended with new yang, netconf, restconf, datastore, and auth sections.
+* The key-value datastore is no longer supported. Use the default text datastore.
+* Added username to rpc calls to prepare for authorization for backend:
+  * clicon_rpc_config_get(h, db, xpath, xt) --> clicon_rpc_config_get(h, db, xpath, username, xt)
+  * clicon_rpc_get(h, xpath, xt) --> clicon_rpc_get(h, xpath, username, xt)
+* Experimental: Added CLICON_TRANSACTION_MOD configuration option. If set,
+  modifications in validation and commit callbacks are written back
+  into the datastore. Requested by Stephen Jones, Netgate.
+* Invalid key to api_path2xml gives warning instead of error and quit.	
+* Added restconf/operations get, see RFC8040 Sec 3.3.2:
+* yang_find_topnode() and api_path2xml() schemanode parameter replaced with yang_class. Replace as follows: 0 -> YC_DATANODE, 1 -> YC_SCHEMANODE
+
+* xml2json: include prefix in translation, so <a:b> is translated to {"a:b" ..}
+* Use `<config>` instead of `<data>` when save/load configuration to file. This
+enables saved files to be used as datastore without any editing. Thanks Matt, Netgate.
+
+* Added Yang "extension" statement. This includes parsing unknown
+  statements and identifying them as extensions or not. However,
+  semantics for specific extensions must still be added.
+
+* Renamed ytype_id and ytype_prefix to yarg_id and yarg_prefix, respectively
+
+* Added cli_show_version()
+
+### Corrected Bugs
+* Showing syntax using CLI commands was broekn and is fixed.
+* Fixed issue https://github.com/clicon/clixon/issues/18 RPC response issues reported by Stephen Jones at Netgate
+* Fixed issue https://github.com/clicon/clixon/issues/17 special character in strings can break RPCs reported by David Cornejo at Netgate.
+  * This was a large rewrite of XML parsing and output due to CharData not correctly encoded according to https://www.w3.org/TR/2008/REC-xml-20081126. 
+* Fixed three-key list entry problem (reported by jdl@netgate)
+* Translate xml->json \n correctly
+* Fix issue: https://github.com/clicon/clixon/issues/15 Replace whole config
+
 ## 3.5.0 (12 February 2018)
 
 ### Major changes:
-* Major Restconf feature update to compy to RFC 8040. Thanks Stephen Jones for getting right.
+* Major Restconf feature update to comply to RFC 8040. Thanks Stephen Jones of Netgate for getting right.
   * GET: Always return object referenced (and nothing else). ie, GET /restconf/data/X returns X. 
   * GET Added support for the following resources: Well-known, top-level resource, and yang library version,
   * GET Single element JSON lists use {list:[element]}, not {list:element}.
@@ -42,7 +159,7 @@
   * New CLICON_XML_SORT configuration option. Default is true. Disable by setting to false.
   * Added yang ordered-by user. The default (ordered-by system) will now sort lists and leaf-lists alphabetically to increase search performance. Note that this may change outputs.
   * If you need legacy order, either set CLICON_XML_SORT to false, or set that list to "ordered-by user".
-  * This replaces XML hash experimental code, ie xml_child_hash variables and all xml_hash_ functions have been removed.
+  * This replaces XML hash experimental code, ie xml_child_hash variables and all xmlv_hash_ functions have been removed.
   * Implementation detail: Cached keys are stored in in yang Y_LIST nodes as cligen vector, see ys_populate_list()  
 
 * Datastore cache introduced: cache XML tree in memory for faster get access.
@@ -102,11 +219,11 @@ SUNET for support, requests, debugging, bugfixes and proposed solutions.
   * In backward compatible mode both .xml and .conf works
   * For migration from old to new, a utility is clixon_cli -x to print new format. Run the command and save in configuration file with .xml suffix instead.
   ```
-    > clixon_cli -f /usr/local/etc/routing.conf -1x
+    > clixon_cli -f /usr/local/etc/example.conf -1x
     <config>
-        <CLICON_CONFIGFILE>/usr/local/etc/routing.xml</CLICON_CONFIGFILE>
+        <CLICON_CONFIGFILE>/usr/local/etc/example.xml</CLICON_CONFIGFILE>
-        <CLICON_YANG_DIR>/usr/local/share/routing/yang</CLICON_YANG_DIR>
+        <CLICON_YANG_DIR>/usr/local/share/example/yang</CLICON_YANG_DIR>
-        <CLICON_BACKEND_DIR>/usr/local/lib/routing/backend</CLICON_BACKEND_DIR>
+        <CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
 	...
    </config>
   ```

Makefile.in

@@ -47,7 +47,7 @@ CC		= @CC@
 CFLAGS  	= @CFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 LIBS    	= @LIBS@
-#INSTALL		= @INSTALL@
+INSTALL		= @INSTALL@
 
 INCLUDES 	= -I. -I@srcdir@ @INCLUDES@
 SHELL		= /bin/sh
@@ -69,8 +69,8 @@ clixon.mk:  clixon.mk.cpp
 	$(CPP) -P -traditional-cpp -x assembler-with-cpp -Dprefix=$(prefix) -Dlocalstatedir=$(localstatedir) -Dsysconfdir=$(sysconfdir) -Ddatadir=$(datadir) -Dlibdir=$(libdir) $< > $@
 
 install:  clixon.mk
-	install -d -m 755 $(DESTDIR)$(datadir)/clixon
+	install -d -m 0755 $(DESTDIR)$(datadir)/clixon
-	install -m 755 clixon.mk $(DESTDIR)$(datadir)/clixon
+	install -m 0644 clixon.mk $(DESTDIR)$(datadir)/clixon
 	for i in $(SUBDIRS) doc; \
 	do (cd $$i; $(MAKE) $(MFLAGS) $@)||exit 1; done; \
 	echo "Install for compilation by: make install-include"

README.md

@@ -4,26 +4,39 @@ Clixon is an automatic configuration manager where you generate
 interactive CLI, NETCONF, RESTCONF and embedded databases with
 transaction support from a YANG specification.
 
-Table of contents
-=================
-  * [Documentation](#documentation)
-  * [Installation](#installation)
-  * [Dependencies](#dependencies)
-  * [Licenses](#licenses)
   * [Background](#background)
-  * [Clixon SDK](#SDK)
+  * [Frequently asked questions](doc/FAQ.md)
+  * [Installation](#installation)
+  * [Licenses](#licenses)
+  * [Support](#support)
+  * [Dependencies](#dependencies)
+  * [Extending](#extending)
+  * [Yang](#yang)
+  * [Netconf](#netconf)
+  * [Restconf](#restconf)
+  * [Datastore](datastore/README.md)
+  * [Authentication and Authorization](#auth)
+  * [Example](example/)
+  * [Changelog](CHANGELOG.md)
+  * [Runtime](#runtime)
+  * [Clixon project page](http://www.clicon.org)
+  * [Tests](test/)
+  * [Reference manual](http://www.clicon.org/doxygen/index.html) (Note: the link may not be up-to-date. It is better to build your own: `cd doc; make doc`)
   
-Documentation
+Background
-=============
+==========
-- [Frequently asked questions](doc/FAQ.md)
+
-- [CHANGELOG](CHANGELOG.md) recent changes.
+Clixon was implemented to provide an open-source generic configuration
-- [XML datastore](datastore/README.md)
+tool. The existing [CLIgen](http://www.cligen.se) tool was for command-lines only, whilke clixon is a system with config-db, xml and rest interfaces. Most of the projects using clixon are for embedded network and measuring devices. But Clixon is more generic than that.
-- [Netconf support](apps/netconf/README.md)
+
-- [Restconf support](apps/restconf/README.md)
+Users of clixon currently include:
-- [Reference manual](http://www.clicon.org/doxygen/index.html) (Note: the link may not be up-to-date. It is better to build your own: `cd doc; make doc`)
+  * [Netgate](https://www.netgate.com)
-- [Routing example](example/README.md)
+  * [CloudMon360](http://cloudmon360.com)
-- [Clicon and Clixon project page](http://www.clicon.org)
+  * [Grideye](http://hagsand.se/grideye)	
-- [Tests](test/README.md)
+  * [Netclean](https://www.netclean.com/solutions/whitebox) # only CLIgen
+  * [Prosilient's PTAnalyzer](https://prosilient.com) # only CLIgen
+
+See also [Clicon project page](http://clicon.org).
 
 Installation
 ============
@@ -36,13 +49,19 @@ A typical installation is as follows:
 ```
 
 One [example application](example/README.md) is provided, a IETF IP YANG datamodel with
-generated CLI and configuration interface.
+generated CLI, Netconf and restconf interface.
+
+Licenses
+========
+Clixon is open-source and dual licensed. Either Apache License, Version 2.0 or GNU
+General Public License Version 2; you choose.
+
+See [LICENSE.md](LICENSE.md) for the license.
 
 Dependencies
 ============
 Clixon depends on the following software packages, which need to exist on the target machine.
-- [CLIgen](http://www.cligen.se) is required for building Clixon. If you need 
+- [CLIgen](http://www.cligen.se) If you need to build and install CLIgen: 
-to build and install CLIgen: 
 ```
     git clone https://github.com/olofhagsand/cligen.git
     cd cligen; configure; make; make install
@@ -50,53 +69,115 @@ to build and install CLIgen:
 - Yacc/bison
 - Lex/Flex
 - Fcgi (if restconf is enabled)
-- [Qdbm](http://fallabs.com/qdbm/) key-value store (if keyvalue datastore is enabled)
 
 There is no yum/apt/ostree package for Clixon (please help?)
 
-Licenses
+Support
+=======
+Clixon interaction is best done posting issues, pull requests, or joining the
+[slack channel](https://clixondev.slack.com).
+[Slack invite](https://join.slack.com/t/clixondev/shared_invite/enQtMzI3OTM4MzA3Nzk3LTA3NWM4OWYwYWMxZDhiYTNhNjRkNjQ1NWI1Zjk5M2JjMDk4MTUzMTljYTZiYmNhODkwMDI2ZTkyNWU3ZWMyN2U). 
+
+Extending
+=========
+Clixon provides a core system and can be used as-is using available
+Yang specifications.  However, an application very quickly needs to
+specialize functions.  Clixon is extended by writing
+plugins for cli and backend. Extensions for netconf and restconf
+are also available.
+
+Plugins are written in C and easiest is to look at
+[example](example/README.md) or consulting the [FAQ](doc/FAQ.md).
+
+Yang
+====
+
+YANG and XML is at the heart of Clixon.  Yang modules are used as a
+specification for handling XML configuration data. The YANG spec is
+used to generate an interactive CLI, netconf and restconf clients. It
+also manages an XML datastore.
+
+Clixon mainly follows [YANG 1.0 RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) with some exceptions:
+- conformance: feature, if-feature, deviation
+- identity, base, identityref
+- list features: min/max-elements, unique
+
+The aim is also to cover new features in YANG 1.1 [YANG RFC 7950](https://www.rfc-editor.org/rfc/rfc7950.txt)
+
+Clixon has its own XML library designed for performance.
+
+Netconf
+=======
+Clixon implements the following NETCONF proposals or standards:
+- [NETCONF Configuration Protocol](http://www.rfc-base.org/txt/rfc-4741.txt)
+- [Using the NETCONF Configuration Protocol over Secure Shell (SSH)](http://www.rfc-base.org/txt/rfc-4742.txt)
+- [NETCONF Event Notifications](http://www.rfc-base.org/txt/rfc-5277.txt)
+
+Some updates are being made to RFC 6241 and RFC 6242. 
+
+Clixon does not yet support the following netconf features:
+
+- :url capability
+- copy-config source config
+- edit-config testopts 
+- edit-config erropts
+- edit-config config-text
+
+Restconf
 ========
-Clixon is dual licensed. Either Apache License, Version 2.0 or GNU
+Clixon restconf is a daemon based on FASTCGI. Instructions are available to
-General Public License Version 2; you choose.
+run with NGINX.
+The implementatation is based on [RFC 8040: RESTCONF Protocol](https://tools.ietf.org/html/rfc8040).
+The following features are supported:
+- OPTIONS, HEAD, GET, POST, PUT, DELETE
+The following are not implemented
+- PATCH
+- query parameters (section 4.9)
+- notifications (sec 6)
+- schema resource
 
-See [LICENSE.md](LICENSE.md) for the license.
+See [more detailed instructions](apps/restconf/README.md).
 
-Background
+Datastore
-==========
+=========
-We implemented Clixon since we needed a generic configuration tool in
+The Clixon datastore is a stand-alone XML based datastore. The idea is
-several projects, including
+to be able to use different datastores backends with the same
-[KTH](http://www.csc.kth.se/~olofh/10G_OSR). Most of these projects
+API.
-were for embedded network and measuring-probe devices. We started with
-something called Clicon which was based on a key-value specification
-and data-store. But as time passed new standards evolved and we
-started adapting it to XML, Yang and netconf. Finally we made Clixon,
-where the legacy key specification has been replaced completely by
-YANG and using XML as configuration data. This means that legacy
-Clicon applications do not run on Clixon.
 
-SDK
+Update: There used to be a key-value plugin based on qdbm but isnow obsoleted. Only a text datastore is implemented.
-===
 
-<img src="doc/clixon_example_sdk.png" alt="clixon sdk" style="width: 200px;"/>
+The datastore is primarily designed to be used by Clixon but can be used
+separately.
+
+See [more detailed instructions](datastore/README.md).
+
+Auth
+====
+
+Authentication is managed outside Clixon using SSH, SSL, Oauth2, etc.
+
+For CLI, login is typically made via SSH. For netconf, SSH netconf
+subsystem can be used. 
+  
+Restconf however needs credentials.  This is done by writing a credentials callback in a restconf plugin. See:
+  * [FAQ](doc/FAQ.md#how-do-i-write-an-authentication-callback).
+  * [Example](example/README.md) has an example how to do this with HTTP basic auth.
+  * I have done this for another project using Oauth2 or (https://github.com/CESNET/Netopeer2/tree/master/server/configuration)
+
+The clients send the ID of the user using a "username" attribute with
+the RPC calls to the backend. Note that the backend trusts the clients
+so the clients can in principle fake a username.
+
+There is an ongoing effort to implement authorization for Clixon
+according to [RFC8341(NACM)](https://tools.ietf.org/html/rfc8341), at
+least a subset of the functionality. See more information here:
+[NACM](README_NACM.md).
+
+
+Runtime
+=======
+
+<img src="doc/clixon_example_sdk.png" alt="clixon sdk" style="width: 180px;"/>
 
 The figure shows the SDK runtime of Clixon.
 
-YANG and XML is at the heart of Clixon.  Yang modules are used as a
-specification for handling XML configuration data. The spec is also
-used to generate an interactive CLI client as well as provide
-[Netconf](apps/netconf/README.md) and
-[Restconf](apps/restconf/README.md) clients.
-
-The [YANG RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) is implemented with the following exceptions:
-- conformance: feature, if-feature, deviation
-- identity, base, identityref
-- list features: min/max-elements, unique, ordered-by
-
-There are also new features in YANG 1.1 [YANG RFC
-7950](https://www.rfc-editor.org/rfc/rfc7950.txt), most of which are
-not implemented.
-
-
-
-
-

README_DEVELOP.md

@@ -1,11 +1,12 @@
-# README for developers Clixon developers
+# README for Clixon developers
 
-1. How to document the code
+  * [Code documentation](#documentation)
-2. How to work in git (branching)
+  * [How to work in git (branching)](#branching)
-3. How the meta-configure stuff works
+  * [How the meta-configure stuff works](#meta-configure)
-4. How to debug
+  * [How to debug](#debug)
 
-## How to document the code
+## Documentation
+How to document the code
 
 ```
 /*! This is a small comment on one line
@@ -26,7 +27,8 @@
  */
 ```
 
-## How to work in git (branching)
+## Branching
+How to work in git (branching)
 
 Basically follows: http://nvie.com/posts/a-successful-git-branching-model/
 only somewhat simplified:
@@ -49,11 +51,16 @@ configure.ac --.
      Makefile.in ---'                    `-> Makefile ---'
 ```
 
-## How to debug
+## Debug
+How to debug
+
+### Configure in debug mode
+```
+   CFLAGS="-g -Wall" INSTALLFLAGS="" ./configure
+```
 
 ### Make your own simplified yang and configuration file.
 ```
-
 cat <<EOF > /tmp/my.yang
 module mymodule{
    container x {
@@ -69,11 +76,11 @@ EOF
 cat <<EOF > /tmp/myconf.xml
 <config>
   <CLICON_CONFIGFILE>/tmp/myconf.xml</CLICON_CONFIGFILE>
-  <CLICON_YANG_DIR>/usr/local/share/routing/yang</CLICON_YANG_DIR>
+  <CLICON_YANG_DIR>/usr/local/share/example/yang</CLICON_YANG_DIR>
   <CLICON_YANG_MODULE_MAIN>example</CLICON_YANG_MODULE_MAIN>
-  <CLICON_SOCK>/usr/local/var/routing/routing.sock</CLICON_SOCK>
+  <CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
-  <CLICON_BACKEND_PIDFILE>/usr/local/var/routing/routing.pidfile</CLICON_BACKEND_PIDFILE>
+  <CLICON_BACKEND_PIDFILE>/usr/local/var/example/example.pidfile</CLICON_BACKEND_PIDFILE>
-  <CLICON_XMLDB_DIR>/usr/local/var/routing</CLICON_XMLDB_DIR>
+  <CLICON_XMLDB_DIR>/usr/local/var/example</CLICON_XMLDB_DIR>
   <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
 </config>
 EOF

README_NACM.md

@@ -0,0 +1,34 @@
+# NACM
+
+Clixon includes an experimental NACM implementation according to [RFC8341(NACM)](https://tools.ietf.org/html/rfc8341).
+
+The support is as follows:
+
+* There is a yang config variable `CLICON_NACM_MODE` to set whether NACM is disabled, uses internal(embedded) NACM configuration, or external configuration. (See yang/clixon-config.yang)
+* If the mode is internal, NACM configurations is expected to be in the regular configuration, managed by regular candidate/runing/commit procedures. This mode may have some problems with bootstrapping.
+* If the mode is `external`, the `CLICON_NACM_FILE` yang config variable contains the name of a separate configuration file containing the NACM configurations. After changes in this file, the backend needs to be restarted.
+* The [example](example/README.md) contains a http basic auth and a NACM backend callback for mandatory state variables.
+* There are two [tests](test/README.md) using internal and external NACM config
+* The backend provides a limited NACM support (when enabled) described below
+
+NACM functionality
+==================
+
+NACM is implemented in the backend and a single access check is made
+in from_client_msg() when an internal netconf RPC has
+just been received and decoded. The code is in nacm_access().
+
+The functionality is as follows:
+* Notification is not supported
+* Groups are supported
+* Rule-lists are supported
+* Rules are supported as follows
+  * module-name: Only '*' supported
+  * access-operations: only '*' and 'exec' supported
+  * rpc-name: fully supported (eg edit-config/get-config, etc)
+  * action: fully supported (permit/deny)
+
+The tests outlines an example of three groups (taken from the RFC): admin, limited and guest:
+* admin: Full access
+* limited: Read access (get and get-config)
+* guest: No access

apps/backend/Makefile.in

@@ -36,6 +36,7 @@ srcdir  	= @srcdir@
 top_srcdir  	= @top_srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 
 prefix 		= @prefix@
@@ -55,8 +56,6 @@ CLIXON_MINOR    = @CLIXON_VERSION_MINOR@
 
 # Use this clixon lib for linking
 CLIXON_LIB	= libclixon.so.$(CLIXON_MAJOR).$(CLIXON_MINOR)
-# Location of system plugins
-CLIXON_BACKEND_SYSDIR = $(libdir)/clixon/plugins/backend
 
 # For dependency. A little strange that we rely on it being built in the src dir
 # even though it may exist in $(libdir). But the new version may not have been installed yet.
@@ -66,23 +65,27 @@ LIBS    	= -L$(top_srcdir)/lib/src @LIBS@ $(top_srcdir)/lib/src/$(CLIXON_LIB) -l
 CPPFLAGS  	= @CPPFLAGS@ -fPIC
 INCLUDES	= -I. -I$(top_srcdir)/lib/src -I$(top_srcdir)/lib -I$(top_srcdir)/include -I$(top_srcdir) @INCLUDES@
 
-# Not accessible from plugin
+# Name of application
-APPSRC  = backend_main.c backend_socket.c backend_client.c \
-	backend_commit.c backend_plugin.c 
-
-APPOBJ  = $(APPSRC:.c=.o)
 APPL	= clixon_backend
 
-#SHLIB	= clixon_backend
+# Not accessible from plugin
-MYNAME		= clixon_backend
+APPSRC  = backend_main.c
-MYLIBLINK       = lib$(MYNAME)$(SH_SUFFIX)
+APPSRC += backend_socket.c
-MYLIB           = $(MYLIBLINK).$(CLIXON_MAJOR).$(CLIXON_MINOR)
+APPSRC += backend_client.c
-MYLIBSO         = $(MYLIBLINK).$(CLIXON_MAJOR)
+APPSRC += backend_commit.c
+APPSRC += backend_plugin.c
+APPOBJ  = $(APPSRC:.c=.o)
 
 # Accessible from plugin
 LIBSRC	= clixon_backend_transaction.c clixon_backend_handle.c
 LIBOBJ	= $(LIBSRC:.c=.o)
 
+# Name of lib
+MYNAME		= clixon_backend
+MYLIBLINK       = lib$(MYNAME)$(SH_SUFFIX)
+MYLIB           = $(MYLIBLINK).$(CLIXON_MAJOR).$(CLIXON_MINOR)
+MYLIBSO         = $(MYLIBLINK).$(CLIXON_MAJOR)
+
 all:	  $(MYLIB) $(APPL) test
 
 clean:
@@ -96,15 +99,15 @@ distclean: clean
 # Also create a libexec/ directory for writeable/temporary files.
 # Put config file in etc/
 install:	install-lib	$(APPL)
-	install -d $(DESTDIR)$(sbindir)
+	install -d -m 0755 $(DESTDIR)$(sbindir)
-	install $(APPL) $(DESTDIR)$(sbindir)
+	install -m 0755 $(INSTALLFLAGS) $(APPL) $(DESTDIR)$(sbindir)
 
 install-lib:	$(MYLIB)
-	install -d $(DESTDIR)$(libdir)
+	install -d -m 0755 $(DESTDIR)$(libdir)
-	install $(MYLIB) $(DESTDIR)$(libdir)
+	install -m 0644 $(INSTALLFLAGS) $(MYLIB) $(DESTDIR)$(libdir)
 	ln -sf $(MYLIB) $(DESTDIR)$(libdir)/$(MYLIBSO)	   # -l:libclixon_config.so.2
 	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon_config.so
-	install -d $(DESTDIR)$(libdir)/clixon/plugins/backend
+	install -d -m 0755 $(DESTDIR)$(libdir)/clixon/plugins/backend
 
 uninstall:
 	rm -f $(DESTDIR)$(sbindir)/$(APPL)
@@ -112,14 +115,14 @@ uninstall:
 	rm -f $(DESTDIR)$(includedir)/clixon/*
 
 install-include: clixon_backend.h clixon_backend_handle.h clixon_backend_transaction.h
-	install -d $(DESTDIR)$(includedir)/clixon
+	install -d -m 0755 $(DESTDIR)$(includedir)/clixon
-	install -m 644 $^ $(DESTDIR)$(includedir)/clixon
+	install -m 0644 $^ $(DESTDIR)$(includedir)/clixon
 
 .SUFFIXES:
 .SUFFIXES: .c .o
 
 .c.o:
-	$(CC) $(INCLUDES) -D__PROGRAM__=\"$(APPL)\" -DCLIXON_BACKEND_SYSDIR=\"$(CLIXON_BACKEND_SYSDIR)\" $(CPPFLAGS) $(CFLAGS) -c $<
+	$(CC) $(INCLUDES) -D__PROGRAM__=\"$(APPL)\" $(CPPFLAGS) $(CFLAGS) -c $<
 
 # Just link test programs
 test.c	:
@@ -129,7 +132,7 @@ test: 	test.c $(LIBOBJ)
 	$(CC) $(INCLUDES) $(LDFLAGS) $< $(LIBOBJ) -L. $(MYLIB) $(LIBS) -o $@
 
 $(APPL) : $(APPOBJ) $(MYLIBLINK) $(LIBDEPS)
-	$(CC) $(LDFLAGS) $(APPOBJ) $(OBJS) -L. $(MYLIB) $(LIBS) -o $@
+	$(CC) $(LDFLAGS) $(APPOBJ) -L. $(MYLIB) $(LIBS) -o $@
 
 $(MYLIB): $(LIBOBJ)
 ifeq ($(HOST_VENDOR),apple)
@@ -147,7 +150,7 @@ TAGS:
 	find . -name '*.[chyl]' -print | etags -
 
 depend:
-	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(SRC) $(APPSRC) > .depend
+	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(LIBSRC) $(APPSRC) > .depend
 
 #include .depend
 

apps/backend/backend_commit.c

@@ -239,13 +239,20 @@ candidate_commit(clicon_handle h,
      if (plugin_transaction_commit(h, td) < 0)
 	 goto done;
 
-     /* 8. Success: Copy candidate to running */
+     /* Optionally write (potentially modified) tree back to candidate */
+     if (clicon_option_bool(h, "CLICON_TRANSACTION_MOD"))
+	 if (xmldb_put(h, candidate, OP_REPLACE, td->td_target, NULL) < 0)
+	     goto done;
+     /* 8. Success: Copy candidate to running 
+      */
+
      if (xmldb_copy(h, candidate, "running") < 0)
 	 goto done;
 
     /* 9. Call plugin transaction end callbacks */
     plugin_transaction_end(h, td);
 
+
     /* 8. Copy running back to candidate in case end functions updated running */
     if (xmldb_copy(h, "running", candidate) < 0){
 	/* ignore errors or signal major setback ? */
@@ -275,35 +282,32 @@ from_client_commit(clicon_handle h,
 {
     int        retval = -1;
     int        piddb;
+    cbuf      *cbx = NULL; /* Assist cbuf */
 
     /* Check if target locked by other client */
     piddb = xmldb_islocked(h, "running");
     if (piddb && mypid != piddb){
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	if ((cbx = cbuf_new()) == NULL){
-		"<error-tag>lock-denied</error-tag>"
+	    clicon_err(OE_XML, errno, "cbuf_new");
-		"<error-type>protocol</error-type>"
+	    goto done;
-		"<error-severity>error</error-severity>"
+	}	
-		"<error-message>Operation failed, lock is already held</error-message>"
+	cprintf(cbx, "<session-id>%d</session-id>", piddb);
-		"<error-info><session-id>%d</session-id></error-info>"
+	if (netconf_lock_denied(cbret, cbuf_get(cbx), "Operation failed, lock is already held") < 0)
-		"</rpc-error></rpc-reply>",
+	    goto done;
-		piddb);
 	goto ok;
     }
     if (candidate_commit(h, "candidate") < 0){ /* Assume validation fail, nofatal */
 	clicon_debug(1, "Commit candidate failed");
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	    if (netconf_invalid_value(cbret, "protocol", clicon_err_reason)< 0)
-		"<error-tag>invalid-value</error-tag>"
+		goto done;
-		"<error-type>protocol</error-type>"
-		"<error-severity>error</error-severity>"
-		"<error-message>%s</error-message>"
-		"</rpc-error></rpc-reply>",
-		clicon_err_reason);
         goto ok;
     }
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");
  ok:
     retval = 0;
-    // done:
+ done:
+    if (cbx)
+	cbuf_free(cbx);
     return retval; /* may be zero if we ignoring errors from commit */
 } /* from_client_commit */
 
@@ -321,33 +325,31 @@ from_client_discard_changes(clicon_handle h,
 {
     int   retval = -1;
     int   piddb;
+    cbuf *cbx = NULL; /* Assist cbuf */
     
     /* Check if target locked by other client */
     piddb = xmldb_islocked(h, "candidate");
     if (piddb && mypid != piddb){
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	if ((cbx = cbuf_new()) == NULL){
-		"<error-tag>lock-denied</error-tag>"
+	    clicon_err(OE_XML, errno, "cbuf_new");
-		"<error-type>protocol</error-type>"
+	    goto done;
-		"<error-severity>error</error-severity>"
+	}	
-		"<error-message>Operation failed, lock is already held</error-message>"
+	cprintf(cbx, "<session-id>%d</session-id>", piddb);
-		"<error-info><session-id>%d</session-id></error-info>"
+	if (netconf_lock_denied(cbret, cbuf_get(cbx), "Operation failed, lock is already held") < 0)
-		"</rpc-error></rpc-reply>",
+	    goto done;
-		piddb);
 	goto ok;
     }
     if (xmldb_copy(h, "running", "candidate") < 0){
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	if (netconf_operation_failed(cbret, "application", clicon_err_reason)< 0)
-		"<error-tag>operation-failed</error-tag>"
+	    goto done;
-		"<error-type>application</error-type>"
-		"<error-severity>error</error-severity>"
-		"<error-info>read-registry</error-info>"
-		"</rpc-error></rpc-reply>");
 	goto ok;
     }
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");
  ok:
     retval = 0;
-    // done:
+ done:
+    if (cbx)
+	cbuf_free(cbx);
     return retval; /* may be zero if we ignoring errors from commit */
 }
 
@@ -367,11 +369,8 @@ from_client_validate(clicon_handle h,
     transaction_data_t *td = NULL;
 
     if (strcmp(db, "candidate") != 0 && strcmp(db, "tmp") != 0){
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	if (netconf_invalid_value(cbret, "protocol", "No such database")< 0)
-		"<error-tag>invalid-value</error-tag>"
+	    goto done;
-		"<error-type>protocol</error-type>"
-		"<error-severity>error</error-severity>"
-		"</rpc-error></rpc-reply>");
 	goto ok;
     }
     clicon_debug(1, "Validate %s",  db);
@@ -383,15 +382,14 @@ from_client_validate(clicon_handle h,
     if (validate_common(h, db, td) < 0){
 	clicon_debug(1, "Validate %s failed",  db);
 	/* XXX: candidate_validate should have proper error handling */
-	cprintf(cbret, "<rpc-reply><rpc-error>"
+	if (netconf_operation_failed(cbret, "application", clicon_err_reason)< 0)
-		"<error-tag>missing-attribute</error-tag>"
+	    goto done;
-		"<error-type>protocol</error-type>"
-		"<error-severity>error</error-severity>"
-		"<error-message>%s</error-message>"
-		"</rpc-error></rpc-reply>",
-		clicon_err_reason);
 	goto ok;
     }
+    /* Optionally write (potentially modified) tree back to candidate */
+     if (clicon_option_bool(h, "CLICON_TRANSACTION_MOD"))
+	 if (xmldb_put(h, "candidate", OP_REPLACE, td->td_target, NULL) < 0)
+	     goto done;
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");
  ok:
     retval = 0;

apps/backend/backend_handle.h

@@ -52,4 +52,8 @@ struct client_entry *backend_client_list(clicon_handle h);
 
 int backend_client_delete(clicon_handle h, struct client_entry *ce);
 
+int backend_nacm_list_set(clicon_handle h, cxobj *xnacm);
+
+cxobj * backend_nacm_list_get(clicon_handle h);
+
 #endif  /* _BACKEND_HANDLE_H_ */

apps/backend/backend_main.c

@@ -86,9 +86,9 @@ backend_terminate(clicon_handle h)
     clicon_debug(1, "%s", __FUNCTION__);
     if ((yspec = clicon_dbspec_yang(h)) != NULL)
 	yspec_free(yspec);
-    plugin_finish(h);
+    clixon_plugin_exit(h);
     /* Delete all backend plugin RPC callbacks */
-    backend_rpc_cb_delete_all(); 
+    rpc_callback_delete_all(); 
     if (pidfile)
 	unlink(pidfile);   
     if (sockpath)
@@ -174,8 +174,8 @@ db_merge(clicon_handle h,
     /* Get data as xml from db1 */
     if (xmldb_get(h, (char*)db1, NULL, 1, &xt) < 0)
 	goto done;
-    /* Merge xml into db2. WIthout commit */
+    /* Merge xml into db2. Without commit */
-    if (xmldb_put(h, (char*)db2, OP_MERGE, xt) < 0)
+    if (xmldb_put(h, (char*)db2, OP_MERGE, xt, NULL) < 0)
 	goto done;
     retval = 0;
  done:
@@ -254,12 +254,65 @@ plugin_start_useroptions(clicon_handle h,
 
     tmp = *(argv-1);
     *(argv-1) = argv0;
-    if (plugin_start_argv(h, argc+1, argv-1) < 0) 
+    if (clixon_plugin_start(h, argc+1, argv-1) < 0) 
 	return -1;
     *(argv-1) = tmp;
     return 0;
 }
 
+/*! Load external NACM file
+ */
+static int
+nacm_load_external(clicon_handle h)
+{
+    int         retval = -1;
+    char       *filename; /* NACM config file */
+    yang_spec  *yspec = NULL;
+    cxobj      *xt = NULL;
+    struct stat st;
+    FILE       *f = NULL;
+    int         fd;
+
+    filename = clicon_option_str(h, "CLICON_NACM_FILE");
+    if (filename == NULL || strlen(filename)==0){
+	clicon_err(OE_UNIX, errno, "CLICON_NACM_FILE not set in NACM external mode");
+	goto done;
+    }
+    if (stat(filename, &st) < 0){
+	clicon_err(OE_UNIX, errno, "%s", filename);
+	goto done;
+    }
+    if (!S_ISREG(st.st_mode)){
+	clicon_err(OE_UNIX, 0, "%s is not a regular file", filename);
+	goto done;
+    }
+    if ((f = fopen(filename, "r")) == NULL) {
+	clicon_err(OE_UNIX, errno, "configure file: %s", filename);
+	return -1;
+    }
+    if ((yspec = yspec_new()) == NULL)
+	goto done;
+    if (yang_parse(h, CLIXON_DATADIR, "ietf-netconf-acm", NULL, yspec) < 0)
+	goto done;
+    fd = fileno(f);
+    /* Read configfile */
+    if (xml_parse_file(fd, "</clicon>", yspec, &xt) < 0)
+	goto done;
+    if (xt == NULL){
+	clicon_err(OE_XML, 0, "No xml tree in %s", filename);
+	goto done;
+    }
+    if (backend_nacm_list_set(h, xt) < 0)
+	goto done;
+    retval = 0;
+ done:
+    if (yspec) /* The clixon yang-spec is not used after this */
+	yspec_free(yspec);
+    if (f)
+	fclose(f);
+    return retval;
+}
+
 /*! Merge xml in filename into database
  */
 static int
@@ -283,7 +336,7 @@ load_extraxml(clicon_handle h,
     if (xml_rootchild(xt, 0, &xt) < 0)
 	goto done;
     /* Merge user reset state */
-    if (xmldb_put(h, (char*)db, OP_MERGE, xt) < 0)
+    if (xmldb_put(h, (char*)db, OP_MERGE, xt, NULL) < 0)
 	goto done;
     retval = 0;
  done:
@@ -306,7 +359,7 @@ startup_mode_none(clicon_handle h)
 	if (xmldb_copy(h, "running", "candidate") < 0)
 	    goto done;
     /* Load plugins and call plugin_init() */
-    if (plugin_initiate(h) != 0) 
+    if (backend_plugin_initiate(h) != 0) 
 	goto done;
     retval = 0;
  done:
@@ -328,7 +381,7 @@ startup_mode_init(clicon_handle h)
 	if (xmldb_copy(h, "running", "candidate") < 0)
 	    goto done;
     /* Load plugins and call plugin_init() */
-    if (plugin_initiate(h) != 0) 
+    if (backend_plugin_initiate(h) != 0) 
 	goto done;
     retval = 0;
  done:
@@ -364,13 +417,13 @@ startup_mode_running(clicon_handle h,
     if (xmldb_copy(h, "running", "candidate") < 0)
 	goto done;
     /* Load plugins and call plugin_init() */
-    if (plugin_initiate(h) != 0) 
+    if (backend_plugin_initiate(h) != 0) 
 	goto done;
     /* Clear tmp db */
     if (db_reset(h, "tmp") < 0)
 	goto done;
     /* Application may define extra xml in its reset function*/
-    if (plugin_reset_state(h, "tmp") < 0)   
+    if (clixon_plugin_reset(h, "tmp") < 0)   
 	goto done;
     /* Get application extra xml from file */
     if (load_extraxml(h, extraxml_file, "tmp") < 0)   
@@ -437,13 +490,13 @@ startup_mode_startup(clicon_handle h,
 	if (xmldb_create(h, "startup") < 0) /* diff */
 	    return -1;
     /* Load plugins and call plugin_init() */
-    if (plugin_initiate(h) != 0) 
+    if (backend_plugin_initiate(h) != 0) 
 	goto done;
     /* Clear tmp db */
     if (db_reset(h, "tmp") < 0)
 	goto done;
     /* Application may define extra xml in its reset function*/
-    if (plugin_reset_state(h, "tmp") < 0)  
+    if (clixon_plugin_reset(h, "tmp") < 0)  
 	goto done;
     /* Get application extra xml from file */
     if (load_extraxml(h, extraxml_file, "tmp") < 0)   
@@ -475,7 +528,8 @@ startup_mode_startup(clicon_handle h,
 }
 
 int
-main(int argc, char **argv)
+main(int    argc,
+     char **argv)
 {
     int           retval = -1;
     char          c;
@@ -497,14 +551,13 @@ main(int argc, char **argv)
     int           xml_cache;
     int           xml_pretty;
     char         *xml_format;
+    char         *nacm_mode;
     
     /* In the startup, logs to stderr & syslog and debug flag set later */
     clicon_log_init(__PROGRAM__, LOG_INFO, CLICON_LOG_STDERR|CLICON_LOG_SYSLOG);
     /* Initiate CLICON handle */
     if ((h = backend_handle_init()) == NULL)
 	return -1;
-    if (backend_plugin_init(h) != 0) 
-	return -1;
     foreground = 0;
     once = 0;
     zap = 0;
@@ -552,6 +605,11 @@ main(int argc, char **argv)
 	    usage(argv[0], h);
 	return -1;
     }
+    /* External NACM file? */
+    nacm_mode = clicon_option_str(h, "CLICON_NACM_MODE");
+    if (nacm_mode && strcmp(nacm_mode, "external") == 0)
+	if (nacm_load_external(h) < 0)
+	    goto done;
     
     /* Now run through the operational args */
     opterr = 1;
@@ -703,13 +761,12 @@ main(int argc, char **argv)
     if ((xml_pretty = clicon_option_bool(h, "CLICON_XMLDB_PRETTY")) >= 0)
 	if (xmldb_setopt(h, "pretty", (void*)(intptr_t)xml_pretty) < 0)
 	    goto done;
-    /* If startup mode is not defined, eg via OPTION or -s, assume old method */
+    /* Startup mode needs to be defined,  */
     startup_mode = clicon_startup_mode(h);
     if (startup_mode == -1){ 	
 	clicon_log(LOG_ERR, "Startup mode undefined. Specify option CLICON_STARTUP_MODE or specify -s option to clicon_backend.\n"); 
 	goto done;
     }
-    else {
     /* Init running db if it is not there
      */
     if (xmldb_exists(h, "running") != 1)
@@ -736,10 +793,9 @@ main(int argc, char **argv)
     /* Initiate the shared candidate. */
     if (xmldb_copy(h, "running", "candidate") < 0)
 	goto done;
-	/* Call plugin_start with user -- options */
+    /* Call backend plugin_start with user -- options */
     if (plugin_start_useroptions(h, argv0, argc, argv) <0)
 	goto done;
-    }
     if (once)
 	goto done;
 

apps/backend/backend_plugin.c

@@ -64,402 +64,137 @@
 #include "backend_plugin.h"
 #include "backend_commit.h"
 
-/*
- * Types
- */
-/* Following are specific to backend. For common see clicon_plugin.h 
- * @note  the following should match the prototypes in clixon_backend.h
- */
-#define PLUGIN_RESET           "plugin_reset"
-typedef int (plgreset_t)(clicon_handle h, const char *db); /* Reset system status */
-
-/*! Plugin callback, if defined called to get state data from plugin
- * @param[in]    h      Clicon handle
- * @param[in]    xpath  String with XPATH syntax. or NULL for all
- * @param[in]    xtop   XML tree, <config/> on entry. 
- * @retval       0      OK
- * @retval      -1      Error
- * @see xmldb_get
- */
-#define PLUGIN_STATEDATA       "plugin_statedata"
-typedef int (plgstatedata_t)(clicon_handle h, char *xpath, cxobj *xtop);
-
-#define PLUGIN_TRANS_BEGIN     "transaction_begin"
-#define PLUGIN_TRANS_VALIDATE  "transaction_validate"
-#define PLUGIN_TRANS_COMPLETE  "transaction_complete"
-#define PLUGIN_TRANS_COMMIT    "transaction_commit"
-#define PLUGIN_TRANS_END       "transaction_end"
-#define PLUGIN_TRANS_ABORT     "transaction_abort"
-
-
-typedef int (trans_cb_t)(clicon_handle h, transaction_data td); /* Transaction cbs */
-
-/* Backend (config) plugins */
-struct plugin {
-    char	       p_name[PATH_MAX]; /* Plugin name */
-    void	      *p_handle;	 /* Dynamic object handle */
-    plginit_t	      *p_init;		 /* Init */
-    plgstart_t	      *p_start;		 /* Start */
-    plgexit_t         *p_exit;		 /* Exit */
-    plgreset_t	      *p_reset;		 /* Reset state */
-    plgstatedata_t    *p_statedata;      /* State-data callback */
-    trans_cb_t        *p_trans_begin;	 /* Transaction start */
-    trans_cb_t        *p_trans_validate; /* Transaction validation */
-    trans_cb_t        *p_trans_complete; /* Transaction validation complete */
-    trans_cb_t        *p_trans_commit;   /* Transaction commit */
-    trans_cb_t        *p_trans_end;	 /* Transaction completed  */
-    trans_cb_t        *p_trans_abort;	 /* Transaction aborted */
-
-};
-
-/*
- * Local variables
- */
-static int _nplugins = 0;
-static struct plugin *_plugins = NULL;
-
-/*! Find a plugin by name and return the dlsym handl
- * Used by libclicon code to find callback funcctions in plugins.
- * @param[in]  h       Clicon handle
- * @param[in]  h       Name of plugin
- * @retval     handle  Plugin handle if found
- * @retval     NULL    Not found
- */
-static void *
-config_find_plugin(clicon_handle h, 
-		   char         *name)
-{
-    int            i;
-    struct plugin *p;
-
-    for (i = 0; i < _nplugins; i++){
-	p = &_plugins[i];
-	if (strcmp(p->p_name, name) == 0)
-	    return p->p_handle;
-    }
-    return NULL;
-}
-
-/*! Initialize plugin code (not the plugins themselves)
- * @param[in]  h       Clicon handle
- * @retval     0       OK
- * @retval    -1       Error
- */
-int
-backend_plugin_init(clicon_handle h)
-{
-    find_plugin_t *fp   = config_find_plugin;
-    clicon_hash_t *data = clicon_data(h);
-
-    /* Register CLICON_FIND_PLUGIN in data hash */
-    if (hash_add(data, "CLICON_FIND_PLUGIN", &fp, sizeof(fp)) == NULL) {
-	clicon_err(OE_UNIX, errno, "failed to register CLICON_FIND_PLUGIN");
-	return -1;
-    }
-    return 0;
-}
-
-/*! Unload a plugin
- * @param[in]  h       Clicon handle
- * @param[in]  plg     Plugin structure
- * @retval     0       OK
- * @retval    -1       Error
- */
-static int
-backend_plugin_unload(clicon_handle  h, 
-		      struct plugin *plg)
-{
-    int   retval=-1;
-    char *error;
-
-    /* Call exit function is it exists */
-    if (plg->p_exit)
-	plg->p_exit(h);
-    
-    dlerror();    /* Clear any existing error */
-    if (dlclose(plg->p_handle) != 0) {
-	error = (char*)dlerror();
-	clicon_err(OE_UNIX, 0, "dlclose: %s", error?error:"Unknown error");
-	goto done;
-	/* Just report */
-    }
-    else 
-	clicon_debug(1, "Plugin '%s' unloaded.", plg->p_name);
-    retval = 0;
- done:
-    return retval;
-}
-
-
-/*! Load a dynamic plugin and call its init-function
- * @param[in]  h       Clicon handle
- * @param[in]  file    The plugin (.so) to load
- * @param[in]  dlflags Arguments to dlopen(3)
- * @retval     plugin  Plugin struct
- * @retval     NULL    Error
- */
-static struct plugin *
-backend_plugin_load (clicon_handle h, 
-		     char         *file, 
-		     int           dlflags)
-{
-    void          *handle;
-    char          *name;
-    struct plugin *new = NULL;
-
-    if ((handle = plugin_load(h, file, dlflags)) == NULL)
-	goto done;
-    if ((new = malloc(sizeof(*new))) == NULL) {
-	clicon_err(OE_UNIX, errno, "dhunk: %s", strerror(errno));
-	dlclose(handle);
-	return NULL;
-    }
-    memset(new, 0, sizeof(*new));
-    name = strrchr(file, '/') ? strrchr(file, '/')+1 : file;
-    clicon_debug(2, "Loading plugin '%s'.", name);
-    snprintf(new->p_name, sizeof(new->p_name), "%*s",
-	     (int)strlen(name)-2, name);
-    new->p_handle   = handle;
-    if ((new->p_start    = dlsym(handle, PLUGIN_START)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_START);
-    if ((new->p_exit     = dlsym(handle, PLUGIN_EXIT)) != NULL)
-	 clicon_debug(2, "%s callback registered.", PLUGIN_EXIT);
-    if ((new->p_reset    = dlsym(handle, PLUGIN_RESET)) != NULL)
-	 clicon_debug(2, "%s callback registered.", PLUGIN_RESET);
-    if ((new->p_statedata    = dlsym(handle, PLUGIN_STATEDATA)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_STATEDATA);
-    if ((new->p_trans_begin    = dlsym(handle, PLUGIN_TRANS_BEGIN)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_BEGIN);
-    if ((new->p_trans_validate = dlsym(handle, PLUGIN_TRANS_VALIDATE)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_VALIDATE);
-    if ((new->p_trans_complete = dlsym(handle, PLUGIN_TRANS_COMPLETE)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_COMPLETE);
-    if ((new->p_trans_commit   = dlsym(handle, PLUGIN_TRANS_COMMIT)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_COMMIT);
-    if ((new->p_trans_end      = dlsym(handle, PLUGIN_TRANS_END)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_END);
-    if ((new->p_trans_abort    = dlsym(handle, PLUGIN_TRANS_ABORT)) != NULL)
-	clicon_debug(2, "%s callback registered.", PLUGIN_TRANS_ABORT);
-    clicon_debug(2, "Plugin '%s' loaded.\n", name);
- done:
-    return new;
-}
-
-/*! Request plugins to reset system state
- * The system 'state' should be the same as the contents of running_db
- * @param[in]  h       Clicon handle
- * @param[in]  dbname  Name of database
- * @retval     0       OK
- * @retval    -1       Error
- */
-int
-plugin_reset_state(clicon_handle h,
-		   const char   *db)
-{ 
-    int            i;
-    struct plugin *p;
-
-    for (i = 0; i < _nplugins; i++)  {
-	p = &_plugins[i];
-	if (p->p_reset) {
-	    clicon_debug(1, "Calling plugin_reset() for %s\n",
-			 p->p_name);
-	    if (((p->p_reset)(h, db)) < 0) {
-		clicon_err(OE_FATAL, 0, "plugin_reset() failed for %s\n",
-			   p->p_name);
-		return -1;
-	    }
-	}
-    }
-    return 0;
-}
-
-/*! Call plugin_start in all plugins
- * @param[in]  h       Clicon handle
- * @param[in]  argc    Command-line arguments
- * @param[in]  argv    Command-line arguments
- * @retval     0       OK
- * @retval    -1       Error
- */
-int
-plugin_start_argv(clicon_handle h, 
-		   int           argc, 
-		   char        **argv)
-{
-    int            i;
-    struct plugin *p;
-
-    for (i = 0; i < _nplugins; i++)  {
-	p = &_plugins[i];
-	if (p->p_start) {
-	    optind = 0;
-	    if (((p->p_start)(h, argc, argv)) < 0) {
-		clicon_err(OE_FATAL, 0, "plugin_start() failed for %s\n",
-			   p->p_name);
-		return -1;
-	    }
-	}
-    }
-    return 0;
-}
-	
-/*! Append plugin to list
- * @param[in]  p       Plugin
- * @retval     0       OK
- * @retval    -1       Error
- */
-static int
-plugin_append(struct plugin *p)
-{
-    struct plugin *new;
-    
-    if ((new = realloc(_plugins, (_nplugins+1) * sizeof (*p))) == NULL) {
-	clicon_err(OE_UNIX, errno, "realloc");
-	return -1;
-    }
-    
-    memset (&new[_nplugins], 0, sizeof(new[_nplugins]));
-    memcpy (&new[_nplugins], p, sizeof(new[_nplugins]));
-    _plugins = new;
-    _nplugins++;
-
-    return 0;
-}
-
-/*! Load backend plugins found in a directory 
- * The plugins must have the '.so' suffix
- * @param[in]  h       Clicon handle
- * @param[in]  dir     Backend plugin directory
- * @retval     0       OK
- * @retval    -1       Error
- */
-static int
-backend_plugin_load_dir(clicon_handle h, 
-		       const char   *dir)
-{
-    int            retval = -1;
-    int            i;
-    int            np = 0;
-    int            ndp;
-    struct stat    st;
-    char           filename[MAXPATHLEN];
-    struct dirent *dp = NULL;
-    struct plugin *new;
-    struct plugin *p = NULL;
-    char           master[MAXPATHLEN];
-    char          *master_plugin;
-
-    /* Format master plugin path */
-    if ((master_plugin = clicon_master_plugin(h)) == NULL){
-	clicon_err(OE_PLUGIN, 0, "clicon_master_plugin option not set");
-	goto quit;
-    }
-    snprintf(master, MAXPATHLEN-1, "%s.so", master_plugin);
-
-    /* Allocate plugin group object */
-    /* Get plugin objects names from plugin directory */
-    if((ndp = clicon_file_dirent(dir, &dp, "(.so)$", S_IFREG))<0)
-	goto quit;
-    
-    /* reset num plugins */
-    np = 0;
-
-    /* Master plugin must be loaded first if it exists. */
-    snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, master);
-    if (stat(filename, &st) == 0) {
-	clicon_debug(1, "Loading master plugin '%.*s' ...", 
-		     (int)strlen(filename), filename);
-
-	new = backend_plugin_load(h, filename, RTLD_NOW|RTLD_GLOBAL);
-	if (new == NULL)
-	    goto quit;
-	if (plugin_append(new) < 0)
-	    goto quit;
-	free(new);
-    }  
-
-    /* Now load the rest. Note plugins is the global variable */
-    for (i = 0; i < ndp; i++) {
-	if (strcmp(dp[i].d_name, master) == 0)
-	    continue; /* Skip master now */
-	snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, dp[i].d_name);
-	clicon_debug(1, "Loading plugin '%.*s' ...",  (int)strlen(filename), filename);
-	new = backend_plugin_load(h, filename, RTLD_NOW);
-	if (new == NULL) 
-	    goto quit;
-	/* Append to 'plugins' */
-	if (plugin_append(new) < 0)
-	    goto quit;
-	free(new);
-    }
-    
-    /* All good. */
-    retval = 0;
-    
-quit:
-    if (retval != 0) {
-	/* XXX p is always NULL */
-	if (_plugins) {
-	    while (--np >= 0){
-		if ((p = &_plugins[np]) == NULL)
-		    continue;
-		backend_plugin_unload(h, p);
-		free(p);
-	    }
-	    free(_plugins);
-	    _plugins=0;
-	}
-    }
-    if (dp)
-	free(dp);
-    return retval;
-}
-
-
 /*! Load a plugin group.
  * @param[in]  h       Clicon handle
  * @retval     0       OK
  * @retval    -1       Error
  */
 int
-plugin_initiate(clicon_handle h)
+backend_plugin_initiate(clicon_handle h)
 {
     char *dir;
 
-    /* First load CLICON system plugins */
+    /* Load application plugins */
-    if (backend_plugin_load_dir(h, CLIXON_BACKEND_SYSDIR) < 0)
+    if ((dir = clicon_backend_dir(h)) == NULL)
-	return -1;
-
-    /* Then load application plugins */
-    dir = clicon_backend_dir(h);
-    /* If backend directory, load the backend plugisn */
-    if (dir && backend_plugin_load_dir(h, dir) < 0)
-	return -1;
-    
 	return 0;
+    return clixon_plugins_load(h, CLIXON_PLUGIN_INIT, dir,
+			       clicon_option_str(h, "CLICON_BACKEND_REGEXP"));
 }
 
-/*! Unload and deallocate all backend plugins
+/*! Request plugins to reset system state
+ * The system 'state' should be the same as the contents of running_db
  * @param[in]  h       Clicon handle
+ * @param[in]  db      Name of database
  * @retval     0       OK
  * @retval    -1       Error
  */
 int
-plugin_finish(clicon_handle h)
+clixon_plugin_reset(clicon_handle h, 
+		    char         *db)
 {
-    int            i;
+    clixon_plugin *cp = NULL;
-    struct plugin *p;
+    plgreset_t    *resetfn;          /* Plugin auth */
+    int            retval = 1;
     
-    for (i = 0; i < _nplugins; i++) {
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	if ((resetfn = cp->cp_api.ca_reset) == NULL)
-	backend_plugin_unload(h, p);
+	    continue;
+	if ((retval = resetfn(h, db)) < 0) {
+	    clicon_debug(1, "plugin_start() failed\n");
+	    return -1;
 	}
-    if (_plugins){
+	break;
-	free(_plugins);
-	_plugins = NULL;
     }
-    _nplugins = 0;
+    return retval;
-    return 0;
+}
+
+/*! Go through all backend statedata callbacks and collect state data
+ * This is internal system call, plugin is invoked (does not call) this function
+ * Backend plugins can register 
+ * @param[in]     h       clicon handle
+ * @param[in]     xpath   String with XPATH syntax. or NULL for all
+ * @param[in,out] xtop    State XML tree is merged with existing tree.
+ * @retval       -1       Error
+ * @retval        0       OK
+ * @retval        1       Statedata callback failed
+ * @note xtop can be replaced
+ */
+int
+clixon_plugin_statedata(clicon_handle        h,
+			char                *xpath,
+			cxobj              **xtop)
+{
+    int            retval = -1;
+    int            i;
+    cxobj         *x = NULL;
+    yang_spec     *yspec;
+    cxobj        **xvec = NULL;
+    size_t         xlen;
+    cxobj         *xc;
+    clixon_plugin  *cp = NULL;
+    plgstatedata_t *fn;          /* Plugin statedata fn */
+    char           *reason = NULL;
+    
+    if ((yspec =  clicon_dbspec_yang(h)) == NULL){
+	clicon_err(OE_YANG, ENOENT, "No yang spec");
+	goto done;
+    }
+    if (*xtop==NULL){
+	clicon_err(OE_CFG, ENOENT, "XML tree expected");
+	goto done;
+    }
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
+	if ((fn = cp->cp_api.ca_statedata) == NULL)
+	    continue;
+	if ((x = xml_new("config", NULL, NULL)) == NULL)
+	    goto done;
+	if (fn(h, xpath, x) < 0){
+	    retval = 1;
+	    goto done; /* Dont quit here on user callbacks */
+	}
+	if (xml_merge(*xtop, x, yspec, &reason) < 0)
+	    goto done;
+	if (reason){
+	    while ((xc = xml_child_i(*xtop, 0)) != NULL)
+		xml_purge(xc);	    
+	    if (netconf_operation_failed_xml(xtop, "rpc", reason)< 0)
+		goto done;
+	    goto ok;
+	}
+	if (x){
+	    xml_free(x);
+	    x = NULL;
+	}
+    }
+    /* Code complex to filter out anything that is outside of xpath */
+    if (xpath_vec(*xtop, xpath?xpath:"/", &xvec, &xlen) < 0)
+	goto done;
+
+    /* If vectors are specified then mark the nodes found and
+     * then filter out everything else,
+     * otherwise return complete tree.
+     */
+    if (xvec != NULL){
+	for (i=0; i<xlen; i++)
+	    xml_flag_set(xvec[i], XML_FLAG_MARK);
+    }
+    /* Remove everything that is not marked */
+    if (!xml_flag(*xtop, XML_FLAG_MARK))
+	if (xml_tree_prune_flagged_sub(*xtop, XML_FLAG_MARK, 1, NULL) < 0)
+	    goto done;
+    /* reset flag */
+    if (xml_apply(*xtop, CX_ELMNT, (xml_applyfn_t*)xml_flag_reset, (void*)XML_FLAG_MARK) < 0)
+	goto done;
+ ok:
+    retval = 0;
+ done:
+    if (reason)
+	free(reason);
+    if (x)
+	xml_free(x);
+    if (xvec)
+	free(xvec);
+    return retval;
 }
 
 /*! Create and initialize transaction */
@@ -510,21 +245,20 @@ int
 plugin_transaction_begin(clicon_handle       h, 
 			  transaction_data_t *td)
 {
-    int            i;
     int            retval = 0;
-    struct plugin *p;
+    clixon_plugin *cp = NULL;
+    trans_cb_t    *fn;
 
-    for (i = 0; i < _nplugins; i++) {
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	if ((fn = cp->cp_api.ca_trans_begin) == NULL)
-	if (p->p_trans_begin) 
+	    continue;
-	    if ((retval = (p->p_trans_begin)(h, (transaction_data)td)) < 0){
+	if ((retval = fn(h, (transaction_data)td)) < 0){
 	    if (!clicon_errno) /* sanity: log if clicon_err() is not called ! */
-		    clicon_log(LOG_NOTICE, "%s: Plugin '%s' %s callback does not make clicon_err call on error", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' transaction_begin callback does not make clicon_err call on error", 
-			       __FUNCTION__, p->p_name, PLUGIN_TRANS_BEGIN);
+			       __FUNCTION__, cp->cp_name);
 
 		break;
 	}
-
     }
     return retval;
 }
@@ -540,18 +274,16 @@ plugin_transaction_validate(clicon_handle       h,
 			    transaction_data_t *td)
 {
     int            retval = 0;
-    int            i;
+    clixon_plugin *cp = NULL;
+    trans_cb_t    *fn;
 
-    struct plugin *p;
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-
+	if ((fn = cp->cp_api.ca_trans_validate) == NULL)
-    for (i = 0; i < _nplugins; i++){
+	    continue;
-	p = &_plugins[i];
+	if ((retval = fn(h, (transaction_data)td)) < 0){
-	if (p->p_trans_validate) 
-	    if ((retval = (p->p_trans_validate)(h, (transaction_data)td)) < 0){
 	    if (!clicon_errno) /* sanity: log if clicon_err() is not called ! */
-		    clicon_log(LOG_NOTICE, "%s: Plugin '%s' %s callback does not make clicon_err call on error", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' transaction_validate callback does not make clicon_err call on error", 
-			       __FUNCTION__, p->p_name, PLUGIN_TRANS_VALIDATE);
+			   __FUNCTION__, cp->cp_name);
-
 	    break;
 	}
     }
@@ -570,17 +302,17 @@ int
 plugin_transaction_complete(clicon_handle       h, 
 			    transaction_data_t *td)
 {
-    int            i;
     int            retval = 0;
-    struct plugin *p;
+    clixon_plugin *cp = NULL;
+    trans_cb_t    *fn;
     
-    for (i = 0; i < _nplugins; i++){
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	if ((fn = cp->cp_api.ca_trans_complete) == NULL)
-	if (p->p_trans_complete) 
+	    continue;
-	    if ((retval = (p->p_trans_complete)(h, (transaction_data)td)) < 0){
+	if ((retval = fn(h, (transaction_data)td)) < 0){
 	    if (!clicon_errno) /* sanity: log if clicon_err() is not called ! */
-		    clicon_log(LOG_NOTICE, "%s: Plugin '%s' %s callback does not make clicon_err call on error", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' trans_complete callback does not make clicon_err call on error", 
-			       __FUNCTION__, p->p_name, PLUGIN_TRANS_COMPLETE);
+			   __FUNCTION__, cp->cp_name);
 	    
 	    break;
 	}
@@ -604,8 +336,8 @@ plugin_transaction_revert(clicon_handle       h,
 {
     int                retval = 0;
     transaction_data_t tr; /* revert transaction */
-    int                i;
+    clixon_plugin     *cp = NULL;
-    struct plugin     *p;
+    trans_cb_t        *fn;
     
     /* Create a new reversed transaction from the original where src and target
        are swapped */
@@ -620,12 +352,12 @@ plugin_transaction_revert(clicon_handle       h,
     tr.td_scvec = td->td_tcvec;
     tr.td_tcvec = td->td_scvec;
 
-    for (i = nr-1; i>=0; i--){
+    while ((cp = clixon_plugin_each_revert(h, cp, nr)) != NULL) {
-	p = &_plugins[i];
+	if ((fn = cp->cp_api.ca_trans_commit) == NULL)
-	if (p->p_trans_commit) 
+	    continue;
-	    if ((p->p_trans_commit)(h, (transaction_data)&tr) < 0){
+	if ((retval = fn(h, (transaction_data)td)) < 0){
-		clicon_log(LOG_NOTICE, "Plugin '%s' %s revert callback failed", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' trans_commit revert callback failed", 
-			   p->p_name, PLUGIN_TRANS_COMMIT);
+			   __FUNCTION__, cp->cp_name);
 		break; 
 	}
     }
@@ -646,18 +378,20 @@ plugin_transaction_commit(clicon_handle       h,
 			  transaction_data_t *td)
 {
     int            retval = 0;
-    int            i;
+    clixon_plugin *cp = NULL;
-    struct plugin *p;
+    trans_cb_t    *fn;
+    int            i=0;
 
-    for (i = 0; i < _nplugins; i++){
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	i++;
-	if (p->p_trans_commit) 
+	if ((fn = cp->cp_api.ca_trans_commit) == NULL)
-	    if ((retval = (p->p_trans_commit)(h, (transaction_data)td)) < 0){
+	    continue;
+	if ((retval = fn(h, (transaction_data)td)) < 0){
 	    if (!clicon_errno) /* sanity: log if clicon_err() is not called ! */
-		    clicon_log(LOG_NOTICE, "%s: Plugin '%s' %s callback does not make clicon_err call on error", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' trans_commit callback does not make clicon_err call on error", 
-			       __FUNCTION__, p->p_name, PLUGIN_TRANS_COMMIT);
+			   __FUNCTION__, cp->cp_name);
 	    /* Make an effort to revert transaction */
-		plugin_transaction_revert(h, td, i); 
+	    plugin_transaction_revert(h, td, i-1); 
 	    break;
 	}
     }
@@ -675,17 +409,16 @@ plugin_transaction_end(clicon_handle h,
 		       transaction_data_t *td)
 {
     int            retval = 0;
-    int            i;
+    clixon_plugin *cp = NULL;
-    struct plugin *p;
+    trans_cb_t    *fn;
     
-    for (i = 0; i < _nplugins; i++) {
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	if ((fn = cp->cp_api.ca_trans_end) == NULL)
-	if (p->p_trans_end) 
+	    continue;
-	    if ((retval = (p->p_trans_end)(h, (transaction_data)td)) < 0){
+	if ((retval = fn(h, (transaction_data)td)) < 0){
 	    if (!clicon_errno) /* sanity: log if clicon_err() is not called ! */
-		    clicon_log(LOG_NOTICE, "%s: Plugin '%s' %s callback does not make clicon_err call on error", 
+		clicon_log(LOG_NOTICE, "%s: Plugin '%s' trans_end callback does not make clicon_err call on error", 
-			       __FUNCTION__, p->p_name, PLUGIN_TRANS_END);
+			   __FUNCTION__, cp->cp_name);
-
 	    break;
 	}
     }
@@ -703,94 +436,14 @@ plugin_transaction_abort(clicon_handle       h,
 			 transaction_data_t *td)
 {
     int            retval = 0;
-    int            i;
+    clixon_plugin *cp = NULL;
-    struct plugin *p;
+    trans_cb_t    *fn;
 
-    for (i = 0; i < _nplugins; i++)  {
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
-	p = &_plugins[i];
+	if ((fn = cp->cp_api.ca_trans_abort) == NULL)
-	if (p->p_trans_abort) 
+	    continue;
-	    (p->p_trans_abort)(h, (transaction_data)td); /* dont abort on error */
+	fn(h, (transaction_data)td); /* dont abort on error */
     }
     return retval;
 }
 	
-/*----------------------------------------------------------------------
- * Backend state data callbacks
- */
-
-/*! Go through all backend statedata callbacks and collect state data
- * This is internal system call, plugin is invoked (does not call) this function
- * Backend plugins can register 
- * @param[in]  h       clicon handle
- * @param[in]  xpath   String with XPATH syntax. or NULL for all
- * @param[in,out] xml  XML tree.
- * @retval -1   Error
- * @retval  0   OK
- * @retval  1   Statedata callback failed
- */
-int
-backend_statedata_call(clicon_handle        h,
-		       char                *xpath,
-		       cxobj               *xtop)
-{
-    int            retval = -1;
-    struct plugin *p;
-    int            i;
-    cxobj         *x = NULL;
-    yang_spec     *yspec;
-    cxobj        **xvec = NULL;
-    size_t         xlen;
-
-    if ((yspec =  clicon_dbspec_yang(h)) == NULL){
-	clicon_err(OE_YANG, ENOENT, "No yang spec");
-	goto done;
-    }
-    if (xtop==NULL){
-	clicon_err(OE_CFG, ENOENT, "XML tree expected");
-	goto done;
-    }
-    for (i = 0; i < _nplugins; i++)  {
-	p = &_plugins[i];
-	if (p->p_statedata) {
-	    if ((x = xml_new("config", NULL, NULL)) == NULL)
-		goto done;
-	    if ((p->p_statedata)(h, xpath, x) < 0){
-		retval = 1;
-		goto done; /* Dont quit here on user callbacks */
-	    }
-	    if (xml_merge(xtop, x, yspec) < 0)
-		goto done;
-	    if (x){
-		xml_free(x);
-		x = NULL;
-	    }
-	}
-    }
-    /* Code complex to filter out anything that is outside of xpath */
-    if (xpath_vec(xtop, xpath?xpath:"/", &xvec, &xlen) < 0)
-	goto done;
-
-    /* If vectors are specified then mark the nodes found and
-     * then filter out everything else,
-     * otherwise return complete tree.
-     */
-    if (xvec != NULL){
-	for (i=0; i<xlen; i++)
-	    xml_flag_set(xvec[i], XML_FLAG_MARK);
-    }
-    /* Remove everything that is not marked */
-    if (!xml_flag(xtop, XML_FLAG_MARK))
-	if (xml_tree_prune_flagged_sub(xtop, XML_FLAG_MARK, 1, NULL) < 0)
-	    goto done;
-    /* reset flag */
-    if (xml_apply(xtop, CX_ELMNT, (xml_applyfn_t*)xml_flag_reset, (void*)XML_FLAG_MARK) < 0)
-	goto done;
-    retval = 0;
- done:
-    if (x)
-	xml_free(x);
-    if (xvec)
-	free(xvec);
-    return retval;
-}
-

apps/backend/backend_plugin.h

@@ -67,14 +67,11 @@ typedef struct {
 /*
  * Prototypes
  */
-int  backend_plugin_init(clicon_handle h);
+int backend_plugin_initiate(clicon_handle h); 
-int  plugin_initiate(clicon_handle h); 
-int  plugin_finish(clicon_handle h);
 
-int  plugin_reset_state(clicon_handle h, const char *db);
+int clixon_plugin_reset(clicon_handle h, char *db);
-int  plugin_start_argv(clicon_handle h, int argc, char **argv);
 
-int backend_statedata_call(clicon_handle h, char *xpath, cxobj *xml);
+int clixon_plugin_statedata(clicon_handle h, char *xpath, cxobj **xtop);
 
 transaction_data_t * transaction_new(void);
 int transaction_free(transaction_data_t *);

apps/backend/clixon_backend.h

@@ -49,62 +49,4 @@
 #include <clixon/clixon_backend_handle.h>
 #include <clixon/clixon_backend_transaction.h>
 
-/*! Clicon Backend plugin callbacks: use these in your backend plugin code 
- */
-
-/*! Called when plugin loaded. Only mandadory callback. All others optional 
- * @see plginit_t
- */
-int plugin_init(clicon_handle h);
-
-/* Called when backend started with cmd-line arguments from daemon call. 
- * @see plgstart_t
- */
-int plugin_start(clicon_handle h, int argc, char **argv);
-
-/* Called just before plugin unloaded. 
- * @see plgexit_t
- */
-int plugin_exit(clicon_handle h);
-
-/*! Reset system state to original state. Eg at reboot before running thru config. 
- * @see plgreset_t 
- */
-int plugin_reset(clicon_handle h, const char *db);
-
-/*! Retreive statedata, add statedata to XML tree
- * @see plgstatedata_ t
- */
-int plugin_statedata(clicon_handle h, char *xpath, cxobj *xtop);
-
-/*! Called before a commit/validate sequence begins. Eg setup state before commit 
- * @see trans_cb_t
- */
-int transaction_begin(clicon_handle h, transaction_data td);
-
-/*! Validate. 
- * @see trans_cb_t
- */
-int transaction_validate(clicon_handle h, transaction_data td);
-
-/* Called after a validation completed succesfully (but before commit). 
- * @see trans_cb_t
- */
-int transaction_complete(clicon_handle h, transaction_data td);
-
-/* Commit. 
- * @see trans_cb_t
- */
-int transaction_commit(clicon_handle h, transaction_data td);
-
-/* Called after a commit sequence completed succesfully. 
- * @see trans_cb_t
- */
-int transaction_end(clicon_handle h, transaction_data td);
-
-/* Called if commit or validate sequence fails. After eventual rollback. 
- * @see trans_cb_t
- */
-int transaction_abort(clicon_handle h, transaction_data td);
-
 #endif /* _CLIXON_BACKEND_H_ */

apps/backend/clixon_backend_handle.c

@@ -52,6 +52,7 @@
 #include <regex.h>
 #include <syslog.h>
 #include <netinet/in.h>
+#include <limits.h>
 
 /* cligen */
 #include <cligen/cligen.h>
@@ -89,6 +90,7 @@ struct backend_handle {
     struct client_entry     *bh_ce_list;   /* The client list */
     int                      bh_ce_nr;     /* Number of clients, just increment */
     struct handle_subscription *bh_subscription; /* Event subscription list */
+    cxobj                   *bh_nacm;      /* NACM external struct */
 };
 
 /*! Creates and returns a clicon config handle for other CLICON API calls
@@ -105,11 +107,14 @@ backend_handle_init(void)
 int
 backend_handle_exit(clicon_handle h)
 {
+    struct backend_handle *bh = handle(h);
     struct client_entry   *ce;
 
     /* only delete client structs, not close sockets, etc, see backend_client_rm */
     while ((ce = backend_client_list(h)) != NULL)
 	backend_client_delete(h, ce);
+    if (bh->bh_nacm)
+	xml_free(bh->bh_nacm);
     clicon_handle_exit(h); /* frees h and options */
     return 0;
 }
@@ -430,110 +435,22 @@ subscription_each(clicon_handle               h,
     return hs;
 }
 
-/*--------------------------------------------------------------------
- * Backend netconf rpc callbacks
- */
-typedef struct {
-    qelem_t 	   rc_qelem;	/* List header */
-    backend_rpc_cb rc_callback;  /* RPC Callback */
-    void	  *rc_arg;	/* Application specific argument to cb */
-    char          *rc_tag;	/* Xml tag when matched, callback called */
-} backend_rpc_cb_entry;
-
-/* List of backend rpc callback entries */
-static backend_rpc_cb_entry *rpc_cb_list = NULL;
-
-/*! Register netconf backend rpc callback
- * Called from plugin to register a callback for a specific netconf XML tag.
- *
- * @param[in]  h       clicon handle
- * @param[in]  cb,     Callback called 
- * @param[in]  arg,    Arg to send to callback 
- * @param[in]  tag     Xml tag when callback is made 
- * @see backend_rpc_cb_call
- */
 int
-backend_rpc_cb_register(clicon_handle  h,
+backend_nacm_list_set(clicon_handle h,
-			backend_rpc_cb cb,
+		      cxobj        *xnacm)
-			void          *arg,       
-			char          *tag)
 {
-    backend_rpc_cb_entry *rc;
+    struct backend_handle *bh = handle(h);
 
-    if ((rc = malloc(sizeof(backend_rpc_cb_entry))) == NULL) {
+    if (bh->bh_nacm)
-	clicon_err(OE_DB, errno, "malloc: %s", strerror(errno));
+	xml_free(bh->bh_nacm);
-	goto catch;
+    bh->bh_nacm = xnacm;
-    }
-    memset (rc, 0, sizeof (*rc));
-    rc->rc_callback = cb;
-    rc->rc_arg  = arg;
-    rc->rc_tag  = strdup(tag); /* XXX strdup memleak */
-    INSQ(rc, rpc_cb_list);
-    return 0;
-catch:
-    if (rc){
-	if (rc->rc_tag)
-	    free(rc->rc_tag);
-	free(rc);
-    }
-    return -1;
-}
-
-/*! Search netconf backend callbacks and invoke if match
- * This is internal system call, plugin is invoked (does not call) this functino
- * @param[in]  h       clicon handle
- * @param[in]  xe      Sub-tree (under xorig) at child of rpc: <rpc><xn></rpc>.
- * @param[in]  ce      Client (session) entry
- * @param[out] cbret   Return XML, error or OK as cbuf
- *
- * @retval -1   Error
- * @retval  0   OK, not found handler.
- * @retval  1   OK, handler called
- * @see backend_rpc_cb_register
- */
-int
-backend_rpc_cb_call(clicon_handle        h,
-		    cxobj               *xe,
-		    struct client_entry *ce,
-		    cbuf                *cbret)
-{
-    backend_rpc_cb_entry *rc;
-    int                   retval = -1;
-
-    if (rpc_cb_list == NULL)
-	return 0;
-    rc = rpc_cb_list;
-    do {
-	if (strcmp(rc->rc_tag, xml_name(xe)) == 0){
-	    if ((retval = rc->rc_callback(h, xe, ce, cbret, rc->rc_arg)) < 0){
-		clicon_debug(1, "%s Error in: %s", __FUNCTION__, rc->rc_tag);
-		goto done;
-	    }
-	    else{
-		retval = 1; /* handled */
-		goto done;
-	    }
-	}
-	rc = NEXTQ(backend_rpc_cb_entry *, rc);
-    } while (rc != rpc_cb_list);
-    retval = 0;
- done:
-    return retval;
-}
-
-/*! Delete all state data callbacks.
- */
-int
-backend_rpc_cb_delete_all(void)
-{
-    backend_rpc_cb_entry *rc;
-
-    while((rc = rpc_cb_list) != NULL) {
-	DELQ(rc, rpc_cb_list, backend_rpc_cb_entry *);
-	if (rc->rc_tag)
-	    free(rc->rc_tag);
-	free(rc);
-    }
     return 0;
 }
 
+cxobj *
+backend_nacm_list_get(clicon_handle h)
+{
+    struct backend_handle *bh = handle(h);
+    
+    return bh->bh_nacm;
+}

apps/backend/clixon_backend_handle.h

@@ -43,30 +43,6 @@
 /*
  * Types
  */
-struct client_entry;
-typedef int (*backend_rpc_cb)(
-    clicon_handle        h,    /* CLicon handle */
-    cxobj               *xe,   /* Request: <rpc><xn></rpc> */
-    struct client_entry *ce,   /* Client session */
-    cbuf                *cbret,/* Reply eg <rpc-reply>... */
-    void                *arg   /* Argument given at register */
-);  
-typedef backend_rpc_cb backend_netconf_cb_t; /* XXX backward compat */
-
-
-/*! Generic downcall registration. 
- * Enables any function to be called from (cli) frontend
- * to backend. Like an RPC on application-level.
- */
-typedef int (*downcall_cb)(clicon_handle h, uint16_t op, uint16_t len, 
-			   void *arg, uint16_t *retlen, void **retarg);
-
-/*
- * Log for netconf notify function (config_client.c)
- */
-int backend_notify(clicon_handle h, char *stream, int level, char *txt);
-int backend_notify_xml(clicon_handle h, char *stream, int level, cxobj *x);
-
 /* subscription callback */
 typedef	int (*subscription_fn_t)(clicon_handle, void *filter, void *arg);
 
@@ -82,6 +58,14 @@ struct handle_subscription{
     void                *hs_arg;    /* Callback argument */
 };
 
+/*
+ * Prototypes
+ */
+/* Log for netconf notify function (config_client.c) */
+int backend_notify(clicon_handle h, char *stream, int level, char *txt);
+int backend_notify_xml(clicon_handle h, char *stream, int level, cxobj *x);
+
+
 struct handle_subscription *subscription_add(clicon_handle h, char *stream, 
 					     enum format_enum format, char *filter, 
 					     subscription_fn_t fn, void *arg);
@@ -92,14 +76,4 @@ int subscription_delete(clicon_handle h, char *stream,
 struct handle_subscription *subscription_each(clicon_handle h,
 				      struct handle_subscription *hprev);
 
-/* XXX backward compat */
-#define backend_netconf_register_callback(a,b,c,d) backend_rpc_cb_register(a,b,c,d)
-int backend_rpc_cb_register(clicon_handle h, backend_rpc_cb cb,	void *arg, 
-			    char *tag);      
-
-int backend_rpc_cb_call(clicon_handle h, cxobj *xe, struct client_entry *ce, 
-			cbuf *cbret);
-
-int backend_rpc_cb_delete_all(void);
-
 #endif /* _CLIXON_BACKEND_HANDLE_H_ */

apps/backend/clixon_backend_transaction.c

@@ -50,6 +50,7 @@
 #include <sys/types.h>
 #include <regex.h>
 #include <netinet/in.h>
+#include <limits.h>
 
 /* cligen */
 #include <cligen/cligen.h>

apps/backend/clixon_backend_transaction.h

@@ -41,21 +41,12 @@
 #define _CLIXON_BACKEND_TRANSACTION_H_
 
 /*
- * Types
+ * Prototypes
  */
-
-/*! Generic downcall registration. 
- * Enables any function to be called from (cli) frontend
- * to backend. Like an RPC on application-level.
- */
-typedef int (*downcall_cb)(clicon_handle h, uint16_t op, uint16_t len, 
-			   void *arg, uint16_t *retlen, void **retarg);
-
 /* Transaction callback data accessors for client plugins
  * (defined in config_dbdep.c)
  * @see transaction_data_t  internal structure
  */
-typedef void *transaction_data;
 uint64_t transaction_id(transaction_data td);
 void   *transaction_arg(transaction_data td);
 cxobj  *transaction_src(transaction_data td);

apps/cli/Makefile.in

@@ -36,6 +36,7 @@ srcdir  	= @srcdir@
 top_srcdir  	= @top_srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 
 prefix 		= @prefix@
@@ -56,8 +57,6 @@ CLIXON_MINOR    = @CLIXON_VERSION_MINOR@
 
 # Use this clixon lib for linking
 CLIXON_LIB	= libclixon.so.$(CLIXON_MAJOR).$(CLIXON_MINOR)
-# Location of system plugins
-CLIXON_CLI_SYSDIR = $(libdir)/clixon/plugins/cli
 
 # For dependency. A little strange that we rely on it being built in the src dir
 # even though it may exist in $(libdir). But the new version may not have been installed yet.
@@ -68,22 +67,33 @@ LIBS          = -L$(top_srcdir)/lib/src @LIBS@  $(top_srcdir)/lib/src/$(CLIXON_L
 CPPFLAGS  	= @CPPFLAGS@ -fPIC
 INCLUDES	= -I. -I$(top_srcdir)/lib -I$(top_srcdir)/include -I$(top_srcdir) @INCLUDES@
 
+# Name of application
 APPL		= clixon_cli
-SRC		= cli_main.c
-OBJS    	= $(SRC:.c=.o)
 
+# Not accessible from plugin
+APPSRC		= cli_main.c
+APPSRC	       += cli_generate.c
+APPOBJ		= $(APPSRC:.c=.o)
+
+# Accessible from plugin
+LIBSRC	        = cli_common.c
+LIBSRC	       += cli_show.c
+LIBSRC	       += cli_handle.c
+LIBSRC	       += cli_plugin.c
+LIBOBJ		= $(LIBSRC:.c=.o)
+
+# Name of lib
 MYNAME		= clixon_cli
 MYLIBLINK       = lib$(MYNAME)$(SH_SUFFIX)
 MYLIB           = $(MYLIBLINK).$(CLIXON_MAJOR).$(CLIXON_MINOR)
 MYLIBSO         = $(MYLIBLINK).$(CLIXON_MAJOR)
 
-LIBSRC		= cli_plugin.c cli_common.c cli_show.c cli_handle.c cli_generate.c
+
-LIBOBJS		= $(LIBSRC:.c=.o)
 
 all:	 $(MYLIB) $(APPL) test
 
 clean:
-	rm -f $(OBJS) $(LIBOBJS) *.core $(APPL) $(MYLIB) $(MYLIBSO) $(MYLIBLINK)
+	rm -f $(LIBOBJ) $(APPOBJ) *.core $(APPL) $(MYLIB) $(MYLIBSO) $(MYLIBLINK)
 
 distclean: clean
 	rm -f Makefile *~ .depend test test.c
@@ -93,19 +103,19 @@ distclean: clean
 # Also create a libexec/ directory for writeable/temporary files.
 # Put config file in etc/
 install: install-lib $(APPL) 
-	install -d $(DESTDIR)$(bindir) 
+	install -d -m 0755 $(DESTDIR)$(bindir)
-	install $(APPL) $(DESTDIR)$(bindir) 
+	install -m 0755 $(INSTALLFLAGS) $(APPL) $(DESTDIR)$(bindir)
 
 install-lib: $(MYLIB)
-	install -d $(DESTDIR)$(libdir) 
+	install -d -m 0755 $(DESTDIR)$(libdir)
-	install $(MYLIB) $(DESTDIR)$(libdir) 
+	install -m 0644 $(INSTALLFLAGS) $(MYLIB) $(DESTDIR)$(libdir)
 	ln -sf $(MYLIB) $(DESTDIR)$(libdir)/$(MYLIBSO)     # -l:libclixon_cli.so.2
 	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon_cli.so
-	install -d $(DESTDIR)$(libdir)/clixon/plugins/cli
+	install -d -m 0755 $(DESTDIR)$(libdir)/clixon/plugins/cli
 
 install-include:	clixon_cli.h clixon_cli_api.h
-	install -d $(DESTDIR)$(includedir)/clixon
+	install -d -m 0755 $(DESTDIR)$(includedir)/clixon
-	install -m 644 $^ $(DESTDIR)$(includedir)/clixon
+	install -m 0644 $^ $(DESTDIR)$(includedir)/clixon
 
 uninstall:
 	rm -f $(DESTDIR)$(bindir)/$(APPL)
@@ -116,7 +126,7 @@ uninstall:
 .SUFFIXES: .c .o
 
 .c.o:
-	$(CC) $(INCLUDES) $(CPPFLAGS) -D__PROGRAM__=\"$(APPL)\" -DCLIXON_CLI_SYSDIR=\"$(CLIXON_CLI_SYSDIR)\" $(CFLAGS) -c $<
+	$(CC) $(INCLUDES) $(CPPFLAGS) -D__PROGRAM__=\"$(APPL)\" $(CFLAGS) -c $<
 
 # Just link test programs
 test.c	:
@@ -125,14 +135,14 @@ test.c	:
 test: 	test.c $(LIBOBJ) 
 	$(CC) $(INCLUDES) $(LDFLAGS) $< $(LIBOBJ) -L. $(MYLIB) $(LIBS) -o $@
 
-$(APPL): $(OBJS) $(MYLIBLINK) $(LIBDEPS)	
+$(APPL): $(APPOBJ) $(MYLIBLINK) $(LIBDEPS)	
-	$(CC) $(LDFLAGS) $(OBJS) -L. $(MYLIB) $(LIBS) -o $@
+	$(CC) $(LDFLAGS) $(APPOBJ) -L. $(MYLIB) $(LIBS) -o $@
 
-$(MYLIB) : $(LIBOBJS)
+$(MYLIB) : $(LIBOBJ)
 ifeq ($(HOST_VENDOR),apple)
-	$(CC) $(LDFLAGS) -shared -undefined dynamic_lookup -o $@ $(LIBOBJS) $(LIBS)
+	$(CC) $(LDFLAGS) -shared -undefined dynamic_lookup -o $@ $(LIBOBJ) $(LIBS)
 else
-	$(CC) $(LDFLAGS) -shared -Wl,-soname,$(MYLIBSO) -o $@ $(LIBOBJS) $(LIBS) -Wl,-soname=$(MYLIBSO)
+	$(CC) $(LDFLAGS) -shared -Wl,-soname,$(MYLIBSO) -o $@ $(LIBOBJ) $(LIBS) -Wl,-soname=$(MYLIBSO)
 endif
 
 # link-name is needed for application linking, eg for clixon_cli and clixon_config
@@ -144,7 +154,7 @@ TAGS:
 	find . -name '*.[chyl]' -print | etags -
 
 depend:
-	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(SRC) $(APPSRC) > .depend
+	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(LIBSRC) $(APPSRC) > .depend
 
 #include .depend
 

apps/cli/cli_common.c

@@ -236,7 +236,7 @@ cli_dbxml(clicon_handle       h,
     if ((xtop = xml_new("config", NULL, NULL)) == NULL)
 	goto done;
     xbot = xtop;
-    if (api_path && api_path2xml(api_path, yspec, xtop, 0, &xbot, &y) < 0)
+    if (api_path && api_path2xml(api_path, yspec, xtop, YC_DATANODE, &xbot, &y) < 0)
 	goto done;
     if ((xa = xml_new("operation", xbot, NULL)) == NULL)
 	goto done;
@@ -829,10 +829,19 @@ save_config_file(clicon_handle h,
     filename = cv_string_get(cv);
     if (clicon_rpc_get_config(h, dbstr,"/", &xt) < 0)
 	goto done;
+    if (xt == NULL){
+	clicon_err(OE_CFG, 0, "get config: empty tree"); /* Shouldnt happen */
+	goto done;
+    }
     if ((xerr = xpath_first(xt, "/rpc-error")) != NULL){
 	clicon_rpc_generate_error("Get configuration", xerr);
 	goto done;
     }
+    /* get-config returns a <data> tree. Save as <config> tree so it can be used
+     * as data-store.
+     */
+    if (xml_name_set(xt, "config") < 0)
+	goto done;
     if ((f = fopen(filename, "w")) == NULL){
 	clicon_err(OE_CFG, errno, "Creating file %s", filename);
 	goto done;

apps/cli/cli_generate.c

@@ -238,7 +238,6 @@ yang2cli_var_sub(clicon_handle h,
 		goto done;
 	    }
 	}
-
 	else{ /* Cligen does not have 'max' keyword in range so need to find actual
 		 max value of type if yang range expression is 0..max 
 	      */

apps/cli/cli_handle.c

@@ -172,12 +172,22 @@ cli_parse_file(clicon_handle h,
 }
 
 int
-cli_susp_hook(clicon_handle h, cli_susphook_t *fn)
+cli_susp_hook(clicon_handle     h,
+	      cligen_susp_cb_t *fn)
 {
     cligen_handle ch = cligen(h);
 
     /* This assume first arg of fn can be treated as void* */
-    return cligen_susp_hook(ch, (cligen_susp_cb_t*)fn); 
+    return cligen_susp_hook(ch, fn); 
+}
+int
+cli_interrupt_hook(clicon_handle       h,
+		   cligen_interrupt_cb_t *fn)
+{
+    cligen_handle ch = cligen(h);
+
+    /* This assume first arg of fn can be treated as void* */
+    return cligen_interrupt_hook(ch, fn); 
 }
 
 char *

apps/cli/cli_handle.h

@@ -39,7 +39,7 @@
 
 /*
  * Prototypes 
- * Internal prototypes. For exported functions see clicon_cli_api.h
+ * Internal prototypes. For exported functions see clixon_cli_api.h
  */
 int cli_parse_file(clicon_handle h,
 		   FILE *f,
@@ -47,7 +47,9 @@ int cli_parse_file(clicon_handle h,
 		   parse_tree *pt,
 		   cvec *globals);
 
-int cli_susp_hook(clicon_handle h, cli_susphook_t *fn);
+int cli_susp_hook(clicon_handle h, cligen_susp_cb_t *fn);
+
+int cli_interrupt_hook(clicon_handle h, cligen_interrupt_cb_t *fn);
 
 char *cli_nomatch(clicon_handle h);
 
@@ -56,8 +58,8 @@ int cli_prompt_set(clicon_handle h, char *prompt);
 int cli_logsyntax_set(clicon_handle h, int status);
 
 /* Internal functions for handling cli groups */
-
 cli_syntax_t *cli_syntax(clicon_handle h);
+
 int cli_syntax_set(clicon_handle h, cli_syntax_t *stx);
 
 #endif  /* _CLI_HANDLE_H_ */

apps/cli/cli_main.c

@@ -53,6 +53,7 @@
 #include <sys/stat.h>
 #include <sys/param.h>
 #include <netinet/in.h>
+#include <pwd.h>
 #include <assert.h>
 #include <libgen.h>
 
@@ -243,17 +244,24 @@ main(int argc, char **argv)
     char        *restarg = NULL; /* what remains after options */
     int          dump_configfile_xml = 0;
     yang_spec   *yspec;
+    struct passwd *pw;
     
     /* Defaults */
+    once = 0;
+
     /* In the startup, logs to stderr & debug flag set later */
     clicon_log_init(__PROGRAM__, LOG_INFO, logdst); 
     /* Initiate CLICON handle */
     if ((h = cli_handle_init()) == NULL)
 	goto done;
-
+    /* Set username to clicon handle. Use in all communication to backend */
-    if (cli_plugin_init(h) != 0) 
+    if ((pw = getpwuid(getuid())) == NULL){
+	clicon_err(OE_UNIX, errno, "getpwuid");
 	goto done;
-    once = 0;
+    }
+    if (clicon_username_set(h, pw->pw_name) < 0)
+	goto done;
+
     cligen_comment_set(cli_cligen(h), '#'); /* Default to handle #! clicon_cli scripts */
 
     /*
@@ -460,7 +468,7 @@ main(int argc, char **argv)
      */
     tmp = *(argv-1);
     *(argv-1) = argv0;
-    cli_plugin_start(h, argc+1, argv-1);
+    clixon_plugin_start(h, argc+1, argv-1);
     *(argv-1) = tmp;
 
     cligen_line_scrolling_set(cli_cligen(h), clicon_option_int(h,"CLICON_CLI_LINESCROLLING"));

apps/cli/cli_plugin.c

@@ -66,15 +66,6 @@
 #include "cli_plugin.h"
 #include "cli_handle.h"
 
-
-/*! Name of master plugin functions
- * More in clicon_plugin.h
- * @note not really used consider documenting or remove
- */
-#define PLUGIN_PROMPT_HOOK   "plugin_prompt_hook"
-#define PLUGIN_PARSE_HOOK    "plugin_parse_hook"
-#define PLUGIN_SUSP_HOOK     "plugin_susp_hook"
-
 /*
  *
  * CLI PLUGIN INTERFACE, INTERNAL SECTION
@@ -84,7 +75,9 @@
 /*! Find syntax mode named 'mode'. Create if specified
  */
 static cli_syntaxmode_t *
-syntax_mode_find(cli_syntax_t *stx, const char *mode, int create)
+syntax_mode_find(cli_syntax_t *stx,
+		 const char   *mode,
+		 int           create)
 {
     cli_syntaxmode_t *m;
 
@@ -113,34 +106,20 @@ syntax_mode_find(cli_syntax_t *stx, const char *mode, int create)
     return m;
 }
 
-/*! Find plugin by name
- */
-static struct cli_plugin *
-plugin_find_cli(cli_syntax_t *stx, char *plgnam)
-{
-    struct cli_plugin *p;
-    
-    if ((p = stx->stx_plugins) != NULL)
-      do {
-	if (strcmp (p->cp_name, plgnam) == 0)
-	  return p;
-	p = NEXTQ(struct cli_plugin *, p);
-      } while (p && p != stx->stx_plugins);
-
-    return NULL;
-}
-
 /*! Generate parse tree for syntax mode 
+ * @param[in]   h     Clicon handle
+ * @param[in]   m     Syntax mode struct
  */
 static int
-gen_parse_tree(clicon_handle h, cli_syntaxmode_t *m)
+gen_parse_tree(clicon_handle     h,
+	       cli_syntaxmode_t *m)
 {
     cligen_tree_add(cli_cligen(h), m->csm_name, m->csm_pt);
     return 0;
 }
 
-
 /*! Append syntax
+ * @param[in]     h       Clicon handle
  */
 static int
 syntax_append(clicon_handle h,
@@ -159,27 +138,18 @@ syntax_append(clicon_handle h,
     return 0;
 }
 
-/*! Unload all plugins in a group
+/*! Remove all cligen syntax modes
+ * @param[in]     h       Clicon handle
  */
 static int
 cli_syntax_unload(clicon_handle h)
 {
     cli_syntax_t            *stx = cli_syntax(h);
-    struct cli_plugin       *p;
     cli_syntaxmode_t        *m;
 
     if (stx == NULL)
 	return 0;
 
-    while (stx->stx_nplugins > 0) {
-	p = stx->stx_plugins;
-	plugin_unload(h, p->cp_handle);
-	clicon_debug(1, "DEBUG: Plugin '%s' unloaded.", p->cp_name);
-	DELQ(p, stx->stx_plugins, struct cli_plugin *);
-	if (p)
-	    free(p);
-	stx->stx_nplugins--;
-    }
     while (stx->stx_nmodes > 0) {
 	m = stx->stx_modes;
 	DELQ(m, stx->stx_modes, cli_syntaxmode_t *);
@@ -239,34 +209,6 @@ clixon_str2fn(char  *name,
    return NULL; 
 }
 
-/*! Load a dynamic plugin object and call it's init-function
- * Note 'file' may be destructively modified
- * @retval plugin-handle should be freed after use
- */
-static plghndl_t 
-cli_plugin_load(clicon_handle h, 
-		char         *file, 
-		int           dlflags)
-{
-    char              *name;
-    plghndl_t         handle = NULL;
-    struct cli_plugin *cp = NULL;
-
-    if ((handle = plugin_load(h, file, dlflags)) == NULL)
-	goto quit;
-    if ((cp = malloc(sizeof (struct cli_plugin))) == NULL) {
-	perror("malloc");
-	goto quit;
-    }
-    memset (cp, 0, sizeof(*cp));
-    name = basename(file);
-    snprintf(cp->cp_name, sizeof(cp->cp_name), "%.*s", (int)strlen(name)-3, name);
-    cp->cp_handle = handle;
-
-quit:
-    return cp;
-}
-
 /*! Append to syntax mode from file
  * @param[in]  h         Clixon handle
  * @param[in]  filename	 Name of file where syntax is specified (in syntax-group dir)
@@ -288,7 +230,7 @@ cli_load_syntax(clicon_handle h,
     char     **vec = NULL;
     int        i, nvec;
     char      *plgnam;
-    struct cli_plugin *p;
+    clixon_plugin *cp;
 
     if (dir)
 	snprintf(filepath, MAXPATHLEN-1, "%s/%s", dir, filename);
@@ -318,8 +260,8 @@ cli_load_syntax(clicon_handle h,
     mode = cvec_find_str(cvv, "CLICON_MODE");
 
     if (plgnam != NULL) { /* Find plugin for callback resolving */
-	if ((p = plugin_find_cli (cli_syntax(h), plgnam)) != NULL)
+	if ((cp = clixon_plugin_find(h, plgnam)) != NULL)
-	    handle = p->cp_handle;
+	    handle = cp->cp_handle;
 	if (handle == NULL){
 	    clicon_err(OE_PLUGIN, 0, "CLICON_PLUGIN set to '%s' in %s but plugin %s.so not found in %s\n", 
 		       plgnam, filename, plgnam, 
@@ -327,7 +269,6 @@ cli_load_syntax(clicon_handle h,
 	    goto done;
 	}
     }
-
     /* Resolve callback names to function pointers. */
     if (cligen_callbackv_str2fn(pt, (cgv_str2fn_t*)clixon_str2fn, handle) < 0){     
 	clicon_err(OE_PLUGIN, 0, "Mismatch between CLIgen file '%s' and CLI plugin file '%s'. Some possible errors:\n\t1. A function given in the CLIgen file does not exist in the plugin (ie link error)\n\t2. The CLIgen spec does not point to the correct plugin .so file (CLICON_PLUGIN=\"%s\" is wrong)", 
@@ -345,7 +286,10 @@ cli_load_syntax(clicon_handle h,
     if ((vec = clicon_strsep(mode, ":", &nvec)) == NULL) 
 	goto done;
     for (i = 0; i < nvec; i++) {
-	if (syntax_append(h, cli_syntax(h), vec[i], pt) < 0) { 
+	if (syntax_append(h,
+			  cli_syntax(h),
+			  vec[i],
+			  pt) < 0) { 
 	    goto done;
 	}
 	if (prompt)
@@ -363,77 +307,7 @@ done:
     return retval;
 }
 
-/*! Load plugins within a directory
+/*! Load a syntax group. Includes both CLI plugin and CLIgen spec syntax files.
- */
-static int
-cli_plugin_load_dir(clicon_handle h,
-		    char         *dir,
-		    cli_syntax_t *stx)
-{
-    int                i;
-    int	               ndp;
-    struct dirent     *dp = NULL;
-    char              *master_plugin;
-    char               master[MAXPATHLEN];
-    char               filename[MAXPATHLEN];
-    struct cli_plugin *cp;
-    struct stat        st;
-    int                retval = -1;
-
-
-    /* Format master plugin path */
-    if ((master_plugin = clicon_master_plugin(h)) == NULL){
-	clicon_err(OE_PLUGIN, 0, "clicon_master_plugin option not set");
-	goto quit;
-    }
-    snprintf(master, MAXPATHLEN-1, "%s.so", master_plugin);
-
-    /* Get plugin objects names from plugin directory */
-    ndp = clicon_file_dirent(dir, &dp, "(.so)$", S_IFREG);
-    if (ndp < 0)
-        goto quit;
-
-    /* Load master plugin first */
-    snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, master);
-    if (stat(filename, &st) == 0) {
-	clicon_debug(1, "DEBUG: Loading master plugin '%s'", master);
-	cp = cli_plugin_load(h, filename, RTLD_NOW|RTLD_GLOBAL);
-	if (cp == NULL)
-	    goto quit;
-	/* Look up certain call-backs in master plugin */
-	stx->stx_prompt_hook = 
-	    dlsym(cp->cp_handle, PLUGIN_PROMPT_HOOK);
-	stx->stx_parse_hook =
-	    dlsym(cp->cp_handle, PLUGIN_PARSE_HOOK);
-	stx->stx_susp_hook =
-	    dlsym(cp->cp_handle, PLUGIN_SUSP_HOOK);
-	INSQ(cp, stx->stx_plugins);
-	stx->stx_nplugins++;
-    }
-
-    /* Load the rest */
-    for (i = 0; i < ndp; i++) {
-	if (strcmp (dp[i].d_name, master) == 0)
-	    continue; /* Skip master now */
-	snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, dp[i].d_name);
-	clicon_debug(1, "DEBUG: Loading plugin '%s'", dp[i].d_name);
-
-	if ((cp = cli_plugin_load (h, filename, RTLD_NOW)) == NULL)
-	    goto quit;
-	INSQ(cp, stx->stx_plugins);
-	stx->stx_nplugins++;
-    }
-
-    retval = 0;
-
- quit:
-    if (dp)
-	free(dp);
-    return retval;
-}
-
-
-/*! Load a syntax group.
  * @param[in]     h       Clicon handle
  */
 int
@@ -448,6 +322,9 @@ cli_syntax_load (clicon_handle h)
     struct dirent     *dp = NULL;
     cli_syntax_t      *stx;
     cli_syntaxmode_t  *m;
+    cligen_susp_cb_t  *fns = NULL;
+    cligen_interrupt_cb_t *fni = NULL;
+    clixon_plugin     *cp;
 
     /* Syntax already loaded.  XXX should we re-load?? */
     if ((stx = cli_syntax(h)) != NULL)
@@ -461,59 +338,62 @@ cli_syntax_load (clicon_handle h)
     /* Allocate plugin group object */
     if ((stx = malloc(sizeof(*stx))) == NULL) {
 	clicon_err(OE_UNIX, errno, "malloc");
-	goto quit;
+	goto done;
     }
     memset (stx, 0, sizeof (*stx));	/* Zero out all */
 
     cli_syntax_set(h, stx);
 
-    /* First load CLICON system plugins. CLIXON_CLI_SYSDIR is defined
+    /* Load cli plugins */
-       in Makefile*/
+    if (plugin_dir &&
-    if (cli_plugin_load_dir(h, CLIXON_CLI_SYSDIR, stx) < 0)
+	clixon_plugins_load(h, CLIXON_PLUGIN_INIT, plugin_dir, NULL)< 0)
-        goto quit;
+	goto done;
-    
-    /* Then load application plugins */
-    if (plugin_dir && cli_plugin_load_dir(h, plugin_dir, stx) < 0)
-        goto quit;
-    
     if (clispec_file){
 	if (cli_load_syntax(h, clispec_file, NULL) < 0)
-	    goto quit;
+	    goto done;
     }
     if (clispec_dir){
 	/* load syntaxfiles */
 	if ((ndp = clicon_file_dirent(clispec_dir, &dp, "(.cli)$", S_IFREG)) < 0)
-	    goto quit;
+	    goto done;
 	/* Load the rest */
 	for (i = 0; i < ndp; i++) {
 	    clicon_debug(1, "DEBUG: Loading syntax '%.*s'", 
 			 (int)strlen(dp[i].d_name)-4, dp[i].d_name);
 	    if (cli_load_syntax(h, dp[i].d_name, clispec_dir) < 0)
-		goto quit;
+		goto done;
 	}
     }
     /* Did we successfully load any syntax modes? */
     if (stx->stx_nmodes <= 0) {
 	retval = 0;
-	goto quit;
+	goto done;
     }	
     /* Parse syntax tree for all modes */
     m = stx->stx_modes;
     do {
 	if (gen_parse_tree(h, m) != 0)
-	    goto quit;
+	    goto done;
 	m = NEXTQ(cli_syntaxmode_t *, m);
     } while (m && m != stx->stx_modes);
 
-
+    /* Set susp and interrupt callbacks into  CLIgen */
-    /* Set callbacks into  CLIgen */
+    cp = NULL;
-    cli_susp_hook(h, cli_syntax(h)->stx_susp_hook);
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
+	if (fns==NULL && (fns = cp->cp_api.ca_suspend) != NULL)
+	    if (cli_susp_hook(h, fns) < 0)
+		goto done;
+	if (fni==NULL && (fni = cp->cp_api.ca_interrupt) != NULL)
+	    if (cli_susp_hook(h, fns) < 0)
+		goto done;
+    }
 
     /* All good. We can now proudly return a new group */
     retval = 0;
 
-quit:
+done:
     if (retval != 0) {
+	clixon_plugin_exit(h);
 	cli_syntax_unload(h);
 	cli_syntax_set(h, NULL);
     }
@@ -522,34 +402,15 @@ quit:
     return retval;
 }
 
-/*! Call plugin_start() in all plugins
+/*! Remove syntax modes and remove syntax
- */
+ * @param[in]     h       Clicon handle
-int
-cli_plugin_start(clicon_handle h, int argc, char **argv)
-{
-    struct cli_plugin *p;
-    cli_syntax_t *stx;
-    plgstart_t *startfun;
-// XXX    int (*startfun)(clicon_handle, int, char **);
-    
-    stx = cli_syntax(h);
-
-    if ((p = stx->stx_plugins) != NULL)
-	do {
-	    startfun = dlsym(p->cp_handle, PLUGIN_START);
-	    if (dlerror() == NULL)
-		startfun(h, argc, argv);
-	    p = NEXTQ(struct cli_plugin *, p);
-	} while (p && p != stx->stx_plugins);
-    
-    return 0;
-}
-
-/*
  */
 int
 cli_plugin_finish(clicon_handle h)
 {
+    /* Remove all CLI plugins */
+    clixon_plugin_exit(h);
+    /* Remove all cligen syntax modes */
     cli_syntax_unload(h);
     cli_syntax_set(h, NULL);
     return 0;
@@ -558,6 +419,7 @@ cli_plugin_finish(clicon_handle h)
 /*! Help function to print a meaningful error string. 
  * Sometimes the libraries specify an error string, if so print that.
  * Otherwise just print 'command error'.
+ * @param[in]  f   File handler to write error to.
  */
 int 
 cli_handler_err(FILE *f)
@@ -679,15 +541,6 @@ clicon_parse(clicon_handle h,
 	    goto done;
 	case CG_NOMATCH: /* no match */
 	    smode = NULL;
-	    if (stx->stx_parse_hook) {
-		/* Try to find a match in upper  modes, a'la IOS. */
-		if ((modename = stx->stx_parse_hook(h, cmd, modename)) != NULL)  {
-		    if ((smode = syntax_mode_find(stx, modename, 0)) != NULL)
-			continue;
-		    else
-			cli_output(f, "Can't find syntax mode '%s'\n", modename);
-		}
-	    }
 	    /*	    clicon_err(OE_CFG, 0, "CLI syntax error: \"%s\": %s", 
 		    cmd, cli_nomatch(h));*/
 	    cli_output(f, "CLI syntax error: \"%s\": %s\n", 
@@ -718,6 +571,7 @@ done:
 }
 
 /*! Read command from CLIgen's cliread() using current syntax mode.
+ * @param[in] h       Clicon handle
  * @retval    string  char* buffer containing CLIgen command
  * @retval    NULL    Fatal error
  */
@@ -728,12 +582,19 @@ clicon_cliread(clicon_handle h)
     char             *pfmt = NULL;
     cli_syntaxmode_t *mode;
     cli_syntax_t     *stx;
+    cli_prompthook_t *fn;
+    clixon_plugin     *cp;
     
     stx = cli_syntax(h);
     mode = stx->stx_active_mode;
-
+    /* Get prompt from plugin callback? */
-    if (stx->stx_prompt_hook)
+    cp = NULL;
-	pfmt = stx->stx_prompt_hook(h, mode->csm_name);
+    while ((cp = clixon_plugin_each(h, cp)) != NULL) {
+	if ((fn = cp->cp_api.ca_prompt) == NULL)
+	    continue;
+	pfmt = fn(h, mode->csm_name);
+	break;
+    }
     if (clicon_quiet_mode(h))
 	cli_prompt_set(h, "");
     else
@@ -745,42 +606,6 @@ clicon_cliread(clicon_handle h)
     return ret;
 }
 
-/*
- * cli_find_plugin
- * Find a plugin by name and return the dlsym handl
- * Used by libclicon code to find callback funcctions in plugins.
- */
-static void *
-cli_find_plugin(clicon_handle h, char *plugin)
-{
-    struct cli_plugin *p;
-    
-    p = plugin_find_cli(cli_syntax(h), plugin);
-    if (p)
-	return p->cp_handle;
-    
-    return NULL;
-}
-
-
-/*! Initialize plugin code (not the plugins themselves)
- */
-int
-cli_plugin_init(clicon_handle h)
-{
-    find_plugin_t *fp = cli_find_plugin;
-    clicon_hash_t *data = clicon_data(h);
-
-    /* Register CLICON_FIND_PLUGIN in data hash */
-    if (hash_add(data, "CLICON_FIND_PLUGIN", &fp, sizeof(fp)) == NULL) {
-	clicon_err(OE_UNIX, errno, "failed to register CLICON_FIND_PLUGIN");
-	return -1;
-    }
-	
-    return 0;
-}
-
-
 /*
  *
  * CLI PLUGIN INTERFACE, PUBLIC SECTION
@@ -788,11 +613,12 @@ cli_plugin_init(clicon_handle h)
  */
 
 
-/*
+/*! Set syntax mode mode for existing current plugin group.
- * Set syntax mode mode for existing current plugin group.
+ * @param[in]     h       Clicon handle
  */
 int
-cli_set_syntax_mode(clicon_handle h, const char *name)
+cli_set_syntax_mode(clicon_handle h,
+		    const char   *name)
 {
     cli_syntaxmode_t *mode;
     
@@ -803,8 +629,8 @@ cli_set_syntax_mode(clicon_handle h, const char *name)
     return 1;
 }
 
-/*
+/*! Get syntax mode name
- * Get syntax mode name
+ * @param[in]     h       Clicon handle
  */
 char *
 cli_syntax_mode(clicon_handle h)
@@ -816,15 +642,15 @@ cli_syntax_mode(clicon_handle h)
     return csm->csm_name;
 }
 
-
+/*! Callback from cli_set_prompt(). Set prompt format for syntax mode
-/*
+ * @param[in]  h       Clicon handle
- * Callback from cli_set_prompt(). Set prompt format for syntax mode
+ * @param[in]  name    Name of syntax mode 
- * Arguments:
+ * @param[in]  prompt  Prompt format
- *   name	: Name of syntax mode 
- *   prompt	: Prompt format
  */
 int
-cli_set_prompt(clicon_handle h, const char *name, const char *prompt)
+cli_set_prompt(clicon_handle h,
+	       const char   *name,
+	       const char   *prompt)
 {
     cli_syntaxmode_t *m;
 
@@ -836,9 +662,14 @@ cli_set_prompt(clicon_handle h, const char *name, const char *prompt)
 }
 
 /*! Format prompt 
+ * @param[out]    prompt  Prompt string to be written
+ * @param[in]     plen    Length of prompt string
+ * @param[in]     fmt     Stdarg fmt string
  */
 static int
-prompt_fmt (char *prompt, size_t plen, char *fmt, ...)
+prompt_fmt (char  *prompt,
+	    size_t plen,
+	    char  *fmt, ...)
 {
   va_list ap;
   char   *s = fmt;
@@ -854,7 +685,6 @@ prompt_fmt (char *prompt, size_t plen, char *fmt, ...)
   }
   
   /* Start with empty string */
-  cprintf(cb, "");
   while(*s) {
       if (*s == '%' && *++s) {
 	  switch(*s) {
@@ -881,7 +711,6 @@ prompt_fmt (char *prompt, size_t plen, char *fmt, ...)
 	  cprintf(cb, "%c", *s);
       s++;
   }
-  
 done:
   if (cb)
       fmt = cbuf_get(cb);
@@ -894,6 +723,7 @@ done:
 }
 
 /*! Return a formatted prompt string
+ * @param[in]     fmt      Format string
  */
 char *
 cli_prompt(char *fmt)
@@ -906,55 +736,3 @@ cli_prompt(char *fmt)
     return prompt;
 }
 
-/*! Find a cli plugin based on name and resolve a function pointer in it.
- * Callback from clicon_dbvars_parse() 
- * Find a cli plugin based on name if given and use dlsym to resolve a 
- * function pointer in it.
- * Call the resolved function to get the cgv populated
- */
-int
-clicon_valcb(void *arg, cvec *vars, cg_var *cgv, char *fname, cg_var *funcarg)
-{
-    char *func;
-    char *plgnam = NULL;
-    void *handle;
-    struct cli_plugin *p;
-    cli_valcb_t *cb;
-    clicon_handle h = (clicon_handle)arg;
-
-    /* Make copy */
-    if ((fname = strdup(fname)) == NULL) {
-	clicon_err(OE_UNIX, errno, "strdup");
-	return -1;
-    }
-
-    /* Extract plugin name if any */
-    if ((func = strstr(fname, "::")) != NULL) {
-	*func = '\0';
-	func += 2;
-	plgnam = fname;
-    }
-    else
-	func = fname;
-    
-    /* If we have specified a plugin name, find the handle to be used
-     * with dlsym()
-     */
-    handle = NULL;
-    if (plgnam && (p = plugin_find_cli(cli_syntax(h), plgnam)))
-	handle = p->cp_handle;
-    
-    /* Look up function pointer */
-    if ((cb = dlsym(handle, func)) == NULL) {
-	clicon_err(OE_UNIX, errno, "unable to find %s()", func);
-	free(fname);
-	return -1;
-    }
-    free(fname);
-
-    if (cb(vars, cgv, funcarg) < 0)
-	return -1;
-
-    return 0;
-}
-

apps/cli/cli_plugin.h

@@ -43,19 +43,7 @@
 /* clicon generic callback pointer */
 typedef void (clicon_callback_t)(clicon_handle h);
 
-/* clicon_set value callback */
+/* List of syntax modes */
-typedef int (cli_valcb_t)(cvec *vars, cg_var *cgv, cg_var *arg);
-
-/* specific to cli. For common see clicon_plugin.h */
-/* Hook to get prompt format before each getline */
-typedef char *(cli_prompthook_t)(clicon_handle, char *mode);
-
-/* Ctrl-Z hook from getline() */   
-typedef int (cli_susphook_t)(clicon_handle, char *, int, int *);
-
-/* CLIgen parse failure hook. Retry other mode? */
-typedef char *(cli_parsehook_t)(clicon_handle, char *, char *);
-
 typedef struct {
     qelem_t    csm_qelem;                               /* List header */
     char       csm_name[256];                      /* Syntax mode name */
@@ -65,32 +53,16 @@ typedef struct {
 
 } cli_syntaxmode_t;
 
-/* A plugin list object */
+/* Plugin group object. Just a single object, not list. part of cli_handle */
-struct cli_plugin {
-    qelem_t cp_qelem;                                     /* List header */
-    char cp_name[256];                                    /* Plugin name */
-    void *cp_handle;                            /* Dynamic object handle */
-};
-
-/* Plugin group object */
 typedef struct  {
-    int stx_nplugins;                                 /* Number of plugins */
-    struct cli_plugin *stx_plugins;                     /* List of plugins */
     int stx_nmodes;                              /* Number of syntax modes */
     cli_syntaxmode_t *stx_active_mode;       /* Current active syntax mode */
     cli_syntaxmode_t *stx_modes;                   /* List of syntax modes */
-    cli_prompthook_t *stx_prompt_hook;                      /* Prompt hook */
-    cli_parsehook_t *stx_parse_hook;                    /* Parse mode hook */
-    cli_susphook_t *stx_susp_hook;           /* Ctrl-Z hook from getline() */
 } cli_syntax_t;
 
 
 void *clixon_str2fn(char *name, void *handle, char **error);
 
-int cli_plugin_start(clicon_handle, int argc, char **argv);
-
-int cli_plugin_init(clicon_handle h);
-
 int clicon_eval(clicon_handle h, char *cmd, cg_obj *match_obj, cvec *vr);
 
 int clicon_parse(clicon_handle h, char *cmd, char **mode, int *result);

apps/cli/cli_show.c

@@ -120,6 +120,7 @@ expand_dbvar(void   *h,
     yang_stmt       *ypath;
     cxobj           *xcur;
     char            *xpathcur;
+    char            *reason = NULL;
     
     if (argv == NULL || cvec_len(argv) != 2){
 	clicon_err(OE_PLUGIN, 0, "%s: requires arguments: <db> <xmlkeyfmt>",
@@ -171,8 +172,10 @@ expand_dbvar(void   *h,
     /* This is primarily to get "y", 
      * xpath2xml would have worked!!
      */
-    if (api_path && api_path2xml(api_path, yspec, xtop, 0, &xbot, &y) < 0)
+    if (api_path && api_path2xml(api_path, yspec, xtop, YC_DATANODE, &xbot, &y) < 0)
 	goto done;
+    if (y==NULL)
+	goto ok;
     /* Special case for leafref. Detect leafref via Yang-type, 
      * Get Yang path element, tentatively add the new syntax to the whole
      * tree and apply the path to that.
@@ -188,8 +191,12 @@ expand_dbvar(void   *h,
 		goto done;
 	    }
 	    xpathcur = ypath->ys_argument;
-	    if (xml_merge(xt, xtop, yspec) < 0) /* Merge xtop into xt */
+	    if (xml_merge(xt, xtop, yspec, &reason) < 0) /* Merge xtop into xt */
 		goto done;
+	    if (reason){
+		cli_output(stderr, "%s\n", reason);
+		goto done;
+	    }	    
 	    if ((xcur = xpath_first(xt, xpath)) == NULL){
 		clicon_err(OE_DB, 0, "xpath %s should return merged content", xpath);
 		goto done;
@@ -236,8 +243,11 @@ expand_dbvar(void   *h,
 	/* XXX RFC3986 decode */
 	cvec_add_string(commands, NULL, bodystr);
     }
+ ok:
     retval = 0;
   done:
+    if (reason)
+	free(reason);
     if (api_path)
 	free(api_path);
     if (xvec)
@@ -506,12 +516,12 @@ cli_show_config(clicon_handle h,
 	    xml2txt(stdout, xc, 0); /* tree-formed text */
 	break;
     case FORMAT_CLI:
-	xc = NULL; /* Dont print xt itself */
+	/* get CLI generatade mode: VARS|ALL */
-	while ((xc = xml_child_each(xt, xc, -1)) != NULL){
 	if ((gt = clicon_cli_genmodel_type(h)) == GT_ERR)
 	    goto done;
+	xc = NULL; /* Dont print xt itself */
+	while ((xc = xml_child_each(xt, xc, -1)) != NULL)
 	    xml2cli(stdout, xc, NULL, gt); /* cli syntax */
-	}
 	break;
     case FORMAT_NETCONF:
 	fprintf(stdout, "<rpc><edit-config><target><candidate/></target><config>\n");
@@ -587,7 +597,14 @@ done:
 	xml_free(xt);
     return retval;
 }
+
 int show_confv_xpath(clicon_handle h, cvec *vars, cvec *argv)
 {
     return show_conf_xpath(h, vars, argv);
 }
+
+int cli_show_version(clicon_handle h, cvec *vars, cvec *argv)
+{
+    cli_output(stdout, "%s\n", CLIXON_VERSION_STRING);
+    return 0;
+}

apps/netconf/Makefile.in

@@ -35,6 +35,7 @@ srcdir  	= @srcdir@
 top_srcdir  	= @top_srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 
 prefix 		= @prefix@
@@ -44,6 +45,7 @@ bindir 		= @bindir@
 libdir		= @libdir@
 mandir		= @mandir@
 libexecdir	= @libexecdir@
+wwwdir  	= /www-data
 localstatedir	= @localstatedir@
 sysconfdir	= @sysconfdir@
 includedir	= @includedir@
@@ -64,22 +66,32 @@ LIBS          = -L$(top_srcdir)/lib/src @LIBS@ $(top_srcdir)/lib/src/$(CLIXON_LI
 CPPFLAGS  	= @CPPFLAGS@ -fPIC
 INCLUDES	= -I. -I$(top_srcdir)/lib/src -I$(top_srcdir)/lib -I$(top_srcdir)/include -I$(top_srcdir) @INCLUDES@
 
+# Name of application
 APPL	 = clixon_netconf
-SRC     = netconf_main.c
-OBJS    = $(SRC:.c=.o)
 
+# Not accessible from plugin
+APPSRC   = netconf_main.c
+APPSRC  += netconf_hello.c
+APPSRC  += netconf_rpc.c 
+APPSRC  += netconf_filter.c
+APPOBJ   = $(APPSRC:.c=.o)
+
+# Accessible from plugin
+LIBSRC   = netconf_lib.c 
+
+
+LIBOBJ   = $(LIBSRC:.c=.o)
+
+# Name of lib
 MYNAME		= clixon_netconf
 MYLIBLINK       = lib$(MYNAME)$(SH_SUFFIX)
 MYLIB           = $(MYLIBLINK).$(CLIXON_MAJOR).$(CLIXON_MINOR)
 MYLIBSO         = $(MYLIBLINK).$(CLIXON_MAJOR)
 
-LIBSRC     = netconf_hello.c netconf_rpc.c netconf_filter.c netconf_lib.c netconf_plugin.c
-LIBOBJS    = $(LIBSRC:.c=.o)
-
 all:	 $(MYLIB) $(APPL)
 
 clean: 
-	rm -f $(OBJS) $(LIBOBJS) *.core $(APPL) $(MYLIB) $(MYLIBSO) $(MYLIBLINK)
+	rm -f $(APPL) $(APPOBJ) $(LIBOBJ) *.core  $(MYLIB) $(MYLIBSO) $(MYLIBLINK)
 
 distclean: clean
 	rm -f Makefile *~ .depend
@@ -89,18 +101,18 @@ distclean: clean
 # Also create a libexec/ directory for writeable/temporary files.
 # Put config file in etc/
 install:	install-lib $(APPL)
-	install -d $(DESTDIR)$(bindir)
+	install -d -m 0755 $(DESTDIR)$(bindir)
-	install $(APPL) $(DESTDIR)$(bindir)
+	install -m 0755 $(INSTALLFLAGS) $(APPL) $(DESTDIR)$(bindir)
 
 install-lib: $(MYLIB)
-	install -d $(DESTDIR)$(libdir) 
+	install -d -m 0755 $(DESTDIR)$(libdir)
-	install $(MYLIB) $(DESTDIR)$(libdir) 
+	install -m 0644 $(INSTALLFLAGS) $(MYLIB) $(DESTDIR)$(libdir)
 	ln -sf $(MYLIB) $(DESTDIR)$(libdir)/$(MYLIBSO)     # -l:libclixon_netconf.so.2
 	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon_netconf.so
 
 install-include:	clixon_netconf.h
-	install -d $(DESTDIR)$(includedir)/clixon
+	install -d -m 0755 $(DESTDIR)$(includedir)/clixon
-	install -m 644 $^ $(DESTDIR)$(includedir)/clixon
+	install -m 0644 $^ $(DESTDIR)$(includedir)/clixon
 
 uninstall:
 	rm -f $(DESTDIR)$(bindir)/$(APPL)
@@ -113,14 +125,14 @@ uninstall:
 .c.o:
 	$(CC) $(INCLUDES) $(CPPFLAGS) -D__PROGRAM__=\"$(APPL)\" $(CFLAGS) -c $<
 
-$(APPL) : $(OBJS) $(MYLIBLINK) $(LIBDEPS)
+$(APPL) : $(APPOBJ) $(MYLIBLINK) $(LIBDEPS)
-	$(CC) $(LDFLAGS) $(OBJS) -L. $(MYLIB) $(LIBS) -o $@
+	$(CC) $(LDFLAGS) $(APPOBJ) -L. $(MYLIB) $(LIBS) -o $@
 
-$(MYLIB) : $(LIBOBJS)
+$(MYLIB) : $(LIBOBJ)
 ifeq ($(HOST_VENDOR),apple)
-	$(CC) $(LDFLAGS) -shared -undefined dynamic_lookup -o $@ $(LIBOBJS) $(LIBS)
+	$(CC) $(LDFLAGS) -shared -undefined dynamic_lookup -o $@ $(LIBOBJ) $(LIBS)
 else
-	$(CC) $(LDFLAGS) -shared -Wl,-soname,$(MYLIBSO) -o $@ $(LIBOBJS) $(LIBS) -Wl,-soname=$(MYLIBSO)
+	$(CC) $(LDFLAGS) -shared -Wl,-soname,$(MYLIBSO) -o $@ $(LIBOBJ) $(LIBS) -Wl,-soname=$(MYLIBSO)
 endif
 
 # link-name is needed for application linking, eg for clixon_cli and clixon_config
@@ -132,7 +144,7 @@ TAGS:
 	find . -name '*.[chyl]' -print | etags -
 
 depend:
-	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(SRC) $(APPSRC) > .depend
+	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(LIBSRC) $(APPSRC) > .depend
 
 #include .depend
 

apps/netconf/clixon_netconf.h

@@ -39,27 +39,12 @@
 #ifndef _CLIXON_NETCONF_H_
 #define _CLIXON_NETCONF_H_
 
-/*
- * Types
- */
-typedef int (*netconf_cb_t)(
-    clicon_handle h, 
-    cxobj  *xn,              /* Request: <rpc><xn></rpc> */
-    cxobj **xret,            /* Return xml tree, eg <rpc-reply>... */
-    void   *arg              /* Argument given at netconf_register_callback() */
-    );  
-
 /*
  * Prototypes
  * (Duplicated. Also in netconf_*.h)
  */
 int netconf_output(int s, cbuf *xf, char *msg);
 
-int netconf_register_callback(clicon_handle h,
-			      netconf_cb_t cb,   /* Callback called */
-			      void *arg,       /* Arg to send to callback */
-			      char *tag);      /* Xml tag when callback is made */
-
 int netconf_xpath(cxobj *xsearch,
 		  cxobj *xfilter, 
 		   cbuf *xf, cbuf *xf_err, 

apps/netconf/netconf_hello.c

@@ -122,7 +122,7 @@ netconf_create_hello(cbuf *xf,            /* msg buffer */
    cprintf(xf, "<capability>urn:ietf:params:netconf:capability:notification:1.0</capability>\n");
    cprintf(xf, "<capability>urn:ietf:params:netconf:capability:startup:1.0</capability>\n");
     cprintf(xf, "</capabilities>");
-    cprintf(xf, "<session-id>%lu</session-id>", 42+session_id);
+    cprintf(xf, "<session-id>%lu</session-id>", (long unsigned int)42+session_id);
     cprintf(xf, "</hello>");
     add_postamble(xf);
     return retval;

apps/netconf/netconf_lib.c

@@ -71,45 +71,48 @@
 enum transport_type    transport = NETCONF_SSH; /* XXX Remove SOAP support */
 int cc_closed = 0; /* XXX Please remove (or at least hide in handle) this global variable */
 
+/*! Add netconf xml postamble of message. I.e, xml after the body of the message.
+ * @param[in]  cb  Netconf packet (cligen buffer)
+ */
 int
-add_preamble(cbuf *xf)
+add_preamble(cbuf *cb)
 {
     if (transport == NETCONF_SOAP)
-	cprintf(xf, "\n<soapenv:Envelope\n xmlns:soapenv=\"http://www.w3.org/2003/05/soap-envelope\">\n"
+	cprintf(cb, "\n<soapenv:Envelope\n xmlns:soapenv=\"http://www.w3.org/2003/05/soap-envelope\">\n"
 	"<soapenv:Body>");
     return 0;
 }
 
-/* 
+/*! Add netconf xml postamble of message. I.e, xml after the body of the message.
- * add_postamble
- * add netconf xml postamble of message. That is, xml after the body of the message.
  * for soap this is the envelope stuff, for ssh this is ]]>]]>
+ * @param[in]  cb  Netconf packet (cligen buffer)
  */
 int
-add_postamble(cbuf *xf)
+add_postamble(cbuf *cb)
 {
     switch (transport){
     case NETCONF_SSH:
-	cprintf(xf, "]]>]]>");     /* Add RFC4742 end-of-message marker */
+	cprintf(cb, "]]>]]>");     /* Add RFC4742 end-of-message marker */
 	break;
     case NETCONF_SOAP:
-	cprintf(xf, "\n</soapenv:Body>" "</soapenv:Envelope>");
+	cprintf(cb, "\n</soapenv:Body>" "</soapenv:Envelope>");
 	break;
     }
     return 0;
 }
 
-/* 
+/*! Add error_preamble
- * add_error_preamble
  * compared to regular messages (see add_preamble), error message differ in some
  * protocols (eg soap) by adding a longer and deeper header.
+ * @param[in]  cb  Netconf packet (cligen buffer)
  */
 int
-add_error_preamble(cbuf *xf, char *reason)
+add_error_preamble(cbuf *cb,
+		   char *reason)
 {
     switch (transport){
     case NETCONF_SOAP:
-	cprintf(xf, "<soapenv:Envelope xmlns:soapenv=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:xml=\"http://www.w3.org/XML/1998/namespace\">"
+	cprintf(cb, "<soapenv:Envelope xmlns:soapenv=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:xml=\"http://www.w3.org/XML/1998/namespace\">"
 		"<soapenv:Body>"
 		"<soapenv:Fault>"
 		"<soapenv:Code>"
@@ -121,26 +124,26 @@ add_error_preamble(cbuf *xf, char *reason)
 		"<detail>", reason);
 	break;
     default:
-	if (add_preamble(xf) < 0)
+	if (add_preamble(cb) < 0)
 	    return -1;
 	break;
     }
     return 0;
 }
 
-/* 
+/*! Add error postamble
- * add_error_postamble
  * compared to regular messages (see add_postamble), error message differ in some
  * protocols (eg soap) by adding a longer and deeper header.
+ * @param[in]  cb  Netconf packet (cligen buffer)
  */
 int
-add_error_postamble(cbuf *xf)
+add_error_postamble(cbuf *cb)
 {
     switch (transport){
     case NETCONF_SOAP:
-	cprintf(xf, "</detail>" "</soapenv:Fault>");
+	cprintf(cb, "</detail>" "</soapenv:Fault>");
     default: /* fall through */
-	if (add_postamble(xf) < 0)
+	if (add_postamble(cb) < 0)
 	    return -1;
 	break;
     }
@@ -150,6 +153,8 @@ add_error_postamble(cbuf *xf)
 
 /*! Get "target" attribute, return actual database given candidate or running
  * Caller must do error handling
+ * @param[in]  xn       XML tree
+ * @param[in]  path
  * @retval     dbname   Actual database file name
  */
 char *
@@ -180,11 +185,11 @@ netconf_get_target(cxobj        *xn,
  */
 int 
 netconf_output(int   s, 
-	       cbuf *xf, 
+	       cbuf *cb, 
 	       char *msg)
 {
-    char *buf = cbuf_get(xf);
+    char *buf = cbuf_get(cb);
-    int   len = cbuf_len(xf);
+    int   len = cbuf_len(cb);
     int   retval = -1;
 
     clicon_debug(1, "SEND %s", msg);
@@ -207,3 +212,4 @@ netconf_output(int   s,
   done:
     return retval;
 }
+

apps/netconf/netconf_main.c

@@ -55,6 +55,7 @@
 #include <netinet/in.h>
 #include <arpa/inet.h>
 #include <assert.h>
+#include <pwd.h>
 #include <netinet/in.h>
 #include <libgen.h>
 
@@ -67,7 +68,6 @@
 #include "clixon_netconf.h"
 #include "netconf_lib.h"
 #include "netconf_hello.h"
-#include "netconf_plugin.h"
 #include "netconf_rpc.h"
 
 /* Command line options to be passed to getopt(3) */
@@ -100,13 +100,9 @@ process_incoming_packet(clicon_handle h,
     /* Parse incoming XML message */
     if (xml_parse_string(str, NULL, &xreq) < 0){ 
 	if ((cbret = cbuf_new()) == NULL){
-	    cprintf(cbret, "<rpc-reply><rpc-error>"
+	    if (netconf_operation_failed(cbret, "rpc", "internal error")< 0)
-		"<error-tag>operation-failed</error-tag>"
+		goto done;
-		"<error-type>rpc</error-type>"
+	    netconf_output(1, cbret, "rpc-error");
-		"<error-severity>error</error-severity>"
-		"<error-message>internal error</error-message>"
-		"</rpc-error></rpc-reply>");
-	    netconf_output(1, cb, "rpc-error");
 	}
 	else
 	    clicon_log(LOG_ERR, "%s: cbuf_new", __FUNCTION__);
@@ -205,7 +201,6 @@ netconf_input_cb(int   s,
 	    retval = 0;
 	    goto done;
 	}
-
 	for (i=0; i<len; i++){
 	    if (buf[i] == 0)
 		continue; /* Skip NULL chars (eg from terminals) */
@@ -268,6 +263,8 @@ netconf_terminate(clicon_handle h)
 {
     yang_spec      *yspec;
 
+    clixon_plugin_exit(h);
+    rpc_callback_delete_all();
     clicon_rpc_close_session(h);
     if ((yspec = clicon_dbspec_yang(h)) != NULL)
 	yspec_free(yspec);
@@ -312,6 +309,8 @@ main(int    argc,
     int              quiet = 0;
     clicon_handle    h;
     int              use_syslog;
+    char            *dir;
+    struct passwd   *pw;
     
     /* Defaults */
     use_syslog = 0;
@@ -322,6 +321,14 @@ main(int    argc,
     if ((h = clicon_handle_init()) == NULL)
 	return -1;
 
+    /* Set username to clicon handle. Use in all communication to backend */
+    if ((pw = getpwuid(getuid())) == NULL){
+	clicon_err(OE_UNIX, errno, "getpwuid");
+	goto done;
+    }
+    if (clicon_username_set(h, pw->pw_name) < 0)
+	goto done;
+
     while ((c = getopt(argc, argv, NETCONF_OPTS)) != -1)
 	switch (c) {
 	case 'h' : /* help */
@@ -379,6 +386,8 @@ main(int    argc,
     argc -= optind;
     argv += optind;
 
+
+
     /* Parse yang database spec file */
     if (yang_spec_main(h) == NULL)
 	goto done;
@@ -388,13 +397,14 @@ main(int    argc,
 	goto done;
 
     /* Initialize plugins group */
-    if (netconf_plugin_load(h) < 0)
+    if ((dir = clicon_netconf_dir(h)) != NULL)
+	if (clixon_plugins_load(h, CLIXON_PLUGIN_INIT, dir, NULL) < 0)
 	    goto done;
 
     /* Call start function is all plugins before we go interactive */
     tmp = *(argv-1);
     *(argv-1) = argv0;
-    netconf_plugin_start(h, argc+1, argv-1);
+    clixon_plugin_start(h, argc+1, argv-1);
     *(argv-1) = tmp;
 
     if (!quiet)
@@ -406,7 +416,6 @@ main(int    argc,
     if (event_loop() < 0)
 	goto done;
   done:
-    netconf_plugin_unload(h);
     netconf_terminate(h);
     clicon_log_init(__PROGRAM__, LOG_INFO, 0); /* Log on syslog no stderr */
     clicon_log(LOG_NOTICE, "%s: %u Terminated\n", __PROGRAM__, getpid());

apps/netconf/netconf_plugin.c

@@ -1,237 +0,0 @@
-/*
- *
-  ***** BEGIN LICENSE BLOCK *****
- 
-  Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
-
-  This file is part of CLIXON.
-
-  Licensed under the Apache License, Version 2.0 (the "License");
-  you may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
-
-  Alternatively, the contents of this file may be used under the terms of
-  the GNU General Public License Version 3 or later (the "GPL"),
-  in which case the provisions of the GPL are applicable instead
-  of those above. If you wish to allow use of your version of this file only
-  under the terms of the GPL, and not to allow others to
-  use your version of this file under the terms of Apache License version 2, 
-  indicate your decision by deleting the provisions above and replace them with
-  the  notice and other provisions required by the GPL. If you do not delete
-  the provisions above, a recipient may use your version of this file under
-  the terms of any one of the Apache License version 2 or the GPL.
-
-  ***** END LICENSE BLOCK *****
-
- * 
- * handling netconf plugins
- */
-
-#ifdef HAVE_CONFIG_H
-#include "clixon_config.h" /* generated by config & autoconf */
-#endif
-
-#include <stdio.h>
-#include <string.h>
-#include <stdlib.h>
-#include <unistd.h>
-#include <stdarg.h>
-#include <syslog.h>
-#include <errno.h>
-#include <assert.h>
-#include <dlfcn.h>
-#include <dirent.h>
-#include <grp.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <sys/param.h>
-#include <netinet/in.h>
-
-/* cligen */
-#include <cligen/cligen.h>
-
-/* clicon */
-#include <clixon/clixon.h>
-
-/* clicon netconf*/
-#include "clixon_netconf.h"
-#include "netconf_lib.h"
-#include "netconf_plugin.h"
-
-/* Database dependency description */
-struct netconf_reg {
-    qelem_t 	 nr_qelem;	/* List header */
-    netconf_cb_t nr_callback;	/* Validation/Commit Callback */
-    void	*nr_arg;	/* Application specific argument to cb */
-    char        *nr_tag;	/* Xml tag when matched, callback called */
-};
-typedef struct netconf_reg netconf_reg_t;
-
-static int nplugins = 0;
-static plghndl_t *plugins = NULL;
-static netconf_reg_t *deps = NULL;
-
-/*! Load all plugins you can find in CLICON_NETCONF_DIR
- */
-int 
-netconf_plugin_load(clicon_handle h)
-{
-    int            retval = -1;
-    char          *dir;
-    int            ndp;
-    struct dirent *dp = NULL;
-    int            i;
-    char           filename[MAXPATHLEN];
-    plghndl_t     *handle;
-
-    /* If no DIR defined, then dont load plugins */
-    if ((dir = clicon_netconf_dir(h)) == NULL){
-	retval = 0;
-	goto quit;
-    }
-
-    /* Get plugin objects names from plugin directory */
-    if((ndp = clicon_file_dirent(dir, &dp, "(.so)$", S_IFREG))<0)
-	goto quit;
-
-    /* Load all plugins */
-    for (i = 0; i < ndp; i++) {
-	snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, dp[i].d_name);
-	clicon_debug(1, "DEBUG: Loading plugin '%.*s' ...", 
-		     (int)strlen(filename), filename);
-	if ((handle = plugin_load(h, filename, RTLD_NOW)) == NULL)
-	    goto quit;
-	if ((plugins = realloc(plugins, (nplugins+1) * sizeof (*plugins))) == NULL) {
-	    clicon_err(OE_UNIX, errno, "realloc");
-	    goto quit;
-	}
-	plugins[nplugins++] = handle;
-    }
-    retval = 0;
-quit:
-    if (dp)
-	free(dp);
-    return retval;
-}
-
-/*! Unload all netconf plugins */
-int
-netconf_plugin_unload(clicon_handle h)
-{
-    int i;
-    netconf_reg_t *nr;
-
-    while((nr = deps) != NULL) {
-	DELQ(nr, deps, netconf_reg_t *);
-	if (nr->nr_tag)
-	    free(nr->nr_tag);
-	free(nr);
-    }
-    for (i = 0; i < nplugins; i++) 
-	plugin_unload(h, plugins[i]);
-    if (plugins){
-	free(plugins);
-	plugins = NULL;
-    }
-    nplugins = 0;
-    return 0;
-}
-
-/*! Call plugin_start in all plugins
- */
-int
-netconf_plugin_start(clicon_handle h, int argc, char **argv)
-{
-    int i;
-    plgstart_t *startfn;
-
-    for (i = 0; i < nplugins; i++) {
-	/* Call exit function is it exists */
-	if ((startfn = dlsym(plugins[i], PLUGIN_START)) == NULL)
-	    break;
-	optind = 0;
-	if (startfn(h, argc, argv) < 0) {
-	    clicon_debug(1, "plugin_start() failed\n");
-	    return -1;
-	}
-    }
-    return 0;
-}
-
-
-/*! Register netconf callback
- * Called from plugin to register a callback for a specific netconf XML tag.
- */
-int
-netconf_register_callback(clicon_handle h,
-			  netconf_cb_t cb,      /* Callback called */
-			  void *arg,        /* Arg to send to callback */
-			  char *tag)        /* Xml tag when callback is made */
-{
-    netconf_reg_t *nr;
-
-    if ((nr = malloc(sizeof(netconf_reg_t))) == NULL) {
-	clicon_err(OE_DB, errno, "malloc: %s", strerror(errno));
-	goto catch;
-    }
-    memset (nr, 0, sizeof (*nr));
-    nr->nr_callback = cb;
-    nr->nr_arg  = arg;
-    nr->nr_tag  = strdup(tag); /* strdup */
-    INSQ(nr, deps);
-    return 0;
-catch:
-    if (nr){
-	if (nr->nr_tag)
-	    free(nr->nr_tag);
-	free(nr);
-    }
-    return -1;
-}
-
-/*! See if there is any callback registered for this tag
- *
- * Look for local (client-side) netconf plugins. This feature may no 
- * longer be necessary as generic RPC:s should be handled by backend.
- *
- * @param[in]  h       clicon handle
- * @param[in]  xn      Sub-tree (under xorig) at child of rpc: <rpc><xn></rpc>.
- * @param[out] xret    Return XML, error or OK
- *
- * @retval -1   Error
- * @retval  0   OK, not found handler.
- * @retval  1   OK, handler called
- */
-int
-netconf_plugin_callbacks(clicon_handle h,
-			 cxobj        *xn, 
-			 cxobj       **xret)
-{
-    int            retval = -1;
-    netconf_reg_t *nreg;
-  
-    if (deps != NULL){
-	nreg = deps;
-	do {
-	    if (strcmp(nreg->nr_tag, xml_name(xn)) == 0){
-		if ((retval = nreg->nr_callback(h, xn, xret, nreg->nr_arg)) < 0)
-		    goto done;
-		retval = 1; /* handled */
-		goto done;
-	    }
-	    nreg = NEXTQ(netconf_reg_t *, nreg);
-	} while (nreg != deps);
-    }
-    retval = 0;
- done:
-    return retval;
-}
-    

apps/netconf/netconf_rpc.c

@@ -65,7 +65,6 @@
 #include "clixon_netconf.h"
 #include "netconf_lib.h"
 #include "netconf_filter.h"
-#include "netconf_plugin.h"
 #include "netconf_rpc.h"
 
 /*
@@ -850,8 +849,10 @@ netconf_create_subscription(clicon_handle h,
     return retval;
 }
 
-/*! See if there is any callback registered for this tag
+/*! See if there is any application defined RPC for this tag
  *
+ * This may either be local client-side or backend. If backend send as netconf 
+ * RPC. 
  * @param[in]  h       clicon handle
  * @param[in]  xn      Sub-tree (under xorig) at child of rpc: <rpc><xn></rpc>.
  * @param[out] xret    Return XML, error or OK
@@ -872,6 +873,8 @@ netconf_application_rpc(clicon_handle h,
     yang_stmt     *youtput;
     cxobj         *xoutput;
     cbuf          *cb = NULL;
+    cbuf          *cbret = NULL;
+    int            ret;
 
     /* First check system / netconf RPC:s */
     if ((cb = cbuf_new()) == NULL){
@@ -890,11 +893,11 @@ netconf_application_rpc(clicon_handle h,
 	//    else
 	//	cprintf(cb, "/%s", xml_name(xn)); /* XXX not accepdted by below */
     /* Find yang rpc statement, return yang rpc statement if found */
-    if (yang_abs_schema_nodeid(yspec, cbuf_get(cb), &yrpc) < 0)
+	if (yang_abs_schema_nodeid(yspec, cbuf_get(cb), Y_RPC, &yrpc) < 0)
 	goto done;
-
     /* Check if found */
     if (yrpc != NULL){
+	/* 1. Check xn arguments with input statement. */
 	if ((yinput = yang_find((yang_node*)yrpc, Y_INPUT, NULL)) != NULL){
 	    xml_spec_set(xn, yinput); /* needed for xml_spec_populate */
 	    if (xml_apply(xn, CX_ELMNT, xml_spec_populate, yinput) < 0)
@@ -905,11 +908,18 @@ netconf_application_rpc(clicon_handle h,
 	    if (xml_yang_validate_add(xn, NULL) < 0)
 		goto done;
 	}
-	/* 
+	if ((cbret = cbuf_new()) == NULL){
-	 * 1. Check xn arguments with input statement.
+	    clicon_err(OE_UNIX, 0, "cbuf_new");
-	 * 2. Send to backend as clicon_msg-encode()
+	    goto done;
-	 * 3. In backend to similar but there call actual backend
+	}
-	 */
+	/* Look for local (client-side) netconf plugins. */
+	if ((ret = rpc_callback_call(h, xn, cbret, NULL)) < 0)
+	    goto done;
+	if (ret == 1){ /* Handled locally */
+	    if (xml_parse_string(cbuf_get(cbret), NULL, xret) < 0)
+		goto done;
+	}
+	else /* Send to backend */
 	    if (clicon_rpc_netconf_xml(h, xml_parent(xn), xret, NULL) < 0)
 		goto done;
 	/* Sanity check of outgoing XML */
@@ -931,11 +941,12 @@ netconf_application_rpc(clicon_handle h,
  done:
     if (cb)
 	cbuf_free(cb);
+    if (cbret)
+	cbuf_free(cbret);
     return retval;
 }
 
 
-
 /*! The central netconf rpc dispatcher. Look at first tag and dispach to sub-functions.
  * Call plugin handler if tag not found. If not handled by any handler, return
  * error.
@@ -1018,12 +1029,8 @@ netconf_rpc_dispatch(clicon_handle h,
 	}
 	/* Others */
 	else {
-	    /* Look for local (client-side) netconf plugins. This feature may no 
+	    /* Look for application-defined RPC. This may either be local
-	     * longer be necessary as generic RPC:s should be handled by backend.
+	       client-side or backend. If backend send as netconf RPC. */
-	     */
-	    if ((retval = netconf_plugin_callbacks(h, xe, xret)) < 0)
-		goto done;
-	    if (retval == 0)
 	    if ((retval = netconf_application_rpc(h, xe, xret)) < 0)
 		goto done;
 	    if (retval == 0){ /* not handled by callback */

apps/restconf/Makefile.in

@@ -35,15 +35,21 @@ srcdir  	= @srcdir@
 top_srcdir  	= @top_srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 
 prefix 		= @prefix@
+datarootdir	= @datarootdir@
 exec_prefix 	= @exec_prefix@
 bindir 		= @bindir@
-wwwdir  	= /www-data
+libdir		= @libdir@
+mandir		= @mandir@
 libexecdir	= @libexecdir@
+wwwdir  	= /www-data
 localstatedir	= @localstatedir@
 sysconfdir	= @sysconfdir@
+includedir	= @includedir@
+HOST_VENDOR     = @host_vendor@
 
 SH_SUFFIX	= @SH_SUFFIX@
 CLIXON_MAJOR    = @CLIXON_VERSION_MAJOR@
@@ -57,23 +63,32 @@ LIBDEPS		= $(top_srcdir)/lib/src/$(CLIXON_LIB)
 
 LIBS    	= -L$(top_srcdir)/lib/src @LIBS@ -l:$(CLIXON_LIB)
 
-CPPFLAGS  	= @CPPFLAGS@
+CPPFLAGS  	= @CPPFLAGS@ -fPIC
 
 INCLUDES	= -I. -I$(top_srcdir)/lib/src -I$(top_srcdir)/lib -I$(top_srcdir)/include -I$(top_srcdir) @INCLUDES@
 
-SRC     = restconf_lib.c
+# Name of application
-SRC    += restconf_methods.c
-
-OBJS    = $(SRC:.c=.o)
-
-APPSRC  = restconf_main.c
-APPOBJ  = $(APPSRC:.c=.o)
 APPL	  = clixon_restconf
 
-all:	 $(APPL)
+# Not accessible from plugin
+APPSRC    = restconf_main.c
+APPSRC   += restconf_methods.c
+APPOBJ    = $(APPSRC:.c=.o)
+
+# Accessible from plugin
+LIBSRC    = restconf_lib.c
+LIBOBJ    = $(LIBSRC:.c=.o)
+
+# Name of lib
+MYNAME	  = clixon_restconf
+MYLIBLINK = lib$(MYNAME)$(SH_SUFFIX)
+MYLIB     = $(MYLIBLINK).$(CLIXON_MAJOR).$(CLIXON_MINOR)
+MYLIBSO   = $(MYLIBLINK).$(CLIXON_MAJOR)
+
+all:	 $(MYLIB) $(APPL)
 
 clean:
-	rm -f $(OBJS) *.core $(APPL) $(APPOBJ)
+	rm -f $(LIBOBJ) *.core $(APPL) $(APPOBJ) $(MYLIB) $(MYLIBSO) $(MYLIBLINK)
 
 distclean: clean
 	rm -f Makefile *~ .depend
@@ -82,14 +97,23 @@ distclean: clean
 # Put other executables in libexec/
 # Also create a libexec/ directory for writeable/temporary files.
 # Put config file in etc/
-install:	$(APPL)
+install:	install-lib $(APPL)
-	install -d $(DESTDIR)$(wwwdir)
+	install -d -m 0755 $(DESTDIR)$(wwwdir)
-	install $(APPL) $(DESTDIR)$(wwwdir)
+	install -m 0755 $(INSTALLFLAGS) $(APPL) $(DESTDIR)$(wwwdir)
 
-install-include:	
+install-lib: $(MYLIB)
+	install -d -m 0755 $(DESTDIR)$(libdir)
+	install -m 0644 $(INSTALLFLAGS) $(MYLIB) $(DESTDIR)$(libdir)
+	ln -sf $(MYLIB) $(DESTDIR)$(libdir)/$(MYLIBSO)     # -l:libclixon_restconf.so.2
+	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon_restconf.so
+
+install-include:	clixon_restconf.h
+	install -d -m 0755 $(DESTDIR)$(includedir)/clixon
+	install -m 0644 $^ $(DESTDIR)$(includedir)/clixon
 
 uninstall:
 	rm -f $(DESTDIR)$(wwwdir)/$(APPL)
+	rm -f $(DESTDIR)$(libdir)/$(MYLIBLINK)*
 
 .SUFFIXES:
 .SUFFIXES: .c .o
@@ -97,14 +121,24 @@ uninstall:
 .c.o:
 	$(CC) $(INCLUDES) -D__PROGRAM__=\"$(APPL)\" $(CPPFLAGS) $(CFLAGS) -c $<
 
-$(APPL) : $(APPOBJ) $(OBJS) $(LIBDEPS)
+$(APPL) : $(APPOBJ) $(MYLIBLINK) $(LIBDEPS)
-	$(CC) $(LDFLAGS) $(APPOBJ) $(OBJS) $(LIBS) -o $@
+	$(CC) $(LDFLAGS) $(APPOBJ) -L. $(MYLIB) $(LIBS) -o $@
+
+$(MYLIB) : $(LIBOBJ)
+ifeq ($(HOST_VENDOR),apple)
+	$(CC) $(LDFLAGS) -shared -undefined dynamic_lookup -o $@ $(LIBOBJ) $(LIBS)
+else
+	$(CC) $(LDFLAGS) -shared -Wl,-soname,$(MYLIBSO) -o $@ $(LIBOBJ) $(LIBS) -Wl,-soname=$(MYLIBSO)
+endif
+
+# link-name is needed for application linking, eg for clixon_cli and clixon_config
+$(MYLIBLINK) : $(MYLIB)
 
 TAGS:
 	find . -name '*.[chyl]' -print | etags -
 
 depend:
-	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(SRC) $(APPSRC) > .depend
+	$(CC) $(DEPENDFLAGS) @DEFS@ $(INCLUDES) $(CFLAGS) -MM $(LIBSRC) $(APPSRC) > .depend
 
 #include .depend
 

apps/restconf/README.md

@@ -1,16 +1,4 @@
 # Clixon Restconf
-### Features
-
-Clixon restconf is a daemon based on FASTCGI. Instructions are available to
-run with NGINX.
-The implementatation is based on [RFC 8040: RESTCONF Protocol](https://tools.ietf.org/html/rfc8040).
-The following featires are supported:
-- OPTIONS, HEAD, GET, POST, PUT, DELETE
-The following are not implemented
-- PATCH
-- query parameters (section 4.9)
-- notifications (sec 6)
-- schema resource
 
 ### Installation using Nginx
 
@@ -32,7 +20,7 @@ sudo /etc/init.d nginx start
 
 Start clixon restconf daemon
 ```
-olof@vandal> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/routing.xml " -s /bin/sh www-data
+olof@vandal> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data
 ```
 
 Make restconf calls with curl
@@ -68,7 +56,7 @@ curl -sX POST -d '{"interfaces":{"interface":{"name":"eth1","type":"eth","enable
 
 Start the restconf fastcgi program with debug flag:
 ```
-sudo su -c "/www-data/clixon_restconf -Df /usr/local/etc/routing.xml" -s /bin/sh www-data
+sudo su -c "/www-data/clixon_restconf -Df /usr/local/etc/example.xml" -s /bin/sh www-data
 ```
 Look at syslog:
 ```

apps/restconf/clixon_restconf.h

@@ -31,26 +31,33 @@
 
   ***** END LICENSE BLOCK *****
 
- *
+ * The exported interface to plugins. External apps (eg frontend restconf plugins)
- * handling netconf plugins
+ * should only include this file (not the restconf_*.h)
- *****************************************************************************/
-#ifndef _NETCONF_PLUGIN_H_
-#define _NETCONF_PLUGIN_H_
-
-/*
- * Types
  */
 
+#ifndef _CLIXON_RESTCONF_H_
+#define _CLIXON_RESTCONF_H_
 
 /*
- * Prototypes
+ * Prototypes (also in restconf_lib.h)
  */
-int netconf_plugin_load(clicon_handle h);
+int restconf_err2code(char *tag);
+const char *restconf_code2reason(int code);
 
-int netconf_plugin_start(clicon_handle h, int argc, char **argv);
+int badrequest(FCGX_Request *r);
+int unauthorized(FCGX_Request *r);
+int forbidden(FCGX_Request *r);
+int notfound(FCGX_Request *r);
+int conflict(FCGX_Request *r);
+int internal_server_error(FCGX_Request *r);
+int notimplemented(FCGX_Request *r);
 
-int netconf_plugin_unload(clicon_handle h);
+int clicon_debug_xml(int dbglevel, char *str, cxobj *cx);
+int test(FCGX_Request *r, int dbg);
+cbuf *readdata(FCGX_Request *r);
+int get_user_cookie(char *cookiestr, char  *attribute, char **val);
+int api_return_err(clicon_handle h, FCGX_Request *r, cxobj *xerr,
+		   int pretty, int use_xml);
 
-int netconf_plugin_callbacks(clicon_handle h, cxobj *xn, cxobj **xret);
 
-#endif  /* _NETCONF_PLUGIN_H_ */
+#endif /* _CLIXON_RESTCONF_H_ */

apps/restconf/restconf_lib.c

@@ -42,7 +42,6 @@
 #include <fcntl.h>
 #include <ctype.h>
 #include <time.h>
-#include <fcgi_stdio.h>
 #include <signal.h>
 #include <dlfcn.h>
 #include <sys/param.h>
@@ -55,6 +54,8 @@
 /* clicon */
 #include <clixon/clixon.h>
 
+#include <fcgi_stdio.h> /* Need to be after clixon_xml-h due to attribute format */
+
 #include "restconf_lib.h"
 
 /* See RFC 8040 Section 7:  Mapping from NETCONF<error-tag> to Status Code
@@ -211,8 +212,10 @@ notfound(FCGX_Request *r)
     clicon_debug(1, "%s", __FUNCTION__);
     path = FCGX_GetParam("DOCUMENT_URI", r->envp);
     FCGX_FPrintF(r->out, "Status: 404\r\n"); /* 404 not found */
+
     FCGX_FPrintF(r->out, "Content-Type: text/html\r\n\r\n");
     FCGX_FPrintF(r->out, "<h1>Not Found</h1>\n");
+    FCGX_FPrintF(r->out, "Not Found\n");
     FCGX_FPrintF(r->out, "The requested URL %s was not found on this server.\n",
 		 path);
     return 0;
@@ -326,6 +329,7 @@ test(FCGX_Request *r,
     printparam(r, "HTTPS", dbg);
     printparam(r, "HTTP_ACCEPT", dbg);
     printparam(r, "HTTP_CONTENT_TYPE", dbg);
+    printparam(r, "HTTP_AUTHORIZATION", dbg);
 #if 0 /* For debug */
     clicon_debug(1, "All environment vars:");
     {
@@ -356,129 +360,6 @@ readdata(FCGX_Request *r)
     return cb;
 }
 
-
-static int nplugins = 0;
-static plghndl_t *plugins = NULL;
-static plgcredentials_t *_credentials_fn = NULL; /* Credentials callback */
-
-/*! Load all plugins you can find in CLICON_RESTCONF_DIR
- */
-int 
-restconf_plugin_load(clicon_handle h)
-{
-    int            retval = -1;
-    char          *dir;
-    int            ndp;
-    struct dirent *dp = NULL;
-    int            i;
-    plghndl_t     *handle;
-    char           filename[MAXPATHLEN];
-
-    if ((dir = clicon_restconf_dir(h)) == NULL){
-	retval = 0;
-	goto quit;
-    }
-    /* Get plugin objects names from plugin directory */
-    if((ndp = clicon_file_dirent(dir, &dp, "(.so)$", S_IFREG))<0)
-	goto quit;
-
-    /* Load all plugins */
-    for (i = 0; i < ndp; i++) {
-	snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, dp[i].d_name);
-	clicon_debug(1, "DEBUG: Loading plugin '%.*s' ...", 
-		     (int)strlen(filename), filename);
-	if ((handle = plugin_load(h, filename, RTLD_NOW)) == NULL)
-	    goto quit;
-	if ((_credentials_fn    = dlsym(handle, PLUGIN_CREDENTIALS)) == NULL)
-	    clicon_debug(1, "Failed to load %s", PLUGIN_CREDENTIALS); 
-	else
-	    clicon_debug(1, "%s callback loaded", PLUGIN_CREDENTIALS); 
-	if ((plugins = realloc(plugins, (nplugins+1) * sizeof (*plugins))) == NULL) {
-	    clicon_err(OE_UNIX, errno, "realloc");
-	    goto quit;
-	}
-	plugins[nplugins++] = handle;
-    }
-    retval = 0;
-quit:
-    if (dp)
-	free(dp);
-    return retval;
-}
-
-
-/*! Unload all restconf plugins */
-int
-restconf_plugin_unload(clicon_handle h)
-{
-    int i;
-
-    for (i = 0; i < nplugins; i++) 
-	plugin_unload(h, plugins[i]);
-    if (plugins){
-	free(plugins);
-	plugins = NULL;
-    }
-    nplugins = 0;
-    return 0;
-}
-
-/*! Call plugin_start in all plugins
- */
-int
-restconf_plugin_start(clicon_handle h, 
-		      int           argc, 
-		      char        **argv)
-{
-    int i;
-    plgstart_t *startfn;
-
-    for (i = 0; i < nplugins; i++) {
-	/* Call exit function is it exists */
-	if ((startfn = dlsym(plugins[i], PLUGIN_START)) == NULL)
-	    break;
-	optind = 0;
-	if (startfn(h, argc, argv) < 0) {
-	    clicon_debug(1, "plugin_start() failed\n");
-	    return -1;
-	}
-    }
-    return 0;
-}
-
-/*! Run the restconf user-defined credentials callback if present
- * The callback is expected to return the authenticated user, or NULL if not
- * authenticasted.
- * If no callback exists, return user "none"
- * @param[in]  h    Clicon handle
- * @param[in]  r    Fastcgi request handle
- * @param[out] user The authenticated user (or NULL). Malloced, must be freed.
- */
-int 
-restconf_credentials(clicon_handle h,     
-		     FCGX_Request *r,
-		     char        **user)
-{
-    int retval = -1;
-
-    clicon_debug(1, "%s", __FUNCTION__);
-    /* If no authentication callback then allow anything. Is this OK? */
-    if (_credentials_fn == NULL){
-	if ((*user = strdup("none")) == NULL){
-	    clicon_err(OE_XML, errno, "strdup");
-	    goto done;
-	}
-	goto ok;
-    }
-    if (_credentials_fn(h, r, user) < 0) 
-	*user = NULL;
- ok:
-    retval = 0;
- done:
-    clicon_debug(1, "%s retval:%d user:%s", __FUNCTION__, retval, *user);
-    return retval;
-}
-
 /*! Parse a cookie string and return value of cookie attribute
  * @param[in]  cookiestr  cookie string according to rfc6265 (modified)
  * @param[in]  attribute  cookie attribute
@@ -505,3 +386,82 @@ get_user_cookie(char  *cookiestr,
 	cvec_free(cvv);
     return retval;
 }
+
+/*! Return restconf error on get/head request
+ * @param[in]  h      Clixon handle
+ * @param[in]  r      Fastcgi request handle
+ * @param[in]  xerr   XML error message from backend
+ * @param[in]  pretty Set to 1 for pretty-printed xml/json output
+ * @param[in]  use_xml Set to 0 for JSON and 1 for XML
+ */
+int
+api_return_err(clicon_handle h,
+	       FCGX_Request *r,
+	       cxobj        *xerr,
+	       int           pretty,
+	       int           use_xml)
+{
+    int        retval = -1;
+    cbuf      *cb = NULL;
+    cxobj     *xtag;
+    char      *tagstr;
+    int        code;	
+    const char *reason_phrase;
+
+    clicon_debug(1, "%s", __FUNCTION__);
+    if ((cb = cbuf_new()) == NULL)
+	goto done;
+    if ((xtag = xpath_first(xerr, "//error-tag")) == NULL){
+	notfound(r);
+	goto ok;
+    }
+    tagstr = xml_body(xtag);
+    code = restconf_err2code(tagstr);
+    if ((reason_phrase = restconf_code2reason(code)) == NULL)
+	reason_phrase="";
+    if (xml_name_set(xerr, "error") < 0)
+	goto done;
+    if (use_xml){
+	if (clicon_xml2cbuf(cb, xerr, 2, pretty) < 0)
+	    goto done;
+    }
+    else
+	if (xml2json_cbuf(cb, xerr, pretty) < 0)
+	    goto done;
+    FCGX_FPrintF(r->out, "Status: %d %s\r\n", code, reason_phrase);
+    FCGX_FPrintF(r->out, "Content-Type: application/yang-data+%s\r\n\r\n",
+		 use_xml?"xml":"json");
+    if (use_xml){
+	if (pretty){
+	    FCGX_FPrintF(r->out, "    <errors xmlns=\"urn:ietf:params:xml:ns:yang:ietf-restconf\">\n", cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "%s", cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "    </errors>\r\n");
+	}
+	else {
+	    FCGX_FPrintF(r->out, "<errors xmlns=\"urn:ietf:params:xml:ns:yang:ietf-restconf\">", cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "%s", cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "</errors>\r\n");
+	}
+    }
+    else{
+	if (pretty){
+	    FCGX_FPrintF(r->out, "{\n");
+	    FCGX_FPrintF(r->out, "  \"ietf-restconf:errors\" : %s\n",
+			 cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "}\r\n");
+	}
+	else{
+	    FCGX_FPrintF(r->out, "{");
+	    FCGX_FPrintF(r->out, "\"ietf-restconf:errors\" : ");
+	    FCGX_FPrintF(r->out, "%s", cbuf_get(cb));
+	    FCGX_FPrintF(r->out, "}\r\n");
+	}
+    }
+ ok:
+    retval = 0;
+ done:
+    clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
+    if (cb)
+        cbuf_free(cb);
+    return retval;
+}

apps/restconf/restconf_lib.h

@@ -37,11 +37,7 @@
 #define _RESTCONF_LIB_H_
 
 /*
- * Constants
+ * Prototypes (also in clixon_restconf.h)
- */
-
-/*
- * Prototypes
  */
 int restconf_err2code(char *tag);
 const char *restconf_code2reason(int code);
@@ -57,12 +53,8 @@ int notimplemented(FCGX_Request *r);
 int clicon_debug_xml(int dbglevel, char *str, cxobj *cx);
 int test(FCGX_Request *r, int dbg);
 cbuf *readdata(FCGX_Request *r);
-
-int restconf_plugin_load(clicon_handle h);
-int restconf_plugin_start(clicon_handle h, int argc, char **argv);
-int restconf_plugin_unload(clicon_handle h);
-int restconf_credentials(clicon_handle h, FCGX_Request *r, char **user);
 int get_user_cookie(char *cookiestr, char  *attribute, char **val);
-
+int api_return_err(clicon_handle h, FCGX_Request *r, cxobj *xerr,
+		   int pretty, int use_xml);
 
 #endif /* _RESTCONF_LIB_H_ */

apps/restconf/restconf_main.c

@@ -39,7 +39,7 @@
  * sudo apt-get install libfcgi-dev
  * gcc -o fastcgi fastcgi.c -lfcgi
 
- * sudo su -c "/www-data/clixon_restconf -Df /usr/local/etc/routing.xml " -s /bin/sh www-data
+ * sudo su -c "/www-data/clixon_restconf -Df /usr/local/etc/example.xml " -s /bin/sh www-data
 
  * This is the interface:
  * api/data/profile=<name>/metric=<name> PUT data:enable=<flag>
@@ -54,7 +54,8 @@
 #include <syslog.h>
 #include <fcntl.h>
 #include <time.h>
-#include <fcgi_stdio.h>
+#include <limits.h>
+
 #include <signal.h>
 #include <sys/time.h>
 #include <sys/wait.h>
@@ -66,6 +67,8 @@
 /* clicon */
 #include <clixon/clixon.h>
 
+#include <fcgi_stdio.h> /* Need to be after clixon_xml-h due to attribute format */
+
 /* restconf */
 #include "restconf_lib.h"
 #include "restconf_methods.h"
@@ -89,7 +92,10 @@
  * @param[in]  pcvec  Vector of path ie DOCUMENT_URI element
  * @param[in]  pi     Offset, where to start pcvec
  * @param[in]  qvec   Vector of query string (QUERY_STRING)
- * @param[in]  dvec   Stream input data
+ * @param[in]  dvec   Stream input daat
+ * @param[in]  pretty Set to 1 for pretty-printed xml/json output
+ * @param[in]  use_xml Set to 0 for JSON and 1 for XML
+ * @param[in]  parse_xml Set to 0 for JSON and 1 for XML for input data
  */
 static int
 api_data(clicon_handle h,
@@ -98,7 +104,10 @@ api_data(clicon_handle h,
 	 cvec         *pcvec, 
 	 int           pi,
 	 cvec         *qvec, 
-	 char         *data)
+	 char         *data,
+	 int           pretty,
+	 int           use_xml,
+	 int           parse_xml)
 {
     int     retval = -1;
     char   *request_method;
@@ -109,17 +118,17 @@ api_data(clicon_handle h,
     if (strcmp(request_method, "OPTIONS")==0)
 	retval = api_data_options(h, r);
     else if (strcmp(request_method, "HEAD")==0)
-	retval = api_data_head(h, r, pcvec, pi, qvec);
+	retval = api_data_head(h, r, pcvec, pi, qvec, pretty, use_xml);
     else if (strcmp(request_method, "GET")==0)
-	retval = api_data_get(h, r, pcvec, pi, qvec);
+	retval = api_data_get(h, r, pcvec, pi, qvec, pretty, use_xml);
     else if (strcmp(request_method, "POST")==0)
-	retval = api_data_post(h, r, api_path, pcvec, pi, qvec, data);
+	retval = api_data_post(h, r, api_path, pcvec, pi, qvec, data, pretty, use_xml, parse_xml);
     else if (strcmp(request_method, "PUT")==0)
-	retval = api_data_put(h, r, api_path, pcvec, pi, qvec, data);
+	retval = api_data_put(h, r, api_path, pcvec, pi, qvec, data, pretty, use_xml, parse_xml);
     else if (strcmp(request_method, "PATCH")==0)
 	retval = api_data_patch(h, r, api_path, pcvec, pi, qvec, data);
     else if (strcmp(request_method, "DELETE")==0)
-	retval = api_data_delete(h, r, api_path, pi);
+	retval = api_data_delete(h, r, api_path, pi, pretty, use_xml);
     else
 	retval = notfound(r);
     clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
@@ -129,11 +138,12 @@ api_data(clicon_handle h,
 /*! Operations REST method, POST
  * @param[in]  h      CLIXON handle
  * @param[in]  r      Fastcgi request handle
- * @param[in]  api_path According to restconf (Sec 3.5.1.1 in [draft])
+ * @param[in]  path   According to restconf (Sec 3.5.1.1 in [draft])
  * @param[in]  pcvec  Vector of path ie DOCUMENT_URI element
  * @param[in]  pi     Offset, where to start pcvec
  * @param[in]  qvec   Vector of query string (QUERY_STRING)
- * @param[in]  dvec   Stream input data
+ * @param[in]  data   Stream input data
+ * @param[in]  parse_xml Set to 0 for JSON and 1 for XML for input data
  */
 static int
 api_operations(clicon_handle h,
@@ -143,7 +153,9 @@ api_operations(clicon_handle h,
 	       int           pi,
 	       cvec         *qvec, 
 	       char         *data,
-	       char         *username)
+	       int           pretty,
+	       int           use_xml,
+	       int           parse_xml)
 {
     int     retval = -1;
     char   *request_method;
@@ -152,16 +164,43 @@ api_operations(clicon_handle h,
     request_method = FCGX_GetParam("REQUEST_METHOD", r->envp);
     clicon_debug(1, "%s method:%s", __FUNCTION__, request_method);
     if (strcmp(request_method, "GET")==0)
-	retval = api_operation_get(h, r, path, pcvec, pi, qvec, data);
+	retval = api_operations_get(h, r, path, pcvec, pi, qvec, data, pretty, use_xml);
     else if (strcmp(request_method, "POST")==0)
-	retval = api_operation_post(h, r, path, pcvec, pi, qvec, data, username);
+	retval = api_operations_post(h, r, path, pcvec, pi, qvec, data,
+				     pretty, use_xml, parse_xml);
     else
 	retval = notfound(r);
     return retval;
 }
 
+/*! Determine the root of the RESTCONF API
+ * @param[in]  h        Clicon handle
+ * @param[in]  r        Fastcgi request handle
+ * @note Hardcoded to "/restconf"
+ * Return see RFC8040 3.1 and RFC7320
+ * In line with the best practices defined by [RFC7320], RESTCONF
+ * enables deployments to specify where the RESTCONF API is located.
+ */
+static int
+api_well_known(clicon_handle h,
+	       FCGX_Request *r)
+{
+    clicon_debug(1, "%s", __FUNCTION__);
+    FCGX_FPrintF(r->out, "Content-Type: application/xrd+xml\r\n");
+    FCGX_FPrintF(r->out, "\r\n");
+    FCGX_SetExitStatus(200, r->out); /* OK */
+    FCGX_FPrintF(r->out, "<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>\n");
+    FCGX_FPrintF(r->out, "   <Link rel='restconf' href='/restconf'/>\n");
+    FCGX_FPrintF(r->out, "</XRD>\r\n");
+
+    return 0;
+}
+
 /*! Retrieve the Top-Level API Resource
+ * @param[in]  h        Clicon handle
+ * @param[in]  r        Fastcgi request handle
  * @note Only returns null for operations and data,...
+ * See RFC8040 3.3
  */
 static int
 api_root(clicon_handle h,
@@ -179,9 +218,11 @@ api_root(clicon_handle h,
     media_accept = FCGX_GetParam("HTTP_ACCEPT", r->envp);
     if (strcmp(media_accept, "application/yang-data+xml")==0)
 	use_xml++;
+    clicon_debug(1, "%s use-xml:%d media-accept:%s", __FUNCTION__, use_xml, media_accept);
     FCGX_SetExitStatus(200, r->out); /* OK */
     FCGX_FPrintF(r->out, "Content-Type: application/yang-data+%s\r\n", use_xml?"xml":"json");
     FCGX_FPrintF(r->out, "\r\n");
+
     if (xml_parse_string("<restconf><data></data><operations></operations><yang-library-version>2016-06-21</yang-library-version></restconf>", NULL, &xt) < 0)
 	goto done;
     if ((cb = cbuf_new()) == NULL){
@@ -235,7 +276,6 @@ api_yang_library_version(clicon_handle h,
     if (xml_rootchild(xt, 0, &xt) < 0)
 	goto done;
     if ((cb = cbuf_new()) == NULL){
-	clicon_err(OE_XML, errno, "cbuf_new");
 	goto done;
     }
     if (use_xml){
@@ -247,8 +287,8 @@ api_yang_library_version(clicon_handle h,
 	    goto done;
     }
     clicon_debug(1, "%s cb%s", __FUNCTION__, cbuf_get(cb));
-    FCGX_FPrintF(r->out, "%s\r\n", cb?cbuf_get(cb):"");
+    FCGX_FPrintF(r->out, "%s\n", cb?cbuf_get(cb):"");
-    FCGX_FPrintF(r->out, "\r\n\r\n");
+    FCGX_FPrintF(r->out, "\n\n");
     retval = 0;
  done:
     if (cb)
@@ -276,17 +316,34 @@ api_restconf(clicon_handle h,
     cvec  *pcvec = NULL; /* for rest api */
     cbuf  *cb = NULL;
     char  *data;
-    char  *username = NULL;
+    int    authenticated = 0;
+    char  *media_accept;
+    char  *media_content_type;
+    int    pretty;
+    int    parse_xml = 0; /* By default expect and parse JSON */
+    int    use_xml = 0;   /* By default use JSON */
+    cbuf  *cbret = NULL;
+    cxobj *xret = NULL;
+    cxobj *xerr;
 
     clicon_debug(1, "%s", __FUNCTION__);
     path = FCGX_GetParam("REQUEST_URI", r->envp);
     query = FCGX_GetParam("QUERY_STRING", r->envp);
+    pretty = clicon_option_bool(h, "CLICON_RESTCONF_PRETTY");
+    /* get xml/json in put and output */
+    media_accept = FCGX_GetParam("HTTP_ACCEPT", r->envp);
+    if (media_accept && strcmp(media_accept, "application/yang-data+xml")==0)
+	use_xml++;
+    media_content_type = FCGX_GetParam("HTTP_CONTENT_TYPE", r->envp);
+    if (media_content_type &&
+	strcmp(media_content_type, "application/yang-data+xml")==0)
+	parse_xml++;
     if ((pvec = clicon_strsep(path, "/", &pn)) == NULL)
 	goto done;
     /* Sanity check of path. Should be /restconf/ */
     if (pn < 2){
-	retval = notfound(r);
+	notfound(r);
-	goto done;
+	goto ok;
     }
     if (strlen(pvec[0]) != 0){
 	retval = notfound(r);
@@ -296,6 +353,8 @@ api_restconf(clicon_handle h,
 	retval = notfound(r);
 	goto done;
     }
+    test(r, 1);
+
     if (pn == 2){
 	retval = api_root(h, r);
 	goto done;
@@ -304,7 +363,7 @@ api_restconf(clicon_handle h,
 	retval = notfound(r);
 	goto done;
     }
-    clicon_debug(1, "method=%s", method);
+    clicon_debug(1, "%s: method=%s", __FUNCTION__, method);
     if (str2cvec(query, '&', '=', &qvec) < 0)
       goto done;
     if (str2cvec(path, '/', '=', &pcvec) < 0) /* rest url eg /album=ricky/foo */
@@ -313,33 +372,45 @@ api_restconf(clicon_handle h,
     if ((cb = readdata(r)) == NULL)
 	goto done;
     data = cbuf_get(cb);
-    clicon_debug(1, "DATA=%s", data);
+    clicon_debug(1, "%s DATA=%s", __FUNCTION__, data);
+
     if (str2cvec(data, '&', '=', &dvec) < 0)
       goto done;
-
-    test(r, 1);
     /* If present, check credentials. See "plugin_credentials" in plugin  
      * See RFC 8040 section 2.5
      */
-    if (restconf_credentials(h, r, &username) < 0)
+    if ((authenticated = clixon_plugin_auth(h, r)) < 0)
+	goto done;
+    clicon_debug(1, "%s auth:%d %s", __FUNCTION__, authenticated, clicon_username_get(h));
+
+    /* If set but no user, we set a dummy user */
+    if (authenticated){
+	if (clicon_username_get(h) == NULL)
+	    clicon_username_set(h, "none");
+    }
+    else{
+	if (netconf_access_denied_xml(&xret, "protocol", "The requested URL was unauthorized") < 0)
+	    goto done;
+	if ((xerr = xpath_first(xret, "//rpc-error")) != NULL){
+	    if (api_return_err(h, r, xerr, pretty, use_xml) < 0)
 		goto done;
-    clicon_debug(1, "%s username:%s", __FUNCTION__, username);
-    clicon_debug(1, "%s credentials ok username:%s (should be non-NULL)",
-		 __FUNCTION__, username);
-    if (username == NULL){
-	unauthorized(r);
 	    goto ok;
 	}
+	goto ok;
+    }
+    clicon_debug(1, "%s auth2:%d %s", __FUNCTION__, authenticated, clicon_username_get(h));
     if (strcmp(method, "yang-library-version")==0){
 	if (api_yang_library_version(h, r) < 0)
 	    goto done;
     }
     else if (strcmp(method, "data") == 0){ /* restconf, skip /api/data */
-	if (api_data(h, r, path, pcvec, 2, qvec, data) < 0)
+	if (api_data(h, r, path, pcvec, 2, qvec, data,
+		     pretty, use_xml, parse_xml) < 0)
 	    goto done;
     }
     else if (strcmp(method, "operations") == 0){ /* rpc */
-	if (api_operations(h, r, path, pcvec, 2, qvec, data, username) < 0)
+	if (api_operations(h, r, path, pcvec, 2, qvec, data,
+			   pretty, use_xml, parse_xml) < 0)
 	    goto done;
     }
     else if (strcmp(method, "test") == 0)
@@ -360,35 +431,20 @@ api_restconf(clicon_handle h,
 	cvec_free(pcvec);
     if (cb)
 	cbuf_free(cb);
-    if (username)
+    if (cbret)
-	free(username);
+	cbuf_free(cbret);
+    if (xret)
+	xml_free(xret);
     return retval;
 }
 
-/*! Process a FastCGI request
- * @param[in]  r        Fastcgi request handle
- */
-static int
-api_well_known(clicon_handle h,
-	       FCGX_Request *r)
-{
-    clicon_debug(1, "%s", __FUNCTION__);
-    FCGX_FPrintF(r->out, "Content-Type: application/xrd+xml\r\n");
-    FCGX_FPrintF(r->out, "\r\n");
-    FCGX_SetExitStatus(200, r->out); /* OK */
-    FCGX_FPrintF(r->out, "<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>\r\n");
-    FCGX_FPrintF(r->out, "   <Link rel='restconf' href='/restconf'/>\r\n");
-    FCGX_FPrintF(r->out, "</XRD>\r\n");
-
-    return 0;
-}
-
 static int
 restconf_terminate(clicon_handle h)
 {
     yang_spec      *yspec;
 
-    clicon_debug(0, "%s", __FUNCTION__);
+    clixon_plugin_exit(h);
+    rpc_callback_delete_all();
     clicon_rpc_close_session(h);
     if ((yspec = clicon_dbspec_yang(h)) != NULL)
 	yspec_free(yspec);
@@ -447,6 +503,7 @@ main(int    argc,
 {
     int           retval = -1;
     int           sock;
+    char	 *argv0 = argv[0];
     FCGX_Request  request;
     FCGX_Request *r = &request;
     char          c;
@@ -454,6 +511,8 @@ main(int    argc,
     char         *path;
     clicon_handle h;
     char         *yangspec=NULL;
+    char         *dir;
+    char	 *tmp;
     
     /* In the startup, logs to stderr & debug flag set later */
     clicon_log_init(__PROGRAM__, LOG_INFO, CLICON_LOG_SYSLOG); 
@@ -510,13 +569,22 @@ main(int    argc,
 	clicon_option_str_set(h, "CLICON_YANG_MODULE_MAIN", yangspec);
 
     /* Initialize plugins group */
-    if (restconf_plugin_load(h) < 0)
+    if ((dir = clicon_restconf_dir(h)) != NULL)
+	if (clixon_plugins_load(h, CLIXON_PLUGIN_INIT, dir, NULL) < 0)
 	    return -1;
 
     /* Parse yang database spec file */
     if (yang_spec_main(h) == NULL)
 	goto done;
 
+    /* Call start function in all plugins before we go interactive 
+       Pass all args after the standard options to plugin_start
+     */
+    tmp = *(argv-1);
+    *(argv-1) = argv0;
+    clixon_plugin_start(h, argc+1, argv-1);
+    *(argv-1) = tmp;
+
     if ((sockpath = clicon_option_str(h, "CLICON_RESTCONF_PATH")) == NULL){
 	clicon_err(OE_CFG, errno, "No CLICON_RESTCONF_PATH in clixon configure file");
 	goto done;
@@ -545,13 +613,12 @@ main(int    argc,
 	    if (strncmp(path, RESTCONF_API_ROOT, strlen(RESTCONF_API_ROOT)) == 0)
 		api_restconf(h, r); /* This is the function */
 	    else if (strncmp(path, RESTCONF_WELL_KNOWN, strlen(RESTCONF_WELL_KNOWN)) == 0) {
-		api_well_known(h, r); /* This is the function */
+		api_well_known(h, r); /*  */
 	    }
 	    else{
 		clicon_debug(1, "top-level %s not found", path);
 		notfound(r);
 	    }
-	    
 	}
 	else
 	    clicon_debug(1, "NULL URI");
@@ -559,7 +626,6 @@ main(int    argc,
     }
     retval = 0;
  done:
-    restconf_plugin_unload(h);
     restconf_terminate(h);
     return retval;
 }

apps/restconf/restconf_methods.h

@@ -46,27 +46,31 @@
  */
 int api_data_options(clicon_handle h, FCGX_Request *r);
 int api_data_head(clicon_handle h, FCGX_Request *r, cvec *pcvec, int pi, 
-		  cvec *qvec);
+		  cvec *qvec, int pretty, int use_xml);
 int api_data_get(clicon_handle h, FCGX_Request *r, cvec *pcvec, int pi, 
-		 cvec *qvec);
+		 cvec *qvec, int pretty, int use_xml);
 int api_data_post(clicon_handle h, FCGX_Request *r, char *api_path,
 		  cvec *pcvec, int pi, 
-		  cvec *qvec, char *data);
+		  cvec *qvec, char *data,
+		  int pretty, int use_xml, int parse_xml);
 int api_data_put(clicon_handle h, FCGX_Request *r, char *api_path, 
 		 cvec *pcvec, int pi, 
-		 cvec *qvec, char *data);
+		 cvec *qvec, char *data,
+		 int pretty, int use_xml, int parse_xml);
 int api_data_patch(clicon_handle h, FCGX_Request *r, char *api_path, 
 		   cvec *pcvec, int pi, 
 		   cvec *qvec, char *data);
-int api_data_delete(clicon_handle h, FCGX_Request *r, char *api_path, int pi);
+int api_data_delete(clicon_handle h, FCGX_Request *r, char *api_path, int pi,
+		    int pretty, int use_xml);
 
-int api_operation_get(clicon_handle h, FCGX_Request *r, 
+int api_operations_get(clicon_handle h, FCGX_Request *r, 
-		       char *path,
-		       cvec *pcvec, int pi, cvec *qvec, char *data);
-
-int api_operation_post(clicon_handle h, FCGX_Request *r, 
 		       char *path,
 		       cvec *pcvec, int pi, cvec *qvec, char *data,
-		       char *username);
+		       int pretty, int use_xml);
+
+int api_operations_post(clicon_handle h, FCGX_Request *r, 
+			char *path,
+			cvec *pcvec, int pi, cvec *qvec, char *data,
+			int pretty, int use_xml, int parse_xml);
 
 #endif /* _RESTCONF_METHODS_H_ */

clixon.conf.cpp.cpp

@@ -1,129 +0,0 @@
-#
-# ***** BEGIN LICENSE BLOCK *****
-# 
-# Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
-#
-# This file is part of CLIXON
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Alternatively, the contents of this file may be used under the terms of
-# the GNU General Public License Version 3 or later (the "GPL"),
-# in which case the provisions of the GPL are applicable instead
-# of those above. If you wish to allow use of your version of this file only
-# under the terms of the GPL, and not to allow others to
-# use your version of this file under the terms of Apache License version 2, 
-# indicate your decision by deleting the provisions above and replace them with
-# the notice and other provisions required by the GPL. If you do not delete
-# the provisions above, a recipient may use your version of this file under
-# the terms of any one of the Apache License version 2 or the GPL.
-#
-# ***** END LICENSE BLOCK *****
-#
-
-#
-# CLIXON options - Default values
-# The origin of this file is run a _first_ time through a pre-processor at 
-# clixon make install time causing autoconf constants (such as "prefix" and 
-# "localstatedir") to be replaced with their installed values.
-# It should be run a _second_ time as a part of installation of the application,
-# in case clixon.mk is included in the application include file, and 
-# "$(APPNAME).conf" rule is accessed.
-# 
-# See clicon_tutorial for more documentation
-
-# Location of configuration-file for default values (this file)
-CLICON_CONFIGFILE      sysconfdir/APPNAME.conf
-
-# Location of YANG module and submodule files. 
-CLICON_YANG_DIR        prefix/share/APPNAME/yang
-
-# Main yang module or absolute filename. If module then search as follows:
-#     <yangdir>/<module>[@<revision>]
-# CLICON_YANG_MODULE_MAIN clicon                
-
-# Option used to construct initial yang file:
-#     <module>[@<revision>]
-CLICON_YANG_MODULE_REVISION
-
-# Location of backend .so plugins
-CLICON_BACKEND_DIR     libdir/APPNAME/backend
-
-# Location of netconf (frontend) .so plugins
-CLICON_NETCONF_DIR    libdir/APPNAME/netconf 
-
-# Location of restconf (frontend) .so plugins
-CLICON_RESTCONF_DIR    libdir/APPNAME/restconf 
-
-# Location of cli frontend .so plugins
-CLICON_CLI_DIR        libdir/APPNAME/cli
-
-# Location of frontend .cli cligen spec files
-CLICON_CLISPEC_DIR    libdir/APPNAME/clispec
-
-# Enabled uses "startup" configuration on boot
-CLICON_USE_STARTUP_CONFIG    0
-
-# Address family for communicating with clixon_backend (UNIX|IPv4|IPv6)
-CLICON_SOCK_FAMILY  UNIX
-
-# If family above is AF_UNIX: Unix socket for communicating with clixon_backend
-# If family above is AF_INET: IPv4 address
-CLICON_SOCK         localstatedir/APPNAME/APPNAME.sock
-
-# Inet socket port for communicating with clixon_backend (only IPv4|IPv6)
-CLICON_SOCK_PORT    4535
-
-# Process-id file
-CLICON_BACKEND_PIDFILE  localstatedir/APPNAME/APPNAME.pidfile
-
-# Group membership to access clixon_backend unix socket
-# CLICON_SOCK_GROUP       clicon
-
-# Set if all configuration changes are committed directly, commit command unnecessary
-# CLICON_AUTOCOMMIT       0
-
-# Name of master plugin (both frontend and backend). Master plugin has special 
-# callbacks for frontends. See clicon user manual for more info.
-# CLICON_MASTER_PLUGIN    master
-
-# Startup CLI mode. This should match the CLICON_MODE in your startup clispec file
-# CLICON_CLI_MODE         base
-
-# Generate code for CLI completion of existing db symbols. Add name="myspec" in 
-# datamodel spec and reference as @myspec.
-# CLICON_CLI_GENMODEL     1
-
-# Generate code for CLI completion of existing db symbols
-# CLICON_CLI_GENMODEL_COMPLETION 1
-
-# How to generate and show CLI syntax: VARS|ALL
-# CLICON_CLI_GENMODEL_TYPE   VARS
-
-# Directory where "running", "candidate" and "startup" are placed
-CLICON_XMLDB_DIR      localstatedir/APPNAME
-
-# XMLDB datastore plugin filename (see datastore/ and clixon_xml_db.[ch])
-CLICON_XMLDB_PLUGIN libdir/xmldb/text.so
-
-# Dont include keys in cvec in cli vars callbacks, ie a & k in 'a <b> k <c>' ignored
-# CLICON_CLI_VARONLY      1
-
-# Set to 0 if you want CLI to wrap to next line.
-# Set to 1 if you  want CLI to scroll sideways when approaching right margin
-# CLICON_CLI_LINESCROLLING      1
-
-# FastCGI unix socket. Should be specified in webserver
-# Eg in nginx: fastcgi_pass unix:/www-data/clicon_restconf.sock;
-CLICON_RESTCONF_PATH /www-data/fastcgi_restconf.sock
-

configure

@@ -632,7 +632,6 @@ CPP
 OBJEXT
 EXEEXT
 ac_ct_CC
-with_keyvalue
 with_restconf
 RANLIB
 AR
@@ -640,6 +639,7 @@ EXE_SUFFIX
 SH_SUFFIX
 AR_SUFFIX
 OBJ_SUFFIX
+INSTALLFLAGS
 CPPFLAGS
 INCLUDES
 LDFLAGS
@@ -707,8 +707,6 @@ ac_user_opts='
 enable_option_checking
 with_cligen
 with_restconf
-with_keyvalue
-with_qdbm
 with_configfile
 '
       ac_precious_vars='build_alias
@@ -1346,8 +1344,6 @@ Optional Packages:
   --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
   --with-cligen=dir       Use CLIGEN here
   --without-restconf      disable support for restconf
-  --with-keyvalue         enable support for key-value xmldb datastore
-  --with-qdbm=dir         Use QDBM here, if keyvalue
   --with-configfile=FILE  set default path to config file
 
 Some influential environment variables:
@@ -2146,13 +2142,14 @@ ac_compiler_gnu=$ac_cv_c_compiler_gnu
 
 
 
-# Default CFLAGS unless set by environment.
+# Default CFLAGS unless set by environment
-: ${CFLAGS="-O2"}
+: ${CFLAGS="-O2 -Wall"}
+: ${INSTALLFLAGS="-s"}
 
 CLIXON_VERSION_MAJOR="3"
-CLIXON_VERSION_MINOR="5"
+CLIXON_VERSION_MINOR="6"
 CLIXON_VERSION_PATCH="0"
-CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}\""
+CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}.PRE\""
 # Fix to specific version (eg 3.5) or head (3)
 CLIGEN_VERSION="3"
 if test "$prefix" = "NONE"; then
@@ -2348,8 +2345,8 @@ test -n "$target_alias" &&
 
 
 
+
  # If yes, compile apps/restconf
- # If yes, compile datastore/keyvalue
 #
 ac_ext=c
 ac_cpp='$CPP $CPPFLAGS'
@@ -3283,12 +3280,12 @@ CPPFLAGS="-DHAVE_CONFIG_H ${CPPFLAGS}"
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: compiler is $CC" >&5
 $as_echo "compiler is $CC" >&6; }
 
-CFLAGS="${CFLAGS} -Wall"
-
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: CPPFLAGS is $CPPFLAGS" >&5
 $as_echo "CPPFLAGS is $CPPFLAGS" >&6; }
 { $as_echo "$as_me:${as_lineno-$LINENO}: result: CFLAGS is $CFLAGS" >&5
 $as_echo "CFLAGS is $CFLAGS" >&6; }
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: INSTALLFLAGS is $INSTALLFLAGS" >&5
+$as_echo "INSTALLFLAGS is $INSTALLFLAGS" >&6; }
 
 for ac_prog in 'bison -y' byacc
 do
@@ -3946,107 +3943,6 @@ fi
 
 fi
 
-# This is for keyvalue datastore (and qdbm)
-
-# Check whether --with-keyvalue was given.
-if test "${with_keyvalue+set}" = set; then :
-  withval=$with_keyvalue;
-else
-  with_keyvalue=no
-fi
-
-if test "x${with_keyvalue}" == xyes; then
-   # This is for qdbm
-
-# Check whether --with-qdbm was given.
-if test "${with_qdbm+set}" = set; then :
-  withval=$with_qdbm;
-fi
-
-   if test "${with_qdbm}"; then
-      echo "Using QDBM here: ${with_qdbm}"
-      CPPFLAGS="-I${with_qdbm}/include ${CPPFLAGS}"
-      LDFLAGS="-L${with_qdbm}/lib ${LDFLAGS}"
-   fi
-   # Problem: depot.h may be in qdbm/depot.h.
-   for ac_header in depot.h
-do :
-  ac_fn_c_check_header_mongrel "$LINENO" "depot.h" "ac_cv_header_depot_h" "$ac_includes_default"
-if test "x$ac_cv_header_depot_h" = xyes; then :
-  cat >>confdefs.h <<_ACEOF
-#define HAVE_DEPOT_H 1
-_ACEOF
-
-else
-  for ac_header in qdbm/depot.h
-do :
-  ac_fn_c_check_header_mongrel "$LINENO" "qdbm/depot.h" "ac_cv_header_qdbm_depot_h" "$ac_includes_default"
-if test "x$ac_cv_header_qdbm_depot_h" = xyes; then :
-  cat >>confdefs.h <<_ACEOF
-#define HAVE_QDBM_DEPOT_H 1
-_ACEOF
-
-else
-  as_fn_error $? "libqdbm-dev required" "$LINENO" 5
-fi
-
-done
-
-fi
-
-done
-
-   { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dpopen in -lqdbm" >&5
-$as_echo_n "checking for dpopen in -lqdbm... " >&6; }
-if ${ac_cv_lib_qdbm_dpopen+:} false; then :
-  $as_echo_n "(cached) " >&6
-else
-  ac_check_lib_save_LIBS=$LIBS
-LIBS="-lqdbm  $LIBS"
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h.  */
-
-/* Override any GCC internal prototype to avoid an error.
-   Use char because int might match the return type of a GCC
-   builtin and then its argument prototype would still apply.  */
-#ifdef __cplusplus
-extern "C"
-#endif
-char dpopen ();
-int
-main ()
-{
-return dpopen ();
-  ;
-  return 0;
-}
-_ACEOF
-if ac_fn_c_try_link "$LINENO"; then :
-  ac_cv_lib_qdbm_dpopen=yes
-else
-  ac_cv_lib_qdbm_dpopen=no
-fi
-rm -f core conftest.err conftest.$ac_objext \
-    conftest$ac_exeext conftest.$ac_ext
-LIBS=$ac_check_lib_save_LIBS
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_qdbm_dpopen" >&5
-$as_echo "$ac_cv_lib_qdbm_dpopen" >&6; }
-if test "x$ac_cv_lib_qdbm_dpopen" = xyes; then :
-  cat >>confdefs.h <<_ACEOF
-#define HAVE_LIBQDBM 1
-_ACEOF
-
-  LIBS="-lqdbm $LIBS"
-
-else
-  as_fn_error $? "libqdbm-dev required" "$LINENO" 5
-fi
-
-   ac_config_files="$ac_config_files datastore/keyvalue/Makefile"
-
-fi
-
 # Set default config file location
 
 # Check whether --with-configfile was given.
@@ -4311,7 +4207,6 @@ _ACEOF
 
 
 
-# See also datastore/keyvalue/Makefile in with_keyvalue clause above
 ac_config_files="$ac_config_files Makefile lib/Makefile lib/src/Makefile lib/clixon/Makefile apps/Makefile apps/cli/Makefile apps/backend/Makefile apps/netconf/Makefile apps/restconf/Makefile include/Makefile etc/Makefile etc/clixonrc example/Makefile example/docker/Makefile extras/rpm/Makefile docker/Makefile docker/cli/Makefile docker/cli/Dockerfile docker/backend/Makefile docker/backend/Dockerfile docker/netconf/Makefile docker/netconf/Dockerfile datastore/Makefile datastore/text/Makefile yang/Makefile doc/Makefile"
 
 cat >confcache <<\_ACEOF
@@ -5005,7 +4900,6 @@ do
   case $ac_config_target in
     "include/clixon_config.h") CONFIG_HEADERS="$CONFIG_HEADERS include/clixon_config.h" ;;
     "lib/clixon/clixon.h") CONFIG_HEADERS="$CONFIG_HEADERS lib/clixon/clixon.h" ;;
-    "datastore/keyvalue/Makefile") CONFIG_FILES="$CONFIG_FILES datastore/keyvalue/Makefile" ;;
     "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;;
     "lib/Makefile") CONFIG_FILES="$CONFIG_FILES lib/Makefile" ;;
     "lib/src/Makefile") CONFIG_FILES="$CONFIG_FILES lib/src/Makefile" ;;

configure.ac

@@ -38,13 +38,14 @@
 
 AC_INIT(lib/clixon/clixon.h.in)
 
-# Default CFLAGS unless set by environment.
+# Default CFLAGS unless set by environment
-: ${CFLAGS="-O2"}
+: ${CFLAGS="-O2 -Wall"}
+: ${INSTALLFLAGS="-s"}
 
 CLIXON_VERSION_MAJOR="3"
-CLIXON_VERSION_MINOR="5"
+CLIXON_VERSION_MINOR="6"
 CLIXON_VERSION_PATCH="0"
-CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}\""
+CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}.PRE\""
 # Fix to specific version (eg 3.5) or head (3) 
 CLIGEN_VERSION="3" 
 if test "$prefix" = "NONE"; then
@@ -77,6 +78,7 @@ AC_SUBST(CFLAGS)
 AC_SUBST(LDFLAGS)
 AC_SUBST(INCLUDES)
 AC_SUBST(CPPFLAGS)
+AC_SUBST(INSTALLFLAGS)
 AC_SUBST(LIBS)
 AC_SUBST(OBJ_SUFFIX)
 AC_SUBST(AR_SUFFIX)
@@ -85,7 +87,6 @@ AC_SUBST(EXE_SUFFIX)
 AC_SUBST(AR)
 AC_SUBST(RANLIB)
 AC_SUBST(with_restconf) # If yes, compile apps/restconf
-AC_SUBST(with_keyvalue) # If yes, compile datastore/keyvalue
 #
 AC_PROG_CC()
 AC_PROG_CPP
@@ -94,10 +95,9 @@ CPPFLAGS="-DHAVE_CONFIG_H ${CPPFLAGS}"
 
 AC_MSG_RESULT(compiler is $CC)	
 
-CFLAGS="${CFLAGS} -Wall" 
-
 AC_MSG_RESULT(CPPFLAGS is $CPPFLAGS)	
 AC_MSG_RESULT(CFLAGS is $CFLAGS)
+AC_MSG_RESULT(INSTALLFLAGS is $INSTALLFLAGS)	
 
 AC_PROG_YACC
 AC_PROG_LEX
@@ -142,25 +142,6 @@ if test "x${with_restconf}" == xyes; then
    AC_CHECK_LIB(fcgi, FCGX_Init,, AC_MSG_ERROR([libfcgi-dev missing])) 
 fi
 
-# This is for keyvalue datastore (and qdbm)
-AC_ARG_WITH([keyvalue],
-            [AS_HELP_STRING([--with-keyvalue],[enable support for key-value xmldb datastore])],
-	    [],
-            [with_keyvalue=no])
-if test "x${with_keyvalue}" == xyes; then
-   # This is for qdbm
-   AC_ARG_WITH(qdbm,  [  --with-qdbm=dir         Use QDBM here, if keyvalue ] )
-   if test "${with_qdbm}"; then
-      echo "Using QDBM here: ${with_qdbm}"
-      CPPFLAGS="-I${with_qdbm}/include ${CPPFLAGS}"
-      LDFLAGS="-L${with_qdbm}/lib ${LDFLAGS}"
-   fi
-   # Problem: depot.h may be in qdbm/depot.h. 
-   AC_CHECK_HEADERS(depot.h,,[AC_CHECK_HEADERS(qdbm/depot.h,,AC_MSG_ERROR(libqdbm-dev required))])
-   AC_CHECK_LIB(qdbm, dpopen,, AC_MSG_ERROR(libqdbm-dev required))
-   AC_CONFIG_FILES(datastore/keyvalue/Makefile)
-fi
-
 # Set default config file location
 AC_ARG_WITH([configfile],
 	    [AS_HELP_STRING([--with-configfile=FILE],[set default path to config file])],
@@ -193,7 +174,6 @@ AC_DEFINE_UNQUOTED(CLIXON_DEFAULT_CONFIG,"${DEFAULT_CONFIG}",[Location for apps
 
 AH_BOTTOM([#include <clixon_custom.h>])
 
-# See also datastore/keyvalue/Makefile in with_keyvalue clause above
 AC_OUTPUT(Makefile
 	  lib/Makefile
 	  lib/src/Makefile 

datastore/Makefile.in

@@ -45,6 +45,7 @@ localstatedir	= @localstatedir@
 sysconfdir	= @sysconfdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 LIBS    	= @LIBS@
 with_restconf	= @with_restconf@
@@ -68,9 +69,6 @@ INCLUDES	= -I. -I$(top_srcdir)/lib/src -I$(top_srcdir)/lib -I$(top_srcdir)/inclu
 
 
 SUBDIRS = text
-ifeq ($(with_keyvalue),yes)
-SUBDIRS += keyvalue
-endif
 
 .PHONY: all clean depend install $(SUBDIRS)
 
@@ -104,8 +102,8 @@ install-include:
 	do (cd $$i ; $(MAKE) $(MFLAGS) $@)||exit 1; done; 
 
 install: $(APPL)
-	install -d $(DESTDIR)$(bindir)
+	install -d -m 0755 $(DESTDIR)$(bindir)
-	install $(APPL) $(DESTDIR)$(bindir)
+	install -m 0755 $(INSTALLFLAGS) $(APPL) $(DESTDIR)$(bindir)
 	for i in $(SUBDIRS); \
 	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done
 

datastore/datastore_client.c

@@ -33,11 +33,11 @@
 
   * Examples: 
 
-./datastore_client -d candidate -b /usr/local/var/routing -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/routing/yang -m ietf-ip get /
+./datastore_client -d candidate -b /usr/local/var/example -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/example/yang -m ietf-ip get /
 
-sudo ./datastore_client -d candidate -b /usr/local/var/routing -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/routing/yang -m ietf-ip put merge /interfaces/interface=eth66 '<config>eth66</config>'
+sudo ./datastore_client -d candidate -b /usr/local/var/example -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/example/yang -m ietf-ip put merge /interfaces/interface=eth66 '<config>eth66</config>'
 
-sudo ./datastore_client -d candidate -b /usr/local/var/routing -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/routing/yang -m ietf-ip put merge / '<config><interfaces><interface><name>eth0</name><enabled>true</enabled></interface></interfaces></config>'
+sudo ./datastore_client -d candidate -b /usr/local/var/example -p /home/olof/src/clixon/datastore/keyvalue/keyvalue.so -y /usr/local/share/example/yang -m ietf-ip put merge / '<config><interfaces><interface><name>eth0</name><enabled>true</enabled></interface></interfaces></config>'
 
  */
 
@@ -124,6 +124,7 @@ main(int argc, char **argv)
     cxobj              *xt = NULL;
     int                 i;
     char               *xpath;
+    cbuf               *cbret = NULL;
 
     /* In the startup, logs to stderr & debug flag set later */
     clicon_log_init(__PROGRAM__, LOG_INFO, CLICON_LOG_STDERR); 
@@ -261,7 +262,9 @@ main(int argc, char **argv)
 	    goto done;
 	if (xml_rootchild(xt, 0, &xt) < 0)
 	    goto done;
-	if (xmldb_put(h, db, op, xt) < 0)
+	if ((cbret = cbuf_new()) == NULL)
+	    goto done;
+	if (xmldb_put(h, db, op, xt, cbret) < 0)
 	    goto done;
     }
     else if (strcmp(cmd, "copy")==0){
@@ -325,6 +328,8 @@ main(int argc, char **argv)
     if (xmldb_plugin_unload(h) < 0)
 	goto done;
   done:
+    if (cbret)
+	cbuf_free(cbret);
     if (xt)
 	xml_free(xt);
     if (h)

datastore/keyvalue/Makefile.in

@@ -47,6 +47,7 @@ sysconfdir	= @sysconfdir@
 VPATH       	= @srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ -rdynamic -fPIC 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 LDFLAGS 	= @LDFLAGS@
 LIBS    	= @LIBS@ 
 DATASTORE       = keyvalue
@@ -80,8 +81,8 @@ distclean: clean
 	$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $<
 
 install: $(PLUGIN)
-	install -d $(DESTDIR)$(libdir)/xmldb
+	install -d -m 0755 $(DESTDIR)$(libdir)/xmldb
-	install $(PLUGIN) $(DESTDIR)$(libdir)/xmldb
+	install -m 0644 $(INSTALLFLAGS) $(PLUGIN) $(DESTDIR)$(libdir)/xmldb
 
 install-include:
 

datastore/keyvalue/clixon_keyvalue.c

@@ -211,7 +211,7 @@ append_listkeys(cbuf      *ckey,
 		       xml_name(xt), keyname);
 	    goto done;
 	}
-	if (percent_encode(xml_body(xkey), &bodyenc) < 0)
+	if (uri_percent_encode(xml_body(xkey), &bodyenc) < 0)
 	    goto done;
 	if (i++)
 	    cprintf(ckey, ",");
@@ -312,7 +312,7 @@ get(char      *dbname,
 	    restval++;
 	}
 	if (i == 1){ /* spec->module->node */
-	    if ((y = yang_find_topnode(ys, name, 0)) == NULL){
+	    if ((y = yang_find_topnode(ys, name, YC_DATANODE)) == NULL){
 		clicon_err(OE_UNIX, errno, "No yang node found: %s", name);
 		goto done;
 	    }
@@ -328,7 +328,7 @@ get(char      *dbname,
 	     * If xml element is a leaf-list, then the next element is expected to
 	     * be a value
 	     */
-	    if (percent_decode(restval, &argdec) < 0)
+	    if (uri_percent_decode(restval, &argdec) < 0)
 		goto done;
 	    if ((xc = xml_find(x, name))==NULL ||
 		(xb = xml_find(xc, argdec))==NULL){
@@ -373,7 +373,7 @@ get(char      *dbname,
 		if (j>=nvalvec)
 		    break;
 		arg = valvec[j++];
-		if (percent_decode(arg, &argdec) < 0)
+		if (uri_percent_decode(arg, &argdec) < 0)
 		    goto done;
 		cprintf(cb, "[%s=%s]", cv_string_get(cvi), argdec);
 		free(argdec);
@@ -391,7 +391,7 @@ get(char      *dbname,
 			break;
 		    arg = valvec[j++];
 		    keyname = cv_string_get(cvi);
-		    if (percent_decode(arg, &argdec) < 0)
+		    if (uri_percent_decode(arg, &argdec) < 0)
 			goto done;
 		    if (create_keyvalues(xc,
 					 ykey,
@@ -681,7 +681,7 @@ put(char               *dbfile,
 	    goto done;
 	break;
     case Y_LEAF_LIST:
-	if (percent_encode(body, &bodyenc) < 0)
+	if (uri_percent_encode(body, &bodyenc) < 0)
 	    goto done;
 	cprintf(cbxk, "=%s", bodyenc);
 	break;
@@ -799,7 +799,7 @@ kv_put(xmldb_handle        xh,
     }
     //	clicon_log(LOG_WARNING, "%s", __FUNCTION__);
     while ((x = xml_child_each(xt, x, CX_ELMNT)) != NULL){
-	if ((ys = yang_find_topnode(yspec, xml_name(x), 0)) == NULL){
+	if ((ys = yang_find_topnode(yspec, xml_name(x), YC_DATANODE)) == NULL){
 	    clicon_err(OE_UNIX, errno, "No yang node found: %s", xml_name(x));
 	    goto done;
 	}

datastore/text/Makefile.in

@@ -47,6 +47,7 @@ HOST_VENDOR     = @host_vendor@
 VPATH       	= @srcdir@
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ -rdynamic -fPIC 
+INSTALLFLAGS  	= @INSTALLFLAGS@
 LDFLAGS 	= @LDFLAGS@
 LIBS    	= @LIBS@ 
 DATASTORE       = text
@@ -84,8 +85,8 @@ distclean: clean
 	$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $<
 
 install: $(PLUGIN)
-	install -d $(DESTDIR)$(libdir)/xmldb
+	install -d -m 0755 $(DESTDIR)$(libdir)/xmldb
-	install $(PLUGIN) $(DESTDIR)$(libdir)/xmldb
+	install -m 0644 $(INSTALLFLAGS) $(PLUGIN) $(DESTDIR)$(libdir)/xmldb
 
 install-include:
 

datastore/text/clixon_xmldb_text.c

@@ -560,6 +560,7 @@ text_get(xmldb_handle xh,
  * @param[in]  x0p Parent of x0
  * @param[in]  x1  xml tree which modifies base
  * @param[in]  op  OP_MERGE, OP_REPLACE, OP_REMOVE, etc 
+ * @param[out] cbret  Initialized cligen buffer. Contains return XML or "".
  * Assume x0 and x1 are same on entry and that y is the spec
  * @see put in clixon_keyvalue.c
  */
@@ -568,7 +569,8 @@ text_modify(cxobj              *x0,
 	    yang_node          *y0,
 	    cxobj              *x0p,
 	    cxobj              *x1,
-	    enum operation_type op)
+	    enum operation_type op,
+	    cbuf               *cbret)
 {
     int        retval = -1;
     char      *opstr;
@@ -594,8 +596,9 @@ text_modify(cxobj              *x0,
 	switch(op){ 
 	case OP_CREATE:
 	    if (x0){
-		clicon_err(OE_XML, 0, "Object to create already exists");
+		if (netconf_data_exists(cbret, "Data already exists; cannot create new resource") < 0)
 		    goto done;
+		goto ok;
 	    }
 	case OP_NONE: /* fall thru */
 	case OP_MERGE:
@@ -631,8 +634,9 @@ text_modify(cxobj              *x0,
 	    break;
 	case OP_DELETE:
 	    if (x0==NULL){
-		clicon_err(OE_XML, 0, "Object to delete does not exist");
+		if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0)
 		    goto done;
+		goto ok;
 	    }
 	case OP_REMOVE: /* fall thru */
 	    if (x0){
@@ -647,8 +651,9 @@ text_modify(cxobj              *x0,
 	switch(op){ 
 	case OP_CREATE:
 	    if (x0){
-		clicon_err(OE_XML, 0, "Object to create already exists");
+		if (netconf_data_exists(cbret, "Data already exists; cannot create new resource") < 0)
 		    goto done;
+		goto ok;
 	    }
 	case OP_REPLACE: /* fall thru */
 	    if (x0){
@@ -704,14 +709,18 @@ text_modify(cxobj              *x0,
 	    while ((x1c = xml_child_each(x1, x1c, CX_ELMNT)) != NULL) {
 		x1cname = xml_name(x1c);
 		yc = yang_find_datanode(y0, x1cname);
-		if (text_modify(x0vec[i++], (yang_node*)yc, x0, x1c, op) < 0)
+		if (text_modify(x0vec[i++], (yang_node*)yc, x0, x1c, op, cbret) < 0)
 		    goto done;
+		/* If xml return - ie netconf error xml tree, then stop and return OK */
+		if (cbuf_len(cbret))
+		    goto ok;
 	    }
 	    break;
 	case OP_DELETE:
 	    if (x0==NULL){
-		clicon_err(OE_XML, 0, "Object to delete does not exist");
+		if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0)
 		    goto done;
+		goto ok;
 	    }
 	case OP_REMOVE: /* fall thru */
 	    if (x0)
@@ -721,27 +730,29 @@ text_modify(cxobj              *x0,
 	    break;
 	} /* CONTAINER switch op */
     } /* else Y_CONTAINER  */
-    // ok:
     xml_sort(x0p, NULL);
+ ok:
     retval = 0;
  done:
     if (x0vec)
 	free(x0vec);
     return retval;
-}
+} /* text_modify */
 
 /*! Modify a top-level base tree x0 with modification tree x1
  * @param[in]  x0  Base xml tree (can be NULL in add scenarios)
  * @param[in]  x1  xml tree which modifies base
  * @param[in]  yspec Top-level yang spec (if y is NULL)
  * @param[in]  op  OP_MERGE, OP_REPLACE, OP_REMOVE, etc 
+ * @param[out] cbret  Initialized cligen buffer. Contains return XML or "".
  * @see text_modify
  */
 static int
 text_modify_top(cxobj              *x0,
 		cxobj              *x1,
 		yang_spec          *yspec,
-		enum operation_type op)
+		enum operation_type op,
+		cbuf               *cbret)
 {
     int        retval = -1;
     char      *x1cname; /* child name */
@@ -759,7 +770,7 @@ text_modify_top(cxobj              *x0,
 	if (xml_operation(opstr, &op) < 0)
 	    goto done;
     /* Special case if x1 is empty, top-level only <config/> */
-    if (!xml_child_nr(x1)){ 
+    if (xml_child_nr(x1) == 0){ 
 	if (xml_child_nr(x0)) /* base tree not empty */
 	    switch(op){ 
 	    case OP_DELETE:
@@ -775,31 +786,43 @@ text_modify_top(cxobj              *x0,
 	else /* base tree empty */
 	    switch(op){ 
 	    case OP_DELETE:
-		clicon_err(OE_XML, 0, "Object to delete does not exist");
+		if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0)
+		    goto done;
+		goto ok;
 		break;
 	    default:
 		break;
 	    }
     }
+    /* Special case top-level replace */
+    if (op == OP_REPLACE || op == OP_DELETE){
+	x0c = NULL;
+	while ((x0c = xml_child_i(x0, 0)) != 0)
+	    xml_purge(x0c);
+    }
     /* Loop through children of the modification tree */
     x1c = NULL;
     while ((x1c = xml_child_each(x1, x1c, CX_ELMNT)) != NULL) {
 	x1cname = xml_name(x1c);
 	/* Get yang spec of the child */
-	if ((yc = yang_find_topnode(yspec, x1cname, 0)) == NULL){
+	if ((yc = yang_find_topnode(yspec, x1cname, YC_DATANODE)) == NULL){
-	    clicon_err(OE_YANG, ENOENT, "No yang spec");
+	    clicon_err(OE_YANG, ENOENT, "XML node %s/%s has no corresponding yang specification (Invalid XML or wrong Yang spec?", x1, x1cname);
 	    goto done;
 	}
 	/* See if there is a corresponding node in the base tree */
 	if (match_base_child(x0, x1c, &x0c, yc) < 0)
 	    goto done;
-	if (text_modify(x0c, (yang_node*)yc, x0, x1c, op) < 0)
+	if (text_modify(x0c, (yang_node*)yc, x0, x1c, op, cbret) < 0)
 	    goto done;
+	/* If xml return - ie netconf error xml tree, then stop and return OK */
+	if (cbuf_len(cbret))
+	    goto ok;
     }
+ ok:
     retval = 0;
  done:
     return retval;
-}
+} /* text_modify_top */
 
 /*! For containers without presence and no children, remove
  * @param[in]   x       XML tree node
@@ -846,7 +869,8 @@ int
 text_put(xmldb_handle        xh,
 	 const char         *db, 
 	 enum operation_type op,
-	 cxobj              *x1)
+	 cxobj              *x1,
+    	 cbuf               *cbret)
 {
     int                 retval = -1;
     struct text_handle *th = handle(xh);
@@ -857,7 +881,15 @@ text_put(xmldb_handle        xh,
     yang_spec          *yspec;
     cxobj              *x0 = NULL;
     struct db_element  *de = NULL;
+    int                 cbretlocal = 0; /* Set if cbret is NULL on entry */
     
+    if (cbret == NULL){
+	if ((cbret = cbuf_new()) == NULL){
+	    clicon_err(OE_XML, errno, "cbuf_new");
+	    goto done;
+	}
+	cbretlocal++;
+    }
     if ((yspec =  th->th_yangspec) == NULL){
 	clicon_err(OE_YANG, ENOENT, "No yang spec");
 	goto done;
@@ -922,8 +954,11 @@ text_put(xmldb_handle        xh,
      * Modify base tree x with modification x1. This is where the
      * new tree is made.
      */
-    if (text_modify_top(x0, x1, yspec, op) < 0)
+    if (text_modify_top(x0, x1, yspec, op, cbret) < 0)
 	goto done;
+    /* If xml return - ie netconf error xml tree, then stop and return OK */
+    if (cbuf_len(cbret))
+	goto ok;
 
     /* Remove NONE nodes if all subs recursively are also NONE */
     if (xml_tree_prune_flagged_sub(x0, XML_FLAG_NONE, 0, NULL) <0)
@@ -973,8 +1008,11 @@ text_put(xmldb_handle        xh,
     }
     else if (clicon_xml2file(f, x0, 0, th->th_pretty) < 0)
 	goto done;
+ ok:
     retval = 0;
  done:
+    if (cbretlocal && cbret)
+	cbuf_free(cbret);
     if (f != NULL)
 	fclose(f);
     if (dbfile)
@@ -1381,7 +1419,7 @@ main(int    argc,
 	    op = OP_REMOVE;
 	else
 	    usage(argv[0]);
-	if (xmldb_put(h, db, op, NULL, xn) < 0)
+	if (xmldb_put(h, db, op, NULL, xn, NULL) < 0)
 	    goto done;
     }
     else

datastore/text/clixon_xmldb_text.h

@@ -40,7 +40,7 @@
  * Prototypes
  */
 int text_get(xmldb_handle h, const char *db, char *xpath, int config, cxobj **xtop);
-int text_put(xmldb_handle h, const char *db, enum operation_type op, cxobj *xt);
+int text_put(xmldb_handle h, const char *db, enum operation_type op, cxobj *xt, cbuf *cbret);
 int text_dump(FILE *f, char *dbfilename, char *rxkey);
 int text_copy(xmldb_handle h, const char *from, const char *to);
 int text_lock(xmldb_handle h, const char *db, int pid);

doc/Doxyfile

@@ -743,7 +743,7 @@ WARN_LOGFILE           =
 # spaces.
 # Note: If this tag is empty the current directory is searched.
 
-INPUT     = ../lib/src/ ../lib/clicon/ ../apps/cli/ ../apps/config/ ../apps/netconf ../apps/dbctrl ../datastore
+INPUT     = ../lib/src/ ../lib/clicon/ ../apps/cli/ ../apps/config/ ../apps/netconf ../apps/dbctrl ../datastore ../datastore/text
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

doc/Doxyfile.graphs

@@ -743,7 +743,7 @@ WARN_LOGFILE           =
 # spaces.
 # Note: If this tag is empty the current directory is searched.
 
-INPUT     = ../lib/src/ ../lib/clicon/ ../apps/cli/ ../apps/backend/ ../apps/restconf/ ../apps/netconf ../apps/dbctrl ../datastore
+INPUT     = ../lib/src/ ../lib/clicon/ ../apps/cli/ ../apps/backend/ ../apps/restconf/ ../apps/netconf ../apps/dbctrl ../datastore ../datastore/text
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

doc/FAQ.md

@@ -1,4 +1,4 @@
-# Clixon FAQ
+i# Clixon FAQ
 
 ## What is Clixon?
 
@@ -24,7 +24,7 @@ Clixon is written in C. The plugins are written in C. The CLI
 specification uses cligen (http://cligen.se)
 
 ## How to best understand Clixon?
-Run the ietf yang routing example, in the example directory.
+Run the Clixon example, in the example directory.
 
 ## How do you build and install Clixon (and the example)?
 Clixon: 
@@ -56,14 +56,14 @@ Build using 'make doc' and aim your browser at doc/html/index.html or
 use the web resource: http://clicon.org/ref/index.html
 
 ## How do you run the example?
-- Start a backend server: 'clixon_backend -Ff /usr/local/etc/routing.xml'
+- Start a backend server: 'clixon_backend -Ff /usr/local/etc/example.xml'
-- Start a cli session: clixon_cli -f /usr/local/etc/routing.xml
+- Start a cli session: clixon_cli -f /usr/local/etc/example.xml
-- Start a netconf session: clixon_netconf -f /usr/local/etc/routing.xml
+- Start a netconf session: clixon_netconf -f /usr/local/etc/example.xml
 
 ## How is configuration data stored?
 Configuration data is stored in an XML datastore. The default is a
 text-based datastore. In the example the datastore are regular files found in
-/usr/local/var/routing/.
+/usr/local/var/example/.
 
 ## What is validate and commit?
 Clixon follows netconf in its validate and commit semantics.
@@ -79,15 +79,15 @@ is the core functionality of a clixon system.
 
 Clixon options are stored in an XML configuration file. The default
 configuration file is /usr/local/etc/clixon.xml. The example
-configuration file is installed at /usr/local/etc/routing.xml. The
+configuration file is installed at /usr/local/etc/example.xml. The
 YANG specification for the configuration file is clixon-config.yang.
 
 You can change where CLixon looks for the configuration FILE as follows:
-  # Provide -f FILE option when starting a program (eg clixon_backend -f FILE)
+  - Provide -f FILE option when starting a program (eg clixon_backend -f FILE)
-  # Provide --with-configfile=FILE when configuring
+  - Provide --with-configfile=FILE when configuring
-  # Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml
+  - Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml
-  # Provide --sysconfig=<dir> when configuring then FILE is <dir>/etc/clixon.xml
+  - Provide --sysconfig=<dir> when configuring then FILE is <dir>/etc/clixon.xml
-  # FILE is /usr/local/etc/clixon.xml
+  - FILE is /usr/local/etc/clixon.xml
 
 ## Can I run Clixon as docker containers?
 Yes, the example works as docker containers as well. backend and cli needs a 
@@ -109,12 +109,12 @@ You may also push the containers with 'make push' but you may then consider chan
 
 As an alternative to cli configuration, you can use netconf. Easiest is to just pipe netconf commands to the clixon_netconf application.
 Example:
-	echo "<rpc><get-config><source><candidate/></source><configuration/></get-config></rpc>]]>]]>" | clixon_netconf -f /usr/local/etc/routing.xml
+	echo "<rpc><get-config><source><candidate/></source><configuration/></get-config></rpc>]]>]]>" | clixon_netconf -f /usr/local/etc/example.xml
 
 However, more useful is to run clixon_netconf as an SSH
 subsystem. Register the subsystem in /etc/sshd_config:
 ```
-	Subsystem netconf /usr/local/bin/clixon_netconf -f /usr/local/etc/routing.xml
+	Subsystem netconf /usr/local/bin/clixon_netconf -f /usr/local/etc/example.xml
 ```
 and then invoke it from a client using
 ```
@@ -149,7 +149,7 @@ cli>
 ```
 or via netconf:
 ```
-clixon_netconf -qf /usr/local/etc/routing.xml 
+clixon_netconf -qf /usr/local/etc/example.xml 
 <rpc><create-subscription><stream>ROUTING</stream></create-subscription></rpc>]]>]]>
 <rpc-reply><ok/></rpc-reply>]]>]]>
 <notification><event>Routing notification</event></notification>]]>]]>
@@ -194,18 +194,39 @@ clixon_backend ... -c extra.xml
 ```
 
 The second way is by programming the plugin_reset() in the backend
-plugin. The example code contains an example on how to do this (see plugin_reset() in routing_backend.c).
+plugin. The example code contains an example on how to do this (see plugin_reset() in example_backend.c).
 
 ## I want to program. How do I extend the example?
-- routing.xml - Change the configuration file
+See [../apps/example] 
+- example.xml - Change the configuration file
 - The yang specifications - This is the central part. It changes the XML, database and the config cli.
-- routing_cli.cli - Change the fixed part of the CLI commands 
+- example_cli.cli - Change the fixed part of the CLI commands 
-- routing_cli.c - Cli C-commands are placed here.
+- example_cli.c - Cli C-commands are placed here.
-- routing_backend.c - Commit and validate functions.
+- example_backend.c - Commit and validate functions.
-- routing_netconf.c - Modify semantics of netconf commands.
+- example_netconf.c - Netconf plugin
+- example_restconf.c - Add restconf authentication, etc.
+
+## How is a plugin initiated?
+Each plugin is initiated with an API struct followed by a plugin init function as follows:
+```
+   static clixon_plugin_api api = {
+      "example",          /* name */
+      clixon_plugin_init, 
+      plugin_start,
+      ... /* more functions here */
+   }
+   clixon_plugin_api *
+   clixon_plugin_init(clicon_handle h)
+   {
+      ...
+      return &api; /* Return NULL on error */
+   }
+```
+For more info see [../example/README.md]
+
 
 ## How do I write a commit function?
-In the example, you write a commit function in routing_backend.c.
+In the example, you write a commit function in example_backend.c.
 Every time a commit is made, transaction_commit() is called in the
 backend.  It has a 'transaction_data td' argument which is used to fetch
 information on added, deleted and changed entries. You access this
@@ -235,9 +256,9 @@ More are found in the doxygen reference.
 
 ## How do I write a CLI callback function?
 
-1. You add an entry in routing_cli.cli
+1. You add an entry in example_cli.cli
 >   example("This is a comment") <var:int32>("This is a variable"), mycallback("myarg");
-2. Then define a function in routing_cli.c
+2. Then define a function in example_cli.c
 >   mycallback(clicon_handle h, cvec *cvv, cvec *arv)
 where 'cvv' contains the value of the variable 'var' and 'argv' contains the string "myarg".
 
@@ -283,10 +304,10 @@ implement the RFC, you need to register an RPC callback in the backend plugin:
 Example:
 ```
 int
-plugin_init(clicon_handle h)
+clixon_plugin_init(clicon_handle h)
 {
 ...
-   backend_rpc_cb_register(h, fib_route, NULL, "fib-route");
+   rpc_callback_register(h, fib_route, NULL, "fib-route");
 ...
 }
 ```
@@ -295,12 +316,31 @@ And then define the callback itself:
 static int 
 fib_route(clicon_handle h,            /* Clicon handle */
 	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-	  struct client_entry *ce,    /* Client session */
 	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-	  void         *arg)          /* Argument given at register */
+	  void         *arg,          /* Client session */
+	  void         *regarg)       /* Argument given at register */
 {
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");    
     return 0;
 }
 ```
 Here, the callback is over-simplified.
+
+## How do I write an authentication callback?
+
+A restconf call may need to be authenticated. 
+You can specify an authentication callback for restconf as follows:
+```
+int
+plugin_credentials(clicon_handle h,     
+		   void         *arg)
+{
+    FCGX_Request *r = (FCGX_Request *)arg;
+    ...
+    clicon_username_set(h, user);
+```
+
+To authenticate, the callback needs to return the value 1 and supply a username.
+
+See [../apps/example/example_restconf.c] plugin_credentials() for
+an example of HTTP basic auth.

etc/Makefile.in

@@ -48,8 +48,8 @@ distclean: clean
 	rm -f Makefile *~ .depend clixonrc
 
 install:	 clixonrc
-	install -m 755 -d $(DESTDIR)$(sysconfdir) 
+	install -m 0755 -d $(DESTDIR)$(sysconfdir)
-	install -m 755 clixonrc $(DESTDIR)$(sysconfdir)
+	install -m 0644 clixonrc $(DESTDIR)$(sysconfdir)
 
 install-include:	
 

example/Makefile.in

@@ -38,27 +38,29 @@ bindir 		= @bindir@
 includedir	= @includedir@
 datarootdir     = @datarootdir@
 
-APPNAME	        = routing
+APPNAME	        = example
 CC		= @CC@
 CFLAGS  	= @CFLAGS@ -rdynamic -fPIC 
+INSTALLFLAGS  	= @INSTALLFLAGS@ 
 
 INCLUDES 	= -I$(includedir) @INCLUDES@ 
 
 BE_PLUGIN       = $(APPNAME)_backend.so
+BE2_PLUGIN      = $(APPNAME)_backend_nacm.so
 CLI_PLUGIN      = $(APPNAME)_cli.so
 NETCONF_PLUGIN  = $(APPNAME)_netconf.so
+RESTCONF_PLUGIN = $(APPNAME)_restconf.so
 
-PLUGINS = $(BE_PLUGIN) $(CLI_PLUGIN) $(NETCONF_PLUGIN)
+PLUGINS = $(BE_PLUGIN) $(BE2_PLUGIN) $(CLI_PLUGIN) $(NETCONF_PLUGIN) $(RESTCONF_PLUGIN)
 
 all:	$(PLUGINS)
 
-# Note: clixon.mk has a rule for:
+# Note: clixon.mk has rules for clixon_DBSPECDIR, clixon_SYSCONFDIR, etc used below
-# $(APPNAME.conf)
 -include $(DESTDIR)$(datarootdir)/clixon/clixon.mk
 
-CLISPECS = routing_cli.cli
+CLISPECS = $(APPNAME)_cli.cli
 
-#YANGSPECS	 = $(APPNAME).yang
+YANGSPECS	 = $(APPNAME).yang
 YANGSPECS	+= ietf-yang-types@2013-07-15.yang
 YANGSPECS	+= ietf-inet-types@2013-07-15.yang
 YANGSPECS	+= ietf-interfaces@2014-05-08.yang
@@ -67,28 +69,39 @@ YANGSPECS	+= ietf-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipv4-unicast-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipv6-unicast-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipsec@2016-03-09.yang
-YANGSPECS	+= example.yang
 
 # Backend plugin
-BE_SRC   = routing_backend.c
+BE_SRC   = $(APPNAME)_backend.c
 BE_OBJ   = $(BE_SRC:%.c=%.o)
 $(BE_PLUGIN): $(BE_OBJ)
-	$(CC) -shared -o $@ -lc $<
+	$(CC) -Wall -shared -o $@ -lc $<
+
+# Secondary NACM backend plugin
+BE2_SRC  = $(APPNAME)_backend_nacm.c
+BE2_OBJ  = $(BE2_SRC:%.c=%.o)
+$(BE2_PLUGIN): $(BE2_OBJ)
+	$(CC) -Wall -shared -o $@ -lc $<
 
 # CLI frontend plugin
-CLI_SRC  = routing_cli.c
+CLI_SRC  = $(APPNAME)_cli.c
 CLI_OBJ  = $(CLI_SRC:%.c=%.o)
 $(CLI_PLUGIN): $(CLI_OBJ)
-	$(CC) -shared -o $@ -lc $^ 
+	$(CC) -Wall -shared -o $@ -lc $^ 
 
 # NETCONF frontend plugin
-NETCONF_SRC  = routing_netconf.c
+NETCONF_SRC  = $(APPNAME)_netconf.c
 NETCONF_OBJ  = $(NETCONF_SRC:%.c=%.o)
 $(NETCONF_PLUGIN): $(NETCONF_OBJ)
-	$(CC) -shared -o $@ -lc $^ 
+	$(CC) -Wall -shared -o $@ -lc $^
 
-SRC     = $(BE_SRC) $(CLI_SRC) $(NETCONF_SRC)
+# RESTCONF frontend plugin
-OBJS    = $(BE_OBJ) $(CLI_OBJ) $(NETCONF_OBJ)
+RESTCONF_SRC  = $(APPNAME)_restconf.c
+RESTCONF_OBJ  = $(RESTCONF_SRC:%.c=%.o)
+$(RESTCONF_PLUGIN): $(RESTCONF_OBJ)
+	$(CC) -Wall -shared -o $@ -lc $^ 
+
+SRC     = $(BE_SRC) $(BE2_SRC) $(CLI_SRC) $(NETCONF_SRC) $(RESTCONF_SRC)
+OBJS    = $(BE_OBJ) $(BE2_OBJ) $(CLI_OBJ) $(NETCONF_OBJ) $(RESTCONF_OBJ)
 
 clean:
 	rm -f $(PLUGINS) $(OBJS) 
@@ -98,20 +111,22 @@ distclean: clean
 	rm -f Makefile *~ .depend
 	(cd docker && $(MAKE) $(MFLAGS) $@)
 
-install: $(YANGSPECS) $(CLISPECS) $(BE_PLUGIN) $(CLI_PLUGIN) $(NETCONF_PLUGIN) $(APPNAME).xml
+install: $(YANGSPECS) $(CLISPECS) $(BE_PLUGIN) $(BE2_PLUGIN) $(CLI_PLUGIN) $(NETCONF_PLUGIN) $(RESTCONF_PLUGIN) $(APPNAME).xml
-	install -d $(DESTDIR)$(clixon_SYSCONFDIR)
+	install -d -m 0755 $(DESTDIR)$(clixon_SYSCONFDIR)
-	install $(APPNAME).xml $(DESTDIR)$(clixon_SYSCONFDIR)
+	install -m 0644 $(APPNAME).xml $(DESTDIR)$(clixon_SYSCONFDIR)
-	install -d $(DESTDIR)$(clixon_DBSPECDIR)/yang
+	install -d -m 0755 $(DESTDIR)$(clixon_DBSPECDIR)/yang
-	install $(YANGSPECS) $(DESTDIR)$(clixon_DBSPECDIR)/yang
+	install -m 0644 $(YANGSPECS) $(DESTDIR)$(clixon_DBSPECDIR)/yang
-	install -d $(DESTDIR)$(clixon_LIBDIR)/cli
+	install -d -m 0755 $(DESTDIR)$(clixon_LIBDIR)/cli
-	install $(CLI_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/cli; 
+	install -m 0644 $(INSTALLFLAGS) $(CLI_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/cli
-	install -d $(DESTDIR)$(clixon_LIBDIR)/backend
+	install -d -m 0755 $(DESTDIR)$(clixon_LIBDIR)/backend
-	install $(BE_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/backend; 
+	install -m 0644 $(INSTALLFLAGS) $(BE_PLUGIN) $(BE2_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/backend
-	install -d $(DESTDIR)$(clixon_LIBDIR)/netconf
+	install -d -m 0755 $(DESTDIR)$(clixon_LIBDIR)/netconf
-	install $(NETCONF_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/netconf; 
+	install -m 0644 $(INSTALLFLAGS) $(NETCONF_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/netconf
-	install -d $(DESTDIR)$(clixon_LIBDIR)/clispec
+	install -d -m 0755 $(DESTDIR)$(clixon_LIBDIR)/restconf
-	install $(CLISPECS) $(DESTDIR)$(clixon_LIBDIR)/clispec; 
+	install -m 0644 $(INSTALLFLAGS) $(RESTCONF_PLUGIN) $(DESTDIR)$(clixon_LIBDIR)/restconf
-	install -d $(DESTDIR)$(clixon_LOCALSTATEDIR)
+	install -d -m 0755 $(DESTDIR)$(clixon_LIBDIR)/clispec
+	install -m 0644 $(CLISPECS) $(DESTDIR)$(clixon_LIBDIR)/clispec
+	install -d -m 0755 $(DESTDIR)$(clixon_LOCALSTATEDIR)
 	(cd docker && $(MAKE) $(MFLAGS) $@)
 
 uninstall: 

example/README.md

@@ -1,4 +1,20 @@
-# Clixon yang routing example
+# Clixon example
+
+This directory contains a Clixon example which includes a simple
+routing example. It contains the following files:
+* example.xml       The configuration file. See yang/clixon-config@<date>.yang for all available fields.
+* example.yang      The yang spec of the example. It mainly includes ietf routing and IP modules.
+* example_cli.cli   CLIgen specification.
+* example_cli.c     CLI callback plugin containing functions called in the cli file above: a generic callback (`mycallback`) and an RPC (`fib_route_rpc`).
+* example_backend.c Backend callback plugin including example of:
+  * transaction callbacks (validate/commit),
+  * notification,
+  * rpc handler
+  * state-data handler, ie non-config data
+* example_backend_nacm.c Secondary backend plugin. Plugins are loaded alphabetically.
+* example_restconf.c Restconf callback plugin containing an HTTP basic authentication callback
+* example_netconf.c Netconf callback plugin
+* Makefile.in       Example makefile where plugins are built and installed
 
 ## Compile and run
 ```
@@ -7,15 +23,23 @@
 ```
 Start backend:
 ```
-    clixon_backend -f /usr/local/etc/routing.xml -I
+    clixon_backend -f /usr/local/etc/example.xml -I
 ```
 Edit cli:
 ```
-    clixon_cli -f /usr/local/etc/routing.xml
+    clixon_cli -f /usr/local/etc/example.xml
 ```
 Send netconf command:
 ```
-    clixon_netconf -f /usr/local/etc/routing.xml
+    clixon_netconf -f /usr/local/etc/example.xml
+```
+Start clixon restconf daemon
+```
+> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data
+```
+Send restconf command
+```
+    curl -G http://127.0.0.1/restconf/data
 ```
 
 ## Setting data example using netconf
@@ -65,6 +89,38 @@ Routing notification
 ...
 ```
 
+## Initializing a plugin
+
+The example includes a restonf, netconf, CLI and two backend plugins.
+Each plugin is initiated with an API struct followed by a plugin init function.
+The content of the API struct is different depending on what kind of plugin it is.
+The plugin init function may also include registering RPC functions, see below is for a backend.
+```
+static clixon_plugin_api api = {
+    "example",          /* name */
+    clixon_plugin_init, 
+    plugin_start,       
+    plugin_exit,        
+    .ca_reset=plugin_reset,/* reset */          
+    .ca_statedata=plugin_statedata, /* statedata */
+    .ca_trans_begin=NULL, /* trans begin */
+    .ca_trans_validate=transaction_validate,/* trans validate */
+    .ca_trans_complete=NULL,                /* trans complete */
+    .ca_trans_commit=transaction_commit,    /* trans commit */
+    .ca_trans_end=NULL,                     /* trans end */
+    .ca_trans_abort=NULL                    /* trans abort */
+};
+
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    /* Optional callback registration for RPC calls */
+    rpc_callback_register(h, fib_route, NULL, "fib-route");
+    /* Return plugin API */
+    return &api; /* Return NULL on error */
+}
+```
+
 ## Operation data
 
 Clixon implements Yang RPC operations by an extension mechanism. The
@@ -95,18 +151,18 @@ In the backend, a callback is registered (fib_route()) which handles the RPC.
 static int 
 fib_route(clicon_handle h, 
 	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-	  struct client_entry *ce,    /* Client session */
 	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-	  void         *arg)          /* Argument given at register */
+	  void         *arg,          /* Client session */
+	  void         *regarg)       /* Argument given at register */
 {
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");    
     return 0;
 }
 int
-plugin_init(clicon_handle h)
+clixon_plugin_init(clicon_handle h)
 {
 ...
-   backend_rpc_cb_register(h, fib_route, NULL, "fib-route");
+   rpc_callback_register(h, fib_route, NULL, "fib-route");
 ...
 }
 ```
@@ -114,7 +170,7 @@ plugin_init(clicon_handle h)
 
 Netconf <get> and restconf GET also returns state data, in contrast to
 config data. 
-
+p
 In YANG state data is specified with "config false;". In the example, interface-state is state data.
 
 To return state data, you need to write a backend state data callback
@@ -126,8 +182,15 @@ a real example would poll or get the interface counters via a system
 call, as well as use the "xpath" argument to identify the requested
 state data.
 
+## Authentication and NACM
+The example contains some stubs for authorization according to [RFC8341(NACM)](https://tools.ietf.org/html/rfc8341):
+* A basic auth HTTP callback, see: example_restconf_credentials() containing three example users: adm1, wilma, and guest, according to the examples in Appendix A in the RFC.
+* A NACM backend plugin reporting the mandatory NACM state variables.
+
 
 ## Run as docker container
+
+(Note not updated)
 ```
 cd docker
 # look in README

example/example.xml

@@ -0,0 +1,20 @@
+<config>
+  <CLICON_CONFIGFILE>/usr/local/etc/example.xml</CLICON_CONFIGFILE>
+  <CLICON_YANG_DIR>/usr/local/share/example/yang</CLICON_YANG_DIR>
+  <CLICON_YANG_MODULE_MAIN>example</CLICON_YANG_MODULE_MAIN>
+  <CLICON_CLI_MODE>example</CLICON_CLI_MODE>
+  <CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
+  <CLICON_NETCONF_DIR>/usr/local/lib/example/netconf</CLICON_NETCONF_DIR>
+  <CLICON_RESTCONF_DIR>/usr/local/lib/example/restconf</CLICON_RESTCONF_DIR>
+  <CLICON_CLI_DIR>/usr/local/lib/example/cli</CLICON_CLI_DIR>
+  <CLICON_CLISPEC_DIR>/usr/local/lib/example/clispec</CLICON_CLISPEC_DIR>
+  <CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
+  <CLICON_BACKEND_PIDFILE>/usr/local/var/example/example.pidfile</CLICON_BACKEND_PIDFILE>
+  <CLICON_CLI_GENMODEL_COMPLETION>1</CLICON_CLI_GENMODEL_COMPLETION>
+  <CLICON_CLI_GENMODEL_TYPE>VARS</CLICON_CLI_GENMODEL_TYPE>
+  <CLICON_XMLDB_DIR>/usr/local/var/example</CLICON_XMLDB_DIR>
+  <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
+  <CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
+  <CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
+  <CLICON_NACM_MODE>disabled</CLICON_NACM_MODE>
+</config>

example/example.yang

@@ -1,10 +1,30 @@
 module example {
+    prefix ex;
     import ietf-ip {
 	prefix ip;
     }
     import ietf-routing {
 	prefix rt;
     }
+    import ietf-netconf-acm {
+	prefix nacm;
+    }
     description
 	"Example code that includes ietf-ip and ietf-routing";
+    rpc client-rpc {
+	description "Example local client-side RPC that is processed by the
+                     the netconf/restconf and not sent to the backend.
+                     This is a clixon implementation detail: some rpc:s
+                     are better processed by the client for API or perf reasons";
+	input {
+	    leaf request {
+		type string;
+	    }
+	}
+	output {
+	    leaf result{
+		type string;
+	    }
+	}
+    }
 }

example/example_backend.c

@@ -77,12 +77,14 @@ transaction_commit(clicon_handle    h,
     int     i;
     size_t  len;
 
+    clicon_debug(1, "%s", __FUNCTION__);
     /* Get all added i/fs */
     if (xpath_vec_flag(target, "//interface", XML_FLAG_ADD, &vec, &len) < 0)
 	return -1;
     if (debug)
 	for (i=0; i<len; i++)             /* Loop over added i/fs */
 	    xml_print(stdout, vec[i]); /* Print the added interface */
+  // done:
     if (vec)
 	free(vec);
     return 0;
@@ -119,13 +121,15 @@ notification_timer_setup(clicon_handle h)
     return event_reg_timeout(t, notification_timer, h, "notification timer");
 }
 
-/*! IETF Routing fib-route rpc */
+/*! IETF Routing fib-route rpc 
+ * @see ietf-routing@2014-10-26.yang  (fib-route)
+ */
 static int 
 fib_route(clicon_handle h,            /* Clicon handle */
 	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-	  struct client_entry *ce,    /* Client session */
 	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-	  void         *arg)          /* Argument given at register */
+	  void         *arg,          /* Client session */
+	  void         *regarg)       /* Argument given at register */
 {
     cprintf(cbret, "<rpc-reply><route>"
 	    "<address-family>ipv4</address-family>"
@@ -134,25 +138,32 @@ fib_route(clicon_handle h,            /* Clicon handle */
     return 0;
 }
 
-/*! Smallest possible RPC declaration for test */
+/*! IETF Routing route-count rpc 
-static int 
+ * @see ietf-routing@2014-10-26.yang  (route-count)
-empty(clicon_handle h,            /* Clicon handle */
+ */
-	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-	  struct client_entry *ce,    /* Client session */
-	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-	  void         *arg)          /* Argument given at register */
-{
-    cprintf(cbret, "<rpc-reply/>");
-    return 0;
-}
-
-/*! IETF Routing route-count rpc */
 static int 
 route_count(clicon_handle h, 
 	    cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-	    struct client_entry *ce,    /* Client session */
 	    cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-	    void         *arg)          /* Argument given at register */
+	    void         *arg,
+	    void         *regarg)          /* Argument given at register */
+{
+    cprintf(cbret, "<rpc-reply><number-of-routes>42</number-of-routes></rpc-reply>");    
+    return 0;
+}
+
+/*! Smallest possible RPC declaration for test 
+ * Yang/XML:
+ * If the RPC operation invocation succeeded and no output parameters
+ * are returned, the <rpc-reply> contains a single <ok/> element defined
+ * in [RFC6241].
+ */
+static int 
+empty(clicon_handle h,            /* Clicon handle */
+      cxobj        *xe,           /* Request: <rpc><xn></rpc> */
+      cbuf         *cbret,        /* Reply eg <rpc-reply>... */
+      void         *arg,          /* client_entry */
+      void         *regarg)       /* Argument given at register */
 {
     cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");
     return 0;
@@ -190,39 +201,6 @@ plugin_statedata(clicon_handle h,
     return retval;
 }
 
-/*! Plugin initialization. Create rpc callbacks
- * plugin_init is called as soon as the plugin has been loaded and is 
- * assumed initialize the plugin's internal state if any as well as register
- * any callbacks, configuration dependencies.
- */
-int
-plugin_init(clicon_handle h)
-{
-    int retval = -1;
-
-    if (notification_timer_setup(h) < 0)
-	goto done;
-    /* Register callback for routing rpc calls */
-    if (backend_rpc_cb_register(h, fib_route, 
-				NULL, 
-				"fib-route"/* Xml tag when callback is made */
-				) < 0)
-	goto done;
-    if (backend_rpc_cb_register(h, route_count, 
-				NULL, 
-				"route-count"/* Xml tag when callback is made */
-				) < 0)
-	goto done;
-    if (backend_rpc_cb_register(h, empty, 
-				NULL, 
-				"empty"/* Xml tag when callback is made */
-				) < 0)
-	goto done;
-    retval = 0;
- done:
-    return retval;
-}
-
 /*! Plugin state reset. Add xml or set state in backend machine.
  * Called in each backend plugin. plugin_reset is called after all plugins
  * have been initialized. This give the application a chance to reset
@@ -250,7 +228,7 @@ plugin_reset(clicon_handle h,
     if (xml_rootchild(xt, 0, &xt) < 0)
 	goto done;
     /* Merge user reset state */
-    if (xmldb_put(h, (char*)db, OP_MERGE, xt) < 0)
+    if (xmldb_put(h, (char*)db, OP_MERGE, xt, NULL) < 0)
 	goto done;
     retval = 0;
  done:
@@ -278,3 +256,54 @@ plugin_start(clicon_handle h,
 {
     return 0;
 }
+
+clixon_plugin_api *clixon_plugin_init(clicon_handle h);
+
+static clixon_plugin_api api = {
+    "example",                              /* name */    
+    clixon_plugin_init,                     /* init */
+    plugin_start,                           /* start */
+    NULL,                                   /* exit */
+    .ca_reset=plugin_reset,                 /* reset */
+    .ca_statedata=plugin_statedata,         /* statedata */
+    .ca_trans_begin=NULL,                   /* trans begin */
+    .ca_trans_validate=transaction_validate,/* trans validate */
+    .ca_trans_complete=NULL,                /* trans complete */
+    .ca_trans_commit=transaction_commit,    /* trans commit */
+    .ca_trans_end=NULL,                     /* trans end */
+    .ca_trans_abort=NULL                    /* trans abort */
+};
+
+/*! Backend plugin initialization
+ * @param[in]  h    Clixon handle
+ * @retval     NULL Error with clicon_err set
+ * @retval     api  Pointer to API struct
+ */
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    clicon_debug(1, "%s backend", __FUNCTION__);
+    if (notification_timer_setup(h) < 0)
+	goto done;
+    /* Register callback for routing rpc calls */
+    if (rpc_callback_register(h, fib_route, 
+			      NULL, 
+			      "fib-route"/* Xml tag when callback is made */
+			      ) < 0)
+	goto done;
+    if (rpc_callback_register(h, route_count, 
+			      NULL, 
+			      "route-count"/* Xml tag when callback is made */
+			      ) < 0)
+	goto done;
+    if (rpc_callback_register(h, empty, 
+			      NULL, 
+			      "empty"/* Xml tag when callback is made */
+			      ) < 0)
+	goto done;
+    /* Return plugin API */
+    return &api;
+ done:
+    return NULL;
+}
+

example/example_backend_nacm.c

@@ -0,0 +1,119 @@
+/*
+ *
+  ***** BEGIN LICENSE BLOCK *****
+ 
+  Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
+
+  This file is part of CLIXON.
+
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+
+  Alternatively, the contents of this file may be used under the terms of
+  the GNU General Public License Version 3 or later (the "GPL"),
+  in which case the provisions of the GPL are applicable instead
+  of those above. If you wish to allow use of your version of this file only
+  under the terms of the GPL, and not to allow others to
+  use your version of this file under the terms of Apache License version 2, indicate
+  your decision by deleting the provisions above and replace them with the 
+  notice and other provisions required by the GPL. If you do not delete
+  the provisions above, a recipient may use your version of this file under
+  the terms of any one of the Apache License version 2 or the GPL.
+
+  ***** END LICENSE BLOCK *****
+
+ * 
+ * IETF yang routing example
+ * Secondary backend for testing more than one backend plugin
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <signal.h>
+#include <unistd.h>
+#include <sys/time.h>
+
+/* clicon */
+#include <cligen/cligen.h>
+
+/* Clicon library functions. */
+#include <clixon/clixon.h>
+
+/* These include signatures for plugin and transaction callbacks. */
+#include <clixon/clixon_backend.h> 
+
+
+/*! Called to get NACM state data
+ * @param[in]    h      Clicon handle
+ * @param[in]    xpath  String with XPATH syntax. or NULL for all
+ * @param[in]    xtop   XML tree, <config/> on entry. 
+ * @retval       0      OK
+ * @retval      -1      Error
+ * @see xmldb_get
+ * @note this example code returns a static statedata used in testing. 
+ * Real code would poll state
+ */
+int 
+nacm_statedata(clicon_handle h, 
+	       char         *xpath,
+	       cxobj        *xstate)
+{
+    int     retval = -1;
+    cxobj **xvec = NULL;
+
+    /* Example of (static) statedata, real code would poll state */
+    if (xml_parse_string("<nacm>"
+			 "<denied-data-writes>0</denied-data-writes>"
+			 "<denied-operations>0</denied-operations>"
+			 "<denied-notifications>0</denied-notifications>"
+			 "</nacm>", NULL, &xstate) < 0)
+	goto done;
+    retval = 0;
+ done:
+    if (xvec)
+	free(xvec);
+    return retval;
+}
+
+int
+plugin_start(clicon_handle h,
+	     int           argc,
+	     char        **argv)
+{
+    return 0;
+}
+
+clixon_plugin_api *clixon_plugin_init(clicon_handle h);
+
+static clixon_plugin_api api = {
+    "nacm",             /* name */           /*--- Common fields.  ---*/
+    clixon_plugin_init, /* init */
+    plugin_start,       /* start */
+    NULL,               /* exit */
+    .ca_reset=NULL,               /* reset */
+    .ca_statedata=nacm_statedata, /* statedata */
+};
+
+/*! Backend plugin initialization
+ * @param[in]  h    Clixon handle
+ * @retval     NULL Error with clicon_err set
+ * @retval     api  Pointer to API struct
+ */
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    clicon_debug(1, "%s backend nacm", __FUNCTION__);
+    return &api;
+}

example/example_cli.c

@@ -52,19 +52,6 @@
 #include <clixon/clixon.h>
 #include <clixon/clixon_cli.h>
 
-/*
- * Plugin initialization
- */
-int
-plugin_init(clicon_handle h)
-{
-    struct timeval tv;
-
-    gettimeofday(&tv, NULL);
-    srandom(tv.tv_usec);
-
-    return 0;
-}
 
 /*! Example cli function */
 int
@@ -107,7 +94,7 @@ fib_route_rpc(clicon_handle h,
     /* User supplied variable in CLI command */
     instance = cvec_find(cvv, "instance"); /* get a cligen variable from vector */
     /* Create XML for fib-route netconf RPC */
-    if (xml_parse_va(&xtop, NULL, "<rpc><fib-route><routing-instance-name>%s</routing-instance-name></fib-route></rpc>", instance) < 0)
+    if (xml_parse_va(&xtop, NULL, "<rpc><fib-route><routing-instance-name>%s</routing-instance-name></fib-route></rpc>", cv_string_get(instance)) < 0)
 	goto done;
     /* Skip top-level */
     xrpc = xml_child_i(xtop, 0);
@@ -125,3 +112,28 @@ fib_route_rpc(clicon_handle h,
     return retval;
 }
 
+static clixon_plugin_api api = {
+    "example",          /* name */
+    clixon_plugin_init, /* init */
+    NULL,               /* start */
+    NULL,               /* exit */
+    .ca_prompt=NULL,    /* cli_prompthook_t */
+    .ca_suspend=NULL,   /* cligen_susp_cb_t */
+    .ca_interrupt=NULL, /* cligen_interrupt_cb_t */
+};
+
+/*! CLI plugin initialization
+ * @param[in]  h    Clixon handle
+ * @retval     NULL Error with clicon_err set
+ * @retval     api  Pointer to API struct
+ */
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    struct timeval tv;
+
+    gettimeofday(&tv, NULL);
+    srandom(tv.tv_usec);
+
+    return &api;
+}

example/example_cli.cli

@@ -1,7 +1,7 @@
 # Common CLI syntax for both server and PMNode operatio mode
-CLICON_MODE="routing";
+CLICON_MODE="example";
 CLICON_PROMPT="%U@%H> ";
-CLICON_PLUGIN="routing_cli";
+CLICON_PLUGIN="example_cli";
 
 # Note, when switching to PT, change datamodel to only @datamodel
 set @datamodel:example, cli_set();
@@ -30,6 +30,7 @@ compare("Compare running and candidate"), compare_dbs((int32)1);
 
 show("Show a particular state of the system"){
     xpath("Show configuration") <xpath:string>("XPATH expression"), show_conf_xpath("candidate");
+    version("Show version"), cli_show_version("candidate", "text", "/");
     compare("Compare candidate and running databases"), compare_dbs((int32)0);{
     		     xml("Show comparison in xml"), compare_dbs((int32)0);
 		     text("Show comparison in text"), compare_dbs((int32)1);

example/example_netconf.c

@@ -46,18 +46,7 @@
 #include <clixon/clixon.h>
 #include <clixon/clixon_netconf.h>
 
-
+/*! Plugin start
-/*
- * Plugin initialization
- */
-int
-plugin_init(clicon_handle h)
-{
-    return 0;
-}
-
-/*
- * Plugin start
  * Called once everything has been initialized, right before
  * the main event loop is entered.
  */
@@ -73,3 +62,40 @@ plugin_exit(clicon_handle h)
     return 0;
 }
 
+/*! Local example netconf rpc callback 
+ */
+int netconf_client_rpc(clicon_handle h, 
+		       cxobj        *xn,      
+		       cbuf         *cbret,    
+		       void         *arg,
+		       void         *regarg)
+{
+    clicon_debug(1, "%s restconf", __FUNCTION__);
+    cprintf(cbret, "<rpc-reply><result>ok</result></rpc-reply>");
+    return 0;
+}
+
+clixon_plugin_api * clixon_plugin_init(clicon_handle h);
+
+static struct clixon_plugin_api api = {
+    "example",          /* name */
+    clixon_plugin_init, /* init */
+    plugin_start,       /* start */
+    plugin_exit         /* exit */
+};
+
+/*! Netconf plugin initialization
+ * @param[in]  h    Clixon handle
+ * @retval     NULL Error with clicon_err set
+ * @retval     api  Pointer to API struct
+ */
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    clicon_debug(1, "%s restconf", __FUNCTION__);
+    /* Register local netconf rpc client (note not backend rpc client) */
+    if (rpc_callback_register(h, netconf_client_rpc, NULL, "client-rpc") < 0)
+	return NULL;
+    return &api;
+}
+

example/example_restconf.c

@@ -0,0 +1,332 @@
+/*
+ *
+  ***** BEGIN LICENSE BLOCK *****
+ 
+  Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
+
+  This file is part of CLIXON.
+
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+
+  Alternatively, the contents of this file may be used under the terms of
+  the GNU General Public License Version 3 or later (the "GPL"),
+  in which case the provisions of the GPL are applicable instead
+  of those above. If you wish to allow use of your version of this file only
+  under the terms of the GPL, and not to allow others to
+  use your version of this file under the terms of Apache License version 2, 
+  indicate your decision by deleting the provisions above and replace them with
+  the  notice and other provisions required by the GPL. If you do not delete
+  the provisions above, a recipient may use your version of this file under
+  the terms of any one of the Apache License version 2 or the GPL.
+
+  ***** END LICENSE BLOCK *****
+ * 
+ */
+
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <ctype.h>
+#include <signal.h>
+#include <fcgi_stdio.h>
+#include <curl/curl.h>
+
+/* cligen */
+#include <cligen/cligen.h>
+
+/* clicon */
+#include <clixon/clixon.h>
+#include <clixon/clixon_restconf.h>
+
+static const char Base64[] =
+	"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+static const char Pad64 = '=';
+
+/* Use http basic auth. Set by starting restonf with:
+   clixon_restconf ... -- -a
+*/
+static int basic_auth = 0;
+
+/* skips all whitespace anywhere.
+   converts characters, four at a time, starting at (or after)
+   src from base - 64 numbers into three 8 bit bytes in the target area.
+   it returns the number of data bytes stored at the target, or -1 on error.
+ @note what is copyright of this?
+ */
+int
+b64_decode(const char *src, 
+	   char       *target, 
+	   size_t      targsize)
+{
+    int tarindex, state, ch;
+    char *pos;
+
+    state = 0;
+    tarindex = 0;
+
+    while ((ch = *src++) != '\0') {
+	if (isspace(ch))	/* Skip whitespace anywhere. */
+	    continue;
+
+	if (ch == Pad64)
+	    break;
+
+	pos = strchr(Base64, ch);
+	if (pos == 0) 		/* A non-base64 character. */
+	    return (-1);
+
+	switch (state) {
+	case 0:
+	    if (target) {
+		if ((size_t)tarindex >= targsize)
+		    return (-1);
+		target[tarindex] = (pos - Base64) << 2;
+	    }
+	    state = 1;
+	    break;
+	case 1:
+	    if (target) {
+		if ((size_t)tarindex + 1 >= targsize)
+		    return (-1);
+		target[tarindex]   |=  (pos - Base64) >> 4;
+		target[tarindex+1]  = ((pos - Base64) & 0x0f)
+		    << 4 ;
+	    }
+	    tarindex++;
+	    state = 2;
+	    break;
+	case 2:
+	    if (target) {
+		if ((size_t)tarindex + 1 >= targsize)
+		    return (-1);
+		target[tarindex]   |=  (pos - Base64) >> 2;
+		target[tarindex+1]  = ((pos - Base64) & 0x03)
+		    << 6;
+	    }
+	    tarindex++;
+	    state = 3;
+	    break;
+	case 3:
+	    if (target) {
+		if ((size_t)tarindex >= targsize)
+		    return (-1);
+		target[tarindex] |= (pos - Base64);
+	    }
+	    tarindex++;
+	    state = 0;
+	    break;
+	default:
+	    return -1;
+	}
+    }
+
+    /*
+     * We are done decoding Base-64 chars.  Let's see if we ended
+     * on a byte boundary, and/or with erroneous trailing characters.
+     */
+
+    if (ch == Pad64) {		/* We got a pad char. */
+	ch = *src++;		/* Skip it, get next. */
+	switch (state) {
+	case 0:		/* Invalid = in first position */
+	case 1:		/* Invalid = in second position */
+	    return (-1);
+
+	case 2:		/* Valid, means one byte of info */
+			/* Skip any number of spaces. */
+	    for ((void)NULL; ch != '\0'; ch = *src++)
+		if (!isspace(ch))
+		    break;
+	    /* Make sure there is another trailing = sign. */
+	    if (ch != Pad64)
+		return (-1);
+	    ch = *src++;		/* Skip the = */
+	    /* Fall through to "single trailing =" case. */
+	    /* FALLTHROUGH */
+
+	case 3:		/* Valid, means two bytes of info */
+			/*
+			 * We know this char is an =.  Is there anything but
+			 * whitespace after it?
+			 */
+	    for ((void)NULL; ch != '\0'; ch = *src++)
+		if (!isspace(ch))
+		    return (-1);
+
+	    /*
+	     * Now make sure for cases 2 and 3 that the "extra"
+	     * bits that slopped past the last full byte were
+	     * zeros.  If we don't check them, they become a
+	     * subliminal channel.
+	     */
+	    if (target && target[tarindex] != 0)
+		return (-1);
+	}
+    } else {
+	/*
+	 * We ended by seeing the end of the string.  Make sure we
+	 * have no partial bytes lying around.
+	 */
+	if (state != 0)
+	    return (-1);
+    }
+
+    return (tarindex);
+}
+
+/*! Process a rest request that requires (cookie) "authentication"
+ * @param[in]  h        Clixon handle
+ * @param[in]  arg      Argument. Here: Fastcgi request handle
+ * @retval -1  Fatal error
+ * @retval  0  Unauth
+ * @retval  1  Auth
+ * @note: Three hardwired users: adm1, wilma, guest w password "bar".
+ * Enabled by passing -- -a to the main function
+ */
+int
+example_restconf_credentials(clicon_handle h,     
+			     void         *arg)
+{
+    int     retval = -1;
+    FCGX_Request *r = (FCGX_Request *)arg;
+    cxobj  *xt = NULL;
+    char   *auth;
+    char   *user = NULL;
+    char   *passwd;
+    char   *passwd2 = "";
+    size_t  authlen;
+    cbuf   *cb = NULL;
+    int     ret;
+
+    /* HTTP basic authentication not enabled, pass with user "none" */
+    if (basic_auth==0)
+	goto ok;
+    /* At this point in the code we must use HTTP basic authentication */
+    if ((auth = FCGX_GetParam("HTTP_AUTHORIZATION", r->envp)) == NULL)
+	goto fail; 
+   if (strlen(auth) < strlen("Basic "))
+	goto fail;
+    if (strncmp("Basic ", auth, strlen("Basic ")))
+	goto fail;
+    auth += strlen("Basic ");
+    authlen = strlen(auth)*2;
+    if ((user = malloc(authlen)) == NULL){
+	clicon_err(OE_UNIX, errno, "malloc");
+	goto done;
+    }
+    memset(user, 0, authlen);
+    if ((ret = b64_decode(auth, user, authlen)) < 0)
+	goto done;
+    /* auth string is on the format user:passwd */
+    if ((passwd = index(user,':')) == NULL)
+	goto fail;
+    *passwd = '\0';
+    passwd++;
+    clicon_debug(1, "%s http user:%s passwd:%s", __FUNCTION__, user, passwd);
+    /* Here get auth sub-tree whjere all the users are */
+    if ((cb = cbuf_new()) == NULL)
+	goto done;
+    /* Hardcoded user/passwd */
+    if (strcmp(user, "wilma")==0 || strcmp(user, "adm1")==0 ||
+	strcmp(user, "quest")==0){
+	passwd2 = "bar";
+    }
+    if (strcmp(passwd, passwd2))
+	goto fail;
+    retval = 1;
+    clicon_debug(1, "%s user:%s", __FUNCTION__, user);
+    if (clicon_username_set(h, user) < 0)
+	goto done;
+ ok:   /* authenticated */
+    retval = 1;
+ done: /* error */
+    clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
+    if (user)
+	free(user);
+    if (cb)
+	cbuf_free(cb);
+    if (xt)
+        xml_free(xt);
+    return retval;
+ fail:  /* unauthenticated */
+    retval = 0;
+    goto done;
+}
+
+/*! Local example restconf rpc callback 
+ */
+int
+restconf_client_rpc(clicon_handle h, 
+		    cxobj        *xn,      
+		    cbuf         *cbret,    
+		    void         *arg,
+		    void         *regarg)
+{
+    //    FCGX_Request *r = (FCGX_Request *)arg;
+    clicon_debug(1, "%s", __FUNCTION__);
+    cprintf(cbret, "<rpc-reply><result>ok</result></rpc-reply>");
+    return 0;
+}
+
+/*! Start example restonf plugin. Set authentication method
+ * Arguments are argc/argv after --
+ * Currently defined: -a  enable http basic authentication
+ * Note hardwired users adm1, wilma and guest
+ */
+int
+example_restconf_start(clicon_handle h,
+		       int           argc,
+		       char        **argv)
+{
+    char c;
+
+    clicon_debug(1, "%s argc:%d", __FUNCTION__, argc);
+    optind = 1;
+    opterr = 0;
+    while ((c = getopt(argc, argv, "a")) != -1){
+	switch (c) {
+	case 'a':
+	    basic_auth = 1;
+	    break;
+	default:
+	    break;
+	}
+    }
+    return 0;
+}
+
+clixon_plugin_api * clixon_plugin_init(clicon_handle h);
+
+static clixon_plugin_api api = {
+    "example",           /* name */
+    clixon_plugin_init,  /* init */
+    example_restconf_start,/* start */
+    NULL,                /* exit */
+    .ca_auth=example_restconf_credentials   /* auth */
+};
+
+/*! Restconf plugin initialization
+ * @param[in]  h    Clixon handle
+ * @retval     NULL Error with clicon_err set
+ * @retval     api  Pointer to API struct
+ */
+clixon_plugin_api *
+clixon_plugin_init(clicon_handle h)
+{
+    clicon_debug(1, "%s restconf", __FUNCTION__);
+    /* Register local netconf rpc client (note not backend rpc client) */
+    if (rpc_callback_register(h, restconf_client_rpc, NULL, "client-rpc") < 0)
+	return NULL;
+    return &api;
+}

example/routing.conf.local

@@ -1,17 +0,0 @@
-# Main YANG module first parsed by parser (in CLICON_YANG_DIR). eg clicon.yang.
-
-# Startup CLI mode. This should match the CLICON_MODE in your startup clispec file
-CLICON_CLI_MODE         routing
-
-# Option used to construct initial yang file:
-#     <module>[@<revision>]
-#CLICON_YANG_MODULE_MAIN ietf-ip
-CLICON_YANG_MODULE_MAIN example
-
-# Option used to construct initial yang file:
-#     <module>[@<revision>]
-#CLICON_YANG_MODULE_REVISION 2014-06-16
-
-# Set to 0 if you want CLI to wrap to next line.
-# Set to 1 if you  want CLI to scroll sideways when approaching right margin
-CLICON_CLI_LINESCROLLING      0

example/routing.xml

@@ -1,18 +0,0 @@
-<config>
-  <CLICON_CONFIGFILE>/usr/local/etc/routing.xml</CLICON_CONFIGFILE>
-  <CLICON_YANG_DIR>/usr/local/share/routing/yang</CLICON_YANG_DIR>
-  <CLICON_YANG_MODULE_MAIN>example</CLICON_YANG_MODULE_MAIN>
-  <CLICON_CLI_MODE>routing</CLICON_CLI_MODE>
-  <CLICON_BACKEND_DIR>/usr/local/lib/routing/backend</CLICON_BACKEND_DIR>
-  <CLICON_NETCONF_DIR>/usr/local/lib/routing/netconf</CLICON_NETCONF_DIR>
-  <CLICON_RESTCONF_DIR>/usr/local/lib/routing/restconf</CLICON_RESTCONF_DIR>
-  <CLICON_CLI_DIR>/usr/local/lib/routing/cli</CLICON_CLI_DIR>
-  <CLICON_CLISPEC_DIR>/usr/local/lib/routing/clispec</CLICON_CLISPEC_DIR>
-  <CLICON_SOCK>/usr/local/var/routing/routing.sock</CLICON_SOCK>
-  <CLICON_BACKEND_PIDFILE>/usr/local/var/routing/routing.pidfile</CLICON_BACKEND_PIDFILE>
-  <CLICON_CLI_GENMODEL_COMPLETION>1</CLICON_CLI_GENMODEL_COMPLETION>
-  <CLICON_XMLDB_DIR>/usr/local/var/routing</CLICON_XMLDB_DIR>
-  <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
-  <CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
-  <CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>
-</config>

include/clixon_config.h.in

@@ -27,9 +27,6 @@
 /* Define to 1 if you have the <crypt.h> header file. */
 #undef HAVE_CRYPT_H
 
-/* Define to 1 if you have the <depot.h> header file. */
-#undef HAVE_DEPOT_H
-
 /* Define to 1 if you have the `inet_aton' function. */
 #undef HAVE_INET_ATON
 
@@ -54,9 +51,6 @@
 /* Define to 1 if you have the `nsl' library (-lnsl). */
 #undef HAVE_LIBNSL
 
-/* Define to 1 if you have the `qdbm' library (-lqdbm). */
-#undef HAVE_LIBQDBM
-
 /* Define to 1 if you have the `socket' library (-lsocket). */
 #undef HAVE_LIBSOCKET
 
@@ -66,9 +60,6 @@
 /* Define to 1 if you have the <memory.h> header file. */
 #undef HAVE_MEMORY_H
 
-/* Define to 1 if you have the <qdbm/depot.h> header file. */
-#undef HAVE_QDBM_DEPOT_H
-
 /* Define to 1 if you have the `sigaction' function. */
 #undef HAVE_SIGACTION
 

include/clixon_config.h.in~

@@ -0,0 +1,142 @@
+/* include/clixon_config.h.in.  Generated from configure.ac by autoheader.  */
+
+/* Clixon data dir for system yang files etc */
+#undef CLIXON_DATADIR
+
+/* Location for apps to find default config file */
+#undef CLIXON_DEFAULT_CONFIG
+
+/* Clixon major release */
+#undef CLIXON_VERSION_MAJOR
+
+/* Clixon minor release */
+#undef CLIXON_VERSION_MINOR
+
+/* Clixon path version */
+#undef CLIXON_VERSION_PATCH
+
+/* Clixon version string */
+#undef CLIXON_VERSION_STRING
+
+/* Define to 1 if you have the `alphasort' function. */
+#undef HAVE_ALPHASORT
+
+/* Define to 1 if you have the <cligen/cligen.h> header file. */
+#undef HAVE_CLIGEN_CLIGEN_H
+
+/* Define to 1 if you have the <crypt.h> header file. */
+#undef HAVE_CRYPT_H
+
+/* Define to 1 if you have the <depot.h> header file. */
+#undef HAVE_DEPOT_H
+
+/* Define to 1 if you have the `inet_aton' function. */
+#undef HAVE_INET_ATON
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#undef HAVE_INTTYPES_H
+
+/* Define to 1 if you have the `cligen' library (-lcligen). */
+#undef HAVE_LIBCLIGEN
+
+/* Define to 1 if you have the `crypt' library (-lcrypt). */
+#undef HAVE_LIBCRYPT
+
+/* Define to 1 if you have the `dl' library (-ldl). */
+#undef HAVE_LIBDL
+
+/* Define to 1 if you have the `fcgi' library (-lfcgi). */
+#undef HAVE_LIBFCGI
+
+/* Define to 1 if you have the `m' library (-lm). */
+#undef HAVE_LIBM
+
+/* Define to 1 if you have the `nsl' library (-lnsl). */
+#undef HAVE_LIBNSL
+
+/* Define to 1 if you have the `qdbm' library (-lqdbm). */
+#undef HAVE_LIBQDBM
+
+/* Define to 1 if you have the `socket' library (-lsocket). */
+#undef HAVE_LIBSOCKET
+
+/* Define to 1 if you have the <linux/if_vlan.h> header file. */
+#undef HAVE_LINUX_IF_VLAN_H
+
+/* Define to 1 if you have the <memory.h> header file. */
+#undef HAVE_MEMORY_H
+
+/* Define to 1 if you have the <qdbm/depot.h> header file. */
+#undef HAVE_QDBM_DEPOT_H
+
+/* Define to 1 if you have the `sigaction' function. */
+#undef HAVE_SIGACTION
+
+/* Define to 1 if you have the `sigvec' function. */
+#undef HAVE_SIGVEC
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#undef HAVE_STDINT_H
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#undef HAVE_STDLIB_H
+
+/* Define to 1 if you have the <strings.h> header file. */
+#undef HAVE_STRINGS_H
+
+/* Define to 1 if you have the <string.h> header file. */
+#undef HAVE_STRING_H
+
+/* Define to 1 if you have the `strlcpy' function. */
+#undef HAVE_STRLCPY
+
+/* Define to 1 if you have the `strndup' function. */
+#undef HAVE_STRNDUP
+
+/* Define to 1 if you have the `strsep' function. */
+#undef HAVE_STRSEP
+
+/* Define to 1 if you have the `strverscmp' function. */
+#undef HAVE_STRVERSCMP
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#undef HAVE_SYS_STAT_H
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#undef HAVE_SYS_TYPES_H
+
+/* Define to 1 if you have the <sys/ucred.h> header file. */
+#undef HAVE_SYS_UCRED_H
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#undef HAVE_UNISTD_H
+
+/* Define to 1 if you have the `versionsort' function. */
+#undef HAVE_VERSIONSORT
+
+/* Define to the address where bug reports for this package should be sent. */
+#undef PACKAGE_BUGREPORT
+
+/* Define to the full name of this package. */
+#undef PACKAGE_NAME
+
+/* Define to the full name and version of this package. */
+#undef PACKAGE_STRING
+
+/* Define to the one symbol short name of this package. */
+#undef PACKAGE_TARNAME
+
+/* Define to the home page for this package. */
+#undef PACKAGE_URL
+
+/* Define to the version of this package. */
+#undef PACKAGE_VERSION
+
+/* Define to 1 if you have the ANSI C header files. */
+#undef STDC_HEADERS
+
+/* Define to 1 if `lex' declares `yytext' as a `char *' by default, not a
+   `char[]'. */
+#undef YYTEXT_POINTER
+
+#include <clixon_custom.h>

include/clixon_config.h~

@@ -0,0 +1,134 @@
+/* include/clixon_config.h.  Generated from clixon_config.h.in by configure.  */
+/* include/clixon_config.h.in.  Generated from configure.ac by autoheader.  */
+
+/* Clixon major release */
+#define CLIXON_VERSION_MAJOR 3
+
+/* Clixon minor release */
+#define CLIXON_VERSION_MINOR 3
+
+/* Clixon path version */
+#define CLIXON_VERSION_PATCH 2
+
+/* Clixon version string */
+#define CLIXON_VERSION_STRING "3.3.2"
+
+/* Define to 1 if you have the `alphasort' function. */
+#define HAVE_ALPHASORT 1
+
+/* Define to 1 if you have the <cligen/cligen.h> header file. */
+#define HAVE_CLIGEN_CLIGEN_H 1
+
+/* Define to 1 if you have the <crypt.h> header file. */
+#define HAVE_CRYPT_H 1
+
+/* Define to 1 if you have the <depot.h> header file. */
+/* #undef HAVE_DEPOT_H */
+
+/* Define to 1 if you have the `inet_aton' function. */
+#define HAVE_INET_ATON 1
+
+/* Define to 1 if you have the <inttypes.h> header file. */
+#define HAVE_INTTYPES_H 1
+
+/* Define to 1 if you have the `crypt' library (-lcrypt). */
+#define HAVE_LIBCRYPT 1
+
+/* Define to 1 if you have the `dl' library (-ldl). */
+#define HAVE_LIBDL 1
+
+/* Define to 1 if you have the `fcgi' library (-lfcgi). */
+#define HAVE_LIBFCGI 1
+
+/* Define to 1 if you have the `m' library (-lm). */
+#define HAVE_LIBM 1
+
+/* Define to 1 if you have the `nsl' library (-lnsl). */
+#define HAVE_LIBNSL 1
+
+/* Define to 1 if you have the `qdbm' library (-lqdbm). */
+#define HAVE_LIBQDBM 1
+
+/* Define to 1 if you have the `socket' library (-lsocket). */
+/* #undef HAVE_LIBSOCKET */
+
+/* Define to 1 if you have the <linux/if_vlan.h> header file. */
+#define HAVE_LINUX_IF_VLAN_H 1
+
+/* Define to 1 if you have the <memory.h> header file. */
+#define HAVE_MEMORY_H 1
+
+/* Define to 1 if you have the <qdbm/depot.h> header file. */
+#define HAVE_QDBM_DEPOT_H 1
+
+/* Define to 1 if you have the `sigaction' function. */
+#define HAVE_SIGACTION 1
+
+/* Define to 1 if you have the `sigvec' function. */
+#define HAVE_SIGVEC 1
+
+/* Define to 1 if you have the <stdint.h> header file. */
+#define HAVE_STDINT_H 1
+
+/* Define to 1 if you have the <stdlib.h> header file. */
+#define HAVE_STDLIB_H 1
+
+/* Define to 1 if you have the <strings.h> header file. */
+#define HAVE_STRINGS_H 1
+
+/* Define to 1 if you have the <string.h> header file. */
+#define HAVE_STRING_H 1
+
+/* Define to 1 if you have the `strlcpy' function. */
+/* #undef HAVE_STRLCPY */
+
+/* Define to 1 if you have the `strndup' function. */
+#define HAVE_STRNDUP 1
+
+/* Define to 1 if you have the `strsep' function. */
+#define HAVE_STRSEP 1
+
+/* Define to 1 if you have the `strverscmp' function. */
+#define HAVE_STRVERSCMP 1
+
+/* Define to 1 if you have the <sys/stat.h> header file. */
+#define HAVE_SYS_STAT_H 1
+
+/* Define to 1 if you have the <sys/types.h> header file. */
+#define HAVE_SYS_TYPES_H 1
+
+/* Define to 1 if you have the <sys/ucred.h> header file. */
+/* #undef HAVE_SYS_UCRED_H */
+
+/* Define to 1 if you have the <unistd.h> header file. */
+#define HAVE_UNISTD_H 1
+
+/* Define to 1 if you have the `versionsort' function. */
+#define HAVE_VERSIONSORT 1
+
+/* Define to the address where bug reports for this package should be sent. */
+#define PACKAGE_BUGREPORT ""
+
+/* Define to the full name of this package. */
+#define PACKAGE_NAME ""
+
+/* Define to the full name and version of this package. */
+#define PACKAGE_STRING ""
+
+/* Define to the one symbol short name of this package. */
+#define PACKAGE_TARNAME ""
+
+/* Define to the home page for this package. */
+#define PACKAGE_URL ""
+
+/* Define to the version of this package. */
+#define PACKAGE_VERSION ""
+
+/* Define to 1 if you have the ANSI C header files. */
+#define STDC_HEADERS 1
+
+/* Define to 1 if `lex' declares `yytext' as a `char *' by default, not a
+   `char[]'. */
+#define YYTEXT_POINTER 1
+
+#include <clixon_custom.h>

lib/clixon/Makefile.in

@@ -44,8 +44,8 @@ depend:
 install:
 
 install-include:	
-	install -m 755 -d $(DESTDIR)$(includedir)/clixon 
+	install -m 0755 -d $(DESTDIR)$(includedir)/clixon
-	install -m 644 *.h $(DESTDIR)$(includedir)/clixon 
+	install -m 0644 *.h $(DESTDIR)$(includedir)/clixon
 
 uninstall:
 	rm -rf $(DESTDIR)$(includedir)/clixon

lib/clixon/clixon.h.in

@@ -83,6 +83,7 @@
 #include <clixon/clixon_xml_db.h>
 #include <clixon/clixon_xsl.h>
 #include <clixon/clixon_json.h>
+#include <clixon/clixon_netconf_lib.h>
 
 /*
  * Global variables generated by Makefile

lib/clixon/clixon_handle.h

@@ -50,6 +50,9 @@ typedef struct {float a;} *clicon_handle;
 typedef void *clicon_handle;
 #endif
 
+/* The dynamicically loadable plugin object handle (should be in clixon_plugin.h) */
+typedef void *plghndl_t;
+
 /*
  * Prototypes
  */

lib/clixon/clixon_netconf_lib.h

@@ -0,0 +1,66 @@
+/*
+ *
+  ***** BEGIN LICENSE BLOCK *****
+ 
+  Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
+
+  This file is part of CLIXON.
+
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+
+  Alternatively, the contents of this file may be used under the terms of
+  the GNU General Public License Version 3 or later (the "GPL"),
+  in which case the provisions of the GPL are applicable instead
+  of those above. If you wish to allow use of your version of this file only
+  under the terms of the GPL, and not to allow others to
+  use your version of this file under the terms of Apache License version 2, 
+  indicate your decision by deleting the provisions above and replace them with
+  the  notice and other provisions required by the GPL. If you do not delete
+  the provisions above, a recipient may use your version of this file under
+  the terms of any one of the Apache License version 2 or the GPL.
+
+  ***** END LICENSE BLOCK *****
+
+ * Netconf library functions. See RFC6241
+ * 
+ */
+#ifndef _CLIXON_NETCONF_LIB_H
+#define _CLIXON_NETCONF_LIB_H
+
+/*
+ * Prototypes
+ */
+int netconf_in_use(cbuf *cb, char *type, char *message);
+int netconf_invalid_value(cbuf *cb, char *type, char *message);
+int netconf_too_big(cbuf *cb, char *type, char *message);
+int netconf_missing_attribute(cbuf *cb,	char *type, char *info, char *message);
+int netconf_bad_attribute(cbuf *cb, char *type, char *info, char *message);
+int netconf_unknown_attribute(cbuf *cb,	char *type, char *info, char *message);
+int netconf_missing_element(cbuf *cb, char *type, char *info, char *message);
+int netconf_bad_element(cbuf *cb, char *type, char *info, char *message);
+int netconf_unknown_element(cbuf *cb, char *type, char *info, char *message);
+int netconf_unknown_namespace(cbuf *cb, char *type, char *info, char *message);
+int netconf_access_denied(cbuf *cb, char *type, char *message);
+int netconf_access_denied_xml(cxobj **xret, char *type, char *message);
+int netconf_lock_denied(cbuf *cb, char *info, char *message);
+int netconf_resource_denied(cbuf *cb, char *type, char *message);
+int netconf_rollback_failed(cbuf *cb, char *type, char *message);
+int netconf_data_exists(cbuf *cb, char *message);
+int netconf_data_missing(cbuf *cb, char *message);
+int netconf_operation_not_supported(cbuf *cb, char *type, char *message);
+int netconf_operation_failed(cbuf *cb, char *type, char *message);
+int netconf_operation_failed_xml(cxobj **xret, char *type, char *message);
+int netconf_malformed_message(cbuf *cb, char *message);
+int netconf_malformed_message_xml(cxobj **xret, char *message);
+
+#endif /* _CLIXON_NETCONF_LIB_H */

lib/clixon/clixon_options.h

@@ -43,23 +43,26 @@
  */
 /* default group membership to access config unix socket */
 #define CLICON_SOCK_GROUP "clicon"
-/* Default name of master plugin */
-#define CLICON_MASTER_PLUGIN "master"
 
 /*
  * Types
  */
 
 /*! Controls how keywords a generated in CLI syntax / prints from object model
- * Example syntax a.b[] $!x $y:
+ * Example YANG: 
- * NONE: a b <x> <y>;
+ *  list a {
- * VARS: a b <x> y <y>;
+ *    key x;
- * ALL:  a b x <x> y <y>;
+ *    leaf x;   
+ *    leaf y;   
+ *  }
+ * NONE: a <x> <y>;
+ * VARS: a <x> y <y>;
+ * ALL:  a x <x> y <y>;
  */
 enum genmodel_type{
     GT_ERR =-1, /* Error  */
     GT_NONE=0,  /* No extra keywords */
-    GT_VARS,    /* Keywords on non-index variables */
+    GT_VARS,    /* Keywords on non-key variables */
     GT_ALL,     /* Keywords on all variables */
 };
 
@@ -137,9 +140,6 @@ static inline char *clicon_sock_group(clicon_handle h){
 static inline char *clicon_backend_pidfile(clicon_handle h){
     return clicon_option_str(h, "CLICON_BACKEND_PIDFILE");
 }
-static inline char *clicon_master_plugin(clicon_handle h){
-    return clicon_option_str(h, "CLICON_MASTER_PLUGIN");
-}
 static inline char *clicon_xmldb_dir(clicon_handle h){
     return clicon_option_str(h, "CLICON_XMLDB_DIR");
 }
@@ -179,4 +179,9 @@ int clicon_xmldb_api_set(clicon_handle h, void *xa_api);
 void *clicon_xmldb_handle_get(clicon_handle h);
 int clicon_xmldb_handle_set(clicon_handle h, void *xh);
 
+/**/
+/* Set and get authorized user name */
+char *clicon_username_get(clicon_handle h);
+int clicon_username_set(clicon_handle h, void *username);
+
 #endif  /* _CLIXON_OPTIONS_H_ */

lib/clixon/clixon_plugin.h

@@ -38,14 +38,32 @@
 #ifndef _CLIXON_PLUGIN_H_
 #define _CLIXON_PLUGIN_H_
 
+/*
+ * Constants
+ */
+/* Hardcoded plugin symbol. Must exist in all plugins to kickstart */
+#define CLIXON_PLUGIN_INIT     "clixon_plugin_init" 
+
 /*
  * Types
  */
-/* The dynamicically loadable plugin object handle */
+/* Dynamicically loadable plugin object handle. @see return value of dlopen(3) */
 typedef void *plghndl_t;
 
-/* Find plugin by name callback.  XXX Should be clicon internal */
+/*! Registered RPC callback function 
-typedef void *(find_plugin_t)(clicon_handle, char *); 
+ * @param[in]  h       Clicon handle 
+ * @param[in]  xn      Request: <rpc><xn></rpc> 
+ * @param[out] cbret   Return xml tree, eg <rpc-reply>..., <rpc-error.. 
+ * @param[in]  arg     Domain specific arg, ec client-entry or FCGX_Request 
+ * @param[in]  regarg  User argument given at rpc_callback_register() 
+ */
+typedef int (*clicon_rpc_cb)(
+    clicon_handle h,       
+    cxobj        *xn,      
+    cbuf         *cbret,   
+    void         *arg,     
+    void         *regarg   
+);  
 
 /*
  * Prototypes
@@ -56,37 +74,135 @@ typedef void *(find_plugin_t)(clicon_handle, char *);
  *   Backend see config_plugin.c
  */
 
-/*! Called when plugin loaded. Only mandadory callback. All others optional 
- * @see plginit_t
- */
-#define PLUGIN_INIT            "plugin_init"
-typedef int (plginit_t)(clicon_handle);	 	       /* Plugin Init */
-
 /* Called when backend started with cmd-line arguments from daemon call. 
- * @see plgstart_t
+ * Call plugin start functions with argc/argv multiple arguments.
+ * Typically the argc/argv are the ones appearing after "--", eg
+ * clicon_cli -f /etc/clicon.xml -- -a myopt
  */
-#define PLUGIN_START           "plugin_start"
 typedef int (plgstart_t)(clicon_handle, int, char **); /* Plugin start */
 
 /* Called just before plugin unloaded. 
  */
-#define PLUGIN_EXIT            "plugin_exit"
 typedef int (plgexit_t)(clicon_handle);		       /* Plugin exit */
 
-/*! Called by restconf
+/*! Called by restconf to check credentials and return username
- * Returns 0 if credentials OK, -1 if failed
  */
-#define PLUGIN_CREDENTIALS      "plugin_credentials"
+
-/* Plugin credentials 
+/* Plugin authorization. Set username option (or not)
- * username should be freed after use
+ * @param[in]  Clicon handle
+ * @param[in]  void*, eg Fastcgihandle request restconf
+ * @retval   0 if credentials OK
+ * @retval  -1 credentials not OK
  */
-typedef int (plgcredentials_t)(clicon_handle, void *, char **username);
+typedef int (plgauth_t)(clicon_handle, void *);
 
-/* Find a function in global namespace or a plugin. XXX clicon internal */
+typedef int (plgreset_t)(clicon_handle h, const char *db); /* Reset system status */
-void *clicon_find_func(clicon_handle h, char *plugin, char *func);
+typedef int (plgstatedata_t)(clicon_handle h, char *xpath, cxobj *xtop);
 
-plghndl_t plugin_load (clicon_handle h, char *file, int dlflags);
+typedef void *transaction_data;
 
-int plugin_unload(clicon_handle h, plghndl_t *handle);
+/* Transaction callbacks */
+typedef int (trans_cb_t)(clicon_handle h, transaction_data td); 
+
+/* Hook to override default prompt with explicit function
+ * Format prompt before each getline 
+ * @param[in] h      Clicon handle
+ * @param[in] mode   Cligen syntax mode
+ * @retval    prompt Prompt to prepend all CLigen command lines
+ */
+typedef char *(cli_prompthook_t)(clicon_handle, char *mode);
+
+/* plugin init struct for the api 
+ * Note: Implicit init function
+ */
+struct clixon_plugin_api;
+typedef struct clixon_plugin_api* (plginit2_t)(clicon_handle);    /* Clixon plugin Init */
+
+struct clixon_plugin_api{
+    /*--- Common fields.  ---*/
+    char              ca_name[PATH_MAX]; /* Name of plugin (given by plugin) */
+    plginit2_t       *ca_init;           /* Clixon plugin Init (implicit) */
+    plgstart_t       *ca_start;          /* Plugin start */
+    plgexit_t        *ca_exit;	         /* Plugin exit */
+    union {
+	struct {
+	    cli_prompthook_t *ci_prompt;         /* Prompt hook */
+	    cligen_susp_cb_t *ci_suspend;        /* Ctrl-Z hook, see cligen getline */
+	    cligen_interrupt_cb_t *ci_interrupt; /* Ctrl-C, see cligen getline */
+	} cau_cli;
+	struct {
+	    plgauth_t        *cr_auth;           /* Auth credentials */
+	} cau_restconf;
+	struct {
+	} cau_netconf;
+	struct {
+	    plgreset_t       *cb_reset;          /* Reset system status (backend only) */
+	    plgstatedata_t   *cb_statedata;      /* Get state data from plugin (backend only) */
+	    trans_cb_t       *cb_trans_begin;	 /* Transaction start */
+	    trans_cb_t       *cb_trans_validate; /* Transaction validation */
+	    trans_cb_t       *cb_trans_complete; /* Transaction validation complete */
+	    trans_cb_t       *cb_trans_commit;   /* Transaction commit */
+	    trans_cb_t       *cb_trans_end;	 /* Transaction completed  */
+	    trans_cb_t       *cb_trans_abort;	 /* Transaction aborted */    
+	} cau_backend;
+
+    } u;
+};
+/* Access fields */
+#define ca_prompt         u.cau_cli.ci_prompt
+#define ca_suspend        u.cau_cli.ci_suspend
+#define ca_interrupt      u.cau_cli.ci_interrupt
+#define ca_auth           u.cau_restconf.cr_auth
+#define ca_reset          u.cau_backend.cb_reset
+#define ca_statedata      u.cau_backend.cb_statedata
+#define ca_trans_begin    u.cau_backend.cb_trans_begin
+#define ca_trans_validate u.cau_backend.cb_trans_validate
+#define ca_trans_complete u.cau_backend.cb_trans_complete
+#define ca_trans_commit   u.cau_backend.cb_trans_commit
+#define ca_trans_end      u.cau_backend.cb_trans_end
+#define ca_trans_abort    u.cau_backend.cb_trans_abort
+
+typedef struct clixon_plugin_api clixon_plugin_api;
+
+/* Internal plugin structure with dlopen() handle and plugin_api
+ */
+struct clixon_plugin{
+    char              cp_name[PATH_MAX]; /* Plugin filename. Note api ca_name is given by plugin itself */
+    plghndl_t         cp_handle;  /* Handle to plugin using dlopen(3) */
+    clixon_plugin_api cp_api;
+};
+typedef struct clixon_plugin clixon_plugin;
+
+/*
+ * Prototypes
+ */
+
+/*! Plugin initialization function. Must appear in all plugins
+ * @param[in]  h    Clixon handle
+ * @retval     api  Pointer to API struct
+ * @see CLIXON_PLUGIN_INIT  default symbol 
+ */
+clixon_plugin_api *clixon_plugin_init(clicon_handle h);
+
+clixon_plugin *clixon_plugin_each(clicon_handle h, clixon_plugin *cpprev);
+
+clixon_plugin *clixon_plugin_each_revert(clicon_handle h, clixon_plugin *cpprev, int nr);
+
+clixon_plugin *clixon_plugin_find(clicon_handle h, char *name);
+
+int clixon_plugins_load(clicon_handle h, char *function, char *dir, char *regexp);
+
+int clixon_plugin_start(clicon_handle h, int argc, char **argv);
+
+int clixon_plugin_exit(clicon_handle h);
+
+int clixon_plugin_auth(clicon_handle h, void *arg);
+
+/* rpc callback API */
+int rpc_callback_register(clicon_handle h, clicon_rpc_cb cb, void *arg, char *tag);
+
+int rpc_callback_delete_all(void);
+
+int rpc_callback_call(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg);
 
 #endif  /* _CLIXON_PLUGIN_H_ */

lib/clixon/clixon_string.h

@@ -74,8 +74,9 @@ static inline char * strdup4(char *str)
 char **clicon_strsep(char *string, char *delim, int  *nvec0);
 char *clicon_strjoin (int argc, char **argv, char *delim);
 int str2cvec(char *string, char delim1, char delim2, cvec **cvp);
-int percent_encode(char *str, char **escp);
+int uri_percent_encode(char *str, char **escp);
-int percent_decode(char *esc, char **str);
+int uri_percent_decode(char *esc, char **str);
+int xml_chardata_encode(char *str, char **escp);
 const char *clicon_int2str(const map_str2int *mstab, int i);
 int clicon_str2int(const map_str2int *mstab, char *str);
 

lib/clixon/clixon_xml.h

@@ -142,8 +142,11 @@ int       clicon_xml2file(FILE *f, cxobj *xn, int level, int prettyprint);
 int       clicon_xml2cbuf(cbuf *xf, cxobj *xn, int level, int prettyprint);
 int       xml_parse_file(int fd, char *endtag, yang_spec *yspec, cxobj **xt);
 int       xml_parse_string(const char *str, yang_spec *yspec, cxobj **xml_top);
+#if defined(__GNUC__) && __GNUC__ >= 3
+int       xml_parse_va(cxobj **xt, yang_spec *yspec, const char *format, ...)  __attribute__ ((format (printf, 3, 4)));
+#else
 int       xml_parse_va(cxobj **xt, yang_spec *yspec, const char *format, ...);
-
+#endif
 int       xmltree2cbuf(cbuf *cb, cxobj *x, int level);
 int       xml_copy_one(cxobj *xn0, cxobj *xn1);
 int       xml_copy(cxobj *x0, cxobj *x1);

lib/clixon/clixon_xml_db.h

@@ -78,7 +78,7 @@ typedef int (xmldb_setopt_t)(xmldb_handle xh, char *optname, void *value);
 typedef int (xmldb_get_t)(xmldb_handle xh, const char *db, char *xpath, int config, cxobj **xtop);
 
 /* Type of xmldb put function */
-typedef int (xmldb_put_t)(xmldb_handle xh, const char *db, enum operation_type op, cxobj *xt);
+typedef int (xmldb_put_t)(xmldb_handle xh, const char *db, enum operation_type op, cxobj *xt, cbuf *cbret);
 
 /* Type of xmldb copy function */
 typedef int (xmldb_copy_t)(xmldb_handle xh, const char *from, const char *to);
@@ -139,7 +139,7 @@ int xmldb_disconnect(clicon_handle h);
 int xmldb_getopt(clicon_handle h, char *optname, void **value);
 int xmldb_setopt(clicon_handle h, char *optname, void *value);
 int xmldb_get(clicon_handle h, const char *db, char *xpath, int config, cxobj **xtop);
-int xmldb_put(clicon_handle h, const char *db, enum operation_type op, cxobj *xt);
+int xmldb_put(clicon_handle h, const char *db, enum operation_type op, cxobj *xt, cbuf *cbret);
 int xmldb_copy(clicon_handle h, const char *from, const char *to);
 int xmldb_lock(clicon_handle h, const char *db, int pid);
 int xmldb_unlock(clicon_handle h, const char *db);

lib/clixon/clixon_xml_map.h

@@ -64,8 +64,8 @@ int xml_spec_populate(cxobj *x, void *arg);
 int api_path2xpath_cvv(yang_spec *yspec, cvec *cvv, int offset, cbuf *xpath);
 int api_path2xpath(yang_spec *yspec, char *api_path, cbuf *xpath);
 int api_path2xml(char *api_path, yang_spec *yspec, cxobj *xtop, 
-		 int schemanode, cxobj **xpathp, yang_node **ypathp);
+		 yang_class nodeclass, cxobj **xpathp, yang_node **ypathp);
-int xml_merge(cxobj *x0, cxobj *x1, yang_spec *yspec);
+int xml_merge(cxobj *x0, cxobj *x1, yang_spec *yspec, char **reason);
 int yang_enum_int_value(cxobj *node, int32_t *val);
 
 #endif  /* _CLIXON_XML_MAP_H_ */

lib/clixon/clixon_yang.h

@@ -31,6 +31,9 @@
 
   ***** END LICENSE BLOCK *****
 
+ * Yang functions
+ * @see https://tools.ietf.org/html/rfc6020 YANG 1.0
+ * @see https://tools.ietf.org/html/rfc7950 YANG 1.1
  */
 
 #ifndef _CLIXON_YANG_H_
@@ -114,6 +117,7 @@ enum rfc_6020{
     Y_TYPEDEF,
     Y_UNIQUE,
     Y_UNITS,
+    Y_UNKNOWN,
     Y_USES,
     Y_VALUE,
     Y_WHEN,
@@ -122,14 +126,45 @@ enum rfc_6020{
     Y_SPEC  /* XXX: NOTE NOT YANG STATEMENT, reserved for top level spec */
 };
 
+/* Type used to group yang nodes used in some functions
+ * See RFC7950 Sec 3
+ */
+enum yang_class{
+    YC_NONE,            /* Someting else,... */
+    YC_DATANODE,        /* See yang_datanode() */
+    YC_DATADEFINITION,  /* See yang_datadefinition() */
+    YC_SCHEMANODE       /* See yang_schemanode() */
+};
+typedef enum yang_class yang_class;
+
 #define YANG_FLAG_MARK 0x01  /* Marker for dynamic algorithms, eg expand */
 
-/* Yang data node */
+/* Yang data node 
+ * See RFC7950 Sec 3:
+ *   o  data node: A node in the schema tree that can be instantiated in a
+ *      data tree.  One of container, leaf, leaf-list, list, anydata, and
+ *      anyxml.
+ */
 #define yang_datanode(y) ((y)->ys_keyword == Y_CONTAINER || (y)->ys_keyword == Y_LEAF || (y)->ys_keyword == Y_LIST || (y)->ys_keyword == Y_LEAF_LIST || (y)->ys_keyword == Y_ANYXML)
 
-/* Yang schema node */
+/* Yang data definition statement
+ * See RFC 7950 Sec 3:
+ *   o  data definition statement: A statement that defines new data
+ *      nodes.  One of "container", "leaf", "leaf-list", "list", "choice",
+ *      "case", "augment", "uses", "anydata", and "anyxml".
+ */
+#define yang_datadefinition(y) (yang_datanode(y) || (y)->ys_keyword == Y_CHOICE || (y)->ys_keyword == Y_CASE || (y)->ys_keyword == Y_AUGMENT || (y)->ys_keyword == Y_USES)
+
+
+/* Yang schema node .
+ * See RFC 7950 Sec 3:
+ *    o  schema node: A node in the schema tree.  One of action, container,
+ *       leaf, leaf-list, list, choice, case, rpc, input, output,
+ *       notification, anydata, and anyxml.
+ */
 #define yang_schemanode(y) (yang_datanode(y) || (y)->ys_keyword == Y_RPC || (y)->ys_keyword == Y_CHOICE || (y)->ys_keyword == Y_CASE || (y)->ys_keyword == Y_INPUT || (y)->ys_keyword == Y_OUTPUT || (y)->ys_keyword == Y_NOTIFICATION)
 
+
 typedef struct yang_stmt yang_stmt; /* forward */
 
 /*! Yang type cache. Yang type statements can cache all typedef info here
@@ -161,7 +196,9 @@ struct yang_stmt{
 					leaf-list, 
 					config: boolean true or false
 					mandatory: boolean true or false
-					fraction-digits for fraction-digits */
+					fraction-digits for fraction-digits
+					unkown-stmt (argument)
+				     */
     cvec              *ys_cvec;      /* List of stmt-specific variables 
 					Y_RANGE: range_min, range_max 
 					Y_LIST: vector of keys
@@ -206,15 +243,15 @@ yang_stmt *ys_dup(yang_stmt *old);
 int        yn_insert(yang_node *yn_parent, yang_stmt *ys_child);
 yang_stmt *yn_each(yang_node *yn, yang_stmt *ys);
 char      *yang_key2str(int keyword);
-char      *ytype_prefix(yang_stmt *ys);
+char      *yarg_prefix(yang_stmt *ys);
-char      *ytype_id(yang_stmt *ys);
+char      *yarg_id(yang_stmt *ys);
 yang_stmt *ys_module(yang_stmt *ys);
 yang_spec *ys_spec(yang_stmt *ys);
 yang_stmt *yang_find_module_by_prefix(yang_stmt *ys, char *prefix);
 yang_stmt *yang_find(yang_node *yn, int keyword, char *argument);
 yang_stmt *yang_find_datanode(yang_node *yn, char *argument);
 yang_stmt *yang_find_schemanode(yang_node *yn, char *argument);
-yang_stmt *yang_find_topnode(yang_spec *ysp, char *name, int schemanode);
+yang_stmt *yang_find_topnode(yang_spec *ysp, char *name, yang_class class);
 int        yang_order(yang_stmt *y);
 int        yang_print(FILE *f, yang_node *yn);
 int        yang_print_cbuf(cbuf *cb, yang_node *yn, int marginal);
@@ -223,11 +260,11 @@ int        yang_parse(clicon_handle h, const char *yang_dir,
 int        yang_apply(yang_node *yn, enum rfc_6020 key, yang_applyfn_t fn, 
 		      void *arg);
 int        yang_abs_schema_nodeid(yang_spec *yspec, char *schema_nodeid, 
-				  yang_stmt **yres);
+				  enum rfc_6020 keyword, yang_stmt **yres);
 int        yang_desc_schema_nodeid(yang_node *yn, char *schema_nodeid, 
-				   yang_stmt **yres);
+				   enum rfc_6020 keyword, yang_stmt **yres);
 cg_var    *ys_parse(yang_stmt *ys, enum cv_type cvtype);
-int        ys_parse_sub(yang_stmt *ys);
+int        ys_parse_sub(yang_stmt *ys, char *extra);
 int        yang_mandatory(yang_stmt *ys);
 int        yang_config(yang_stmt *ys);
 yang_spec *yang_spec_netconf(clicon_handle h);

lib/clixon/clixon_yang_type.h

@@ -68,7 +68,6 @@ char      *cv2yang_type(enum cv_type cv_type);
 yang_stmt *yang_find_identity(yang_stmt *ys, char *identity);
 int        ys_cv_validate(cg_var *cv, yang_stmt *ys, char **reason);
 int        clicon_type2cv(char *type, char *rtype, enum cv_type *cvtype);
-char      *ytype_id(yang_stmt *ys);
 int        yang_type_get(yang_stmt *ys, char **otype, yang_stmt **restype, 
 			 int *options, cg_var **mincv, cg_var **maxcv, char **pattern,
                          uint8_t *fraction_digits);

lib/src/Makefile.in

@@ -52,6 +52,7 @@ CLIXON_MINOR    = @CLIXON_VERSION_MINOR@
 VPATH       	= @srcdir@
 CC		= @CC@
 CFLAGS  	= -fPIC @CFLAGS@
+INSTALLFLAGS  	= @INSTALLFLAGS@
 LDFLAGS 	= @LDFLAGS@
 LIBS    	= @LIBS@ 
 
@@ -68,7 +69,7 @@ SRC     = clixon_sig.c clixon_log.c clixon_err.c clixon_event.c \
 	  clixon_json.c clixon_yang.c clixon_yang_type.c \
 	  clixon_hash.c clixon_options.c clixon_plugin.c \
 	  clixon_proto.c clixon_proto_client.c \
-	  clixon_xsl.c clixon_sha1.c clixon_xml_db.c
+	  clixon_xsl.c clixon_sha1.c clixon_xml_db.c clixon_netconf_lib.c
 
 YACCOBJS := lex.clixon_xml_parse.o clixon_xml_parse.tab.o \
 	    lex.clixon_yang_parse.o  clixon_yang_parse.tab.o \
@@ -181,8 +182,8 @@ install: install-lib
 install-include:
 
 install-lib: $(MYLIB)
-	install -m 755 -d $(DESTDIR)$(libdir) 
+	install -m 0755 -d $(DESTDIR)$(libdir)
-	install -m 755 $(MYLIB) $(DESTDIR)$(libdir) 
+	install -m 0644 $(INSTALLFLAGS) $(MYLIB) $(DESTDIR)$(libdir)
 	ln -sf $(MYLIB) $(DESTDIR)$(libdir)/$(MYLIBSO)     # -l:libclixon.so.3
 	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon.so
 

lib/src/clixon_file.c

@@ -62,8 +62,7 @@
 #include "clixon_string.h"
 #include "clixon_file.h"
 
-/*
+/*! qsort "compar" for directory alphabetically sorting, see qsort(3)
- * qsort function
  */
 static int
 clicon_file_dirent_sort(const void* arg1, 
@@ -79,8 +78,7 @@ clicon_file_dirent_sort(const void* arg1,
 #endif /* HAVE_STRVERSCMP */
 }
 
-
+/*! Return alphabetically sorted files from a directory matching regexp
-/*! Return sorted matching files from a directory
  * @param[in]  dir     Directory path 
  * @param[out] ent     Entries pointer, will be filled in with dir entries. Free
  *                     after use
@@ -120,27 +118,23 @@ clicon_file_dirent(const char     *dir,
    struct dirent *new = NULL;
    struct dirent *dvecp = NULL;
 
-
    *ent = NULL;
    nent = 0;
-
    if (regexp && (res = regcomp(&re, regexp, REG_EXTENDED)) != 0) {
        regerror(res, &re, errbuf, sizeof(errbuf));
        clicon_err(OE_DB, 0, "regcomp: %s", errbuf);
        return -1;
    }
-
+   if ((dirp = opendir(dir)) == NULL) {
-   if ((dirp = opendir (dir)) == NULL) {
      if (errno == ENOENT) /* Dir does not exist -> return 0 matches */
        retval = 0;
      else
        clicon_err(OE_UNIX, errno, "opendir(%s)", dir);
      goto quit;
    }
-   
+   for (res = readdir_r(dirp, &dent, &dresp); 
-   for (res = readdir_r (dirp, &dent, &dresp); 
 	dresp; 
-	res = readdir_r (dirp, &dent, &dresp)) {
+	res = readdir_r(dirp, &dent, &dresp)) {
        if (res != 0) {
 	   clicon_err(OE_UNIX, 0, "readdir: %s", strerror(errno));
 	   goto quit;
@@ -162,13 +156,12 @@ clicon_file_dirent(const char     *dir,
 	   if ((type & st.st_mode) == 0)
 	       continue;
        }
-       
        if ((tmp = realloc(new, (nent+1)*sizeof(*dvecp))) == NULL) {
 	   clicon_err(OE_UNIX, errno, "realloc");
 	   goto quit;
        }
        new = tmp;
-       memcpy (&new[nent], &dent, sizeof(dent));
+       memcpy(&new[nent], &dent, sizeof(dent));
        nent++;
 
    } /* while */

lib/src/clixon_handle.c

@@ -50,9 +50,9 @@
 #include "clixon_queue.h"
 #include "clixon_hash.h"
 #include "clixon_handle.h"
+#include "clixon_log.h"
 #include "clixon_err.h"
 #include "clixon_yang.h"
-#include "clixon_plugin.h"
 #include "clixon_options.h"
 
 #define CLICON_MAGIC 0x99aafabe
@@ -61,7 +61,8 @@
 
 /*! Internal structure of basic handle. Also header of all other handles.
  * @note If you change here, you must also change the structs below:
- * @see struct cli_handle, struct backend_handle
+ * @see struct cli_handle
+ * @see struct backend_handle
  */
 struct clicon_handle {
     int                ch_magic;    /* magic (HDR) */
@@ -70,6 +71,9 @@ struct clicon_handle {
 };
 
 /*! Internal call to allocate a CLICON handle. 
+ *
+ * @param[in]  size  Size of handle (internal) struct.
+ * @retval   h   Clicon handle
  *
  * There may be different variants of handles with some common options.
  * So far the only common options is a MAGIC cookie for sanity checks and 
@@ -102,6 +106,7 @@ clicon_handle_init0(int size)
 
 /*! Basic CLICON init functions returning a handle for API access.
  *
+ * @retval   h   Clicon handle
  * This is the first call to CLICON basic API which returns a handle to be 
  * used in the API functions. There are other clicon_init functions for more 
  * elaborate applications (cli/backend/netconf). This should be used by the most
@@ -114,6 +119,7 @@ clicon_handle_init(void)
 }
 
 /*! Deallocate clicon handle, including freeing handle data.
+ * @param[in]  h   Clicon handle
  * @Note: handle 'h' cannot be used in calls after this
  */
 int
@@ -131,9 +137,10 @@ clicon_handle_exit(clicon_handle h)
     return 0;
 }
 
-/*
+/*! Check struct magic number for sanity checks
- * Check struct magic number for sanity checks
+ * @param[in]  h   Clicon handle
- * return 0 if OK, -1 if fail.
+ * @retval     0   Sanity check OK
+ * @retval    -1   Sanity check failed
  */
 int
 clicon_handle_check(clicon_handle h)
@@ -144,8 +151,8 @@ clicon_handle_check(clicon_handle h)
     return ch->ch_magic == CLICON_MAGIC ? 0 : -1;
 }
 
-/* 
+/*! Return clicon options (hash-array) given a handle.
- * Return clicon options (hash-array) given a handle.
+ * @param[in]  h        Clicon handle
  */
 clicon_hash_t *
 clicon_options(clicon_handle h)
@@ -155,8 +162,8 @@ clicon_options(clicon_handle h)
     return ch->ch_copt;
 }
 
-/* 
+/*! Return clicon data (hash-array) given a handle.
- * Return clicon data (hash-array) given a handle.
+ * @param[in]  h        Clicon handle
  */
 clicon_hash_t *
 clicon_data(clicon_handle h)
@@ -165,4 +172,3 @@ clicon_data(clicon_handle h)
 
     return ch->ch_data;
 }
-

lib/src/clixon_json.c

@@ -216,6 +216,9 @@ json_str_escape(char *str)
     for (i=0;i<strlen(str);i++)
 	switch (str[i]){
 	case '\n':
+	    snew[j++]='\\';
+	    snew[j++]='n';
+	    break;
 	case '\"':
 	case '\\':
 	    snew[j++]='\\';
@@ -301,10 +304,12 @@ xml2json1_cbuf(cbuf                  *cb,
 	break;
     }
     case NO_ARRAY:
-	if (!flat)
+	if (!flat){
-	    cprintf(cb, "%*s\"%s\": ", 
+	    cprintf(cb, "%*s\"", pretty?(level*JSON_INDENT):0, "");
-		    pretty?(level*JSON_INDENT):0, "", 
+	    if (xml_namespace(x))
-		    xml_name(x));
+		cprintf(cb, "%s:", xml_namespace(x));
+	    cprintf(cb, "%s\": ", xml_name(x));
+	}
 	switch (childt){
 	case NULL_CHILD:
 	    cprintf(cb, "null");
@@ -320,9 +325,10 @@ xml2json1_cbuf(cbuf                  *cb,
 	break;
     case FIRST_ARRAY:
     case SINGLE_ARRAY:
-	cprintf(cb, "%*s\"%s\": ", 
+	cprintf(cb, "%*s\"", pretty?(level*JSON_INDENT):0, "");
-		pretty?(level*JSON_INDENT):0, "", 
+	if (xml_namespace(x))
-		xml_name(x));
+	    cprintf(cb, "%s:", xml_namespace(x));
+	cprintf(cb, "%s\": ", xml_name(x));
 	level++;
 	cprintf(cb, "[%s%*s", 
 		pretty?"\n":"",

lib/src/clixon_json_parse.l

@@ -89,6 +89,7 @@ exp ({integer}|{real})[eE][+-]{integer}
 %%
 <START>[ \t]            
 <START>\n               { _JY->jy_linenum++; }
+<START>\r               { }
 <START><<EOF>>          { return J_EOF; }
 <START>\{               { return *yytext; }
 <START>\}               { return *yytext; }

lib/src/clixon_netconf_lib.c

@@ -0,0 +1,936 @@
+/*
+ *
+  ***** BEGIN LICENSE BLOCK *****
+ 
+  Copyright (C) 2009-2018 Olof Hagsand and Benny Holmgren
+
+  This file is part of CLIXON.
+
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+
+  Alternatively, the contents of this file may be used under the terms of
+  the GNU General Public License Version 3 or later (the "GPL"),
+  in which case the provisions of the GPL are applicable instead
+  of those above. If you wish to allow use of your version of this file only
+  under the terms of the GPL, and not to allow others to
+  use your version of this file under the terms of Apache License version 2, 
+  indicate your decision by deleting the provisions above and replace them with
+  the  notice and other provisions required by the GPL. If you do not delete
+  the provisions above, a recipient may use your version of this file under
+  the terms of any one of the Apache License version 2 or the GPL.
+
+  ***** END LICENSE BLOCK *****
+
+ * Netconf library functions. See RFC6241
+ */
+
+#ifdef HAVE_CONFIG_H
+#include "clixon_config.h" /* generated by config & autoconf */
+#endif
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <limits.h>
+#include <fnmatch.h>
+#include <stdint.h>
+#include <assert.h>
+
+/* cligen */
+#include <cligen/cligen.h>
+
+/* clixon */
+#include "clixon_queue.h"
+#include "clixon_hash.h"
+#include "clixon_string.h"
+#include "clixon_err.h"
+#include "clixon_handle.h"
+#include "clixon_yang.h"
+#include "clixon_log.h"
+#include "clixon_xml.h"
+#include "clixon_netconf_lib.h"
+
+/*! Create Netconf in-use error XML tree according to RFC 6241 Appendix A
+ *
+ * The request requires a resource that already is in use.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type     Error type: "application" or "protocol"
+ * @param[in]  message  Error message
+ */
+int
+netconf_in_use(cbuf *cb,
+	       char *type,
+	       char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+    
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>in-use</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf invalid-value error XML tree according to RFC 6241 Appendix A
+ *
+ * The request specifies an unacceptable value for one or more parameters.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_invalid_value(cbuf *cb,
+		      char *type,
+		      char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>invalid-value</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf too-big error XML tree according to RFC 6241 Appendix A
+ *
+ * The request or response (that would be generated) is 
+ * too large for the implementation to handle.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "transport", "rpc", "application", "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_too_big(cbuf *cb,
+		char *type,
+		char *message)
+{
+    int     retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>too-big</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf missing-attribute error XML tree according to RFC 6241 App A
+ *
+ * An expected attribute is missing.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "rpc", "application" or "protocol"
+ * @param[in]  info    bad-attribute or bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_missing_attribute(cbuf *cb,
+			  char *type,
+			  char *info,
+			  char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>missing-attribute</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf bad-attribute error XML tree according to RFC 6241 App A
+ *
+ * An attribute value is not correct; e.g., wrong type,
+ * out of range, pattern mismatch.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "rpc", "application" or "protocol"
+ * @param[in]  info    bad-attribute or bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_bad_attribute(cbuf *cb,
+		      char *type,
+		      char *info,
+		      char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>bad-attribute</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf unknwon-attribute error XML tree according to RFC 6241 App A
+ *
+ * An unexpected attribute is present.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "rpc", "application" or "protocol"
+ * @param[in]  info    bad-attribute or bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_unknown_attribute(cbuf *cb,
+			  char *type,
+			  char *info,
+			  char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>unknown-attribute</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf missing-element error XML tree according to RFC 6241 App A
+ *
+ * An expected element is missing.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  info    bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_missing_element(cbuf      *cb, 
+			char      *type,
+			char      *info,
+			char      *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>missing-element</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf bad-element error XML tree according to RFC 6241 App A
+ *
+ * An element value is not correct; e.g., wrong type, out of range, 
+ * pattern mismatch.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  info    bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_bad_element(cbuf *cb,
+		    char *type,
+		    char *info,
+		    char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>bad-element</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf unknown-element error XML tree according to RFC 6241 App A
+ *
+ * An unexpected element is present.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  info    bad-element xml
+ * @param[in]  message Error message
+ */
+int
+netconf_unknown_element(cbuf *cb,
+			char *type,
+			char *info,
+			char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>unknown-element</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf unknown-namespace error XML tree according to RFC 6241 App A
+ *
+ * An unexpected namespace is present.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  info    bad-element or bad-namespace xml
+ * @param[in]  message Error message
+ */
+int
+netconf_unknown_namespace(cbuf *cb,
+			  char *type,
+			  char *info,
+			  char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>unknown-namespace</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		type, info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf access-denied error XML tree according to RFC 6241 App A
+ *
+ * An expected element is missing.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_access_denied(cbuf *cb,
+		      char *type,
+		      char *message)
+{
+    int retval = -1;
+    char *encstr = NULL;
+    
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>access-denied</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf access-denied error XML tree according to RFC 6241 App A
+ *
+ * An expected element is missing.
+ * @param[out] xret    Error XML tree 
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_access_denied_xml(cxobj **xret,
+			  char   *type,
+			  char   *message)
+{
+    int   retval =-1;
+    cxobj *xerr;
+    
+    if (*xret == NULL){
+	if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL)
+	    goto done;
+    }
+    else if (xml_name_set(*xret, "rpc-reply") < 0)
+	goto done;
+    if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL)
+	goto done;
+    if (xml_parse_va(&xerr, NULL, "<error-tag>access-denied</error-tag>"
+		     "<error-type>%s</error-type>"
+		     "<error-severity>error</error-severity>", type) < 0)
+	goto done;
+    if (message && xml_parse_va(&xerr, NULL, "<error-message>%s</error-message>",
+			 message) < 0)
+	goto done;
+    retval = 0;
+ done:
+    return retval;
+}
+
+/*! Create Netconf lock-denied error XML tree according to RFC 6241 App A
+ *
+ * Access to the requested lock is denied because the lock is currently held
+ * by another entity.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  info    session-id xml
+ * @param[in]  message Error message
+ */
+int
+netconf_lock_denied(cbuf *cb,
+		    char *info,
+		    char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>lock-denied</error-tag>"
+		"<error-type>protocol</error-type>"
+		"<error-info>%s</error-info>"
+		"<error-severity>error</error-severity>",
+		info) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf resource-denied error XML tree according to RFC 6241 App A
+ *
+ * An expected element is missing.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "transport, "rpc", "application", "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_resource_denied(cbuf *cb,
+			char *type,
+			char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+    
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>resource-denied</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf rollback-failed error XML tree according to RFC 6241 App A
+ *
+ * Request to roll back some configuration change (via rollback-on-error or 
+ * <discard-changes> operations) was not completed for some reason.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_rollback_failed(cbuf *cb,
+			char *type,
+			char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>rollback-failed</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf data-exists error XML tree according to RFC 6241 Appendix A
+ *
+ * Request could not be completed because the relevant
+ * data model content already exists.  For example,
+ * a "create" operation was attempted on data that already exists.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  message Error message
+ */
+int
+netconf_data_exists(cbuf      *cb, 
+		    char      *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>data-exists</error-tag>"
+		"<error-type>application</error-type>"
+		"<error-severity>error</error-severity>") <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf data-missing error XML tree according to RFC 6241 App A
+ *
+ * Request could not be completed because the relevant data model content 
+ * does not exist.  For example, a "delete" operation was attempted on
+ * data that does not exist.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  message Error message
+ */
+int
+netconf_data_missing(cbuf *cb,
+		     char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>data-missing</error-tag>"
+		"<error-type>application</error-type>"
+		"<error-severity>error</error-severity>") <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf operation-not-supported error XML according to RFC 6241 App A
+ *
+ * Request could not be completed because the requested operation is not
+ * supported by this implementation.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_operation_not_supported(cbuf *cb,
+				char *type,
+				char *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>operation-not-supported</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf operation-failed error XML tree according to RFC 6241 App A
+ *
+ * Request could not be completed because the requested operation failed for
+ * some reason not covered by any other error condition.
+ * @param[out] cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]  type    Error type: "rpc", "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_operation_failed(cbuf  *cb,
+			 char  *type,
+			 char  *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>operation-failed</error-tag>"
+		"<error-type>%s</error-type>"
+		"<error-severity>error</error-severity>",
+		type) <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") < 0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf operation-failed error XML tree according to RFC 6241 App A
+ *
+ * Request could not be completed because the requested operation failed for
+ * some reason not covered by any other error condition.
+ * @param[out] xret    Error XML tree 
+ * @param[in]  type    Error type: "rpc", "application" or "protocol"
+ * @param[in]  message Error message
+ */
+int
+netconf_operation_failed_xml(cxobj **xret,
+			     char  *type,
+			     char  *message)
+{
+    int   retval =-1;
+    cxobj *xerr;
+    
+    if (*xret == NULL){
+	if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL)
+	    goto done;
+    }
+    else if (xml_name_set(*xret, "rpc-reply") < 0)
+	goto done;
+    if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL)
+	goto done;
+    if (xml_parse_va(&xerr, NULL, "<error-tag>operation-failed</error-tag>"
+		     "<error-type>%s</error-type>"
+		     "<error-severity>error</error-severity>", type) < 0)
+	goto done;
+    if (message && xml_parse_va(&xerr, NULL, "<error-message>%s</error-message>",
+			 message) < 0)
+	goto done;
+    retval = 0;
+ done:
+    return retval;
+}
+
+/*! Create Netconf malformed-message error XML tree according to RFC 6241 App A
+ *
+ * A message could not be handled because it failed to be parsed correctly.  
+ * For example, the message is not well-formed XML or it uses an
+ * invalid character set.
+ * @param[out]  cb      CLIgen buf. Error XML is written in this buffer
+ * @param[in]   message Error message
+ * @note New in :base:1.1
+ */
+int
+netconf_malformed_message(cbuf  *cb,
+			  char  *message)
+{
+    int   retval = -1;
+    char *encstr = NULL;
+
+    if (cprintf(cb, "<rpc-reply><rpc-error>"
+		"<error-tag>malformed-message</error-tag>"
+		"<error-type>rpc</error-type>"
+		"<error-severity>error</error-severity>") <0)
+	goto err;
+    if (message){
+	if (xml_chardata_encode(message, &encstr) < 0)
+	    goto done;
+	if (cprintf(cb, "<error-message>%s</error-message>", encstr) < 0)
+	    goto err;
+    }
+    if (cprintf(cb, "</rpc-error></rpc-reply>") <0)
+	goto err;
+    retval = 0;
+ done:
+    if (encstr)
+	free(encstr);
+    return retval;
+ err:
+    clicon_err(OE_XML, errno, "cprintf");
+    goto done;
+}
+
+/*! Create Netconf malformed-message error XML tree according to RFC 6241 App A
+ *
+ * A message could not be handled because it failed to be parsed correctly.  
+ * For example, the message is not well-formed XML or it uses an
+ * invalid character set.
+ * @param[out] xret    Error XML tree 
+ * @param[in]  message Error message
+ * @note New in :base:1.1
+ */
+int
+netconf_malformed_message_xml(cxobj **xret,
+			     char  *message)
+{
+    int   retval =-1;
+    cxobj *xerr;
+    
+    if (*xret == NULL){
+	if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL)
+	    goto done;
+    }
+    else if (xml_name_set(*xret, "rpc-reply") < 0)
+	goto done;
+    if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL)
+	goto done;
+    if (xml_parse_va(&xerr, NULL, "<error-tag>malformed-message</error-tag>"
+		     "<error-type>rpc</error-type>"
+		     "<error-severity>error</error-severity>") < 0)
+	goto done;
+    if (message && xml_parse_va(&xerr, NULL, "<error-message>%s</error-message>",
+			 message) < 0)
+	goto done;
+    retval = 0;
+ done:
+    return retval;
+}

lib/src/clixon_options.c

@@ -64,9 +64,9 @@
 #include "clixon_handle.h"
 #include "clixon_log.h"
 #include "clixon_yang.h"
-#include "clixon_plugin.h"
 #include "clixon_options.h"
 #include "clixon_xml.h"
+#include "clixon_plugin.h"
 #include "clixon_xsl.h"
 #include "clixon_xml_map.h"
 
@@ -209,7 +209,7 @@ clicon_options_main(clicon_handle h)
     /* If file ends with .xml, assume it is new format */
     if ((suffix = rindex(configfile, '.')) != NULL){
 	suffix++;
-	xml = strcmp(suffix,"xml") == 0;
+	xml = strcmp(suffix, "xml") == 0;
     }
     if (xml){     /* Read clixon yang file */
 	if ((yspec = yspec_new()) == NULL)
@@ -715,3 +715,32 @@ clicon_xmldb_handle_set(clicon_handle h,
 }
 
 
+/*! Get authorized user name
+ * @param[in]  h   Clicon handle
+ * @retval     xh  XMLDB storage handle. If not connected return NULL
+ */
+char *
+clicon_username_get(clicon_handle h)
+{
+    clicon_hash_t  *cdat = clicon_data(h);
+
+    return (char*)hash_value(cdat, "username", NULL);
+}
+
+/*! Set authorized user name
+ * @param[in]  h   Clicon handle
+ * @param[in]  xh  XMLDB storage handle. If NULL reset it
+ * @note Just keep note of it, dont allocate it or so.
+ */
+int
+clicon_username_set(clicon_handle h, 
+		    void         *username)
+{
+    clicon_hash_t  *cdat = clicon_data(h);
+
+    if (username == NULL)
+	return hash_del(cdat, "username");
+    return hash_add(cdat, "username", username, strlen(username)+1)==NULL?-1:0;
+}
+
+

lib/src/clixon_plugin.c

@@ -38,123 +38,467 @@
 
 #include <stdio.h>
 #include <stdlib.h>
+#include <stdint.h>
 #include <string.h>
 #include <errno.h>
 #include <dlfcn.h>
+#include <dirent.h>
+
+#include <sys/stat.h>
+#include <sys/param.h>
+
+/* cligen */
+#include <cligen/cligen.h>
 
 #include "clixon_err.h"
 #include "clixon_queue.h"
 #include "clixon_hash.h"
 #include "clixon_log.h"
+#include "clixon_file.h"
 #include "clixon_handle.h"
+#include "clixon_yang.h"
+#include "clixon_xml.h"
 #include "clixon_plugin.h"
 
+/* List of plugins XXX 
+ * 1. Place in clixon handle not global variables
+ * 2. Use qelem circular lists
+ */
+static clixon_plugin *_clixon_plugins = NULL;  /* List of plugins (of client) */
+static int            _clixon_nplugins = 0;  /* Number of plugins */
 
-static find_plugin_t *
+/*! Iterator over clixon plugins
-clicon_find_plugin(clicon_handle h)
+ *
+ * @note Never manipulate the plugin during operation or using the
+ * same object recursively
+ *
+ * @param[in]  h       Clicon handle
+ * @param[in] plugin   previous plugin, or NULL on init
+ * @code
+ *   clicon_plugin *cp = NULL;
+ *   while ((cp = clixon_plugin_each(h, cp)) != NULL) {
+ *     ...
+ *   }
+ * @endcode
+ * @note Not optimized, alwasy iterates from the start of the list
+ */
+clixon_plugin *
+clixon_plugin_each(clicon_handle  h,
+		   clixon_plugin *cpprev)
 {
-    void *p;
+    int            i;
-    find_plugin_t *fp = NULL;
+    clixon_plugin *cp;
-    clicon_hash_t *data = clicon_data(h);
+    clixon_plugin *cpnext = NULL; 
 
-    if ((p = hash_value(data, "CLICON_FIND_PLUGIN", NULL)) != NULL)
+    if (cpprev == NULL)
-	memcpy(&fp, p, sizeof(fp));
+	cpnext = _clixon_plugins;
-
+    else{
-    return fp;
+	for (i = 0; i < _clixon_nplugins; i++) {
+	    cp = &_clixon_plugins[i];
+	    if (cp == cpprev)
+		break;
+	    cp = NULL;
+	}
+	if (cp && i < _clixon_nplugins-1)
+	    cpnext = &_clixon_plugins[i+1];
+    }
+    return cpnext;
 }
 
-
+/*! Reverse iterator over clixon plugins, iterater from nr to 0
-/*! Return a function pointer based on name of plugin and function.
+ *
- * If plugin is specified, ask daemon registered function to return 
+ * @note Never manipulate the plugin during operation or using the
- * the dlsym handle of the plugin.
+ * same object recursively
+ *
+ * @param[in]  h       Clicon handle
+ * @param[in] plugin   previous plugin, or NULL on init
+ * @code
+ *   clicon_plugin *cp = NULL;
+ *   while ((cp = clixon_plugin_each_revert(h, cp, nr)) != NULL) {
+ *     ...
+ *   }
+ * @endcode
+ * @note Not optimized, alwasy iterates from the start of the list
  */
-void *
+clixon_plugin *
-clicon_find_func(clicon_handle h, char *plugin, char *func)
+clixon_plugin_each_revert(clicon_handle  h,
+			  clixon_plugin *cpprev,
+			  int            nr)
 {
-    find_plugin_t *plgget;
+    int            i;
-    void          *dlhandle = NULL;
+    clixon_plugin *cp;
+    clixon_plugin *cpnext = NULL; 
 
-    if (plugin) {
+    if (cpprev == NULL)
-	/* find clicon_plugin_get() in global namespace */
+	cpnext = &_clixon_plugins[nr-1];
-	if ((plgget = clicon_find_plugin(h)) == NULL) {
+    else{
-	    clicon_err(OE_UNIX, errno, "Specified plugin not supported");
+	for (i = nr-1; i >= 0; i--) {
+	    cp = &_clixon_plugins[i];
+	    if (cp == cpprev)
+		break;
+	    cp = NULL;
+	}
+	if (cp && i > 0)
+	    cpnext = &_clixon_plugins[i-1];
+    }
+    return cpnext;
+}
+
+/*! Find plugin by name
+ * @param[in]  h    Clicon handle
+ * @param[in]  name Plugin name
+ * @retval     p    Plugin if found
+ * @retval     NULL Not found
+ */
+clixon_plugin *
+clixon_plugin_find(clicon_handle h,
+		   char         *name)
+{
+    int            i;
+    clixon_plugin *cp = NULL;
+
+    for (i = 0; i < _clixon_nplugins; i++) {
+	cp = &_clixon_plugins[i];
+	if (strcmp(cp->cp_name, name) == 0)
+	    return cp;
+    }
     return NULL;
-	}
-	dlhandle = plgget(h, plugin);
-    }
-    
-    return dlsym(dlhandle, func);
 }
 
 /*! Load a dynamic plugin object and call its init-function
- * Note 'file' may be destructively modified
  * @param[in]  h       Clicon handle
  * @param[in]  file    Which plugin to load
+ * @param[in]  function Which function symbol to load and call
  * @param[in]  dlflags See man(3) dlopen
+ * @retval     cp      Clixon plugin structure
+ * @retval     NULL    Error
+ * @see clixon_plugins_load  Load all plugins
  */
-plghndl_t 
+static clixon_plugin *
-plugin_load(clicon_handle h, 
+plugin_load_one(clicon_handle   h, 
 		char           *file,
+		char           *function,
 		int             dlflags)
 {
     char          *error;
     void          *handle = NULL;
-    plginit_t *initfn;
+    plginit2_t    *initfn;
+    clixon_plugin_api *api = NULL;
+    clixon_plugin *cp = NULL;
+    char          *name;
+    char          *p;
 
     clicon_debug(1, "%s", __FUNCTION__);
     dlerror();    /* Clear any existing error */
-    if ((handle = dlopen (file, dlflags)) == NULL) {
+    if ((handle = dlopen(file, dlflags)) == NULL) {
         error = (char*)dlerror();
 	clicon_err(OE_PLUGIN, errno, "dlopen: %s\n", error ? error : "Unknown error");
 	goto done;
     }
-    /* call plugin_init() if defined */
+    /* call plugin_init() if defined, eg CLIXON_PLUGIN_INIT or CLIXON_BACKEND_INIT */
-    if ((initfn = dlsym(handle, PLUGIN_INIT)) == NULL){
+    if ((initfn = dlsym(handle, function)) == NULL){
-	clicon_err(OE_PLUGIN, errno, "Failed to find plugin_init when loading restconf plugin %s", file);
+	clicon_err(OE_PLUGIN, errno, "Failed to find %s when loading clixon plugin %s", CLIXON_PLUGIN_INIT, file);
 	goto err;
     }
     if ((error = (char*)dlerror()) != NULL) {
 	clicon_err(OE_UNIX, 0, "dlsym: %s: %s", file, error);
 	goto done;
     }
-    if (initfn(h) != 0) {
+    if ((api = initfn(h)) == NULL) {
 	clicon_err(OE_PLUGIN, errno, "Failed to initiate %s", strrchr(file,'/')?strchr(file, '/'):file);
 	if (!clicon_errno) 	/* sanity: log if clicon_err() is not called ! */
 	    clicon_err(OE_DB, 0, "Unknown error: %s: plugin_init does not make clicon_err call on error",
 		       file);
 	goto err;
     }
+    /* Note: sizeof clixon_plugin_api which is largest of clixon_plugin_api:s */
+    if ((cp = (clixon_plugin *)malloc(sizeof(struct clixon_plugin))) == NULL){
+	clicon_err(OE_UNIX, errno, "malloc");
+	goto done;
+    }
+    memset(cp, 0, sizeof(struct clixon_plugin));
+    cp->cp_handle = handle;
+    /* Extract string after last '/' in filename, if any */
+    name = strrchr(file, '/') ? strrchr(file, '/')+1 : file;
+    /* strip extension, eg .so from name */
+    if ((p=strrchr(name, '.')) != NULL)
+	*p = '\0';
+    /* Copy name to struct */
+    memcpy(cp->cp_name, name, strlen(name)+1);
+
+    snprintf(cp->cp_name, sizeof(cp->cp_name), "%*s",
+	     (int)strlen(name), name);
+    cp->cp_api = *api;
+    clicon_debug(1, "%s", __FUNCTION__);
  done:
-    return handle;
+    return cp;
  err:
     if (handle)
 	dlclose(handle);
     return NULL;
 }
 
-
+/*! Load a set of plugin objects from a directory and and call their init-function
-/*! Unload a plugin
  * @param[in]  h     Clicon handle
- * @param[in]  handle   Clicon handle
+ * @param[in]  function Which function symbol to load and call (eg CLIXON_PLUGIN_INIT)
+ * @param[in]  dir   Directory. .so files in this dir will be loaded.
+ * @param[in]  regexp Regexp for matching files in plugin directory. Default *.so.
+ * @retval     0     OK
+ * @retval     -1    Error
  */
 int
-plugin_unload(clicon_handle h, 
+clixon_plugins_load(clicon_handle h,
-	      plghndl_t    *handle)
+    		    char         *function,
+		    char         *dir,
+    		    char         *regexp)
 {
-    int        retval = 0;
+    int            retval = -1;
-    char      *error;
+    int            ndp;
+    struct dirent *dp = NULL;
+    int            i;
+    char           filename[MAXPATHLEN];
+    clixon_plugin *cp;
+
+    clicon_debug(1, "%s", __FUNCTION__); 
+    /* Get plugin objects names from plugin directory */
+    if((ndp = clicon_file_dirent(dir, &dp,
+				 regexp?regexp:"(.so)$", S_IFREG))<0)
+	goto done;
+    /* Load all plugins */
+    for (i = 0; i < ndp; i++) {
+	snprintf(filename, MAXPATHLEN-1, "%s/%s", dir, dp[i].d_name);
+	clicon_debug(1, "DEBUG: Loading plugin '%.*s' ...", 
+		     (int)strlen(filename), filename);
+	if ((cp = plugin_load_one(h, filename, function, RTLD_NOW)) == NULL)
+	    goto done;
+	_clixon_nplugins++;
+	if ((_clixon_plugins = realloc(_clixon_plugins, _clixon_nplugins*sizeof(clixon_plugin))) == NULL) {
+	    clicon_err(OE_UNIX, errno, "realloc");
+	    goto done;
+	}
+	_clixon_plugins[_clixon_nplugins-1] = *cp;
+	free(cp);
+    }
+    retval = 0;
+done:
+    if (dp)
+	free(dp);
+    return retval;
+}
+
+/*! Call plugin_start in all plugins
+ * @param[in]  h       Clicon handle
+ * @param[in]  argc    
+ * @param[in]  argv
+ * Call plugin start functions (if defined) with argc/argv multiple
+ * arguments.
+ * Typically the argc/argv are the ones appearing after "--", eg
+ * clicon_cli -f /etc/clicon.xml -- -a myopt
+ * In the example above argc=3 and
+ * argv[0]: clicon_cli
+ * argv[1]: -a
+ * argv[2]: myopt
+ */
+int
+clixon_plugin_start(clicon_handle h, 
+		    int           argc, 
+		    char        **argv)
+{
+    clixon_plugin *cp;
+    int            i;
+    plgstart_t    *startfn;          /* Plugin start */
+    
+    for (i = 0; i < _clixon_nplugins; i++) {
+	cp = &_clixon_plugins[i];
+	if ((startfn = cp->cp_api.ca_start) == NULL)
+	    continue;
+	//	optind = 0;
+	if (startfn(h, argc, argv) < 0) {
+	    clicon_debug(1, "plugin_start() failed\n");
+	    return -1;
+	}
+    }
+    return 0;
+}
+
+/*! Unload all plugins: call exit function and close shared handle
+ * @param[in]  h       Clicon handle
+ */
+int
+clixon_plugin_exit(clicon_handle h)
+{
+    clixon_plugin *cp;
     plgexit_t     *exitfn;
+    int            i;
+    char          *error;
     
-    /* Call exit function is it exists */
+    for (i = 0; i < _clixon_nplugins; i++) {
-    exitfn = dlsym(handle, PLUGIN_EXIT);
+	cp = &_clixon_plugins[i];
-    if (dlerror() == NULL)
+	if ((exitfn = cp->cp_api.ca_exit) == NULL)
-	exitfn(h);
+	    continue;
-
+	if (exitfn(h) < 0) {
-    dlerror();    /* Clear any existing error */
+	    clicon_debug(1, "plugin_exit() failed\n");
-    if (dlclose(handle) != 0) {
+	    return -1;
+	}
+	if (dlclose(cp->cp_handle) != 0) {
 	    error = (char*)dlerror();
 	    clicon_err(OE_PLUGIN, errno, "dlclose: %s\n", error ? error : "Unknown error");
-	/* Just report */
+	}
+    }
+    if (_clixon_plugins){
+	free(_clixon_plugins);
+	_clixon_plugins = NULL;
+    }
+    _clixon_nplugins = 0;
+    return 0;
+}
+
+
+/*! Run the restconf user-defined credentials callback if present
+ * Find first authentication callback and call that, then return.
+ * The callback is to set the authenticated user
+ * @param[in]  h    Clicon handle
+ * @param[in]  arg  Argument, such as fastcgi handler for restconf
+ * @retval    -1    Error
+ * @retval     0    Not authenticated
+ * @retval     1    Authenticated 
+ * @note If authenticated either a callback was called and clicon_username_set() 
+ *       Or no callback was found.
+ */
+int
+clixon_plugin_auth(clicon_handle h, 
+		   void         *arg)
+{
+    clixon_plugin *cp;
+    int            i;
+    plgauth_t     *authfn;          /* Plugin auth */
+    int            retval = 1;
+    
+    for (i = 0; i < _clixon_nplugins; i++) {
+	cp = &_clixon_plugins[i];
+	if ((authfn = cp->cp_api.ca_auth) == NULL)
+	    continue;
+	if ((retval = authfn(h, arg)) < 0) {
+	    clicon_debug(1, "plugin_auth() failed\n");
+	    return -1;
+	}
+	break;
     }
     return retval;
 }
+
+/*--------------------------------------------------------------------
+ * RPC callbacks for both client/frontend and backend plugins.
+ * RPC callbacks are explicitly registered in the plugin_init() function
+ * with a tag and a function
+ * WHen the the tag is encountered, the callback is called.
+ * Primarily backend, but also netconf and restconf frontend plugins.
+ * CLI frontend so far have direct callbacks, ie functions in the cligen
+ * specification are directly dlsym:ed to the CLI plugin.
+ * It would be possible to use this rpc registering API for CLI plugins as well.
+ * 
+ * @note may have a problem if callbacks are of different types
+ */
+typedef struct {
+    qelem_t 	  rc_qelem;	/* List header */
+    clicon_rpc_cb rc_callback;  /* RPC Callback */
+    void	  *rc_arg;	/* Application specific argument to cb */
+    char          *rc_tag;	/* Xml/json tag when matched, callback called */
+} rpc_callback_t;
+
+/* List of rpc callback entries */
+static rpc_callback_t *rpc_cb_list = NULL;
+
+/*! Register a RPC callback
+ * Called from plugin to register a callback for a specific netconf XML tag.
+ *
+ * @param[in]  h       clicon handle
+ * @param[in]  cb,     Callback called 
+ * @param[in]  arg,    Domain-specific argument to send to callback 
+ * @param[in]  tag     Xml tag when callback is made 
+ * @see rpc_callback_call
+ */
+int
+rpc_callback_register(clicon_handle  h,
+		      clicon_rpc_cb  cb,
+		      void          *arg,       
+		      char          *tag)
+{
+    rpc_callback_t *rc;
+
+    if ((rc = malloc(sizeof(rpc_callback_t))) == NULL) {
+	clicon_err(OE_DB, errno, "malloc: %s", strerror(errno));
+	goto done;
+    }
+    memset (rc, 0, sizeof (*rc));
+    rc->rc_callback = cb;
+    rc->rc_arg  = arg;
+    rc->rc_tag  = strdup(tag); /* XXX strdup memleak */
+    INSQ(rc, rpc_cb_list);
+    return 0;
+ done:
+    if (rc){
+	if (rc->rc_tag)
+	    free(rc->rc_tag);
+	free(rc);
+    }
+    return -1;
+}
+
+/*! Delete all RPC callbacks
+ */
+int
+rpc_callback_delete_all(void)
+{
+    rpc_callback_t *rc;
+
+    while((rc = rpc_cb_list) != NULL) {
+	DELQ(rc, rpc_cb_list, rpc_callback_t *);
+	if (rc->rc_tag)
+	    free(rc->rc_tag);
+	free(rc);
+    }
+    return 0;
+}
+
+/*! Search RPC callbacks and invoke if XML match with tag
+ *
+ * @param[in]  h       clicon handle
+ * @param[in]  xn      Sub-tree (under xorig) at child of rpc: <rpc><xn></rpc>.
+ * @param[out] xret    Return XML, error or OK
+ * @param[in]  arg   Domain-speific arg (eg client_entry)
+ *
+ * @retval -1   Error
+ * @retval  0   OK, not found handler.
+ * @retval  1   OK, handler called
+ */
+int
+rpc_callback_call(clicon_handle h,
+		  cxobj        *xe, 
+		  cbuf         *cbret,
+		  void         *arg)
+{
+    int            retval = -1;
+    rpc_callback_t *rc;
+
+    if (rpc_cb_list == NULL)
+	return 0;
+    rc = rpc_cb_list;
+    do {
+	if (strcmp(rc->rc_tag, xml_name(xe)) == 0){
+	    if ((retval = rc->rc_callback(h, xe, cbret, arg, rc->rc_arg)) < 0){
+		clicon_debug(1, "%s Error in: %s", __FUNCTION__, rc->rc_tag);
+		goto done;
+	    }
+	    else{
+		retval = 1; /* handled */
+		goto done;
+	    }
+	}
+	rc = NEXTQ(rpc_callback_t *, rc);
+    } while (rc != rpc_cb_list);
+    retval = 0;
+ done:
+    clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
+    return retval;
+}

lib/src/clixon_proto_client.c

@@ -47,6 +47,7 @@
 #include <errno.h>
 #include <assert.h>
 #include <unistd.h>
+#include <sys/param.h>
 #include <sys/stat.h>
 #include <sys/socket.h>
 #include <sys/syslog.h>
@@ -60,9 +61,9 @@
 #include "clixon_hash.h"
 #include "clixon_handle.h"
 #include "clixon_yang.h"
-#include "clixon_plugin.h"
 #include "clixon_options.h"
 #include "clixon_xml.h"
+#include "clixon_plugin.h"
 #include "clixon_xsl.h"
 #include "clixon_proto.h"
 #include "clixon_err.h"
@@ -90,6 +91,7 @@ clicon_rpc_msg(clicon_handle      h,
     cxobj             *xret = NULL;
     yang_spec         *yspec;
 
+    clicon_debug(1, "%s request:%s", __FUNCTION__, msg->op_body);
     if ((sock = clicon_sock(h)) == NULL){
 	clicon_err(OE_FATAL, 0, "CLICON_SOCK option not set");
 	goto done;
@@ -200,7 +202,7 @@ clicon_rpc_netconf_xml(clicon_handle  h,
     return retval;
 }
 
-/*! Generate clicon error function call from Netconf error message
+/*! Generate and log clicon error function call from Netconf error message
  * @param[in]  xerr    Netconf error message on the level: <rpc-reply><rpc-error>
  */
 int
@@ -264,10 +266,14 @@ clicon_rpc_get_config(clicon_handle       h,
     cbuf              *cb = NULL;
     cxobj             *xret = NULL;
     cxobj             *xd;
+    char              *username;
     
     if ((cb = cbuf_new()) == NULL)
 	goto done;
-    cprintf(cb, "<rpc><get-config><source><%s/></source>", db);
+    cprintf(cb, "<rpc");
+    if ((username = clicon_username_get(h)) != NULL)
+	cprintf(cb, " username=\"%s\"", username);
+    cprintf(cb, "><get-config><source><%s/></source>", db);
     if (xpath && strlen(xpath))
 	cprintf(cb, "<filter type=\"xpath\" select=\"%s\"/>", xpath);
     cprintf(cb, "</get-config></rpc>");
@@ -303,7 +309,7 @@ clicon_rpc_get_config(clicon_handle       h,
  * @param[in] op         Operation on database item: OP_MERGE, OP_REPLACE
  * @param[in] xml        XML string. Ex: <config><a>..</a><b>...</b></config>
  * @retval    0          OK
- * @retval   -1          Error
+ * @retval   -1          Error and logged to syslog
  * @note xml arg need to have <config> as top element
  * @code
  * if (clicon_rpc_edit_config(h, "running", OP_MERGE, 
@@ -322,10 +328,14 @@ clicon_rpc_edit_config(clicon_handle       h,
     cbuf              *cb = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
     if ((cb = cbuf_new()) == NULL)
 	goto done;
-    cprintf(cb, "<rpc><edit-config><target><%s/></target>", db);
+    cprintf(cb, "<rpc");
+    if ((username = clicon_username_get(h)) != NULL)
+	cprintf(cb, " username=\"%s\"", username);
+    cprintf(cb, "><edit-config><target><%s/></target>", db);
     cprintf(cb, "<default-operation>%s</default-operation>", 
 	    xml_operation2str(op));
     if (xmlstr)
@@ -356,6 +366,8 @@ clicon_rpc_edit_config(clicon_handle       h,
  * @param[in] h        CLICON handle
  * @param[in] db1      src database, eg "running"
  * @param[in] db2      dst database, eg "startup"
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  * @code
  * if (clicon_rpc_copy_config(h, "running", "startup") < 0)
  *    err;
@@ -370,8 +382,12 @@ clicon_rpc_copy_config(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><copy-config><source><%s/></source><target><%s/></target></copy-config></rpc>", db1, db2)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><copy-config><source><%s/></source><target><%s/></target></copy-config></rpc>",
+				 username?username:"",
+				 db1, db2)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -391,6 +407,8 @@ clicon_rpc_copy_config(clicon_handle h,
 /*! Send a request to backend to delete a config database
  * @param[in] h        CLICON handle
  * @param[in] db       database, eg "running"
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  * @code
  * if (clicon_rpc_delete_config(h, "startup") < 0)
  *    err;
@@ -404,8 +422,11 @@ clicon_rpc_delete_config(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><delete-config><target><%s/></target></delete-config></rpc>", db)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><delete-config><target><%s/></target></delete-config></rpc>",
+				 username?username:"", db)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -425,6 +446,8 @@ clicon_rpc_delete_config(clicon_handle h,
 /*! Lock a database
  * @param[in] h        CLICON handle
  * @param[in] db       database, eg "running"
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_lock(clicon_handle h, 
@@ -434,8 +457,11 @@ clicon_rpc_lock(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><lock><target><%s/></target></lock></rpc>", db)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><lock><target><%s/></target></lock></rpc>",
+				 username?username:"", db)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -455,6 +481,8 @@ clicon_rpc_lock(clicon_handle h,
 /*! Unlock a database
  * @param[in] h        CLICON handle
  * @param[in] db       database, eg "running"
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_unlock(clicon_handle h, 
@@ -464,8 +492,10 @@ clicon_rpc_unlock(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><unlock><target><%s/></target></unlock></rpc>", db)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><unlock><target><%s/></target></unlock></rpc>", username?username:"", db)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -512,10 +542,14 @@ clicon_rpc_get(clicon_handle       h,
     cbuf              *cb = NULL;
     cxobj             *xret = NULL;
     cxobj             *xd;
+    char              *username;
 
     if ((cb = cbuf_new()) == NULL)
 	goto done;
-    cprintf(cb, "<rpc><get>");
+    cprintf(cb, "<rpc");
+    if ((username = clicon_username_get(h)) != NULL)
+	cprintf(cb, " username=\"%s\"", username);
+    cprintf(cb, "><get>");
     if (xpath && strlen(xpath))
 	cprintf(cb, "<filter type=\"xpath\" select=\"%s\"/>", xpath);
     cprintf(cb, "</get></rpc>");
@@ -547,6 +581,8 @@ clicon_rpc_get(clicon_handle       h,
 
 /*! Close a (user) session
  * @param[in] h        CLICON handle
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_close_session(clicon_handle h)
@@ -555,8 +591,11 @@ clicon_rpc_close_session(clicon_handle h)
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><close-session/></rpc>")) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><close-session/></rpc>",
+				 username?username:"")) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -576,6 +615,8 @@ clicon_rpc_close_session(clicon_handle h)
 /*! Kill other user sessions
  * @param[in] h           CLICON handle
  * @param[in] session_id  Session id of other user session
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_kill_session(clicon_handle h,
@@ -585,8 +626,11 @@ clicon_rpc_kill_session(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><kill-session><session-id>%d</session-id></kill-session></rpc>", session_id)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><kill-session><session-id>%d</session-id></kill-session></rpc>",
+				 username?username:"", session_id)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -607,6 +651,7 @@ clicon_rpc_kill_session(clicon_handle h,
  * @param[in] h        CLICON handle
  * @param[in] db       Name of database
  * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_validate(clicon_handle h, 
@@ -616,8 +661,10 @@ clicon_rpc_validate(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><validate><source><%s/></source></validate></rpc>", db)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><validate><source><%s/></source></validate></rpc>", username?username:"", db)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -637,6 +684,7 @@ clicon_rpc_validate(clicon_handle h,
 /*! Commit changes send a commit request to backend daemon
  * @param[in] h          CLICON handle
  * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_commit(clicon_handle h)
@@ -645,8 +693,10 @@ clicon_rpc_commit(clicon_handle h)
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><commit/></rpc>")) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><commit/></rpc>", username?username:"")) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -666,6 +716,7 @@ clicon_rpc_commit(clicon_handle h)
 /*! Discard all changes in candidate / revert to running
  * @param[in] h        CLICON handle
  * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_discard_changes(clicon_handle h)
@@ -674,8 +725,10 @@ clicon_rpc_discard_changes(clicon_handle h)
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><discard-changes/></rpc>")) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><discard-changes/></rpc>", username?username:"")) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;
@@ -697,6 +750,9 @@ clicon_rpc_discard_changes(clicon_handle h)
  * @param{in]   stream   name of notificatio/log stream (CLICON is predefined)
  * @param{in]   filter   message filter, eg xpath for xml notifications
  * @param[out]  s0       socket returned where notification mesages will appear
+ * @retval      0        OK
+ * @retval      -1       Error and logged to syslog
+
  * @note When using netconf create-subsrciption,status and format is not supported
  */
 int
@@ -709,11 +765,14 @@ clicon_rpc_create_subscription(clicon_handle    h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><create-subscription>"
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><create-subscription>"
 				 "<stream>%s</stream>"
 				 "<filter>%s</filter>"
 				 "</create-subscription></rpc>", 
+				 username?username:"",
 				 stream?stream:"", filter?filter:"")) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, s0) < 0)
@@ -734,6 +793,8 @@ clicon_rpc_create_subscription(clicon_handle    h,
 /*! Send a debug request to backend server
  * @param[in] h        CLICON handle
  * @param[in] level    Debug level
+ * @retval    0        OK
+ * @retval   -1        Error and logged to syslog
  */
 int
 clicon_rpc_debug(clicon_handle h, 
@@ -743,8 +804,10 @@ clicon_rpc_debug(clicon_handle h,
     struct clicon_msg *msg = NULL;
     cxobj             *xret = NULL;
     cxobj             *xerr;
+    char              *username;
 
-    if ((msg = clicon_msg_encode("<rpc><debug><level>%d</level></debug></rpc>", level)) == NULL)
+    username = clicon_username_get(h);
+    if ((msg = clicon_msg_encode("<rpc username=\"%s\"><debug><level>%d</level></debug></rpc>", username?username:"", level)) == NULL)
 	goto done;
     if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
 	goto done;

lib/src/clixon_string.c

@@ -140,7 +140,7 @@ clicon_strjoin(int         argc,
 }
 
 static int
-unreserved(unsigned char in)
+uri_unreserved(unsigned char in)
 {
     switch(in) {
     case '0': case '1': case '2': case '3': case '4':
@@ -163,11 +163,17 @@ unreserved(unsigned char in)
     return 0;
 }
 
-/*! Percent encoding according to RFC 3896 
+/*! Percent encoding according to RFC 3986 URI Syntax
- * @param[out]  esc   Deallocate with free()
+ * @param[in]   str    Not-encoded input string     
+ * @param[out]  escp   Encoded/escaped malloced output string
+ * @retval      0      OK
+ * @retval     -1      Error
+ * @see RFC 3986 Uniform Resource Identifier (URI): Generic Syntax
+ * @see uri_percent_decode
+ * @see xml_chardata_encode
  */
 int
-percent_encode(char  *str, 
+uri_percent_encode(char  *str, 
 		   char **escp)
 {
     int   retval = -1;
@@ -184,7 +190,7 @@ percent_encode(char  *str,
     memset(esc, 0, len);
     j = 0;
     for (i=0; i<strlen(str); i++){
-	if (unreserved(str[i]))
+	if (uri_unreserved(str[i]))
 	    esc[j++] = str[i];
 	else{
 	    snprintf(&esc[j], 4, "%%%02X", str[i]&0xff);
@@ -199,11 +205,16 @@ percent_encode(char  *str,
     return retval;
 }
 
-/*! Percent decoding according to RFC 3896 
+/*! Percent decoding according to RFC 3986 URI Syntax
- * @param[out]  str   Deallocate with free()
+ * @param[in]   esc    Escaped/encoded input string     
+ * @param[out]  strp   Decoded malloced output string. Deallocate with free()
+ * @retval      0      OK
+ * @retval     -1      Error
+ * @see RFC 3986 Uniform Resource Identifier (URI): Generic Syntax
+ * @see uri_percent_encode
  */
 int
-percent_decode(char  *esc, 
+uri_percent_decode(char  *esc, 
 		   char **strp)
 {
     int   retval = -1;
@@ -243,6 +254,98 @@ percent_decode(char  *esc,
     return retval;
 }
 
+/*! Escape characters according to XML definition
+ * @param[in]   str    Not-encoded input string     
+ * @param[out]  escp   Encoded/escaped malloced output string
+ * @retval      0      OK
+ * @retval     -1      Error
+ * @see https://www.w3.org/TR/2008/REC-xml-20081126/#syntax chapter 2.6
+ * @see uri_percent_encode
+ * @see AMPERSAND mode in clixon_xml_parse.l
+ * @code
+ *   char *encstr = NULL;
+ *   char *val = "a<>b";
+ *   if (xml_chardata_encode(str, &encstr) < 0)
+ *      err;
+ *   if (encstr)
+ *      free(encstr);
+ * @endcode
+ * Essentially encode as follows:
+ *     & -> "&amp; "   must
+ *     < -> "&lt; "    must
+ *     > -> "&gt; "    must for backward compatibility
+ *     ' -> "&apos; "  may
+ *     ' -> "&quot; "  may
+ * Optionally >
+ */
+int
+xml_chardata_encode(char  *str, 
+		    char **escp)
+{
+    int   retval = -1;
+    char *esc = NULL;
+    int   l;
+    int   len;
+    int   i, j;
+    
+    len = 0;
+    for (i=0; i<strlen(str); i++){
+	switch (str[i]){
+	case '&':
+	    len += strlen("&amp; ");
+	    break;
+	case '<':
+	    len += strlen("&lt; ");
+	    break;
+	case '>':
+	    len += strlen("&gt; ");
+	    break;
+	default:
+	    len++;
+	}
+    }
+    len++; /* trailing \0 */
+    if ((esc = malloc(len)) == NULL){
+	clicon_err(OE_UNIX, errno, "malloc"); 
+	goto done;
+    }
+    memset(esc, 0, len);
+    j = 0;
+    for (i=0; i<strlen(str); i++){
+	switch (str[i]){
+	case '&':
+	    if ((l=snprintf(&esc[j], 7, "&amp; ")) < 0){
+		clicon_err(OE_UNIX, errno, "snprintf");
+		goto done;
+	    }
+	    j += l;
+	    break;
+	case '<':
+	    if ((l=snprintf(&esc[j], 6, "&lt; ")) < 0){
+		clicon_err(OE_UNIX, errno, "snprintf");
+		goto done;
+	    }
+	    j += l;
+	    break;
+	case '>':
+	    if ((l=snprintf(&esc[j], 6, "&gt; ")) < 0){
+		clicon_err(OE_UNIX, errno, "snprintf");
+		goto done;
+	    }
+	    j += l;
+	    break;
+	default:
+	    esc[j++] = str[i];
+	}
+    }
+    *escp = esc;
+    retval = 0;
+ done:
+    if (retval < 0 && esc)
+	free(esc);
+    return retval;
+}
+
 /*! Split a string into a cligen variable vector using 1st and 2nd delimiter 
  * Split a string first into elements delimited by delim1, then into
  * pairs delimited by delim2.
@@ -295,7 +398,7 @@ str2cvec(char  *string,
 	    *(snext++) = '\0';
 	if ((val = index(s, delim2)) != NULL){
 	    *(val++) = '\0';
-	    if (percent_decode(val, &valu) < 0)
+	    if (uri_percent_decode(val, &valu) < 0)
 		goto err;
 	    if ((cv = cvec_add(cvv, CGV_STRING)) == NULL){
 		clicon_err(OE_UNIX, errno, "cvec_add");

lib/src/clixon_xml.c

@@ -57,7 +57,6 @@
 #include "clixon_err.h"
 #include "clixon_log.h"
 #include "clixon_string.h"
-
 #include "clixon_queue.h"
 #include "clixon_hash.h"
 #include "clixon_handle.h"
@@ -165,7 +164,7 @@ xml_name_set(cxobj *xn,
     }
     if (name){
 	if ((xn->x_name = strdup(name)) == NULL){
-	    clicon_err(OE_XML, errno, "%s: strdup", __FUNCTION__);
+	    clicon_err(OE_XML, errno, "strdup");
 	    return -1;
 	}
     }
@@ -198,7 +197,7 @@ xml_namespace_set(cxobj *xn,
     }
     if (namespace){
 	if ((xn->x_namespace = strdup(namespace)) == NULL){
-	    clicon_err(OE_XML, errno, "%s: strdup", __FUNCTION__);
+	    clicon_err(OE_XML, errno, "strdup");
 	    return -1;
 	}
     }
@@ -289,7 +288,7 @@ xml_value_set(cxobj *xn,
     }
     if (val){
 	if ((xn->x_value = strdup(val)) == NULL){
-	    clicon_err(OE_XML, errno, "%s: strdup", __FUNCTION__);
+	    clicon_err(OE_XML, errno, "strdup");
 	    return -1;
 	}
     }
@@ -434,7 +433,7 @@ xml_child_i_set(cxobj *xt,
 
 /*! Iterator over xml children objects
  *
- * NOTE: Never manipulate the child-list during operation or using the
+ * @note Never manipulate the child-list during operation or using the
  * same object recursively, the function uses an internal field to remember the
  * index used. It works as long as the same object is not iterated concurrently. 
  *
@@ -481,7 +480,7 @@ xml_child_append(cxobj *x,
     x->x_childvec_len++;
     x->x_childvec = realloc(x->x_childvec, x->x_childvec_len*sizeof(cxobj*));
     if (x->x_childvec == NULL){
-	clicon_err(OE_XML, errno, "%s: realloc", __FUNCTION__);
+	clicon_err(OE_XML, errno, "realloc");
 	return -1;
     }
     x->x_childvec[x->x_childvec_len-1] = xc;
@@ -539,7 +538,7 @@ xml_new(char      *name,
     cxobj *x;
     
     if ((x = malloc(sizeof(cxobj))) == NULL){
-	clicon_err(OE_XML, errno, "%s: malloc", __FUNCTION__);
+	clicon_err(OE_XML, errno, "malloc");
 	return NULL;
     }
     memset(x, 0, sizeof(cxobj));
@@ -951,7 +950,7 @@ xml_free(cxobj *x)
  * XML printing functions. Output a parse tree to file, string cligen buf
  *------------------------------------------------------------------------*/
 
-/*! Print an XML tree structure to an output stream
+/*! Print an XML tree structure to an output stream and encode chars "<>&"
  *
  * Uses clicon_xml2cbuf internally
  *
@@ -976,13 +975,17 @@ clicon_xml2file(FILE  *f,
     int    hasbody;
     int    haselement;
     char  *val;
+    char  *encstr = NULL; /* xml encoded string */
     
     name = xml_name(x);
     namespace = xml_namespace(x);
     switch(xml_type(x)){
     case CX_BODY:
-	if ((val = xml_value(x)) != NULL) /* incomplete tree */
+	if ((val = xml_value(x)) == NULL) /* incomplete tree */
-	    fprintf(f, "%s", xml_value(x));
+	    break;
+	if (xml_chardata_encode(val, &encstr) < 0)
+	    goto done;
+	fprintf(f, "%s", encstr);
 	break;
     case CX_ATTR:
 	fprintf(f, " ");
@@ -1045,6 +1048,8 @@ clicon_xml2file(FILE  *f,
     }/* switch */
     retval = 0;
  done:
+    if (encstr)
+	free(encstr);
     return retval;
 }
 
@@ -1064,8 +1069,7 @@ xml_print(FILE  *f,
     return clicon_xml2file(f, xn, 0, 1);
 }
 
-
+/*! Print an XML tree structure to a cligen buffer and encode chars "<>&"
-/*! Print an XML tree structure to a cligen buffer
  *
  * @param[in,out] cb          Cligen buffer to write to
  * @param[in]     xn          Clicon xml tree
@@ -1094,12 +1098,18 @@ clicon_xml2cbuf(cbuf  *cb,
     int    hasbody;
     int    haselement;
     char  *namespace;
+    char  *encstr = NULL; /* xml encoded string */
+    char  *val;
     
     name = xml_name(x);
     namespace = xml_namespace(x);
     switch(xml_type(x)){
     case CX_BODY:
-	cprintf(cb, "%s", xml_value(x));
+	if ((val = xml_value(x)) == NULL) /* incomplete tree */
+	    break;
+	if (xml_chardata_encode(val, &encstr) < 0)
+	    goto done;
+	cprintf(cb, "%s", encstr);
 	break;
     case CX_ATTR:
 	cprintf(cb, " ");
@@ -1131,7 +1141,6 @@ clicon_xml2cbuf(cbuf  *cb,
 	    default:
 		break;
 	    }
-	
 	/* Check for special case <a/> instead of <a></a> */
 	if (hasbody==0 && haselement==0) 
 	    cprintf(cb, "/>");
@@ -1159,6 +1168,8 @@ clicon_xml2cbuf(cbuf  *cb,
     }/* switch */
     retval = 0;
  done:
+    if (encstr)
+	free(encstr);
     return retval;
 }
 /*! Print actual xml tree datastructures (not xml), mainly for debugging
@@ -1303,15 +1314,14 @@ xml_parse_file(int        fd,
     if (endtag != NULL)
 	endtaglen = strlen(endtag);
     if ((xmlbuf = malloc(xmlbuflen)) == NULL){
-	clicon_err(OE_XML, errno, "%s: malloc", __FUNCTION__);
+	clicon_err(OE_XML, errno, "malloc");
 	goto done;
     }
     memset(xmlbuf, 0, xmlbuflen);
     ptr = xmlbuf;
     while (1){
 	if ((ret = read(fd, &ch, 1)) < 0){
-	    clicon_err(OE_XML, errno, "%s: read: [pid:%d]\n", 
+	    clicon_err(OE_XML, errno, "read: [pid:%d]\n", 
-		    __FUNCTION__,
 		    (int)getpid());
 	    break;
 	}
@@ -1334,7 +1344,7 @@ xml_parse_file(int        fd,
 	    oldxmlbuflen = xmlbuflen;
 	    xmlbuflen *= 2;
 	    if ((xmlbuf = realloc(xmlbuf, xmlbuflen)) == NULL){
-		clicon_err(OE_XML, errno, "%s: realloc", __FUNCTION__);
+		clicon_err(OE_XML, errno, "realloc");
 		goto done;
 	    }
 	    memset(xmlbuf+oldxmlbuflen, 0, xmlbuflen-oldxmlbuflen);
@@ -1424,10 +1434,7 @@ xml_parse_va(cxobj     **xtop,
     va_start(args, format);
     len = vsnprintf(str, len, format, args) + 1;
     va_end(args);
-    if (*xtop == NULL)
+    if (xml_parse_string(str, yspec, xtop) < 0)
-	if ((*xtop = xml_new(XML_TOP_SYMBOL, NULL, NULL)) == NULL)
-	    goto done;
-    if (_xml_parse(str, yspec, *xtop) < 0)
 	goto done;
     retval = 0;
  done:
@@ -1447,7 +1454,7 @@ xml_copy_one(cxobj *x0,
     xml_type_set(x1, xml_type(x0));
     if (xml_value(x0)){ /* malloced string */
 	if ((x1->x_value = strdup(x0->x_value)) == NULL){
-	    clicon_err(OE_XML, errno, "%s: strdup", __FUNCTION__);
+	    clicon_err(OE_XML, errno, "strdup");
 	    return -1;
 	}
     }
@@ -1456,7 +1463,7 @@ xml_copy_one(cxobj *x0,
 	    return -1;
     if (xml_cv_get(x0)){
       if ((cv1 = cv_dup(xml_cv_get(x0))) == NULL){
-	clicon_err(OE_XML, errno, "%s: cv_dup", __FUNCTION__);
+	clicon_err(OE_XML, errno, "cv_dup");
 	return -1;
       }
       if ((xml_cv_set(x1, cv1)) < 0)
@@ -1553,7 +1560,7 @@ cxvec_append(cxobj   *x,
     int retval = -1;
 
     if ((*vec = realloc(*vec, sizeof(cxobj *) * (*len+1))) == NULL){
-	clicon_err(OE_XML, errno, "%s: realloc", __FUNCTION__);
+	clicon_err(OE_XML, errno, "realloc");
 	goto done;
     }
     (*vec)[(*len)++] = x;

lib/src/clixon_xml_db.c

@@ -43,9 +43,11 @@
 #include <fcntl.h>
 #include <time.h>
 #include <signal.h>
+#include <libgen.h>
 #include <sys/time.h>
 #include <sys/wait.h>
-#include <libgen.h>
+#include <sys/param.h>
+
 
 /* cligen */
 #include <cligen/cligen.h>
@@ -322,7 +324,7 @@ xmldb_setopt(clicon_handle h,
  * @param[in]  dbname Name of database to search in (filename including dir path
  * @param[in]  xpath  String with XPATH syntax. or NULL for all
  * @param[in]  config If set only configuration data, else also state
- * @param[out] xtop   Single XML tree. Free with xml_free()
+ * @param[out] xret   Single return XML tree. Free with xml_free()
  * @retval     0      OK
  * @retval     -1     Error
  * @code
@@ -339,7 +341,7 @@ xmldb_get(clicon_handle h,
 	  const char   *db, 
 	  char         *xpath,
 	  int           config,
-	  cxobj       **xtop)
+	  cxobj       **xret)
 {
     int               retval = -1;
     xmldb_handle      xh;
@@ -357,11 +359,11 @@ xmldb_get(clicon_handle h,
 	clicon_err(OE_DB, 0, "Not connected to datastore plugin");
 	goto done;
     }
-    retval = xa->xa_get_fn(xh, db, xpath, config, xtop);
+    retval = xa->xa_get_fn(xh, db, xpath, config, xret);
 #if DEBUG
     if (retval == 0) { 
 	 cbuf *cb = cbuf_new();
-	 clicon_xml2cbuf(cb, *xtop, 0, 0);
+	 clicon_xml2cbuf(cb, *xret, 0, 0);
 	 clicon_log(LOG_WARNING, "%s: db:%s xpath:%s xml:%s", 
 		    __FUNCTION__, db, xpath, cbuf_get(cb));
 	 cbuf_free(cb);
@@ -377,25 +379,29 @@ xmldb_get(clicon_handle h,
  * @param[in]  db     running or candidate
  * @param[in]  op     Top-level operation, can be superceded by other op in tree
  * @param[in]  xt     xml-tree. Top-level symbol is dummy
+ * @param[out] cbret  Initialized cligen buffer or NULL. On exit contains XML or "".
  * @retval     0      OK
  * @retval     -1     Error
  * The xml may contain the "operation" attribute which defines the operation.
  * @code
  *   cxobj     *xt;
+ *   cxobj     *xret = NULL;
  *   if (xml_parse_string("<a>17</a>", yspec, &xt) < 0)
  *     err;
- *   if (xmldb_put(xh, "running", OP_MERGE, xt) < 0)
+ *   if (xmldb_put(xh, "running", OP_MERGE, xt, cbret) < 0)
  *     err;
  * @endcode
  * @note that you can add both config data and state data. In comparison,
  *  xmldb_get has a parameter to get config data only.
+ * @note if xret is non-null, it may contain error message
  *
  */
 int 
 xmldb_put(clicon_handle       h, 
 	  const char         *db, 
 	  enum operation_type op, 
-	  cxobj              *xt)
+	  cxobj              *xt,
+	  cbuf               *cbret)
 {
     int               retval = -1;
     xmldb_handle      xh;
@@ -425,7 +431,7 @@ xmldb_put(clicon_handle       h,
 	cbuf_free(cb);
     }
 #endif
-    retval = xa->xa_put_fn(xh, db, op, xt);
+    retval = xa->xa_put_fn(xh, db, op, xt, cbret);
  done:
     return retval;
 }

lib/src/clixon_xml_map.c

@@ -61,10 +61,11 @@
 #include <errno.h>
 #include <ctype.h>
 #include <string.h>
-#include <arpa/inet.h>
-#include <assert.h>
 #include <syslog.h>
 #include <fcntl.h>
+#include <assert.h>
+#include <arpa/inet.h>
+#include <sys/param.h>
 #include <netinet/in.h>
 
 /* cligen */
@@ -79,28 +80,15 @@
 #include "clixon_string.h"
 #include "clixon_yang.h"
 #include "clixon_yang_type.h"
-#include "clixon_plugin.h"
 #include "clixon_options.h"
 #include "clixon_xml.h"
+#include "clixon_plugin.h"
 #include "clixon_xsl.h"
 #include "clixon_log.h"
 #include "clixon_err.h"
 #include "clixon_xml_sort.h"
 #include "clixon_xml_map.h"
 
-/*
- * A node is a leaf if it contains a body.
- */
-static cxobj *
-leaf(cxobj *xn)
-{
-    cxobj *xc = NULL;
-
-    while ((xc = xml_child_each(xn, xc, CX_BODY)) != NULL)
-	break;
-    return xc;
-}
-
 /*! x is element and has eactly one child which in turn has none */
 static int
 tleaf(cxobj *x)
@@ -117,6 +105,7 @@ tleaf(cxobj *x)
 
 /*! Translate XML -> TEXT
  * @param[in]  level  print 4 spaces per level in front of each line
+ * XXX rewrite using YANG and remove encrypted password KLUDGE
  */
 int 
 xml2txt(FILE  *f, 
@@ -127,25 +116,18 @@ xml2txt(FILE  *f,
     int    children=0;
     char  *term = NULL;
     int    retval = -1;
-    int    encr=0;
 
     xe = NULL;     /* count children */
     while ((xe = xml_child_each(x, xe, -1)) != NULL)
 	children++;
     if (!children){ 
 	if (xml_type(x) == CX_BODY){
-	    /* Kludge for escaping encrypted passwords */
-	    if (strcmp(xml_name(xml_parent(x)), "encrypted-password")==0)
-		encr++;
 	    term = xml_value(x);
 	}
 	else{
 	    fprintf(f, "%*s", 4*level, "");
 	    term = xml_name(x);
 	}
-	if (encr)
-	    fprintf(f, "\"%s\";\n", term);
-	else
 	fprintf(f, "%s;\n", term);
 	retval = 0;
 	goto done;
@@ -183,67 +165,60 @@ xml2cli(FILE              *f,
 {
     int              retval = -1;
     cxobj           *xe = NULL;
-    char            *term;
+    cbuf            *cbpre = NULL;
-    int              bool;
+    yang_stmt       *ys;
-    int              nr;
+    int              match;
-    int              i;
+    char            *body;
-    cbuf            *cbpre;
 
+    ys = xml_spec(x);
+    if (ys->ys_keyword == Y_LEAF || ys->ys_keyword == Y_LEAF_LIST){
+	if (prepend0)
+	    fprintf(f, "%s", prepend0);
+	body = xml_body(x);
+	if (gt == GT_ALL || gt == GT_VARS)
+	    fprintf(f, "%s ", xml_name(x));
+	if (index(body, ' '))
+	    fprintf(f, "\"%s\"", body);
+	else
+	    fprintf(f, "%s", body);
+	fprintf(f, "\n");
+	goto ok;
+    }
     /* Create prepend variable string */
     if ((cbpre = cbuf_new()) == NULL){
 	clicon_err(OE_PLUGIN, errno, "cbuf_new");	
 	goto done;
     }
-    nr = xml_child_nr(x);
-    if (!nr){
-	if (xml_type(x) == CX_BODY)
-	    term = xml_value(x);
-	else
-	    term = xml_name(x);
-	if (prepend0)
-	    fprintf(f, "%s ", prepend0);
-	if (index(term, ' '))
-	    fprintf(f, "\"%s\"\n", term);
-	else
-	    fprintf(f, "%s\n", term);
-	retval = 0;
-	goto done;
-    }
     if (prepend0)
 	cprintf(cbpre, "%s", prepend0);
-/* bool determines when to print a variable keyword:
+    cprintf(cbpre, "%s ", xml_name(x));
-   !leaf           T for all (ie parameter)
+
-   index GT_NONE   F
+    if (ys->ys_keyword == Y_LIST){
-   index GT_VARS   F
+	/* If list then first loop through keys */
-   index GT_ALL    T
-   !index GT_NONE  F
-   !index GT_VARS  T
-   !index GT_ALL   T
- */
-    bool = !leaf(x) || gt == GT_ALL || (gt == GT_VARS);
-//    bool = (!x->xn_index || gt == GT_ALL);
-    if (bool){
-	if (cbuf_len(cbpre))
-	    cprintf(cbpre, " ");
-	cprintf(cbpre, "%s", xml_name(x));
-    }
 	xe = NULL;
-    /* First child is unique, then add that, before looping. */
-    i = 0;
 	while ((xe = xml_child_each(x, xe, -1)) != NULL){
-	/* Dont call this if it is index and there are other following */
+	    if ((match = yang_key_match((yang_node*)ys, xml_name(xe))) < 0)
-	if (0 && i < nr-1) 
+		goto done;
-	    ;
+	    if (!match)
-	else
+		continue;
+	    if (gt == GT_ALL)
+		cprintf(cbpre, "%s ", xml_name(xe));
+	    cprintf(cbpre, "%s ", xml_body(xe));
+	}
+    }
+    /* Then loop through all other (non-keys) */
+    xe = NULL;
+    while ((xe = xml_child_each(x, xe, -1)) != NULL){
+	if (ys->ys_keyword == Y_LIST){
+	    if ((match = yang_key_match((yang_node*)ys, xml_name(xe))) < 0)
+		goto done;
+	    if (match)
+		continue;
+	}
 	if (xml2cli(f, xe, cbuf_get(cbpre), gt) < 0)
 	    goto done;
-	if (0){ /* assume index is first, otherwise need one more while */
-	    if (gt == GT_ALL)
-		cprintf(cbpre, " %s", xml_name(xe));
-	    cprintf(cbpre, " %s", xml_value(xml_child_i(xe, 0)));
-	}
-	i++;
     }
+ ok:
     retval = 0;
   done:
     if (cbpre)
@@ -268,7 +243,6 @@ validate_leafref(cxobj     *xt,
     char        *leafrefbody;
     char        *leafbody;
 
-
     if ((leafrefbody = xml_body(xt)) == NULL)
 	return 0;
     if ((ypath = yang_find((yang_node*)ytype, Y_PATH, NULL)) == NULL){
@@ -316,9 +290,11 @@ xml_yang_validate_add(cxobj   *xt,
     yang_stmt *ys;
     char      *body;
     
-    /* if not given by argument (overide) use default link */
+    /* if not given by argument (overide) use default link 
-    if ((ys = xml_spec(xt)) != NULL){
+       and !Node has a config sub-statement and it is false */
+    if ((ys = xml_spec(xt)) != NULL && yang_config(ys) != 0){
 	switch (ys->ys_keyword){
+	case Y_INPUT:
 	case Y_LIST:
 	    /* fall thru */
 	case Y_CONTAINER:
@@ -326,6 +302,8 @@ xml_yang_validate_add(cxobj   *xt,
 		yc = ys->ys_stmt[i];
 		if (yc->ys_keyword != Y_LEAF)
 		    continue;
+		if (yang_config(yc)==0)
+		    continue;
 		if (yang_mandatory(yc) && xml_find(xt, yc->ys_argument)==NULL){
 		    clicon_err(OE_CFG, 0,"Missing mandatory variable: %s",
 			       yc->ys_argument);
@@ -385,8 +363,10 @@ xml_yang_validate_all(cxobj   *xt,
     yang_stmt *ys;
     yang_stmt *ytype;
     
-    /* if not given by argument (overide) use default link */
+    /* if not given by argument (overide) use default link 
-    if ((ys = xml_spec(xt)) != NULL){
+       and !Node has a config sub-statement and it is false */
+    if ((ys = xml_spec(xt)) != NULL &&
+	yang_config(ys) != 0){
 	switch (ys->ys_keyword){
 	case Y_LEAF:
 	    /* fall thru */
@@ -587,7 +567,7 @@ yang_next(yang_node *y,
     yang_stmt *ys;
 
     if (y->yn_keyword == Y_SPEC)
-	ys = yang_find_topnode((yang_spec*)y, name, 0);
+	ys = yang_find_topnode((yang_spec*)y, name, YC_DATANODE);
     else
 	ys = yang_find_datanode(y, name);
     if (ys == NULL)
@@ -833,7 +813,6 @@ yang2api_path_fmt(yang_stmt *ys,
     return retval;
 }
 
-
 /*! Transform an xml key format and a vector of values to an XML key
  * Used for actual key, eg in clicon_rpc_change(), xmldb_put_xkey()
  * Example: 
@@ -896,7 +875,7 @@ api_path_fmt2api_path(char  *api_path_fmt,
 		    clicon_err(OE_UNIX, errno, "cv2str_dup");
 		    goto done;
 		}
-		if (percent_encode(str, &strenc) < 0)
+		if (uri_percent_encode(str, &strenc) < 0)
 		    goto done;
 		cprintf(cb, "%s", strenc); 
 		free(strenc); strenc = NULL;
@@ -1313,7 +1292,7 @@ xml_spec_populate(cxobj  *x,
 	(yp = xml_spec(xp)) != NULL)
 	y = yang_find_datanode((yang_node*)yp, xml_name(x));
     else
-	y = yang_find_topnode(yspec, name, 0); /* still NULL for config */
+	y = yang_find_topnode(yspec, name, YC_DATANODE); /* still NULL for config */
 #endif
     if (y)
 	xml_spec_set(x, y);
@@ -1367,7 +1346,7 @@ api_path2xpath_cvv(yang_spec *yspec,
 	clicon_debug(1, "[%d] cvname:%s", i, name);
 	clicon_debug(1, "cv2str%d", cv2str(cv, NULL, 0));
 	if (i == offset){
-	    if ((y = yang_find_topnode(yspec, name, 0)) == NULL){
+	    if ((y = yang_find_topnode(yspec, name, YC_DATANODE)) == NULL){
 		clicon_err(OE_UNIX, errno, "No yang node found: %s", name);
 		goto done;
 	    }
@@ -1448,7 +1427,7 @@ api_path2xpath(yang_spec *yspec,
  * @param[in]   nvec      Length of vec
  * @param[in]   x0        Xpath tree so far
  * @param[in]   y0        Yang spec for x0
- * @param[in]   schemanode If set use schema nodes otherwise data nodes.
+ * @param[in]   nodeclass Set to schema nodes, data nodes, etc
  * @param[out]  xpathp    Resulting xml tree 
  * @param[out]  ypathp    Yang spec matching xpathp
  * @see api_path2xml
@@ -1458,7 +1437,7 @@ api_path2xml_vec(char             **vec,
 		 int                nvec,
 		 cxobj             *x0,
 		 yang_node         *y0,
-		 int                schemanode,
+		 yang_class         nodeclass,
 		 cxobj            **xpathp,
 		 yang_node        **ypathp)
 {
@@ -1490,7 +1469,7 @@ api_path2xml_vec(char             **vec,
     if ((restval_enc = index(name, '=')) != NULL){
 	*restval_enc = '\0';
 	restval_enc++;
-	if (percent_decode(restval_enc, &restval) < 0)
+	if (uri_percent_decode(restval_enc, &restval) < 0)
 	    goto done;
     }
     /* Split into prefix and localname, ignore prefix for now */
@@ -1500,10 +1479,10 @@ api_path2xml_vec(char             **vec,
 	name = local;
     }
     if (y0->yn_keyword == Y_SPEC){ /* top-node */
-	y = yang_find_topnode((yang_spec*)y0, name, schemanode);
+	y = yang_find_topnode((yang_spec*)y0, name, nodeclass);
     }
     else {
-	y = schemanode?yang_find_schemanode((yang_node*)y0, name):
+	y = (nodeclass==YC_SCHEMANODE)?yang_find_schemanode((yang_node*)y0, name):
 	    yang_find_datanode((yang_node*)y0, name);
     }
     if (y == NULL){
@@ -1572,7 +1551,7 @@ api_path2xml_vec(char             **vec,
     }
     if (api_path2xml_vec(vec+1, nvec-1, 
 			 x, (yang_node*)y, 
-			 schemanode,
+			 nodeclass,
 			 xpathp, ypathp) < 0)
 	goto done;
     retval = 0;
@@ -1588,7 +1567,7 @@ api_path2xml_vec(char             **vec,
  * @param[in]     api_path   API-path as defined in RFC 8040
  * @param[in]     yspec      Yang spec
  * @param[in,out] xtop       Incoming XML tree
- * @param[in]     schemanode If set use schema nodes otherwise data nodes.
+ * @param[in]     nodeclass  Set to schema nodes, data nodes, etc
  * @param[out]    xbotp      Resulting xml tree (end of xpath)
  * @param[out]    ybotp      Yang spec matching xbotp
  * @example
@@ -1605,7 +1584,7 @@ int
 api_path2xml(char       *api_path,
 	     yang_spec  *yspec,
 	     cxobj      *xtop,
-	     int         schemanode,
+	     yang_class  nodeclass,
 	     cxobj     **xbotp,
 	     yang_node **ybotp)
 {
@@ -1614,8 +1593,8 @@ api_path2xml(char       *api_path,
     int    nvec;
 
     if (*api_path!='/'){
-	clicon_err(OE_DB, 0, "Invalid key: %s", api_path);
+	clicon_log(LOG_WARNING, "Invalid key: %s (must start with '/')", api_path);
-	goto done;
+	goto ok;
     }
     if ((vec = clicon_strsep(api_path, "/", &nvec)) == NULL)
 	goto done;
@@ -1628,9 +1607,10 @@ api_path2xml(char       *api_path,
     }
     nvec--; /* NULL-terminated */
     if (api_path2xml_vec(vec+1, nvec, 
-			 xtop, (yang_node*)yspec, schemanode,
+			 xtop, (yang_node*)yspec, nodeclass,
 			 xbotp, ybotp) < 0)
 	goto done;
+ ok:
     retval = 0;
  done:
     if (vec)
@@ -1643,6 +1623,9 @@ api_path2xml(char       *api_path,
  * @param[in]  y0  Yang spec corresponding to xml-node x0. NULL if x0 is NULL
  * @param[in]  x0p Parent of x0
  * @param[in]  x1  xml tree which modifies base
+ * @param[out] reason If retval=0 a malloced string
+ * @retval     0     OK. If reason is set, Yang error
+ * @retval    -1     Error
  * Assume x0 and x1 are same on entry and that y is the spec
  * @see put in clixon_keyvalue.c
  */
@@ -1650,7 +1633,8 @@ static int
 xml_merge1(cxobj              *x0,
 	   yang_node          *y0,
 	   cxobj              *x0p,
-	   cxobj              *x1)
+	   cxobj              *x1,
+	   char              **reason)
 {
     int        retval = -1;
     char      *x1name;
@@ -1698,24 +1682,35 @@ xml_merge1(cxobj              *x0,
 	    x1cname = xml_name(x1c);
 	    /* Get yang spec of the child */
 	    if ((yc = yang_find_datanode(y0, x1cname)) == NULL){
-		clicon_err(OE_YANG, errno, "No yang node found: %s", x1cname);
+		if (reason && (*reason = strdup("XML node has no corresponding yang specification (Invalid XML or wrong Yang spec?")) == NULL){
+		    clicon_err(OE_UNIX, errno, "strdup");
 		    goto done;
 		}
+		break;
+	    }
 	    /* See if there is a corresponding node in the base tree */
 	    x0c = NULL;
 	    if (yc && match_base_child(x0, x1c, &x0c, yc) < 0)
 		goto done;
-	    if (xml_merge1(x0c, (yang_node*)yc, x0, x1c) < 0)
+	    if (xml_merge1(x0c, (yang_node*)yc, x0, x1c, reason) < 0)
 		goto done;
+	    if (*reason != NULL)
+		goto ok;
 	}
     } /* else Y_CONTAINER  */
-    // ok:
+ ok:
     retval = 0;
  done:
     return retval;
 }
 
 /*! Merge XML trees x1 into x0 according to yang spec yspec
+ * @param[in]  x0    Base xml tree (can be NULL in add scenarios)
+ * @param[in]  x1    xml tree which modifies base
+ * @param[in]  yspec Yang spec
+ * @param[out] reason If retval=0 a malloced string. Needs to be freed by caller
+ * @retval     0     OK. If reason is set, Yang error
+ * @retval    -1     Error
  * @note both x0 and x1 need to be top-level trees
  * @see text_modify_top as more generic variant (in datastore text)
  * @note returns -1 if YANG do not match, you may want to have a softer error
@@ -1723,7 +1718,8 @@ xml_merge1(cxobj              *x0,
 int
 xml_merge(cxobj     *x0,
 	  cxobj     *x1,
-	  yang_spec *yspec)
+	  yang_spec *yspec,
+	  char     **reason)
 {
     int        retval = -1;
     char      *x1cname; /* child name */
@@ -1736,17 +1732,22 @@ xml_merge(cxobj     *x0,
     while ((x1c = xml_child_each(x1, x1c, CX_ELMNT)) != NULL) {
 	x1cname = xml_name(x1c);
 	/* Get yang spec of the child */
-	if ((yc = yang_find_topnode(yspec, x1cname, 0)) == NULL){
+	if ((yc = yang_find_topnode(yspec, x1cname, YC_DATANODE)) == NULL){
-	    clicon_err(OE_YANG, ENOENT, "No yang spec");
+	    if (reason && (*reason = strdup("XML node has no corresponding yang specification (Invalid XML or wrong Yang spec?")) == NULL){
+		clicon_err(OE_UNIX, errno, "strdup");
 		goto done;
 	    }
+	    break;
+	}
 	/* See if there is a corresponding node in the base tree */
 	if (match_base_child(x0, x1c, &x0c, yc) < 0)
 	    goto done;
-	if (xml_merge1(x0c, (yang_node*)yc, x0, x1c) < 0)
+	if (xml_merge1(x0c, (yang_node*)yc, x0, x1c, reason) < 0)
 	    goto done;
+	if (*reason != NULL)
+	    break;
     }
-    retval = 0;
+    retval = 0; /* OK */
  done:
     return retval;
 }

lib/src/clixon_xml_parse.h

@@ -51,6 +51,7 @@ struct xml_parse_yacc_arg{
     cxobj      *ya_xparent;      /* xml parent element*/
     int         ya_skipspace;    /* If set, remove all non-terminal bodies (strip pretty-print) */
     yang_spec  *ya_yspec;        /* If set, top-level yang-spec */
+    int         ya_lex_state;    /* lex start condition (AMPERSAND) */
 };
 
 extern char *clixon_xml_parsetext;

lib/src/clixon_xml_parse.l

@@ -76,6 +76,7 @@ int clixon_xml_parsewrap(void)
 
 %x START
 %s STATEA
+%s AMPERSAND
 %s CMNT
 %s STR
 %s TEXTDECL
@@ -98,14 +99,21 @@ int clixon_xml_parsewrap(void)
 <START>\>             { BEGIN(STATEA); return *clixon_xml_parsetext; }
 
 <START>\"             { BEGIN(STR); return *clixon_xml_parsetext; }
-<START>.               { clixon_xml_parselval.string = yytext; return CHAR; /*XXX:optimize*/ }
+<START>.              { clixon_xml_parselval.string = yytext; return CHARDATA; /*XXX:optimize*/ }
-
 
 <STATEA>"</"          { BEGIN(START); return BSLASH; }
 <STATEA>"<!--"        { BEGIN(CMNT); return BCOMMENT; }
 <STATEA>\<            { BEGIN(START); return *clixon_xml_parsetext; }
-<STATEA>\n                 { clixon_xml_parselval.string = yytext;_YA->ya_linenum++; return (CHAR);}
+<STATEA>&             { _YA->ya_lex_state =STATEA;BEGIN(AMPERSAND);}
-<STATEA>.                  { clixon_xml_parselval.string = yytext; return CHAR; /*XXX:optimize*/}
+<STATEA>\n            { clixon_xml_parselval.string = yytext;_YA->ya_linenum++; return (CHARDATA);}
+<STATEA>.             { clixon_xml_parselval.string = yytext; return CHARDATA; /*XXX:optimize*/}
+	/* @see xml_chardata_encode */
+<AMPERSAND>"amp; "     {BEGIN(_YA->ya_lex_state); clixon_xml_parselval.string = "&"; return CHARDATA;}
+<AMPERSAND>"lt; "      {BEGIN(_YA->ya_lex_state); clixon_xml_parselval.string = "<"; return CHARDATA;}
+<AMPERSAND>"gt; "      {BEGIN(_YA->ya_lex_state); clixon_xml_parselval.string = ">"; return CHARDATA;}
+<AMPERSAND>"apos; "    {BEGIN(_YA->ya_lex_state); clixon_xml_parselval.string = "'"; return CHARDATA;}
+<AMPERSAND>"aquot; "   {BEGIN(_YA->ya_lex_state); clixon_xml_parselval.string = "'"; return CHARDATA;}
+
 
 <CMNT>"-->"           { BEGIN(START); return ECOMMENT; }
 <CMNT>\n              _YA->ya_linenum++;
@@ -117,15 +125,15 @@ int clixon_xml_parsewrap(void)
 <TEXTDECL>\"             { BEGIN(STRDQ); return *clixon_xml_parsetext; }
 <TEXTDECL>\'             { BEGIN(STRSQ); return *clixon_xml_parsetext; }
 
-<STR>[^\"]+           { clixon_xml_parselval.string = strdup(yytext); return CHAR; }
+<STR>[^\"]+           { clixon_xml_parselval.string = strdup(yytext); return CHARDATA; }
 <STR>\"               { BEGIN(START); return *clixon_xml_parsetext; }
 
-<STRDQ>1\.[0-9]+       { clixon_xml_parselval.string = strdup(yytext);  return CHAR; }
+<STRDQ>1\.[0-9]+       { clixon_xml_parselval.string = strdup(yytext);  return CHARDATA; }
-<STRDQ>[^\"]+          { clixon_xml_parselval.string = strdup(yytext);  return CHAR; }
+<STRDQ>[^\"]+          { clixon_xml_parselval.string = strdup(yytext);  return CHARDATA; }
 <STRDQ>\"              { BEGIN(TEXTDECL); return *clixon_xml_parsetext; }
 
-<STRSQ>1\.[0-9]+       { clixon_xml_parselval.string = strdup(yytext);   return CHAR; }
+<STRSQ>1\.[0-9]+       { clixon_xml_parselval.string = strdup(yytext);   return CHARDATA; }
-<STRSQ>[^\']+          { clixon_xml_parselval.string = strdup(yytext);   return CHAR; }
+<STRSQ>[^\']+          { clixon_xml_parselval.string = strdup(yytext);   return CHARDATA; }
 <STRSQ>\'              { BEGIN(TEXTDECL); return *clixon_xml_parsetext; }
 
 %%

lib/src/clixon_xml_parse.y

@@ -41,7 +41,7 @@
 
 %start topxml
 
-%token <string> NAME CHAR 
+%token <string> NAME CHARDATA 
 %token VER ENC
 %token BSLASH ESLASH 
 %token BTEXT ETEXT
@@ -329,15 +329,15 @@ topxml      : list
 dcl         : BTEXT info encode ETEXT { clicon_debug(3, "dcl->info encode"); }
             ;
 
-info        : VER '=' '\"' CHAR '\"' 
+info        : VER '=' '\"' CHARDATA '\"' 
                  { if (xml_parse_version(_YA, $4) <0) YYABORT; }
-            | VER '=' '\'' CHAR '\'' 
+            | VER '=' '\'' CHARDATA '\'' 
          	 { if (xml_parse_version(_YA, $4) <0) YYABORT; }
             |
             ;
 
-encode      : ENC '=' '\"' CHAR '\"' {free($4);}
+encode      : ENC '=' '\"' CHARDATA '\"' {free($4);}
-            | ENC '=' '\'' CHAR '\'' {free($4);}
+            | ENC '=' '\'' CHARDATA '\'' {free($4);}
             ;
 
 element     : '<' qname  attrs element1 
@@ -372,8 +372,8 @@ list        : list content { clicon_debug(3, "list -> list content"); }
 
 content     : element      { clicon_debug(3, "content -> element"); }
             | comment      { clicon_debug(3, "content -> comment"); }
-            | CHAR         { if (xml_parse_content(_YA, $1) < 0) YYABORT;  
+            | CHARDATA         { if (xml_parse_content(_YA, $1) < 0) YYABORT;  
-                             clicon_debug(3, "content -> CHAR %s", $1); }
+                             clicon_debug(3, "content -> CHARDATA %s", $1); }
             |              { clicon_debug(3, "content -> "); }
             ;
 
@@ -394,7 +394,7 @@ attqname    : NAME   {$$ = $1;}
             ;
 
 
-attvalue    : '\"' CHAR '\"'   { $$=$2; /* $2 must be consumed */}
+attvalue    : '\"' CHARDATA '\"'   { $$=$2; /* $2 must be consumed */}
             | '\"'  '\"'       { $$=strdup(""); /* $2 must be consumed */}
             ;
 

lib/src/clixon_xml_sort.c

@@ -93,7 +93,7 @@ xml_child_spec(char       *name,
     if (xp && (yparent = xml_spec(xp)) != NULL)
 	y = yang_find_datanode((yang_node*)yparent, name);
     else if (yspec)
-	y = yang_find_topnode(yspec, name, 0); /* still NULL for config */
+	y = yang_find_topnode(yspec, name, YC_DATANODE); /* still NULL for config */
     else
 	y = NULL;
     *yresult = y;
@@ -167,7 +167,7 @@ xml_cmp(const void* arg1,
     return equal;
 }
 
-/*!
+/*! Compare xml object
  * @param[in] yangi  Yang order
  * @param[in]  keynr    Length of keyvec/keyval vector when applicable
  * @param[in]  keyvec   Array of of yang key identifiers
@@ -192,36 +192,38 @@ xml_cmp1(cxobj        *x,
     int   i;
     char *keyname;
     char *key;
+    int   match = 0;
 
     /* Check if same yang spec (order in yang stmt list) */
     switch (keyword){
     case Y_CONTAINER: /* Match with name */
     case Y_LEAF: /* Match with name */
-	return strcmp(name, xml_name(x));
+	match = strcmp(name, xml_name(x));
 	break;
     case Y_LEAF_LIST: /* Match with name and value */
 	if (userorder && yang_find((yang_node*)y, Y_ORDERED_BY, "user") != NULL)
 	    *userorder=1;
 	b=xml_body(x);
-	return strcmp(keyval[0], b);
+	match = strcmp(keyval[0], b);
 	break;
     case Y_LIST: /* Match with array of key values */
 	if (userorder && yang_find((yang_node*)y, Y_ORDERED_BY, "user") != NULL)
 	    *userorder=1;
+	/* All must match */
 	for (i=0; i<keynr; i++){
 	    keyname = keyvec[i];
 	    key = keyval[i];
 	    /* Eg return "e0" in <if><name>e0</name></name></if> given "name" */
 	    if ((b = xml_find_body(x, keyname)) == NULL)
 		break; /* error case */
-	    return strcmp(key, b);
+	    if ((match = strcmp(key, b)) != 0)
+		break;
 	}
-	return 0;
 	break;
     default:
 	break;
     }
-    return 0; /* should not reach here */
+    return match; /* should not reach here */
 }
 
 /*! Sort children of an XML node
@@ -522,7 +524,7 @@ xml_sort_verify(cxobj *x0,
     return retval;
 }
 
-/*! Given child tree x1c, find matching child in base tree x0
+/*! Given child tree x1c, find matching child in base tree x0 and return as x0cp
  * param[in]  x0   Base tree node
  * param[in]  x1c  Modification tree child
  * param[in]  yc   Yang spec of tree child
@@ -567,7 +569,10 @@ match_base_child(cxobj     *x0,
 	break;
     case Y_LIST: /* Match with key values */
 	cvk = yc->ys_cvec; /* Use Y_LIST cache, see ys_populate_list() */
-	/* Count number of key indexes */
+	/* Count number of key indexes 
+	 * Then create two vectors one with names and one with values of x1c,
+	 * ec: keyvec: [a,b,c]  keyval: [1,2,3]
+	 */
 	cvi = NULL; keynr = 0;
 	while ((cvi = cvec_each(cvk, cvi)) != NULL) 
 	    keynr++;
@@ -591,7 +596,7 @@ match_base_child(cxobj     *x0,
     default:
 	break;
     }
-    /* Get match */
+    /* Get match. Sorting mode(optimized) or not?*/
     if (xml_child_sort==0)
 	*x0cp = xml_match(x0, xml_name(x1c), yc->ys_keyword, keynr, keyvec, keyval);
     else{

lib/src/clixon_yang.c

@@ -32,6 +32,8 @@
   ***** END LICENSE BLOCK *****
 
  * Yang functions
+ * @see https://tools.ietf.org/html/rfc6020 YANG 1.0
+ * @see https://tools.ietf.org/html/rfc7950 YANG 1.1
  */
 
 #ifdef HAVE_CONFIG_H
@@ -66,6 +68,7 @@
 #include "clixon_file.h"
 #include "clixon_yang.h"
 #include "clixon_hash.h"
+#include "clixon_xml.h"
 #include "clixon_plugin.h"
 #include "clixon_options.h"
 #include "clixon_yang_type.h"
@@ -134,6 +137,7 @@ static const map_str2int ykmap[] = {
     {"typedef",          Y_TYPEDEF}, 
     {"unique",           Y_UNIQUE}, 
     {"units",            Y_UNITS},
+    {"unknown",          Y_UNKNOWN}, 
     {"uses",             Y_USES}, 
     {"value",            Y_VALUE}, 
     {"when",             Y_WHEN}, 
@@ -401,6 +405,94 @@ yang_find(yang_node *yn,
     }
     return match ? ys : NULL;
 }
+#ifdef NOTYET
+/*! Prototype more generic than yang_find_datanode and yang_find_schemanode
+ */
+yang_stmt *
+yang_find_class(yang_node *yn, 
+		char      *argument,
+		yang_class class)
+{
+    yang_stmt *ys = NULL;
+    yang_stmt *yc = NULL;
+    yang_stmt *ysmatch = NULL;
+    int        i, j;
+    int        ok;
+
+    for (i=0; i<yn->yn_len; i++){
+	ys = yn->yn_stmt[i];
+	switch(class){
+	case YC_NONE:
+	    ok = 1;
+	    break;
+	case YC_DATANODE:
+	    ok = yang_datanode(ys);
+	    break;
+	case YC_DATADEFINITION:
+	    ok = yang_datadefinition(ys);
+	    break;
+	case YC_SCHEMANODE:
+	    ok = yang_schemanode(ys);
+	    break;
+	}
+	if (!ok)
+	    continue;
+	switch(class){
+	case YC_NONE:
+	    if (argument == NULL)
+		ysmatch = ys;
+	    else
+		if (ys->ys_argument && strcmp(argument, ys->ys_argument) == 0)
+		    ysmatch = ys;
+	    if (ysmatch)
+		goto match;
+	    break;
+	case YC_DATANODE:
+	case YC_DATADEFINITION:
+	    if (argument == NULL)
+		ysmatch = ys;
+	    else
+		if (ys->ys_argument && strcmp(argument, ys->ys_argument) == 0)
+		    ysmatch = ys;
+	    if (ysmatch)
+		goto match;
+	    break;
+	case YC_SCHEMANODE:
+	    if (ys->ys_keyword == Y_CHOICE){ /* Look for its children */
+	    for (j=0; j<ys->ys_len; j++){
+		yc = ys->ys_stmt[j];
+		if (yc->ys_keyword == Y_CASE) /* Look for its children */
+		    ysmatch = yang_find_class((yang_node*)yc, argument, class);
+		else{
+		    if (yang_schemanode(yc)){
+			if (argument == NULL)
+			    ysmatch = yc;
+			else
+			    if (yc->ys_argument && strcmp(argument, yc->ys_argument) == 0)
+				ysmatch = yc;
+		    }
+		}
+		if (ysmatch)
+		    goto match;
+	    }
+	} /* Y_CHOICE */
+	else{
+	    if (argument == NULL)
+		ysmatch = ys;
+	    else
+		if (ys->ys_argument && strcmp(argument, ys->ys_argument) == 0)
+		    ysmatch = ys;
+	    if (ysmatch)
+		goto match;
+
+	}
+	break;
+	} /* switch */
+    } /* for */
+ match:
+    return ysmatch;
+}
+#endif /* NOTYET */
 
 /*! Find child data node with matching argument (container, leaf, etc)
  *
@@ -454,6 +546,8 @@ yang_find_datanode(yang_node *yn,
 }
 
 /*! Find child schema node with matching argument (container, leaf, etc)
+ * @param[in]  yn         Yang node, current context node.
+ * @param[in]  argument   if NULL, match any(first) argument.
  * @note XXX unify code with yang_find_datanode?
  * @see yang_find_datanode
  */
@@ -504,7 +598,7 @@ yang_find_schemanode(yang_node *yn,
 /*! Find first matching data node in all (sub)modules in a yang spec
  *
  * @param[in]  ysp        Yang specification
- * @param[in]  name       if NULL, match any(first) argument. XXX is that really a case?
+ * @param[in]  argument   if NULL, match any(first) argument. XXX is that really a case?
  * @param[in]  schemanode If set look for schema nodes, otherwise only data nodes
  * A yang specification has modules as children which in turn can have 
  * syntax-nodes as children. This function goes through all the modules to
@@ -513,8 +607,8 @@ yang_find_schemanode(yang_node *yn,
  */
 yang_stmt *
 yang_find_topnode(yang_spec *ysp, 
-		  char      *name,
+		  char      *argument,
-		  int        schemanode)
+		  yang_class class)
 {
     yang_stmt *ys = NULL;
     yang_stmt *yc = NULL;
@@ -522,13 +616,22 @@ yang_find_topnode(yang_spec *ysp,
 
     for (i=0; i<ysp->yp_len; i++){
 	ys = ysp->yp_stmt[i];
-	if (schemanode){
+	switch (class){
-	    if ((yc = yang_find_schemanode((yang_node*)ys, name)) != NULL)
+	case YC_NONE:
+	    if ((yc = yang_find((yang_node*)ys, 0, argument)) != NULL)
 		return yc;
+	    break;
+	case YC_DATANODE:
+	    if ((yc = yang_find_datanode((yang_node*)ys, argument)) != NULL)
+		return yc;
+	    break;
+	case YC_SCHEMANODE:
+	    if ((yc = yang_find_schemanode((yang_node*)ys, argument)) != NULL)
+		return yc;
+	    break;
+	case YC_DATADEFINITION:
+	    break; /* nyi */
 	}
-	else
-	    if ((yc = yang_find_datanode((yang_node*)ys, name)) != NULL)
-		return yc;
     }
     return NULL;
 }
@@ -668,13 +771,15 @@ ys_spec(yang_stmt *ys)
     return (yang_spec*)ys;
 }
 
-/* Extract id from type argument. two cases:
+/* Assume argument is id on the type: <[prefix:]id>, return 'id'
- * argument is prefix:id, 
- * argument is id,        
  * Just return string from id
+ * @param[in] ys      A yang statement
+ * @retval    NULL    No id (argument is NULL)
+ * @retval    id      Pointer to identifier
+ * @see yarg_prefix
  */
 char*
-ytype_id(yang_stmt *ys)
+yarg_id(yang_stmt *ys)
 {
     char   *id;
     
@@ -685,13 +790,14 @@ ytype_id(yang_stmt *ys)
     return id;
 }
 
-/* Extract prefix from type argument. two cases:
+/* Assume argument is id on the type: <[prefix:]id>, return 'prefix'
- * argument is prefix:id, 
+ * @param[in] ys      A yang statement
- * argument is id,        
+ * @retval    NULL    No prefix
- * return either NULL or a new prefix string that needs to be freed by caller.
+ * @retval    prefix  Malloced string that needs to be freed by caller.
+ * @see yarg_id
  */
 char*
-ytype_prefix(yang_stmt *ys)
+yarg_prefix(yang_stmt *ys)
 {
     char   *id;
     char   *prefix = NULL;
@@ -822,6 +928,10 @@ yang_print_cbuf(cbuf      *cb,
     yang_stmt *ys = NULL;
 
     while ((ys = yn_each(yn, ys)) != NULL) {
+	if (ys->ys_keyword == Y_UNKNOWN){ /* dont print unknown - proxy for extension*/
+	    cprintf(cb, "%*s", marginal-1, "");
+	}
+	else
 	    cprintf(cb, "%*s%s", marginal, "", yang_key2str(ys->ys_keyword));
 	if (ys->ys_argument){
 	    if (quotedstring(ys->ys_argument))
@@ -974,7 +1084,7 @@ ys_populate_range(yang_stmt *ys,
 			  &options, NULL, NULL, NULL, &fraction_digits) < 0)
 	goto done;
     restype = yrestype?yrestype->ys_argument:NULL;
-    origtype = ytype_id((yang_stmt*)yparent);
+    origtype = yarg_id((yang_stmt*)yparent);
     /* This handles non-resolved also */
     if (clicon_type2cv(origtype, restype, &cvtype) < 0) 
 	goto done;
@@ -1209,7 +1319,7 @@ yang_augment_node(yang_stmt *ys,
     clicon_debug(1, "%s %s", __FUNCTION__, schema_nodeid);
 
     /* Find the target */
-    if (yang_abs_schema_nodeid(ysp, schema_nodeid, &yss) < 0)
+    if (yang_abs_schema_nodeid(ysp, schema_nodeid, -1, &yss) < 0)
 	goto done;
     if (yss == NULL)
 	goto ok;
@@ -1291,8 +1401,8 @@ yang_expand(yang_node *yn)
 	switch(ys->ys_keyword){
 	case Y_USES:
 	    /* Split argument into prefix and name */
-	    name    = ytype_id(ys);     /* This is uses/grouping name to resolve */
+	    name = yarg_id(ys);     /* This is uses/grouping name to resolve */
-	    prefix  = ytype_prefix(ys); /* And this its prefix */
+	    prefix  = yarg_prefix(ys); /* And this its prefix */
 	    if (ys_grouping_resolve(ys, prefix, name, &ygrouping) < 0)
 		goto done;
 	    if (prefix)
@@ -1638,7 +1748,7 @@ ys_schemanode_check(yang_stmt *ys,
 	/* fallthru */
     case Y_REFINE:
     case Y_UNIQUE:
-	if (yang_desc_schema_nodeid(yp, ys->ys_argument, &yres) < 0)
+	if (yang_desc_schema_nodeid(yp, ys->ys_argument, -1, &yres) < 0)
 	    goto done;
 	if (yres == NULL){
 	    clicon_err(OE_YANG, 0, "schemanode sanity check of %d %s", 
@@ -1649,7 +1759,7 @@ ys_schemanode_check(yang_stmt *ys,
 	break;
     case Y_DEVIATION:
 	yspec = ys_spec(ys);
-	if (yang_abs_schema_nodeid(yspec, ys->ys_argument, &yres) < 0)
+	if (yang_abs_schema_nodeid(yspec, ys->ys_argument, -1, &yres) < 0)
 	    goto done;
 	if (yres == NULL){
 	    clicon_err(OE_YANG, 0, "schemanode sanity check of %s", ys->ys_argument);
@@ -1792,6 +1902,7 @@ yang_apply(yang_node     *yn,
  * @param[in]  yn    Yang node. Find next yang stmt and return that if match.
  * @param[in]  vec   Vector of nodeid's in a schema node identifier, eg a/b
  * @param[in]  nvec  Length of vec
+ * @param[in]  keyword       A schemode of this type, or -1 if any
  * @param[out] yres  Result yang statement node, or NULL if not found
  * @retval    -1     Error, with clicon_err called
  * @retval     0     OK
@@ -1800,6 +1911,7 @@ static int
 schema_nodeid_vec(yang_node  *yn,
 		  char      **vec, 
 		  int         nvec,
+		  enum rfc_6020 keyword,
 		  yang_stmt **yres)
 {
     int              retval = -1;
@@ -1830,6 +1942,8 @@ schema_nodeid_vec(yang_node  *yn,
 	    ys = yn->yn_stmt[i];
 	    if (!yang_schemanode(ys))
 		continue;
+	    if (keyword != -1 && keyword != ys->ys_keyword)
+		continue;
 	    /* some keys dont have arguments, match on key */
 	    if (ys->ys_keyword == Y_INPUT || ys->ys_keyword == Y_OUTPUT){
 		if (strcmp(nodeid, yang_key2str(ys->ys_keyword)) == 0){
@@ -1856,7 +1970,7 @@ schema_nodeid_vec(yang_node  *yn,
 	goto ok;
     }
     /* recursive call using ynext */
-    if (schema_nodeid_vec(ynext, vec+1, nvec-1, yres) < 0)
+    if (schema_nodeid_vec(ynext, vec+1, nvec-1, keyword, yres) < 0)
 	goto done;
  ok:
     retval = 0;
@@ -1867,15 +1981,22 @@ schema_nodeid_vec(yang_node  *yn,
 /*! Given an absolute schema-nodeid (eg /a/b/c) find matching yang spec  
  * @param[in]  yspec         Yang specification.
  * @param[in]  schema_nodeid Absolute schema-node-id, ie /a/b
- * @retval     NULL          Error, with clicon_err called
+ * @param[in]  keyword       A schemode of this type, or -1 if any
- * @retval     yres          First yang node matching schema nodeid
+ * @param[out] yres          Result yang statement node, or NULL if not found
+ * @retval    -1             Error, with clicon_err called
+ * @retval     0             OK (if yres set then found, if yres=0 then not found)
  * Assume schema nodeid:s have prefixes, (actually the first).
  * @see yang_desc_schema_nodeid
+ * @see RFC7950 6.5
+   o  schema node: A node in the schema tree.  One of action, container,
+      leaf, leaf-list, list, choice, case, rpc, input, output,
+      notification, anydata, and anyxml.
  * Used in yang: deviation, top-level augment
  */
 int
 yang_abs_schema_nodeid(yang_spec  *yspec,
 		       char       *schema_nodeid,
+		       enum rfc_6020 keyword,
 		       yang_stmt **yres)
 {
     int              retval = -1;
@@ -1885,6 +2006,7 @@ yang_abs_schema_nodeid(yang_spec  *yspec,
     char            *id;
     char            *prefix = NULL;
     yang_stmt       *yprefix;
+    yang_stmt       *ys;
 
     /* check absolute schema_nodeid */
     if (schema_nodeid[0] != '/'){
@@ -1903,8 +2025,8 @@ yang_abs_schema_nodeid(yang_spec  *yspec,
     }
     /* split <prefix>:<id> */
     if ((id = strchr(vec[1], ':')) == NULL){ /* no prefix */
-	clicon_err(OE_YANG, 0, "Absolute schema nodeid must have prefix");
+	clicon_log(LOG_WARNING, "%s: Absolute schema nodeid must have prefix", __FUNCTION__);
-	goto done;
+	goto ok;
     }
     if ((prefix = strdup(vec[1])) == NULL){
 	clicon_err(OE_UNIX, errno, "%s: strdup", __FUNCTION__); 
@@ -1920,8 +2042,8 @@ yang_abs_schema_nodeid(yang_spec  *yspec,
 	}
     }
     if (ymod == NULL){ /* Try with topnode */
-	yang_stmt       *ys;
+
-	if ((ys = yang_find_topnode(yspec, id, 1)) == NULL){
+	if ((ys = yang_find_topnode(yspec, id, YC_SCHEMANODE)) == NULL){
 	    clicon_err(OE_YANG, 0, "Module with id:%s:%s not found", prefix,id);
 	    goto done;
 	}
@@ -1930,8 +2052,9 @@ yang_abs_schema_nodeid(yang_spec  *yspec,
 	    goto done;
 	}
     }
-    if (schema_nodeid_vec((yang_node*)ymod, vec+1, nvec-1, yres) < 0)
+    if (schema_nodeid_vec((yang_node*)ymod, vec+1, nvec-1, keyword, yres) < 0)
 	goto done;
+ ok: /* yres may not be set */
     retval = 0;
  done:
     if (vec)
@@ -1944,14 +2067,17 @@ yang_abs_schema_nodeid(yang_spec  *yspec,
 /*! Given a descendant schema-nodeid (eg a/b/c) find matching yang spec  
  * @param[in]  yn            Yang node
  * @param[in]  schema_nodeid Descendant schema-node-id, ie a/b
- * @retval     NULL          Error, with clicon_err called
+ * @param[in]  keyword       A schemode of this type, or -1 if any
- * @retval     yres          First yang node matching schema nodeid
+ * @param[out] yres          First yang node matching schema nodeid
+ * @retval     0             OK
+ * @retval    -1             Error, with clicon_err called
  * @see yang_schema_nodeid
  * Used in yang: unique, refine, uses augment
  */
 int
 yang_desc_schema_nodeid(yang_node  *yn, 
 			char       *schema_nodeid,
+			enum rfc_6020 keyword,
 			yang_stmt **yres)
 {
     int              retval = -1;
@@ -1967,7 +2093,7 @@ yang_desc_schema_nodeid(yang_node  *yn,
     }
     if ((vec = clicon_strsep(schema_nodeid, "/", &nvec)) == NULL)
 	goto done;
-    if (schema_nodeid_vec(yn, vec, nvec, yres) < 0)
+    if (schema_nodeid_vec(yn, vec, nvec, keyword, yres) < 0)
 	goto done;
     retval = 0;
  done:
@@ -2016,6 +2142,8 @@ ys_parse(yang_stmt   *ys,
  * Specific syntax checks  and variable creation for stand-alone yang statements.
  * That is, siblings, etc, may not be there. Complete checks are made in
  * ys_populate instead.
+ * @param[in] ys    yang statement
+ * @param[in] extra Yang extra for cornercases (unknown/extension)
  *
  * The cv:s created in parse-tree as follows:
  * fraction-digits : Create cv as uint8, check limits [1:8] (must be made in 1st pass)
@@ -2023,13 +2151,19 @@ ys_parse(yang_stmt   *ys,
  * @see ys_populate
  */
 int
-ys_parse_sub(yang_stmt *ys)
+ys_parse_sub(yang_stmt *ys,
+	     char      *extra)
 {
     int        retval = -1;
+    int        cvret;
+    char      *reason = NULL;
+    yang_stmt *ymod;
+    uint8_t    fd;
+    char      *prefix = NULL;
+    char      *name;
 
     switch (ys->ys_keyword){
-    case Y_FRACTION_DIGITS:{
+    case Y_FRACTION_DIGITS:
-	uint8_t fd;
 	if (ys_parse(ys, CGV_UINT8) == NULL) 
 	    goto done;
 	fd = cv_uint8_get(ys->ys_cv);
@@ -2038,12 +2172,41 @@ ys_parse_sub(yang_stmt *ys)
 	    goto done;
 	}
 	break;
+    case Y_UNKNOWN:
+	if (extra == NULL)
+	    break;
+	/* Find extension, if found, store it as unknown, if not,
+	   break for error */
+	prefix  = yarg_prefix(ys); /* And this its prefix */
+	name    = yarg_id(ys);  /* This is the type to resolve */
+	if ((ymod = yang_find_module_by_prefix(ys, prefix)) == NULL)
+	    goto ok; /* shouldnt happen */
+	if (yang_find((yang_node*)ymod, Y_EXTENSION, name) == NULL){
+	    clicon_err(OE_YANG, errno, "Extension %s:%s not found", prefix, name);
+	    goto done;
 	}
+	if ((ys->ys_cv = cv_new(CGV_STRING)) == NULL){
+	    clicon_err(OE_YANG, errno, "%s: cv_new", __FUNCTION__); 
+	    goto done;
+	}
+	if ((cvret = cv_parse1(extra, ys->ys_cv, &reason)) < 0){ /* error */
+	    clicon_err(OE_YANG, errno, "parsing cv");
+	    goto done;
+	}
+	if (cvret == 0){ /* parsing failed */
+	    clicon_err(OE_YANG, errno, "Parsing CV: %s", reason);
+	    goto done;
+	}
+	free(extra);
+	break;
     default:
 	break;
     }
+    ok:
     retval = 0;
   done:
+    if (prefix)
+	free(prefix);
     return retval;
 }