Merge branch 'develop'

CHANGELOG.md

@@ -1,69 +1,143 @@
 # Clixon CHANGELOG
 
-- Added new backend plugin callback: "plugin_statedata()" for retreiving state data
+## Upcoming 3.3.2
 
-- Added yang dir with ietf-netconf and clixon-config yang specs for internal usage.
+### Major changes:
+* Changed top-level netconf get-config and get to return `<data>` instead of `<data><config>` to comply to the RFC.
+  * If you use direct netconf get or get-config calls, you may need to handle the return XML differently.
+  * RESTCONF and CLI is not affected.
+  * Example: 
+```
+  Query: 
+    <rpc><get/></rpc>  
+  New reply: 
+    <rpc-reply>
+       <data>
+          <a/> # Example data model
+       </data>
+    </rpc-reply>
 
-- Added state data: Netconf <get> operation introduced; Error when
+  Old reply: 
-  adding state data in <edit-config>.
+    <rpc-reply>
+       <data>
+          <config>  # Removed
+             <a/>
+          </config> # Removed
+       </data>
+    </rpc-reply>
+```
 
-- Fixed bug where cli set of leaf-list were doubled, eg cli set foo -> foofoo
+* Added support for yang presence and no-presence containers. Previous default was "presence".
+  * Empty containers will be removed unless you have used the "presence" yang declaration.
+  * Example YANG without presence: 
+```
+     container 
+        nopresence { 
+          leaf j { 
+             type string; 
+          } 
+     }
+```
+If you submit "nopresence" without a leaf, it will automatically be removed:
+```
+     <nopresence/> # removed
+     <nopresence>  # not removed
+        <j>hello</j>
+     </nopresence>
+```
 
-- Restricted yang (sub)module file match to match RFC6020 exactly
+* Added YANG RPC support for netconf and CLI. With example rpc documentation and testcase. This replaces the previous "downcall" mechanism.
+  * This means you can make netconf rpc calls as defined by YANG.
+  * However you need to register an RPC backend callback using the backend_rpc_cb_register() function. See documentation and example for more details.
+  * Note that RESTCONF PUT for RCP calls is not yet supported.
+  * Example, the following YANG RPC definition enables you to run a netconf rpc.
+```
+      YANG:
+      rpc myrpc {
+         input {
+	   leaf name {
+	      type string;
+           }
+         }
+      }
+      NETCONF:
+      <rpc><myrpc><name>hello</name><rpc>
+```
 
-- Generalized yang type resolution to all included (sub)modules not just the topmost
+* Enhanced leafref functionality: 
+  * Validation for leafref forward and backward references; 
+  * CLI completion for generated cli leafrefs for both absolute and relative paths.
+  * Example, relative path:
+```
+         leaf ifname {
+             type leafref {
+                 path "../../interface/name";
+             }
+         }
+```
 	
-- Generic map_str2int generic mapping tables
+* Added state data: Netconf `<get>` operation, new backend plugin callback: "plugin_statedata()" for retreiving state data.
+  * You can use netconf: `<rpc><get/></rpc>` and it will return both config and state data.
+  * Restconf GET will return state data also, if defined.
+  * You need to define state data in a backend callback. See the example and documentation for more details.
 
-- Removed vector return values from xmldb_get()
+### Minor changes:
+* Removed 'margin' parameter of yang_print().
+* Extended example with ietf-routing (not only ietf-ip) for rpc operations.
+* Added yang dir with ietf-netconf and clixon-config yang specs for internal usage.
+* Fixed bug where cli set of leaf-list were doubled, eg cli set foo -> foofoo
+* Restricted yang (sub)module file match to match RFC6020 exactly
+* Generic map_str2int generic mapping tables
+* Removed vector return values from xmldb_get()
+* Generalized yang type resolution to all included (sub)modules not just the topmost
 	
 ## 3.3.1 June 7 2017
 
-- Fixed yang leafref cli completion.
+* Fixed yang leafref cli completion for absolute paths.
 
-- Removed non-standard api_path extension from the internal netconf protocol so that the internal netcinf is now fully standard.
+* Removed non-standard api_path extension from the internal netconf protocol so that the internal netconf is now fully standard.
 
-- Strings in xmldb_put not properly encoded, eg eth/0 became eth.00000
+* Strings in xmldb_put not properly encoded, eg eth/0 became eth.00000
 	
 ## 3.3.0
 
 May 2017	
 	
-- Datastore text module is now default.
+* Datastore text module is now default.
 
-- Refined netconf "none" semantics in tests and text datastore
+* Refined netconf "none" semantics in tests and text datastore
 
-- Moved apps/dbctrl to datastore/
+* Moved apps/dbctrl to datastore/
 
-- Added connect/disconnect/getopt/setopt and handle to xmldb API
+* Added connect/disconnect/getopt/setopt and handle to xmldb API
 
-- Added datastore 'text'
+* Added datastore 'text'
 
-- Configure (autoconf) changes
+* Configure (autoconf) changes
   Removed libcurl dependency
   Disable restconf (and fastcgi) with configure --disable-restconf
   Disable keyvalue datastore (and qdbm) with configure --disable-keyvalue
 
-- Created xmldb plugin api
+* Created xmldb plugin api
   Moved qdbm, chunk and  xmldb to datastore keyvalue directories
   Removed all other clixon dependency on chunk code
 	
-- cli_copy_config added as generic cli command
+* cli_copy_config added as generic cli command
-- cli_show_config added as generic cli command
+* cli_show_config added as generic cli command
   Replace all show_confv*() and show_conf*() with cli_show_config()
   Example: replace:
      show_confv_as_json("candidate","/sender");
   with:
      cli_show_config("candidate","json","/sender");
-- Alternative yang spec option -y added to all applications
+* Alternative yang spec option -y added to all applications
-- Many clicon special string functions have been removed
+* Many clicon special string functions have been removed
-- The netconf support has been extended with lock/unlock
+* The netconf support has been extended with lock/unlock
-- clicon_rpc_call() has been removed and should be replaced by extending the
+* clicon_rpc_call() has been removed and should be replaced by extending the
   internal netconf protocol. 
   See downcall() function in example/routing_cli.c and 
   routing_downcall() in example/routing_backend.c
-- Replace clicon_rpc_xmlput with clicon_rpc_edit_config
+* Replace clicon_rpc_xmlput with clicon_rpc_edit_config
-- Removed xmldb daemon. All xmldb acceses is made backend daemon. 
+* Removed xmldb daemon. All xmldb acceses is made backend daemon. 
   No direct accesses by clients to xmldb API.
   Instead use the rpc calls in clixon_proto_client.[ch]
   In clients (eg cli/netconf) replace xmldb_get() in client code with 
@@ -74,48 +148,48 @@ May 2017
     clicon_rpc_get_config(h, dbstr, api_path, &xt);
     xpath_vec(xt, api_path, &xvec, &xlen)
 
-- clicon_rpc_change() is replaced with clicon_rpc_edit_config().
+* clicon_rpc_change() is replaced with clicon_rpc_edit_config().
   Note modify argument 5:
      clicon_rpc_change(h, db, op, apipath, "value") 
   to:
-     clicon_rpc_edit_config(h, db, op, apipath, "<config>value</config>") 
+     clicon_rpc_edit_config(h, db, op, apipath, `"<config>value</config>"`) 
 
-- xmdlb_put_xkey() and xmldb_put_tree() have been folded into xmldb_put()
+* xmdlb_put_xkey() and xmldb_put_tree() have been folded into xmldb_put()
   Replace xmldb_put_xkey with xmldb_put as follows:
      xmldb_put_xkey(h, "candidate", cbuf_get(cb), str, OP_REPLACE);
   with
-     clicon_xml_parse(&xml, "<config>%s</config>", str);
+     clicon_xml_parse(&xml, `"<config>%s</config>"`, str);
      xmldb_put(h, "candidate", OP_REPLACE, cbuf_get(cb), xml);
      xml_free(xml);
 
-- Change internal protocol from clicon_proto.h to netconf.
+* Change internal protocol from clicon_proto.h to netconf.
   This means that the internal protocol defined in clixon_proto.[ch] is removed
 
-- Netconf startup configuration support. Set CLICON_USE_STARTUP_CONFIG to 1 to
+* Netconf startup configuration support. Set CLICON_USE_STARTUP_CONFIG to 1 to
   enable. Eg, if backend_main is started with -CIr startup will be copied to
   running.
 
-- Added ".." as valid step in xpath
+* Added ".." as valid step in xpath
 
-- Use restconf format for internal xmldb keys. Eg /a/b=3,4
+* Use restconf format for internal xmldb keys. Eg /a/b=3,4
 
-- List keys with special characters RFC 3986 encoded.	
+* List keys with special characters RFC 3986 encoded.	
 
-- Replaced cli expand functions with single to multiple args
+* Replaced cli expand functions with single to multiple args
   This change is _not_ backward compatible
   This effects all calls to expand_dbvar() or user-defined
   expand callbacks
 
-- Replaced cli callback functions with single arg to multiple args
+* Replaced cli callback functions with single arg to multiple args
   This change is _not_ backward compatible.
   You are affected if you 
   (1) use system callbacks (i.e. in clixon_cli_api.h)
   (2) write your own cli callbacks
 
   If you use cli callbacks, you need to rewrite cli callbacks from eg:
-     load("Comment") <filename:string>,load_config_file("filename replace");
+     `load("Comment") <filename:string>,load_config_file("filename replace");`
   to:
-     load("Comment") <filename:string>,load_config_file("filename", "replace");
+     `load("Comment") <filename:string>,load_config_file("filename", "replace");`
 
   If you write your own, you need to change the callback signature from;
     int cli_callback(clicon_handle h, cvec *vars, cg_var *arg)
@@ -128,9 +202,9 @@ May 2017
   load_config_file, save_config_file, delete_all, discard_changes, cli_notify,
   show_yang, show_conf_xpath
 
-- Added --with-cligen and --with-qdbm configure options
+* Added --with-cligen and --with-qdbm configure options
-- Added union type check for non-cli (eg xml) input 
+* Added union type check for non-cli (eg xml) input 
-- Empty yang type. Relaxed yang types for unions, eg two strings with different length.
+* Empty yang type. Relaxed yang types for unions, eg two strings with different length.
 	
 Dec 2016: Dual license: both GPLv3 and APLv2
 	

README.md

@@ -15,7 +15,8 @@ Table of contents
 
 Documentation
 =============
-- [Frequently asked questions](doc/FAQ.md)
+- [Frequently asked questions](doc/FAQ.md
+- [CHANGELOG](CHANGELOG.md) recent changes.
 - [XML datastore](datastore/README.md)
 - [Netconf support](apps/netconf/README.md)
 - [Restconf support](apps/restconf/README.md)
@@ -58,7 +59,7 @@ Licenses
 Clixon is dual license. Either Apache License, Version 2.0 or GNU
 General Public License Version 2. You choose.
 
-See [LICENSE.md](LICENSE.md) for license, [CHANGELOG](CHANGELOG.md) for recent changes.
+See [LICENSE.md](LICENSE.md) for license.
 
 Background
 ==========
@@ -90,7 +91,7 @@ The [YANG RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) is implemented w
 - object-references
 - if-feature
 - unique
-- rpc
+
 
 
 

apps/backend/backend_client.c

@@ -241,16 +241,16 @@ from_client_get_config(clicon_handle h,
 		"</rpc-error></rpc-reply>");
 	goto ok;
     }
-    cprintf(cbret, "<rpc-reply><data>");
+    cprintf(cbret, "<rpc-reply>");
-    if (xret!=NULL){
+    if (xret==NULL)
-	if (xml_child_nr(xret)){
+	cprintf(cbret, "<data/>");
-	    if (xml_name_set(xret, "config") < 0)
+    else{
-		goto done;
+	if (xml_name_set(xret, "data") < 0)
-	    if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
+	    goto done;
-		goto done;
+	if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
-	}
+	    goto done;
     }
-    cprintf(cbret, "</data></rpc-reply>");
+    cprintf(cbret, "</rpc-reply>");
  ok:
     retval = 0;
  done:
@@ -293,17 +293,16 @@ from_client_get(clicon_handle h,
     assert(xret);
     if (backend_statedata_call(h, selector, xret) < 0)
 	goto done;
-    cprintf(cbret, "<rpc-reply><data>");
+    cprintf(cbret, "<rpc-reply>");
-    /* if empty only <config/> */
+    if (xret==NULL)
-    if (xret!=NULL){
+	cprintf(cbret, "<data/>");
-	if (xml_child_nr(xret)){
+    else{
-	    if (xml_name_set(xret, "config") < 0)
+	if (xml_name_set(xret, "data") < 0)
-		goto done;
+	    goto done;
-	    if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
+	if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
-		goto done;
+	    goto done;
-	}
     }
-    cprintf(cbret, "</data></rpc-reply>");
+    cprintf(cbret, "</rpc-reply>");
  ok:
     retval = 0;
  done:

apps/backend/backend_commit.c

@@ -89,15 +89,19 @@ generic_validate(yang_spec          *yspec,
     int             retval = -1;
     cxobj          *x1;
     cxobj          *x2;
-    int             i;
     yang_stmt      *ys;
+    int             i;
+
+    /* All entries */
+    if (xml_apply(td->td_target, CX_ELMNT, 
+		  (xml_applyfn_t*)xml_yang_validate_all, NULL) < 0)
+	goto done;
 
     /* changed entries */
     for (i=0; i<td->td_clen; i++){
 	x1 = td->td_scvec[i]; /* source changed */
 	x2 = td->td_tcvec[i]; /* target changed */
-	ys = xml_spec(x1);
+	if (xml_yang_validate_add(x2, NULL) < 0)
-	if (xml_yang_validate(x2, ys) < 0)
 	    goto done;
     }
     /* deleted entries */
@@ -113,11 +117,8 @@ generic_validate(yang_spec          *yspec,
     /* added entries */
     for (i=0; i<td->td_alen; i++){
 	x2 = td->td_avec[i];
-	ys = xml_spec(x2);
+	if (xml_apply0(x2, CX_ELMNT, 
-	if (xml_yang_validate(x2, ys) < 0)
+		      (xml_applyfn_t*)xml_yang_validate_add, NULL) < 0)
-	    goto done;
-	if (xml_apply(x2, CX_ELMNT, 
-		      (xml_applyfn_t*)xml_yang_validate, NULL) < 0)
 	    goto done;
     }
     retval = 0;

apps/cli/cli_generate.c

@@ -275,9 +275,10 @@ yang2cli_var_sub(clicon_handle h,
     if (helptext)
 	cprintf(cb0, "(\"%s\")", helptext);
     if (completion){
+#if 0
 	if (type && (strcmp(type, "leafref") == 0)){
 	    yang_stmt *ypath;
-
+	    /* XXX only for absolute xpath */
 	    if ((ypath = yang_find((yang_node*)ytype, Y_PATH, NULL)) == NULL){
 		clicon_err(OE_XML, 0, "leafref should have path sub");
 		goto done;
@@ -290,6 +291,7 @@ yang2cli_var_sub(clicon_handle h,
 		    ypath->ys_argument);
 	}
 	else
+#endif
 	    if (cli_expand_var_generate(h, ys, cvtype, cb0, 
 					options, fraction_digits) < 0)
 		goto done;

apps/cli/cli_main.c

@@ -297,7 +297,8 @@ main(int argc, char **argv)
 	    char *dir = dirname(str);
 	    hash_del(clicon_options(h), (char*)"CLICON_YANG_MODULE_REVISION");
 	    clicon_option_str_set(h, "CLICON_YANG_MODULE_MAIN", basename(optarg));
-	    clicon_option_str_set(h, "CLICON_YANG_DIR", strdup(dir));
+	    clicon_option_str_set(h, "CLICON_YANG_DIR", dir);
+	    free(str);
 	    break;
 	}
 	default:

apps/cli/cli_show.c

@@ -112,12 +112,24 @@ expand_dbvar(void   *h,
     int              j;
     int              k;
     cg_var          *cv;
+    yang_spec       *yspec;
+    cxobj           *xtop = NULL; /* xpath root */
+    cxobj           *xbot = NULL; /* xpath, NULL if datastore */
+    yang_node       *y = NULL; /* yang spec of xpath */
+    yang_stmt       *ytype;
+    yang_stmt       *ypath;
+    cxobj           *xcur;
+    char            *xpathcur;
 
     if (argv == NULL || cvec_len(argv) != 2){
 	clicon_err(OE_PLUGIN, 0, "%s: requires arguments: <db> <xmlkeyfmt>",
 		   __FUNCTION__);
 	goto done;
     }
+    if ((yspec = clicon_dbspec_yang(h)) == NULL){
+	clicon_err(OE_FATAL, 0, "No DB_SPEC");
+	goto done;
+    }
     if ((cv = cvec_i(argv, 0)) == NULL){
 	clicon_err(OE_PLUGIN, 0, "%s: Error when accessing argument <db>");
 	goto done;
@@ -147,11 +159,40 @@ expand_dbvar(void   *h,
 	clicon_rpc_generate_error(xerr);
 	goto done;
     }
+    xcur = xt; /* default top-of-tree */
+    xpathcur = xpath;
+    /* Create config top-of-tree */
+    if ((xtop = xml_new("config", NULL)) == NULL)
+	goto done;
+    xbot = xtop;
+    if (api_path && api_path2xml(api_path, yspec, xtop, &xbot, &y) < 0)
+	goto done;
+    /* Special case for leafref. Detect leafref via Yang-type, 
+     * Get Yang path element, tentatively add the new syntax to the whole
+     * tree and apply the path to that.
+     * Last, the reference point for the xpath code below is changed to 
+     * the point of the tentative new xml.
+     * Here the whole syntax tree is loaded, and it would be better to offload
+     * such operations to the datastore by a generic xpath function.
+     */
+    if ((ytype = yang_find((yang_node*)y, Y_TYPE, NULL)) != NULL)
+	if (strcmp(ytype->ys_argument, "leafref")==0){
+	    if ((ypath = yang_find((yang_node*)ytype, Y_PATH, NULL)) == NULL){
+		clicon_err(OE_DB, 0, "Leafref %s requires path statement", ytype->ys_argument);
+		goto done;
+	    }
+	    xpathcur = ypath->ys_argument;
+	    if (xml_merge(xt, xtop, yspec) < 0)
+		goto done;
+	    if ((xcur = xpath_first(xt, xpath)) == NULL){
+		clicon_err(OE_DB, 0, "xpath %s should return merged content", xpath);
+		goto done;
+	    }
+	}
     /* One round to detect duplicates 
-     * XXX The code below would benefit from some cleanup
      */
     j = 0;
-    if (xpath_vec(xt, xpath, &xvec, &xlen) < 0) 
+    if (xpath_vec(xcur, xpathcur, &xvec, &xlen) < 0) 
 	goto done;
     for (i = 0; i < xlen; i++) {
 	char *str;
@@ -194,6 +235,8 @@ expand_dbvar(void   *h,
   done:
     if (xvec)
 	free(xvec);
+    if (xtop)
+	xml_free(xtop);
     if (xt)
 	xml_free(xt);
     if (xpath) 
@@ -344,7 +387,7 @@ show_yang(clicon_handle h,
   }
   else
     yn = (yang_node*)yspec;
-  yang_print(stdout, yn, 0);
+  yang_print(stdout, yn);
   return 0;
 }
 int show_yangv(clicon_handle h, cvec *vars, cvec *argv)

apps/netconf/netconf_plugin.c

@@ -196,6 +196,28 @@ catch:
     return -1;
 }
     
+/*! Struct to carry info into and out of ys_find_rpc callback
+ */
+typedef struct {
+    char      *name; /* name of rpc */
+    yang_stmt *yrpc; /* matching yang statement */
+} find_rpc_arg;
+
+/*! Check yang rpc statement, return yang rpc statement if found 
+ */
+static int 
+ys_find_rpc(yang_stmt *ys, 
+	    void      *arg)
+{
+    find_rpc_arg *fra = (find_rpc_arg*)arg;
+
+    if (strcmp(fra->name, ys->ys_argument) == 0){
+	fra->yrpc = ys;
+	return 1; /* handled */
+    }
+   return 0;
+}
+
 /*! See if there is any callback registered for this tag
  *
  * @param[in]  h       clicon handle
@@ -211,21 +233,73 @@ netconf_plugin_callbacks(clicon_handle h,
 			 cxobj        *xn, 
 			 cxobj       **xret)
 {
+    int            retval = -1;
     netconf_reg_t *nreg;
-    int            retval;
+    yang_spec     *yspec;
+    yang_stmt     *yrpc;
+    yang_stmt     *yinput;
+    yang_stmt     *youtput;
+    cxobj         *xoutput;
+    find_rpc_arg   fra = {0,0};
+    int            ret;
 
-    if (deps == NULL)
+    if (deps != NULL){
-	return 0;
+	nreg = deps;
-    nreg = deps;
+	do {
-    do {
+	    if (strcmp(nreg->nr_tag, xml_name(xn)) == 0){
-	if (strcmp(nreg->nr_tag, xml_name(xn)) == 0){
+		if ((retval = nreg->nr_callback(h, xn, xret, nreg->nr_arg)) < 0)
-	    if ((retval = nreg->nr_callback(h, xn, xret, nreg->nr_arg)) < 0)
+		    goto done;
-		return -1;
+		retval = 1; /* handled */
-	    else
+		goto done;
-		return 1; /* handled */
+	    }
+	    nreg = NEXTQ(netconf_reg_t *, nreg);
+	} while (nreg != deps);
+    }
+    if ((yspec =  clicon_dbspec_yang(h)) == NULL){
+	clicon_err(OE_YANG, ENOENT, "No yang spec");
+	goto done;
+    }
+    /* Find yang rpc statement, return yang rpc statement if found */
+    fra.name = xml_name(xn);
+    if ((ret = yang_apply((yang_node*)yspec, Y_RPC, ys_find_rpc, &fra)) < 0)
+	goto done;
+    /* Check if found */
+    if (ret == 1){
+	yrpc = fra.yrpc;
+	if ((yinput = yang_find((yang_node*)yrpc, Y_INPUT, NULL)) != NULL){
+	    xml_spec_set(xn, yinput); /* needed for xml_spec_populate */
+	    if (xml_apply(xn, CX_ELMNT, xml_spec_populate, yinput) < 0)
+		goto done;
+	    if (xml_apply(xn, CX_ELMNT, 
+			  (xml_applyfn_t*)xml_yang_validate_all, NULL) < 0)
+		goto done;
+	    if (xml_yang_validate_add(xn, NULL) < 0)
+		goto done;
 	}
-	nreg = NEXTQ(netconf_reg_t *, nreg);
+	/* 
-    } while (nreg != deps);
+	 * 1. Check xn arguments with input statement.
-    return 0;
+	 * 2. Send to backend as clicon_msg-encode()
+	 * 3. In backend to similar but there call actual backend
+	 */
+	if (clicon_rpc_netconf_xml(h, xml_parent(xn), xret, NULL) < 0)
+	    goto done;
+	/* Sanity check of outgoing XML */
+	if ((youtput = yang_find((yang_node*)yrpc, Y_OUTPUT, NULL)) != NULL){
+	    xoutput=xpath_first(*xret, "/");
+	    xml_spec_set(xoutput, youtput); /* needed for xml_spec_populate */
+	    if (xml_apply(xoutput, CX_ELMNT, xml_spec_populate, yinput) < 0)
+		goto done;
+	    if (xml_apply(xoutput, CX_ELMNT, 
+			  (xml_applyfn_t*)xml_yang_validate_all, NULL) < 0)
+		goto done;
+	    if (xml_yang_validate_add(xoutput, NULL) < 0)
+		goto done;
+	}
+	retval = 1; /* handled by callback */
+	goto done;
+    }
+    retval = 0;
+ done:
+    return retval;
 }
     

apps/netconf/netconf_rpc.c

@@ -159,7 +159,7 @@ netconf_get_config(clicon_handle h,
 	     goto done;	
 	 if (xfilter &&
 	     (xfilterconf = xpath_first(xfilter, "//configuration"))!= NULL &&
-	     (xconf = xpath_first(*xret, "/rpc-reply/data/configuration")) != NULL){
+	     (xconf = xpath_first(*xret, "/rpc-reply/data")) != NULL){
 	     /* xml_filter removes parts of xml tree not matching */
 	     if ((strcmp(xml_name(xfilterconf), xml_name(xconf))!=0) ||
 		 xml_filter(xfilterconf, xconf) < 0){
@@ -571,7 +571,7 @@ netconf_get(clicon_handle h,
 	     goto done;	
 	 if (xfilter &&
 	     (xfilterconf = xpath_first(xfilter, "//configuration"))!= NULL &&
-	     (xconf = xpath_first(*xret, "/rpc-reply/data/configuration")) != NULL){
+	     (xconf = xpath_first(*xret, "/rpc-reply/data")) != NULL){
 	     /* xml_filter removes parts of xml tree not matching */
 	     if ((strcmp(xml_name(xfilterconf), xml_name(xconf))!=0) ||
 		 xml_filter(xfilterconf, xconf) < 0){

datastore/keyvalue/clixon_keyvalue.c

@@ -673,7 +673,7 @@ put(char               *dbfile,
     clicon_debug(1, "%s xk0:%s ys:%s", __FUNCTION__, xk0, ys->ys_argument);
     if (debug){
 	xml_print(stderr, xt);
-	//	yang_print(stderr, (yang_node*)ys, 0);
+	//	yang_print(stderr, (yang_node*)ys);
     }
     if ((opstr = xml_find_value(xt, "operation")) != NULL)
 	if (xml_operation(opstr, &op) < 0)

datastore/text/clixon_xmldb_text.c

@@ -665,6 +665,50 @@ text_modify_top(cxobj              *x0,
     return retval;
 }
 
+/*! For containers without presence and no children, remove
+ * @param[in]   x       XML tree node
+ * @note This should really be unnecessary since yspec should be set on creation
+ * @code
+ * xml_apply(xc, CX_ELMNT, xml_spec_populate, yspec)
+ * @endcode
+ * See section 7.5.1 in rfc6020bis-02.txt:
+ * No presence:
+ * those that exist only for organizing the hierarchy of data nodes:
+ * the container has no meaning of its own, existing
+ * only to contain child nodes.  This is the default style.
+ * (Remove these if no children)
+ * Presence:
+ * the presence of the container itself is
+ * configuration data, representing a single bit of configuration data.
+ * The container acts as both a configuration knob and a means of
+ * organizing related configuration.  These containers are explicitly
+ * created and deleted.
+ * (Dont touch these)
+ */
+int
+xml_container_presence(cxobj  *x, 
+		       void   *arg)
+{
+    int        retval = -1;
+    char      *name;
+    yang_stmt *y;  /* yang node */
+
+    name = xml_name(x);
+    if ((y = (yang_stmt*)xml_spec(x)) == NULL){
+	clicon_log(LOG_WARNING, "%s: no xml_spec(%s)", __FUNCTION__, name);
+	retval = 0;
+	goto done;
+    }
+    /* Mark node that is: container, have no children, dont have presence */
+    if (y->ys_keyword == Y_CONTAINER && 
+	xml_child_nr(x)==0 &&
+	yang_find((yang_node*)y, Y_PRESENCE, NULL) == NULL)
+	xml_flag_set(x, XML_FLAG_MARK); /* Mark, remove later */
+    retval = 0;
+ done:
+    return retval;
+}
+
 /*! Modify database provided an xml tree and an operation
  * This is a clixon datastore plugin of the the xmldb api
  * @see xmldb_put
@@ -745,6 +789,12 @@ text_put(xmldb_handle        xh,
     if (xml_apply(x0, CX_ELMNT, (xml_applyfn_t*)xml_flag_reset, 
 		  (void*)XML_FLAG_NONE) < 0)
 	goto done;
+    /* Mark non-presence containers that do not have children */
+    if (xml_apply(x0, CX_ELMNT, (xml_applyfn_t*)xml_container_presence, NULL) < 0)
+	goto done;
+    /* Remove (prune) nodes that are marked (non-presence containers w/o children) */
+    if (xml_tree_prune_flagged(x0, XML_FLAG_MARK, 1) < 0)
+	goto done;
     // output:
     /* Print out top-level xml tree after modification to file */
     if ((cb = cbuf_new()) == NULL){

doc/FAQ.md

@@ -197,8 +197,10 @@ They are documented in [CLIgen tutorial](https://github.com/olofhagsand/cligen/b
 ## How do I write a validation function?
 Similar to a commit function, but instead write the transaction_validate() function.
 Check for inconsistencies in the XML trees and if they fail, make an clicon_err() call.
+```
     clicon_err(OE_PLUGIN, 0, "Route %s lacks ipv4 addr", name);
     return -1;
+```
 The validation or commit will then be aborted.
 
 ## How do I write a state data callback function?
@@ -210,3 +212,45 @@ To return state data, you need to write a backend state data callback
 with the name "plugin_statedata()" where you return an XML tree.
 
 Please look at the example for an example on how to write a state data callback.
+
+## How do I write an RPC function?
+
+A YANG RPC is an application specific operation. Example:
+```
+   rpc fib-route {
+      input {
+         leaf inarg { type string; }
+      }
+      output {
+         leaf outarg { type string; }
+      }
+   }
+```
+which defines the fib-route operation present in the example (the arguments have been changed).
+
+Clixon automatically relays the RPC to the clixon backend. To
+implement the RFC, you need to register an RPC callback in the backend plugin:
+Example:
+```
+int
+plugin_init(clicon_handle h)
+{
+...
+   backend_rpc_cb_register(h, fib_route, NULL, "fib-route");
+...
+}
+```
+And then define the callback itself:
+```
+static int 
+fib_route(clicon_handle h,            /* Clicon handle */
+	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
+	  struct client_entry *ce,    /* Client session */
+	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
+	  void         *arg)          /* Argument given at register */
+{
+    cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");    
+    return 0;
+}
+```
+Here, the callback is over-simplified.

example/Makefile.in

@@ -67,6 +67,7 @@ YANGSPECS	+= ietf-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipv4-unicast-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipv6-unicast-routing@2014-10-26.yang
 YANGSPECS	+= ietf-ipsec@2016-03-09.yang
+YANGSPECS	+= example.yang
 
 # Backend plugin
 BE_SRC   = routing_backend.c

example/README.md

@@ -65,22 +65,52 @@ Routing notification
 ...
 ```
 
-## Extending
+## Operation data
 
-Clixon has an extension mechanism which can be used to make extended internal
+Clixon implements Yang RPC operations by an extension mechanism. The
-netconf messages to the backend configuration engine. You may need this to
+extension mechanism enables you to add application-specific
-make some special operation that is not covered by standard
+operations. It works by adding user-defined callbacks for added
-netconf functions. The example has a simple "echo" downcall
+netconf operations. It is possible to use the extension mechanism
-mechanism that simply echoes what is sent down and is included for
+independent of the yang rpc construct, but it is recommended to use
-reference. A more realistic downcall would perform some action, such as
+that, and the example includes such an example:
-reading some status.
 
 Example:
 ```
-cli> downcall "This is a  string"
+cli> rpc ipv4
-This is a string
+<rpc-reply>
+   <ok/>
+</rpc-reply>
 ```
 
+The example works by creating a netconf rpc call and sending it to the backend: (see the fib_route_rpc() function).
+```
+  <rpc>
+    <fib-route>
+      <routing-instance-name>ipv4</routing-instance-name>
+    </fib-route>
+   </rpc>
+```
+
+The backend in turn registers a callback (fib_route()) which handles the RPC.
+```
+static int 
+fib_route(clicon_handle h, 
+	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
+	  struct client_entry *ce,    /* Client session */
+	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
+	  void         *arg)          /* Argument given at register */
+{
+    cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");    
+    return 0;
+}
+int
+plugin_init(clicon_handle h)
+{
+...
+   backend_rpc_cb_register(h, fib_route, NULL, "fib-route");
+...
+}
+```
 ## State data
 
 Netconf <get> and restconf GET also returns state data, in contrast to

example/example.yang

@@ -0,0 +1,10 @@
+module example {
+    import ietf-ip {
+      prefix ip;
+    }
+    import ietf-routing {
+      prefix rt;
+    }
+    description
+      "Example code that includes ietf-ip and ietf-routing";
+}

example/routing.conf.local

@@ -5,11 +5,12 @@ CLICON_CLI_MODE         routing
 
 # Option used to construct initial yang file:
 #     <module>[@<revision>]
-CLICON_YANG_MODULE_MAIN ietf-ip
+#CLICON_YANG_MODULE_MAIN ietf-ip
+CLICON_YANG_MODULE_MAIN example
 
 # Option used to construct initial yang file:
 #     <module>[@<revision>]
-CLICON_YANG_MODULE_REVISION 2014-06-16
+#CLICON_YANG_MODULE_REVISION 2014-06-16
 
 # Generate code for CLI completion of existing db symbols
 # CLICON_CLI_GENMODEL_COMPLETION 0

example/routing_backend.c

@@ -119,14 +119,30 @@ notification_timer_setup(clicon_handle h)
     return event_reg_timeout(t, notification_timer, h, "notification timer");
 }
 
+/*! IETF Routing fib-route rpc */
 static int 
-routing_downcall(clicon_handle h, 
+fib_route(clicon_handle h,            /* Clicon handle */
-		 cxobj        *xe,           /* Request: <rpc><xn></rpc> */
+	  cxobj        *xe,           /* Request: <rpc><xn></rpc> */
-		 struct client_entry *ce,    /* Client session */
+	  struct client_entry *ce,    /* Client session */
-		 cbuf         *cbret,        /* Reply eg <rpc-reply>... */
+	  cbuf         *cbret,        /* Reply eg <rpc-reply>... */
-		 void         *arg)          /* Argument given at register */
+	  void         *arg)          /* Argument given at register */
 {
-    cprintf(cbret, "<rpc-reply><ok>%s</ok></rpc-reply>", xml_body(xe));    
+    cprintf(cbret, "<rpc-reply><route>"
+	    "<address-family>ipv4</address-family>"
+	    "<next-hop><next-hop-list>2.3.4.5</next-hop-list></next-hop>"
+	    "</route></rpc-reply>");    
+    return 0;
+}
+
+/*! IETF Routing route-count rpc */
+static int 
+route_count(clicon_handle h, 
+	    cxobj        *xe,           /* Request: <rpc><xn></rpc> */
+	    struct client_entry *ce,    /* Client session */
+	    cbuf         *cbret,        /* Reply eg <rpc-reply>... */
+	    void         *arg)          /* Argument given at register */
+{
+    cprintf(cbret, "<rpc-reply><ok/></rpc-reply>");    
     return 0;
 }
 
@@ -173,10 +189,15 @@ plugin_init(clicon_handle h)
 
     if (notification_timer_setup(h) < 0)
 	goto done;
-    /* Register callback for netconf application-specific rpc call */
+    /* Register callback for routing rpc calls */
-    if (backend_rpc_cb_register(h, routing_downcall, 
+    if (backend_rpc_cb_register(h, fib_route, 
 				NULL, 
-				"myrouting"/* Xml tag when callback is made */
+				"fib-route"/* Xml tag when callback is made */
+				) < 0)
+	goto done;
+    if (backend_rpc_cb_register(h, route_count, 
+				NULL, 
+				"route-count"/* Xml tag when callback is made */
 				) < 0)
 	goto done;
     retval = 0;

example/routing_cli.c

@@ -92,40 +92,36 @@ mycallback(clicon_handle h, cvec *cvv, cvec *argv)
     return retval;
 }
 
-/*! get argument and send as string to backend as RPC (which returns the string) 
+/*! Example "downcall": ietf-routing fib-route RPC */
- */
 int
-downcall(clicon_handle h, 
+fib_route_rpc(clicon_handle h, 
-	 cvec         *vars, 
+	      cvec         *cvv, 
-	 cvec         *argv)
+	      cvec         *argv)
 {
-    int      retval = -1;
+    int        retval = -1;
-    struct clicon_msg *msg = NULL;
+    cg_var    *instance;
-    char    *str="";
+    cxobj     *xtop = NULL;
-    cg_var  *cv;
+    cxobj     *xrpc;
-    cxobj   *xret=NULL;
+    cxobj     *xret = NULL;
-    cxobj   *xerr;
-    cxobj   *xdata;
 
-    if (cvec_len(vars)==2){
+    /* User supplied variable in CLI command */
-	if ((cv = cvec_i(vars, 1)) != NULL)
+    instance = cvec_find(cvv, "instance"); /* get a cligen variable from vector */
-	    str = cv_string_get(cv);
+    /* Create XML for fib-route netconf RPC */
-    }
+    if (clicon_xml_parse(&xtop, "<rpc><fib-route><routing-instance-name>%s</routing-instance-name></fib-route></rpc>", instance) < 0)
-    if ((msg = clicon_msg_encode("<rpc><myrouting>%s</myrouting></rpc>", str)) == NULL)
 	goto done;
-    if (clicon_rpc_msg(h, msg, &xret, NULL) < 0)
+    /* Skip top-level */
+    xrpc = xml_child_i(xtop, 0);
+    /* Send to backend */
+    if (clicon_rpc_netconf_xml(h, xrpc, &xret, NULL) < 0)
 	goto done;
-    if ((xerr = xpath_first(xret, "//rpc-error")) != NULL){
+    /* Print result */
-	clicon_rpc_generate_error(xerr);
+    xml_print(stdout, xml_child_i(xret, 0));
-	goto done;
-    }
-    if ((xdata = xpath_first(xret, "//ok")) != NULL)
-	cli_output(stdout, "%s\n", xml_body(xdata));
     retval = 0;
-  done:
+ done:
     if (xret)
-    	xml_free(xret);
+	xml_free(xret);
-    if (msg)
+    if (xtop)
-	free(msg);
+	xml_free(xtop);
     return retval;
 }
+

example/routing_cli.cli

@@ -4,10 +4,10 @@ CLICON_PROMPT="%U@%H> ";
 CLICON_PLUGIN="routing_cli";
 
 # Note, when switching to PT, change datamodel to only @datamodel
-set @datamodel:ietf-ip, cli_set();
+set @datamodel:example, cli_set();
-merge @datamodel:ietf-ip, cli_merge();
+merge @datamodel:example, cli_merge();
-create @datamodel:ietf-ip, cli_create();
+create @datamodel:example, cli_create();
-delete("Delete a configuration item") @datamodel:ietf-ip, cli_del();
+delete("Delete a configuration item") @datamodel:example, cli_del();
 
 validate("Validate changes"), cli_validate();
 commit("Commit the changes"), cli_commit();
@@ -49,7 +49,7 @@ load("Load configuration from XML file") <filename:string>("Filename (local file
     merge("Merge file with existent candidate"), load_config_file("filename", "merge");
 }
 example("This is a comment") <var:int32>("Just a random number"), mycallback("myarg");
-downcall("This is a downcall") <str:rest>, downcall();
+rpc("fib-route rpc") <instance:string>("routing instance"), fib_route_rpc("myarg");
 notify("Get notifications from backend"), cli_notify("ROUTING", "1", "text");
 no("Negate") notify("Get notifications from backend"), cli_notify("ROUTING", "0", "xml");
 lock,cli_lock("candidate");

lib/clixon/clixon_xml_map.h

@@ -53,7 +53,8 @@ enum {
  */
 int xml2txt(FILE *f, cxobj *x, int level);
 int xml2cli(FILE *f, cxobj *x, char *prepend, enum genmodel_type gt);
-int xml_yang_validate(cxobj *xt, yang_stmt *ys) ;
+int xml_yang_validate_add(cxobj *xt, void *arg);
+int xml_yang_validate_all(cxobj *xt, void *arg);
 int xml2cvec(cxobj *xt, yang_stmt *ys, cvec **cvv0);
 int cvec2xml_1(cvec *cvv, char *toptag, cxobj *xp, cxobj **xt0);
 int xml_diff(yang_spec *yspec, cxobj *xt1, cxobj *xt2, 	 

lib/clixon/clixon_yang.h

@@ -54,7 +54,7 @@
  * - Cant use the symbols in this file because yacc needs token definitions
  */
 enum rfc_6020{
-    Y_ANYXML,
+    Y_ANYXML = 0,
     Y_ARGUMENT,
     Y_AUGMENT,
     Y_BASE,
@@ -204,11 +204,12 @@ yang_stmt *yang_find(yang_node *yn, int keyword, char *argument);
 yang_stmt *yang_find_syntax(yang_node *yn, char *argument);
 yang_stmt *yang_find_topnode(yang_spec *ysp, char *name);
 
+int        yang_print(FILE *f, yang_node *yn);
 int        yang_print_cbuf(cbuf *cb, yang_node *yn, int marginal);
-int        yang_print(FILE *f, yang_node *yn, int marginal);
 int        yang_parse(clicon_handle h, const char *yang_dir, 
 		      const char *module, const char *revision, yang_spec *ysp);
-int        yang_apply(yang_node *yn, yang_applyfn_t fn, void *arg);
+int        yang_apply(yang_node *yn, enum rfc_6020 key, yang_applyfn_t fn, 
+		      void *arg);
 yang_node *yang_xpath_abs(yang_node *yn, char *xpath);
 yang_node *yang_xpath(yang_node *yn, char *xpath);
 cg_var    *ys_parse(yang_stmt *ys, enum cv_type cvtype);

lib/src/clixon_proto_client.c

@@ -272,8 +272,8 @@ clicon_rpc_get_config(clicon_handle       h,
     /* Send xml error back: first check error, then ok */
     if ((xd = xpath_first(xret, "/rpc-reply/rpc-error")) != NULL)
 	xd = xml_parent(xd); /* point to rpc-reply */
-    else if ((xd = xpath_first(xret, "/rpc-reply/data/config")) == NULL)
+    else if ((xd = xpath_first(xret, "/rpc-reply/data")) == NULL)
-	if ((xd = xml_new("config", NULL)) == NULL)
+	if ((xd = xml_new("data", NULL)) == NULL)
 	    goto done;
     if (xt){
 	if (xml_rm(xd) < 0)
@@ -521,8 +521,8 @@ clicon_rpc_get(clicon_handle       h,
     /* Send xml error back: first check error, then ok */
     if ((xd = xpath_first(xret, "/rpc-reply/rpc-error")) != NULL)
 	xd = xml_parent(xd); /* point to rpc-reply */
-    else if ((xd = xpath_first(xret, "/rpc-reply/data/config")) == NULL)
+    else if ((xd = xpath_first(xret, "/rpc-reply/data")) == NULL)
-	if ((xd = xml_new("config", NULL)) == NULL)
+	if ((xd = xml_new("data", NULL)) == NULL)
 	    goto done;
     if (xt){
 	if (xml_rm(xd) < 0)

lib/src/clixon_xml_map.c

@@ -274,16 +274,62 @@ xml2cli(FILE              *f,
     return retval;
 }
 
+/*! Validate an xml node of type leafref, ensure the value is one of that path's reference
+ * @param[in]  xt    XML leaf node of type leafref
+ * @param[in]  ytype Yang type statement belonging to the XML node
+ */
+static int
+validate_leafref(cxobj     *xt,
+		 yang_stmt *ytype)
+{
+    int          retval = -1;
+    yang_stmt   *ypath;
+    cxobj      **xvec = NULL;
+    cxobj       *x;
+    int          i;
+    size_t       xlen = 0;
+    char        *leafrefbody;
+    char        *leafbody;
 
-/*! Validate a single XML node with yang specification
+
- * -  If no value and mandatory flag set in spec, report error.
+    if ((leafrefbody = xml_body(xt)) == NULL)
- * - Validate value versus spec, and report error if no match. Currently 
+	return 0;
- *   only int ranges and string regexp checked.
+    if ((ypath = yang_find((yang_node*)ytype, Y_PATH, NULL)) == NULL){
- * @retval  0 OK
+	clicon_err(OE_DB, 0, "Leafref %s requires path statement", ytype->ys_argument);
-  */
+	goto done;
+    }
+    if (xpath_vec(xt, ypath->ys_argument, &xvec, &xlen) < 0) 
+	goto done;
+    for (i = 0; i < xlen; i++) {
+	x = xvec[i];
+	if ((leafbody = xml_body(x)) == NULL)
+	    continue;
+	if (strcmp(leafbody, leafrefbody) == 0)
+	    break;
+    }
+    if (i==xlen){
+	clicon_err(OE_DB, 0, "Leafref validation failed, no such leaf: %s",
+		   leafrefbody);
+	goto done;
+    }
+    retval = 0;
+ done:
+    if (xvec)
+	free(xvec);
+    return retval;
+}
+
+/*! Validate a single XML node with yang specification for added entry
+ * 1. Check if mandatory leafs present as subs.
+ * 2. Check leaf values, eg int ranges and string regexps.
+ * @param[in]  xt  XML node to be validated
+ * @retval     0   Valid OK
+ * @retval    -1   Validation failed
+ * @see xml_yang_validate_all
+ */
 int
-xml_yang_validate(cxobj     *xt, 
+xml_yang_validate_add(cxobj   *xt, 
-		  yang_stmt *ys0)
+		      void    *arg)
 {
     int        retval = -1;
     cg_var    *cv = NULL;
@@ -294,7 +340,7 @@ xml_yang_validate(cxobj     *xt,
     char      *body;
     
     /* if not given by argument (overide) use default link */
-    ys = ys0?ys0:xml_spec(xt);
+    ys = xml_spec(xt);
     switch (ys->ys_keyword){
     case Y_LIST:
 	/* fall thru */
@@ -346,6 +392,43 @@ xml_yang_validate(cxobj     *xt,
     return retval;
 }
 
+/*! Validate a single XML node with yang specification for all (not only added) entries
+ * 1. Check leafrefs. Eg you delete a leaf and a leafref references it.
+ * @param[in]  xt  XML node to be validated
+ * @retval     0   Valid OK
+ * @retval    -1   Validation failed
+ * @see xml_yang_validate_add
+ */
+int
+xml_yang_validate_all(cxobj   *xt, 
+		      void    *arg)
+{
+    int        retval = -1;
+    yang_stmt *ys;
+    yang_stmt *ytype;
+    
+    /* if not given by argument (overide) use default link */
+    ys = xml_spec(xt);
+    switch (ys->ys_keyword){
+    case Y_LEAF:
+	/* fall thru */
+    case Y_LEAF_LIST:
+	/* Special case if leaf is leafref, then first check against
+	   current xml tree
+	 */
+	if ((ytype = yang_find((yang_node*)ys, Y_TYPE, NULL)) != NULL &&
+	    strcmp(ytype->ys_argument, "leafref") == 0)
+	    if (validate_leafref(xt, ytype) < 0)
+		goto done;
+	break;
+    default:
+	break;
+    }
+    retval = 0;
+ done:
+    return retval;
+}
+
 /*! Translate a single xml node to a cligen variable vector. Note not recursive 
  * @param[in]  xt   XML tree containing one top node
  * @param[in]  ys   Yang spec containing type specification of top-node of xt
@@ -1908,3 +1991,4 @@ xml_merge(cxobj     *x0,
  done:
     return retval;
 }
+

lib/src/clixon_xsl.c

@@ -435,6 +435,7 @@ recursive_find(cxobj   *xn,
  *  - @<attr>=<value>
  *  - <number>
  *  - <name>=<value> # RelationalExpr '=' RelationalExpr
+ *  - <name>=current()<xpath> XXX
  * @see https://www.w3.org/TR/xpath/#predicates
  */
 static int
@@ -565,6 +566,7 @@ xpath_find(struct xpath_element *xe,
     cxobj         *xv;
     int            descendants = 0;
     cxobj        **vec1 = NULL;
+    cxobj         *xparent;
     size_t         vec1len = 0;
     struct xpath_predicate *xp;
 
@@ -587,10 +589,13 @@ xpath_find(struct xpath_element *xe,
     case A_SELF:
 	break;
     case A_PARENT:
+	j = 0;
 	for (i=0; i<vec0len; i++){
 	    xv = vec0[i];
-	    vec0[i] = xml_parent(xv);
+	    if ((xparent = xml_parent(xv)) != NULL)
+		vec0[j++] = xparent;
 	}
+	vec0len = j;
 	break;
     case A_ROOT: /* set list to NULL */
 	x = vec0[0];

lib/src/clixon_yang.c

@@ -373,7 +373,7 @@ yn_each(yang_node *yn,
  *
  * @param[in]  yn         Yang node, current context node.
  * @param[in]  keyword    if 0 match any keyword
- * @param[in]  argument   if NULL, match any argument.
+ * @param[in]  argument   String compare w wrgument. if NULL, match any.
  * This however means that if you actually want to match only a yang-stmt with 
  * argument==NULL you cannot, but I have not seen any such examples.
  * @see yang_find_syntax
@@ -682,43 +682,41 @@ quotedstring(char *s)
 }
 
 /*! Print yang specification to file
+ * @param[in]  f         File to print to.
+ * @param[in]  yn        Yang node to print
  * @see yang_print_cbuf
  */
 int
 yang_print(FILE      *f, 
-	   yang_node *yn, 
+	   yang_node *yn)
-	   int        marginal)
 {
-    yang_stmt *ys = NULL;
+    int        retval = -1;
+    cbuf      *cb = NULL;
 
-    while ((ys = yn_each(yn, ys)) != NULL) {
+    if ((cb = cbuf_new()) == NULL){
-	fprintf(f, "%*s%s", marginal, "", yang_key2str(ys->ys_keyword));
+	clicon_err(OE_YANG, errno, "%s: cbuf_new", __FUNCTION__);
-	fflush(f);
+	goto done;
-	if (ys->ys_argument){
-	    if (quotedstring(ys->ys_argument))
-		fprintf(f, " \"%s\"", ys->ys_argument);
-	    else
-		fprintf(f, " %s", ys->ys_argument);
-	}
-	if (ys->ys_len){
-	    fprintf(f, " {\n");
-	    yang_print(f, (yang_node*)ys, marginal+3);
-	    fprintf(f, "%*s%s\n", marginal, "", "}");
-	}
-	else
-	    fprintf(f, ";\n");
     }
-    return 0;
+    if (yang_print_cbuf(cb, yn, 0) < 0)
+	goto done;
+    fprintf(f, "%s", cbuf_get(cb));
+    if (cb)
+	cbuf_free(cb);
+    retval = 0;
+ done:
+    return retval;
 }
 
 /*! Print yang specification to cligen buf
+ * @param[in]  cb        Cligen buffer. This is where the pretty print is.
+ * @param[in]  yn        Yang node to print
+ * @param[in]  marginal  Tab indentation, mainly for recursion.
  * @code
  *  cbuf *cb = cbuf_new();
  *  yang_print_cbuf(cb, yn, 0);
  *  // output is in cbuf_buf(cb);
  *  cbuf_free(cb);
  * @endcode
- * @see yang_print
  */
 int
 yang_print_cbuf(cbuf      *cb,
@@ -1587,17 +1585,18 @@ yang_parse(clicon_handle h,
     clicon_dbspec_name_set(h, ymod->ys_argument);
 
     /* Resolve all types */
-    yang_apply((yang_node*)ysp, ys_resolve_type, NULL);
+    yang_apply((yang_node*)ysp, Y_TYPE, ys_resolve_type, NULL);
 
     /* Step 2: Macro expansion of all grouping/uses pairs. Expansion needs marking */
     if (yang_expand((yang_node*)ysp) < 0)
 	goto done;
-    yang_apply((yang_node*)ymod, ys_flag_reset, (void*)YANG_FLAG_MARK);
+    yang_apply((yang_node*)ymod, -1, ys_flag_reset, (void*)YANG_FLAG_MARK);
-    /* Step 4: Go through parse tree and populate it with cv types */
+
-    if (yang_apply((yang_node*)ysp, ys_populate, NULL) < 0)
+    /* Step 3: Go through parse tree and populate it with cv types */
+    if (yang_apply((yang_node*)ysp, -1, ys_populate, NULL) < 0)
 	goto done;
 
-    /* Step 3: Top-level augmentation of all modules */
+    /* Step 4: Top-level augmentation of all modules */
     if (yang_augment_spec(ysp) < 0)
 	goto done;
 
@@ -1606,7 +1605,6 @@ yang_parse(clicon_handle h,
     return retval;
 }
 
-
 /*! Apply a function call recursively on all yang-stmt s recursively
  *
  * Recursively traverse all yang-nodes in a parse-tree and apply fn(arg) for 
@@ -1614,34 +1612,49 @@ yang_parse(clicon_handle h,
  * argument as args.
  * The tree is traversed depth-first, which at least guarantees that a parent is
  * traversed before a child.
- * @param[in]  xn   XML node
+ * @param[in]  yn   yang node
- * @param[in]  type matching type or -1 for any
+ * @param[in]  key  yang keyword to use as filer or -1 for all
  * @param[in]  fn   Callback
  * @param[in]  arg  Argument
+ * @retval    -1    Error, aborted at first error encounter
+ * @retval     0    OK, all nodes traversed
+ * @retval     n    OK, aborted at first encounter of first match
  * @code
  * int ys_fn(yang_stmt *ys, void *arg)
  * {
  *   return 0;
  * }
- * yang_apply((yang_node*)ys, ys_fn, NULL);
+ * yang_apply((yang_node*)ys, Y_TYPE, ys_fn, NULL);
  * @endcode
  * @note do not delete or move around any children during this function
  */
 int
 yang_apply(yang_node     *yn, 
+	   enum rfc_6020  keyword,
 	   yang_applyfn_t fn, 
 	   void          *arg)
 {
     int        retval = -1;
     yang_stmt *ys = NULL;
     int        i;
+    int        ret;
 
     for (i=0; i<yn->yn_len; i++){
 	ys = yn->yn_stmt[i];
-	if (fn(ys, arg) < 0)
+	if (keyword == -1 || keyword == ys->ys_keyword){
+	    if ((ret = fn(ys, arg)) < 0)
+		goto done;
+	    if (ret > 0){
+		retval = ret;
+		goto done;
+	    }
+	}
+	if ((ret = yang_apply((yang_node*)ys, keyword, fn, arg)) < 0)
 	    goto done;
-	if (yang_apply((yang_node*)ys, fn, arg) < 0)
+	if (ret > 0){
+	    retval = ret;
 	    goto done;
+	}
     }
     retval = 0;
   done:
@@ -1929,7 +1942,7 @@ yang_spec_main(clicon_handle h,
 	goto done;
     clicon_dbspec_yang_set(h, yspec);	
     if (printspec)
-	yang_print(f, (yang_node*)yspec, 0);
+	yang_print(f, (yang_node*)yspec);
     retval = 0;
   done:
     return retval;

lib/src/clixon_yang_type.c

@@ -90,7 +90,6 @@ static const map_str2int ytmap[] = {
     {"int16",       CGV_INT16},  
     {"int64",       CGV_INT64},
     {"leafref",     CGV_STRING},  /* XXX */
-
     {"uint8",       CGV_UINT8}, 
     {"uint16",      CGV_UINT16},
     {"uint32",      CGV_UINT32},
@@ -105,12 +104,6 @@ yang_builtin(char *type)
 {
     if (clicon_str2int(ytmap, type) != -1)
 	return 1;
-#if 0
-    const struct map_str2int *yt;
-    for (yt = &ytmap[0]; yt->ms_str; yt++)
-       if (strcmp(yt->ms_str, type) == 0)
-           return 1;
-#endif
     return 0;
 }
 
@@ -225,8 +218,6 @@ ys_resolve_type(yang_stmt *ys,
     uint8_t           fraction = 0;
     yang_stmt        *resolved = NULL;
  
-    if (ys->ys_keyword != Y_TYPE)
-        return 0;
     if (yang_type_resolve((yang_stmt*)ys->ys_parent, ys, &resolved,
 			  &options, &mincv, &maxcv, &pattern, &fraction) < 0)
 	goto done;
@@ -857,6 +848,7 @@ yang_type_resolve(yang_stmt   *ys,
     ylength   = yang_find((yang_node*)ytype, Y_LENGTH, NULL);
     ypattern  = yang_find((yang_node*)ytype, Y_PATTERN, NULL);
     yfraction = yang_find((yang_node*)ytype, Y_FRACTION_DIGITS, NULL);
+
     /* Check if type is basic type. If so, return that */
     if (prefix == NULL && yang_builtin(type)){
 	*yrestype = ytype; 
@@ -867,8 +859,10 @@ yang_type_resolve(yang_stmt   *ys,
 
     /* Not basic type. Now check if prefix which means we look in other module */
     if (prefix){ /* Go to top and find import that matches */
-	if ((ymod = yang_find_module_by_prefix(ys, prefix)) == NULL)
+	if ((ymod = yang_find_module_by_prefix(ys, prefix)) == NULL){
+	    clicon_err(OE_DB, 0, "Module not resolved: %s", prefix);
 	    goto done;
+	}
 	if ((rytypedef = yang_find((yang_node*)ymod, Y_TYPEDEF, type)) == NULL)
 	    goto ok; /* unresolved */
     }

test/README.md

@@ -4,8 +4,10 @@ This directory contains testing code for clixon and the example
 routing application:
 - clixon    A top-level script clones clixon in /tmp and starts all.sh. You can copy this file (review it first) and place as cron script
 - all.sh    Run through all tests named 'test*.sh' in this directory. Therefore, if you place a test in this directory matching 'test*.sh' it will be run automatically. 
-- test1.sh  CLI tests
+- test_cli.sh       CLI tests
-- test2.sh  Netconf tests
+- test_netconf.sh   Netconf tests
-- test3.sh  Restconf tests
+- test_restconf.sh  Restconf tests
-- test4.sh  Yang tests
+- test_yang.sh      Yang tests for constructs not in the example.
-- test5.sh  Datastore tests
+- test_leafref.sh   Yang leafref tests
+- test_datastore.sh Datastore tests
+

test/test6.sh

@@ -1,64 +0,0 @@
-#!/bin/bash
-# Test6: Yang specifics: rpc and state info
-
-# include err() and new() functions
-. ./lib.sh
-
-# For memcheck
-# clixon_netconf="valgrind --leak-check=full --show-leak-kinds=all clixon_netconf"
-clixon_netconf=clixon_netconf
-clixon_cli=clixon_cli
-
-cat <<EOF > /tmp/rpc.yang
-module ietf-ip{
-     rpc fib-route {
-       input {
-         leaf name {
-           type string;
-           mandatory "true";
-         }
-         leaf destination-address {
-           type string;
-         }
-       }
-       output {
-         container route {
-           leaf address{
-              type string;
-           }
-           leaf address{
-              type string;
-           } 
-         }
-       }
-     }
-}
-EOF
-
-# kill old backend (if any)
-new "kill old backend"
-sudo clixon_backend -zf $clixon_cf -y /tmp/rpc
-if [ $? -ne 0 ]; then
-    err
-fi
-
-new "start backend"
-# start new backend
-sudo clixon_backend -If $clixon_cf -y /tmp/rpc
-if [ $? -ne 0 ]; then
-    err
-fi
-new "netconf rpc (notyet)"
-#expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/rpc" "<rpc><fib-route><name></name></fib-route></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
-
-new "Kill backend"
-# Check if still alive
-pid=`pgrep clixon_backend`
-if [ -z "$pid" ]; then
-    err "backend already dead"
-fi
-# kill backend
-sudo clixon_backend -zf $clixon_cf
-if [ $? -ne 0 ]; then
-    err "kill backend"
-fi

test/test_cli.sh

@@ -31,8 +31,8 @@ new "cli tests"
 new "cli configure top"
 expectfn "$clixon_cli -1f $clixon_cf set interfaces" ""
 
-new "cli show configuration top"
+new "cli show configuration top (no presence)"
-expectfn "$clixon_cli -1f $clixon_cf show conf cli" "^interfaces$"
+expectfn "$clixon_cli -1f $clixon_cf show conf cli" ""
 
 new "cli configure delete top"
 expectfn "$clixon_cli -1f $clixon_cf delete interfaces" ""
@@ -87,7 +87,7 @@ expectfn "$clixon_cli -1f $clixon_cf -l o debug level 1" ""
 expectfn "$clixon_cli -1f $clixon_cf -l o debug level 0" ""
 
 new "cli downcall"
-expectfn "$clixon_cli -1f $clixon_cf -l o downcall \"This is a test =====\"" "^\"This is a test =====\"$"
+expectfn "$clixon_cli -1f $clixon_cf -l o rpc ipv4" "^<rpc-reply>"
 
 new "Kill backend"
 # Check if still alive

test/test_leafref.sh

@@ -0,0 +1,104 @@
+#!/bin/bash
+# Test7: Yang specifics: leafref
+
+# include err() and new() functions
+. ./lib.sh
+
+# For memcheck
+# clixon_netconf="valgrind --leak-check=full --show-leak-kinds=all clixon_netconf"
+clixon_netconf=clixon_netconf
+clixon_cli=clixon_cli
+
+cat <<EOF > /tmp/leafref.yang
+module example{
+     typedef admin-status{
+        type string;
+     }
+     list interface {
+         key "name";
+         leaf name {
+             type string;
+         }
+         leaf admin-status {
+             type admin-status;
+         }
+         list address {
+             key "ip";
+             leaf ip {
+                 type string;
+             }
+         }
+     }
+     container default-address {
+         leaf ifname {
+             type leafref {
+                 path "../../interface/name";
+             }
+         }
+         leaf address {
+             type leafref {
+                 path "../../interface[name=eth0]"
+                    + "/address/ip";
+             }
+         }
+    }
+}
+EOF
+#                 path "../../interface[name = current()/../ifname]"
+
+# kill old backend (if any)
+new "kill old backend"
+sudo clixon_backend -zf $clixon_cf -y /tmp/leafref
+if [ $? -ne 0 ]; then
+    err
+fi
+
+new "start backend"
+# start new backend
+sudo clixon_backend -If $clixon_cf -y /tmp/leafref
+if [ $? -ne 0 ]; then
+    err
+fi
+
+new "leafref base config"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><edit-config><target><candidate/></target><config><interface><name>eth0</name><admin-status>up</admin-status><address><ip>192.0.2.1</ip></address><address><ip>192.0.2.2</ip></address></interface><interface><name>lo</name><admin-status>up</admin-status><address><ip>127.0.0.1</ip></address></interface></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "leafref get config"
+expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source></get-config></rpc>]]>]]>' '^<rpc-reply><data><interface><name>eth0</name>'
+
+new "leafref base commit"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><commit/></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "leafref add wrong ref"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><edit-config><target><candidate/></target><config><default-address><ifname>eth3</ifname><address>10.0.4.6</address></default-address></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "leafref validate"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>" "^<rpc-reply><rpc-error><error-tag>missing-attribute</error-tag>"
+
+new "leafref discard-changes"
+expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><discard-changes/></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "leafref add correct ref"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><edit-config><target><candidate/></target><config><default-address><ifname>eth0</ifname><address>192.0.2.2</address></default-address></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "leafref validate (ok)"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>"
+
+new "leafref delete leaf"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><edit-config><target><candidate/></target><config><interface operation=\"delete\"><name>eth0</name></interface></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>"
+
+new "leafref validate (should fail)"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/leafref" "<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>"  "^<rpc-reply><rpc-error><error-tag>missing-attribute</error-tag>"
+
+
+new "Kill backend"
+# Check if still alive
+pid=`pgrep clixon_backend`
+if [ -z "$pid" ]; then
+    err "backend already dead"
+fi
+# kill backend
+sudo clixon_backend -zf $clixon_cf
+if [ $? -ne 0 ]; then
+    err "kill backend"
+fi

test/test_netconf.sh

@@ -36,7 +36,7 @@ new "Add subtree eth/0/0 using none and create which should add eth/0/0"
 expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><edit-config><target><candidate/></target><config><interfaces><interface operation="create"><name>eth/0/0</name><type>eth</type></interface></interfaces></config><default-operation>none</default-operation> </edit-config></rpc>]]>]]>' "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "Check eth/0/0 added using xpath"
-expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth/0/0]"/></get-config></rpc>]]>]]>' "^<rpc-reply><data><config><interfaces><interface><name>eth/0/0</name><type>eth</type><enabled>true</enabled></interface></interfaces></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth/0/0]"/></get-config></rpc>]]>]]>' "^<rpc-reply><data><interfaces><interface><name>eth/0/0</name><type>eth</type><enabled>true</enabled></interface></interfaces></data></rpc-reply>]]>]]>$"
 
 new "Re-create same eth/0/0 which should generate error"
 expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><edit-config><target><candidate/></target><config><interfaces><interface operation="create"><name>eth/0/0</name><type>eth</type></interface></interfaces></config><default-operation>none</default-operation> </edit-config></rpc>]]>]]>' "^<rpc-reply><rpc-error>"
@@ -44,8 +44,8 @@ expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><edit-config><target><candidate
 new "Delete eth/0/0 using none config"
 expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><edit-config><target><candidate/></target><config><interfaces><interface operation="delete"><name>eth/0/0</name><type>eth</type></interface></interfaces></config><default-operation>none</default-operation> </edit-config></rpc>]]>]]>'  "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
-#new "Check deleted eth/0/0"
+new "Check deleted eth/0/0 (non-presence container)"
-#expecteof "$clixon_netconf -qf $clixon_cf" '<rpc message-id="101"><get-config><source><candidate/></source></get-config></rpc>]]>]]>' '^<rpc-reply message-id="101"><data><config><interfaces/></config></data></rpc-reply>]]>]]>$'
+expecteof "$clixon_netconf -qf $clixon_cf" '<rpc message-id="101"><get-config><source><candidate/></source></get-config></rpc>]]>]]>' '^<rpc-reply message-id="101"><data/></rpc-reply>]]>]]>$'
 
 new "Re-Delete eth/0/0 using none should generate error"
 expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><edit-config><target><candidate/></target><config><interfaces><interface operation="delete"><name>eth/0/0</name><type>eth</type></interface></interfaces></config><default-operation>none</default-operation> </edit-config></rpc>]]>]]>'  "^<rpc-reply><rpc-error>"
@@ -54,10 +54,10 @@ new "netconf edit config"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><edit-config><target><candidate/></target><config><interfaces><interface><name>eth/0/0</name></interface><interface><name>eth1</name><enabled>true</enabled><ipv4><address><ip>9.2.3.4</ip><prefix-length>24</prefix-length></address></ipv4></interface></interfaces></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "netconf get config xpath"
-expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth1]/enabled"/></get-config></rpc>]]>]]>' "^<rpc-reply><data><config><interfaces><interface><name>eth1</name><enabled>true</enabled></interface></interfaces></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth1]/enabled"/></get-config></rpc>]]>]]>' "^<rpc-reply><data><interfaces><interface><name>eth1</name><enabled>true</enabled></interface></interfaces></data></rpc-reply>]]>]]>$"
 
 new "netconf get config xpath parent"
-expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth1]/enabled/../.."/></get-config></rpc>]]>]]>' "^<rpc-reply><data><config><interfaces><interface><name>eth/0/0</name><enabled>true</enabled></interface><interface><name>eth1</name><enabled>true</enabled><ipv4><enabled>true</enabled><forwarding>false</forwarding><address><ip>9.2.3.4</ip><prefix-length>24</prefix-length></address></ipv4></interface></interfaces></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf" '<rpc><get-config><source><candidate/></source><filter type="xpath" select="/interfaces/interface[name=eth1]/enabled/../.."/></get-config></rpc>]]>]]>' "^<rpc-reply><data><interfaces><interface><name>eth/0/0</name><enabled>true</enabled></interface><interface><name>eth1</name><enabled>true</enabled><ipv4><enabled>true</enabled><forwarding>false</forwarding><address><ip>9.2.3.4</ip><prefix-length>24</prefix-length></address></ipv4></interface></interfaces></data></rpc-reply>]]>]]>$"
 
 new "netconf validate missing type"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>" "^<rpc-reply><rpc-error>"
@@ -81,7 +81,7 @@ new "netconf edit config replace"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><edit-config><target><candidate/></target><config><interfaces><interface><name>eth2</name><type>eth</type></interface></interfaces></config><default-operation>merge</default-operation></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "netconf get replaced config"
-expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><get-config><source><candidate/></source></get-config></rpc>]]>]]>" "^<rpc-reply><data><config><interfaces><interface><name>eth1</name><type>eth</type><enabled>true</enabled></interface><interface><name>eth2</name><type>eth</type><enabled>true</enabled></interface></interfaces></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><get-config><source><candidate/></source></get-config></rpc>]]>]]>" "^<rpc-reply><data><interfaces><interface><name>eth1</name><type>eth</type><enabled>true</enabled></interface><interface><name>eth2</name><type>eth</type><enabled>true</enabled></interface></interfaces></data></rpc-reply>]]>]]>$"
 
 new "netconf discard-changes"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><discard-changes/></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
@@ -111,7 +111,7 @@ new "copy startup"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><copy-config><target><startup/></target><source><candidate/></source></copy-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "netconf get startup"
-expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><get-config><source><startup/></source></get-config></rpc>]]>]]>" "^<rpc-reply><data><config><interfaces><interface><name>eth1</name><type>eth</type><enabled>true</enabled></interface></interfaces></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><get-config><source><startup/></source></get-config></rpc>]]>]]>" "^<rpc-reply><data><interfaces><interface><name>eth1</name><type>eth</type><enabled>true</enabled></interface></interfaces></data></rpc-reply>]]>]]>$"
 
 new "netconf delete startup"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><delete-config><target><startup/></target></delete-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
@@ -119,6 +119,9 @@ expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><delete-config><target><startup
 new "netconf check empty startup"
 expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><get-config><source><startup/></source></get-config></rpc>]]>]]>" "^<rpc-reply><data/></rpc-reply>]]>]]>$"
 
+new "netconf rpc"
+expecteof "$clixon_netconf -qf $clixon_cf" "<rpc><fib-route><routing-instance-name>ipv4</routing-instance-name><destination-address><address-family>ipv4</address-family></destination-address></fib-route></rpc>]]>]]>" "^<rpc-reply><route><address-family>ipv4</address-family><next-hop><next-hop-list>"
+
 new "netconf subscription"
 expectwait "$clixon_netconf -qf $clixon_cf" "<rpc><create-subscription><stream>ROUTING</stream></create-subscription></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]><notification><event>Routing notification</event></notification>]]>]]>$" 30
 

test/test_yang.sh

@@ -10,7 +10,7 @@ clixon_netconf=clixon_netconf
 clixon_cli=clixon_cli
 
 cat <<EOF > /tmp/test.yang
-module ietf-ip{
+module example{
    container x {
     list y {
       key "a b";
@@ -35,7 +35,15 @@ module ietf-ip{
     leaf g {
       type string;  
     }
-    container h {
+    container nopresence {
+      description "No presence should be removed if no children";
+      leaf j {
+        type string;
+      }
+    }
+    container presence {
+      description "Presence should not be removed even if no children";
+      presence "even if empty should remain";
       leaf j {
         type string;
       }
@@ -70,29 +78,33 @@ new "netconf commit"
 expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><commit/></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "netconf get config xpath"
-expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/y[a=1][b=2]\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><config><x><y><a>1</a><b>2</b><c>5</c></y></x></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/y[a=1][b=2]\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><x><y><a>1</a><b>2</b><c>5</c></y></x></data></rpc-reply>]]>]]>$"
 
 new "netconf edit leaf-list"
 expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><edit-config><target><candidate/></target><config><x><f><e>hej</e><e>hopp</e></f></x></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
 
 new "netconf get leaf-list"
-expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/f/e\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><config><x><f><e>hej</e><e>hopp</e></f></x></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/f/e\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><x><f><e>hej</e><e>hopp</e></f></x></data></rpc-reply>]]>]]>$"
 
 new "netconf get leaf-list path"
-expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/f[e=hej]\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><config><x><f><e>hej</e><e>hopp</e></f></x></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/f[e=hej]\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><x><f><e>hej</e><e>hopp</e></f></x></data></rpc-reply>]]>]]>$"
 
-new "netconf get (state data XXX should be some)"
+new "netconf get (should be some)"
-expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get><filter type=\"xpath\" select=\"/\"/></get></rpc>]]>]]>" "^<rpc-reply><data><config><x><y><a>1</a><b>2</b><c>5</c></y><d/></x></config></data></rpc-reply>]]>]]>$"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get><filter type=\"xpath\" select=\"/\"/></get></rpc>]]>]]>" "^<rpc-reply><data><x><y><a>1</a><b>2</b><c>5</c></y><d/></x></data></rpc-reply>]]>]]>$"
 
 new "cli set leaf-list"
 expectfn "$clixon_cli -1f $clixon_cf -y /tmp/test set x f e foo" ""
 
 new "cli show leaf-list"
 expectfn "$clixon_cli -1f $clixon_cf -y /tmp/test show xpath /x/f/e" "<e>foo</e>"
-
 new "netconf set state data (not allowed)"
 expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><edit-config><target><candidate/></target><config><state><op>42</op></state></config></edit-config></rpc>]]>]]>" "^<rpc-reply><rpc-error><error-tag>invalid-value"
 
+new "netconf set presence and not present"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><edit-config><target><candidate/></target><config><x><nopresence/><presence/></x></config></edit-config></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
+
+new "netconf get"
+expecteof "$clixon_netconf -qf $clixon_cf -y /tmp/test" "<rpc><get-config><source><candidate/></source><filter type=\"xpath\" select=\"/x/*presence\"/></get-config></rpc>]]>]]>" "^<rpc-reply><data><x><presence/></x></data></rpc-reply>]]>]]>$"
 
 new "Kill backend"
 # Check if still alive