Merge branch 'master' into develop 3.3.3

CHANGELOG.md

@@ -1,37 +1,61 @@
 # Clixon Changelog
 
-## 3.3.3 (Upcoming)
+## 3.3.3 (25 November 2017)
+
+Thanks to Matthew Smith, Joe Loeliger at Netgate; Fredrik Pettai at
+SUNET for support, requests, debugging, bugfixes and proposed solutions.
 
-### Known issues
 ### Major changes:
-* Clixon can now be compiled and run on Apple Darwin.
-
 * Performance improvements
   * Added xml hash lookup instead of linear search for better performance of large lists. To disable, undefine XML_CHILD_HASH in clixon_custom.h
-  * netconf client was limited to 8K byte messages. Now limit is 2^32 bytes
+  * Netconf client was limited to 8K byte messages. New limit is 2^32 bytes.
 
 * XML and YANG-based configuration file.
-  * New configuration files have .xml suffix, old have .conf. Old config files till work for backward compatibility.
+  * New configuration files have .xml suffix, old have .conf.
   * The yang model is yang/clixon-config.yang.
-  * A migration utility is clixon_cli -x to print new format, eg:
+  * You can run backward compatible mode using `configure --with-config-compat`
-```
+  * In backward compatible mode both .xml and .conf works
-clixon_cli -f /usr/local/etc/routing.conf -1x
+  * For migration from old to new, a utility is clixon_cli -x to print new format. Run the command and save in configuration file with .xml suffix instead.
-```
+  ```
+    > clixon_cli -f /usr/local/etc/routing.conf -1x
+    <config>
+        <CLICON_CONFIGFILE>/usr/local/etc/routing.xml</CLICON_CONFIGFILE>
+        <CLICON_YANG_DIR>/usr/local/share/routing/yang</CLICON_YANG_DIR>
+        <CLICON_BACKEND_DIR>/usr/local/lib/routing/backend</CLICON_BACKEND_DIR>
+	...
+   </config>
+  ```
   
-* Introducing backend daemon startup modes. The flags -IRCr and option CLICON_USE_STARTUP_CONFIG are replaced with command-line option -s <mode> and option CLICON_STARTUP_MODE. You need to replace the starting of clixon_backend as follows:
+* Simplified backend daemon startup modes.
-  * -I replace with -s "init" (or use of CLICON_STARTUP_MODE option)
+  * The flags -IRCr are replaced with command-line option -s <mode>
-  * -CIr replace with -s "running" 
+  * You use the -s to select the mode. Example: `clixon_backend -s running`
-  * (no-option) replace with -s "none"
+  * You may also add a default method in the configuration file: `<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>`
-  * CLICON_USE_STARTUP_CONFIG=1 replace with -s "startup"
+  * The configuration option CLICON_USE_STARTUP_CONFIG is obsolete
-Backward compatibility is enabled by defining BACKEND_STARTUP_BACKWARD_COMPAT in include/clixon_custom.h
+  * Command-ine option `-I` is replaced with `-s init` 
+  * `-CIr` is replaced with `-s running`
+  * Use `-s none` if you request no action on startu
+  * Backward compatibility is enabled by:
+  ```
+      configure --with-startup-compat
+  ```
+  * You can run in backward compatible mode where both -IRCr and -s options works. But if -s is given, -IRCr options willbe ignored.
+
+* Extra XML has been added along with the new startup modes. Requested by Netgate.
+  * You can add extra XML with the -c option to the backend daemon on startup:
+  ```
+      clixon_backend ... -c extra.xml
+   ```
+  * You can also add extra XML by programming the plugin_reset() in the backend
+plugin. The example application shows how.
+
+* Clixon can now be compiled and run on Apple Darwin. Thanks SUNET.
 
 ### Minor changes:
+* Fixed DESTDIR make install/uninstall and break immediately on errors
 * Disabled key-value datastore. Enable with --with-keyvalue
-* Removed mandatory requirements for BACKEND, NETCONF, RESTCONF and CLI dirs.
+* Removed mandatory requirements for BACKEND, NETCONF, RESTCONF and CLI dirs in the configuration file. If these are not given, no plugins will be loaded of that type.
-* When user callbacks such as statedata() call returns -1, clixon_backend no
+
-  longer silently exits. Instead a log is printed and an RPC error is returned.
+* Restconf: http cookie sent as attribute in rpc restconf_post operations to backend as "id" XML attribute.
-* Added Floating point and negative number support to JSON
-* Restconf: http cookie sent as attribute in rpc restconf_post operations to backend.
 * Added option CLICON_CLISPEC_FILE as complement to CLICON_CLISPEC_DIR to
   specify single CLI specification file, not only directory containing files.
 	
@@ -41,19 +65,25 @@ Backward compatibility is enabled by defining BACKEND_STARTUP_BACKWARD_COMPAT in
 	cli_tree_active_set, cli_tree.
 
 * Added a format parameter to clicon_rpc_generate_error() and changed error
-  printouts for backend errors, such as commit and validate. Example of the
+  printouts for backend errors, such as commit and validate. (Thanks netgate).
-  new format:
+  Example of the new format:
 
 ```
-> commit
+  > commit
-Sep 27 18:11:58: Commit failed. Edit and try again or discard changes:
+  Sep 27 18:11:58: Commit failed. Edit and try again or discard changes:
-protocol invalid-value Missing mandatory variable: type
+  protocol invalid-value Missing mandatory variable: type
 ```
 
-* Added event_poll function.
+* Added event_poll() function to check if data is available on specific file descriptor.
 
-* Support for non-line scrolling in CLI, eg wrap lines. Set with:
+* Support for non-line scrolling in CLI, eg wrap lines. Thanks to Jon Loeliger for proposed solution. Set in configuration file with:
-  CLICON_CLI_LINESCROLLING 0
+  <CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
+
+### Corrected Bugs
+* Added floating point and negative number support to JSON
+* When user callbacks such as statedata() call returns -1, clixon_backend no
+  longer silently exits. Instead a log is printed and an RPC error is returned.
+  Cred to Matt, netgate for pointing this out.
 
 ## 3.3.2 (Aug 27 2017)
 

Makefile.in

@@ -63,7 +63,7 @@ $(SUBDIRS):
 
 depend:
 	for i in $(SUBDIRS) doc example docker; \
-		do (cd $$i && $(MAKE) $(MFLAGS) depend); done
+	do (cd $$i && $(MAKE) $(MFLAGS) depend); done
 
 # template clixon.conf file
 clixon.conf.cpp:  clixon.conf.cpp.cpp
@@ -73,23 +73,23 @@ clixon.mk:  clixon.mk.cpp
 	$(CPP) -P -traditional-cpp -x assembler-with-cpp -Dprefix=$(prefix) -Dlocalstatedir=$(localstatedir) -Dsysconfdir=$(sysconfdir) -Ddatadir=$(datadir) -Dlibdir=$(libdir) $< > $@
 
 install:	clixon.conf.cpp clixon.mk
-	for i in $(SUBDIRS) doc; \
-		do (cd $$i && $(MAKE) $(MFLAGS) $@); done; \
 	install -d -m 755 $(DESTDIR)$(datadir)/clixon
 	install -m 755 clixon.conf.cpp $(DESTDIR)$(datadir)/clixon
 	install -m 755 clixon.mk $(DESTDIR)$(datadir)/clixon
+	for i in $(SUBDIRS) doc; \
+	do (cd $$i; $(MAKE) $(MFLAGS) $@)||exit 1; done; \
 	echo "Install for compilation by: make install-include"
 
 install-include:
 	for i in $(SUBDIRS) doc; \
-		do (cd $$i && $(MAKE) $(MFLAGS) $@); done; \
+	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done; \
 	echo "To install example app: cd example; make; make install"
 
 uninstall:
 	for i in $(SUBDIRS) doc example docker; \
-	do (cd $$i && $(MAKE) $(MFLAGS) $@); done; 
+	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done; 
-	rm -f $(datadir)/clixon/clixon.conf.cpp
+	rm -f $(DESTDIR)$(datadir)/clixon/clixon.conf.cpp
-	rm -f $(datadir)/clixon/clixon.mk
+	rm -f $(DESTDIR)$(datadir)/clixon/clixon.mk
 
 doc:	
 	cd $@; $(MAKE) $(MFLAGS) $@

apps/Makefile.in

@@ -61,15 +61,15 @@ $(SUBDIRS):
 
 install-include:
 	for i in $(SUBDIRS); \
-	do (cd $$i ; $(MAKE) $(MFLAGS) $@); done; 
+	do (cd $$i && $(MAKE) $(MFLAGS) $@||exit 1); done; 
 
 install:
 	for i in $(SUBDIRS); \
-	do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done
 
 uninstall:
 	for i in $(SUBDIRS); \
-	do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i; $(MAKE) $(MFLAGS) $@)||exit 1; done
 
 clean:
 	for i in $(SUBDIRS); \

apps/backend/Makefile.in

@@ -107,9 +107,9 @@ install-lib:	$(MYLIB)
 	install -d $(DESTDIR)$(libdir)/clixon/plugins/backend
 
 uninstall:
-	rm -f $(sbindir)/$(APPL)
+	rm -f $(DESTDIR)$(sbindir)/$(APPL)
-	rm -f $(libdir)/$(MYLIB)
+	rm -f $(DESTDIR)$(libdir)/$(MYLIBLINK)*
-	rm -f $(includedir)/clixon/*
+	rm -f $(DESTDIR)$(includedir)/clixon/*
 
 install-include: clixon_backend.h clixon_backend_handle.h clixon_backend_transaction.h
 	install -d $(DESTDIR)$(includedir)/clixon

apps/backend/backend_client.c

@@ -275,6 +275,7 @@ from_client_get(clicon_handle h,
     cxobj *xfilter;
     char  *selector = "/";
     cxobj *xret = NULL;
+    int    ret;
     
     if ((xfilter = xml_find(xe, "filter")) != NULL)
 	if ((selector = xml_find_value(xfilter, "select"))==NULL)
@@ -291,18 +292,30 @@ from_client_get(clicon_handle h,
     }
     /* Get state data from plugins as defined by plugin_statedata(), if any */
     assert(xret);
-    if (backend_statedata_call(h, selector, xret) < 0)
+    clicon_err_reset();
+    if ((ret = backend_statedata_call(h, selector, xret)) < 0)
 	goto done;
-    cprintf(cbret, "<rpc-reply>");
+    if (ret == 0){ /* OK */
-    if (xret==NULL)
+	cprintf(cbret, "<rpc-reply>");
-	cprintf(cbret, "<data/>");
+	if (xret==NULL)
-    else{
+	    cprintf(cbret, "<data/>");
-	if (xml_name_set(xret, "data") < 0)
+	else{
-	    goto done;
+	    if (xml_name_set(xret, "data") < 0)
-	if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
+		goto done;
-	    goto done;
+	    if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0)
+		goto done;
+	}
+	cprintf(cbret, "</rpc-reply>");
+    }
+    else { /* 1 Error from callback */
+	cprintf(cbret, "<rpc-reply><rpc-error>"
+		"<error-tag>operation-failed</error-tag>"
+		"<error-type>rpc</error-type>"
+		"<error-severity>error</error-severity>"
+		"<error-message>Internal error:%s</error-message>"
+		"</rpc-error></rpc-reply>", clicon_err_reason);
+	clicon_log(LOG_NOTICE, "%s Error in backend_statedata_call:%s", __FUNCTION__, xml_name(xe));
     }
-    cprintf(cbret, "</rpc-reply>");
  ok:
     retval = 0;
  done:

apps/backend/backend_main.c

@@ -73,7 +73,7 @@
 #include "backend_handle.h"
 
 /* Command line options to be passed to getopt(3) */
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
 #define BACKEND_OPTS "hD:f:d:b:Fzu:P:1s:c:IRCrg:py:x:" /* substitute s: for IRCc:r */
 #else
 #define BACKEND_OPTS "hD:f:d:b:Fzu:P:1s:c:g:py:x:" /* substitute s: for IRCc:r */
@@ -142,12 +142,12 @@ usage(char *argv0, clicon_handle h)
     	    "    -P <file>\tPid filename (default: %s)\n"
 	    "    -s <mode>\tSpecify backend startup mode: none|startup|running|init (replaces -IRCr\n"
 	    "    -c <file>\tLoad extra xml configuration, but don't commit.\n"
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
     	    "    -I\t\tInitialize running state database\n"
     	    "    -R\t\tCall plugin_reset() in plugins to reset system state in running db (use with -I)\n"
 	    "    -C\t\tCall plugin_reset() in plugins to reset system state in candidate db (use with -I)\n"
 	    "    -r\t\tReload running database\n"
-#endif /* BACKEND_STARTUP_BACKWARD_COMPAT */
+#endif /* BACKEND_STARTUP_COMPAT */
 	    "    -p \t\tPrint database yang specification\n"
 	    "    -g <group>\tClient membership required to this group (default: %s)\n"
 	    "    -y <file>\tOverride yang spec file (dont include .yang suffix)\n"
@@ -271,7 +271,7 @@ plugin_start_useroptions(clicon_handle h,
     return 0;
 }
 
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
 /*! Initialize running-config from file application configuration
  *
  * @param[in] h                clicon handle
@@ -424,7 +424,7 @@ fragmented_startup_mode(clicon_handle h,
  done:
     return retval;
 }
-#endif /* BACKEND_STARTUP_BACKWARD_COMPAT */
+#endif /* BACKEND_STARTUP_COMPAT */
 
 /*! Merge xml in filename into database
  */
@@ -523,6 +523,9 @@ startup_mode_running(clicon_handle h,
     /* Load plugins and call plugin_init() */
     if (plugin_initiate(h) != 0) 
 	goto done;
+    /* Clear running db */
+    if (db_reset(h, "running") < 0)
+	goto done;
     /* Clear tmp db */
     if (db_reset(h, "tmp") < 0)
 	goto done;
@@ -565,6 +568,9 @@ startup_mode_startup(clicon_handle h,
     /* Load plugins and call plugin_init() */
     if (plugin_initiate(h) != 0) 
 	goto done;
+    /* Clear running db */
+    if (db_reset(h, "running") < 0)
+	goto done;
     /* Clear tmp db */
     if (db_reset(h, "tmp") < 0)
 	goto done;
@@ -593,7 +599,7 @@ main(int argc, char **argv)
     int           foreground;
     int           once;
     enum startup_mode_t startup_mode;
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
     int           init_rundb = 0;
     int           reset_state_running = 0;
     int           reset_state_candidate = 0;
@@ -712,7 +718,7 @@ main(int argc, char **argv)
 	case 'c': /* Load application config */
 	    extraxml_file = optarg;
 	    break;
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
 	case 'I': /* Initiate running db */
 	    init_rundb++;
 	    break;
@@ -725,7 +731,7 @@ main(int argc, char **argv)
 	case 'r': /* Reload running */
 	    reload_running++;
 	    break;
-#endif /* BACKEND_STARTUP_BACKWARD_COMPAT */
+#endif /* BACKEND_STARTUP_COMPAT */
 	case 'g': /* config socket group */
 	    clicon_option_str_set(h, "CLICON_SOCK_GROUP", optarg);
 	    break;
@@ -818,7 +824,7 @@ main(int argc, char **argv)
     if (yang_spec_main(h, stdout, printspec) < 0)
 	goto done;
 
-    /* Set options: database dir aqnd yangspec (could be hidden in connect?)*/
+    /* Set options: database dir and yangspec (could be hidden in connect?)*/
     if (xmldb_setopt(h, "dbdir", clicon_xmldb_dir(h)) < 0)
 	goto done;
     if (xmldb_setopt(h, "yangspec", clicon_dbspec_yang(h)) < 0)
@@ -826,7 +832,7 @@ main(int argc, char **argv)
     /* If startup mode is not defined, eg via OPTION or -s, assume old method */
     startup_mode = clicon_startup_mode(h);
     if (startup_mode == -1){ 	/* Old style, fragmented mode, phase out */
-#ifdef BACKEND_STARTUP_BACKWARD_COMPAT
+#ifdef BACKEND_STARTUP_COMPAT
 	if (fragmented_startup_mode(h,
 				    argv0, argc, argv,
 				    reload_running, init_rundb,

apps/backend/backend_plugin.c

@@ -726,6 +726,7 @@ plugin_transaction_abort(clicon_handle       h,
  * @param[in,out] xml  XML tree.
  * @retval -1   Error
  * @retval  0   OK
+ * @retval  1   Statedata callback failed
  */
 int
 backend_statedata_call(clicon_handle        h,
@@ -753,8 +754,10 @@ backend_statedata_call(clicon_handle        h,
 	if (p->p_statedata) {
 	    if ((x = xml_new("config", NULL)) == NULL)
 		goto done;
-	    if ((p->p_statedata)(h, xpath, x) < 0)
+	    if ((p->p_statedata)(h, xpath, x) < 0){
-		goto done;
+		retval = 1;
+		goto done; /* Dont quit here on user callbacks */
+	    }
 	    if (xml_merge(xtop, x, yspec) < 0)
 		goto done;
 	    if (x){
@@ -763,27 +766,25 @@ backend_statedata_call(clicon_handle        h,
 	    }
 	}
     }
-    {
+    /* Code complex to filter out anything that is outside of xpath */
-	/* Code complex to filter out anything that is outside of xpath */
+    if (xpath_vec(xtop, xpath?xpath:"/", &xvec, &xlen) < 0)
-	if (xpath_vec(xtop, xpath?xpath:"/", &xvec, &xlen) < 0)
+	goto done;
-	    goto done;
 
-	/* If vectors are specified then mark the nodes found and
+    /* If vectors are specified then mark the nodes found and
-	 * then filter out everything else,
+     * then filter out everything else,
-	 * otherwise return complete tree.
+     * otherwise return complete tree.
-	 */
+     */
-	if (xvec != NULL){
+    if (xvec != NULL){
-	    for (i=0; i<xlen; i++)
+	for (i=0; i<xlen; i++)
-		xml_flag_set(xvec[i], XML_FLAG_MARK);
+	    xml_flag_set(xvec[i], XML_FLAG_MARK);
-	}
-	/* Remove everything that is not marked */
-	if (!xml_flag(xtop, XML_FLAG_MARK))
-	    if (xml_tree_prune_flagged_sub(xtop, XML_FLAG_MARK, 1, NULL) < 0)
-		goto done;
-	/* reset flag */
-	if (xml_apply(xtop, CX_ELMNT, (xml_applyfn_t*)xml_flag_reset, (void*)XML_FLAG_MARK) < 0)
-	    goto done;
     }
+    /* Remove everything that is not marked */
+    if (!xml_flag(xtop, XML_FLAG_MARK))
+	if (xml_tree_prune_flagged_sub(xtop, XML_FLAG_MARK, 1, NULL) < 0)
+	    goto done;
+    /* reset flag */
+    if (xml_apply(xtop, CX_ELMNT, (xml_applyfn_t*)xml_flag_reset, (void*)XML_FLAG_MARK) < 0)
+	goto done;
     retval = 0;
  done:
     if (x)

apps/cli/Makefile.in

@@ -108,9 +108,9 @@ install-include:	clixon_cli.h clixon_cli_api.h
 	install -m 644 $^ $(DESTDIR)$(includedir)/clixon
 
 uninstall:
-	rm -f $(bindir)/$(APPL)
+	rm -f $(DESTDIR)$(bindir)/$(APPL)
-	rm -f $(libdir)/$(MYLIB)
+	rm -f $(DESTDIR)$(libdir)/$(MYLIBLINK)*
-	rm -f $(includedir)/clixon/*
+	rm -f $(DESTDIR)$(includedir)/clixon/*
 
 .SUFFIXES:
 .SUFFIXES: .c .o

apps/netconf/Makefile.in

@@ -103,9 +103,9 @@ install-include:	clixon_netconf.h
 	install -m 644 $^ $(DESTDIR)$(includedir)/clixon
 
 uninstall:
-	rm -f $(bindir)/$(APPL)
+	rm -f $(DESTDIR)$(bindir)/$(APPL)
-	rm -f $(libdir)/$(MYLIB)
+	rm -f $(DESTDIR)$(libdir)/$(MYLIBLINK)*
-	rm -f $(includedir)/clixon/*
+	rm -f $(DESTDIR)$(includedir)/clixon/*
 
 .SUFFIXES:
 .SUFFIXES: .c .o

apps/restconf/Makefile.in

@@ -89,7 +89,7 @@ install:	$(APPL)
 install-include:	
 
 uninstall:
-	rm -f $(wwwdir)/$(APPL)
+	rm -f $(DESTDIR)$(wwwdir)/$(APPL)
 
 .SUFFIXES:
 .SUFFIXES: .c .o

configure

@@ -632,6 +632,8 @@ CPP
 OBJEXT
 EXEEXT
 ac_ct_CC
+with_config_compat
+with_startup_compat
 with_keyvalue
 with_restconf
 RANLIB
@@ -708,6 +710,8 @@ with_cligen
 with_restconf
 with_keyvalue
 with_qdbm
+with_startup_compat
+with_config_compat
 '
       ac_precious_vars='build_alias
 host_alias
@@ -1346,6 +1350,8 @@ Optional Packages:
   --without-restconf      disable support for restconf
   --with-keyvalue         enable support for key-value xmldb datastore
   --with-qdbm=dir         Use QDBM here, if keyvalue
+  --with-startup-compat   Backward compatibility of backend startup commands
+  --with-config-compat    Backward compatibility of ocnfiguration file
 
 Some influential environment variables:
   CC          C compiler command
@@ -2185,8 +2191,8 @@ _ACEOF
 
  # Bind to specific CLIgen version
 
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: CLIXON version is ${CLIXON_VERSION}_PRE" >&5
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: CLIXON version is ${CLIXON_VERSION}" >&5
-$as_echo "CLIXON version is ${CLIXON_VERSION}_PRE" >&6; }
+$as_echo "CLIXON version is ${CLIXON_VERSION}" >&6; }
 
 ac_aux_dir=
 for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do
@@ -2341,6 +2347,8 @@ test -n "$target_alias" &&
 
  # If yes, compile apps/restconf
  # If yes, compile datastore/keyvalue
+ # If yes, backward compatible backend startup
+ # If yes, backward compatible .conf configuration
 
 #
 ac_ext=c
@@ -3946,9 +3954,7 @@ else
   with_keyvalue=no
 fi
 
-echo "keyvalue:${with_keyvalue}"
 if test "x${with_keyvalue}" == xyes; then
-   echo "yes keyvalue"
    # This is for qdbm
 
 # Check whether --with-qdbm was given.
@@ -4038,6 +4044,40 @@ fi
 
 fi
 
+# This is for backward compatibility of backend startup commands
+
+# Check whether --with-startup_compat was given.
+if test "${with_startup_compat+set}" = set; then :
+  withval=$with_startup_compat;
+else
+  with_startup_compat=no
+fi
+
+if test "x${with_startup_compat}" == xyes; then
+
+cat >>confdefs.h <<_ACEOF
+#define BACKEND_STARTUP_COMPAT $with_startup_compat
+_ACEOF
+
+fi
+
+# This is for backward compatibility of .conf configuration file
+
+# Check whether --with-config_compat was given.
+if test "${with_config_compat+set}" = set; then :
+  withval=$with_config_compat;
+else
+  with_config_compat=no
+fi
+
+if test "x${with_config_compat}" == xyes; then
+
+cat >>confdefs.h <<_ACEOF
+#define CONFIG_COMPAT $with_config_compat
+_ACEOF
+
+fi
+
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for crypt in -lcrypt" >&5
 $as_echo_n "checking for crypt in -lcrypt... " >&6; }
 if ${ac_cv_lib_crypt_crypt+:} false; then :

configure.ac

@@ -63,7 +63,7 @@ AC_SUBST(CLIXON_VERSION_MAJOR)
 AC_SUBST(CLIXON_VERSION_MINOR)
 AC_SUBST(CLIGEN_VERSION) # Bind to specific CLIgen version
 
-AC_MSG_RESULT(CLIXON version is ${CLIXON_VERSION}_PRE)
+AC_MSG_RESULT(CLIXON version is ${CLIXON_VERSION})
 
 AC_CANONICAL_TARGET
 AC_SUBST(CC)
@@ -80,6 +80,8 @@ AC_SUBST(AR)
 AC_SUBST(RANLIB)
 AC_SUBST(with_restconf) # If yes, compile apps/restconf
 AC_SUBST(with_keyvalue) # If yes, compile datastore/keyvalue
+AC_SUBST(with_startup_compat) # If yes, backward compatible backend startup
+AC_SUBST(with_config_compat) # If yes, backward compatible .conf configuration
 
 #
 AC_PROG_CC()
@@ -141,9 +143,7 @@ AC_ARG_WITH([keyvalue],
             [AS_HELP_STRING([--with-keyvalue],[enable support for key-value xmldb datastore])],
 	    [],
             [with_keyvalue=no])
-echo "keyvalue:${with_keyvalue}"
 if test "x${with_keyvalue}" == xyes; then
-   echo "yes keyvalue" 
    # This is for qdbm
    AC_ARG_WITH(qdbm,  [  --with-qdbm=dir         Use QDBM here, if keyvalue ] )
    if test "${with_qdbm}"; then
@@ -156,6 +156,24 @@ if test "x${with_keyvalue}" == xyes; then
    AC_CHECK_LIB(qdbm, dpopen,, AC_MSG_ERROR(libqdbm-dev required))
 fi
 
+# This is for backward compatibility of backend startup commands
+AC_ARG_WITH([startup_compat],
+            [AS_HELP_STRING([--with-startup-compat],[Backward compatibility of backend startup commands])],
+	    [],
+            [with_startup_compat=no])
+if test "x${with_startup_compat}" == xyes; then
+   AC_DEFINE_UNQUOTED(BACKEND_STARTUP_COMPAT, $with_startup_compat, [Backward compatible backend startup command-line options])
+fi
+
+# This is for backward compatibility of .conf configuration file
+AC_ARG_WITH([config_compat],
+            [AS_HELP_STRING([--with-config-compat],[Backward compatibility of ocnfiguration file])],
+	    [],
+            [with_config_compat=no])
+if test "x${with_config_compat}" == xyes; then
+   AC_DEFINE_UNQUOTED(CONFIG_COMPAT, $with_config_compat, [Backward compatible of .conf configuration files])
+fi
+
 AC_CHECK_LIB(crypt, crypt)
 AC_CHECK_HEADERS(crypt.h)
 

datastore/Makefile.in

@@ -101,18 +101,18 @@ $(SUBDIRS):
 
 install-include:
 	for i in $(SUBDIRS); \
-	do (cd $$i ; $(MAKE) $(MFLAGS) $@); done; 
+	do (cd $$i ; $(MAKE) $(MFLAGS) $@)||exit 1; done; 
 
 install: $(APPL)
 	install -d $(DESTDIR)$(bindir)
 	install $(APPL) $(DESTDIR)$(bindir)
 	for i in $(SUBDIRS); \
-	do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done
 
 uninstall:
-	rm -f $(bindir)/$(APPL)
+	rm -f $(DESTDIR)$(bindir)/$(APPL)
 	for i in $(SUBDIRS); \
-	do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done
 
 clean:
 	rm -f *.core $(APPL) $(APPOBJ)

etc/Makefile.in

@@ -52,8 +52,10 @@ install:	 clixonrc
 	install -m 755 clixonrc $(DESTDIR)$(sysconfdir)
 
 install-include:	
+	rm -f $(DESTDIR)$(sysconfdir)/clixonrc
 
 uninstall:
 
+
 depend:
 

include/clixon_config.h.in

@@ -1,5 +1,8 @@
 /* include/clixon_config.h.in.  Generated from configure.ac by autoheader.  */
 
+/* Backward compatible backend startup command-line options */
+#undef BACKEND_STARTUP_COMPAT
+
 /* Clixon data dir for system yang files etc */
 #undef CLIXON_DATADIR
 
@@ -15,6 +18,9 @@
 /* Clixon version string */
 #undef CLIXON_VERSION_STRING
 
+/* Backward compatible of .conf configuration files */
+#undef CONFIG_COMPAT
+
 /* Define to 1 if you have the `alphasort' function. */
 #undef HAVE_ALPHASORT
 
@@ -33,6 +39,9 @@
 /* Define to 1 if you have the <inttypes.h> header file. */
 #undef HAVE_INTTYPES_H
 
+/* Define to 1 if you have the `cligen' library (-lcligen). */
+#undef HAVE_LIBCLIGEN
+
 /* Define to 1 if you have the `crypt' library (-lcrypt). */
 #undef HAVE_LIBCRYPT
 

include/clixon_custom.h

@@ -48,9 +48,3 @@ int strverscmp (__const char *__s1, __const char *__s2);
  */
 #define XML_CHILD_HASH 1
 
-/* Backward compatible clixon backend daemon startup sequences
- * This has been replaced with -s <mode> and CLICON_STARTUP_MODE
- * define if enable backward compatible behaviour
- * Remove in 3.3.4
- */
-#undef BACKEND_STARTUP_BACKWARD_COMPAT

lib/Makefile.in

@@ -55,20 +55,20 @@ $(SUBDIRS):
 		(cd $@; $(MAKE) $(MFLAGS) all)
 
 install:
-		for i in $(SUBDIRS); \
+	for i in $(SUBDIRS); \
-		do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i && $(MAKE) $(MFLAGS) $@); done
 
 install-include:
-		for i in $(SUBDIRS); \
+	for i in $(SUBDIRS); \
-		do (cd $$i; $(MAKE) $(MFLAGS) $@); done; \
+	do (cd $$i; $(MAKE) $(MFLAGS) $@)||exit 1; done; \
-		(cd clixon; $(MAKE) $(MFLAGS) $@)
+	(cd clixon; $(MAKE) $(MFLAGS) $@)
 
 uninstall:
-		for i in $(SUBDIRS); \
+	for i in $(SUBDIRS); \
-		do (cd $$i; $(MAKE) $(MFLAGS) $@); done
+	do (cd $$i; $(MAKE) $(MFLAGS) $@)||exit 1; done
 
 config.status:	configure
-		$(SHELL) config.status --recheck
+	$(SHELL) config.status --recheck
 
 clean:
 	for i in $(SUBDIRS); \

lib/clixon/Makefile.in

@@ -48,7 +48,7 @@ install-include:
 	install -m 644 *.h $(DESTDIR)$(includedir)/clixon 
 
 uninstall:
-	rm -f $(includedir)/clixon
+	rm -rf $(DESTDIR)$(includedir)/clixon
 
 clean:
 

lib/src/Makefile.in

@@ -187,7 +187,7 @@ install-lib: $(MYLIB)
 	ln -sf $(MYLIBSO) $(DESTDIR)$(libdir)/$(MYLIBLINK) # -l:libclixon.so
 
 uninstall: 
-	rm -f $(libdir)/$(MYLIB)
+	rm -f $(DESTDIR)$(libdir)/$(MYLIBLINK)*
 
 TAGS:
 	find . -name '*.[chyl]' -print | etags -

lib/src/clixon_options.c

@@ -185,6 +185,7 @@ clicon_option_readfile_xml(clicon_hash_t *copt,
     return retval;
 }
 
+#ifdef CONFIG_COMPAT
 
 /*! Read filename and set values to global options registry
  * For legacy configuration file, ie not xml
@@ -344,6 +345,7 @@ clicon_option_sanity(clicon_hash_t *copt)
  done:
     return retval;
 }
+#endif /* CONFIG_COMPAT */
 
 /*! Initialize option values
  *
@@ -386,6 +388,7 @@ clicon_options_main(clicon_handle h)
 	    yspec_free(yspec);
     }
     else {
+#ifdef CONFIG_COMPAT
 	/* Set default options */
 	if (clicon_option_default(copt) < 0)  /* init registry from file */
 	    goto done;
@@ -394,6 +397,10 @@ clicon_options_main(clicon_handle h)
 	    goto done;
 	if (clicon_option_sanity(copt) < 0)
 	    goto done;
+#else /* CONFIG_COMPAT */
+	clicon_err(OE_CFG, 0, "%s: suffix %s not recognized (Run ./configure --with-config-compat?)", configfile, suffix);
+	goto done;
+#endif /* CONFIG_COMPAT */
     }
     retval = 0;
  done:

test/lib.sh

@@ -1,13 +1,17 @@
 #!/bin/bash
 
 testnr=0
-testnname=
+testname=
 clixon_cf=/usr/local/etc/routing.xml
 # error and exit, arg is optional extra errmsg
 err(){
   echo "Error in Test$testnr [$testname]:"
-  echo "Expected: $1"
+  if [ $# -gt 0 ]; then 
-  echo "Received: $2"
+      echo "Expected: $1"
+  fi
+  if [ $# -gt 1 ]; then 
+      echo "Received: $2"
+  fi
   exit $testnr
 }
 

test/test_install.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+# Install test
+
+# include err() and new() functions
+. ./lib.sh
+
+DIR=/tmp/clixoninstall
+
+new "Set up installdir $DIR"
+rm -rf $DIR
+mkdir $DIR
+
+new "Make DESTDIR install"
+(cd ..; make DESTDIR=$DIR install)
+if [ $? -ne 0 ]; then
+    err
+fi
+
+new "Check installed files"
+if [ ! -d $DIR/usr ]; then
+    err $DIR/usr
+fi
+if [ ! -d $DIR/www-data ]; then
+    err $DIR/www-data
+fi
+if [ ! -f $DIR/usr/local/share/clixon/clixon.mk ]; then
+    err $DIR/usr/local/share/clixon/clixon.mk
+fi
+if [ ! -f $DIR/usr/local/share/clixon/clixon.conf.cpp ]; then
+    err $DIR/usr/local/share/clixon/clixon.conf.cpp
+fi
+if [ ! -f $DIR/usr/local/share/clixon/clixon-config* ]; then
+    err $DIR/usr/local/share/clixon/clixon-config*
+fi
+if [ ! -h $DIR/usr/local/lib/libclixon.so ]; then
+    err $DIR/usr/local/lib/libclixon.so
+fi
+if [ ! -h $DIR/usr/local/lib/libclixon_backend.so ]; then
+    err $DIR/usr/local/lib/libclixon_backend.so
+fi
+
+new "Make DESTDIR install include"
+(cd ..; make DESTDIR=$DIR install-include)
+if [ $? -ne 0 ]; then
+    err
+fi
+new "Check installed includes"
+if [ ! -f $DIR/usr/local/include/clixon/clixon.h ]; then
+    err $DIR/usr/local/include/clixon/clixon.h
+fi
+new "Make DESTDIR uninstall"
+(cd ..; make DESTDIR=$DIR uninstall)
+if [ $? -ne 0 ]; then
+    err
+fi
+
+new "Check remaining files"
+f=$(find $DIR -type f)
+if [ -n "$f" ]; then
+    err "$f"
+fi
+
+new "Check remaining symlinks"
+l=$(find $DIR -type l)
+if [ -n "$l" ]; then
+    err "$l"
+fi

yang/Makefile.in

@@ -54,7 +54,9 @@ distclean: clean
 	rm -f Makefile *~ .depend
 
 install: $(YANGSPECS) 
-	install -d $(DESTDIR)/yang
+	echo $(DESTDIR)$(datarootdir)/clixon/clixon.mk
+	echo $(DESTDIR)$(clixon_DATADIR)
+	install -d $(DESTDIR)$(clixon_DATADIR)
 	install $(YANGSPECS) $(DESTDIR)$(clixon_DATADIR)
 
 uninstall: