experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'develop'

Browse source
This commit is contained in:
Olof hagsand 2019-03-27 16:32:01 +01:00
commit 4902f7cf1d
28 changed files with 1877 additions and 1002 deletions
Download patch file Download diff file Expand all files Collapse all files

19
CHANGELOG.md
View file

@ -16,23 +16,25 @@
* Enable with CLICON_XMLDB_MODSTATE config option * Enable with CLICON_XMLDB_MODSTATE config option
* Check modules-state tags when loading a datastore at startup * Check modules-state tags when loading a datastore at startup
* Check which modules match, and which do not. * Check which modules match, and which do not.
* Loading of "extra" XML. * Loading of "extra" XML, such as from a file.
* Detection of in-compatible XML and Yang models in the startup configuration. * Detection of in-compatible XML and Yang models in the startup configuration.
* A user can register upgrade callbacks per module/revision when in-compatible XML is encountered (`update_callback_register`). * A user can register upgrade callbacks per module/revision when in-compatible XML is encountered (`update_callback_register`).
* See the [example](example/example_backend.c) and [test](test/test_upgrade_interfaces.sh].
* A "failsafe" mode allowing a user to repair the startup on errors or failed validation. * A "failsafe" mode allowing a user to repair the startup on errors or failed validation.
* Major rewrite of `backend_main.c` and a new module `backend_startup.c` * Major rewrite of `backend_main.c` and a new module `backend_startup.c`
* New yang changelog experimental feature for automatic upgrade
* Yang module clixon-yang-changelog@2019-03-21.yang based on draft-wang-netmod-module-revision-management-01
* Two config options control:
* CLICON_YANG_CHANGELOG enables the yang changelog feature
* CLICON_YANG_CHANGELOG_FILE where the changelog resides
* Datastore files contain RFC7895 module-state information * Datastore files contain RFC7895 module-state information
* Added modules-state diff parameter to xmldb_get datastore function * Added modules-state diff parameter to xmldb_get datastore function
* Set config option `CLICON_XMLDB_MODSTATE` to true * Set config option `CLICON_XMLDB_MODSTATE` to true
* Enable this if you wish to use the upgrade feature in the new startup functionality. * Enable this if you wish to use the upgrade feature in the new startup functionality.
* Note that this adds bytes to your configs * Note that this adds bytes to your configs
* New xml changelog experimental feature for automatic upgrade
* Yang module clixon-xml-changelog@2019-03-21.yang based on draft-wang-netmod-module-revision-management-01
* Two config options control:
* CLICON_XML_CHANGELOG enables the yang changelog feature
* CLICON_XML_CHANGELOG_FILE where the changelog resides
### API changes on existing features (you may need to change your code) ### API changes on existing features (you may need to change your code)
* Renamed `xml_insert` to `xml_wrap_all`.
* Added modules-state diff parameter to xmldb_get datastore function for startup scenarios. Set this to NULL in normal cases. * Added modules-state diff parameter to xmldb_get datastore function for startup scenarios. Set this to NULL in normal cases.
* `rpc_callback_register` added a namespace parameter. Example: * `rpc_callback_register` added a namespace parameter. Example:
``` ```
@ -50,7 +52,10 @@
``` ```
### Minor changes ### Minor changes
* Added synatctic check for yang status: current, deprectated or obsolete. * Added syntactic check for yang status: current, deprecated or obsolete.
* Added `xml_wrap` function that adds an XML node above a node as a wrapper
* also renamed `xml_insert` to `xml_wrap_all`.
* Added `clicon_argv_get()` function to get the user command-line options, ie the args in `-- <args>`. This is an alternative to using them passed to `plugin_start()`.
* Made Makefile concurrent so that it can be compiled with -jN * Made Makefile concurrent so that it can be compiled with -jN
* Added flags to example backend to control its behaviour: * Added flags to example backend to control its behaviour:
* Start with `-- -r` to run the reset plugin * Start with `-- -r` to run the reset plugin

288
apps/backend/backend_commit.c
View file

@ -140,6 +140,194 @@ generic_validate(yang_spec *yspec,
goto done; goto done;
} }
/*! Common startup validation
* Get db, upgrade it w potential transformed XML, populate it w yang spec,
* sort it, validate it by triggering a transaction
* and call application callback validations.
* @param[in] h Clicon handle
* @param[in] db The startup database. The wanted backend state
* @param[out] xtr Transformed XML
* @param[out] cbret CLIgen buffer w error stmt if retval = 0
* @retval -1 Error - or validation failed (but cbret not set)
* @retval 0 Validation failed (with cbret set)
* @retval 1 Validation OK
* @note Need to differentiate between error and validation fail
*
* 1. Parse startup XML (or JSON)
* 2. If syntax failure, call startup-cb(ERROR), copy failsafe db to
* candidate and commit. Done
* 3. Check yang module versions between backend and init config XML. (msdiff)
* 4. Validate startup db. (valid)
* 5. If valid fails, call startup-cb(Invalid, msdiff), keep startup in candidate and commit failsafe db. Done.
* 6. Call startup-cb(OK, msdiff) and commit.
*/
static int
startup_common(clicon_handle h,
char *db,
transaction_data_t *td,
cbuf *cbret)
{
int retval = -1;
yang_spec *yspec;
int ret;
modstate_diff_t *msd = NULL;
cxobj *xt = NULL;
/* If CLICON_XMLDB_MODSTATE is enabled, then get the db XML with
* potentially non-matching module-state in msd
*/
if (clicon_option_bool(h, "CLICON_XMLDB_MODSTATE"))
if ((msd = modstate_diff_new()) == NULL)
goto done;
if (xmldb_get(h, db, "/", 1, &xt, msd) < 0)
goto done;
if (msd){
if ((ret = clixon_module_upgrade(h, xt, msd, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
}
if ((yspec = clicon_dbspec_yang(h)) == NULL){
clicon_err(OE_YANG, 0, "Yang spec not set");
goto done;
}
/* After upgrading, XML tree needs to be sorted and yang spec populated */
if (xml_apply0(xt, CX_ELMNT, xml_spec_populate, yspec) < 0)
goto done;
if (xml_apply0(xt, CX_ELMNT, xml_sort, NULL) < 0)
goto done;
/* Handcraft transition with with only add tree */
td->td_target = xt;
if (cxvec_append(td->td_target, &td->td_avec, &td->td_alen) < 0)
goto done;
/* 4. Call plugin transaction start callbacks */
if (plugin_transaction_begin(h, td) < 0)
goto done;
/* 5. Make generic validation on all new or changed data.
Note this is only call that uses 3-values */
if ((ret = generic_validate(yspec, td, cbret)) < 0)
goto done;
if (ret == 0)
goto fail; /* STARTUP_INVALID */
/* 6. Call plugin transaction validate callbacks */
if (plugin_transaction_validate(h, td) < 0)
goto done;
/* 7. Call plugin transaction complete callbacks */
if (plugin_transaction_complete(h, td) < 0)
goto done;
retval = 1;
done:
if (msd)
modstate_diff_free(msd);
return retval;
fail:
retval = 0;
goto done;
}
/*! Read startup db, check upgrades and validate it, return upgraded XML
*
* @param[in] h Clicon handle
* @param[in] db The startup database. The wanted backend state
* @param[out] xtr (Potentially) transformed XML
* @param[out] cbret CLIgen buffer w error stmt if retval = 0
* @retval -1 Error - or validation failed (but cbret not set)
* @retval 0 Validation failed (with cbret set)
* @retval 1 Validation OK
*/
int
startup_validate(clicon_handle h,
char *db,
cxobj **xtr,
cbuf *cbret)
{
int retval = -1;
int ret;
transaction_data_t *td = NULL;
/* Handcraft a transition with only target and add trees */
if ((td = transaction_new()) == NULL)
goto done;
if ((ret = startup_common(h, db, td, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
if (xtr){
*xtr = td->td_target;
td->td_target = NULL;
}
retval = 1;
done:
if (td)
transaction_free(td);
return retval;
fail: /* cbret should be set */
retval = 0;
goto done;
}
/*! Read startup db, check upgrades and commit it
*
* @param[in] h Clicon handle
* @param[in] db The startup database. The wanted backend state
* @param[out] cbret CLIgen buffer w error stmt if retval = 0
* @retval -1 Error - or validation failed (but cbret not set)
* @retval 0 Validation failed (with cbret set)
* @retval 1 Validation OK
*/
int
startup_commit(clicon_handle h,
char *db,
cbuf *cbret)
{
int retval = -1;
int ret;
transaction_data_t *td = NULL;
/* Handcraft a transition with only target and add trees */
if ((td = transaction_new()) == NULL)
goto done;
if ((ret = startup_common(h, db, td, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
/* 8. Call plugin transaction commit callbacks */
if (plugin_transaction_commit(h, td) < 0)
goto done;
/* 9, write (potentially modified) tree to running
* XXX note here startup is copied to candidate, which may confuse everything
*/
if ((ret = xmldb_put(h, "running", OP_REPLACE, td->td_target,
clicon_username_get(h), cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
/* 10. Call plugin transaction end callbacks */
plugin_transaction_end(h, td);
/* 11. Copy running back to candidate in case end functions updated running
* XXX: room for improvement: candidate and running may be equal.
* Copy only diffs?
*/
if (xmldb_copy(h, "running", "candidate") < 0){
/* ignore errors or signal major setback ? */
clicon_log(LOG_NOTICE, "Error in rollback, trying to continue");
goto done;
}
retval = 1;
done:
if (td)
transaction_free(td);
return retval;
fail: /* cbret should be set */
retval = 0;
goto done;
}
/*! Validate a candidate db and comnpare to running /*! Validate a candidate db and comnpare to running
* Get both source and dest datastore, validate target, compute diffs * Get both source and dest datastore, validate target, compute diffs
* and call application callback validations. * and call application callback validations.
@ -152,10 +340,10 @@ generic_validate(yang_spec *yspec,
* (only done for generic_validate) * (only done for generic_validate)
*/ */
static int static int
validate_common(clicon_handle h, from_validate_common(clicon_handle h,
char *candidate, char *candidate,
transaction_data_t *td, transaction_data_t *td,
cbuf *cbret) cbuf *cbret)
{ {
int retval = -1; int retval = -1;
yang_spec *yspec; yang_spec *yspec;
@ -248,94 +436,6 @@ validate_common(clicon_handle h,
goto done; goto done;
} }
/*! Validate a candidate db and comnpare to running XXX Experimental
* Get both source and dest datastore, validate target, compute diffs
* and call application callback validations.
* @param[in] h Clicon handle
* @param[in] candidate The candidate database. The wanted backend state
* @retval -1 Error - or validation failed (but cbret not set)
* @retval 0 Validation failed (with cbret set)
* @retval 1 Validation OK
* @note Need to differentiate between error and validation fail
* 1. Parse startup XML (or JSON)
* 2. If syntax failure, call startup-cb(ERROR), copy failsafe db to
* candidate and commit. Done
* 3. Check yang module versions between backend and init config XML. (msdiff)
* 4. Validate startup db. (valid)
* 5. If valid fails, call startup-cb(Invalid, msdiff), keep startup in candidate and commit failsafe db. Done.
* 6. Call startup-cb(OK, msdiff) and commit.
*/
int
startup_validate(clicon_handle h,
char *db,
cbuf *cbret)
{
int retval = -1;
yang_spec *yspec;
int ret;
modstate_diff_t *msd = NULL;
transaction_data_t *td = NULL;
/* Handcraft a transition with only target and add trees */
if ((td = transaction_new()) == NULL)
goto done;
/* 2. Parse xml trees
* This is the state we are going to
* Note: xmsdiff contains non-matching modules
* Only if CLICON_XMLDB_MODSTATE is enabled
*/
if (clicon_option_bool(h, "CLICON_XMLDB_MODSTATE"))
if ((msd = modstate_diff_new()) == NULL)
goto done;
if (xmldb_get(h, db, "/", 1, &td->td_target, msd) < 0)
goto done;
if ((ret = clixon_module_upgrade(h, td->td_target, msd, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
/* Handcraft transition with with only add tree */
if (cxvec_append(td->td_target, &td->td_avec, &td->td_alen) < 0)
goto done;
/* 4. Call plugin transaction start callbacks */
if (plugin_transaction_begin(h, td) < 0)
goto done;
if ((yspec = clicon_dbspec_yang(h)) == NULL){
clicon_err(OE_YANG, 0, "Yang spec not set");
goto done;
}
/* 5. Make generic validation on all new or changed data.
Note this is only call that uses 3-values */
if ((ret = generic_validate(yspec, td, cbret)) < 0)
goto done;
if (ret == 0)
goto fail; /* STARTUP_INVALID */
/* 6. Call plugin transaction validate callbacks */
if (plugin_transaction_validate(h, td) < 0)
goto done;
/* 7. Call plugin transaction complete callbacks */
if (plugin_transaction_complete(h, td) < 0)
goto done;
retval = 1;
done:
if (td)
transaction_free(td);
if (msd)
modstate_diff_free(msd);
return retval;
fail: /* cbret should be set */
if (cbuf_len(cbret)==0){
clicon_err(OE_CFG, EINVAL, "Validation fail but cbret not set");
goto done;
}
retval = 0;
goto done;
}
/*! Do a diff between candidate and running, then start a commit transaction /*! Do a diff between candidate and running, then start a commit transaction
* *
* The code reverts changes if the commit fails. But if the revert * The code reverts changes if the commit fails. But if the revert
@ -365,7 +465,7 @@ candidate_commit(clicon_handle h,
/* Common steps (with validate). Load candidate and running and compute diffs /* Common steps (with validate). Load candidate and running and compute diffs
* Note this is only call that uses 3-values * Note this is only call that uses 3-values
*/ */
if ((ret = validate_common(h, candidate, td, cbret)) < 0) if ((ret = from_validate_common(h, candidate, td, cbret)) < 0)
goto done; goto done;
if (ret == 0) if (ret == 0)
goto fail; goto fail;
@ -580,7 +680,7 @@ from_client_validate(clicon_handle h,
if ((td = transaction_new()) == NULL) if ((td = transaction_new()) == NULL)
goto done; goto done;
/* Common steps (with commit) */ /* Common steps (with commit) */
if ((ret = validate_common(h, db, td, cbret)) < 1){ if ((ret = from_validate_common(h, db, td, cbret)) < 1){
clicon_debug(1, "Validate %s failed", db); clicon_debug(1, "Validate %s failed", db);
if (ret < 0){ if (ret < 0){
if (netconf_operation_failed(cbret, "application", clicon_err_reason)< 0) if (netconf_operation_failed(cbret, "application", clicon_err_reason)< 0)

3
apps/backend/backend_commit.h
View file

@ -40,7 +40,8 @@
/* /*
* Prototypes * Prototypes
*/ */
int startup_validate(clicon_handle h, char *db, cbuf *cbret); int startup_validate(clicon_handle h, char *db, cxobj **xtr, cbuf *cbret);
int startup_commit(clicon_handle h, char *db, cbuf *cbret);
int candidate_commit(clicon_handle h, char *db, cbuf *cbret); int candidate_commit(clicon_handle h, char *db, cbuf *cbret);
int from_client_commit(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg, void *regarg); int from_client_commit(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg, void *regarg);

9
apps/backend/backend_main.c
View file

@ -98,7 +98,7 @@ backend_terminate(clicon_handle h)
close(ss); close(ss);
if ((x = clicon_module_state_get(h)) != NULL) if ((x = clicon_module_state_get(h)) != NULL)
xml_free(x); xml_free(x);
if ((x = clicon_yang_changelog_get(h)) != NULL) if ((x = clicon_xml_changelog_get(h)) != NULL)
xml_free(x); xml_free(x);
if ((yspec = clicon_dbspec_yang(h)) != NULL) if ((yspec = clicon_dbspec_yang(h)) != NULL)
yspec_free(yspec); yspec_free(yspec);
@ -509,6 +509,7 @@ main(int argc,
argc -= optind; argc -= optind;
argv += optind; argv += optind;
clicon_argv_set(h, argv0, argc, argv);
clicon_log_init(__PROGRAM__, debug?LOG_DEBUG:LOG_INFO, logdst); clicon_log_init(__PROGRAM__, debug?LOG_DEBUG:LOG_INFO, logdst);
/* Defer: Wait to the last minute to print help message */ /* Defer: Wait to the last minute to print help message */
@ -645,8 +646,8 @@ main(int argc,
goto done; goto done;
/* Must be after netconf_module_load, but before startup code */ /* Must be after netconf_module_load, but before startup code */
if (clicon_option_bool(h, "CLICON_YANG_CHANGELOG")) if (clicon_option_bool(h, "CLICON_XML_CHANGELOG"))
if (clixon_yang_changelog_init(h) < 0) if (clixon_xml_changelog_init(h) < 0)
goto done; goto done;
/* Save modules state of the backend (server). Compare with startup XML */ /* Save modules state of the backend (server). Compare with startup XML */
@ -705,6 +706,8 @@ main(int argc,
} }
if (status != STARTUP_OK){ if (status != STARTUP_OK){
if (cbuf_len(cbret))
clicon_log(LOG_NOTICE, "%s: %u %s", __PROGRAM__, getpid(), cbuf_get(cbret));
if (startup_failsafe(h) < 0){ if (startup_failsafe(h) < 0){
goto done; goto done;
} }

25
apps/backend/backend_startup.c
View file

@ -161,15 +161,10 @@ startup_mode_startup(clicon_handle h,
if (xmldb_create(h, db) < 0) /* diff */ if (xmldb_create(h, db) < 0) /* diff */
return -1; return -1;
} }
if ((ret = startup_validate(h, db, cbret)) < 0) if ((ret = startup_commit(h, db, cbret)) < 0)
goto done; goto done;
if (ret == 0) if (ret == 0)
goto fail; goto fail;
/* Commit startup */
if (candidate_commit(h, db, cbret) < 1) /* diff */
goto fail;
if (ret == 0) /* shouldnt happen (we already validate) */
goto fail;
retval = 1; retval = 1;
done: done:
return retval; return retval;
@ -241,6 +236,7 @@ startup_extraxml(clicon_handle h,
int retval = -1; int retval = -1;
char *db = "tmp"; char *db = "tmp";
int ret; int ret;
cxobj *xt = NULL; /* Potentially upgraded XML */
/* Clear tmp db */ /* Clear tmp db */
if (startup_db_reset(h, db) < 0) if (startup_db_reset(h, db) < 0)
@ -256,11 +252,17 @@ startup_extraxml(clicon_handle h,
if (ret == 0) if (ret == 0)
goto fail; goto fail;
} }
/* Validate tmp (unless empty?) */ /* Validate the tmp db and return possibly upgraded xml in xt
if ((ret = startup_validate(h, db, cbret)) < 0) */
if ((ret = startup_validate(h, db, &xt, cbret)) < 0)
goto done; goto done;
if (ret == 0) if (ret == 0)
goto fail; goto fail;
/* Write (potentially modified) xml tree xt back to tmp
*/
if ((ret = xmldb_put(h, "tmp", OP_REPLACE, xt,
clicon_username_get(h), cbret)) < 0)
goto done;
/* Merge tmp into running (no commit) */ /* Merge tmp into running (no commit) */
if ((ret = db_merge(h, db, "running", cbret)) < 0) if ((ret = db_merge(h, db, "running", cbret)) < 0)
goto fail; goto fail;
@ -268,6 +270,8 @@ startup_extraxml(clicon_handle h,
goto fail; goto fail;
retval = 1; retval = 1;
done: done:
if (xt)
xml_free(xt);
if (xmldb_delete(h, "tmp") != 0 && errno != ENOENT) if (xmldb_delete(h, "tmp") != 0 && errno != ENOENT)
return -1; return -1;
return retval; return retval;
@ -300,15 +304,16 @@ startup_failsafe(clicon_handle h)
if ((ret = xmldb_exists(h, db)) < 0) if ((ret = xmldb_exists(h, db)) < 0)
goto done; goto done;
if (ret == 0){ /* No it does not exist, fail */ if (ret == 0){ /* No it does not exist, fail */
clicon_err(OE_DB, 0, "No failsafe database"); clicon_err(OE_DB, 0, "Startup failed and no Failsafe database found, exiting");
goto done; goto done;
} }
if ((ret = candidate_commit(h, db, cbret)) < 0) /* diff */ if ((ret = candidate_commit(h, db, cbret)) < 0) /* diff */
goto done; goto done;
if (ret == 0){ if (ret == 0){
clicon_err(OE_DB, 0, "Failsafe database validation failed %s", cbuf_get(cbret)); clicon_err(OE_DB, 0, "Startup failed, Failsafe database validation failed %s", cbuf_get(cbret));
goto done; goto done;
} }
clicon_log(LOG_NOTICE, "Startup failed, Failsafe database loaded ");
retval = 0; retval = 0;
done: done:
if (cbret) if (cbret)

260
example/example_backend.c
View file

@ -34,6 +34,8 @@
*/ */
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <stdint.h>
#include <inttypes.h>
#include <string.h> #include <string.h>
#include <errno.h> #include <errno.h>
#include <signal.h> #include <signal.h>
@ -64,6 +66,12 @@ static int _reset = 0;
*/ */
static int _state = 0; static int _state = 0;
/*! Variable to control upgrade callbacks.
* If set, call test-case for upgrading ietf-interfaces, otherwise call
* auto-upgrade
*/
static int _upgrade = 0;
/* forward */ /* forward */
static int example_stream_timer_setup(clicon_handle h); static int example_stream_timer_setup(clicon_handle h);
@ -246,12 +254,10 @@ example_statedata(clicon_handle h,
return retval; return retval;
} }
#ifdef keep_as_example /*! Testcase upgrade function moving interfaces-state to interfaces
/*! Registered Upgrade callback function
* @param[in] h Clicon handle * @param[in] h Clicon handle
* @param[in] xn XML tree to be updated * @param[in] xn XML tree to be updated
* @param[in] modname Name of module * @param[in] ns Namespace of module (for info)
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD * @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system) * @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register() * @param[in] arg User argument given at rpc_callback_register()
@ -259,22 +265,182 @@ example_statedata(clicon_handle h,
* @retval 1 OK * @retval 1 OK
* @retval 0 Invalid * @retval 0 Invalid
* @retval -1 Error * @retval -1 Error
* @see clicon_upgrade_cb
* @see test_upgrade_interfaces.sh
* @see upgrade_interfaces_2016
* This example shows a two-step upgrade where the 2014 function does:
* - Move /if:interfaces-state/if:interface/if:admin-status to
* /if:interfaces/if:interface/
* - Move /if:interfaces-state/if:interface/if:statistics to
* /if:interfaces/if:interface/
* - Rename /interfaces/interface/description to descr
*/ */
static int static int
upgrade_all(clicon_handle h, upgrade_interfaces_2014(clicon_handle h,
cxobj *xn, cxobj *xt,
char *modname, char *ns,
char *modns, uint32_t from,
uint32_t from, uint32_t to,
uint32_t to, void *arg,
void *arg, cbuf *cbret)
cbuf *cbret)
{ {
fprintf(stderr, "%s XML:%s mod:%s %s from:%d to:%d\n", __FUNCTION__, xml_name(xn), int retval = -1;
modname, modns, from, to); yang_spec *yspec;
return 1; yang_stmt *ym;
cxobj **vec = NULL;
cxobj *xc;
cxobj *xi; /* xml /interfaces-states/interface node */
cxobj *x;
cxobj *xif; /* xml /interfaces/interface node */
size_t vlen;
int i;
char *name;
/* Get Yang module for this namespace. Note it may not exist (if obsolete) */
yspec = clicon_dbspec_yang(h);
if ((ym = yang_find_module_by_namespace(yspec, ns)) == NULL)
goto ok; /* shouldnt happen */
clicon_debug(1, "%s module %s", __FUNCTION__, ym?ym->ys_argument:"none");
/* Get all XML nodes with that namespace */
if (xml_namespace_vec(h, xt, ns, &vec, &vlen) < 0)
goto done;
for (i=0; i<vlen; i++){
xc = vec[i];
/* Iterate through interfaces-state */
if (strcmp(xml_name(xc),"interfaces-state") == 0){
/* Note you cannot delete or move xml objects directly under xc
* in the loop (eg xi objects) but you CAN move children of xi
*/
xi = NULL;
while ((xi = xml_child_each(xc, xi, CX_ELMNT)) != NULL) {
if (strcmp(xml_name(xi), "interface"))
continue;
if ((name = xml_find_body(xi, "name")) == NULL)
continue; /* shouldnt happen */
/* Get corresponding /interfaces/interface entry */
xif = xpath_first(xt, "/interfaces/interface[name=\"%s\"]", name);
/* - Move /if:interfaces-state/if:interface/if:admin-status to
* /if:interfaces/if:interface/ */
if ((x = xml_find(xi, "admin-status")) != NULL && xif){
if (xml_addsub(xif, x) < 0)
goto done;
}
/* - Move /if:interfaces-state/if:interface/if:statistics to
* /if:interfaces/if:interface/*/
if ((x = xml_find(xi, "statistics")) != NULL){
if (xml_addsub(xif, x) < 0)
goto done;
}
}
}
else if (strcmp(xml_name(xc),"interfaces") == 0){
/* Iterate through interfaces */
xi = NULL;
while ((xi = xml_child_each(xc, xi, CX_ELMNT)) != NULL) {
if (strcmp(xml_name(xi), "interface"))
continue;
/* Rename /interfaces/interface/description to descr */
if ((x = xml_find(xi, "description")) != NULL)
if (xml_name_set(x, "descr") < 0)
goto done;
}
}
}
ok:
retval = 1;
done:
if (vec)
free(vec);
return retval;
}
/*! Testcase upgrade function removing interfaces-state
* @param[in] h Clicon handle
* @param[in] xn XML tree to be updated
* @param[in] ns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
* @see clicon_upgrade_cb
* @see test_upgrade_interfaces.sh
* @see upgrade_interfaces_2014
* The 2016 function does:
* - Delete /if:interfaces-state
* - Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr
* - Change type /interfaces/interface/statistics/in-octets to decimal64 with
* fraction-digits 3 and divide all values with 1000
*/
static int
upgrade_interfaces_2016(clicon_handle h,
cxobj *xt,
char *ns,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret)
{
int retval = -1;
yang_spec *yspec;
yang_stmt *ym;
cxobj **vec = NULL;
cxobj *xc;
cxobj *xi;
cxobj *x;
cxobj *xb;
size_t vlen;
int i;
/* Get Yang module for this namespace. Note it may not exist (if obsolete) */
yspec = clicon_dbspec_yang(h);
if ((ym = yang_find_module_by_namespace(yspec, ns)) == NULL)
goto ok; /* shouldnt happen */
clicon_debug(1, "%s module %s", __FUNCTION__, ym?ym->ys_argument:"none");
/* Get all XML nodes with that namespace */
if (xml_namespace_vec(h, xt, ns, &vec, &vlen) < 0)
goto done;
for (i=0; i<vlen; i++){
xc = vec[i];
/* Delete /if:interfaces-state */
if (strcmp(xml_name(xc), "interfaces-state") == 0)
xml_purge(xc);
/* Iterate through interfaces */
else if (strcmp(xml_name(xc),"interfaces") == 0){
/* Iterate through interfaces */
xi = NULL;
while ((xi = xml_child_each(xc, xi, CX_ELMNT)) != NULL) {
if (strcmp(xml_name(xi), "interface"))
continue;
/* Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr */
if ((x = xml_find(xi, "descr")) != NULL)
if (xml_wrap(x, "docs") < 0)
goto done;
/* Change type /interfaces/interface/statistics/in-octets to
* decimal64 with fraction-digits 3 and divide values with 1000
*/
if ((x = xpath_first(xi, "statistics/in-octets")) != NULL){
if ((xb = xml_body_get(x)) != NULL){
uint64_t u64;
cbuf *cb = cbuf_new();
parse_uint64(xml_value(xb), &u64, NULL);
cprintf(cb, "%" PRIu64 ".%03d", u64/1000, (int)(u64%1000));
xml_value_set(xb, cbuf_get(cb));
cbuf_free(cb);
}
}
}
}
}
ok:
retval = 1;
done:
if (vec)
free(vec);
return retval;
} }
#endif
/*! Plugin state reset. Add xml or set state in backend machine. /*! Plugin state reset. Add xml or set state in backend machine.
* Called in each backend plugin. plugin_reset is called after all plugins * Called in each backend plugin. plugin_reset is called after all plugins
@ -335,7 +501,7 @@ example_reset(clicon_handle h,
* *
* plugin_start is called once everything has been initialized, right before * plugin_start is called once everything has been initialized, right before
* the main event loop is entered. * the main event loop is entered.
* From the CLI, command line options can be passed to the * From the cli/backend, command line options can be passed to the
* plugins by using "-- <args>" where <args> is any choice of * plugins by using "-- <args>" where <args> is any choice of
* options specific to the application. These options are passed to the * options specific to the application. These options are passed to the
* plugin_start function via the argc and argv arguments which * plugin_start function via the argc and argv arguments which
@ -343,22 +509,9 @@ example_reset(clicon_handle h,
*/ */
int int
example_start(clicon_handle h, example_start(clicon_handle h,
int argc, int argc,
char **argv) char **argv)
{ {
char c;
opterr = 0;
optind = 1;
while ((c = getopt(argc, argv, "rs")) != -1)
switch (c) {
case 'r':
_reset = 1;
break;
case 's':
_state = 1;
break;
}
return 0; return 0;
} }
@ -389,13 +542,36 @@ static clixon_plugin_api api = {
* @param[in] h Clixon handle * @param[in] h Clixon handle
* @retval NULL Error with clicon_err set * @retval NULL Error with clicon_err set
* @retval api Pointer to API struct * @retval api Pointer to API struct
* In this example, you can pass -r, -s, -u to control the behaviour, mainly
* for use in the test suites.
*/ */
clixon_plugin_api * clixon_plugin_api *
clixon_plugin_init(clicon_handle h) clixon_plugin_init(clicon_handle h)
{ {
struct timeval retention = {0,0}; struct timeval retention = {0,0};
int argc; /* command-line options (after --) */
char **argv;
char c;
clicon_debug(1, "%s backend", __FUNCTION__); clicon_debug(1, "%s backend", __FUNCTION__);
if (clicon_argv_get(h, &argc, &argv) < 0)
goto done;
opterr = 0;
optind = 1;
while ((c = getopt(argc, argv, "rsu")) != -1)
switch (c) {
case 'r':
_reset = 1;
break;
case 's':
_state = 1;
break;
case 'u':
_upgrade = 1;
break;
}
/* Example stream initialization: /* Example stream initialization:
* 1) Register EXAMPLE stream * 1) Register EXAMPLE stream
* 2) setup timer for notifications, so something happens on stream * 2) setup timer for notifications, so something happens on stream
@ -444,14 +620,18 @@ clixon_plugin_init(clicon_handle h)
"copy-config" "copy-config"
) < 0) ) < 0)
goto done; goto done;
/* Called after the regular system copy_config callback */ /* Upgrade callback: if you start the backend with -- -u you will get the
if (upgrade_callback_register(h, yang_changelog_upgrade, * test interface example. Otherwise the auto-upgrade feature is enabled.
NULL, */
NULL, NULL, if (_upgrade){
0, 0 if (upgrade_callback_register(h, upgrade_interfaces_2014, "urn:example:interfaces", 0, 0, NULL) < 0)
) < 0) goto done;
goto done; if (upgrade_callback_register(h, upgrade_interfaces_2016, "urn:example:interfaces", 0, 0, NULL) < 0)
goto done;
}
else
if (upgrade_callback_register(h, xml_changelog_upgrade, NULL, 0, 0, NULL) < 0)
goto done;
/* Return plugin API */ /* Return plugin API */
return &api; return &api;

2
lib/clixon/clixon.h.in
View file

@ -90,7 +90,7 @@
#include <clixon/clixon_json.h> #include <clixon/clixon_json.h>
#include <clixon/clixon_netconf_lib.h> #include <clixon/clixon_netconf_lib.h>
#include <clixon/clixon_nacm.h> #include <clixon/clixon_nacm.h>
#include <clixon/clixon_yang_changelog.h> #include <clixon/clixon_xml_changelog.h>
/* /*
* Global variables generated by Makefile * Global variables generated by Makefile

10
lib/clixon/clixon_options.h
View file

@ -210,8 +210,12 @@ int clicon_socket_set(clicon_handle h, int s);
cxobj *clicon_module_state_get(clicon_handle h); cxobj *clicon_module_state_get(clicon_handle h);
int clicon_module_state_set(clicon_handle h, cxobj *xms); int clicon_module_state_set(clicon_handle h, cxobj *xms);
/*! Set and get module revision changelog */ /*! Set and get yang/xml module revision changelog */
cxobj *clicon_yang_changelog_get(clicon_handle h); cxobj *clicon_xml_changelog_get(clicon_handle h);
int clicon_yang_changelog_set(clicon_handle h, cxobj *xchlog); int clicon_xml_changelog_set(clicon_handle h, cxobj *xchlog);
/*! Set and get user command-line options (after --) */
int clicon_argv_get(clicon_handle h, int *argc, char ***argv);
int clicon_argv_set(clicon_handle h, char *argv0, int argc, char **argv);
#endif /* _CLIXON_OPTIONS_H_ */ #endif /* _CLIXON_OPTIONS_H_ */

15
lib/clixon/clixon_plugin.h
View file

@ -70,8 +70,7 @@ typedef int (*clicon_rpc_cb)(
/*! Registered Upgrade callback function /*! Registered Upgrade callback function
* @param[in] h Clicon handle * @param[in] h Clicon handle
* @param[in] xn XML tree to be updated * @param[in] xn XML tree to be updated
* @param[in] modname Name of module * @param[in] namespace Namespace of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD * @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system) * @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register() * @param[in] arg User argument given at rpc_callback_register()
@ -83,8 +82,7 @@ typedef int (*clicon_rpc_cb)(
typedef int (*clicon_upgrade_cb)( typedef int (*clicon_upgrade_cb)(
clicon_handle h, clicon_handle h,
cxobj *xn, cxobj *xn,
char *modname, char *namespace,
char *modns,
uint32_t from, uint32_t from,
uint32_t to, uint32_t to,
void *arg, void *arg,
@ -203,6 +201,11 @@ struct clixon_plugin_api{
#define ca_trans_end u.cau_backend.cb_trans_end #define ca_trans_end u.cau_backend.cb_trans_end
#define ca_trans_abort u.cau_backend.cb_trans_abort #define ca_trans_abort u.cau_backend.cb_trans_abort
/*
* Macros
*/
#define upgrade_callback_register(h, cb, namespace, from, to, arg) upgrade_callback_reg_fn((h), (cb), #cb, (namespace), (from), (to), (arg))
typedef struct clixon_plugin_api clixon_plugin_api; typedef struct clixon_plugin_api clixon_plugin_api;
/* Internal plugin structure with dlopen() handle and plugin_api /* Internal plugin structure with dlopen() handle and plugin_api
@ -246,8 +249,8 @@ int rpc_callback_delete_all(clicon_handle h);
int rpc_callback_call(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg); int rpc_callback_call(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg);
/* upgrade callback API */ /* upgrade callback API */
int upgrade_callback_register(clicon_handle h, clicon_upgrade_cb cb, void *arg, char *name, char *namespace, uint32_t from, uint32_t to); int upgrade_callback_reg_fn(clicon_handle h, clicon_upgrade_cb cb, const char *strfn, char *namespace, uint32_t from, uint32_t to, void *arg);
int upgrade_callback_delete_all(clicon_handle h); int upgrade_callback_delete_all(clicon_handle h);
int upgrade_callback_call(clicon_handle h, cxobj *xt, char *modname, char *modns, uint32_t from, uint32_t to, cbuf *cbret); int upgrade_callback_call(clicon_handle h, cxobj *xt, char *namespace, uint32_t from, uint32_t to, cbuf *cbret);
#endif /* _CLIXON_PLUGIN_H_ */ #endif /* _CLIXON_PLUGIN_H_ */

4
lib/clixon/clixon_xml.h
View file

@ -127,7 +127,9 @@ int xml_cv_set(cxobj *x, cg_var *cv);
cxobj *xml_find(cxobj *xn_parent, char *name); cxobj *xml_find(cxobj *xn_parent, char *name);
int xml_addsub(cxobj *xp, cxobj *xc); int xml_addsub(cxobj *xp, cxobj *xc);
cxobj *xml_insert(cxobj *xt, char *tag); cxobj *xml_wrap_all(cxobj *xp, char *tag);
cxobj *xml_wrap(cxobj *xc, char *tag);
#define xml_insert(x,t) xml_wrap_all((x),(t))
int xml_purge(cxobj *xc); int xml_purge(cxobj *xc);
int xml_child_rm(cxobj *xp, int i); int xml_child_rm(cxobj *xp, int i);
int xml_rm(cxobj *xc); int xml_rm(cxobj *xc);

11
lib/clixon/clixon_yang_changelog.h → lib/clixon/clixon_xml_changelog.h
View file

@ -34,13 +34,14 @@
* YANG module revision change management. * YANG module revision change management.
* See draft-wang-netmod-module-revision-management-01 * See draft-wang-netmod-module-revision-management-01
*/ */
#ifndef _CLIXON_YANG_CHANGELOG_H #ifndef _CLIXON_XML_CHANGELOG_H
#define _CLIXON_YANG_CHANGELOG_H #define _CLIXON_XML_CHANGELOG_H
/* /*
* Prototypes * Prototypes
*/ */
int yang_changelog_upgrade(clicon_handle h, cxobj *xn, char *modname, char *modns, uint32_t from, uint32_t to, void *arg, cbuf *cbret); int xml_changelog_upgrade(clicon_handle h, cxobj *xn, char *namespace, uint32_t from, uint32_t to, void *arg, cbuf *cbret);
int clixon_yang_changelog_init(clicon_handle h); int clixon_xml_changelog_init(clicon_handle h);
int xml_namespace_vec(clicon_handle h, cxobj *xt, char *namespace, cxobj ***vec, size_t *veclen);
#endif /* _CLIXON_YANG_CHANGELOG_H */ #endif /* _CLIXON_XML_CHANGELOG_H */

4
lib/clixon/clixon_yang_module.h
View file

@ -51,8 +51,8 @@
* This is in state of flux so it needss to be conatained and easily changed. * This is in state of flux so it needss to be conatained and easily changed.
*/ */
typedef struct { typedef struct {
cxobj *md_del; /* yang mdoule state deletes */ cxobj *md_del; /* yang module state deletes */
cxobj *md_mod; /* yang mdoule state modifications */ cxobj *md_mod; /* yang module state modifications */
} modstate_diff_t; } modstate_diff_t;
/* /*

2
lib/src/Makefile.in
View file

@ -70,7 +70,7 @@ SRC = clixon_sig.c clixon_log.c clixon_err.c clixon_event.c \
clixon_string.c clixon_handle.c \ clixon_string.c clixon_handle.c \
clixon_xml.c clixon_xml_sort.c clixon_xml_map.c clixon_file.c \ clixon_xml.c clixon_xml_sort.c clixon_xml_map.c clixon_file.c \
clixon_json.c clixon_yang.c clixon_yang_type.c clixon_yang_module.c \ clixon_json.c clixon_yang.c clixon_yang_type.c clixon_yang_module.c \
clixon_yang_cardinality.c clixon_yang_changelog.c \ clixon_yang_cardinality.c clixon_xml_changelog.c \
clixon_hash.c clixon_options.c clixon_plugin.c \ clixon_hash.c clixon_options.c clixon_plugin.c \
clixon_proto.c clixon_proto_client.c \ clixon_proto.c clixon_proto_client.c \
clixon_xpath.c clixon_xpath_ctx.c clixon_sha1.c \ clixon_xpath.c clixon_xpath_ctx.c clixon_sha1.c \

4
lib/src/clixon_netconf_lib.c
View file

@ -1043,8 +1043,8 @@ netconf_module_load(clicon_handle h)
if (yang_spec_parse_module(h, "clixon-rfc5277", NULL, yspec)< 0) if (yang_spec_parse_module(h, "clixon-rfc5277", NULL, yspec)< 0)
goto done; goto done;
/* YANG module revision change management */ /* YANG module revision change management */
if (clicon_option_bool(h, "CLICON_YANG_CHANGELOG")) if (clicon_option_bool(h, "CLICON_XML_CHANGELOG"))
if (yang_spec_parse_module(h, "clixon-yang-changelog", NULL, yspec)< 0) if (yang_spec_parse_module(h, "clixon-xml-changelog", NULL, yspec)< 0)
goto done; goto done;
retval = 0; retval = 0;
done: done:

67
lib/src/clixon_options.c
View file

@ -1019,17 +1019,17 @@ clicon_module_state_set(clicon_handle h,
* @see draft-wang-netmod-module-revision-management-01 * @see draft-wang-netmod-module-revision-management-01
*/ */
cxobj * cxobj *
clicon_yang_changelog_get(clicon_handle h) clicon_xml_changelog_get(clicon_handle h)
{ {
clicon_hash_t *cdat = clicon_data(h); clicon_hash_t *cdat = clicon_data(h);
void *p; void *p;
if ((p = hash_value(cdat, "yang-changelog", NULL)) != NULL) if ((p = hash_value(cdat, "xml-changelog", NULL)) != NULL)
return *(cxobj **)p; return *(cxobj **)p;
return NULL; return NULL;
} }
/*! Set yang module changelog /*! Set xml module changelog
* @param[in] h Clicon handle * @param[in] h Clicon handle
* @param[in] s Module revision changelog XML tree * @param[in] s Module revision changelog XML tree
* @retval 0 OK * @retval 0 OK
@ -1037,12 +1037,69 @@ clicon_yang_changelog_get(clicon_handle h)
* @see draft-wang-netmod-module-revision-management-01 * @see draft-wang-netmod-module-revision-management-01
*/ */
int int
clicon_yang_changelog_set(clicon_handle h, clicon_xml_changelog_set(clicon_handle h,
cxobj *xchlog) cxobj *xchlog)
{ {
clicon_hash_t *cdat = clicon_data(h); clicon_hash_t *cdat = clicon_data(h);
if (hash_add(cdat, "yang_changelog", &xchlog, sizeof(xchlog))==NULL) if (hash_add(cdat, "xml-changelog", &xchlog, sizeof(xchlog))==NULL)
return -1;
return 0;
}
/*! Get user clicon command-line options argv, argc (after --)
* @param[in] h Clicon handle
* @param[out] argc
* @param[out] argv
* @retval 0 OK
* @retval -1 Error
*/
int
clicon_argv_get(clicon_handle h,
int *argc,
char ***argv)
{
clicon_hash_t *cdat = clicon_data(h);
void *p;
if ((p = hash_value(cdat, "argc", NULL)) == NULL)
return -1;
*argc = *(int*)p;
if ((p = hash_value(cdat, "argv", NULL)) == NULL)
return -1;
*argv = *(char***)p;
return 0;
}
/*! Set clicon user command-line options argv, argc (after --)
* @param[in] h Clicon handle
* @param[in] prog argv[0] - the program name
* @param[in] argc Length of argv
* @param[in] argv Array of command-line options
* @retval 0 OK
* @retval -1 Error
*/
int
clicon_argv_set(clicon_handle h,
char *prgm,
int argc,
char **argv)
{
clicon_hash_t *cdat = clicon_data(h);
char **argvv = NULL;
/* add space for null-termination and argv[0] program name */
if ((argvv = calloc(argc+2, sizeof(char*))) == NULL){
clicon_err(OE_UNIX, errno, "calloc");
return -1;
}
memcpy(argvv+1, argv, argc*sizeof(char*));
argvv[0] = prgm;
if (hash_add(cdat, "argv", &argvv, sizeof(argvv))==NULL)
return -1;
argc += 1;
if (hash_add(cdat, "argc", &argc, sizeof(argc))==NULL)
return -1; return -1;
return 0; return 0;
} }

65
lib/src/clixon_plugin.c
View file

@ -539,13 +539,14 @@ rpc_callback_call(clicon_handle h,
* revision. * revision.
*/ */
typedef struct { typedef struct {
qelem_t uc_qelem; /* List header */ qelem_t uc_qelem; /* List header */
clicon_upgrade_cb uc_callback; /* RPC Callback */ clicon_upgrade_cb uc_callback; /* RPC Callback */
void *uc_arg; /* Application specific argument to cb */ const char *uc_fnstr; /* Stringified fn name for debug */
char *uc_name; /* Module name */ void *uc_arg; /* Application specific argument to cb */
char *uc_namespace; /* Module namespace ??? */ char *uc_namespace; /* Module namespace */
uint32_t uc_from; /* Module revision (from) or 0 in YYYYMMDD format */ uint32_t uc_rev; /* Module revision (to) in YYYYMMDD format or 0 */
uint32_t uc_to; /* Module revision (to) in YYYYMMDD format */ uint32_t uc_from; /* Module revision (from) or 0 in YYYYMMDD format */
} upgrade_callback_t; } upgrade_callback_t;
/* List of rpc callback entries XXX hang on handle */ /* List of rpc callback entries XXX hang on handle */
@ -555,23 +556,23 @@ static upgrade_callback_t *upgrade_cb_list = NULL;
* *
* @param[in] h clicon handle * @param[in] h clicon handle
* @param[in] cb Callback called * @param[in] cb Callback called
* @param[in] fnstr Stringified function for debug
* @param[in] arg Domain-specific argument to send to callback * @param[in] arg Domain-specific argument to send to callback
* @param[in] name Module name (if NULL all modules) * @param[in] namespace Module namespace (if NULL all modules)
* @param[in] namespace Module namespace (NOTE not relevant) * @param[in] rev To module revision (0 means module obsoleted)
* @param[in] from From module revision (0 from any revision) * @param[in] from From module revision (0 from any revision)
* @param[in] to To module revision (0 means module obsoleted)
* @retval 0 OK * @retval 0 OK
* @retval -1 Error * @retval -1 Error
* @see upgrade_callback_call which makes the actual callback * @see upgrade_callback_call which makes the actual callback
*/ */
int int
upgrade_callback_register(clicon_handle h, upgrade_callback_reg_fn(clicon_handle h,
clicon_upgrade_cb cb, clicon_upgrade_cb cb,
void *arg, const char *fnstr,
char *name, char *namespace,
char *namespace, uint32_t revision,
uint32_t from, uint32_t from,
uint32_t to) void *arg)
{ {
upgrade_callback_t *uc; upgrade_callback_t *uc;
@ -581,19 +582,16 @@ upgrade_callback_register(clicon_handle h,
} }
memset(uc, 0, sizeof(*uc)); memset(uc, 0, sizeof(*uc));
uc->uc_callback = cb; uc->uc_callback = cb;
uc->uc_fnstr = fnstr;
uc->uc_arg = arg; uc->uc_arg = arg;
if (name)
uc->uc_name = strdup(name);
if (namespace) if (namespace)
uc->uc_namespace = strdup(namespace); uc->uc_namespace = strdup(namespace);
uc->uc_rev = revision;
uc->uc_from = from; uc->uc_from = from;
uc->uc_to = to;
ADDQ(uc, upgrade_cb_list); ADDQ(uc, upgrade_cb_list);
return 0; return 0;
done: done:
if (uc){ if (uc){
if (uc->uc_name)
free(uc->uc_name);
if (uc->uc_namespace) if (uc->uc_namespace)
free(uc->uc_namespace); free(uc->uc_namespace);
free(uc); free(uc);
@ -610,8 +608,6 @@ upgrade_callback_delete_all(clicon_handle h)
while((uc = upgrade_cb_list) != NULL) { while((uc = upgrade_cb_list) != NULL) {
DELQ(uc, upgrade_cb_list, upgrade_callback_t *); DELQ(uc, upgrade_cb_list, upgrade_callback_t *);
if (uc->uc_name)
free(uc->uc_name);
if (uc->uc_namespace) if (uc->uc_namespace)
free(uc->uc_namespace); free(uc->uc_namespace);
free(uc); free(uc);
@ -622,7 +618,7 @@ upgrade_callback_delete_all(clicon_handle h)
/*! Search Upgrade callbacks and invoke if module match /*! Search Upgrade callbacks and invoke if module match
* *
* @param[in] h clicon handle * @param[in] h clicon handle
* @param[in] xt XML tree to be updated * @param[in] xt Top-level XML tree to be updated (includes other ns as well)
* @param[in] modname Name of module * @param[in] modname Name of module
* @param[in] modns Namespace of module (for info) * @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD * @param[in] from From revision on the form YYYYMMDD
@ -631,13 +627,12 @@ upgrade_callback_delete_all(clicon_handle h)
* @retval -1 Error * @retval -1 Error
* @retval 0 Invalid - cbret contains reason as netconf * @retval 0 Invalid - cbret contains reason as netconf
* @retval 1 OK * @retval 1 OK
* @see upgrade_callback_register which registers the callbacks * @see upgrade_callback_reg_fn which registers the callbacks
*/ */
int int
upgrade_callback_call(clicon_handle h, upgrade_callback_call(clicon_handle h,
cxobj *xt, cxobj *xt,
char *modname, char *namespace,
char *modns,
uint32_t from, uint32_t from,
uint32_t to, uint32_t to,
cbuf *cbret) cbuf *cbret)
@ -659,15 +654,21 @@ upgrade_callback_call(clicon_handle h,
* - Registered from revision >= from AND * - Registered from revision >= from AND
* - Registered to revision <= to (which includes case both 0) * - Registered to revision <= to (which includes case both 0)
*/ */
if (uc->uc_name == NULL || strcmp(uc->uc_name, modname)==0) if (uc->uc_namespace == NULL || strcmp(uc->uc_namespace, namespace)==0)
if ((uc->uc_from == 0) || if ((uc->uc_from == 0) ||
(uc->uc_from >= from && uc->uc_to <= to)){ (uc->uc_from >= from && uc->uc_rev <= to)){
if ((ret = uc->uc_callback(h, xt, modname, modns, from, to, uc->uc_arg, cbret)) < 0){ if ((ret = uc->uc_callback(h, xt, namespace, from, to, uc->uc_arg, cbret)) < 0){
clicon_debug(1, "%s Error in: %s", __FUNCTION__, uc->uc_name); clicon_debug(1, "%s Error in: %s", __FUNCTION__, uc->uc_namespace);
goto done; goto done;
} }
if (ret == 0) if (ret == 0){
if (cbuf_len(cbret)==0){
clicon_err(OE_CFG, 0, "Validation fail %s(%s): cbret not set",
uc->uc_fnstr, namespace);
goto done;
}
goto fail; goto fail;
}
nr++; nr++;
} }
uc = NEXTQ(upgrade_callback_t *, uc); uc = NEXTQ(upgrade_callback_t *, uc);

75
lib/src/clixon_xml.c
View file

@ -560,7 +560,8 @@ xml_child_nr_type(cxobj *xn,
/*! Get a specific child /*! Get a specific child
* @param[in] xn xml node * @param[in] xn xml node
* @param[in] i the number of the child, eg order in children vector * @param[in] i the number of the child, eg order in children vector
* @retval child in XML tree, or NULL if no such child, or empty child * @retval xml The child xml node
* @retval NULL if no such child, or empty child
*/ */
cxobj * cxobj *
xml_child_i(cxobj *xn, xml_child_i(cxobj *xn,
@ -633,6 +634,8 @@ xml_child_each(cxobj *xparent,
int i; int i;
cxobj *xn = NULL; cxobj *xn = NULL;
if (xparent == NULL)
return NULL;
for (i=xprev?xprev->_x_vector_i+1:0; i<xparent->x_childvec_len; i++){ for (i=xprev?xprev->_x_vector_i+1:0; i<xparent->x_childvec_len; i++){
xn = xparent->x_childvec[i]; xn = xparent->x_childvec[i];
if (xn == NULL) if (xn == NULL)
@ -805,7 +808,7 @@ xml_find(cxobj *x_up,
* @param[in] xc Child xml node to insert under xp * @param[in] xc Child xml node to insert under xp
* @retval 0 OK * @retval 0 OK
* @retval -1 Error * @retval -1 Error
* @see xml_insert * @see xml_wrap
*/ */
int int
xml_addsub(cxobj *xp, xml_addsub(cxobj *xp,
@ -833,30 +836,55 @@ xml_addsub(cxobj *xp,
return 0; return 0;
} }
/*! Insert a new element (xc) under an xml node (xp), move all children to xc. /*! Wrap a new node between a parent xml node (xp) and all its children
* Before: xp --> xt * Before: xp --> xc*
* After: xp --> xc --> xt * After: xp --> xw --> xc*
* @param[in] xp Parent xml node
* @param[in] tag Name of new xml child
* @retval xw Return the new child (xw)
* @see xml_addsub
* @see xml_wrap (wrap s single node)
*/
cxobj *
xml_wrap_all(cxobj *xp,
char *tag)
{
cxobj *xw; /* new wrap node */
if ((xw = xml_new(tag, NULL, NULL)) == NULL)
goto done;
while (xp->x_childvec_len)
if (xml_addsub(xw, xml_child_i(xp, 0)) < 0)
goto done;
if (xml_addsub(xp, xw) < 0)
goto done;
done:
return xw;
}
/*! Wrap a new element above a single xml node (xc) with new tag
* Before: xp --> xc # specific child
* After: xp --> xt(tag) --> xc
* @param[in] xp Parent xml node * @param[in] xp Parent xml node
* @param[in] tag Name of new xml child * @param[in] tag Name of new xml child
* @retval xc Return the new child (xc) * @retval xc Return the new child (xc)
* @see xml_addsub * @see xml_addsub (give the parent)
* The name of the function is somewhat misleading * @see xml_wrap_all (wrap all children of a node, not just one)
*/ */
cxobj * cxobj *
xml_insert(cxobj *xp, xml_wrap(cxobj *xc,
char *tag) char *tag)
{ {
cxobj *xc; /* new child */ cxobj *xw; /* new wrap node */
cxobj *xp; /* parent */
if ((xc = xml_new(tag, NULL, NULL)) == NULL) xp = xml_parent(xc);
goto catch; if ((xw = xml_new(tag, xp, NULL)) == NULL)
while (xp->x_childvec_len) goto done;
if (xml_addsub(xc, xml_child_i(xp, 0)) < 0) if (xml_addsub(xw, xc) < 0)
goto catch; goto done;
if (xml_addsub(xp, xc) < 0) done:
goto catch; return xw;
catch:
return xc;
} }
/*! Remove and free an xml node child from xml parent /*! Remove and free an xml node child from xml parent
@ -958,7 +986,7 @@ xml_rm(cxobj *xc)
/*! Return a child sub-tree, while removing parent and all other children /*! Return a child sub-tree, while removing parent and all other children
* Given a root xml node, and the i:th child, remove the child from its parent * Given a root xml node, and the i:th child, remove the child from its parent
* and return it, remove the parent and all other children. * and return it, remove the parent and all other children. (unwrap)
* Before: xp-->[..xc..] * Before: xp-->[..xc..]
* After: xc * After: xc
* @param[in] xp xml parent node. Will be deleted * @param[in] xp xml parent node. Will be deleted
@ -1006,7 +1034,7 @@ xml_rootchild(cxobj *xp,
/*! Return a child sub-tree, while removing parent and all other children /*! Return a child sub-tree, while removing parent and all other children
* Given a root xml node, remove the child from its parent * Given a root xml node, remove the child from its parent
* , remove the parent and all other children. * , remove the parent and all other children. (unwrap)
* Before: xp-->[..xc..] * Before: xp-->[..xc..]
* After: xc * After: xc
* @param[in] xp xml parent node. Must be root. Will be deleted * @param[in] xp xml parent node. Must be root. Will be deleted
@ -1042,8 +1070,7 @@ xml_rootchild_node(cxobj *xp,
return retval; return retval;
} }
/*! Help function to sorting: enumerate all children according to present order
/*! help function to sorting: enumerate all children according to present order
* This is so that the child itself know its present order in a list. * This is so that the child itself know its present order in a list.
* When sorting by "ordered by user", the order should remain in its present * When sorting by "ordered by user", the order should remain in its present
* state. * state.
@ -1612,8 +1639,6 @@ _xml_parse(const char *str,
goto done; goto done;
if (xml_apply0(xt, CX_ELMNT, xml_sort, NULL) < 0) if (xml_apply0(xt, CX_ELMNT, xml_sort, NULL) < 0)
goto done; goto done;
if (xml_apply0(xt, -1, xml_sort_verify, NULL) < 0)
goto done;
} }
retval = 0; retval = 0;
done: done:

516
lib/src/clixon_xml_changelog.c Normal file
View file

@ -0,0 +1,516 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand and Benny Holmgren
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* YANG module revision change management.
* See draft-wang-netmod-module-revision-management-01
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <fnmatch.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/syslog.h>
#include <fcntl.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon_queue.h"
#include "clixon_hash.h"
#include "clixon_string.h"
#include "clixon_err.h"
#include "clixon_handle.h"
#include "clixon_yang.h"
#include "clixon_log.h"
#include "clixon_xml.h"
#include "clixon_options.h"
#include "clixon_xml_map.h"
#include "clixon_yang_module.h"
#include "clixon_xml_changelog.h"
#include "clixon_xpath_ctx.h"
#include "clixon_xpath.h"
static int
changelog_rename(clicon_handle h,
cxobj *xt,
cxobj *xw,
char *tag)
{
int retval = -1;
xp_ctx *xctx = NULL;
char *str = NULL;
if (tag == NULL){
clicon_err(OE_XML, 0, "tag required");
goto done;
}
if (xpath_vec_ctx(xw, tag, &xctx) < 0)
goto done;
if (ctx2string(xctx, &str) < 0)
goto done;
if (!strlen(str)){
clicon_err(OE_XML, 0, "invalid rename tag: \"%s\"", str);
goto done;
}
if (xml_name_set(xw, str) < 0)
goto done;
// ok:
retval = 1;
done:
if (xctx)
ctx_free(xctx);
if (str)
free(str);
return retval;
// fail:
retval = 0;
goto done;
}
/* replace target XML */
static int
changelog_replace(clicon_handle h,
cxobj *xt,
cxobj *xw,
cxobj *xnew)
{
int retval = -1;
cxobj *x;
/* create a new node by parsing fttransform string and insert it at
target */
if (xnew == NULL){
clicon_err(OE_XML, 0, "new required");
goto done;
}
/* replace: remove all children of target */
while ((x = xml_child_i(xw, 0)) != NULL)
if (xml_purge(x) < 0)
goto done;
/* replace: first single node under <new> */
if (xml_child_nr(xnew) != 1){
clicon_err(OE_XML, 0, "Single child to <new> required");
goto done;
}
x = xml_child_i(xnew, 0);
/* Copy from xnew to (now) empty target */
if (xml_copy(x, xw) < 0)
goto done;
retval = 1;
done:
return retval;
}
/* create a new node by parsing "new" and insert it at
target */
static int
changelog_insert(clicon_handle h,
cxobj *xt,
cxobj *xw,
cxobj *xnew)
{
int retval = -1;
cxobj *x;
if (xnew == NULL){
clicon_err(OE_XML, 0, "new required");
goto done;
}
/* replace: add all new children to target */
while ((x = xml_child_i(xnew, 0)) != NULL)
if (xml_addsub(xw, x) < 0)
goto done;
// ok:
retval = 1;
done:
return retval;
// fail:
retval = 0;
goto done;
}
/* delete target */
static int
changelog_delete(clicon_handle h,
cxobj *xt,
cxobj *xw)
{
int retval = -1;
if (xml_purge(xw) < 0)
goto done;
retval = 1;
done:
return retval;
}
/* Move target node to location */
static int
changelog_move(clicon_handle h,
cxobj *xt,
cxobj *xw,
char *dst)
{
int retval = -1;
cxobj *xp; /* destination parent node */
if ((xp = xpath_first(xt, "%s", dst)) == NULL){
clicon_err(OE_XML, 0, "path required");
goto done;
}
if (xml_addsub(xp, xw) < 0)
goto done;
retval = 1;
done:
return retval;
}
/*! Perform a changelog operation
* @param[in] h Clicon handle
* @param[in] xt XML to upgrade
* @param[in] xi Changelog item
* @note XXX error handling!
* @note XXX xn --> xt xpath may not match
*/
static int
changelog_op(clicon_handle h,
cxobj *xt,
cxobj *xi)
{
int retval = -1;
char *op;
char *whenxpath; /* xpath to when */
char *tag; /* xpath to extra path (move) */
char *dst; /* xpath to extra path (move) */
cxobj *xnew; /* new xml (insert, replace) */
char *wxpath; /* xpath to where (target-node) */
cxobj **wvec = NULL; /* Vector of where(target) nodes */
size_t wlen;
cxobj *xw;
int ret;
xp_ctx *xctx = NULL;
int i;
if ((op = xml_find_body(xi, "op")) == NULL)
goto ok;
/* get common variables that may be used in the operations below */
tag = xml_find_body(xi, "tag");
dst = xml_find_body(xi, "dst");
xnew = xml_find(xi, "new");
whenxpath = xml_find_body(xi, "when");
if ((wxpath = xml_find_body(xi, "where")) == NULL)
goto ok;
/* Get vector of target nodes meeting the where requirement */
if (xpath_vec(xt, "%s", &wvec, &wlen, wxpath) < 0)
goto done;
for (i=0; i<wlen; i++){
xw = wvec[i];
/* If 'when' exists and is false, skip this target */
if (whenxpath){
if (xpath_vec_ctx(xw, whenxpath, &xctx) < 0)
goto done;
if ((ret = ctx2boolean(xctx)) < 0)
goto done;
if (xctx){
ctx_free(xctx);
xctx = NULL;
}
if (ret == 0)
continue;
}
/* Now switch on operation */
if (strcmp(op, "rename") == 0){
ret = changelog_rename(h, xt, xw, tag);
}
else if (strcmp(op, "replace") == 0){
ret = changelog_replace(h, xt, xw, xnew);
}
else if (strcmp(op, "insert") == 0){
ret = changelog_insert(h, xt, xw, xnew);
}
else if (strcmp(op, "delete") == 0){
ret = changelog_delete(h, xt, xw);
}
else if (strcmp(op, "move") == 0){
ret = changelog_move(h, xt, xw, dst);
}
else{
clicon_err(OE_XML, 0, "Unknown operation: %s", op);
goto done;
}
if (ret < 0)
goto done;
if (ret == 0)
goto fail;
}
ok:
retval = 1;
done:
if (wvec)
free(wvec);
if (xctx)
ctx_free(xctx);
return retval;
fail:
retval = 0;
clicon_debug(1, "%s fail op:%s ", __FUNCTION__, op);
goto done;
}
/*! Iterate through one changelog item
* @param[in] h Clicon handle
* @param[in] xt Changelog list
* @param[in] xn XML to upgrade
*/
static int
changelog_iterate(clicon_handle h,
cxobj *xt,
cxobj *xch)
{
int retval = -1;
cxobj **vec = NULL;
size_t veclen;
int ret;
int i;
if (xpath_vec(xch, "step", &vec, &veclen) < 0)
goto done;
/* Iterate through changelog items */
for (i=0; i<veclen; i++){
if ((ret = changelog_op(h, xt, vec[i])) < 0)
goto done;
if (ret == 0)
goto fail;
}
retval = 1;
done:
clicon_debug(1, "%s retval: %d", __FUNCTION__, retval);
if (vec)
free(vec);
return retval;
fail:
retval = 0;
goto done;
}
/*! Automatic upgrade using changelog
* @param[in] h Clicon handle
* @param[in] xt Top-level XML tree to be updated (includes other ns as well)
* @param[in] namespace Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
* @see upgrade_callback_register where this function should be registered
*/
int
xml_changelog_upgrade(clicon_handle h,
cxobj *xt,
char *namespace,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret)
{
int retval = -1;
cxobj *xchlog; /* changelog */
cxobj **vec = NULL;
cxobj *xch;
size_t veclen;
char *b;
int ret;
int i;
uint32_t f;
uint32_t t;
/* Check if changelog enabled */
if (!clicon_option_bool(h, "CLICON_XML_CHANGELOG"))
goto ok;
/* Get changelog */
if ((xchlog = clicon_xml_changelog_get(h)) == NULL)
goto ok;
/* Iterate and find relevant changelog entries in the interval:
* - find all changelogs in the interval: [from, to]
* - note it t=0 then no changelog is applied
*/
if (xpath_vec(xchlog, "changelog[namespace=\"%s\"]",
&vec, &veclen, namespace) < 0)
goto done;
/* Get all changelogs in the interval [from,to]*/
for (i=0; i<veclen; i++){
xch = vec[i];
f = t = 0;
if ((b = xml_find_body(xch, "revfrom")) != NULL)
if (ys_parse_date_arg(b, &f) < 0)
goto done;
if ((b = xml_find_body(xch, "revision")) != NULL)
if (ys_parse_date_arg(b, &t) < 0)
goto done;
if ((f && from>f) || to<t)
continue;
if ((ret = changelog_iterate(h, xt, xch)) < 0)
goto done;
if (ret == 0)
goto fail;
}
ok:
retval = 1;
done:
if (vec)
free(vec);
return retval;
fail:
retval = 0;
goto done;
}
/*! Initialize module revision. read changelog, etc
*/
int
clixon_xml_changelog_init(clicon_handle h)
{
int retval = -1;
char *filename;
int fd = -1;
cxobj *xt = NULL;
yang_spec *yspec;
cbuf *cbret = NULL;
int ret;
yspec = clicon_dbspec_yang(h);
if ((filename = clicon_option_str(h, "CLICON_XML_CHANGELOG_FILE")) != NULL){
if ((fd = open(filename, O_RDONLY)) < 0){
clicon_err(OE_UNIX, errno, "open(%s)", filename);
goto done;
}
if (xml_parse_file(fd, NULL, yspec, &xt) < 0)
goto done;
if (xml_rootchild(xt, 0, &xt) < 0)
goto done;
if ((cbret = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
goto done;
}
if ((ret = xml_yang_validate_all(xt, cbret)) < 0)
goto done;
if (ret==1 && (ret = xml_yang_validate_add(xt, cbret)) < 0)
goto done;
if (ret == 0){ /* validation failed */
clicon_err(OE_YANG, 0, "validation failed: %s", cbuf_get(cbret));
goto done;
}
if (clicon_xml_changelog_set(h, xt) < 0)
goto done;
xt = NULL;
}
retval = 0;
done:
if (fd != -1)
close(fd);
if (xt)
xml_free(xt);
if (cbret)
cbuf_free(cbret);
return retval;
}
/*! Given a top-level XML tree and a namespace, return a vector of matching XML nodes
* @param[in] h Clicon handle
* @param[in] xt Top-level XML tree, with children marked with namespaces
* @param[in] namespace The namespace to select
* @param[out] vecp Vector containining XML nodes w namespace. Null-terminated.
* @param[out] veclenp Length of vector
* @note Need to free vec after use with free()
* Example
* xt ::= <config><a xmlns="urn:example:a"/><aaa xmlns="urn:example:a"/><a xmlns="urn:example:b"/></config
* namespace ::= urn:example:a
* out:
* vec ::= [<a xmlns="urn:example:a"/>, <aaa xmlns="urn:example:a"/>, NULL]
*/
int
xml_namespace_vec(clicon_handle h,
cxobj *xt,
char *namespace,
cxobj ***vecp,
size_t *veclenp)
{
int retval = -1;
cxobj **xvec = NULL;
size_t xlen;
cxobj *xc;
char *ns;
int i;
/* Allocate upper bound on length (ie could be too large) + a NULL element
* (event though we use veclen)
*/
xlen = xml_child_nr_type(xt, CX_ELMNT)+1;
if ((xvec = calloc(xlen, sizeof(cxobj*))) == NULL){
clicon_err(OE_UNIX, errno, "calloc");
goto done;
}
/* Iterate and find xml nodes with assoctaed namespace */
xc = NULL;
i = 0;
while ((xc = xml_child_each(xt, xc, CX_ELMNT)) != NULL) {
if (xml2ns(xc, NULL, &ns) < 0) /* Get namespace of XML */
goto done;
if (strcmp(namespace, ns))
continue; /* no match */
xvec[i++] = xc;
}
*vecp = xvec;
*veclenp = i;
retval = 0;
done:
return retval;
}

11
lib/src/clixon_xpath.c
View file

@ -830,7 +830,6 @@ xp_union(xp_ctx *xc1,
return retval; return retval;
} }
/*! Evaluate an XPATH on an XML tree /*! Evaluate an XPATH on an XML tree
* The initial sequence of steps selects a set of nodes relative to a context node. * The initial sequence of steps selects a set of nodes relative to a context node.
@ -1038,11 +1037,20 @@ xp_eval(xp_ctx *xc,
} }
/*! Given XML tree and xpath, returns xpath context /*! Given XML tree and xpath, returns xpath context
* This is a raw form of xpath where you can do type conversion, etc,
* not just a nodeset.
* @param[in] xcur XML-tree where to search * @param[in] xcur XML-tree where to search
* @param[in] xpath String with XPATH 1.0 syntax * @param[in] xpath String with XPATH 1.0 syntax
* @param[out] xrp Return XPATH context * @param[out] xrp Return XPATH context
* @retval 0 OK * @retval 0 OK
* @retval -1 Error * @retval -1 Error
* @code
* xp_ctx *xc = NULL;
* if (xpath_vec_ctx(x, xpath, &xc) < 0)
* err;
* if (xc)
* ctx_free(xc);
* @endcode
*/ */
int int
xpath_vec_ctx(cxobj *xcur, xpath_vec_ctx(cxobj *xcur,
@ -1295,6 +1303,7 @@ xpath_vec_flag(cxobj *xcur,
} }
/*! Given XML tree and xpath, returns boolean /*! Given XML tree and xpath, returns boolean
* Returns true if the nodeset is non-empty
* @param[in] xcur xml-tree where to search * @param[in] xcur xml-tree where to search
* @param[in] xpath stdarg string with XPATH 1.0 syntax * @param[in] xpath stdarg string with XPATH 1.0 syntax
* @retval 1 True * @retval 1 True

249
lib/src/clixon_yang_changelog.c
View file

@ -1,249 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand and Benny Holmgren
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* YANG module revision change management.
* See draft-wang-netmod-module-revision-management-01
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <fnmatch.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/syslog.h>
#include <fcntl.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon_queue.h"
#include "clixon_hash.h"
#include "clixon_string.h"
#include "clixon_err.h"
#include "clixon_handle.h"
#include "clixon_yang.h"
#include "clixon_log.h"
#include "clixon_xml.h"
#include "clixon_options.h"
#include "clixon_xml_map.h"
#include "clixon_yang_module.h"
#include "clixon_yang_changelog.h"
#include "clixon_xpath_ctx.h"
#include "clixon_xpath.h"
#if 0
/*! Make a specific change
<index>0001</index>
<change-operation>create</change-operation>
<data-definition>
<target-node>
/a:system/a:y;
</target-node>
</data-definition>
*/
static int
upgrade_op(cxobj *x)
{
int retval = -1;
xml_print(stderr, x);
retval = 0;
// done:
return retval;
}
static int
upgrade_deleted(clicon_handle h,
char *name,
cxobj *xs)
{
int retval = -1;
fprintf(stderr, "%s \"%s\" belongs to a removed module\n", __FUNCTION__, name);
retval = 0;
// done:
return retval;
}
/*!
* @param[in] xs Module state
*/
static int
upgrade_modified(clicon_handle h,
char *name,
char *namespace,
cxobj *xs,
cxobj *xch)
{
int retval = -1;
char *mname;
yang_spec *yspec = NULL;
yang_stmt *ymod;
yang_stmt *yrev;
char *mrev;
cxobj **vec = NULL;
size_t veclen;
int i;
fprintf(stderr, "%s: \"%s\" belongs to an upgraded module\n", __FUNCTION__, name);
yspec = clicon_dbspec_yang(h);
/* We need module-name of XML since changelog uses that (change in changelog?)*/
mname = xml_find_body(xs, "name");
/* Look up system module (alt send it via argument) */
if ((ymod = yang_find_module_by_name(yspec, mname)) == NULL)
goto done;
if ((yrev = yang_find((yang_node*)ymod, Y_REVISION, NULL)) == NULL)
goto done;
mrev = yrev->ys_argument;
/* Look up in changelog */
if (xpath_vec(xch, "module[name=\"%s\" and revision=\"%s\"]/revision-change-log",
&vec, &veclen, mname, mrev) < 0)
goto done;
/* Iterate through changelog */
for (i=0; i<veclen; i++)
if (upgrade_op(vec[i]) < 0)
goto done;
retval = 0;
done:
if (vec)
free(vec);
return retval;
}
#endif
/*! Automatic upgrade using changelog
* @param[in] h Clicon handle
* @param[in] xn XML tree to be updated
* @param[in] modname Name of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
*/
int
yang_changelog_upgrade(clicon_handle h,
cxobj *xn,
char *modname,
char *modns,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret)
{
// cxobj *xchlog; /* changelog */
// cxobj **vec = NULL;
// size_t veclen;
if (!clicon_option_bool(h, "CLICON_YANG_CHANGELOG"))
goto ok;
/* Get changelog */
// xchlog = clicon_yang_changelog_get(h);
/* Get changelog entries for module between from and to
* (if to=0 we may not know name, need to use namespace)
*/
#if 0
if (xpath_vec(xchlog, "module[name=\"%s\" and revision=\"%s\"]/revision-change-log",
&vec, &veclen, modname, mrev) < 0)
goto done;
#endif
ok:
return 1;
}
/*! Initialize module revision. read changelog, etc
*/
int
clixon_yang_changelog_init(clicon_handle h)
{
int retval = -1;
char *filename;
int fd = -1;
cxobj *xt = NULL;
yang_spec *yspec;
cbuf *cbret = NULL;
int ret;
yspec = clicon_dbspec_yang(h);
if ((filename = clicon_option_str(h, "CLICON_YANG_CHANGELOG_FILE")) != NULL){
if ((fd = open(filename, O_RDONLY)) < 0){
clicon_err(OE_UNIX, errno, "open(%s)", filename);
goto done;
}
if (xml_parse_file(fd, NULL, yspec, &xt) < 0)
goto done;
if (xml_rootchild(xt, 0, &xt) < 0)
goto done;
if ((cbret = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
goto done;
}
if ((ret = xml_yang_validate_all(xt, cbret)) < 0)
goto done;
if (ret==1 && (ret = xml_yang_validate_add(xt, cbret)) < 0)
goto done;
if (ret == 0){ /* validation failed */
clicon_err(OE_YANG, 0, "validation failed: %s", cbuf_get(cbret));
goto done;
}
if (clicon_yang_changelog_set(h, xt) < 0)
goto done;
xt = NULL;
}
retval = 0;
done:
if (fd != -1)
close(fd);
if (xt)
xml_free(xt);
if (cbret)
cbuf_free(cbret);
return retval;
}

140
lib/src/clixon_yang_module.c
View file

@ -375,12 +375,72 @@ yang_modules_state_get(clicon_handle h,
return retval; return retval;
} }
/*! For single module state with namespace, get revisions and send upgrade callbacks
* @param[in] h Clicon handle
* @param[in] xt Top-level XML tree to be updated (includes other ns as well)
* @param[in] xs XML module state (for one yang module)
* @param[in] xvec Help vector where to store XML child nodes (??)
* @param[in] xlen Length of xvec
* @param[in] ns0 Namespace of module state we are looking for
* @param[in] deleted If set, dont look for system yang module and "to" rev
* @param[out] cbret Netconf error message if invalid
* @retval 1 OK
* @retval 0 Validation failed (cbret set)
* @retval -1 Error
*/
static int
mod_ns_upgrade(clicon_handle h,
cxobj *xt,
cxobj *xs,
char *ns,
int deleted,
cbuf *cbret)
{
int retval = -1;
char *b; /* string body */
yang_stmt *ymod;
yang_stmt *yrev;
uint32_t from = 0;
uint32_t to = 0;
int ret;
yang_spec *yspec;
/* Make upgrade callback for this XML, specifying the module
* namespace, from and to revision.
*/
if ((b = xml_find_body(xs, "revision")) != NULL) /* Module revision */
if (ys_parse_date_arg(b, &from) < 0)
goto done;
if (deleted)
to = 0;
else { /* Look up system module (alt send it via argument) */
yspec = clicon_dbspec_yang(h);
if ((ymod = yang_find_module_by_namespace(yspec, ns)) == NULL)
goto fail;
if ((yrev = yang_find((yang_node*)ymod, Y_REVISION, NULL)) == NULL)
goto fail;
if (ys_parse_date_arg(yrev->ys_argument, &to) < 0)
goto done;
}
if ((ret = upgrade_callback_call(h, xt, ns, from, to, cbret)) < 0)
goto done;
if (ret == 0) /* XXX ignore and continue? */
goto fail;
retval = 1;
done:
return retval;
fail:
retval = 0;
goto done;
}
/*! Upgrade XML /*! Upgrade XML
* @param[in] h Clicon handle * @param[in] h Clicon handle
* @param[in] xt XML tree (to upgrade) * @param[in] xt XML tree (to upgrade)
* @param[in] msd Modules-state differences of xt * @param[in] msd Modules-state differences of xt
* @param[out] cbret Netconf error message if invalid
* @retval 1 OK * @retval 1 OK
* @retval 0 Validation failed * @retval 0 Validation failed (cbret set)
* @retval -1 Error * @retval -1 Error
*/ */
int int
@ -390,67 +450,37 @@ clixon_module_upgrade(clicon_handle h,
cbuf *cbret) cbuf *cbret)
{ {
int retval = -1; int retval = -1;
cxobj *xc; /* XML child of data */ char *ns; /* Namespace */
char *namespace; cxobj *xs; /* XML module state */
cxobj *xs; /* XML module state */
char *xname; /* XML top-level symbol name */
int state; /* 0: no changes, 1: deleted, 2: modified */
char *modname;
yang_spec *yspec;
yang_stmt *ymod;
yang_stmt *yrev;
char *rev;
uint32_t from;
uint32_t to;
int ret; int ret;
/* Iterate through db XML top-level - get namespace info */ if (msd == NULL)
xc = NULL; goto ok;
while ((xc = xml_child_each(xt, xc, CX_ELMNT)) != NULL) { /* Iterate through xml modified module state */
xname = xml_name(xc); /* xml top-symbol name */ xs = NULL;
if (xml2ns(xc, NULL, &namespace) < 0) /* Get namespace of XML */ while ((xs = xml_child_each(msd->md_mod, xs, CX_ELMNT)) != NULL) {
/* Extract namespace */
if ((ns = xml_find_body(xs, "namespace")) == NULL)
goto done; goto done;
if (namespace == NULL){ /* Extract revisions and make callbacks */
clicon_log(LOG_DEBUG, "XML %s lacks namespace", xname); if ((ret = mod_ns_upgrade(h, xt, xs, ns, 0, cbret)) < 0)
goto fail;
}
/* Look up module-state via namespace of XML */
state = 0; /* XML matches system modules */
if (msd){
if ((xs = xpath_first(msd->md_del, "module[namespace=\"%s\"]", namespace)) != NULL)
state = 1; /* XML belongs to a removed module */
else if ((xs = xpath_first(msd->md_mod, "module[namespace=\"%s\"]", namespace)) != NULL)
state = 2; /* XML belongs to an outdated module */
}
/* Pick up more data from data store module-state */
from = to = 0;
modname = NULL;
if (state && xs && msd){ /* sanity: XXX what about no msd?? */
modname = xml_find_body(xs, "name"); /* Module name */
if ((rev = xml_find_body(xs, "revision")) != NULL) /* Module revision */
if (ys_parse_date_arg(rev, &from) < 0)
goto done;
if (state > 1){
yspec = clicon_dbspec_yang(h);
/* Look up system module (alt send it via argument) */
if ((ymod = yang_find_module_by_name(yspec, modname)) == NULL)
goto fail;
if ((yrev = yang_find((yang_node*)ymod, Y_REVISION, NULL)) == NULL)
goto fail;
if (ys_parse_date_arg(yrev->ys_argument, &to) < 0)
goto done;
}
}
/* Make upgrade callback for this XML, specifying the module name,
* namespace, from and to revision.
* XXX: namespace may be known but not module!!
*/
if ((ret = upgrade_callback_call(h, xc, modname, namespace, from, to, NULL)) < 0)
goto done; goto done;
if (ret == 0) if (ret == 0)
goto fail; goto fail;
} }
/* Iterate through xml deleted module state */
xs = NULL;
while ((xs = xml_child_each(msd->md_del, xs, CX_ELMNT)) != NULL) {
/* Extract namespace */
if ((ns = xml_find_body(xs, "namespace")) == NULL)
continue;
/* Extract revisions and make callbacks (now w deleted=1) */
if ((ret = mod_ns_upgrade(h, xt, xs, ns, 1, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
}
ok:
retval = 1; retval = 1;
done: done:
return retval; return retval;

1
test/test_order.sh
View file

@ -36,7 +36,6 @@ cat <<EOF > $cfg
<CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK> <CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK>
<CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR> <CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE> <CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_CLI_GENMODEL_COMPLETION>1</CLICON_CLI_GENMODEL_COMPLETION>
<CLICON_XMLDB_DIR>$dbdir</CLICON_XMLDB_DIR> <CLICON_XMLDB_DIR>$dbdir</CLICON_XMLDB_DIR>
<CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN> <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
</clixon-config> </clixon-config>

267
test/test_upgrade_changelog.sh → test/test_upgrade_auto.sh
View file

@ -3,7 +3,7 @@
# Ways of changes (operation-type) are: # Ways of changes (operation-type) are:
# create, delete, move, modify # create, delete, move, modify
# In this example, example-a has the following changes: # In this example, example-a has the following changes:
# - Create y, delete x, modify host-name, move z # - Create y, delete x, replace host-name, move z
# example-b is completely obsoleted # example-b is completely obsoleted
# Magic line must be first in script (see README.md) # Magic line must be first in script (see README.md)
@ -16,84 +16,111 @@ changelog=$dir/changelog.xml # Module revision changelog
changelog2=$dir/changelog2.xml # From draft appendix changelog2=$dir/changelog2.xml # From draft appendix
exa01y=$dir/example-a@2017-12-01.yang exa01y=$dir/example-a@2017-12-01.yang
exa20y=$dir/example-a@2017-12-20.yang exa20y=$dir/example-a@2017-12-20.yang
exb20y=$dir/example-b@2017-12-20.yang
# draft-wang-netmod-module-revision-management-01 # draft-wang-netmod-module-revision-management-01
# 3.2.1 and 4.1 example-a revision 2017-12-01 # 3.2.1 and 4.1 example-a revision 2017-12-01
cat <<EOF > $exa01y cat <<EOF > $exa01y
module example-a{ module example-a{
yang-version 1.1; yang-version 1.1;
namespace "urn:example:a"; namespace "urn:example:a";
prefix "a"; prefix "a";
organization "foo."; organization "foo.";
contact "fo@example.com"; contact "fo@example.com";
description description
"foo."; "foo.";
revision 2017-12-01 { revision 2017-12-01 {
description "Initial revision."; description "Initial revision.";
} }
container system { container system {
leaf a { leaf a {
type string; type string;
description "no change"; description "no change";
} }
leaf x { leaf b {
type string; type string;
description "delete"; description "rename tag";
} }
leaf host-name { leaf x {
type uint32; type string;
description "modify type"; description "delete";
} }
leaf z { leaf host-name {
description "move to alt"; type uint32;
type string; description "modify type";
} }
} leaf z {
container alt { description "move to alt";
} type string;
} }
}
container alt {
}
}
EOF EOF
# 3.2.1 and 4.1 example-a revision 2017-12-20 # 3.2.1 and 4.1 example-a revision 2017-12-20
cat <<EOF > $exa20y cat <<EOF > $exa20y
module example-a{ module example-a {
yang-version 1.1; yang-version 1.1;
namespace "urn:example:a"; namespace "urn:example:a";
prefix "a"; prefix "a";
organization "foo."; organization "foo.";
contact "fo@example.com"; contact "fo@example.com";
description description
"foo."; "foo.";
revision 2017-12-20 { revision 2017-12-20 {
description "Create y, delete x, modify host-name, move z"; description "Create y, delete x, replace host-name, move z";
} }
revision 2017-12-01 { revision 2017-12-01 {
description "Initial revision."; description "Initial revision.";
} }
container system { container system {
leaf a { leaf a {
type string; type string;
description "no change"; description "no change";
} }
leaf host-name { leaf c {
type string; type string;
description "modify type"; description "rename tag";
} }
leaf y { leaf host-name {
type string; type string;
description "create"; description "replace";
} }
} leaf y {
container alt { type string;
leaf z { description "create";
description "move to alt"; }
type string; }
} container alt {
} leaf z {
description "move to alt";
type string;
}
}
}
EOF
# 3.2.1 and 4.1 example-a revision 2017-12-20
cat <<EOF > $exb20y
module example-b {
yang-version 1.1;
namespace "urn:example:b";
prefix "b";
organization "foo.";
contact "fo@example.com";
description
"foo.";
revision 2017-12-20 {
description "Remove all";
}
} }
EOF EOF
@ -125,16 +152,23 @@ cat <<EOF > $dir/startup_db
</modules-state> </modules-state>
<system xmlns="urn:example:a"> <system xmlns="urn:example:a">
<a>dont change me</a> <a>dont change me</a>
<b>rename me</b>
<host-name>modify me</host-name> <host-name>modify me</host-name>
<x>remove me</x> <x>remove me</x>
<z>move me</z> <z>move me</z>
</system> </system>
<alt xmlns="urn:example:a">
</alt>
<system-b xmlns="urn:example:b"> <system-b xmlns="urn:example:b">
<b>Obsolete</b> <b>Obsolete</b>
</system-b> </system-b>
</config> </config>
EOF EOF
# Wanted new XML
XML='<system xmlns="urn:example:a"><a>dont change me</a><c>rename me</c><host-name>i am modified</host-name><y>created</y></system><alt xmlns="urn:example:a"><z>move me</z></alt>'
# Create configuration # Create configuration
cat <<EOF > $cfg cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config"> <clixon-config xmlns="http://clicon.org/config">
@ -147,8 +181,8 @@ cat <<EOF > $cfg
<CLICON_XMLDB_DIR>$dir</CLICON_XMLDB_DIR> <CLICON_XMLDB_DIR>$dir</CLICON_XMLDB_DIR>
<CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN> <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
<CLICON_XMLDB_MODSTATE>true</CLICON_XMLDB_MODSTATE> <CLICON_XMLDB_MODSTATE>true</CLICON_XMLDB_MODSTATE>
<CLICON_MODULE_REVISION>true</CLICON_MODULE_REVISION> <CLICON_XML_CHANGELOG>true</CLICON_XML_CHANGELOG>
<CLICON_MODULE_REVISION_CHANGELOG>$changelog</CLICON_MODULE_REVISION_CHANGELOG> <CLICON_XML_CHANGELOG_FILE>$changelog</CLICON_XML_CHANGELOG_FILE>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR> <CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR> <CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE> <CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
@ -157,54 +191,52 @@ EOF
# Changelog of example-a: # Changelog of example-a:
cat <<EOF > $changelog cat <<EOF > $changelog
<yang-modules xmlns="http://clicon.org/yang-changelog"> <changelogs xmlns="http://clicon.org/xml-changelog">
<module> <changelog>
<name>example-b</name> <namespace>urn:example:b</namespace>
<revision>2017-12-01</revision> <revfrom>2017-12-01</revfrom>
<!--obsolete-->
</module>
<module>
<name>example-a</name>
<revision>2017-12-20</revision> <revision>2017-12-20</revision>
<backward-compatible>true</backward-compatible> <step>
<revision-change-log> <name>1</name>
<index>0001</index> <op>delete</op>
<change-operation>create</change-operation> <where>/b:system-b</where>
<data-definition> </step>
<target-node> </changelog>
/a:system/a:y; <changelog>
</target-node> <namespace>urn:example:a</namespace>
</data-definition> <revfrom>2017-12-01</revfrom>
</revision-change-log> <revision>2017-12-20</revision>
<revision-change-log> <step>
<index>0002</index> <name>0</name>
<change-operation>delete</change-operation> <op>rename</op>
<data-definition> <where>/a:system/a:b</where>
<target-node> <tag>"c"</tag>
/a:system/a:x; </step>
</target-node> <step>
</data-definition> <name>1</name>
</revision-change-log> <op>insert</op>
<revision-change-log> <where>/a:system</where>
<index>0003</index> <new><y>created</y></new>
<change-operation>modify</change-operation> </step>
<data-definition> <step>
<target-node> <name>2</name>
/a:system/a:host-name; <op>delete</op>
</target-node> <where>/a:system/a:x</where>
</data-definition> </step>
</revision-change-log> <step>
<revision-change-log> <name>3</name>
<index>0004</index> <op>replace</op>
<change-operation>move</change-operation> <where>/a:system/a:host-name</where>
<data-definition> <new><host-name>i am modified</host-name></new>
<target-node> </step>
/a:system/a:z; <step>
</target-node> <name>4</name>
</data-definition> <op>move</op>
</revision-change-log> <where>/a:system/a:z</where>
</module> <dst>/a:alt</dst>
</yang-modules> </step>
</changelog>
</changelogs>
EOF EOF
# Start new system from old datastore # Start new system from old datastore
@ -234,9 +266,8 @@ start_restconf -f $cfg
new "waiting" new "waiting"
sleep $RCWAIT sleep $RCWAIT
new "Check failsafe (work in progress)"
new "Check running db content" new "Check running db content"
expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' '^<rpc-reply><data><system xmlns="urn:example:a"><a>Failsafe</a></system></data></rpc-reply>]]>]]>$' expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' "^<rpc-reply><data>$XML</data></rpc-reply>]]>]]>$"
new "Kill restconf daemon" new "Kill restconf daemon"
stop_restconf stop_restconf

333
test/test_upgrade_interfaces.sh Executable file
View file

@ -0,0 +1,333 @@
#!/bin/bash
# Upgrade a module by registering a manually programmed callback
# The usecase is insipred by the ietf-interfaces upgrade from
# 2014-05-08 to 2018-02-20.
# That includes moving parts from interfaces-state to interfaces and then
# deprecating the whole /interfaces-state tree.
# A preliminary change list is in Appendix A of
# draft-wang-netmod-module-revision-management-01
# The example here is simplified and also extended.
# It has also been broken up into two parts to test a series of upgrades.
# These are the operations (authentic):
# Move /if:interfaces-state/if:interface/if:admin-status to
# /if:interfaces/if:interface/
# Move /if:interfaces-state/if:interface/if:statistics to
# if:interfaces/if:interface/
# Delete /if:interfaces-state
# These are extra added for test:
# Rename /interfaces/interface/description to /interfaces/interface/descr
# Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr
# Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000
#
# Magic line must be first in script (see README.md)
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi
APPNAME=example
cfg=$dir/conf.xml
if2014=$dir/interfaces@2014-05-08.yang
if2018=$dir/interfaces@2018-02-20.yang
# Original simplified version - note all is config to allow for storing in
# datastore
cat <<EOF > $if2014
module interfaces{
yang-version 1.1;
namespace "urn:example:interfaces";
prefix "if";
import ietf-yang-types {
prefix yang;
}
revision 2014-05-08 {
description
"Initial revision.";
reference
"RFC 7223: A YANG Data Model for Interface Management";
}
feature if-mib {
description
"This feature indicates that the device implements
the IF-MIB.";
reference
"RFC 2863: The Interfaces Group MIB";
}
container interfaces {
description
"Interface configuration parameters.";
list interface {
key "name";
leaf name {
type string;
}
leaf description {
type string;
}
leaf type {
type string;
mandatory true;
}
leaf link-up-down-trap-enable {
if-feature if-mib;
type enumeration {
enum enabled;
enum disabled;
}
}
}
}
container interfaces-state {
list interface {
key "name";
leaf name {
type string;
}
leaf admin-status {
if-feature if-mib;
type enumeration {
enum up;
enum down;
enum testing;
}
mandatory true;
}
container statistics {
leaf in-octets {
type yang:counter64;
}
leaf in-unicast-pkts {
type yang:counter64;
}
}
}
}
}
EOF
cat <<EOF > $if2018
module interfaces{
yang-version 1.1;
namespace "urn:example:interfaces";
prefix "if";
import ietf-yang-types {
prefix yang;
}
revision 2018-02-20 {
description
"Updated to support NMDA.";
reference
"RFC 8343: A YANG Data Model for Interface Management";
}
revision 2014-05-08 {
description
"Initial revision.";
reference
"RFC 7223: A YANG Data Model for Interface Management";
}
feature if-mib {
description
"This feature indicates that the device implements
the IF-MIB.";
reference
"RFC 2863: The Interfaces Group MIB";
}
container interfaces {
description
"Interface configuration parameters.";
list interface {
key "name";
leaf name {
type string;
}
container docs{
description "Original description is wrapped and renamed";
leaf descr {
type string;
}
}
leaf type {
type string;
mandatory true;
}
leaf link-up-down-trap-enable {
if-feature if-mib;
type enumeration {
enum enabled;
enum disabled;
}
}
leaf admin-status {
if-feature if-mib;
type enumeration {
enum up;
enum down;
enum testing;
}
mandatory true;
}
container statistics {
leaf in-octets {
type decimal64{
fraction-digits 3;
}
}
leaf in-unicast-pkts {
type yang:counter64;
}
}
}
}
}
EOF
# Create startup db revision example-a and example-b 2017-12-01
# this should be automatically upgraded to 2017-12-20
cat <<EOF > $dir/startup_db
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>interfaces</name>
<revision>2014-05-08</revision>
<namespace>urn:example:interfaces</namespace>
</module>
</modules-state>
<interfaces xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<type>eth</type>
<description>First interface</description>
</interface>
<interface>
<name>e1</name>
<type>eth</type>
</interface>
</interfaces>
<interfaces-state xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<admin-status>up</admin-status>
<statistics>
<in-octets>54326432</in-octets>
<in-unicast-pkts>8458765</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<admin-status>down</admin-status>
</interface>
<interface>
<name>e2</name>
<admin-status>testing</admin-status>
</interface>
</interfaces-state>
</config>
EOF
# Wanted new XML
# Note interface e2 is not moved
cat <<EOF > $dir/wanted
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>interfaces</name>
<revision>2018-02-20</revision>
<namespace>urn:example:interfaces</namespace>
</module>
</modules-state>
<interfaces xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<type>eth</type>
<admin-status>up</admin-status>
<docs><descr>First interface</descr></docs>
<statistics>
<in-octets>54326.432</in-octets>
<in-unicast-pkts>8458765</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<type>eth</type>
<admin-status>down</admin-status>
</interface>
</interfaces>
</config>
EOF
XML='<interfaces xmlns="urn:example:interfaces"><interface><name>e0</name><docs><descr>First interface</descr></docs><type>eth</type><admin-status>up</admin-status><statistics><in-octets>54326.432</in-octets><in-unicast-pkts>8458765</in-unicast-pkts></statistics></interface><interface><name>e1</name><type>eth</type><admin-status>down</admin-status></interface></interfaces>'
# Create configuration
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
<CLICON_YANG_MAIN_DIR>$dir</CLICON_YANG_MAIN_DIR>
<CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK>
<CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>$dir</CLICON_XMLDB_DIR>
<CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
<CLICON_XMLDB_MODSTATE>true</CLICON_XMLDB_MODSTATE>
<CLICON_XML_CHANGELOG>false</CLICON_XML_CHANGELOG>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
</clixon-config>
EOF
# Start new system from old datastore
mode=startup
# -u means trigger example upgrade
new "test params: -s $mode -f $cfg -- -u"
# Bring your own backend
if [ $BE -ne 0 ]; then
# kill old backend (if any)
new "kill old backend"
sudo clixon_backend -zf $cfg
if [ $? -ne 0 ]; then
err
fi
new "start backend -s $mode -f $cfg -- -u"
start_backend -s $mode -f $cfg -- -u
fi
new "waiting"
sleep $RCWAIT
new "kill old restconf daemon"
sudo pkill -u www-data clixon_restconf
new "start restconf daemon"
start_restconf -f $cfg
new "waiting"
sleep $RCWAIT
new "Check running db content"
expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' "^<rpc-reply><data>$XML</data></rpc-reply>]]>]]>$"
new "Kill restconf daemon"
stop_restconf
if [ $BE -ne 0 ]; then
new "Kill backend"
# Check if premature kill
pid=`pgrep -u root -f clixon_backend`
if [ -z "$pid" ]; then
err "backend already dead"
fi
# kill backend
stop_backend -f $cfg
rm -rf $dir
fi

2
yang/clixon/Makefile.in
View file

@ -43,7 +43,7 @@ CLIXON_DATADIR = @CLIXON_DATADIR@
YANGSPECS = clixon-config@2019-03-05.yang YANGSPECS = clixon-config@2019-03-05.yang
YANGSPECS += clixon-lib@2019-01-02.yang YANGSPECS += clixon-lib@2019-01-02.yang
YANGSPECS += clixon-rfc5277@2008-07-01.yang YANGSPECS += clixon-rfc5277@2008-07-01.yang
YANGSPECS += clixon-yang-changelog@2019-03-21.yang YANGSPECS += clixon-xml-changelog@2019-03-21.yang
APPNAME = clixon # subdir ehere these files are installed APPNAME = clixon # subdir ehere these files are installed

24
yang/clixon/clixon-config@2019-03-05.yang
View file

@ -392,6 +392,18 @@ module clixon-config {
info. When loaded at startup, a check is made if the system info. When loaded at startup, a check is made if the system
yang modules match"; yang modules match";
} }
leaf CLICON_XML_CHANGELOG {
type boolean;
default false;
description "If true enable automatic upgrade using yang clixon
changelog.";
}
leaf CLICON_XML_CHANGELOG_FILE {
type string;
description "Name of file with module revision changelog.
If CLICON_XML_CHANGELOG is true, Clixon
reads the module changelog from this file.";
}
leaf CLICON_USE_STARTUP_CONFIG { leaf CLICON_USE_STARTUP_CONFIG {
type int32; type int32;
default 0; default 0;
@ -428,18 +440,6 @@ module clixon-config {
data. If enabled, module info will appear when doing data. If enabled, module info will appear when doing
netconf get or restconf GET"; netconf get or restconf GET";
} }
leaf CLICON_YANG_CHANGELOG {
type boolean;
default false;
description "If true enable automatic upgrade using yang clixon
changelog.";
}
leaf CLICON_YANG_CHANGELOG_FILE {
type string;
description "Name of file with module revision changelog.
If CLICON_YANG_CHANGELOG is true, Clixon
reads the module changelog from this file.";
}
leaf CLICON_MODULE_SET_ID { leaf CLICON_MODULE_SET_ID {
type string; type string;
default "0"; default "0";

145
yang/clixon/clixon-xml-changelog@2019-03-21.yang Normal file
View file

@ -0,0 +1,145 @@
module clixon-xml-changelog {
yang-version 1.1;
namespace "http://clicon.org/xml-changelog";
prefix ml;
import ietf-yang-library {
prefix yanglib;
}
import ietf-yang-types {
prefix yang;
}
organization "Clixon";
contact
"Olof Hagsand <olof@hagsand.se>";
description
"This is experimental XML changelog module with several influences:
1) draft-wang-netmod-module-revision-management-01, by:
Qin Wu <mailto:bill.wu@huawei.com>
Zitao Wang <mailto:wangzitao@huawei.com>
2) XProc https://www.w3.org/TR/xproc/#xpath-context";
revision 2019-03-21 {
description
"Initial Clixon derived version";
}
typedef operation_type {
description
"From: https://en.wikipedia.org/wiki/XML_pipeline:
Rename - renames elements or attributes without modifying the content
Replace - replaces elements or attributes
Insert - adds a new data element to the output stream at a specified point
Delete - removes an element or attribute (also known as pruning the input tree)
Wrap - wraps elements with additional elements
Reorder - changes the order of elements
More inspiration in XProc: https://www.w3.org/TR/xproc/#ex2";
type enumeration{
enum rename {
description
"Rename the 'where' node, ie XML label
Synopsis: rename(where:targets, when:bool, tag:string)";
}
enum replace {
description
"Replace the target data node modification is given by the leaf
transform which is a string with %s where the original value
is inserted.
Synopsis: replace(where:targets, when:bool, new:xml)";
}
enum insert {
description
"Create new data nodes and insert under an existing node.
Synopsis: insert(where:parents, when:bool, new:xml)";
}
enum delete {
description
"Delete the target node.
Synopsis: delete(where:parents, when:bool)";
}
enum move {
description
"Move the target node(Added).
Synopsis: move(where:parents, when:bool, dst:node)";
}
}
}
container changelogs {
config false;
list changelog {
key "namespace revision";
leaf namespace {
type string;
description
"The YANG namespace identifying a module or submodule.
XML needs to be identified by namespace, translation to
module name may not always be possible.";
}
leaf revision {
type yanglib:revision-identifier;
description
"The YANG module or submodule revision date.
This is the actual date of the changlelog items.
Note however if the terminate flag is set, this is a virtual
revision just in place to terminate the XML, such as removing or
moving items,.";
}
leaf revfrom {
type yanglib:revision-identifier;
description
"Optional revision from date. This changelog is effective in the
range [from,to]. If from is not given the changelog is open-ended.
Several changelogs may be applied if the upgrade spans multiple
ranges: [from0,to0],..[fromN,toN]";
}
list step {
description
"List for module revision change log";
key "name";
leaf name {
type string;
description
"Unique step name";
}
leaf op {
type operation_type;
mandatory true;
description
"This leaf indicate the change operation, such as create, move, delete, modify, etc.";
}
leaf where {
type yang:xpath1.0;
mandatory true;
description
"Identifies the target data node for update.
for move, modify or delete the target-node points to
the data node of the old version.
For create, it is the parent where it should be
inserted.";
}
leaf when {
type yang:xpath1.0;
description
"Boolean XPATH. Execute this step if this xpath exists
and evaluates to true";
}
leaf tag {
description
"For rename, a string XPath definining the new tag.";
type yang:xpath1.0;
}
leaf dst {
description
"For move, a destination XPath definining the parent where
to insert.";
type yang:xpath1.0;
}
anydata new {
description
"If op is replace or insert, new XML for the new node.";
}
}
}
}
}

326
yang/clixon/clixon-yang-changelog@2019-03-21.yang
View file

@ -1,326 +0,0 @@
module clixon-yang-changelog {
yang-version 1.1;
namespace "http://clicon.org/yang-changelog";
prefix ml;
import ietf-yang-library {
prefix yanglib;
}
import ietf-yang-types {
prefix yang;
}
organization "Clixon";
contact
"Olof Hagsand <olof@hagsand.se>";
description
"This is experimentalYANG changelog derived from:
draft-wang-netmod-module-revision-management-01
with the following contacts and references:
WG Web: <https://datatracker.ietf.org/wg/netmod/>
WG List: <mailto:netmod@ietf.org>
Author: Qin Wu
<mailto:bill.wu@huawei.com>
Zitao Wang
<mailto:wangzitao@huawei.com>";
reference "draft-wang-netmod-module-revision-management-01";
revision 2019-03-21 {
description
"Initial Clixon derived version";
}
identity operation-type {
description
"Abstract base identity for the operation type ";
}
identity create {
base operation-type;
description
"Denotes create new data nodes";
}
identity delete {
base operation-type;
description
"Denotes delete the target node";
}
identity move {
base operation-type;
description
"Denote move the target node.";
}
identity modify {
base operation-type;
description
"Denote modify the target data node.";
}
identity statement-type {
description
"Base identity for statement type";
}
identity feature-statement {
base statement-type;
description
"feature statement, if this type be chose, it means that the
feature or if-feature statement been modified";
}
identity identity-statement {
base statement-type;
description
"identity statement, if this type be chose, it means that the
identity statement been modified, for example, add new identity, etc.";
}
identity grouping-statement {
base statement-type;
description
"grouping statement, if this type be chose, it means that the grouping
statement been modified.";
}
identity typedef-statement {
base statement-type;
description
"typedef statement, if this type be chose, it means that the typedef
statement been modified.";
}
identity augment-statement {
base statement-type;
description
"augment statement, if this type be chose, it means that the augment
statement been modified.";
}
identity rpc-statement {
base statement-type;
description
"rpc statement, if this type be chose, it means that the rpc
statement been modified.";
}
identity notification-statement {
base statement-type;
description
"notification statement, if this type be chose, it means that the notification
statement been modified.";
}
extension purpose {
argument name;
description
"The purpose can be used to mark the data nodes change purpose.
The name argument can be specified in the following recommended mode
- bug-fix, which can help user to understand the data nodes' changes present bug fix,
- new-function, which can help user to understand the data nodes' changes present new function,
- nmda-conform, which can help user to understand the data nodes' changes conform to NMDA,
and note that the user can argument the purpose name according to their sepcific requirements.";
}
grouping data-definition {
container data-definition {
leaf target-node {
type yang:xpath1.0;
mandatory true;
description
"Identifies the target data node for update.
Notice that, if the update-type equal to move or delete,
this target-node must point to the data node of old version.
\t
For example, suppose the target node is a YANG leaf named a,
and the previous version is:
\t
container foo {
leaf a { type string; }
leaf b { type int32; }
}
\t
the new version is:
container foo {
leaf b {type int32;}
}
\t
Therefore, the targe-node should be /foo/a.";
}
leaf location-point {
type yang:xpath1.0;
description
"Identifies the location point where the updates happened.";
}
leaf where {
when "derived-from-or-self(../../change-operation, 'move')" {
description
"This leaf only applies for 'move'
updates.";
}
type enumeration {
enum "before" {
description
"Insert or move a data node before the data resource
identified by the 'point' parameter.";
}
enum "after" {
description
"Insert or move a data node after the data resource
identified by the 'point' parameter.";
}
enum "first" {
description
"Insert or move a data node so it becomes ordered
as the first entry.";
}
enum "last" {
description
"Insert or move a data node so it becomes ordered
as the last entry.";
}
}
default "last";
description
"Identifies where a data resource will be inserted
or moved.";
}
anydata data-definition {
when "derived-from-or-self(../../change-operation, 'modify')" {
description
"This nodes only be present when
the 'change-operation' equal to 'modify'.";
}
description
"This nodes used for present the definitions before updated.
And this nodes only be present when
the 'change-operation' equal to 'modify'.";
}
description
"Container for data statement";
}
description
"Grouping for data definition";
}
grouping other-statement {
container other-statement {
leaf statement-name {
type identityref {
base statement-type;
}
description
"Statement name, for example, identity, feature, typedef, etc.";
}
anydata statement-definition {
description
"This nodes used for present new the definitions.";
}
list substatements {
key "statement-name";
leaf statement-name {
type identityref {
base statement-type;
}
description
"Statement name, for example, identity, feature, typedef, etc.";
}
anydata substatement-definition {
description
"This nodes used for present new the definitions.";
}
description
"List for substatements updates";
}
description
"Container for header statement updates";
}
description
"Grouping for header statement";
}
grouping change-log {
list revision-change-log {
key "index";
leaf index {
type uint32;
description
"Index for module change log";
}
leaf change-operation {
type identityref {
base operation-type;
}
mandatory true;
description
"This leaf indicate the change operation, such as create, move, delete, modify, etc.";
}
choice yang-statements {
description
"Choice for various YANG statements that have been impacted.";
case data-definition-statement {
uses data-definition;
}
case other-statement {
uses other-statement;
}
}
description
"List for module revision change log";
}
description
"Grouping for module revision change log";
}
container yang-modules {
config false;
list module {
key "name revision";
leaf name {
type yang:yang-identifier;
description
"The YANG module or submodule name.";
}
leaf revision {
type yanglib:revision-identifier;
description
"The YANG module or submodule revision date. If no revision
statement is present in the YANG module or submodule, this
leaf is not instantiated.";
}
leaf backward-compatible {
type boolean;
description
"Indicates whether it is a backward compatible version.
If this parameter is set to true, it means that this version is
a backwards compatible version";
}
uses change-log;
description
"List for module updated log";
}
description
"This container present the modules updated log.";
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module" {
description
"Augment the yang library with backward compatibility indication.";
leaf backward-compatible {
type boolean;
description
"backward compatibility indication.";
}
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module/yanglib:submodule" {
description
"Augment the yang library with backward compatibility indication.";
leaf backward-compatible {
type boolean;
description
"backward compatibility indication.";
}
}
}