experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

* New yang changelog experimental feature for automatic upgrade

Browse source 
* Added modules-state diff parameter to xmldb_get datastore function for startup scenarios.
* Allowed Yang extended Xpath functions (syntax only):
  * re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
* XSD regular expression handling of dash(`-`)
  *: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`.
* YANG Anydata treated same as Anyxml
This commit is contained in:
Olof hagsand 2019-03-21 17:42:53 +01:00
parent 434f0b930e
commit 3f68cca06c
37 changed files with 1475 additions and 351 deletions
Download patch file Download diff file Expand all files Collapse all files

17
CHANGELOG.md
View file

@ -1,6 +1,6 @@
# Clixon Changelog
## 3.10.0 (Upcoming)
## 3.10.0/4.0.0 (Upcoming)
### Major New features
* Persistent CLI history: [Preserve CLI command history across sessions. The up/down arrows](https://github.com/clicon/clixon/issues/79)
@ -18,16 +18,22 @@
* Check which modules match, and which do not.
* Loading of "extra" XML.
* Detection of in-compatible XML and Yang models in the startup configuration.
* An upgrade callback when in-compatible XML is encountered (`ca_upgrade`)
* A user can register upgrade callbacks per module/revision when in-compatible XML is encountered (`update_callback_register`).
* A "failsafe" mode allowing a user to repair the startup on errors or failed validation.
* Major rewrite of `backend_main.c` and a new module `backend_startup.c`
* New yang changelog experimental feature for automatic upgrade
* Yang module clixon-yang-changelog@2019-03-21.yang based on draft-wang-netmod-module-revision-management-01
* Two config options control:
* CLICON_YANG_CHANGELOG enables the yang changelog feature
* CLICON_YANG_CHANGELOG_FILE where the changelog resides
* Datastore files contain RFC7895 module-state information
* Added modules-state parameter to xmldb_get datastore function
* Added modules-state diff parameter to xmldb_get datastore function
* Set config option `CLICON_XMLDB_MODSTATE` to true
* Enable this if you wish to use the upgrade feature in the new startup functionality.
* Note that this adds bytes to your configs
### API changes on existing features (you may need to change your code)
* Added modules-state diff parameter to xmldb_get datastore function for startup scenarios. Set this to NULL in normal cases.
* `rpc_callback_register` added a namespace parameter. Example:
```
rpc_callback_register(h, empty_rpc, NULL, "urn:example:clixon", "empty");
@ -62,6 +68,11 @@
* Added libgen.h for baseline()
### Corrected Bugs
* Allowed Yang extended Xpath functions (syntax only):
* re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
* XSD regular expression handling of dash(`-`)
*: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`.
* YANG Anydata treated same as Anyxml
* Bugfix: [Nodes from more than one of the choice's branches exist at the same time](https://github.com/clicon/clixon/issues/81)
* Note it may still be possible to load a file with multiple choice elements via netconf, but it will not pass validate.
* Bugfix: Default NACM policies applied even if NACM is disabled

1
README.md
View file

@ -113,6 +113,7 @@ However, the following YANG syntax modules are not implemented:
- unique
- action
- refine
- Yang extended Xpath functions: re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
Restrictions on Yang types are as follows:
- Submodules cannot re-use a prefix in an import statement that is already used for another imported module in the module that the submodule belongs to. (see https://github.com/clicon/clixon/issues/60)

16
apps/backend/backend_commit.c
View file

@ -273,7 +273,7 @@ startup_validate(clicon_handle h,
int retval = -1;
yang_spec *yspec;
int ret;
cxobj *xms = NULL;
modstate_diff_t *msd = NULL;
transaction_data_t *td = NULL;
/* Handcraft a transition with only target and add trees */
@ -282,11 +282,17 @@ startup_validate(clicon_handle h,
/* 2. Parse xml trees
* This is the state we are going to
* Note: xmsdiff contains non-matching modules
* Only if CLICON_XMLDB_MODSTATE is enabled
*/
if (xmldb_get(h, db, "/", 1, &td->td_target, &xms) < 0)
if (clicon_option_bool(h, "CLICON_XMLDB_MODSTATE"))
if ((msd = modstate_diff_new()) == NULL)
goto done;
if (xmldb_get(h, db, "/", 1, &td->td_target, msd) < 0)
goto done;
if (xms && clixon_plugin_upgrade(h, xms) < 0)
if ((ret = clixon_module_upgrade(h, td->td_target, msd, cbret)) < 0)
goto done;
if (ret == 0)
goto fail;
/* Handcraft transition with with only add tree */
if (cxvec_append(td->td_target, &td->td_avec, &td->td_alen) < 0)
@ -318,8 +324,8 @@ startup_validate(clicon_handle h,
done:
if (td)
transaction_free(td);
if (xms)
xml_free(xms);
if (msd)
modstate_diff_free(msd);
return retval;
fail: /* cbret should be set */
if (cbuf_len(cbret)==0){

10
apps/backend/backend_main.c
View file

@ -96,7 +96,10 @@ backend_terminate(clicon_handle h)
clicon_debug(1, "%s", __FUNCTION__);
if ((ss = clicon_socket_get(h)) != -1)
close(ss);
modules_state_cache_set(h, NULL);
if ((x = clicon_module_state_get(h)) != NULL)
xml_free(x);
if ((x = clicon_yang_changelog_get(h)) != NULL)
xml_free(x);
if ((yspec = clicon_dbspec_yang(h)) != NULL)
yspec_free(yspec);
if ((yspec = clicon_config_yang(h)) != NULL)
@ -639,6 +642,11 @@ main(int argc,
if (backend_rpc_init(h) < 0)
goto done;
/* Must be after netconf_module_load, but before startup code */
if (clicon_option_bool(h, "CLICON_YANG_CHANGELOG"))
if (clixon_yang_changelog_init(h) < 0)
goto done;
/* Save modules state of the backend (server). Compare with startup XML */
if (startup_module_state(h, yspec) < 0)
goto done;

25
apps/backend/backend_plugin.c
View file

@ -156,31 +156,6 @@ clixon_plugin_statedata(clicon_handle h,
return retval;
}
/*! Call configuration upgrade routines in backend plugins
* @param[in] h Clicon handle
* @param[in] xms XML tree of module state differences
* @retval 0 OK
* @retval -1 Error in one (first) of user callbacks
*/
int
clixon_plugin_upgrade(clicon_handle h,
cxobj *xmodst)
{
int retval = -1;
clixon_plugin *cp = NULL;
upgrade_cb_t *fn; /* Plugin configuration upgrade fn */
while ((cp = clixon_plugin_each(h, cp)) != NULL) {
if ((fn = cp->cp_api.ca_upgrade) == NULL)
continue;
if (fn(h, xmodst) < 0)
goto done;
}
retval = 0;
done:
return retval;
}
/*! Create and initialize transaction */
transaction_data_t *
transaction_new(void)

1
apps/backend/backend_plugin.h
View file

@ -72,7 +72,6 @@ int backend_plugin_initiate(clicon_handle h);
int clixon_plugin_reset(clicon_handle h, char *db);
int clixon_plugin_statedata(clicon_handle h, yang_spec *yspec, char *xpath, cxobj **xtop);
int clixon_plugin_upgrade(clicon_handle h, cxobj *xmodst);
transaction_data_t * transaction_new(void);
int transaction_free(transaction_data_t *);

78
datastore/clixon_xmldb_text.c
View file

@ -364,12 +364,11 @@ singleconfigroot(cxobj *xt,
static int
text_get_nocache(struct text_handle *th,
const char *db,
char *xpath,
int config,
cxobj **xtop,
cxobj **xms)
const char *db,
char *xpath,
int config,
cxobj **xtop,
modstate_diff_t *msd)
{
int retval = -1;
char *dbfile = NULL;
@ -385,7 +384,6 @@ text_get_nocache(struct text_handle *th,
clicon_err(OE_YANG, ENOENT, "No yang spec");
goto done;
}
if (text_db2file(th, db, &dbfile) < 0)
goto done;
if (dbfile==NULL){
@ -559,7 +557,7 @@ xml_copy_marked(cxobj *x0,
* @param[in] th Datastore text handle
* @param[in] yspec Top-level yang spec
* @param[in] xt XML tree
* @param[out] xmsp If set, return modules-state differences
* @param[out] msd If set, return modules-state differences
*
* Read mst (module-state-tree) from xml tree (if any) and compare it with
* the system state mst.
@ -575,7 +573,7 @@ static int
text_read_modstate(struct text_handle *th,
yang_spec *yspec,
cxobj *xt,
cxobj **xmsp)
modstate_diff_t *msd)
{
int retval = -1;
cxobj *xmodst;
@ -585,16 +583,19 @@ text_read_modstate(struct text_handle *th,
char *name; /* module name */
char *mrev; /* file revision */
char *srev; /* system revision */
cxobj *xmsd = NULL; /* Local modules-state diff tree */
if ((xmodst = xml_find_type(xt, NULL, "modules-state", CX_ELMNT)) == NULL){
/* 1) There is no modules-state info in the file */
}
else if (th->th_modst){
/* Create a diff tree */
if (xml_parse_string("<modules-state xmlns=\"urn:ietf:params:xml:ns:yang:ietf-yang-library\"/>", yspec, &xmsd) < 0)
else if (th->th_modst && msd){
/* Create diff trees */
if (xml_parse_string("<modules-state xmlns=\"urn:ietf:params:xml:ns:yang:ietf-yang-library\"/>", yspec, &msd->md_del) < 0)
goto done;
if (xml_rootchild(xmsd, 0, &xmsd) < 0)
if (xml_rootchild(msd->md_del, 0, &msd->md_del) < 0)
goto done;
if (xml_parse_string("<modules-state xmlns=\"urn:ietf:params:xml:ns:yang:ietf-yang-library\"/>", yspec, &msd->md_mod) < 0)
goto done;
if (xml_rootchild(msd->md_mod, 0, &msd->md_mod) < 0)
goto done;
/* 3) For each module state m in the file */
@ -608,7 +609,7 @@ text_read_modstate(struct text_handle *th,
// fprintf(stderr, "%s: Module %s: not in system\n", __FUNCTION__, name);
if ((xm2 = xml_dup(xm)) == NULL)
goto done;
if (xml_addsub(xmsd, xm2) < 0)
if (xml_addsub(msd->md_del, xm2) < 0)
goto done;
continue;
}
@ -626,7 +627,7 @@ text_read_modstate(struct text_handle *th,
// fprintf(stderr, "%s: Module %s: file \"%s\" and system \"%s\" revisions do not match\n", __FUNCTION__, name, mrev, srev);
if ((xm2 = xml_dup(xm)) == NULL)
goto done;
if (xml_addsub(xmsd, xm2) < 0)
if (xml_addsub(msd->md_mod, xm2) < 0)
goto done;
}
}
@ -639,14 +640,8 @@ text_read_modstate(struct text_handle *th,
if (xml_purge(xmodst) < 0)
goto done;
}
if (xmsp){
*xmsp = xmsd;
xmsd = NULL;
}
retval = 0;
done:
if (xmsd)
xml_free(xmsd);
return retval;
}
@ -655,14 +650,14 @@ text_read_modstate(struct text_handle *th,
* @param[in] db Symbolic database name, eg "candidate", "running"
* @param[in] yspec Top-level yang spec
* @param[out] xp XML tree read from file
* @param[out] xmsp If set, return modules-state differences
* @param[out] msd If set, return modules-state differences
*/
static int
text_readfile(struct text_handle *th,
const char *db,
yang_spec *yspec,
cxobj **xp,
cxobj **xmsp)
modstate_diff_t *msd)
{
int retval = -1;
cxobj *x0 = NULL;
@ -702,7 +697,7 @@ text_readfile(struct text_handle *th,
/* From Clixon 3.10,datastore files may contain module-state defining
* which modules are used in the file.
*/
if (text_read_modstate(th, yspec, x0, xmsp) < 0)
if (text_read_modstate(th, yspec, x0, msd) < 0)
goto done;
if (xp){
*xp = x0;
@ -728,18 +723,18 @@ text_readfile(struct text_handle *th,
* @param[in] xpath String with XPATH syntax. or NULL for all
* @param[in] config If set only configuration data, else also state
* @param[out] xret Single return XML tree. Free with xml_free()
* @param[out] xms If set, return modules-state differences
* @param[out] msd If set, return modules-state differences
* @retval 0 OK
* @retval -1 Error
* @see xmldb_get the generic API function
*/
static int
text_get_cache(struct text_handle *th,
const char *db,
char *xpath,
int config,
cxobj **xtop,
cxobj **xms)
const char *db,
char *xpath,
int config,
cxobj **xtop,
modstate_diff_t *msd)
{
int retval = -1;
yang_spec *yspec;
@ -759,7 +754,7 @@ text_get_cache(struct text_handle *th,
de = hash_value(th->th_dbs, db, NULL);
if (de == NULL || de->de_xml == NULL){ /* Cache miss, read XML from file */
/* If there is no xml x0 tree (in cache), then read it from file */
if (text_readfile(th, db, yspec, &x0t, xms) < 0)
if (text_readfile(th, db, yspec, &x0t, msd) < 0)
goto done;
/* XXX: should we validate file if read from disk?
* Argument against: we may want to have a semantically wrong file and wish
@ -828,24 +823,25 @@ text_get_cache(struct text_handle *th,
* @param[in] xpath String with XPATH syntax. or NULL for all
* @param[in] config If set only configuration data, else also state
* @param[out] xret Single return XML tree. Free with xml_free()
* @param[out] msd If set, return modules-state differences
* @retval 0 OK
* @retval -1 Error
* @see xmldb_get the generic API function
*/
int
text_get(xmldb_handle xh,
const char *db,
char *xpath,
int config,
cxobj **xtop,
cxobj **xms)
text_get(xmldb_handle xh,
const char *db,
char *xpath,
int config,
cxobj **xtop,
modstate_diff_t *msd)
{
struct text_handle *th = handle(xh);
if (th->th_cache)
return text_get_cache(th, db, xpath, config, xtop, xms);
return text_get_cache(th, db, xpath, config, xtop, msd);
else
return text_get_nocache(th, db, xpath, config, xtop, xms);
return text_get_nocache(th, db, xpath, config, xtop, msd);
}
/*! Modify a base tree x0 with x1 with yang spec y according to operation op
@ -1019,7 +1015,7 @@ text_modify(struct text_handle *th,
can be modified in its entirety only.
Any "operation" attributes present on subelements of an anyxml
node are ignored by the NETCONF server.*/
if (y0->yn_keyword == Y_ANYXML){
if (y0->yn_keyword == Y_ANYXML || y0->yn_keyword == Y_ANYDATA){
if (op == OP_NONE)
break;
if (op==OP_MERGE && !permit && xnacm){

2
datastore/clixon_xmldb_text.h
View file

@ -39,7 +39,7 @@
/*
* Prototypes
*/
int text_get(xmldb_handle h, const char *db, char *xpath, int config, cxobj **xtop, cxobj **xms);
int text_get(xmldb_handle h, const char *db, char *xpath, int config, cxobj **xtop, modstate_diff_t *xms);
int text_put(xmldb_handle h, const char *db, enum operation_type op, cxobj *xt, char *username, cbuf *cbret);
int text_dump(FILE *f, char *dbfilename, char *rxkey);
int text_copy(xmldb_handle h, const char *from, const char *to);

50
example/example_backend.c
View file

@ -246,24 +246,35 @@ example_statedata(clicon_handle h,
return retval;
}
/*! Upgrade configuration from one version to another
* @param[in] h Clicon handle
* @param[in] xms Module state differences
* @retval 0 OK
* @retval -1 Error
#ifdef keep_as_example
/*! Registered Upgrade callback function
* @param[in] h Clicon handle
* @param[in] xn XML tree to be updated
* @param[in] modname Name of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
*/
int
example_upgrade(clicon_handle h,
cxobj *xms)
static int
upgrade_all(clicon_handle h,
cxobj *xn,
char *modname,
char *modns,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret)
{
int retval = -1;
if (xms)
clicon_log_xml(LOG_NOTICE, xms, "%s", __FUNCTION__);
retval = 0;
// done:
return retval;
fprintf(stderr, "%s XML:%s mod:%s %s from:%d to:%d\n", __FUNCTION__, xml_name(xn),
modname, modns, from, to);
return 1;
}
#endif
/*! Plugin state reset. Add xml or set state in backend machine.
* Called in each backend plugin. plugin_reset is called after all plugins
@ -366,7 +377,6 @@ static clixon_plugin_api api = {
example_exit, /* exit */
.ca_reset=example_reset, /* reset */
.ca_statedata=example_statedata, /* statedata */
.ca_upgrade=example_upgrade, /* upgrade configuration */
.ca_trans_begin=NULL, /* trans begin */
.ca_trans_validate=transaction_validate,/* trans validate */
.ca_trans_complete=NULL, /* trans complete */
@ -434,6 +444,14 @@ clixon_plugin_init(clicon_handle h)
"copy-config"
) < 0)
goto done;
/* Called after the regular system copy_config callback */
if (upgrade_callback_register(h, yang_changelog_upgrade,
NULL,
NULL, NULL,
0, 0
) < 0)
goto done;
/* Return plugin API */
return &api;

1
lib/clixon/clixon.h.in
View file

@ -90,6 +90,7 @@
#include <clixon/clixon_json.h>
#include <clixon/clixon_netconf_lib.h>
#include <clixon/clixon_nacm.h>
#include <clixon/clixon_yang_changelog.h>
/*
* Global variables generated by Makefile

6
lib/clixon/clixon_options.h
View file

@ -206,8 +206,12 @@ int clicon_startup_status_set(clicon_handle h, enum startup_status status);
int clicon_socket_get(clicon_handle h);
int clicon_socket_set(clicon_handle h, int s);
/*! Set and set module state cache */
/*! Set and get module state cache */
cxobj *clicon_module_state_get(clicon_handle h);
int clicon_module_state_set(clicon_handle h, cxobj *xms);
/*! Set and get module revision changelog */
cxobj *clicon_yang_changelog_get(clicon_handle h);
int clicon_yang_changelog_set(clicon_handle h, cxobj *xchlog);
#endif /* _CLIXON_OPTIONS_H_ */

41
lib/clixon/clixon_plugin.h
View file

@ -67,6 +67,30 @@ typedef int (*clicon_rpc_cb)(
void *regarg
);
/*! Registered Upgrade callback function
* @param[in] h Clicon handle
* @param[in] xn XML tree to be updated
* @param[in] modname Name of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
*/
typedef int (*clicon_upgrade_cb)(
clicon_handle h,
cxobj *xn,
char *modname,
char *modns,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret
);
/*
* Prototypes
*/
@ -128,14 +152,6 @@ enum startup_status{
STARTUP_OK /* Everything OK (may still be modules-mismatch) */
};
/*! Upgrade configuration callback given a diff of yang module state
* @param[in] h Clicon handle
* @param[in] xms XML tree of module state differences
* @retval 0 OK
* @retval -1 Error
*/
typedef int (upgrade_cb_t)(clicon_handle, cxobj *xms);
/* plugin init struct for the api
* Note: Implicit init function
*/
@ -162,7 +178,6 @@ struct clixon_plugin_api{
struct { /* backend-specific */
plgreset_t *cb_reset; /* Reset system status */
plgstatedata_t *cb_statedata; /* Get state data from plugin (backend only) */
upgrade_cb_t *cb_upgrade; /* Upgrade callback */
trans_cb_t *cb_trans_begin; /* Transaction start */
trans_cb_t *cb_trans_validate; /* Transaction validation */
trans_cb_t *cb_trans_complete; /* Transaction validation complete */
@ -181,7 +196,6 @@ struct clixon_plugin_api{
#define ca_auth u.cau_restconf.cr_auth
#define ca_reset u.cau_backend.cb_reset
#define ca_statedata u.cau_backend.cb_statedata
#define ca_upgrade u.cau_backend.cb_upgrade
#define ca_trans_begin u.cau_backend.cb_trans_begin
#define ca_trans_validate u.cau_backend.cb_trans_validate
#define ca_trans_complete u.cau_backend.cb_trans_complete
@ -228,9 +242,12 @@ int clixon_plugin_auth(clicon_handle h, void *arg);
/* rpc callback API */
int rpc_callback_register(clicon_handle h, clicon_rpc_cb cb, void *arg, char *namespace, char *name);
int rpc_callback_delete_all(void);
int rpc_callback_call(clicon_handle h, cxobj *xe, cbuf *cbret, void *arg);
/* upgrade callback API */
int upgrade_callback_register(clicon_handle h, clicon_upgrade_cb cb, void *arg, char *name, char *namespace, uint32_t from, uint32_t to);
int upgrade_callback_delete_all(void);
int upgrade_callback_call(clicon_handle h, cxobj *xt, char *modname, char *modns, uint32_t from, uint32_t to, cbuf *cbret);
#endif /* _CLIXON_PLUGIN_H_ */

4
lib/clixon/clixon_xml_db.h
View file

@ -75,7 +75,7 @@ typedef int (xmldb_getopt_t)(xmldb_handle xh, char *optname, void **value);
typedef int (xmldb_setopt_t)(xmldb_handle xh, char *optname, void *value);
/* Type of xmldb get function */
typedef int (xmldb_get_t)(xmldb_handle xh, const char *db, char *xpath, int config, cxobj **xtop, cxobj **xmodst);
typedef int (xmldb_get_t)(xmldb_handle xh, const char *db, char *xpath, int config, cxobj **xtop, modstate_diff_t *msd);
/* Type of xmldb put function */
typedef int (xmldb_put_t)(xmldb_handle xh, const char *db, enum operation_type op, cxobj *xt, char *username, cbuf *cbret);
@ -138,7 +138,7 @@ int xmldb_connect(clicon_handle h);
int xmldb_disconnect(clicon_handle h);
int xmldb_getopt(clicon_handle h, char *optname, void **value);
int xmldb_setopt(clicon_handle h, char *optname, void *value);
int xmldb_get(clicon_handle h, const char *db, char *xpath, int config, cxobj **xtop, cxobj **xmodst);
int xmldb_get(clicon_handle h, const char *db, char *xpath, int config, cxobj **xtop, modstate_diff_t *msd);
int xmldb_put(clicon_handle h, const char *db, enum operation_type op, cxobj *xt, char *username, cbuf *cbret);
int xmldb_copy(clicon_handle h, const char *from, const char *to);
int xmldb_lock(clicon_handle h, const char *db, int pid);

4
lib/clixon/clixon_yang.h
View file

@ -150,7 +150,7 @@ typedef enum yang_class yang_class;
* data tree. One of container, leaf, leaf-list, list, anydata, and
* anyxml.
*/
#define yang_datanode(y) ((y)->ys_keyword == Y_CONTAINER || (y)->ys_keyword == Y_LEAF || (y)->ys_keyword == Y_LIST || (y)->ys_keyword == Y_LEAF_LIST || (y)->ys_keyword == Y_ANYXML)
#define yang_datanode(y) ((y)->ys_keyword == Y_CONTAINER || (y)->ys_keyword == Y_LEAF || (y)->ys_keyword == Y_LIST || (y)->ys_keyword == Y_LEAF_LIST || (y)->ys_keyword == Y_ANYXML || (y)->ys_keyword == Y_ANYDATA)
/* Yang data definition statement
* See RFC 7950 Sec 3:
@ -283,6 +283,8 @@ int yang_abs_schema_nodeid(yang_spec *yspec, yang_stmt *ys,
enum rfc_6020 keyword, yang_stmt **yres);
int yang_desc_schema_nodeid(yang_node *yn, char *schema_nodeid,
enum rfc_6020 keyword, yang_stmt **yres);
int ys_parse_date_arg(char *datearg, uint32_t *dateint);
cg_var *ys_parse(yang_stmt *ys, enum cv_type cvtype);
int ys_parse_sub(yang_stmt *ys, char *extra);
int yang_mandatory(yang_stmt *ys);

46
lib/clixon/clixon_yang_changelog.h Normal file
View file

@ -0,0 +1,46 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* YANG module revision change management.
* See draft-wang-netmod-module-revision-management-01
*/
#ifndef _CLIXON_YANG_CHANGELOG_H
#define _CLIXON_YANG_CHANGELOG_H
/*
* Prototypes
*/
int yang_changelog_upgrade(clicon_handle h, cxobj *xn, char *modname, char *modns, uint32_t from, uint32_t to, void *arg, cbuf *cbret);
int clixon_yang_changelog_init(clicon_handle h);
#endif /* _CLIXON_YANG_CHANGELOG_H */

11
lib/clixon/clixon_yang_module.h
View file

@ -46,15 +46,26 @@
* Types
*/
/* Struct contataining module state differences between two modules or two
* revisions of same module.
* This is in state of flux so it needss to be conatained and easily changed.
*/
typedef struct {
cxobj *md_del; /* yang mdoule state deletes */
cxobj *md_mod; /* yang mdoule state modifications */
} modstate_diff_t;
/*
* Prototypes
*/
modstate_diff_t * modstate_diff_new(void);
int modstate_diff_free(modstate_diff_t *);
int modules_state_cache_set(clicon_handle h, cxobj *msx);
int yang_modules_init(clicon_handle h);
char *yang_modules_revision(clicon_handle h);
int yang_modules_state_get(clicon_handle h, yang_spec *yspec, char *xpath,
int brief, cxobj **xret);
int clixon_module_upgrade(clicon_handle h, cxobj *xt, modstate_diff_t *msd, cbuf *cb);
#endif /* _CLIXON_YANG_MODULE_H_ */

2
lib/src/Makefile.in
View file

@ -70,7 +70,7 @@ SRC = clixon_sig.c clixon_log.c clixon_err.c clixon_event.c \
clixon_string.c clixon_handle.c \
clixon_xml.c clixon_xml_sort.c clixon_xml_map.c clixon_file.c \
clixon_json.c clixon_yang.c clixon_yang_type.c clixon_yang_module.c \
clixon_yang_cardinality.c \
clixon_yang_cardinality.c clixon_yang_changelog.c \
clixon_hash.c clixon_options.c clixon_plugin.c \
clixon_proto.c clixon_proto_client.c \
clixon_xpath.c clixon_xpath_ctx.c clixon_sha1.c \

1
lib/src/clixon_nacm.c
View file

@ -65,6 +65,7 @@
#include "clixon_netconf_lib.h"
#include "clixon_xpath_ctx.h"
#include "clixon_xpath.h"
#include "clixon_yang_module.h"
#include "clixon_xml_db.h"
#include "clixon_nacm.h"

4
lib/src/clixon_netconf_lib.c
View file

@ -1042,6 +1042,10 @@ netconf_module_load(clicon_handle h)
goto done;
if (yang_spec_parse_module(h, "clixon-rfc5277", NULL, yspec)< 0)
goto done;
/* YANG module revision change management */
if (clicon_option_bool(h, "CLICON_YANG_CHANGELOG"))
if (yang_spec_parse_module(h, "clixon-yang-changelog", NULL, yspec)< 0)
goto done;
retval = 0;
done:
return retval;

37
lib/src/clixon_options.c
View file

@ -989,7 +989,6 @@ cxobj *
clicon_module_state_get(clicon_handle h)
{
clicon_hash_t *cdat = clicon_data(h);
void *p;
if ((p = hash_value(cdat, "module_state_cache", NULL)) != NULL)
@ -999,7 +998,7 @@ clicon_module_state_get(clicon_handle h)
/*! Set module state cache
* @param[in] h Clicon handle
* @param[in] s Open socket (or -1 to close)
* @param[in] xms Module state cache XML tree
* @retval 0 OK
* @retval -1 Error
*/
@ -1013,3 +1012,37 @@ clicon_module_state_set(clicon_handle h,
return -1;
return 0;
}
/*! Get yang module changelog
* @param[in] h Clicon handle
* @retval xch Module revision changelog XML tree
* @see draft-wang-netmod-module-revision-management-01
*/
cxobj *
clicon_yang_changelog_get(clicon_handle h)
{
clicon_hash_t *cdat = clicon_data(h);
void *p;
if ((p = hash_value(cdat, "yang-changelog", NULL)) != NULL)
return *(cxobj **)p;
return NULL;
}
/*! Set yang module changelog
* @param[in] h Clicon handle
* @param[in] s Module revision changelog XML tree
* @retval 0 OK
* @retval -1 Error
* @see draft-wang-netmod-module-revision-management-01
*/
int
clicon_yang_changelog_set(clicon_handle h,
cxobj *xchlog)
{
clicon_hash_t *cdat = clicon_data(h);
if (hash_add(cdat, "yang_changelog", &xchlog, sizeof(xchlog))==NULL)
return -1;
return 0;
}

164
lib/src/clixon_plugin.c
View file

@ -407,23 +407,25 @@ clixon_plugin_auth(clicon_handle h,
* When namespace and name match, the callback is made
*/
typedef struct {
qelem_t rc_qelem; /* List header */
qelem_t rc_qelem; /* List header */
clicon_rpc_cb rc_callback; /* RPC Callback */
void *rc_arg; /* Application specific argument to cb */
char *rc_namespace;/* Namespace to combine with name tag */
char *rc_name; /* Xml/json tag/name */
void *rc_arg; /* Application specific argument to cb */
char *rc_namespace;/* Namespace to combine with name tag */
char *rc_name; /* Xml/json tag/name */
} rpc_callback_t;
/* List of rpc callback entries */
/* List of rpc callback entries XXX hang on handle */
static rpc_callback_t *rpc_cb_list = NULL;
/*! Register a RPC callback by appending a new RPC to the list
*
* @param[in] h clicon handle
* @param[in] cb, Callback called
* @param[in] arg, Domain-specific argument to send to callback
* @param[in] cb Callback called
* @param[in] arg Domain-specific argument to send to callback
* @param[in] namespace namespace of rpc
* @param[in] name RPC name
* @retval 0 OK
* @retval -1 Error
* @see rpc_callback_call which makes the actual callback
*/
int
@ -530,3 +532,151 @@ rpc_callback_call(clicon_handle h,
clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
return retval;
}
/*--------------------------------------------------------------------
* Upgrade callbacks for backend upgrade of datastore
* Register upgrade callbacks in plugin_init() with a module and a "from" and "to"
* revision.
*/
typedef struct {
qelem_t uc_qelem; /* List header */
clicon_upgrade_cb uc_callback; /* RPC Callback */
void *uc_arg; /* Application specific argument to cb */
char *uc_name; /* Module name */
char *uc_namespace; /* Module namespace ??? */
uint32_t uc_from; /* Module revision (from) or 0 in YYYYMMDD format */
uint32_t uc_to; /* Module revision (to) in YYYYMMDD format */
} upgrade_callback_t;
/* List of rpc callback entries XXX hang on handle */
static upgrade_callback_t *upgrade_cb_list = NULL;
/*! Register an upgrade callback by appending the new callback to the list
*
* @param[in] h clicon handle
* @param[in] cb Callback called
* @param[in] arg Domain-specific argument to send to callback
* @param[in] name Module name (if NULL all modules)
* @param[in] namespace Module namespace (NOTE not relevant)
* @param[in] from From module revision (0 from any revision)
* @param[in] to To module revision (0 means module obsoleted)
* @retval 0 OK
* @retval -1 Error
* @see upgrade_callback_call which makes the actual callback
*/
int
upgrade_callback_register(clicon_handle h,
clicon_upgrade_cb cb,
void *arg,
char *name,
char *namespace,
uint32_t from,
uint32_t to)
{
upgrade_callback_t *uc;
if ((uc = malloc(sizeof(upgrade_callback_t))) == NULL) {
clicon_err(OE_DB, errno, "malloc: %s", strerror(errno));
goto done;
}
memset(uc, 0, sizeof(*uc));
uc->uc_callback = cb;
uc->uc_arg = arg;
if (name)
uc->uc_name = strdup(name);
if (namespace)
uc->uc_namespace = strdup(namespace);
uc->uc_from = from;
uc->uc_to = to;
ADDQ(uc, upgrade_cb_list);
return 0;
done:
if (uc){
if (uc->uc_name)
free(uc->uc_name);
if (uc->uc_namespace)
free(uc->uc_namespace);
free(uc);
}
return -1;
}
/*! Delete all Upgrade callbacks
*/
int
upgrade_callback_delete_all(void)
{
upgrade_callback_t *uc;
while((uc = upgrade_cb_list) != NULL) {
DELQ(uc, upgrade_cb_list, upgrade_callback_t *);
if (uc->uc_name)
free(uc->uc_name);
if (uc->uc_namespace)
free(uc->uc_namespace);
free(uc);
}
return 0;
}
/*! Search Upgrade callbacks and invoke if module match
*
* @param[in] h clicon handle
* @param[in] xt XML tree to be updated
* @param[in] modname Name of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[out] cbret Return XML (as string in CLIgen buffer), on invalid
* @retval -1 Error
* @retval 0 Invalid - cbret contains reason as netconf
* @retval 1 OK
* @see upgrade_callback_register which registers the callbacks
*/
int
upgrade_callback_call(clicon_handle h,
cxobj *xt,
char *modname,
char *modns,
uint32_t from,
uint32_t to,
cbuf *cbret)
{
int retval = -1;
upgrade_callback_t *uc;
int nr = 0; /* How many callbacks */
int ret;
if (upgrade_cb_list == NULL)
return 0;
uc = upgrade_cb_list;
do {
/* For matching an upgrade callback:
* - No module name registered (matches all modules) OR
* - Names match
* AND
* - No registered from revision (matches all revisions) OR
* - Registered from revision >= from AND
* - Registered to revision <= to (which includes case both 0)
*/
if (uc->uc_name == NULL || strcmp(uc->uc_name, modname)==0)
if ((uc->uc_from == 0) ||
(uc->uc_from >= from && uc->uc_to <= to)){
if ((ret = uc->uc_callback(h, xt, modname, modns, from, to, uc->uc_arg, cbret)) < 0){
clicon_debug(1, "%s Error in: %s", __FUNCTION__, uc->uc_name);
goto done;
}
if (ret == 0)
goto fail;
nr++;
}
uc = NEXTQ(upgrade_callback_t *, uc);
} while (uc != upgrade_cb_list);
retval = 1;
done:
clicon_debug(1, "%s retval:%d", __FUNCTION__, retval);
return retval;
fail:
retval =0;
goto done;
}

33
lib/src/clixon_string.c
View file

@ -636,17 +636,20 @@ clixon_trim(char *str)
}
/*! Transform from XSD regex to posix ERE
* The usecase is that Yang (RFC7950) supports XSD regexpressions but CLIgen supports
* Current translations:
* \d --> [0-9]
* The usecase is that Yang (RFC7950) supports XSD regular expressions but
* CLIgen supports POSIX ERE
* POSIX ERE regexps according to man regex(3).
* @param[in] xsd Input regex string according XSD
* @param[out] posix Output (malloced) string according to POSIX ERE
* @see https://www.w3.org/TR/2004/REC-xmlschema-2-20041028/#regexs
* @see https://www.regular-expressions.info/posixbrackets.html#class translation
* @see https://www.regular-expressions.info/xml.html
* Translation is not complete but covers some character sequences:
* \d decimal digit
* \w alphanum + underscore
* \w all characters except the set of "punctuation", "separator" and
* "other" characters: #x0000-#x10FFFF]-[\p{P}\p{Z}\p{C}]
* \i letters + underscore and colon
* \c XML Namechar, see: https://www.w3.org/TR/2008/REC-xml-20081126/#NT-NameChar
*/
int
regexp_xsd2posix(char *xsd,
@ -657,6 +660,7 @@ regexp_xsd2posix(char *xsd,
char x;
int i;
int esc;
int minus = 0;
if ((cb = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
@ -668,17 +672,24 @@ regexp_xsd2posix(char *xsd,
if (esc){
esc = 0;
switch (x){
case '-': /* \- is translated to -], ie must be last in bracket */
minus++;
break;
case 'c': /* xml namechar */
cprintf(cb, "[0-9a-zA-Z\\\\.\\\\-_:]");
cprintf(cb, "[0-9a-zA-Z._:-]"); /* also interpunct */
break;
case 'd':
cprintf(cb, "[0-9]");
break;
case 'w':
cprintf(cb, "[0-9a-zA-Z_\\\\-]");
case 'i': /* initial */
cprintf(cb, "[a-zA-Z_:]");
break;
case 'W':
cprintf(cb, "[^0-9a-zA-Z_\\\\-]");
case 'w': /* word */
//cprintf(cb, "[0-9a-zA-Z_\\\\-]")
cprintf(cb, "[^[:punct:][:space:][:cntrl:]]");
break;
case 'W': /* inverse of \w */
cprintf(cb, "[[:punct:][:space:][:cntrl:]]");
break;
case 's':
cprintf(cb, "[ \t\r\n]");
@ -693,6 +704,10 @@ regexp_xsd2posix(char *xsd,
}
else if (x == '\\')
esc++;
else if (x == ']' && minus){
cprintf(cb, "-]");
minus = 0;
}
else
cprintf(cb, "%c", x);
}

17
lib/src/clixon_xml_db.c
View file

@ -62,6 +62,7 @@
#include "clixon_xml.h"
#include "clixon_plugin.h"
#include "clixon_options.h"
#include "clixon_yang_module.h"
#include "clixon_xml_db.h"
/* Set to log get and put requests */
@ -325,7 +326,7 @@ xmldb_setopt(clicon_handle h,
* @param[in] xpath String with XPATH syntax. or NULL for all
* @param[in] config If set only configuration data, else also state
* @param[out] xret Single return XML tree. Free with xml_free()
* @param[out] xms If set, return modules-state differences
* @param[out] msd If set, return modules-state differences
* @retval 0 OK
* @retval -1 Error
* @code
@ -338,12 +339,12 @@ xmldb_setopt(clicon_handle h,
* @see xpath_vec
*/
int
xmldb_get(clicon_handle h,
const char *db,
char *xpath,
int config,
cxobj **xret,
cxobj **xms)
xmldb_get(clicon_handle h,
const char *db,
char *xpath,
int config,
cxobj **xret,
modstate_diff_t *msd)
{
int retval = -1;
xmldb_handle xh;
@ -361,7 +362,7 @@ xmldb_get(clicon_handle h,
clicon_err(OE_DB, 0, "Not connected to datastore plugin");
goto done;
}
retval = xa->xa_get_fn(xh, db, xpath, config, xret, xms);
retval = xa->xa_get_fn(xh, db, xpath, config, xret, msd);
#if DEBUG
if (retval == 0) {
cbuf *cb = cbuf_new();

3
lib/src/clixon_xpath.c
View file

@ -973,7 +973,7 @@ xp_eval(xp_ctx *xc,
break;
}
/* Eval second child c0
* Note, some operators 8like locationpath, need transitive context (use_xr0)
* Note, some operators like locationpath, need transitive context (use_xr0)
*/
if (xs->xs_c1)
if (xp_eval(use_xr0?xr0:xc, xs->xs_c1, &xr1) < 0)
@ -1060,6 +1060,7 @@ xpath_vec_ctx(cxobj *xcur,
goto done;
if (xpath_parse_init(&xy) < 0)
goto done;
clicon_debug(2,"%s",__FUNCTION__);
if (clixon_xpath_parseparse(&xy) != 0) { /* yacc returns 1 on error */
clicon_log(LOG_NOTICE, "XPATH error: on line %d", xy.xy_linenum);
if (clicon_errno == 0)

6
lib/src/clixon_xpath_parse.l
View file

@ -109,6 +109,12 @@ real ({digit}+[.]{digit}*)|({digit}*[.]{digit}+)
<TOKEN>last { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>position { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>count { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>re-match { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>deref { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>derived-from { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>derived-from-or-self { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>enum-value { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>bit-is-set { clixon_xpath_parselval.string = strdup(yytext); return FUNCTIONNAME; }
<TOKEN>@ { return *yytext; }
<TOKEN>ancestor:: { clixon_xpath_parselval.intval = A_ANCESTOR; return AXISNAME; }

17
lib/src/clixon_xpath_parse.y
View file

@ -69,6 +69,7 @@
%type <intval> axisspec
%type <string> string
%type <stack> args
%type <stack> expr
%type <stack> andexpr
%type <stack> relexpr
@ -164,8 +165,8 @@ xp_new(enum xp_type type,
double d0,
char *s0,
char *s1,
xpath_tree *c0,
xpath_tree *c1)
xpath_tree *c0,
xpath_tree *c1)
{
xpath_tree *xs = NULL;
@ -251,7 +252,7 @@ nodetest : '*' { $$=xp_new(XP_NODE,A_NAN,0.0, NULL, NULL, NULL,
| NAME { $$=xp_new(XP_NODE,A_NAN,0.0, NULL, $1, NULL, NULL); clicon_debug(2,"nodetest-> name(%s)",$1); }
| NAME ':' NAME { $$=xp_new(XP_NODE,A_NAN,0.0, $1, $3, NULL, NULL);clicon_debug(2,"nodetest-> name(%s) : name(%s)", $1, $3); }
| NAME ':' '*' { $$=xp_new(XP_NODE,A_NAN,0.0, $1, NULL, NULL, NULL);clicon_debug(2,"nodetest-> name(%s) : *", $1); }
| NODETYPE '(' ')' { $$=xp_new(XP_NODE_FN,A_NAN,0.0, $1, NULL, NULL, NULL); clicon_debug(2,"nodetest-> nodetype()"); }
| NODETYPE '(' ')' { $$=xp_new(XP_NODE_FN,A_NAN,0.0, $1, NULL, NULL, NULL); clicon_debug(1,"nodetest-> nodetype():%s", $1); }
;
/* evaluates to boolean */
@ -266,13 +267,15 @@ primaryexpr : '(' expr ')' { $$=xp_new(XP_PRI0,A_NAN,0.0, NULL, NULL, $2
| APOST string APOST { $$=xp_new(XP_PRIME_STR,A_NAN,0.0, $2, NULL, NULL, NULL);clicon_debug(2,"primaryexpr-> ' string '"); }
| APOST APOST { $$=xp_new(XP_PRIME_STR,A_NAN,0.0, NULL, NULL, NULL, NULL);clicon_debug(2,"primaryexpr-> ' '"); }
| FUNCTIONNAME '(' ')' { $$=xp_new(XP_PRIME_FN,A_NAN,0.0, $1, NULL, NULL, NULL);clicon_debug(2,"primaryexpr-> functionname ( arguments )"); }
| FUNCTIONNAME '(' args ')' { $$=xp_new(XP_PRIME_FN,A_NAN,0.0, $1, NULL, $3, NULL);clicon_debug(2,"primaryexpr-> functionname ( arguments )"); }
;
/* XXX Adding this between FUNCTIONNAME() breaks parser,..
arguments : arguments expr { clicon_debug(2,"arguments-> arguments expr"); }
| { clicon_debug(2,"arguments-> "); }
args : args ',' expr { $$=xp_new(XP_EXP,A_NAN,0.0,NULL,NULL,$1, $3);
clicon_debug(2,"args -> args expr");}
| expr { $$=xp_new(XP_EXP,A_NAN,0.0,NULL,NULL,$1, NULL);
clicon_debug(2,"args -> expr "); }
;
*/
string : string CHAR {
int len = strlen($1);
$$ = realloc($1, len+strlen($2) + 1);

34
lib/src/clixon_yang.c
View file

@ -173,9 +173,6 @@ static const map_str2int ykmap[] = {
{NULL, -1}
};
/* forward declaration */
static int ys_parse_date_arg(char *str, uint32_t *date);
/*! Create new yang specification
* @retval yspec Free with yspec_free()
* @retval NULL Error
@ -2920,37 +2917,40 @@ yang_desc_schema_nodeid(yang_node *yn,
return retval;
}
/*! parse yang date-arg string
/*! parse yang date-arg string and return a uint32 useful for arithmetics
* @param[in] datearg yang revision string as "YYYY-MM-DD"
* @param[out] dateint Integer version as YYYYMMDD
* @retval 0 OK
* @retval -1 Error, eg str is not on the format "YYYY-MM-DD"
*/
static int
ys_parse_date_arg(char *str,
uint32_t *date)
int
ys_parse_date_arg(char *datearg,
uint32_t *dateint)
{
int retval = -1;
int i;
uint32_t d = 0;
if (strlen(str) != 10 || str[4] != '-' || str[7] != '-'){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", str);
if (strlen(datearg) != 10 || datearg[4] != '-' || datearg[7] != '-'){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", datearg);
goto done;
}
if ((i = cligen_tonum(4, str)) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", str);
if ((i = cligen_tonum(4, datearg)) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", datearg);
goto done;
}
d = i*10000; /* year */
if ((i = cligen_tonum(2, &str[5])) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", str);
if ((i = cligen_tonum(2, &datearg[5])) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", datearg);
goto done;
}
d += i*100; /* month */
if ((i = cligen_tonum(2, &str[8])) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", str);
if ((i = cligen_tonum(2, &datearg[8])) < 0){
clicon_err(OE_YANG, EINVAL, "Revision date %s, but expected: YYYY-MM-DD", datearg);
goto done;
}
d += i; /* day */
*date = d;
*dateint = d;
retval = 0;
done:
return retval;

249
lib/src/clixon_yang_changelog.c Normal file
View file

@ -0,0 +1,249 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand and Benny Holmgren
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* YANG module revision change management.
* See draft-wang-netmod-module-revision-management-01
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <fnmatch.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/syslog.h>
#include <fcntl.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon_queue.h"
#include "clixon_hash.h"
#include "clixon_string.h"
#include "clixon_err.h"
#include "clixon_handle.h"
#include "clixon_yang.h"
#include "clixon_log.h"
#include "clixon_xml.h"
#include "clixon_options.h"
#include "clixon_xml_map.h"
#include "clixon_yang_module.h"
#include "clixon_yang_changelog.h"
#include "clixon_xpath_ctx.h"
#include "clixon_xpath.h"
#if 0
/*! Make a specific change
<index>0001</index>
<change-operation>create</change-operation>
<data-definition>
<target-node>
/a:system/a:y;
</target-node>
</data-definition>
*/
static int
upgrade_op(cxobj *x)
{
int retval = -1;
xml_print(stderr, x);
retval = 0;
// done:
return retval;
}
static int
upgrade_deleted(clicon_handle h,
char *name,
cxobj *xs)
{
int retval = -1;
fprintf(stderr, "%s \"%s\" belongs to a removed module\n", __FUNCTION__, name);
retval = 0;
// done:
return retval;
}
/*!
* @param[in] xs Module state
*/
static int
upgrade_modified(clicon_handle h,
char *name,
char *namespace,
cxobj *xs,
cxobj *xch)
{
int retval = -1;
char *mname;
yang_spec *yspec = NULL;
yang_stmt *ymod;
yang_stmt *yrev;
char *mrev;
cxobj **vec = NULL;
size_t veclen;
int i;
fprintf(stderr, "%s: \"%s\" belongs to an upgraded module\n", __FUNCTION__, name);
yspec = clicon_dbspec_yang(h);
/* We need module-name of XML since changelog uses that (change in changelog?)*/
mname = xml_find_body(xs, "name");
/* Look up system module (alt send it via argument) */
if ((ymod = yang_find_module_by_name(yspec, mname)) == NULL)
goto done;
if ((yrev = yang_find((yang_node*)ymod, Y_REVISION, NULL)) == NULL)
goto done;
mrev = yrev->ys_argument;
/* Look up in changelog */
if (xpath_vec(xch, "module[name=\"%s\" and revision=\"%s\"]/revision-change-log",
&vec, &veclen, mname, mrev) < 0)
goto done;
/* Iterate through changelog */
for (i=0; i<veclen; i++)
if (upgrade_op(vec[i]) < 0)
goto done;
retval = 0;
done:
if (vec)
free(vec);
return retval;
}
#endif
/*! Automatic upgrade using changelog
* @param[in] h Clicon handle
* @param[in] xn XML tree to be updated
* @param[in] modname Name of module
* @param[in] modns Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
*/
int
yang_changelog_upgrade(clicon_handle h,
cxobj *xn,
char *modname,
char *modns,
uint32_t from,
uint32_t to,
void *arg,
cbuf *cbret)
{
// cxobj *xchlog; /* changelog */
// cxobj **vec = NULL;
// size_t veclen;
if (!clicon_option_bool(h, "CLICON_YANG_CHANGELOG"))
goto ok;
/* Get changelog */
// xchlog = clicon_yang_changelog_get(h);
/* Get changelog entries for module between from and to
* (if to=0 we may not know name, need to use namespace)
*/
#if 0
if (xpath_vec(xchlog, "module[name=\"%s\" and revision=\"%s\"]/revision-change-log",
&vec, &veclen, modname, mrev) < 0)
goto done;
#endif
ok:
return 1;
}
/*! Initialize module revision. read changelog, etc
*/
int
clixon_yang_changelog_init(clicon_handle h)
{
int retval = -1;
char *filename;
int fd = -1;
cxobj *xt = NULL;
yang_spec *yspec;
cbuf *cbret = NULL;
int ret;
yspec = clicon_dbspec_yang(h);
if ((filename = clicon_option_str(h, "CLICON_YANG_CHANGELOG_FILE")) != NULL){
if ((fd = open(filename, O_RDONLY)) < 0){
clicon_err(OE_UNIX, errno, "open(%s)", filename);
goto done;
}
if (xml_parse_file(fd, NULL, yspec, &xt) < 0)
goto done;
if (xml_rootchild(xt, 0, &xt) < 0)
goto done;
if ((cbret = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
goto done;
}
if ((ret = xml_yang_validate_all(xt, cbret)) < 0)
goto done;
if (ret==1 && (ret = xml_yang_validate_add(xt, cbret)) < 0)
goto done;
if (ret == 0){ /* validation failed */
clicon_err(OE_YANG, 0, "validation failed: %s", cbuf_get(cbret));
goto done;
}
if (clicon_yang_changelog_set(h, xt) < 0)
goto done;
xt = NULL;
}
retval = 0;
done:
if (fd != -1)
close(fd);
if (xt)
xml_free(xt);
if (cbret)
cbuf_free(cbret);
return retval;
}

301
lib/src/clixon_yang_module.c
View file

@ -72,9 +72,35 @@
#include "clixon_xpath_ctx.h"
#include "clixon_xpath.h"
#include "clixon_options.h"
#include "clixon_plugin.h"
#include "clixon_netconf_lib.h"
#include "clixon_yang_module.h"
modstate_diff_t *
modstate_diff_new(void)
{
modstate_diff_t *md;
if ((md = malloc(sizeof(modstate_diff_t))) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
return NULL;
}
memset(md, 0, sizeof(modstate_diff_t));
return md;
}
int
modstate_diff_free(modstate_diff_t *md)
{
if (md == NULL)
return 0;
if (md->md_del)
free(md->md_del);
if (md->md_mod)
free(md->md_mod);
free(md);
return 0;
}
/*! Init the Yang module library
*
* Load RFC7895 yang spec, module-set-id, etc.
@ -167,7 +193,7 @@ int
modules_state_cache_set(clicon_handle h,
cxobj *msx)
{
int retval = -1;
int retval = -1;
cxobj *x; /* module state cache XML */
if ((x = clicon_module_state_get(h)) != NULL)
@ -185,35 +211,6 @@ modules_state_cache_set(clicon_handle h,
return retval;
}
/*! Get modules state according to RFC 7895
* @param[in] h Clicon handle
* @param[in] yspec Yang spec
* @param[in] xpath XML Xpath
* @param[in] brief Just name,revision and uri (no cache)
* @param[in,out] xret Existing XML tree, merge x into this
* @retval -1 Error (fatal)
* @retval 0 OK
* @retval 1 Statedata callback failed
* @notes NYI: schema, deviation
x +--ro modules-state
x +--ro module-set-id string
x +--ro module* [name revision]
x +--ro name yang:yang-identifier
x +--ro revision union
+--ro schema? inet:uri
x +--ro namespace inet:uri
+--ro feature* yang:yang-identifier
+--ro deviation* [name revision]
| +--ro name yang:yang-identifier
| +--ro revision union
+--ro conformance-type enumeration
+--ro submodule* [name revision]
+--ro name yang:yang-identifier
+--ro revision union
+--ro schema? inet:uri
* @see netconf_create_hello
*/
#if 1
/*! Actually build the yang modules state XML tree
*/
static int
@ -298,7 +295,34 @@ yms_build(clicon_handle h,
done:
return retval;
}
/*! Get modules state according to RFC 7895
* @param[in] h Clicon handle
* @param[in] yspec Yang spec
* @param[in] xpath XML Xpath
* @param[in] brief Just name,revision and uri (no cache)
* @param[in,out] xret Existing XML tree, merge x into this
* @retval -1 Error (fatal)
* @retval 0 OK
* @retval 1 Statedata callback failed
* @notes NYI: schema, deviation
x +--ro modules-state
x +--ro module-set-id string
x +--ro module* [name revision]
x +--ro name yang:yang-identifier
x +--ro revision union
+--ro schema? inet:uri
x +--ro namespace inet:uri
+--ro feature* yang:yang-identifier
+--ro deviation* [name revision]
| +--ro name yang:yang-identifier
| +--ro revision union
+--ro conformance-type enumeration
+--ro submodule* [name revision]
+--ro name yang:yang-identifier
+--ro revision union
+--ro schema? inet:uri
* @see netconf_create_hello
*/
int
yang_modules_state_get(clicon_handle h,
yang_spec *yspec,
@ -350,162 +374,87 @@ yang_modules_state_get(clicon_handle h,
xml_free(x);
return retval;
}
#else
/*! Upgrade XML
* @param[in] h Clicon handle
* @param[in] xt XML tree (to upgrade)
* @param[in] msd Modules-state differences of xt
* @retval 1 OK
* @retval 0 Validation failed
* @retval -1 Error
*/
int
yang_modules_state_get(clicon_handle h,
yang_spec *yspec,
char *xpath,
int brief,
cxobj **xret)
clixon_module_upgrade(clicon_handle h,
cxobj *xt,
modstate_diff_t *msd,
cbuf *cbret)
{
int retval = -1;
cxobj *x = NULL;
cbuf *cb = NULL;
yang_stmt *ylib = NULL; /* ietf-yang-library */
yang_stmt *yns = NULL; /* namespace */
yang_stmt *ymod; /* generic module */
yang_stmt *ys;
yang_stmt *yc;
char *msid; /* modules-set-id */
char *module = "ietf-yang-library";
cxobj *x1;
int retval = -1;
cxobj *xc; /* XML child of data */
char *namespace;
cxobj *xs; /* XML module state */
char *xname; /* XML top-level symbol name */
int state; /* 0: no changes, 1: deleted, 2: modified */
char *modname;
yang_spec *yspec;
yang_stmt *ymod;
yang_stmt *yrev;
char *rev;
uint32_t from;
uint32_t to;
int ret;
msid = clicon_option_str(h, "CLICON_MODULE_SET_ID");
if (modules_state_cache_get(h, msid, &x) < 0)
goto done;
if (x != NULL){ /* Yes a cache (but no duplicate) */
if (xpath_first(x, "%s", xpath)){
if ((x1 = xml_dup(x)) == NULL)
goto done;
x = x1;
}
else
x = NULL;
}
else { /* No cache -> build the tree */
if ((ylib = yang_find((yang_node*)yspec, Y_MODULE, module)) == NULL &&
(ylib = yang_find((yang_node*)yspec, Y_SUBMODULE, module)) == NULL){
clicon_err(OE_YANG, 0, "%s not found", module);
/* Iterate through db XML top-level - get namespace info */
xc = NULL;
while ((xc = xml_child_each(xt, xc, CX_ELMNT)) != NULL) {
xname = xml_name(xc); /* xml top-symbol name */
if (xml2ns(xc, NULL, &namespace) < 0) /* Get namespace of XML */
goto done;
if (namespace == NULL){
clicon_log(LOG_DEBUG, "XML %s lacks namespace", xname);
goto fail;
}
if ((yns = yang_find((yang_node*)ylib, Y_NAMESPACE, NULL)) == NULL){
clicon_err(OE_YANG, 0, "%s yang namespace not found", module);
goto done;
/* Look up module-state via namespace of XML */
state = 0; /* XML matches system modules */
if (msd){
if ((xs = xpath_first(msd->md_del, "module[namespace=\"%s\"]", namespace)) != NULL)
state = 1; /* XML belongs to a removed module */
else if ((xs = xpath_first(msd->md_mod, "module[namespace=\"%s\"]", namespace)) != NULL)
state = 2; /* XML belongs to an outdated module */
}
if ((cb = cbuf_new()) == NULL){
clicon_err(OE_UNIX, 0, "clicon buffer");
goto done;
}
cprintf(cb,"<modules-state xmlns=\"%s\">", yns->ys_argument);
cprintf(cb,"<module-set-id>%s</module-set-id>", msid);
/* Pick up more data from data store module-state */
from = to = 0;
modname = NULL;
if (state && xs && msd){ /* sanity: XXX what about no msd?? */
modname = xml_find_body(xs, "name"); /* Module name */
if ((rev = xml_find_body(xs, "revision")) != NULL) /* Module revision */
if (ys_parse_date_arg(rev, &from) < 0)
goto done;
if (state > 1){
yspec = clicon_dbspec_yang(h);
/* Look up system module (alt send it via argument) */
if ((ymod = yang_find_module_by_name(yspec, modname)) == NULL)
goto fail;
if ((yrev = yang_find((yang_node*)ymod, Y_REVISION, NULL)) == NULL)
goto fail;
if (ys_parse_date_arg(yrev->ys_argument, &to) < 0)
goto done;
<<<<<<< HEAD
ymod = NULL;
while ((ymod = yn_each((yang_node*)yspec, ymod)) != NULL) {
if (ymod->ys_keyword != Y_MODULE &&
ymod->ys_keyword != Y_SUBMODULE)
continue;
cprintf(cb,"<module>");
cprintf(cb,"<name>%s</name>", ymod->ys_argument);
if ((ys = yang_find((yang_node*)ymod, Y_REVISION, NULL)) != NULL)
cprintf(cb,"<revision>%s</revision>", ys->ys_argument);
else
cprintf(cb,"<revision></revision>");
=======
ymod = NULL;
while ((ymod = yn_each((yang_node*)yspec, ymod)) != NULL) {
if (ymod->ys_keyword != Y_MODULE &&
ymod->ys_keyword != Y_SUBMODULE)
continue;
cprintf(cb,"<module>");
cprintf(cb,"<name>%s</name>", ymod->ys_argument);
if ((ys = yang_find((yang_node*)ymod, Y_REVISION, NULL)) != NULL)
cprintf(cb,"<revision>%s</revision>", ys->ys_argument);
else
cprintf(cb,"<revision></revision>");
if (!brief){
>>>>>>> modules-state
if ((ys = yang_find((yang_node*)ymod, Y_NAMESPACE, NULL)) != NULL)
cprintf(cb,"<namespace>%s</namespace>", ys->ys_argument);
else
cprintf(cb,"<namespace></namespace>");
<<<<<<< HEAD
/* This follows order in rfc 7895: feature, conformance-type, submodules */
yc = NULL;
=======
}
/* This follows order in rfc 7895: feature, conformance-type, submodules */
yc = NULL;
if (!brief)
>>>>>>> modules-state
while ((yc = yn_each((yang_node*)ymod, yc)) != NULL) {
switch(yc->ys_keyword){
case Y_FEATURE:
if (yc->ys_cv && cv_bool_get(yc->ys_cv))
cprintf(cb,"<feature>%s</feature>", yc->ys_argument);
break;
default:
break;
}
}
<<<<<<< HEAD
cprintf(cb, "<conformance-type>implement</conformance-type>");
yc = NULL;
while ((yc = yn_each((yang_node*)ymod, yc)) != NULL) {
switch(yc->ys_keyword){
case Y_SUBMODULE:
cprintf(cb,"<submodule>");
cprintf(cb,"<name>%s</name>", yc->ys_argument);
if ((ys = yang_find((yang_node*)yc, Y_REVISION, NULL)) != NULL)
cprintf(cb,"<revision>%s</revision>", ys->ys_argument);
else
cprintf(cb,"<revision></revision>");
cprintf(cb,"</submodule>");
break;
default:
break;
}
=======
if (!brief)
cprintf(cb, "<conformance-type>implement</conformance-type>");
yc = NULL;
if (!brief)
while ((yc = yn_each((yang_node*)ymod, yc)) != NULL) {
switch(yc->ys_keyword){
case Y_SUBMODULE:
cprintf(cb,"<submodule>");
cprintf(cb,"<name>%s</name>", yc->ys_argument);
if ((ys = yang_find((yang_node*)yc, Y_REVISION, NULL)) != NULL)
cprintf(cb,"<revision>%s</revision>", ys->ys_argument);
else
cprintf(cb,"<revision></revision>");
cprintf(cb,"</submodule>");
break;
default:
break;
>>>>>>> modules-state
}
cprintf(cb,"</module>");
}
cprintf(cb,"</modules-state>");
if (xml_parse_string(cbuf_get(cb), yspec, &x) < 0){
if (netconf_operation_failed_xml(xret, "protocol", clicon_err_reason)< 0)
goto done;
retval = 1;
goto done;
}
if (modules_state_cache_set(h, x) < 0)
/* Make upgrade callback for this XML, specifying the module name,
* namespace, from and to revision.
* XXX: namespace may be known but not module!!
*/
if ((ret = upgrade_callback_call(h, xc, modname, namespace, from, to, NULL)) < 0)
goto done;
if (ret == 0)
goto fail;
}
if (x && netconf_trymerge(x, yspec, xret) < 0)
goto done;
retval = 0;
retval = 1;
done:
if (x)
xml_free(x);
if (cb)
cbuf_free(cb);
return retval;
fail:
retval = 0;
goto done;
}
#endif

28
test/test_type.sh
View file

@ -196,7 +196,12 @@ module example{
pattern '\w{4}';
}
}
leaf minus{
description "Problem with minus";
type string{
pattern '[a-zA-Z_][a-zA-Z0-9_\-.]*';
}
}
}
EOF
@ -535,13 +540,13 @@ new "cli yang pattern \d error"
expectfn "$clixon_cli -1f $cfg -l o -y $fyang set digit4 01b2" 255 "^$"
new "cli yang pattern \w ok"
expectfn "$clixon_cli -1f $cfg -l o -y $fyang set word4 a2-_" 0 "^$"
expectfn "$clixon_cli -1f $cfg -l o -y $fyang set word4 abc9" 0 "^$"
new "cli yang pattern \w error"
expectfn "$clixon_cli -1f $cfg -l o -y $fyang set word4 ab%d3" 255 "^$"
expectfn "$clixon_cli -1f $cfg -l o -y $fyang set word4 ab%3" 255 "^$"
new "netconf pattern \w"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><edit-config><target><candidate/></target><config><word4 xmlns="urn:example:clixon">a-_9</word4></config></edit-config></rpc>]]>]]>' "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><edit-config><target><candidate/></target><config><word4 xmlns="urn:example:clixon">aXG9</word4></config></edit-config></rpc>]]>]]>' "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
new "netconf pattern \w valid"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>' "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
@ -552,6 +557,21 @@ expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><edit-config><target><can
new "netconf pattern \w valid"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>' '^<rpc-reply><rpc-error><error-type>application</error-type><error-tag>bad-element</error-tag><error-info><bad-element>word4</bad-element></error-info><error-severity>error</error-severity><error-message>regexp match fail: "ab%d3" does not match \\w{4}</error-message></rpc-error></rpc-reply>]]>]]>$'
new "netconf discard-changes"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "<rpc><discard-changes/></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
#------ minus
new "type with minus"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '<rpc><edit-config><target><candidate/></target><config><minus xmlns="urn:example:clixon">my-name</minus></config></edit-config></rpc>]]>]]>' "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
new "validate minus"
expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "<rpc><validate><source><candidate/></source></validate></rpc>]]>]]>" "^<rpc-reply><ok/></rpc-reply>]]>]]>$"
#new "cli type with minus"
#expectfn "$clixon_cli -1f $cfg -l o -y $fyang set name my-name" 0 "^$"
if [ $BE -eq 0 ]; then
exit # BE
fi

255
test/test_upgrade_changelog.sh Executable file
View file

@ -0,0 +1,255 @@
#!/bin/bash
# Auto-upgrade using draft-wang-netmod-module-revision-management
# Ways of changes (operation-type) are:
# create, delete, move, modify
# In this example, example-a has the following changes:
# - Create y, delete x, modify host-name, move z
# example-b is completely obsoleted
# Magic line must be first in script (see README.md)
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi
APPNAME=example
cfg=$dir/conf.xml
changelog=$dir/changelog.xml # Module revision changelog
changelog2=$dir/changelog2.xml # From draft appendix
exa01y=$dir/example-a@2017-12-01.yang
exa20y=$dir/example-a@2017-12-20.yang
# draft-wang-netmod-module-revision-management-01
# 3.2.1 and 4.1 example-a revision 2017-12-01
cat <<EOF > $exa01y
module example-a{
yang-version 1.1;
namespace "urn:example:a";
prefix "a";
organization "foo.";
contact "fo@example.com";
description
"foo.";
revision 2017-12-01 {
description "Initial revision.";
}
container system {
leaf a {
type string;
description "no change";
}
leaf x {
type string;
description "delete";
}
leaf host-name {
type uint32;
description "modify type";
}
leaf z {
description "move to alt";
type string;
}
}
container alt {
}
}
EOF
# 3.2.1 and 4.1 example-a revision 2017-12-20
cat <<EOF > $exa20y
module example-a{
yang-version 1.1;
namespace "urn:example:a";
prefix "a";
organization "foo.";
contact "fo@example.com";
description
"foo.";
revision 2017-12-20 {
description "Create y, delete x, modify host-name, move z";
}
revision 2017-12-01 {
description "Initial revision.";
}
container system {
leaf a {
type string;
description "no change";
}
leaf host-name {
type string;
description "modify type";
}
leaf y {
type string;
description "create";
}
}
container alt {
leaf z {
description "move to alt";
type string;
}
}
}
EOF
# Create failsafe db
cat <<EOF > $dir/failsafe_db
<config>
<system xmlns="urn:example:a">
<a>Failsafe</a>
</system>
</config>
EOF
# Create startup db revision example-a and example-b 2017-12-01
# this should be automatically upgraded to 2017-12-20
cat <<EOF > $dir/startup_db
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>example-a</name>
<revision>2017-12-01</revision>
<namespace>urn:example:a</namespace>
</module>
<module>
<name>example-b</name>
<revision>2017-12-01</revision>
<namespace>urn:example:b</namespace>
</module>
</modules-state>
<system xmlns="urn:example:a">
<a>dont change me</a>
<host-name>modify me</host-name>
<x>remove me</x>
<z>move me</z>
</system>
<system-b xmlns="urn:example:b">
<b>Obsolete</b>
</system-b>
</config>
EOF
# Create configuration
cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_YANG_MAIN_DIR>$dir</CLICON_YANG_MAIN_DIR>
<CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK>
<CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
<CLICON_BACKEND_PIDFILE>/usr/local/var/$APPNAME/$APPNAME.pidfile</CLICON_BACKEND_PIDFILE>
<CLICON_XMLDB_DIR>$dir</CLICON_XMLDB_DIR>
<CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
<CLICON_XMLDB_MODSTATE>true</CLICON_XMLDB_MODSTATE>
<CLICON_MODULE_REVISION>true</CLICON_MODULE_REVISION>
<CLICON_MODULE_REVISION_CHANGELOG>$changelog</CLICON_MODULE_REVISION_CHANGELOG>
<CLICON_CLISPEC_DIR>/usr/local/lib/$APPNAME/clispec</CLICON_CLISPEC_DIR>
<CLICON_CLI_DIR>/usr/local/lib/$APPNAME/cli</CLICON_CLI_DIR>
<CLICON_CLI_MODE>$APPNAME</CLICON_CLI_MODE>
</clixon-config>
EOF
# Changelog of example-a:
cat <<EOF > $changelog
<yang-modules xmlns="http://clicon.org/yang-changelog">
<module>
<name>example-b</name>
<revision>2017-12-01</revision>
<!--obsolete-->
</module>
<module>
<name>example-a</name>
<revision>2017-12-20</revision>
<backward-compatible>true</backward-compatible>
<revision-change-log>
<index>0001</index>
<change-operation>create</change-operation>
<data-definition>
<target-node>
/a:system/a:y;
</target-node>
</data-definition>
</revision-change-log>
<revision-change-log>
<index>0002</index>
<change-operation>delete</change-operation>
<data-definition>
<target-node>
/a:system/a:x;
</target-node>
</data-definition>
</revision-change-log>
<revision-change-log>
<index>0003</index>
<change-operation>modify</change-operation>
<data-definition>
<target-node>
/a:system/a:host-name;
</target-node>
</data-definition>
</revision-change-log>
<revision-change-log>
<index>0004</index>
<change-operation>move</change-operation>
<data-definition>
<target-node>
/a:system/a:z;
</target-node>
</data-definition>
</revision-change-log>
</module>
</yang-modules>
EOF
# Start new system from old datastore
mode=startup
new "test params: -s $mode -f $cfg"
# Bring your own backend
if [ $BE -ne 0 ]; then
# kill old backend (if any)
new "kill old backend"
sudo clixon_backend -zf $cfg
if [ $? -ne 0 ]; then
err
fi
new "start backend -s $mode -f $cfg"
start_backend -s $mode -f $cfg
fi
new "waiting"
sleep $RCWAIT
new "kill old restconf daemon"
sudo pkill -u www-data clixon_restconf
new "start restconf daemon"
start_restconf -f $cfg
new "waiting"
sleep $RCWAIT
new "Check failsafe (work in progress)"
new "Check running db content"
expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' '^<rpc-reply><data><system xmlns="urn:example:a"><a>Failsafe</a></system></data></rpc-reply>]]>]]>$'
new "Kill restconf daemon"
stop_restconf
if [ $BE -ne 0 ]; then
new "Kill backend"
# Check if premature kill
pid=`pgrep -u root -f clixon_backend`
if [ -z "$pid" ]; then
err "backend already dead"
fi
# kill backend
stop_backend -f $cfg
rm -rf $dir
fi

2
test/test_upgrade_repair.sh
View file

@ -1,7 +1,7 @@
#!/bin/bash
# Start clixon with module A rec 2019
# Load startup with non-compatible and invalid module A with rev 0814-01-28
# Go into fail-safe with invalid startup
# Repair by copying startup into candidate, editing and commit it
# Magic line must be first in script (see README.md)
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi

4
test/test_xpath.sh
View file

@ -199,4 +199,8 @@ expecteof "$clixon_util_xpath -f $xml3 -p bbb[ccc='bar']" 0 "" "^nodeset:0:<bbb
new "xpath bbb[ccc='fie']"
expecteof "$clixon_util_xpath -f $xml3 -p bbb[ccc='fie']" 0 "" "^nodeset:$"
# Just syntax - no semantic meaning
new "xpath derived-from-or-self"
expecteof "$clixon_util_xpath -f $xml3 -p 'derived-from-or-self(../../change-operation,modify)'" 0 "" "derived-from-or-self"
rm -rf $dir

1
yang/clixon/Makefile.in
View file

@ -43,6 +43,7 @@ CLIXON_DATADIR = @CLIXON_DATADIR@
YANGSPECS = clixon-config@2019-03-05.yang
YANGSPECS += clixon-lib@2019-01-02.yang
YANGSPECS += clixon-rfc5277@2008-07-01.yang
YANGSPECS += clixon-yang-changelog@2019-03-21.yang
APPNAME = clixon # subdir ehere these files are installed

12
yang/clixon/clixon-config@2019-03-05.yang
View file

@ -428,6 +428,18 @@ module clixon-config {
data. If enabled, module info will appear when doing
netconf get or restconf GET";
}
leaf CLICON_YANG_CHANGELOG {
type boolean;
default false;
description "If true enable automatic upgrade using yang clixon
changelog.";
}
leaf CLICON_YANG_CHANGELOG_FILE {
type string;
description "Name of file with module revision changelog.
If CLICON_YANG_CHANGELOG is true, Clixon
reads the module changelog from this file.";
}
leaf CLICON_MODULE_SET_ID {
type string;
default "0";

326
yang/clixon/clixon-yang-changelog@2019-03-21.yang Normal file
View file

@ -0,0 +1,326 @@
module clixon-yang-changelog {
yang-version 1.1;
namespace "http://clicon.org/yang-changelog";
prefix ml;
import ietf-yang-library {
prefix yanglib;
}
import ietf-yang-types {
prefix yang;
}
organization "Clixon";
contact
"Olof Hagsand <olof@hagsand.se>";
description
"This is experimentalYANG changelog derived from:
draft-wang-netmod-module-revision-management-01
with the following contacts and references:
WG Web: <https://datatracker.ietf.org/wg/netmod/>
WG List: <mailto:netmod@ietf.org>
Author: Qin Wu
<mailto:bill.wu@huawei.com>
Zitao Wang
<mailto:wangzitao@huawei.com>";
reference "draft-wang-netmod-module-revision-management-01";
revision 2019-03-21 {
description
"Initial Clixon derived version";
}
identity operation-type {
description
"Abstract base identity for the operation type ";
}
identity create {
base operation-type;
description
"Denotes create new data nodes";
}
identity delete {
base operation-type;
description
"Denotes delete the target node";
}
identity move {
base operation-type;
description
"Denote move the target node.";
}
identity modify {
base operation-type;
description
"Denote modify the target data node.";
}
identity statement-type {
description
"Base identity for statement type";
}
identity feature-statement {
base statement-type;
description
"feature statement, if this type be chose, it means that the
feature or if-feature statement been modified";
}
identity identity-statement {
base statement-type;
description
"identity statement, if this type be chose, it means that the
identity statement been modified, for example, add new identity, etc.";
}
identity grouping-statement {
base statement-type;
description
"grouping statement, if this type be chose, it means that the grouping
statement been modified.";
}
identity typedef-statement {
base statement-type;
description
"typedef statement, if this type be chose, it means that the typedef
statement been modified.";
}
identity augment-statement {
base statement-type;
description
"augment statement, if this type be chose, it means that the augment
statement been modified.";
}
identity rpc-statement {
base statement-type;
description
"rpc statement, if this type be chose, it means that the rpc
statement been modified.";
}
identity notification-statement {
base statement-type;
description
"notification statement, if this type be chose, it means that the notification
statement been modified.";
}
extension purpose {
argument name;
description
"The purpose can be used to mark the data nodes change purpose.
The name argument can be specified in the following recommended mode
- bug-fix, which can help user to understand the data nodes' changes present bug fix,
- new-function, which can help user to understand the data nodes' changes present new function,
- nmda-conform, which can help user to understand the data nodes' changes conform to NMDA,
and note that the user can argument the purpose name according to their sepcific requirements.";
}
grouping data-definition {
container data-definition {
leaf target-node {
type yang:xpath1.0;
mandatory true;
description
"Identifies the target data node for update.
Notice that, if the update-type equal to move or delete,
this target-node must point to the data node of old version.
\t
For example, suppose the target node is a YANG leaf named a,
and the previous version is:
\t
container foo {
leaf a { type string; }
leaf b { type int32; }
}
\t
the new version is:
container foo {
leaf b {type int32;}
}
\t
Therefore, the targe-node should be /foo/a.";
}
leaf location-point {
type yang:xpath1.0;
description
"Identifies the location point where the updates happened.";
}
leaf where {
when "derived-from-or-self(../../change-operation, 'move')" {
description
"This leaf only applies for 'move'
updates.";
}
type enumeration {
enum "before" {
description
"Insert or move a data node before the data resource
identified by the 'point' parameter.";
}
enum "after" {
description
"Insert or move a data node after the data resource
identified by the 'point' parameter.";
}
enum "first" {
description
"Insert or move a data node so it becomes ordered
as the first entry.";
}
enum "last" {
description
"Insert or move a data node so it becomes ordered
as the last entry.";
}
}
default "last";
description
"Identifies where a data resource will be inserted
or moved.";
}
anydata data-definition {
when "derived-from-or-self(../../change-operation, 'modify')" {
description
"This nodes only be present when
the 'change-operation' equal to 'modify'.";
}
description
"This nodes used for present the definitions before updated.
And this nodes only be present when
the 'change-operation' equal to 'modify'.";
}
description
"Container for data statement";
}
description
"Grouping for data definition";
}
grouping other-statement {
container other-statement {
leaf statement-name {
type identityref {
base statement-type;
}
description
"Statement name, for example, identity, feature, typedef, etc.";
}
anydata statement-definition {
description
"This nodes used for present new the definitions.";
}
list substatements {
key "statement-name";
leaf statement-name {
type identityref {
base statement-type;
}
description
"Statement name, for example, identity, feature, typedef, etc.";
}
anydata substatement-definition {
description
"This nodes used for present new the definitions.";
}
description
"List for substatements updates";
}
description
"Container for header statement updates";
}
description
"Grouping for header statement";
}
grouping change-log {
list revision-change-log {
key "index";
leaf index {
type uint32;
description
"Index for module change log";
}
leaf change-operation {
type identityref {
base operation-type;
}
mandatory true;
description
"This leaf indicate the change operation, such as create, move, delete, modify, etc.";
}
choice yang-statements {
description
"Choice for various YANG statements that have been impacted.";
case data-definition-statement {
uses data-definition;
}
case other-statement {
uses other-statement;
}
}
description
"List for module revision change log";
}
description
"Grouping for module revision change log";
}
container yang-modules {
config false;
list module {
key "name revision";
leaf name {
type yang:yang-identifier;
description
"The YANG module or submodule name.";
}
leaf revision {
type yanglib:revision-identifier;
description
"The YANG module or submodule revision date. If no revision
statement is present in the YANG module or submodule, this
leaf is not instantiated.";
}
leaf backward-compatible {
type boolean;
description
"Indicates whether it is a backward compatible version.
If this parameter is set to true, it means that this version is
a backwards compatible version";
}
uses change-log;
description
"List for module updated log";
}
description
"This container present the modules updated log.";
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module" {
description
"Augment the yang library with backward compatibility indication.";
leaf backward-compatible {
type boolean;
description
"backward compatibility indication.";
}
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module/yanglib:submodule" {
description
"Augment the yang library with backward compatibility indication.";
leaf backward-compatible {
type boolean;
description
"backward compatibility indication.";
}
}
}

1
yang/standard/Makefile.in
View file

@ -52,7 +52,6 @@ YANGSPECS += ietf-netconf@2011-06-01.yang
YANGSPECS += ietf-netconf-acm@2018-02-14.yang
YANGSPECS += ietf-restconf-monitoring@2017-01-26.yang
YANGSPECS += ietf-netconf-monitoring@2010-10-04.yang
YANGSPECS += ietf-module-revision@2018-08-08.yang
all: