experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

upgrade example and test

Browse source
This commit is contained in:
Olof hagsand 2019-03-27 18:04:01 +01:00
parent 4902f7cf1d
commit be59bd48d8
7 changed files with 88 additions and 103 deletions
Download patch file Download diff file Expand all files Collapse all files

6
doc/FAQ.md
View file

@ -37,6 +37,7 @@
* [I want to add a hook to an existing operation, can I do that?](#i-want-to-add-a-hook-to-an-existing-operation-can-i-do-that) * [I want to add a hook to an existing operation, can I do that?](#i-want-to-add-a-hook-to-an-existing-operation-can-i-do-that)
* [How do I write an authentication callback?](#how-do-i-write-an-authentication-callback) * [How do I write an authentication callback?](#how-do-i-write-an-authentication-callback)
* [What about access control?](#what-about-access-control) * [What about access control?](#what-about-access-control)
* [Does Clixon support upgrade?](#does-clixon-support-upgrade)
* [How do I write a CLI translator function?](#how-do-i-write-a-cli-translator-function) * [How do I write a CLI translator function?](#how-do-i-write-a-cli-translator-function)
## What is Clixon? ## What is Clixon?
@ -570,6 +571,11 @@ Control Model defined in [RFC8341](https://tools.ietf.org/html/rfc8341)
Incoming RPC and data node access points are supported with some Incoming RPC and data node access points are supported with some
limitations. See the (README)(../README.md) for more information. limitations. See the (README)(../README.md) for more information.
## Does Clixon support upgrade?
Yes. Clixon provides a callback interface where datastores can be
upgraded. This is described in [the startup doc](startup.md).
## How do I write a CLI translator function? ## How do I write a CLI translator function?
The CLI can perform variable translation. This is useful if you want to The CLI can perform variable translation. This is useful if you want to

118
doc/startup.md
View file

@ -120,7 +120,7 @@ module-state of the backend daemon, a set of _upgrade_ callbacks are
made. This allows a user to program upgrade funtions in the backend made. This allows a user to program upgrade funtions in the backend
plugins to automatically upgrade the XML to the current version. plugins to automatically upgrade the XML to the current version.
Clixon also has a system-provided [automatic upgrading method](#automatic-upgrades) based on Yang changelogs covered in a separate chapter. Clixon also has an experimental [automatic upgrading method](#automatic-upgrades) based on Yang changelogs covered in a separate chapter.
A user registers upgrade callbacks based on module and revision A user registers upgrade callbacks based on module and revision
ranges. A user can register many callbacks, or choose wildcards. ranges. A user can register many callbacks, or choose wildcards.
@ -138,37 +138,36 @@ callback per module and revision range that will be called in a series.
A user registers upgrade callbacks in the backend `clixon_plugin_init()` function. The signature of upgrade callback is as follows: A user registers upgrade callbacks in the backend `clixon_plugin_init()` function. The signature of upgrade callback is as follows:
``` ```
int upgrade_callback_register(clicon_handle h, upgrade_callback_register(h, cb, namespace, from, revision, arg);
clicon_upgrade_cb cb,
char *namespace,
uint32_t from,
uint32_t to,
void *arg)
``` ```
where: where:
* `h` is the Clicon handle, * `h` is the Clicon handle,
* `cb` is the name of the callback function, * `cb` is the name of the callback function,
* `namespace` defines a Yang module. NULL denotes all modules. Note that module `name` is not used. * `namespace` defines a Yang module. NULL denotes all modules. Note that module `name` is not used (XML uses namespace, whereas JSON uses name, XML is more common).
* `revision` is the revision date to where the upgrade is made. It is either the same revision as the Clixon system module, or an older version. In the latter case, it i recommended to provide an upgrade function to the most recent revision. If revision is `0` it means that this module is not present in the system and is _obsolete_. * `from` is a revision date indicated an optional start date of the upgrade. This allows for defining a partial upgrade. It can also be `0` to denote any version.
* `from` is a revision date indicated an optional start date of the upgrade. This allows for defining a partial upgrade. It can also be `0` to denote any old version. * `revision` is the revision date "to" where the upgrade is made. It is either the same revision as the Clixon system module, or an older version. In the latter case, you can provide another upgrade callback to the most recent revision.
* `arg` is a user defined argument which can be passed to the callback. * `arg` is a user defined argument which can be passed to the callback.
One example of registering a "catch-all" upgrade: One example of registering a "catch-all" upgrade:
``` ```
upgrade_callback_register(h, yang_changelog_upgrade, NULL, 0, 0, NULL); upgrade_callback_register(h, xml_changelog_upgrade, NULL, 0, 0, NULL);
``` ```
Another example are fine-grained stepwise upgrades of a single module: Another example are fine-grained stepwise upgrades of a singlemodule [upgrade example](#example-upgrade):
``` ```
upgrade_callback_register(h, upgrade_a_1, "urn:example:a", upgrade_callback_register(h, upgrade_2016, "urn:example:interfaces",
20171201, 20180101, NULL); 20140508, 20160101, NULL);
upgrade_callback_register(h, upgrade_a_2, "urn:example:a", upgrade_callback_register(h, upgrade_2018, "urn:example:interfaces",
20180101, 20190101, NULL); 20160101, 20180220, NULL);
``` ```
```
In the latter case, one upgrade function updates data modelled by `A` 20140508 20160101 20180220
from revision 2017-12-01 to 2018-01-01; a separate one updates from ------+--------------+--------------+-------->
2018-01-01 to 2019-01-01. These are run in series. upgrade_2016 upgrade_2018
```
In the latter case, the first callback upgrades
from revision 2014-05-08 to 2016-01-01; while the second makes upgrades from
2016-01-01 to 2018-02-20. These are run in series.
### Upgrade callback ### Upgrade callback
@ -181,61 +180,15 @@ Note the following:
* Upgrade callbacks _will_ be called if the datastore contains a version of a module that is older than the module loaded in Clixon. * Upgrade callbacks _will_ be called if the datastore contains a version of a module that is older than the module loaded in Clixon.
* Upgrade callbacks _will_ also be called if the datastore contains a version of a module that is not present in Clixon - an obsolete module. * Upgrade callbacks _will_ also be called if the datastore contains a version of a module that is not present in Clixon - an obsolete module.
An example upgrade callback: Re-using the previous stepwise example, if a datastore is loaded based on revision 20140508 by a system supporting revision 20180220, the following two callbacks will be made:
``` ```
/*! Automatic upgrade of module A upgrade_2016(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret);
* @param[in] h Clicon handle upgrade_2018(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret);
* @param[in] xn XML tree to be updated
* @param[in] namespace Namespace of module (for info)
* @param[in] from From revision on the form YYYYMMDD
* @param[in] to To revision on the form YYYYMMDD (0 not in system)
* @param[in] arg User argument given at rpc_callback_register()
* @param[out] cbret Return xml tree, eg <rpc-reply>..., <rpc-error..
* @retval 1 OK
* @retval 0 Invalid
* @retval -1 Error
*/
int
upgrade_a_1(clicon_handle h,
cxobj *xn,
char *namespace,
uint32_t revision,
uint32_t from,
void *arg,
cbuf *cbret)
{
int retval = -1;
yang_spec *yspec;
yang_stmt *ym;
cxobj **vec = NULL;
cxobj *xc;
size_t vlen;
int i;
/* Get Yang module for this namespace. Note it may not exist */
yspec = clicon_dbspec_yang(h);
if ((ym = yang_find_module_by_namespace(yspec, ns)) == NULL)
goto ok; /* shouldnt happen */
/* Get all XML nodes with that namespace */
if (xml_namespace_vec(h, xt, ns, &vec, &vlen) < 0)
goto done;
for (i=0; i<vlen; i++){
xc = vec[i];
/* Insert code here to transform nodes of module */
}
ok:
retval = 1;
done:
if (vec)
free(vec);
return retval;
}
``` ```
Note that the example shown is a template for an upgrade function. It Note that the example shown is a template for an upgrade function. It
gets the nodes of an yang module given by `namespace` and the gets the nodes of an yang module given by `namespace` and the
(outdated) `from` revision, and iterates through them. Actual (outdated) `from` revision, and iterates through them. Actual
upgrading code may be implemented by a user. upgrading code is in [the example](../example/example_backend.c).
If no action is made by the upgrade calback, and thus the XML is not If no action is made by the upgrade calback, and thus the XML is not
upgraded, the next step is XML/Yang validation. upgraded, the next step is XML/Yang validation.
@ -247,6 +200,19 @@ However, if the validation fails, the backend will try to enter the
failsafe mode so that the user may perform manual upgarding of the failsafe mode so that the user may perform manual upgarding of the
configuration. configuration.
### Example upgrade
[The example](../example/example_backend.c) and [test](../test/test_upgrade_interfaces.sh) shos the code for upgrading of an interface module. The example is inspired by the ietf-interfaces module that made a subset of the upgrades shown in the examples.
The code is split in two steps. The `upgrade_2016` callback does the following transforms:
* Move /if:interfaces-state/if:interface/if:admin-status to /if:interfaces/if:interface/
* Move /if:interfaces-state/if:interface/if:statistics to if:interfaces/if:interface/
* Rename /interfaces/interface/description to /interfaces/interface/descr
While the `upgrade_2018` callback does the following transforms:
* Delete /if:interfaces-state
* Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr
* Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000
## Extra XML ## Extra XML
@ -358,10 +324,10 @@ The example shown in this Section is also available as a regression [test script
## Automatic upgrades ## Automatic upgrades
Clixon supports an experimental yang changelog feature based on Clixon supports an EXPERIMENTAL xml changelog feature based on
"draft-wang-netmod-module-revision-management-01" (Zitao Wang et al) "draft-wang-netmod-module-revision-management-01" (Zitao Wang et al)
where changes to the Yang model are documented and loaded into where changes to the Yang model are documented and loaded into
Clixon. Clixon. The implementation is not complete.
When upgrading, the system parses the changelog and tries to upgrade When upgrading, the system parses the changelog and tries to upgrade
the datastore automatically. This featire is experimental and has the datastore automatically. This featire is experimental and has
@ -369,10 +335,10 @@ several limitations.
You enable the automatic upgrading by registering the changelog upgrade method in `clixon_plugin_ini()` using wildcards: You enable the automatic upgrading by registering the changelog upgrade method in `clixon_plugin_ini()` using wildcards:
``` ```
upgrade_callback_register(h, yang_changelog_upgrade, NULL, 0, 0, NULL); upgrade_callback_register(h, xml_changelog_upgrade, NULL, 0, 0, NULL);
``` ```
The transformation is defined by a list of changelogs. Each changelog defined how a module (defined by a namespace) is transformed from an old revision to a nnew. Example: The transformation is defined by a list of changelogs. Each changelog defined how a module (defined by a namespace) is transformed from an old revision to a nnew. Example from [test_upgrade_auto.sh](../test/test_upgrade_auto.sh)
``` ```
<changelogs xmlns="http://clicon.org/xml-changelog"> <changelogs xmlns="http://clicon.org/xml-changelog">
<changelog> <changelog>
@ -389,12 +355,12 @@ Each changelog consists of set of (orderered) steps:
<name>1</name> <name>1</name>
<op>insert</op> <op>insert</op>
<where>/a:system</where> <where>/a:system</where>
<transform>&lt;y&gt;created&lt;/y&gt;</transform> <new><y>created</y></new>
</step> </step>
<step> <step>
<name>2</name> <name>2</name>
<op>delete</op> <op>delete</op>
<where>/a:system/a:x</where> <where>/a:system/a:x</where>
</step> </step>
``` ```
Each step has an (atomic) operation: Each step has an (atomic) operation:

36
example/example_backend.c
View file

@ -267,7 +267,7 @@ example_statedata(clicon_handle h,
* @retval -1 Error * @retval -1 Error
* @see clicon_upgrade_cb * @see clicon_upgrade_cb
* @see test_upgrade_interfaces.sh * @see test_upgrade_interfaces.sh
* @see upgrade_interfaces_2016 * @see upgrade_2016
* This example shows a two-step upgrade where the 2014 function does: * This example shows a two-step upgrade where the 2014 function does:
* - Move /if:interfaces-state/if:interface/if:admin-status to * - Move /if:interfaces-state/if:interface/if:admin-status to
* /if:interfaces/if:interface/ * /if:interfaces/if:interface/
@ -276,13 +276,13 @@ example_statedata(clicon_handle h,
* - Rename /interfaces/interface/description to descr * - Rename /interfaces/interface/description to descr
*/ */
static int static int
upgrade_interfaces_2014(clicon_handle h, upgrade_2016(clicon_handle h,
cxobj *xt, cxobj *xt,
char *ns, char *ns,
uint32_t from, uint32_t from,
uint32_t to, uint32_t to,
void *arg, void *arg,
cbuf *cbret) cbuf *cbret)
{ {
int retval = -1; int retval = -1;
yang_spec *yspec; yang_spec *yspec;
@ -367,7 +367,7 @@ upgrade_interfaces_2014(clicon_handle h,
* @retval -1 Error * @retval -1 Error
* @see clicon_upgrade_cb * @see clicon_upgrade_cb
* @see test_upgrade_interfaces.sh * @see test_upgrade_interfaces.sh
* @see upgrade_interfaces_2014 * @see upgrade_2016
* The 2016 function does: * The 2016 function does:
* - Delete /if:interfaces-state * - Delete /if:interfaces-state
* - Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr * - Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr
@ -375,13 +375,13 @@ upgrade_interfaces_2014(clicon_handle h,
* fraction-digits 3 and divide all values with 1000 * fraction-digits 3 and divide all values with 1000
*/ */
static int static int
upgrade_interfaces_2016(clicon_handle h, upgrade_2018(clicon_handle h,
cxobj *xt, cxobj *xt,
char *ns, char *ns,
uint32_t from, uint32_t from,
uint32_t to, uint32_t to,
void *arg, void *arg,
cbuf *cbret) cbuf *cbret)
{ {
int retval = -1; int retval = -1;
yang_spec *yspec; yang_spec *yspec;
@ -624,9 +624,9 @@ clixon_plugin_init(clicon_handle h)
* test interface example. Otherwise the auto-upgrade feature is enabled. * test interface example. Otherwise the auto-upgrade feature is enabled.
*/ */
if (_upgrade){ if (_upgrade){
if (upgrade_callback_register(h, upgrade_interfaces_2014, "urn:example:interfaces", 0, 0, NULL) < 0) if (upgrade_callback_register(h, upgrade_2016, "urn:example:interfaces", 20140508, 20160101, NULL) < 0)
goto done; goto done;
if (upgrade_callback_register(h, upgrade_interfaces_2016, "urn:example:interfaces", 0, 0, NULL) < 0) if (upgrade_callback_register(h, upgrade_2018, "urn:example:interfaces", 20160101, 20180220, NULL) < 0)
goto done; goto done;
} }
else else

2
lib/clixon/clixon_plugin.h
View file

@ -204,7 +204,7 @@ struct clixon_plugin_api{
/* /*
* Macros * Macros
*/ */
#define upgrade_callback_register(h, cb, namespace, from, to, arg) upgrade_callback_reg_fn((h), (cb), #cb, (namespace), (from), (to), (arg)) #define upgrade_callback_register(h, cb, namespace, from, rev, arg) upgrade_callback_reg_fn((h), (cb), #cb, (namespace), (from), (rev), (arg))
typedef struct clixon_plugin_api clixon_plugin_api; typedef struct clixon_plugin_api clixon_plugin_api;

4
lib/src/clixon_plugin.c
View file

@ -559,8 +559,8 @@ static upgrade_callback_t *upgrade_cb_list = NULL;
* @param[in] fnstr Stringified function for debug * @param[in] fnstr Stringified function for debug
* @param[in] arg Domain-specific argument to send to callback * @param[in] arg Domain-specific argument to send to callback
* @param[in] namespace Module namespace (if NULL all modules) * @param[in] namespace Module namespace (if NULL all modules)
* @param[in] rev To module revision (0 means module obsoleted)
* @param[in] from From module revision (0 from any revision) * @param[in] from From module revision (0 from any revision)
* @param[in] revision To module revision (0 means module obsoleted)
* @retval 0 OK * @retval 0 OK
* @retval -1 Error * @retval -1 Error
* @see upgrade_callback_call which makes the actual callback * @see upgrade_callback_call which makes the actual callback
@ -570,8 +570,8 @@ upgrade_callback_reg_fn(clicon_handle h,
clicon_upgrade_cb cb, clicon_upgrade_cb cb,
const char *fnstr, const char *fnstr,
char *namespace, char *namespace,
uint32_t revision,
uint32_t from, uint32_t from,
uint32_t revision,
void *arg) void *arg)
{ {
upgrade_callback_t *uc; upgrade_callback_t *uc;

23
test/test_feature.sh
View file

@ -1,5 +1,21 @@
#!/bin/bash #!/bin/bash
# Yang features. if-feature. and schema resources according to RFC7895 # Yang features. if-feature.
# The test has a example module with FEATURES A and B, where A is enabled and
# B is disabled.
# It also uses an ietf-router module where (only) router-id is enabled
# Also check modules-state (RFC7895) announces the enabled features.
#
# From RFC7950:
# 7.20.1 Schema nodes tagged with an "if-feature" statement are _ignored_ by
# the server unless the server supports the given feature expression.
# 8.1: There MUST be no nodes tagged with "if-feature" present if the
# "if-feature" expression evaluates to "false" in the server.
# - Should the server just "ignore" these nodes or actively reject them?
#
# Clixon has a strict implementation of the features so that setting
# data with disabled features is same as if they are not present in the Yang.
# Which means no cli syntax or edit operations were syntactically allowed
# (and therefore invalid).
# Magic line must be first in script (see README.md) # Magic line must be first in script (see README.md)
s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi
@ -31,7 +47,7 @@ cat <<EOF > $cfg
EOF EOF
cat <<EOF > $fyang cat <<EOF > $fyang
module $APPNAME{ module example{
yang-version 1.1; yang-version 1.1;
namespace "urn:example:clixon"; namespace "urn:example:clixon";
prefix ex; prefix ex;
@ -44,9 +60,6 @@ module $APPNAME{
feature B{ feature B{
description "This test feature is disabled"; description "This test feature is disabled";
} }
feature C{
description "This test feature is disabled";
}
leaf x{ leaf x{
if-feature A; if-feature A;
type "string"; type "string";

2
test/test_upgrade_interfaces.sh
View file

@ -270,7 +270,7 @@ cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config"> <clixon-config xmlns="http://clicon.org/config">
<CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE> <CLICON_CONFIGFILE>$cfg</CLICON_CONFIGFILE>
<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR> <CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>
<CLICON_FEATURE>*:*</CLICON_FEATURE> <CLICON_FEATURE>interfaces:if-mib</CLICON_FEATURE>
<CLICON_YANG_MAIN_DIR>$dir</CLICON_YANG_MAIN_DIR> <CLICON_YANG_MAIN_DIR>$dir</CLICON_YANG_MAIN_DIR>
<CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK> <CLICON_SOCK>/usr/local/var/$APPNAME/$APPNAME.sock</CLICON_SOCK>
<CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR> <CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>