experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

more upgrade-interfaces test cases

Browse source
This commit is contained in:
Olof hagsand 2019-03-28 10:56:53 +01:00
parent 772d93963c
commit b10c3c5727
3 changed files with 174 additions and 91 deletions
Download patch file Download diff file Expand all files Collapse all files

21
doc/startup.md
View file

@ -113,7 +113,7 @@ a set of current yang models that loads a datastore with old or even
obsolete data. obsolete data.
Note that this feature is only available if Note that this feature is only available if
[module-state](module-state) in the datastore is enabled. [module-state](#module-state) in the datastore is enabled.
If the module-state of the startup configuration does not match the If the module-state of the startup configuration does not match the
module-state of the backend daemon, a set of _upgrade_ callbacks are module-state of the backend daemon, a set of _upgrade_ callbacks are
@ -153,7 +153,7 @@ One example of registering a "catch-all" upgrade:
upgrade_callback_register(h, xml_changelog_upgrade, NULL, 0, 0, NULL); upgrade_callback_register(h, xml_changelog_upgrade, NULL, 0, 0, NULL);
``` ```
Another example are fine-grained stepwise upgrades of a singlemodule [upgrade example](#example-upgrade): Another example are fine-grained stepwise upgrades of a single module [upgrade example](#example-upgrade):
``` ```
upgrade_callback_register(h, upgrade_2016, "urn:example:interfaces", upgrade_callback_register(h, upgrade_2016, "urn:example:interfaces",
20140508, 20160101, NULL); 20140508, 20160101, NULL);
@ -175,20 +175,19 @@ When Clixon loads a startup datastore with outdated modules, the matching
upgrade callbacks will be called. upgrade callbacks will be called.
Note the following: Note the following:
* Upgrade callbacks will _not_ be called for data that is up-to-date with the current system * Upgrade callbacks _will_ _not_ be called for data that is up-to-date with the current system
* Upgrade callbacks will _not_ be called if there is no module-state in the datastore, or if module-state support is disabled. * Upgrade callbacks _will_ _not_ be called if there is no module-state in the datastore, or if module-state support is disabled.
* Upgrade callbacks _will_ be called if the datastore contains a version of a module that is older than the module loaded in Clixon. * Upgrade callbacks _will_ be called if the datastore contains a version of a module that is older than the module loaded in Clixon.
* Upgrade callbacks _will_ also be called if the datastore contains a version of a module that is not present in Clixon - an obsolete module. * Upgrade callbacks _will_ also be called if the datastore contains a version of a module that is not present in Clixon - an obsolete module.
Re-using the previous stepwise example, if a datastore is loaded based on revision 20140508 by a system supporting revision 20180220, the following two callbacks will be made: Re-using the previous stepwise example, if a datastore is loaded based on revision 20140508 by a system supporting revision 2018-02-20, the following two callbacks are made:
``` ```
upgrade_2016(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret); upgrade_2016(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret);
upgrade_2018(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret); upgrade_2018(h, <xml>, "urn:example:interfaces", 20140508, 20180220, NULL, cbret);
``` ```
Note that the example shown is a template for an upgrade function. It Note that the example shown is a template for an upgrade function. It
gets the nodes of an yang module given by `namespace` and the gets the nodes of an yang module given by `namespace` and the
(outdated) `from` revision, and iterates through them. Actual (outdated) `from` revision, and iterates through them.
upgrading code is in [the example](../example/example_backend.c).
If no action is made by the upgrade calback, and thus the XML is not If no action is made by the upgrade calback, and thus the XML is not
upgraded, the next step is XML/Yang validation. upgraded, the next step is XML/Yang validation.
@ -202,18 +201,22 @@ configuration.
### Example upgrade ### Example upgrade
[The example](../example/example_backend.c) and [test](../test/test_upgrade_interfaces.sh) shos the code for upgrading of an interface module. The example is inspired by the ietf-interfaces module that made a subset of the upgrades shown in the examples. The example and shows the code for upgrading of an interface module. The example is inspired by the ietf-interfaces module that made a subset of the upgrades shown in the examples.
The code is split in two steps. The `upgrade_2016` callback does the following transforms: The code is split in two steps. The `upgrade_2016` callback does the following transforms:
* Move /if:interfaces-state/if:interface/if:admin-status to /if:interfaces/if:interface/ * Move /if:interfaces-state/if:interface/if:admin-status to /if:interfaces/if:interface/
* Move /if:interfaces-state/if:interface/if:statistics to if:interfaces/if:interface/ * Move /if:interfaces-state/if:interface/if:statistics to if:interfaces/if:interface/
* Rename /interfaces/interface/description to /interfaces/interface/descr * Rename /interfaces/interface/description to /interfaces/interface/descr
While the `upgrade_2018` callback does the following transforms: The `upgrade_2018` callback does the following transforms:
* Delete /if:interfaces-state * Delete /if:interfaces-state
* Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr * Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr
* Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000 * Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000
Please consult the `upgrade_2016` and `upgrade_2018` functions in [the
example](../example/example_backend.c) and
[test](../test/test_upgrade_interfaces.sh) for more details.
## Extra XML ## Extra XML
If the Yang validation succeeds and the startup configuration has been committed to the running database, a user may add "extra" XML. If the Yang validation succeeds and the startup configuration has been committed to the running database, a user may add "extra" XML.

2
lib/src/clixon_options.c
View file

@ -419,7 +419,7 @@ clicon_option_str_set(clicon_handle h,
* @endcode * @endcode
* Note that -1 can be both error and value. * Note that -1 can be both error and value.
* This means that it should be used together with clicon_option_exists() and * This means that it should be used together with clicon_option_exists() and
* supply a defualt value as shown in the example. * supply a default value as shown in the example.
*/ */
int int
clicon_option_int(clicon_handle h, clicon_option_int(clicon_handle h,

242
test/test_upgrade_interfaces.sh
View file

@ -8,16 +8,15 @@
# draft-wang-netmod-module-revision-management-01 # draft-wang-netmod-module-revision-management-01
# The example here is simplified and also extended. # The example here is simplified and also extended.
# It has also been broken up into two parts to test a series of upgrades. # It has also been broken up into two parts to test a series of upgrades.
# These are the operations (authentic): # These are the operations (authentic move/delete are from ietf-interfaces):
# Move /if:interfaces-state/if:interface/if:admin-status to # Move /if:interfaces-state/if:interface/if:admin-status to (2016)
# /if:interfaces/if:interface/ # /if:interfaces/if:interface/
# Move /if:interfaces-state/if:interface/if:statistics to # Move /if:interfaces-state/if:interface/if:statistics to (2016)
# if:interfaces/if:interface/ # if:interfaces/if:interface/
# Delete /if:interfaces-state # Delete /if:interfaces-state (2018)
# These are extra added for test: # Rename /interfaces/interface/description to /interfaces/interface/descr (2016)
# Rename /interfaces/interface/description to /interfaces/interface/descr # Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr (2018)
# Wrap /interfaces/interface/descr to /interfaces/interface/docs/descr # Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000 (2018)
# Change type /interfaces/interface/statistics/in-octets to decimal64 and divide all values with 1000
# #
# Magic line must be first in script (see README.md) # Magic line must be first in script (see README.md)
@ -185,9 +184,8 @@ module interfaces{
} }
EOF EOF
# Create startup db revision from 2014-05-08 to be upgraded to 2018-02-20
# Create startup db revision example-a and example-b 2017-12-01 # This is 2014 syntax
# this should be automatically upgraded to 2017-12-20
cat <<EOF > $dir/startup_db cat <<EOF > $dir/startup_db
<config> <config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library"> <modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
@ -230,41 +228,6 @@ cat <<EOF > $dir/startup_db
</config> </config>
EOF EOF
# Wanted new XML
# Note interface e2 is not moved
cat <<EOF > $dir/wanted
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>interfaces</name>
<revision>2018-02-20</revision>
<namespace>urn:example:interfaces</namespace>
</module>
</modules-state>
<interfaces xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<type>eth</type>
<admin-status>up</admin-status>
<docs><descr>First interface</descr></docs>
<statistics>
<in-octets>54326.432</in-octets>
<in-unicast-pkts>8458765</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<type>eth</type>
<admin-status>down</admin-status>
</interface>
</interfaces>
</config>
EOF
XML='<interfaces xmlns="urn:example:interfaces"><interface><name>e0</name><docs><descr>First interface</descr></docs><type>eth</type><admin-status>up</admin-status><statistics><in-octets>54326.432</in-octets><in-unicast-pkts>8458765</in-unicast-pkts></statistics></interface><interface><name>e1</name><type>eth</type><admin-status>down</admin-status></interface></interfaces>'
# Create configuration # Create configuration
cat <<EOF > $cfg cat <<EOF > $cfg
<clixon-config xmlns="http://clicon.org/config"> <clixon-config xmlns="http://clicon.org/config">
@ -285,49 +248,166 @@ cat <<EOF > $cfg
</clixon-config> </clixon-config>
EOF EOF
# Start new system from old datastore # Start from startup and upgrade, check running
mode=startup testrun(){
runxml=$1
# -u means trigger example upgrade # -u means trigger example upgrade
new "test params: -s $mode -f $cfg -- -u" new "test params: -s startup -f $cfg -- -u"
# Bring your own backend # Bring your own backend
if [ $BE -ne 0 ]; then if [ $BE -ne 0 ]; then
# kill old backend (if any) # kill old backend (if any)
new "kill old backend" new "kill old backend"
sudo clixon_backend -zf $cfg sudo clixon_backend -zf $cfg
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
err err
fi
new "start backend -s startup -f $cfg -- -u"
start_backend -s startup -f $cfg -- -u
fi fi
new "start backend -s $mode -f $cfg -- -u" new "waiting"
start_backend -s $mode -f $cfg -- -u sleep $RCWAIT
fi
new "waiting"
sleep $RCWAIT
new "kill old restconf daemon" new "kill old restconf daemon"
sudo pkill -u www-data clixon_restconf sudo pkill -u www-data clixon_restconf
new "start restconf daemon" new "start restconf daemon"
start_restconf -f $cfg start_restconf -f $cfg
new "waiting" new "waiting"
sleep $RCWAIT sleep $RCWAIT
new "Check running db content" new "Check running db content"
expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' "^<rpc-reply><data>$XML</data></rpc-reply>]]>]]>$" expecteof "$clixon_netconf -qf $cfg" 0 '<rpc><get-config><source><running/></source></get-config></rpc>]]>]]>' "^<rpc-reply><data>$runxml</data></rpc-reply>]]>]]>$"
new "Kill restconf daemon" new "Kill restconf daemon"
stop_restconf stop_restconf
if [ $BE -ne 0 ]; then
new "Kill backend"
# Check if premature kill
pid=`pgrep -u root -f clixon_backend`
if [ -z "$pid" ]; then
err "backend already dead"
fi
# kill backend
stop_backend -f $cfg
fi
}
XML='<interfaces xmlns="urn:example:interfaces"><interface><name>e0</name><docs><descr>First interface</descr></docs><type>eth</type><admin-status>up</admin-status><statistics><in-octets>54326.432</in-octets><in-unicast-pkts>8458765</in-unicast-pkts></statistics></interface><interface><name>e1</name><type>eth</type><admin-status>down</admin-status></interface></interfaces>'
new "1. Upgrade from 2014 to 2018-02-20"
testrun "$XML"
# This is "2016" syntax
cat <<EOF > $dir/startup_db
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>interfaces</name>
<revision>2016-01-01</revision>
<namespace>urn:example:interfaces</namespace>
</module>
</modules-state>
<interfaces xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<admin-status>up</admin-status>
<type>eth</type>
<descr>First interface</descr>
<statistics>
<in-octets>54326432</in-octets>
<in-unicast-pkts>8458765</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<type>eth</type>
<admin-status>down</admin-status>
</interface>
</interfaces>
<interfaces-state xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<admin-status>down</admin-status>
<statistics>
<in-octets>946743234</in-octets>
<in-unicast-pkts>218347</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<admin-status>up</admin-status>
</interface>
<interface>
<name>e2</name>
<admin-status>testing</admin-status>
</interface>
</interfaces-state>
</config>
EOF
# 2. Upgrade from intermediate 2016-01-01 to 2018-02-20
new "2. Upgrade from intermediate 2016-01-01 to 2018-02-20"
testrun "$XML"
# Again 2014 syntax
cat <<EOF > $dir/startup_db
<config>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>42</module-set-id>
<module>
<name>interfaces</name>
<revision>2014-05-08</revision>
<namespace>urn:example:interfaces</namespace>
</module>
</modules-state>
<interfaces xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<type>eth</type>
<description>First interface</description>
</interface>
<interface>
<name>e1</name>
<type>eth</type>
</interface>
</interfaces>
<interfaces-state xmlns="urn:example:interfaces">
<interface>
<name>e0</name>
<admin-status>up</admin-status>
<statistics>
<in-octets>54326432</in-octets>
<in-unicast-pkts>8458765</in-unicast-pkts>
</statistics>
</interface>
<interface>
<name>e1</name>
<admin-status>down</admin-status>
</interface>
<interface>
<name>e2</name>
<admin-status>testing</admin-status>
</interface>
</interfaces-state>
</config>
EOF
rm $if2018
# Original XML
XML='<interfaces xmlns="urn:example:interfaces"><interface><name>e0</name><description>First interface</description><type>eth</type></interface><interface><name>e1</name><type>eth</type></interface></interfaces><interfaces-state xmlns="urn:example:interfaces"><interface><name>e0</name><admin-status>up</admin-status><statistics><in-octets>54326432</in-octets><in-unicast-pkts>8458765</in-unicast-pkts></statistics></interface><interface><name>e1</name><admin-status>down</admin-status></interface><interface><name>e2</name><admin-status>testing</admin-status></interface></interfaces-state>'
new "3. No 2018 (upgrade) model -> dont trigger upgrade"
testrun "$XML"
#rm $if2014
#new "4. No model at all"
#testrun "$XML"
if [ $BE -ne 0 ]; then if [ $BE -ne 0 ]; then
new "Kill backend"
# Check if premature kill
pid=`pgrep -u root -f clixon_backend`
if [ -z "$pid" ]; then
err "backend already dead"
fi
# kill backend
stop_backend -f $cfg
rm -rf $dir rm -rf $dir
fi fi