experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
clixon/CHANGELOG.md
Olof hagsand 10d841f4be Added make test and travis experiment
2019-02-04 14:13:20 +01:00

896 lines
44 KiB
Markdown
Raw Blame History

# Clixon Changelog
## 3.9.0 (Preliminary Target: Mid-January 2019)
### Planned new features
* [Roadmap](ROADMAP.md)
### Major New features
* Correct XML namespace handling
* XML multiple modules was based on non-strict semantics so that yang modules were found by iterating thorugh namespaces until a match was made. This did not adhere to proper [XML namespace handling](https://www.w3.org/TR/2009/REC-xml-names-20091208) as well as strict Netconf and Restconf namespace handling, which causes problems with overlapping names and false positives, and most importantly, with standard conformance.
* There are still the following non-strict namespace handling:
* Everything in ietf-netconf base syntax with namespace `urn:ietf:params:xml:ns:netconf:base:1.0` is default and need not be explicitly given
* edit-config xpath select statement does not support namespaces
* notifications do not support namespaces.
* Below see netconf old (but wrong) netconf RPC:
```
<rpc>
<my-own-method/>
</rpc>
<rpc-reply>
<route>
<address-family>ipv4</address-family>
</route>
</rpc-reply>
```
This is the currently correct Netconf RPC:
```
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> # xmlns may be ommitted
<my-own-method xmlns="urn:example:my-own">
</rpc>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<route xmlns="urn:ietf:params:xml:ns:yang:ietf-routing">
<address-family>ipv4</address-family>
</route>
</rpc-reply>
```
* Another example for restconf rpc with new correct syntax. Note that while Netconf uses xmlns attribute syntax, Restconf uses module name prefix. First the request:
```
POST http://localhost/restconf/operations/example:example)
Content-Type: application/yang-data+json
{
"example:input":{
"x":0
}
}
```
then the reply:
```
HTTP/1.1 200 OK
{
"example:output": {
"x": "0",
"y": "42"
}
}
```
* To keep previous non-strict namespace handling (backwards compatible), set CLICON_XML_NS_STRICT to false.
* See https://github.com/clicon/clixon/issues/49
* Yang code upgrade (RFC7950)
* YANG parser cardinality checked (https://github.com/clicon/clixon/issues/48)
* See https://github.com/clicon/clixon/issues/84
* RPC method input parameters validated
* see https://github.com/clicon/clixon/issues/47
* Support of submodule, include and belongs-to.
* Parsing of standard yang files supported, such as:
* https://github.com/openconfig/public - except [https://github.com/clicon/clixon/issues/60].
* See [test/test_openconfig.sh]
* https://github.com/YangModels/yang - except vendor-specific specs
* See [test/test_yangmodels.sh]
* Improved "unknown" handling
* More precise Yang validation and better error messages
* Example: adding bad-, missing-, or unknown-element error messages, instead of operation-failed.
* Validation of mandatory choice and recursive mandatory containers
* Yang load file configure options changed
* `CLICON_YANG_DIR` is changed from a single directory to a path of directories
* Note CLIXON_DATADIR (=/usr/local/share/clixon) need to be in the list
* CLICON_YANG_MAIN_FILE Provides a filename with a single module filename.
* CLICON_YANG_MAIN_DIR Provides a directory where all yang modules should be loaded.
* NACM (RFC8341)
* Experimental support, no performance enhancements and need further testing
* Incoming RPC Message validation is supported (3.4.4)
* Data Node Access validation is supported (3.4.5), except:
* rule-type data-node path is not supported
* Outgoing noitification aithorization is _not_ supported (3.4.6)
* RPC:s are supported _except_:
* `copy-config`for other src/target combinations than running/startup (3.2.6)
* `commit` - NACM is applied to candidate and running operations only (3.2.8)
* Client-side RPC:s are _not_ supported.
* Recovery user "_nacm_recovery" added.
### API changes on existing features (you may need to change your code)
* Added `username` argument on `xmldb_put()` datastore function for NACM data-node write checks
* Rearranged yang files
* Moved and updated all standard ietf and iana yang files from example and yang/ to `yang/standard`.
* Moved clixon yang files from yang to `yang/clixon`
* New configure option to disable standard yang files: `./configure --disable-stdyangs`
* This is to make it easier to use standard IETF/IANA yang files in separate directory
* Renamed example yang from example.yang -> clixon-example.yang
* clixon_cli -p (printspec) changed semantics to add new yang path dir (see minor changes).
* Date-and-time type now properly uses ISO 8601 UTC timezone designators.
* Eg 2008-09-21T18:57:21.003456 is changed to 2008-09-21T18:57:21.003456Z
* Renamed yang file `ietf-netconf-notification@2008-07-01.yang` to `clixon-rfc5277`.
* Fixed validation problems, see [https://github.com/clicon/clixon/issues/62]
* Name confusion, the file is manually constructed from the rfc.
* Changed prefix to `ncevent`
* Stricter YANG choice validation leads to enforcement of structures like: `choice c{ mandatory true; leaf x` statements. `x` was not previously enforced.
* Many hand-crafted validation messages have been removed and replaced with generic validations, which may lead to changed rpc-error messages.
* CLICON_XML_SORT option (in clixon-config.yang) has been removed and set to true permanently. Unsorted XML lists leads to slower performance and old obsolete code can be removed.
* Strict namespace setting can be a problem when upgrading existing database files, such as startup-db or persistent running-db, or any other saved XML file.
* Removed `delete-config` support for candidate db since it is not supported in RFC6241.
* Switched the order of `error-type` and `error-tag` in all netconf and restconf error messages to comply to RFC order.
* Yang parser is stricter (see above) which may break parsing of existing yang specs.
* XML namespace handling is corrected (see above)
* For backward compatibility set config option CLICON_XML_NS_LOOSE
* Yang parser functions have changed signatures. Please check the source if you call these functions.
* Add `<CLICON_YANG_DIR>/usr/local/share/clixon</CLICON_YANG_DIR>` to your configuration file, or corresponding CLICON_DATADIR directory for Clixon system yang files.
* Change all @datamodel:tree to @datamodel in all CLI specification files
* If you generate CLI code from the model (CLIXON_CLI_GENMODEL).
* For backward compatibility, define CLICON_CLI_MODEL_TREENAME_PATCH in clixon_custom.h
### Minor changes
* Added make test from top-level
* Added `xml_rootchild_node()` lib function as variant of `xml_rootchild()`
* Added -o "<option>=<value>" command-line option to all programs: backend, cli, netconf, restconf.
* Any config option from file can be overrided by giving them on command-line.
* Added -p <dir> command-line option to all programs: backend, cli, netconf, restconf.
* -p adds a new dir to the yang path dir. (same as -o CLICON_YAN_DIR=<dir>)
* Cligen uses posix regex while yang uses XSD. It differs in some aspects. A translator function has been added for `\d` -> `[0-9]` translation, there may be more.
* Added new clixon-lib yang module for internal netconf protocol. Currently only extends the standard with a debug RPC.
* Added three-valued return values for several validate functions where -1 is fatal error, 0 is validation failed and 1 is validation OK.
* This includes: `xmldb_put`, `xml_yang_validate_all`, `xml_yang_validate_add`, `xml_yang_validate_rpc`, `api_path2xml`, `api_path2xpath`
* Added new xml functions for specific types: `xml_child_nr_notype`, `xml_child_nr_notype`, `xml_child_i_type`, `xml_find_type`.
* Added example_rpc RPC to example backend
* Renamed xml_namespace[_set]() to xml_prefix[_set]()
* Changed all make tags --> make TAGS
* Keyvalue datastore removed (it has been disabled since 3.3.3)
* Removed return value ymodp from yang parse functions (eg yang_parse()).
* New config option: CLICON_CLI_MODEL_TREENAME defining name of generated syntax tree if CLIXON_CLI_GENMODEL is set.
* XML parser conformance to W3 spec
* Names lexically correct (NCName)
* Syntactically Correct handling of '<?' (processing instructions) and '<?xml' (XML declaration)
* XML prolog syntax for 'well-formed' XML
* <!DOCTYPE (ie DTD) is not supported.
### Corrected Bugs
* xml_cmp() compares numeric nodes based on string value [https://github.com/clicon/clixon/issues/64]
* xml_cmp() respects 'ordered-by user' for state nodes, which violates RFC 7950 [https://github.com/clicon/clixon/issues/63]. (Thanks JDL)
* XML<>JSON conversion problems [https://github.com/clicon/clixon/issues/66]
* CDATA sections stripped from XML when converted to JSON
* Restconf returns error when RPC generates "ok" reply [https://github.com/clicon/clixon/issues/69]
* xsd regular expression support for character classes [https://github.com/clicon/clixon/issues/68]
* added support for \c, \d, \w, \W, \s, \S.
* Removing newlines from XML data [https://github.com/clicon/clixon/issues/65]
* [ietf-netconf-notification@2008-07-01.yang validation problem #62](https://github.com/clicon/clixon/issues/62)
* Ignore CR(\r) in yang files for DOS files
* Keyword "min" (not only "max") can be used in built-in types "range" and "length" statements.
* Support for empty yang string added, eg `default "";`
* Removed CLI generation for yang notifications (and other non-data yang nodes)
* Some restconf error messages contained "rpc-reply" or "rpc-error" which have now been removed.
* getopt return value changed from char to int (https://github.com/clicon/clixon/issues/58)
* Netconf/Restconf RPC extra input arguments are ignored (https://github.com/clicon/clixon/issues/47)
### Known issues
* debug rpc added in example application (should be in clixon-config).
## 3.8.0 (6 Nov 2018)
### Major New features
* YANG Features
* Yang 1.1 feature and if-feature according to RFC 7950 7.20.1 and 7.20.2.
* See https://github.com/clicon/clixon/issues/41
* Features are declared via CLICON_FEATURE in the configuration file. Example below shows enabling (1) a specific feature; (2) all features in a module; (3) all features in all modules:
```
<CLICON_FEATURE>ietf-routing:router-id</CLICON_FEATURE>
<CLICON_FEATURE>ietf-routing:*</CLICON_FEATURE>
<CLICON_FEATURE>*:*</CLICON_FEATURE>
```
* logical combination of features not implemented, eg if-feature "not foo or bar and baz";
* ietf-netconf yang module added with candidate, validate, startup and xpath features enabled.
* YANG module library
* YANG modules according to RFC 7895 and implemented by ietf-yang-library.yang
* Enabled by configuration option CLICON_MODULE_LIBRARY_RFC7895 - enabled by default
* RFC 7895 defines a module-set-id. Configure option CLICON_MODULE_SET_ID is set and changed when modules change.
* Yang 1.1 notification support (RFC 7950: Sec 7.16)
* New event streams implementation with replay
* Generic stream support (both CLI, Netconf and Restconf)
* Added stream discovery according to RFC 5277 for netconf and RFC 8040 for restconf
* Enabled by configure options CLICON_STREAM_DISCOVERY_RFC5277 and CLICON_STREAM_DISCOVERY_RFC8040
* Configure option CLICON_STREAM_RETENTION is default number of seconds before dropping replay buffers
* See clicon_stream.[ch] for details
* Restconf stream notification support according to RFC8040
* start-time and stop-time query parameters
* Fork fcgi handler for multiple concurrent streams
* Set access/subscribe base URL with: CLICON_STREAM_URL (default "https://localhost") and CLICON_STREAM_PATH (default "streams")
* Replay support: start-time and stop-time query parameter support
* See [apps/restconf/README.md] for more details.
* Alternative restconf streams using pub/sub enabled by ./configure --enable-publish
* Set publish URL base with: CLICON_STREAM_PUB (default http://localhost/pub)
* RFC5277 Netconf replay supported
* Replay support is only in-memory and not persistent. External time-series DB could be added.
### API changes on existing features (you may need to change your code)
* Netconf hello capability updated to YANG 1.1 RFC7950 Sec 5.6.4
* Added urn:ietf:params:netconf:capability:yang-library:1.0
* Thanks @SCadilhac for helping out, see https://github.com/clicon/clixon/issues/39
* Major rewrite of event streams (as described above)
* If you used old event callbacks API, you need to switch to the streams API
* See clixon_stream.[ch]
* Old streams API which needs to be modified:
* clicon_log_register_callback() removed
* subscription_add() --> stream_add()
* stream_cb_add() --> stream_ss_add()
* stream_cb_delete() --> stream_ss_delete()
* backend_notify() and backend_notify_xml() - use stream_notify() instead
* Example uses "NETCONF" stream instead of "ROUTING"
* clixon_restconf and clixon_netconf changed command-line option from -D to -D `<level>` aligning with cli and backend
* Unified log handling for all clicon applications using command-line option: `-l e|o|s|f<file>`.
* The options stand for e:stderr, o:stdout, s: syslog, f:file
* Added file logging (`-l f` or `-l f<file>`) for cases where neither syslog nor stderr is usefpul.
* clixon_netconf -S is obsolete. Use `clixon_netconf -l s` instead.
* Comply to RFC 8040 3.5.3.1 rule: api-identifier = [module-name ":"] identifier
* The "module-name" was a no-op before.
* This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/foobar:modules-state
* Generilized top-level yang parsing functions
* Clarified semantics of main yang module:
* Command-line option -y MUST specify a filename
* Configure option CLICON_YANG_MODULE_MAIN MUST specify a module name
* yang_parse() changed to take either filename or module name and revision.
* Removed clicon_dbspec_name() and clicon_dbspec_name_set().
* Replace calls to yang_spec_main() with yang_spec_parse_module(). See for
example backend_main() and others if you need details.
### Minor changes
* Renamed test/test_auth*.sh tests to test/test_nacm*.sh
* YANG keywords "action" and "belongs-to" implemented by syntactically by parser (but not proper semantics).
* clixon-config YAML file has new revision: 2018-10-21.
* Allow new lines in CLI prompts
* uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters
* Added Configure option CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option).
* New function: clicon_conf_xml() returns configuration tree
* Obsoleted COMPAT_CLIV and COMPAT_XSL that were optional in 3.7
* Added command-line option `-t <timeout>` for clixon_netconf - quit after max time.
### Corrected Bugs
* No space after ampersand escaped characters in XML https://github.com/clicon/clixon/issues/52
* Thanks @SCadilhac
* Single quotes for XML attributes https://github.com/clicon/clixon/issues/51
* Thanks @SCadilhac
* Fixed https://github.com/clicon/clixon/issues/46 Issue with empty values in leaf-list
* Thanks @achernavin22
* Identity without any identityref:s caused SEGV
* Memory error in backend transaction revert
* Set dir /www-data with www-data as owner, see https://github.com/clicon/clixon/issues/37
### Known issues
* Netconf/Restconf RPC extra input arguments are ignored
* https://github.com/clicon/clixon/issues/47
* Yang sub-command cardinality not checked.
* https://github.com/clicon/clixon/issues/48
* Issue with bare axis names (XPath 1.0)
* https://github.com/clicon/clixon/issues/54
* Top-level Yang symbol cannot be called "config" in any imported yang file.
* datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanity.
* Namespace name relabeling is not supported.
* Eg: if "des" is defined as prefix for an imported module, then a relabeling using xmlns is not supported, such as:
```
<crypto xmlns:x="urn:example:des">x:des3</crypto>
```
## 3.7.0 (22 July 2018)
### Major New features
* YANG "must" and "when" Xpath-basedconstraints according to RFC 7950 Sec 7.5.3 and 7.21.5.
* Must and when Xpath constrained checked at validation/commit.
* YANG "identity" and "identityref", according to RFC 7950 Sec 7.18 and 9.10.
* Identities checked at validation/commit.
* CLI completion support of identity values.
* Example extended with iana-if-type RFC 7224 interface identities.
* Improved support for XPATH 1.0 according to https://www.w3.org/TR/xpath-10 using yacc/lex,
* Full suport of boolean constraints for "when"/"must", not only nodesets.
* See also API changes below.
* CDATA XML support (patch by David Cornejo, Netgate)
* Encode and decode (parsing) support
* Added cligen variable translation. Useful for processing input such as hashing, encryption.
* More info in example and FAQ.
* Example:
```
cli> translate value HAL
cli> show configuration
translate {
value IBM;
}
```
### API changes on existing features (you may need to change your code)
* YANG identity, identityref, must, and when validation support may cause applications that had not strictly enforced identities and constraints before.
* Restconf operations changed from prefix to module name.
* Proper specification for an operation is POST /restconf/operations/<module_name>:<rpc_procedure> HTTP/1.1
* See https://github.com/clicon/clixon/issues/31, https://github.com/clicon/clixon/pull/32 and https://github.com/clicon/clixon/issues/30
* Thanks David Cornejo and Dmitry Vakhrushev of Netgate for pointing this out.
* New XPATH 1.0 leads to some API changes and corrections
* Due to an error in the previous implementation, all XPATH calls on the form `x[a=str]` where `str` is a string (not a number or XML symbol), must be changed to: `x[a='str'] or x[a="str"]`
* This includes all calls to `xpath_vec, xpath_first`, etc.
* In CLI specs, calls to cli_copy_config() must change 2nd argument from `x[%s=%s]` to `x[%s='%s']`
* In CLI specs, calls to cli_show_config() may need to change third argument, eg
* `cli_show_config("running","text","/profile[name=%s]","name")` to `cli_show_config("running","text","/profile[name='%s']","name")`
* xpath_each() is removed
* The old API can be enabled by setting COMPAT_XSL in include/clixon_custom.h and recompile.
* Makefile change: Removed the make include file: clixon.mk and clixon.mk.in
* These generated the Makefile variables: clixon_DBSPECDIR, clixon_SYSCONFDIR, clixon_LOCALSTATEDIR, clixon_LIBDIR, clixon_DATADIR which have been replaced by generic autoconf variables instead.
* Removed cli callback vector functions. Set COMPAT_CLIV if you need to keep these functions in include/clixon_custom.h.
* Replace functions as follows in CLI SPEC files:
* cli_setv --> cli_set
* cli_mergev --> cli_merge
* cli_delv --> cli_del
* cli_debug_cliv --> cli_debug_cli
* cli_debug_backendv --> cli_debug_backend
* cli_set_modev --> cli_set_mode
* cli_start_shellv --> cli_start_shell
* cli_quitv --> cli_quit
* cli_commitv --> cli_commit
* cli_validatev --> cli_validate
* compare_dbsv --> compare_dbs
* load_config_filev --> load_config_file
* save_config_filev --> save_config_file
* delete_allv --> delete_all
* discard_changesv --> discard_changes
* cli_notifyv --> cli_notify
* show_yangv --> show_yang
* show_confv_xpath --> show_conf_xpath
* Changed `plugin_init()` backend return semantics: If returns NULL, _without_ calling clicon_err(), the module is disabled.
### Minor changes
* Clixon docker upgrade
* Updated the docker image build and example to a single clixon docker image.
* Example pushed to https://hub.docker.com/r/olofhagsand/clixon_example/
* Added systemd example files under example/systemd
* Added util subdir, with dedicated standalone xml, json, yang and xpath parser utility test programs.
* Validation of yang bits type space-separated list value
* Added -U <user> command line to clixon_cli and clixon_netconf for changing user.
* This is primarily for NACM pseudo-user tests
* Added a generated CLI show command that works on the generated parse tree with auto completion.
* A typical call is: show @datamodel:example, cli_show_auto("candidate", "json");
* The example contains a more elaborate example.
* Thanks ngashok for request, see https://github.com/clicon/clixon/issues/24
* Added XML namespace (xmlns) validation
* for eg <a xmlns:x="uri"><x:b/></a>
* ./configure extended with --enable-debug flag
* CFLAGS=-g ./configure deprecated
### Corrected Bugs
* Prefix of rpc was ignored (thanks Dmitri at netgate)
* https://github.com/clicon/clixon/issues/30
* Added cli return value also for single commands (eg -1)
* Fixed JSON unbalanced braces resulting in assert.
### Known issues
* Namespace name relabeling is not supported.
* Eg: if "des" is defined as prefix for an imported module, then a relabeling using xmlns is not supported, such as:
```
<crypto xmlns:x="urn:example:des">x:des3</crypto>
```
## 3.6.1 (29 May 2018)
### Corrected Bugs
* https://github.com/clicon/clixon/issues/23 clixon_cli failing with error
* The example included a reference to nacm yang file which did not exist and was not used
* Added clixon-config@2018-04-30.yang
## 3.6.0 (30 April 2018)
### Major changes:
* Experimental NACM RFC8341 Network Configuration Access Control Model, see [NACM](README_NACM.md).
* New CLICON_NACM_MODE config option, default is disabled.
* New CLICON_NACM_FILE config option, if CLICON_NACM_MODE is "external"
* Added username attribute to all internal RPC:s from frontend to backend
* Added NACM backend module in example
* Restructure and more generic plugin API for cli, backend, restconf, and netconf. See example further down and the [example](example/README.md)
* Changed `plugin_init()` to `clixon_plugin_init()` returning an api struct with function pointers. There are no other hardcoded plugin functions.
* Master plugins have been removed. Plugins are loaded alphabetically. You can ensure plugin load order by prefixing them with an ordering number, for example.
* Plugin RPC callback interface have been unified between backend, netconf and restconf.
* Backend RPC register callback function (Netconf RPC or restconf operation POST) has been changed from:
`backend_rpc_cb_register()` to `rpc_callback_register()`
* Backend RPC callback signature has been changed from:
`int cb(clicon_handle h, cxobj *xe, struct client_entry *ce, cbuf *cbret, void *arg)`
to:
`int cb(clicon_handle h, cxobj *xe, struct client_entry *ce, cbuf *cbret, void *arg)`
* Frontend netconf and restconf plugins can register callbacks as well with same API as backends.
* Moved specific plugin functions from apps/ to generic functions in lib/
* New config option CLICON_BACKEND_REGEXP to match backend plugins (if you do not want to load all).
* Added authentication plugin callback (ca_auth)
* Added clicon_username_get() / clicon_username_set()
* Removed some obscure plugin code that seem not to be used (please report if needed!)
* CLI parse hook
* CLICON_FIND_PLUGIN
* clicon_valcb()
* CLIXON_BACKEND_SYSDIR
* CLIXON_CLI_SYSDIR
* CLICON_MASTER_PLUGIN config variable
* Example of migrating a backend plugin module:
* Add all callbacks in a clixon_plugin_api struct
* Rename plugin_init() -> clixon_plugin_init() and return api as function value
* Rename backend_rpc_cb_register() -> rpc_callback_register() for any RPC/restconf operation POST calls
```
/* This is old style with hardcoded function names (eg plugin_start) */
int plugin_start(clicon_handle h, int argc, char **argv)
{
return 0;
}
int
plugin_init(clicon_handle h)
{
return 0;
}
/* This is new style with all function names in api struct */
clixon_plugin_api *clixon_plugin_init(clicon_handle h);
static clixon_plugin_api api = {
"example", /* name */
clixon_plugin_init, /* init */
NULL, /* start */
NULL, /* exit */
.ca_auth=plugin_credentials /* restconf specific: auth */
};
clixon_plugin_api *clixon_plugin_init(clicon_handle h)
{
return &api; /* Return NULL on error */
}
```
* Builds and installs a new restconf library: `libclixon_restconf.so` and clixon_restconf.h
* The restconf library can be included by a restconf plugin.
* Example code in example/Makefile.in and example/restconf_lib.c
* Restconf error handling for get, put and post. (thanks Stephen Jones, Netgate)
* Available both as xml and json (set accept header).
* Proper RFC 6241 Netconf error handling
* New functions added in clixon_netconf_lib.[ch]
* Datastore code modified for RFC 6241
### Minor changes:
* INSTALLFLAGS added with default value -s(strip).
* For debug do: CFLAGS=-g INSTALLFLAGS= ./configure
* plugin_start() callbacks added for restconf
* Authentication
* Example extended with http basic authentication for restconf
* Documentation in FAQ.md
* Updated ietf-netconf-acm to ietf-netconf-acm@2018-02-14.yang from RFC 8341
* The Clixon example has changed name from "routing" to "example" affecting all config files, plugins, tests, etc.
* Secondary backend plugin added
* Removed username to rpc calls (added below)
* README.md extended with new yang, netconf, restconf, datastore, and auth sections.
* The key-value datastore is no longer supported. Use the default text datastore.
* Added username to rpc calls to prepare for authorization for backend:
* clicon_rpc_config_get(h, db, xpath, xt) --> clicon_rpc_config_get(h, db, xpath, username, xt)
* clicon_rpc_get(h, xpath, xt) --> clicon_rpc_get(h, xpath, username, xt)
* Experimental: Added CLICON_TRANSACTION_MOD configuration option. If set,
modifications in validation and commit callbacks are written back
into the datastore. Requested by Stephen Jones, Netgate.
* Invalid key to api_path2xml gives warning instead of error and quit.
* Added restconf/operations get, see RFC8040 Sec 3.3.2:
* yang_find_topnode() and api_path2xml() schemanode parameter replaced with yang_class. Replace as follows: 0 -> YC_DATANODE, 1 -> YC_SCHEMANODE
* xml2json: include prefix in translation, so <a:b> is translated to {"a:b" ..}
* Use `<config>` instead of `<data>` when save/load configuration to file. This
enables saved files to be used as datastore without any editing. Thanks Matt, Netgate.
* Added Yang "extension" statement. This includes parsing unknown
statements and identifying them as extensions or not. However,
semantics for specific extensions must still be added.
* Renamed ytype_id and ytype_prefix to yarg_id and yarg_prefix, respectively
* Added cli_show_version()
### Corrected Bugs
* Showing syntax using CLI commands was broekn and is fixed.
* Fixed issue https://github.com/clicon/clixon/issues/18 RPC response issues reported by Stephen Jones at Netgate
* Fixed issue https://github.com/clicon/clixon/issues/17 special character in strings can break RPCs reported by David Cornejo at Netgate.
* This was a large rewrite of XML parsing and output due to CharData not correctly encoded according to https://www.w3.org/TR/2008/REC-xml-20081126.
* Fixed three-key list entry problem (reported by jdl@netgate)
* Translate xml->json \n correctly
* Fix issue: https://github.com/clicon/clixon/issues/15 Replace whole config
## 3.5.0 (12 February 2018)
### Major changes:
* Major Restconf feature update to comply to RFC 8040. Thanks Stephen Jones of Netgate for getting right.
* GET: Always return object referenced (and nothing else). ie, GET /restconf/data/X returns X.
* GET Added support for the following resources: Well-known, top-level resource, and yang library version,
* GET Single element JSON lists use {list:[element]}, not {list:element}.
* PUT Whole datastore
### Minor changes:
* Changed signature of plugin_credentials() restconf callback. Added a "user" parameter. To enable authentication and in preparation for access control a la RFC 6536.
* Added RFC 6536 ietf-netconf-acm@2012-02-22.yang access control (but not implemented).
* The following backward compatible options to configure have been _obsoleted_. If you havent already migrated this code you must do this now.
* `configure --with-startup-compat`. Configure option CLICON_USE_STARTUP_CONFIG is also obsoleted.
* `configure --with-config-compat`. The template clicon.conf.cpp files are also removed.
* `configure --with-xml-compat`
* New configuration option: CLICON_RESTCONF_PRETTY. Default true. Set to false to get more compact Restconf output.
* Default configure file handling generalized by Renato Botelho/Matt Smith. Config file FILE is selected in the following priority order:
* Provide -f FILE option when starting a program (eg clixon_backend -f FILE)
* Provide --with-configfile=FILE when configuring
* Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml
* Provide --sysconfig=<dir> when configuring then FILE is <dir>/etc/clixon.xml
* FILE is /usr/local/etc/clixon.xml
### Corrected Bugs
* yang max keyword was not supported for string type. Corrected by setting "max" to MAXPATHLEN
* Corrected "No yang spec" printed on tty when using leafref in CLI.
* Fixed error in xml2cvec. If a (for example) int8 value has range error (eg 1000), it was treated as an error and the program terminated. Now this is just logged and skipped. Reported by Fredrik Pettai.
### Known issues
## 3.4.0 (1 January 2018)
### Major changes:
* Optimized search performance for large lists by sorting and binary search.
* New CLICON_XML_SORT configuration option. Default is true. Disable by setting to false.
* Added yang ordered-by user. The default (ordered-by system) will now sort lists and leaf-lists alphabetically to increase search performance. Note that this may change outputs.
* If you need legacy order, either set CLICON_XML_SORT to false, or set that list to "ordered-by user".
* This replaces XML hash experimental code, ie xml_child_hash variables and all xmlv_hash_ functions have been removed.
* Implementation detail: Cached keys are stored in in yang Y_LIST nodes as cligen vector, see ys_populate_list()
* Datastore cache introduced: cache XML tree in memory for faster get access.
* Reads are cached. Writes are written to disk.
* New CLICON_XMLDB_CACHE configuration option. Default is true. To disable set to false.
* With cache, you cannot have multiple backends (with single datastore). You need to have a single backend.
* Thanks netgate for proposing this.
* Changed C functional API for XML creation and parsing for better coherency and closer YANG/XML integration. This may require your action.
* New yang spec parameter has been added to most functions (default NULL) and functions have been removed and renamed. You may need to change the XML calls as follows.
* xml_new(name, parent) --> xml_new(name, xn_parent, yspec)
* xml_new_spec(name, parent, spec) --> xml_new(name, parent, spec)
* clicon_xml_parse(&xt, format, ...) --> xml_parse_va(&xt, yspec, format, ...)
* clicon_xml_parse_file(fd, &xt, endtag) --> xml_parse_file(fd, endtag, yspec, &xt)
* clicon_xml_parse_string(&str, &xt) --> xml_parse_string(str, yspec, &xt)
* clicon_xml_parse_str(str, &xt) --> xml_parse_string(str, yspec, &xt)
* xml_parse(str, xt) --> xml_parse_string(str, yspec, &xt)
* Backward compatibility is enabled by (will be removed in 3.5.0:
```
configure --with-xml-compat
```
### Minor changes:
* Better semantic versioning, eg MAJOR/MINOR/PATCH, where increment in PATCH does not change API.
* Added CLICON_XMLDB_PRETTY option. If set to false, XML database files will be more compact.
* Added CLICON_XMLDB_FORMAT option. Default is "xml". If set to "json", XML database files uses JSON format.
* Clixon_backend now returns -1/255 on error instead of 0. Useful for systemd restarts, for example.
* Experimental: netconf yang rpc. That is, using ietf-netconf@2011-06-01.yang
formal specification instead of hardcoded C-code.
### Corrected Bugs
* Fixed bug that deletes running on startup if backup started with -m running.
When clixon starts again, running is lost.
The error was that the running (or startup) configuration may fail when
clixon backend starts.
The fix now makes a copy of running and copies it back on failure.
* datastore/keyvalue/Makefile was left behind on make distclean. Fixed by conditional configure. Thanks renato@netgate.com.
* Escape " in JSON names and strings and values
### Known issues
* Please use text datastore, key-value datastore no up-to-date
## 3.3.3 (25 November 2017)
Thanks to Matthew Smith, Joe Loeliger at Netgate; Fredrik Pettai at
SUNET for support, requests, debugging, bugfixes and proposed solutions.
### Major changes:
* Performance improvements
* Added xml hash lookup instead of linear search for better performance of large lists. To disable, undefine XML_CHILD_HASH in clixon_custom.h
* Netconf client was limited to 8K byte messages. New limit is 2^32 bytes.
* XML and YANG-based configuration file.
* New configuration files have .xml suffix, old have .conf.
* The yang model is yang/clixon-config.yang.
* You can run backward compatible mode using `configure --with-config-compat`
* In backward compatible mode both .xml and .conf works
* For migration from old to new, a utility is clixon_cli -x to print new format. Run the command and save in configuration file with .xml suffix instead.
```
> clixon_cli -f /usr/local/etc/example.conf -1x
<config>
<CLICON_CONFIGFILE>/usr/local/etc/example.xml</CLICON_CONFIGFILE>
<CLICON_YANG_DIR>/usr/local/share/example/yang</CLICON_YANG_DIR>
<CLICON_BACKEND_DIR>/usr/local/lib/example/backend</CLICON_BACKEND_DIR>
...
</config>
```
* Simplified backend daemon startup modes.
* The flags -IRCr are replaced with command-line option -s <mode>
* You use the -s to select the mode. Example: `clixon_backend -s running`
* You may also add a default method in the configuration file: `<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>`
* The configuration option CLICON_USE_STARTUP_CONFIG is obsolete
* Command-ine option `-I` is replaced with `-s init`
* `-CIr` is replaced with `-s running`
* Use `-s none` if you request no action on startu
* Backward compatibility is enabled by:
```
configure --with-startup-compat
```
* You can run in backward compatible mode where both -IRCr and -s options works. But if -s is given, -IRCr options willbe ignored.
* Extra XML has been added along with the new startup modes. Requested by Netgate.
* You can add extra XML with the -c option to the backend daemon on startup:
```
clixon_backend ... -c extra.xml
```
* You can also add extra XML by programming the plugin_reset() in the backend
plugin. The example application shows how.
* Clixon can now be compiled and run on Apple Darwin. Thanks SUNET.
### Minor changes:
* Fixed DESTDIR make install/uninstall and break immediately on errors
* Disabled key-value datastore. Enable with --with-keyvalue
* Removed mandatory requirements for BACKEND, NETCONF, RESTCONF and CLI dirs in the configuration file. If these are not given, no plugins will be loaded of that type.
* Restconf: http cookie sent as attribute in rpc restconf_post operations to backend as "id" XML attribute.
* Added option CLICON_CLISPEC_FILE as complement to CLICON_CLISPEC_DIR to
specify single CLI specification file, not only directory containing files.
* Replaced the following cli_ functions with their original cligen_functions:
cli_exiting, cli_set_exiting, cli_comment,
cli_set_comment, cli_tree_add, cli_tree_active,
cli_tree_active_set, cli_tree.
* Added a format parameter to clicon_rpc_generate_error() and changed error
printouts for backend errors, such as commit and validate. (Thanks netgate).
Example of the new format:
```
> commit
Sep 27 18:11:58: Commit failed. Edit and try again or discard changes:
protocol invalid-value Missing mandatory variable: type
```
* Added event_poll() function to check if data is available on specific file descriptor.
* Support for non-line scrolling in CLI, eg wrap lines. Thanks to Jon Loeliger for proposed solution. Set in configuration file with:
<CLICON_CLI_LINESCROLLING>0</CLICON_CLI_LINESCROLLING>
### Corrected Bugs
* Added floating point and negative number support to JSON
* When user callbacks such as statedata() call returns -1, clixon_backend no
longer silently exits. Instead a log is printed and an RPC error is returned.
Cred to Matt, netgate for pointing this out.
## 3.3.2 (Aug 27 2017)
### Known issues
* Please use text datastore, key-value datastore no up-to-date
* leafref occuring within lists: cli expansion does not work
### Major changes:
* Added support for YANG anyxml.
* Changed top-level netconf get-config and get to return `<data>` instead of `<data><config>` to comply to the RFC.
* If you use direct netconf get or get-config calls, you may need to handle the return XML differently.
* RESTCONF and CLI is not affected.
* Example:
```
Query:
<rpc><get/></rpc>
New reply:
<rpc-reply>
<data>
<a/> # Example data model
</data>
</rpc-reply>
Old reply:
<rpc-reply>
<data>
<config> # Removed
<a/>
</config> # Removed
</data>
</rpc-reply>
```
* Added support for yang presence and no-presence containers. Previous default was "presence".
* Empty containers will be removed unless you have used the "presence" yang declaration.
* Example YANG without presence:
```
container
nopresence {
leaf j {
type string;
}
}
```
If you submit "nopresence" without a leaf, it will automatically be removed:
```
<nopresence/> # removed
<nopresence> # not removed
<j>hello</j>
</nopresence>
```
* Added YANG RPC support for netconf, restconf and CLI. With example rpc documentation and testcase. This replaces the previous "downcall" mechanism.
* This means you can make netconf/restconf rpc calls
* However you need to register an RPC backend callback using the backend_rpc_cb_register() function. See documentation and example for more details.
* Example, the following YANG RPC definition enables you to run a netconf rpc.
```
YANG:
rpc myrpc {
input {
leaf name {
type string;
}
}
}
NETCONF:
<rpc><myrpc><name>hello</name><rpc>
RESTCONF:
curl -sS -X POST -d {"input":{"name":"hello"}} http://localhost/restconf/operations/myroute'
```
* Enhanced leafref functionality:
* Validation for leafref forward and backward references;
* CLI completion for generated cli leafrefs for both absolute and relative paths.
* Example, relative path:
```
leaf ifname {
type leafref {
path "../../interface/name";
}
}
```
* Added state data: Netconf `<get>` operation, new backend plugin callback: "plugin_statedata()" for retreiving state data.
* You can use netconf: `<rpc><get/></rpc>` and it will return both config and state data.
* Restconf GET will return state data also, if defined.
* You need to define state data in a backend callback. See the example and documentation for more details.
### Minor Changes
* Added xpath support for predicate: current(), eg /interface[name=current()/../name]
* Added prefix parsing of xpath, allowing eg /p:x/p:y, but prefix ignored.
* Corrected Yang union CLI generation and type validation. Recursive unions did not work.
* Corrected Yang pattern type escaping problem, ie '\.' did not work properly. This requires update of cligen as well.
* Compliance with RFC: Rename yang xpath to schema_nodeid and syntaxnode to datanode.
* Main yang module (CLICON_YANG_MODULE_MAIN or -y) can be an absolute file name.
* Removed 'margin' parameter of yang_print().
* Extended example with ietf-routing (not only ietf-ip) for rpc operations.
* Added yang dir with ietf-netconf and clixon-config yang specs for internal usage.
* Fixed bug where cli set of leaf-list were doubled, eg cli set foo -> foofoo
* Restricted yang (sub)module file match to match RFC6020 exactly
* Generic map_str2int generic mapping tables
* Removed vector return values from xmldb_get()
* Generalized yang type resolution to all included (sub)modules not just the topmost
## 3.3.1 (June 7 2017)
* Fixed yang leafref cli completion for absolute paths.
* Removed non-standard api_path extension from the internal netconf protocol so that the internal netconf is now fully standard.
* Strings in xmldb_put not properly encoded, eg eth/0 became eth.00000
## 3.3.0 (May 2017)
* Datastore text module is now default.
* Refined netconf "none" semantics in tests and text datastore
* Moved apps/dbctrl to datastore/
* Added connect/disconnect/getopt/setopt and handle to xmldb API
* Added datastore 'text'
* Configure (autoconf) changes
Removed libcurl dependency
Disable restconf (and fastcgi) with configure --disable-restconf
Disable keyvalue datastore (and qdbm) with configure --disable-keyvalue
* Created xmldb plugin api
Moved qdbm, chunk and xmldb to datastore keyvalue directories
Removed all other clixon dependency on chunk code
* cli_copy_config added as generic cli command
* cli_show_config added as generic cli command
Replace all show_confv*() and show_conf*() with cli_show_config()
Example: replace:
show_confv_as_json("candidate","/sender");
with:
cli_show_config("candidate","json","/sender");
* Alternative yang spec option -y added to all applications
* Many clicon special string functions have been removed
* The netconf support has been extended with lock/unlock
* clicon_rpc_call() has been removed and should be replaced by extending the
internal netconf protocol.
See downcall() function in example/routing_cli.c and
routing_downcall() in example/routing_backend.c
* Replace clicon_rpc_xmlput with clicon_rpc_edit_config
* Removed xmldb daemon. All xmldb acceses is made backend daemon.
No direct accesses by clients to xmldb API.
Instead use the rpc calls in clixon_proto_client.[ch]
In clients (eg cli/netconf) replace xmldb_get() in client code with
clicon_rpc_get_config().
If you use the vector arguments of xmldb_get(), replace as follows:
xmldb_get(h, db, api_path, &xt, &xvec, &xlen);
with
clicon_rpc_get_config(h, dbstr, api_path, &xt);
xpath_vec(xt, api_path, &xvec, &xlen)
* clicon_rpc_change() is replaced with clicon_rpc_edit_config().
Note modify argument 5:
clicon_rpc_change(h, db, op, apipath, "value")
to:
clicon_rpc_edit_config(h, db, op, apipath, `"<config>value</config>"`)
* xmdlb_put_xkey() and xmldb_put_tree() have been folded into xmldb_put()
Replace xmldb_put_xkey with xmldb_put as follows:
xmldb_put_xkey(h, "candidate", cbuf_get(cb), str, OP_REPLACE);
with
clicon_xml_parse(&xml, `"<config>%s</config>"`, str);
xmldb_put(h, "candidate", OP_REPLACE, cbuf_get(cb), xml);
xml_free(xml);
* Change internal protocol from clicon_proto.h to netconf.
This means that the internal protocol defined in clixon_proto.[ch] is removed
* Netconf startup configuration support. Set CLICON_USE_STARTUP_CONFIG to 1 to
enable. Eg, if backend_main is started with -CIr startup will be copied to
running.
* Added ".." as valid step in xpath
* Use restconf format for internal xmldb keys. Eg /a/b=3,4
* List keys with special characters RFC 3986 encoded.
* Replaced cli expand functions with single to multiple args
This change is _not_ backward compatible
This effects all calls to expand_dbvar() or user-defined
expand callbacks
* Replaced cli callback functions with single arg to multiple args
This change is _not_ backward compatible.
You are affected if you
(1) use system callbacks (i.e. in clixon_cli_api.h)
(2) write your own cli callbacks
If you use cli callbacks, you need to rewrite cli callbacks from eg:
`load("Comment") <filename:string>,load_config_file("filename replace");`
to:
`load("Comment") <filename:string>,load_config_file("filename", "replace");`
If you write your own, you need to change the callback signature from;
```
int cli_callback(clicon_handle h, cvec *vars, cg_var *arg)
```
to:
```
int cli_callback(clicon_handle h, cvec *vars, cvec *argv)
```
and rewrite the code to handle argv instead of arg.
These are the system functions affected:
cli_set, cli_merge, cli_del, cli_debug_backend, cli_set_mode,
cli_start_shell, cli_quit, cli_commit, cli_validate, compare_dbs,
load_config_file, save_config_file, delete_all, discard_changes, cli_notify,
show_yang, show_conf_xpath
* Added --with-cligen and --with-qdbm configure options
* Added union type check for non-cli (eg xml) input
* Empty yang type. Relaxed yang types for unions, eg two strings with different length.
## (Dec 2016)
* Dual license: both GPLv3 and APLv2
## (Feb 2016)
* Forked new clixon repository from clicon
Reference in a new issue View git blame Copy permalink