Save state of netconf chunked framing between inputs
Treat EBADF in internal protocol (triggered by freebsd+notifications)
Test: chunked encoding: use printf instead of echo
* First hello is 1.0 EOM framing, then successing rpc is chunked framing
* See
* [Netconf framing](https://github.com/clicon/clixon/issues/50), and
* [Clixon does not switch to chunked framing after NETCONF 1.1 is negotiated](https://github.com/clicon/clixon/issues/314)
* C:
* Moved netconf framing code from netconf application to clixon lib
* Test:
* New expecteof_netconf and adjusted other expect scripts to handle NETCONF 1.1 framing
* Note that edits to the candidate database or locks will be lost
* To force the CLI to exit if backend restarts, undef `PROTO_RESTART_RECONNECT`
* This is an effect of the fix of [Broken pipe error seen in client (cli) when backend restarts and CLICON_SOCK is recreated](https://github.com/clicon/clixon/issues/312), the CLI behavior on backend restart is changed.
* Added `eof` parameter to `clicon_rpc()` and `clicon_rpc1()` and error handling modified
* Updated command-line usage of cli,netconf,backend apps in sync with clixon-docs user manual
* Documented differences between protocol stubs for external vs internal netconf
* Added ancestor config cache indicating wether the node or an ancestor is config false or true
* Improved yang cardinality lookup
* Added yang_init(), called from all apps using yang
NETCONF hello errors, such as wrong session-id, prefix, namespace terminates session
* Instead of returning an rpc-error reply
Changed YANG PATCH enabling:
* Now: `./configure --enable-yang-patch`
* Before: set YANG_PATCH constant in `include/clixon_custom.h`
Refactored Makefile for static linking
This include plugin module init and exit functions
New type: clixon_plugin_t exposed via public API while struct clixon_plugin is private
- Makefile changes for static linkage using "LINKAGE=static"
- Moved nacm external init to later stage in backend_main
* Hello message semantics has been made stricter according to RFC 6241 Sec 8.1, for example:
* A client MUST send a <hello> element.
* Each peer MUST send at least the base NETCONF capability, "urn:ietf:params:netconf:base:1.1" (or 1.0 for RFC 4741)
* The netconf client will terminate (close the socket) if the client does not comply
* You can set `CLICON_NETCONF_HELLO_OPTIONAL` to true to use the old behavior of essentially ignoring hellos.
* New clixon-config@2020-03-08.yang revision
* Added: `CLICON_NETCONF_HELLO_OPTIONAL`
* The base capability has been changed to "urn:ietf:params:netconf:base:1.1" following RFC6241.
* Netconf split lines input (input fragments) fixed
* Netconf input split on several lines, eg using stdin: "<a>\nfoo</a>]]>]]>" could under some circumstances be split so that only "</a>]]>]]>" be properly processed. This could also happen to a socket receiving a sub-string and then after a delay receive the rest.
* Fixed by storing residue and add that to the input string if later input is received on the same socket.
* Use [https://github.com/clicon/libevhtp](https://github.com/clicon/libevhtp) instead of [https://github.com/criticalstack/libevhtp](https://github.com/criticalstack/libevhtp) as a source of the evhtp source
* A new configuration option `CLICON_CONFIGDIR` has been added for loading of extra config files
* If not given, only the main configfile is loaded.
* If given, and if the directory exists, the files in this directory will be loaded alphabetically AFTER the main config file in the following way:
* leaf values are overwritten
* leaf-list values are appended
* You can override file setting with `-E <dir>` command-line option.
* New clixon-config@2020-10-01.yang revision
* Added option for configuration directory: `CLICON_CONFIGDIR`
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
* All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
* Requests such as: `<rpc><edit-config>...` will not be accepted.
* You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
* This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
* Added: stats RPC for clixon XML and memory statistics.
* Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
* ca_daemon: Called just after a server has "daemonized", ie put in background.
* ca_trans_commit_done: Called when all plugin commits have been done.
* Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
* `clixon_xml_parse_string()`
* `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
* `clixon_json_parse_string()`
* `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
* `xml_bind_yang_rpc()`
* `xml_bind_yang_rpc_reply()`
* `xml_bind_yang()`
* `xml_bind_yang0()`
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
* Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
* Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
* Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
w-level general upgrades and as a complement to module-specific upgrade.
* Called on startup after initial XML parsing, but before module-specific upgrades
* Enabled by definign the `.ca_datastore_upgrade`
* [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
* `xml_spec_populate` family of functions extended with three-value return values
* -1: error, 0: parse OK, 1: parse and YANG binding OK.
* `xml_parse` and `json_parse` API changes
* Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
* Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
* New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
* Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
* Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
Major New features
* New and updated search functions using xpath, api-path and instance-id
* New search functions using api-path and instance_id:
* C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
* Binary search optimization in lists for indexed leafs in all three formats.
* This improves search performance to O(logN) which is drastical improvements for large lists.
* You can also register explicit indexes for making binary search (not only list keys)
* For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)
API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
* Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
* New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"
Minor changes
* Test framework
* Added `-- -S <file>` command-line to main example to be able to return any state to main example.
* Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
* clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
* partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
* New clixon-lib@2019-08-13.yang revision
* Bugfix: If `ietf-netconf.yang` was imported from any yang module, client/backend communication stops working.
* Restconf RFC 8040 increased feature compliance
* Cache-Control: no-cache added in HTTP responses (RFC Section 5.5)
* Restconf monitoring capabilities (RFC Section 9.1)
* Added support for Yang extensions
* New plugin callback: ca_extension
* Main backend example includes example code on how to implement a Yang extension in a plugin.
* JSON changes
* Non-pretty-print output removed all extra spaces.
* Example: `{"nacm-example:x": 42}` --> {"nacm-example:x":42}`
* Empty JSON container changed from `null` to `{}`.
* Empty list and leafs remain as `null`
* Removed unnecessary configure dependencies
* libnsl, libcrypt, libm, if_vlan,...
* pseudo-plugin added, to enable callbacks also for main programs. Useful for extensions
* Yang Unique statements with multiple schema identifiers did not work on some platforms due to memory error.
The reason is to be conservative with the API. However, less used functions, such as xpath_vec_bool(), xpath_vec_ctx() and xpath_vec_flag() are changed with a new `nsc`parameter, which should be set to NULL in most cases.
