Commit graph

253 commits

Author SHA1 Message Date
Olof hagsand
a77cd27abb * Fixed: [namespace prefix nc is not supported](https://github.com/clicon/clixon/issues/143)
* Added more sanity checks on incoming top-level rpc and hello messages, including verifying top-level namespace
2020-10-17 18:04:46 +02:00
Olof hagsand
6465022f57 Find non-config nodes and mark/return error with better error message 2020-10-15 13:55:44 +02:00
Olof hagsand
5c67bcd364 validation of list keys rearrangement of code 2020-10-08 15:32:23 +02:00
Olof hagsand
9984cfa1c8 Fixed: [default state data returned with get-config](https://github.com/clicon/clixon/issues/140)
* Generalized default code for both config and state
2020-09-27 17:14:28 +02:00
Olof hagsand
27bb0a1d2b * Fixed: Restconf failed put/post could leave residue in candidate causing errors in next put/post
* Added -v option for backend plugins to generate validation error
2020-09-10 18:09:17 +02:00
Olof hagsand
804b329301 rearranged default code to be outside of copying in xml_get inner loop, and that no defaults are computed if no yang binding is made 2020-09-09 15:52:54 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
dc1ad560f9 * Bundle internal NETCONF on RESTCONF: A RESTCONF operation could produce several (up to four) internal NETCONF messages between RESTCONF server and backend. These have now been bundled into one.
* NACM recovery user session is now properly enforced. This means that if `CLICON_NACM_CREDENTIALS` is `except` (default), then a specific `CLICON_NACM_RECOVERY_USER` can make any edits and bypass NACM rules.
* If a default value is replaced by an actual value, RESTCONF return values have changed from `204 No Content` to `201 Created`
* clixon-config.yang: Removed default valude of CLICON_NACM_RECOVERY_USER
2020-08-11 14:49:30 +02:00
Olof hagsand
a7737c283c restricted NACM recovery user 2020-08-06 21:43:27 +02:00
Olof hagsand
65733ffe69 * Added yang-binding yb parameter to xmldb_get0() and all xmldb get functions. 2020-08-03 13:57:04 +02:00
Olof hagsand
23bcee8185 * Netconf lock/unlock behaviour changed to adhere to RFC 6241
* Changed commit lock error tag from "lock denied" to "in-use".
  * Changed unlock error message from "lock is already held" to #lock not active" or "lock held by other session".
  * Fixed [lock candidate succeeded even though it is modified #110](https://github.com/clicon/clixon/issues/110)
2020-07-28 16:13:34 +02:00
Olof hagsand
57b104bc25 * Checked yang binding validity which lead to:
* Changed and enhanced several `bad-element` error replies to `unknown-element` with more detailed error-message.
2020-07-25 16:09:24 +02:00
Olof hagsand
ff5462ecac Replaced the global variable debug with access function: clicon_debug_get(). 2020-06-09 16:04:49 +02:00
Olof hagsand
14b03f63a3 all clixon event functions prepended with clixon_
Replaced fcgi getparam code with clixon-data parameters
2020-06-05 16:19:03 +02:00
Olof hagsand
a455b15f03 Changed --with-restconf directive to have package as argument, with either null, evhtp or fcgi.
This effects configure script and makefiles
2020-06-04 11:26:38 +02:00
Olof hagsand
29235d5547 Removed and enabled code for compile options: RESTART_PLUGIN_RPC, XML_NEW_DIFFERENTIATE, OPTIMIZE_45_BIND and OPTIMIZE_45_SORT 2020-05-11 15:29:58 +02:00
Olof hagsand
bf04131e54 Added decriptive error message when plugins produce invalid state XML. 2020-05-10 17:10:07 +02:00
Olof hagsand
82529a2f16 * Experimental optimzations of yang-bind and sort for large lists
* Enabled by compile-time options: `OPTIMIZE_45_BIND` and `OPTIMIZE_45_SORT`
2020-05-01 17:08:32 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
1c99bd6a9b * xml-stats moved from clixon-config.yang as state data to an rpc datastatsin clixon-lib.yang
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
2020-04-23 22:58:57 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
476f1d84d3 Fix a bind rpc assert on unknown namespace 2020-04-23 11:06:25 +02:00
Olof hagsand
ba59e22fd7 NACM RFC341 datanode paths, read operation 2020-04-14 11:38:29 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
916c1aabd7 NULL->/ for state callbacks. Clarified socket close error 2020-03-26 20:42:42 +01:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
b726c5008e renamed xml_spec_populate*() to xml_bind_yang*() 2020-03-17 16:11:58 +01:00
Olof hagsand
71efe18f94 NACM datanode write rules have been changed from looking at datastore being chekend (eg running/candidate/startup) to *only* look at running.
Fixed: NACM datanode write problem: read/write/exec default rules did not work.
2020-03-12 14:58:30 +01:00
Olof hagsand
cdcffa768f * Replaced compile option VALIDATE_STATE_XML with runtime option CLICON_VALIDATE_STATE_XML.
* XML childvec Compile error
2020-03-04 11:46:30 +01:00
Olof hagsand
0f54899ae4 * Added option: CLICON_CLI_BUF_START and CLICON_CLI_BUF_THRESHOLD so you can change the start and
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
  * Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
  * Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
  * Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
2020-03-04 11:27:43 +01:00
Olof hagsand
2d521d52c8 * Added clixon-stats in clixon-config.yang for clixon XML and memory statistics. 2020-03-01 10:59:24 +01:00
Olof hagsand
a71c256898 * New clixon-config@2020-02-22.yang revision
* C-code changes:
  - Replaced stream uri:s w constants
  - Replaced large debug print code with single clicon_log_xml
  - Restconf put and post handling refactored using new parse API
2020-02-24 15:25:06 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
c7d6f69a85 * Session-id CLI functionality delayed: "lazy evaluation"
* From a cli perspective this is a revert to 4.1 behaviour, where the cli does not immediately exit on start if the backend is not running, but with the new session-id function
2020-02-05 15:37:25 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
d68a829862 * Added namespace-context parameter nsc to xpath_first and xpath_vec, (xpath_vec_nsc and xpath_first_nsc` are removed). 2019-12-07 20:46:13 +01:00
Olof hagsand
835f9030d2 * State callbacks provided by user are validated. If they are invalid an internal error is returned.
* Fixed multi-namespace for augmented state which was not covered in 4.2.0.
* The multi-namespace augment state may rearrange the XML namespace attributes.
* Mandatory variables can no longer be deleted.
2019-11-11 21:03:11 +01:00
Olof hagsand
84c94b2c0e recover user as config option instead of constant 2019-10-24 20:43:03 +02:00
Olof hagsand
44138c0071 internal netconf hello mechanism to obtain session-id 2019-10-24 11:46:06 +02:00
Olof Hagsand
9a6478e267 Merge branch 'master' of https://github.com/clicon/clixon 2019-10-19 20:58:07 +00:00
Olof Hagsand
28a62ecb11 memleak 2019-10-19 20:58:03 +00:00
Olof hagsand
1fa45067a7 typo 2019-10-19 22:56:25 +02:00
Olof hagsand
b624e911b8 * Internal backend socket protocol changed: uint32_t session-id added, see clixon_proto.h
* * Changed session-id handing. Instead of using pid of peer process, a proper session id generated by the server is used, following RFC6241.
2019-10-19 19:52:09 +02:00
Olof hagsand
279614d64f * The backend socket has now support of credentials of peer clients
* Added: CLICON_NACM_CREDENTIALS and CLICON_NACM_RECOVERY_USER
2019-10-18 19:33:23 +02:00
Olof hagsand
03acf8e19c * XPATH canonical form implemented for NETCONF get and get-config. 2019-10-09 22:13:04 +02:00
Olof hagsand
d9136c8972 * Stricter handling of multi-namespace handling
* This occurs in cases where there are more than one XML namespaces in a config tree, such as `augment`:ed trees.
  * Affects all parts of the system, including datastore, backend, restconf and cli.
* Invalid api-path syntax (eg non-matching yang) error changed from 412 operation-failed to 400 Bad request invalid-value, or unknown-element.
2019-09-29 14:45:08 +02:00
Olof hagsand
ee329ee382 * RESTCONF "depth" query parameter supported
* C API change: Added `depth` parameter to function `clicon_xml2cbuf`, default is -1.
2019-08-17 10:54:13 +02:00
Olof hagsand
10a2dbe8ec * Added experimental binary search API function: xml_binsearch
* Added content parameter to `clicon_rpc_get` (-1 or CONTENT_ALL is default)
* Reverted change in clixon-lib.yang
2019-08-14 11:13:24 +02:00
Olof hagsand
8b7b7b0f60 * RESTCONF "content" query parameter supported
* New clixon-lib@2019-08-13.yang revision
* Bugfix: If `ietf-netconf.yang` was imported from any yang module, client/backend communication stops working.
2019-08-13 13:21:11 +02:00