Commit graph

83 commits

Author SHA1 Message Date
Olof hagsand
783b0a4857 * YANG when statement in conjunction with grouping/uses/augment
* Several cases were not implemented fully according to RFC 7950
    * Do not extend default values if when statements evaluate to false
    * Do not allow edit-config of nodes if when statements evaluate to false (Sec 8.3.2)
    * If a key leaf is defined in a grouping that is used in a list, the "uses" statement MUST NOT have a "when" statement. (See 7.21.5)
  * See [yang uses's substatement when has no effect #218](https://github.com/clicon/clixon/issues/2$
2021-05-13 14:26:02 +02:00
Olof hagsand
ac51cb0293 * Fix: XPath:s used in netconf (eg get-config) did not correctly access default values 2021-05-07 13:07:34 +02:00
Olof hagsand
9087694b58 Fixed: [Backend can not read datastore with container named config #147](https://github.com/clicon/clixon/issues/147) 2021-03-13 17:15:00 +01:00
Olof hagsand
5692072d36 * Fixed: [CLIXON is not waiting for the hello message #184](https://github.com/clicon/clixon/issues/184)
* Hello message semantics has been made stricter according to RFC 6241 Sec 8.1, for example:
  * A client MUST send a <hello> element.
  * Each peer MUST send at least the base NETCONF capability, "urn:ietf:params:netconf:base:1.1" (or 1.0 for RFC 4741)
  * The netconf client will terminate (close the socket) if the client does not comply
  * You can set `CLICON_NETCONF_HELLO_OPTIONAL` to true to use the old behavior of essentially ignoring hellos.
* New clixon-config@2020-03-08.yang revision
  * Added: `CLICON_NETCONF_HELLO_OPTIONAL`
* The base capability has been changed to "urn:ietf:params:netconf:base:1.1" following RFC6241.
2021-03-10 14:18:16 +01:00
Olof hagsand
b7991d9b39 * Made a separate Clixon datastore XML/JSON top-level symbol
* Replaces the hardcoded "config" keyword.
  * Implemented by a compile-time option called `DATASTORE_TOP_SYMBOL` option in clixon_custom.h
* Tests: added endtest to all tests. Removed all premature exits if BE=0
2021-03-05 14:15:15 +01:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
804b329301 rearranged default code to be outside of copying in xml_get inner loop, and that no defaults are computed if no yang binding is made 2020-09-09 15:52:54 +02:00
Olof hagsand
1a2e074539 - Added newline after restconf evhtp GET reply 2020-08-24 22:31:12 +02:00
Olof hagsand
25245a12d2 code documentation 2020-07-12 10:59:12 +02:00
Olof hagsand
593ed2ef3b Added const char * in multiple places 2020-07-02 19:25:48 +02:00
shmuels
58ea4eba27 Changed code so that it will be compatible with c++.
1. Added #ifdef extern "C" to public header files.
2. Changed variables that are key words in c++, to different names (like ns instead of namespace).
3. Changed char* to const char* for constant strings.
2020-07-02 15:00:53 +03:00
Olof hagsand
6629f3d780 * Added new function xml_child_index_each() to iterate over the children of an XML node according to the order defined by an explicit index variable. 2020-05-20 15:21:59 +02:00
Olof hagsand
1c99bd6a9b * xml-stats moved from clixon-config.yang as state data to an rpc datastatsin clixon-lib.yang
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
2020-04-23 22:58:57 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
04017c97ba Length of xml vector in many structs changed from to since it is a vector size, not byte size. This includes transaction_data_t 2020-04-21 10:17:09 +02:00
Olof hagsand
5be5b02a84 NACM write path delete works as well 2020-04-20 22:29:09 +02:00
Olof hagsand
8897df202e NACM RFC8341 datanode write paths 2020-04-18 17:45:47 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
aa191152cb minor things for clixon.4.4 2020-03-27 11:20:35 +01:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
c5e2039cac Removed prefix parameter to xml_new
Added xml_new_body() by mgsmith
2020-03-17 11:18:22 +01:00
Olof hagsand
28fad0303a * Fixed: Datastore read on startup got fixed default values.
* Fixed: Default values only worked on leafs, not typedefs.
2020-03-13 14:49:37 +01:00
Olof hagsand
7425a3b520 * C-API change: xml_new() changed from xml_new(name, xp, ys) to xml_new(name, prefix, xp, type)
* If you have used, `ys`, add `xml_spec_set(x, ys)` after the statement
  * If you have `xml_type_set(x, TYPE)` or `xml_prefix_set(x, PREFIX)` immediately after the statement, you can remove those and set them directly as: `xml_new(name, PREFIX, xp, TYPE)`
2020-03-12 22:29:28 +01:00
Olof hagsand
0f54899ae4 * Added option: CLICON_CLI_BUF_START and CLICON_CLI_BUF_THRESHOLD so you can change the start and
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
  * Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
  * Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
  * Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
2020-03-04 11:27:43 +01:00
Olof hagsand
75b20936d3 Moved code from clixon_xml.c to namespace and io functions to reduce size of core xml file 2020-03-02 09:43:57 +01:00
Olof hagsand
451adfaf1f * Explicit search indexes
* Added object-based `clixon_xvec` as a new programming construct for contiguous XML object vectors.
2020-02-28 12:16:16 +01:00
Olof hagsand
a71c256898 * New clixon-config@2020-02-22.yang revision
* C-code changes:
  - Replaced stream uri:s w constants
  - Replaced large debug print code with single clicon_log_xml
  - Restconf put and post handling refactored using new parse API
2020-02-24 15:25:06 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
9575d10887 use cbuf for xml value code 2019-11-18 21:16:12 +01:00
Olof hagsand
d9136c8972 * Stricter handling of multi-namespace handling
* This occurs in cases where there are more than one XML namespaces in a config tree, such as `augment`:ed trees.
  * Affects all parts of the system, including datastore, backend, restconf and cli.
* Invalid api-path syntax (eg non-matching yang) error changed from 412 operation-failed to 400 Bad request invalid-value, or unknown-element.
2019-09-29 14:45:08 +02:00
Olof hagsand
ee329ee382 * RESTCONF "depth" query parameter supported
* C API change: Added `depth` parameter to function `clicon_xml2cbuf`, default is -1.
2019-08-17 10:54:13 +02:00
Olof hagsand
f319c18374 RESTCONF JSON identity had wrong namespace in sub-objetcs 2019-08-09 14:39:38 +02:00
Olof hagsand
bb2eaa3cc7 * Corrected problem with namespace context cache, was not always cleared when adding new subtrees. 2019-08-08 11:29:01 +02:00
Olof hagsand
c97346921b Netconf operation attribute namespace check is enforced 2019-07-31 16:45:48 +02:00
Olof hagsand
2d9d204f69 * RESTCONF "insert" and "point" query parameters supported
* Yang Netconf leaf/leaf-list insert support
  * For "ordered-by user" leafs and leaf-lists, the insert and value/key attributes are supported according to RFC7950 Sections 7.7.9 and 7.8.6

* Fixed RESTCONF api-path leaf-list selection was not made properly
2019-07-31 14:04:27 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
c79baf1b1f * Optimized validation of large lists
* New xmldb_get1() returning actual cache - not a copy. This has lead to some householding instead of just deleting the copy
  * xml_diff rewritten to work linearly instead of O(2)
  * New xml_insert function using tree search. The new code uses this in insertion xmldb_put and defaults. (Note previous xml_insert renamed to xml_wrap_all)
2019-04-14 14:36:41 +02:00
Olof hagsand
e29cd7cfb9 * Optimized validation by making xml_diff work on raw cache tree (not copies)
* xmldb_get() removed unnecessary config option
2019-04-07 15:55:53 +02:00
Olof hagsand
b5f1f7346e Structural C-code change: Merged yang_spec and yang_node types into yang_stmt 2019-04-02 12:27:33 +02:00
Olof hagsand
6ff36a2894 * Added xml_wrap function that adds an XML node above a node as a wrapper
* also renamed `xml_insert` to `xml_wrap_all`.
* Added `clicon_argv_get()` function to get the user command-line options, ie the args in `-- <args>`. This is an alternative to using them passed to `plugin_start()`.
2019-03-27 16:04:14 +01:00
Olof hagsand
98fb6eceef * Clixon configuration file top-level symbols has changed to clixon-configand namespace check is enforced.
* clixon-config YAML file has new revision: 2019-03-05.
  * New URN and changed top-level symbol to `clixon-config`
* Removed obsolete `_CLICON_XML_NS_STRICT` variable and `CLICON_XML_NS_STRICT` config option.
2019-03-06 09:42:55 +01:00
Olof Hagsand
ff3ff0daa9 * There was a problem with ordered-by-user for XML children that appeared in so\
me circumstances and difficult to trigger. Entries entered by the user did not \
appear in the order they were entered. This should now be fixed by adding an enumeration to xml children for sorting ordered by user properly
2019-02-27 18:20:23 +01:00
Olof Hagsand
ceb5df2050 * New xml_isancestor xml function
* Fixed memleak test for upgrade
2019-02-27 16:33:51 +01:00
Olof hagsand
560110b4e8 * New backend startup and upgrade support, see [doc/startup.md] for details
* Datastore files contain RFC7895 module-state information
2019-02-26 15:48:15 +01:00
Olof hagsand
ea8c84a372 Added new log function: clicon_log_xml() for logging XML tree 2019-02-13 11:35:49 +01:00
Olof hagsand
14343810ca xml_cmp() compares numeric nodes based on string value [https://github.com/clicon/clixon/issues/64] 2019-02-04 11:49:53 +01:00
Olof hagsand
ffecebf32a * NACM Data node READ access module support (RFC8341 3.4.5)
* Access control points added for `get` and `get-config` in addition to incoming rpc.
   * RFC 8341 Example A.2 implemented, see: [test/test_nacm_module.sh]
* Added `username` argument on `xmldb_put()` datastore function for NACM data-node write checks
* Added `xml_rootchild_node()` lib function as variant of `xml_rootchild()`
2019-01-27 13:26:15 +01:00
Olof hagsand
0103d58994 2019 2019-01-02 15:48:30 +01:00