Commit graph

46 commits

Author SHA1 Message Date
Olof hagsand
77b4468eb3 Updated copyright statements to 2022 2022-01-18 16:36:00 +01:00
Olof hagsand
5388aace12 Refactoring for better performance of large auto-cli specs
* Fixed: [very slow execution of load_set_file #288](https://github.com/clicon/clixon/issues/288)
* New `clixon-lib@2021-11-11.yang` revision
  * Modified option: RPC stats extended with YANG stats
* Modified `clixon-config@2021-11-11.yang` revision
  * Added option:
    * CLICON_PLUGIN_CALLBACK_CHECK
    * Enable to make plugin context check before and after all callbacks.
* Added statistics for YANG: number of objects and memory used
* Use the treeref no-copy option of CLIgen to reduce memory
* Refactored cli-generation/autocli-start code
* Refactored cligen glue functions to use cligen_eval directly (remove clicon_eval,clixon_cligen_eval)
2021-11-25 12:04:05 +01:00
Olof hagsand
0626de9431 * RPC replies now verified with YANG
* Stricter checking of outgoing RPC replies from server
  * See [RPC output not verified by yang](https://github.com/clicon/clixon/issues/283)
  * This lead to some corrections of RPC replies in system code
2021-11-16 22:12:14 +01:00
Olof hagsand
51316d5d61 Fixed: [type leafref in type union ineffective](https://github.com/clicon/clixon/issues/277)
* Leafrefs and identityrefs in unions were not validated correctly
2021-11-12 10:52:29 +01:00
Olof hagsand
78f5a6983c Pagination draft 2021-09-21 11:22:40 +02:00
Olof hagsand
980718178a * YANG Leafref feature update
* Closer adherence to RFC 7950. Some of this is changed behavior, some is new feature.
  * Essentially instead of looking at the referring leaf, context is referred(target) node
  * Validation uses referred node
    * Validation changed to use type of referred node, instead of just "string"
  * Auto-cli
    * Changed to use type of referred node for typecheck
    * Completion uses referred node
  * Required instance / less strict validation
    * New: Leafrefs must refer to existing data leaf ONLY IF YANG `required-instance` is true
    * Previous: All leafrefs must refer to existing data leaf node
* Fixed: [Autocli does not offer completions for leafref to identityref #254](https://github.com/clicon/clixon/issues/254)
2021-08-16 17:14:27 +02:00
Olof hagsand
603f9724ce * Fixed: YANG when was not properly implemented for LEAF default values 2021-08-03 13:38:31 +02:00
Olof hagsand
00645ee52b * Added linenumbers to all YANG symbols for better debug and errors
* Improved error messages for YANG identityref:s and leafref:s by adding original line numbers
2021-08-03 12:53:37 +02:00
Olof hagsand
babdc6f496 * Fixed: [OpenConfig BGP afi-safi and when condition issues #249](https://github.com/clicon/clixon/issues/249)
* YANG when was not properly implemented for default values
* Improved error message on leafref validation errors
2021-08-02 15:48:38 +02:00
Olof hagsand
dee081646c * Fixed: SEGV in clixon_netconf_lib functions from internal errors including validation.
* Check xerr argument both before and after call on netconf lib functions
2021-07-27 10:31:16 +02:00
Olof hagsand
89f843f919 * Fixed: [uses oc-if:interface-ref error with openconfig #233](https://github.com/clicon/clixon/issues/233) 2021-07-01 11:55:45 +02:00
Olof hagsand
b40dafb691 * Event exit API changed to a single decrementing counter where 1 means exit.
* Removed: `clicon_exit_reset()`
  * Changed: `clicon_exit_set()` --> `clixon_exit_set(int nr)`
  * Changed: `clicon_exit_get()` --> `clixon_exit_get()`
* native restconf special case upgrade handling from http/1 -> http/2 and restconf restart: delay the restart using event exit counter
2021-06-16 11:58:13 +02:00
Olof hagsand
965cce5e5d * Added SM_RUNNING_STARTUP to translation table
* Fixed https://github.com/clicon/clixon/issues/224 yet again by addressing a mixed xml and yang namespace xpath case
2021-05-27 15:05:50 +02:00
Olof hagsand
e0c3f5467c * Added new startup-mode: running-startup: First try running db, if it is empty try startup db.
* See [Can startup mode to be extended to support running-startup mode? #234](https://github.com/clicon/clixon/issues/234)
* Improved error message on failed MUST condition
2021-05-27 13:18:01 +02:00
Olof hagsand
ec4d26febf * Fixed again: [xpath function count did not work properly #224](https://github.com/clicon/clixon/issues/224)
* Must statements namespace context were given by yang node, but this is not correct, should be xml
2021-05-26 14:00:20 +02:00
Olof hagsand
783b0a4857 * YANG when statement in conjunction with grouping/uses/augment
* Several cases were not implemented fully according to RFC 7950
    * Do not extend default values if when statements evaluate to false
    * Do not allow edit-config of nodes if when statements evaluate to false (Sec 8.3.2)
    * If a key leaf is defined in a grouping that is used in a list, the "uses" statement MUST NOT have a "when" statement. (See 7.21.5)
  * See [yang uses's substatement when has no effect #218](https://github.com/clicon/clixon/issues/2$
2021-05-13 14:26:02 +02:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
e46f6f4a36 * Error-type changed from protocol to application for data-not-unique netconf/restconf errors
* Added new revision of main example yang: `clixon-example@2020-12-01.yang`
* Fixed [YANG: key statement in rpc/notification list #148](https://github.com/clicon/clixon/issues/148)
  * Do not check uniqueness among lists without keys
2020-12-01 18:58:20 +01:00
Olof hagsand
c32950c8a9 * Added message-id attributes in error and hello replies
* See [namespace prefix nc is not supported in full #154](https://github.com/clicon/clixon/issues/154)
* Removed mandatory loading of clixon_restconf.yang
2020-12-01 12:14:30 +01:00
Olof hagsand
6eb18da5e9 * Multi-socket feature (eg IPv4/IPv6 http/https) to restconf evhtp
* Added by-ref parameter to `ys_cv_validate()` returning which sub-yang spec was validated in a union.
2020-11-15 12:34:40 +01:00
Olof hagsand
1614bcd19c Cleanup master for 4.9.0.PRE 2020-10-19 20:06:20 +02:00
Olof hagsand
4d35ace95f fix of performance enhancement commit for unique constraint 2020-10-16 15:43:37 +02:00
Olof hagsand
3d1ac2692d Fixed: Performance enhancement of unique list check (of duplicate keys) 2020-10-16 12:48:51 +02:00
Olof hagsand
f8ac8cc14b typo on list check 2020-10-08 16:04:31 +02:00
Olof hagsand
5c67bcd364 validation of list keys rearrangement of code 2020-10-08 15:32:23 +02:00
Olof hagsand
469cbe0d64 Fixed: Accepted added subtrees containing lists with duplicate keys 2020-10-08 15:15:22 +02:00
Olof hagsand
8ae647c32b * More explanatory validation error messages for when and augments error messages.
* Example: error-message: `Mandatory variable` -> `Mandatory variable of edit-config in module ietf-netconf`.
2020-10-07 20:57:49 +02:00
Olof hagsand
9984cfa1c8 Fixed: [default state data returned with get-config](https://github.com/clicon/clixon/issues/140)
* Generalized default code for both config and state
2020-09-27 17:14:28 +02:00
Olof hagsand
da54dae396 Added sanity check that a yang module name matches the filename 2020-09-25 13:39:13 +02:00
Olof hagsand
c616aa1569 Added support for XPATH functions:
* `contains`,
  * `derived-from` and `derived-from-or-self`
    * in particular in augment/when statements as shown in eg RFC 7950.
2020-09-22 22:40:42 +02:00
Olof hagsand
1ef3c0db7a Fixed: Do not check min/max constraints on state data in config validate code 2020-09-08 13:25:28 +02:00
Olof hagsand
2c96cf71ea Changed main example: removed translate yang and replaced testcases w table/parameter 2020-09-08 11:39:55 +02:00
Olof hagsand
558abb1f9b Corrected error message for list min/max-value to comply to RFC 7950: a proper path is now returned, peviously only the list symbol. it is also exposed in the CLI correctly.
* Example: `<error-path>/c/a1</error-path>`
2020-08-27 15:52:31 +02:00
Olof hagsand
cb58b2bb12 log warning, error doc 2020-07-26 16:16:32 +02:00
Vladimir Isaev
cc8e727f85 Added error message for each CLICON_YANG_UNKNOWN_ANYDATA occasion 2020-07-24 15:46:38 +03:00
Olof hagsand
839dc3231d Merge branch 'mycpp/new-test' of https://github.com/shmuelnatan/clixon into shmuelnatan-mycpp/new-test 2020-07-02 17:20:38 +02:00
shmuels
58ea4eba27 Changed code so that it will be compatible with c++.
1. Added #ifdef extern "C" to public header files.
2. Changed variables that are key words in c++, to different names (like ns instead of namespace).
3. Changed char* to const char* for constant strings.
2020-07-02 15:00:53 +03:00
Olof hagsand
99b01040a7 * Fixed: [Double free when using libxml2 as regex engine #117](https://github.com/clicon/clixon/issues/117)
* added libxml2 support in test_pattern.sh when libxml2 is configured
2020-07-02 13:26:15 +02:00
Olof hagsand
dafc6d10e0 Added option to treat unknown XML (wrt YANG) as anydata. 2020-05-03 22:03:33 +02:00
Olof hagsand
12d1b67250 * Stricter validation detecting duplicate container or leaf in XML.
* Eg `<x><a/><a/></x>` is invalid if `a` is anything else than leaf or leaf-list
2020-05-02 17:30:02 +02:00
Olof hagsand
82529a2f16 * Experimental optimzations of yang-bind and sort for large lists
* Enabled by compile-time options: `OPTIMIZE_45_BIND` and `OPTIMIZE_45_SORT`
2020-05-01 17:08:32 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
c1903916f9 * Created two sub-files (clixon_validate.c and clixon_api_path.c) from large lib/src/clixon_xml_map.c source file. 2020-01-01 11:12:54 +01:00