Commit graph

254 commits

Author SHA1 Message Date
Olof hagsand
1db43ca45b yang augment sanity checks 2020-10-05 15:49:22 +02:00
Olof hagsand
b3545871c0 * Added stricter check on schema-node identifier checking, such as for augments.
* These checks are now made at YANG loading time
2020-09-29 20:53:24 +02:00
Olof hagsand
c616aa1569 Added support for XPATH functions:
* `contains`,
  * `derived-from` and `derived-from-or-self`
    * in particular in augment/when statements as shown in eg RFC 7950.
2020-09-22 22:40:42 +02:00
Olof hagsand
1ef3c0db7a Fixed: Do not check min/max constraints on state data in config validate code 2020-09-08 13:25:28 +02:00
Olof hagsand
a61da8715d Fixed: [min-elements attribute prevents from deleting list entries #133](https://github.com/clicon/clixon/issues/133) 2020-09-08 10:47:43 +02:00
Olof hagsand
576533ed82 Changed singnature of ys_real_module and improved error handling 2020-08-27 11:01:09 +02:00
Olof hagsand
25f67d1eb9 Fixed: [(CLI) the description of a used grouping is shown instead of the encapsulating container #124](https://github.com/clicon/clixon/issues/124) 2020-08-19 15:55:55 +02:00
Olof hagsand
918bf6223b added CR delimiters to if-feature 2020-07-27 13:00:56 +02:00
Olof hagsand
f7b640810b * Enhanced Clixon if-feature handling:
* If-feature now supports and/or lists, such as: `if-feature "a and b"` and `if-feature "a or b or c"`. However, full if-feature-expr including `not` and nested boolean experessions is still not supported.
  * Sanity check: if an `if-feature` statement exists, a corresponding `feature` statement must exists that declares that feature.
2020-07-27 12:06:36 +02:00
Olof hagsand
490f7d8023 Two CLICON_YANG_UNKNOWN_ANYDATA issues: segv when exiting cli and diff errors on commit 2020-07-17 14:18:09 +02:00
Olof hagsand
62446db752 Fixed CLICON_YANG_UNKNOWN_ANYDATA for config and state data. This feature introduced in 4.5 didnt really work. 2020-07-14 19:56:42 +02:00
Olof hagsand
839dc3231d Merge branch 'mycpp/new-test' of https://github.com/shmuelnatan/clixon into shmuelnatan-mycpp/new-test 2020-07-02 17:20:38 +02:00
shmuels
58ea4eba27 Changed code so that it will be compatible with c++.
1. Added #ifdef extern "C" to public header files.
2. Changed variables that are key words in c++, to different names (like ns instead of namespace).
3. Changed char* to const char* for constant strings.
2020-07-02 15:00:53 +03:00
Olof hagsand
99b01040a7 * Fixed: [Double free when using libxml2 as regex engine #117](https://github.com/clicon/clixon/issues/117)
* added libxml2 support in test_pattern.sh when libxml2 is configured
2020-07-02 13:26:15 +02:00
Olof hagsand
9c82e97072 Added http support for evhtp (not only https) 2020-06-29 14:41:08 +02:00
Olof hagsand
0adcd94f3f * Added new cli show functions to work with cligen_output for cligen pageing to work. To acheive this, add a callback function as follows:
* xml2txt(...) --> xml2txt_cb(..., cligen_output)
  * xml2cli(...) --> xml2cli_cb(..., cligen_output)
  * clicon_xml2file(...) --> clicon_xml2file_cb(..., cligen_output)
  * xml2json(...) --> xml2json_cb(..., cligen_output)
  * yang_print(...) --> yang_print_cb(..., cligen_output)
2020-06-17 14:04:19 +02:00
Olof hagsand
e898dda016 * Auto-CLI enhancements
* A generated clispec including state (default @datanodestate) also generated along with the config clispec tree (default @datanode)
  * New mode `GT_HIDE` set by option `CLICON_CLI_GENMODEL_TYPE` to collapse non-presence containers that only contain a single list
  * Added a prfix for cli_show_config/cli_show_auto so that it can produce parseable output
  * Thanks dcornejo@netgate.com for trying it out and suggestions
2020-06-13 12:05:26 +02:00
Olof hagsand
fdf335fb2f new function ; Added valgrind mem check in leak tests; pattern CDATA tests 2020-05-29 12:39:07 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
ba59e22fd7 NACM RFC341 datanode paths, read operation 2020-04-14 11:38:29 +02:00
Olof hagsand
0fd71ec372 loosen yang check of keys in lists to not do it in rpc declarations 2020-03-27 15:44:05 +01:00
Olof hagsand
28fad0303a * Fixed: Datastore read on startup got fixed default values.
* Fixed: Default values only worked on leafs, not typedefs.
2020-03-13 14:49:37 +01:00
Olof hagsand
0e94937ccf Sanity check of mandatory key statement for Yang LISTs.
* If fails, exit with error message, eg: `Yang error: Sanity check failed: LIST vsDataContainer lacks key statement which MUST be present (See RFC 7950 Sec 7.8.2)`
  * Can be disabled by setting `CLICON_CLICON_YANG_LIST_CHECK` to `false`
2020-03-09 15:48:33 +01:00
Olof hagsand
75b20936d3 Moved code from clixon_xml.c to namespace and io functions to reduce size of core xml file 2020-03-02 09:43:57 +01:00
Olof hagsand
760cdda49a Fixed: Search function checked only own not for config false statement, should have checked all ancestors. 2020-02-28 14:05:56 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof Hagsand
a87e9cb246 clang warnings 2019-12-23 14:03:42 +00:00
Olof hagsand
52629d7b35 Added "canonical" global namespace context: nsctx_global 2019-11-15 17:04:13 +01:00
Olof hagsand
03acf8e19c * XPATH canonical form implemented for NETCONF get and get-config. 2019-10-09 22:13:04 +02:00
Olof hagsand
6df434093e * Restconf top-level operations GET root resource modified to comply with
RFC 8040 Sec 3.1
  * non-pretty print remove all spaces, eg `{"operations":{"clixon-example:client-rpc":[null]`
  * Replaced JSON `null` with `[null]` as proper empty JSON leaf/leaf-list encoding.
* [Cannot write to config using restconf example #91](https://github.com/clicon/clixon/issues/91)
  * Updated restconf documentation (the example was wrong)
* [clixon-lib yang revision file name update #92](https://github.com/clicon/clixon/issues/92)
  * Clixon-lib yang file had conflicting filename and internal yang revision.
  * This was only detected in the use-case when a whole dir was loaded.
  * Inserted sanity check in all yang parse routines.
  * Committed updated clixon-lib yang file that triggered the error
2019-08-24 15:30:43 +02:00
Olof hagsand
aa14f8ac2c RESTCONF PUT/POST -d {} media is enforced 2019-08-06 15:47:46 +02:00
Olof hagsand
70ebfa4d80 * Identity/identityref mapped between XML and JSON 2019-07-28 18:09:55 +02:00
Olof Hagsand
5f7d011654 memleaks 2019-07-25 11:39:17 +00:00
Olof hagsand
e7b60619da * Pushed tag to 4.0.1.PRE
* Restconf RFC 8040 increased feature compliance
  * Cache-Control: no-cache added in HTTP responses (RFC Section 5.5)
  * Restconf monitoring capabilities (RFC Section 9.1)
* Added support for Yang extensions
  * New plugin callback: ca_extension
  * Main backend example includes example code on how to implement a Yang extension in a plugin.
* JSON changes
  * Non-pretty-print output removed all extra spaces.
    * Example: `{"nacm-example:x": 42}` --> {"nacm-example:x":42}`
  * Empty JSON container changed from `null` to `{}`.
    * Empty list and leafs remain as `null`
* Removed unnecessary configure dependencies
  * libnsl, libcrypt, libm, if_vlan,...
* pseudo-plugin added, to enable callbacks also for main programs. Useful for extensions
* Yang Unique statements with multiple schema identifiers did not work on some platforms due to memory error.
2019-07-23 22:11:14 +02:00
Olof hagsand
6d46087109 Documented bug [Yang identityref XML encoding is not general #90](https://github.com/clicon/clixon/issues/90) 2019-07-11 12:11:45 +02:00
Olof hagsand
e44685a100 Replaced strchr with nodeid_split
Experimental code for replacing identityref lists with module:id instead of prefix:id
2019-07-10 17:18:34 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
dfa3970ab2 RESTCONF PUT list key problems 2019-06-08 16:32:56 +02:00
Olof Hagsand
42a974bf78 mem problem in refine/augment code 2019-06-05 09:54:13 +00:00
Olof hagsand
dc45600074 Yang 'refine' feature supported, According to RFC 7950 7.13.2 2019-06-05 10:23:49 +02:00
Olof hagsand
2aeb925521 * JSON parse and print improvements
* Integrated parsing with namespace translation and yang spec lookup
2019-06-03 16:40:54 +02:00
Olof hagsand
5706703ab4 Added support for inverted regexps 2019-05-29 14:39:36 +02:00
Olof hagsand
2fe185d683 * Support for multiple patterns as described in RFC7950 Section 9.4.7
* Added regex cache to type resolution
* Added compiled regexp parameter as part of internal yang type resolution functions
* All internal `ys_populate_*()` functions (except ys_populate()) have switched parameters: `clicon_handle, yang_stmt *)`
2019-05-29 11:39:09 +02:00
Olof hagsand
bc54f2d04c * Regexp improvements
* Added check for libxml in configure';
  * Added clixon_util_regexp utility function
* Yang state get improvements
  * Integrated state and config into same tree on retrieval, not separate trees
  * Added cli functions `cli_show_config_state()` and `cli_show_auto_state()` for showing combined config and state info.
  * Added integrated state in the main example: `interface/oper-state`.
  * Added performance tests for getting state, see [test/test_perf_state.sh].
2019-05-20 16:03:29 +02:00
Olof hagsand
d775eb5374 * Improved submodule implementation, as part of [Yang submodule import prefix restrictions #60](https://github.com/clicon/clixon/issues/60). 2019-05-14 14:09:02 +02:00
Olof hagsand
c529847790 * Yang "min-element" and "max-element" feature supported
* New Clixon Yang RPC: ping. To check if backup is running.
* Fixed support for multiple datanodes in a choice/case statement.
2019-05-07 11:47:30 +02:00
Olof hagsand
2fc37d2470 Yang "unique" feature supported according to RFC 7950 7.8.3 2019-05-03 14:03:10 +02:00