Commit graph

46 commits

Author SHA1 Message Date
Olof hagsand
a00e36caa2 * Full RFC 7950 if-feature-expr support (Section 7.20.2)
* Previous implementation did not handle nested if-feature expressions
  * As part of fixing: [YANG if-feature does not support nested boolean expression](https://github.com/clicon/clixon/issues/341)
  * Added new yacc/lex parser for if-feature-expr string
2022-06-26 10:58:43 +02:00
Olof hagsand
f1300d7a12 * [Adding feature to top level container doesn't work](https://github.com/clicon/clixon/issues/322)
* Instead of removing YANG which is disabled by `if-feature`, replace it with an yang `anydata` node.
  * This means XML specified by such YANG is ignored, and it is not an error to access it
  * Note the similarity with `CLICON_YANG_UNKNOWN_ANYDATA`
2022-04-06 14:04:11 +02:00
Olof hagsand
c038c9a27f * Implementation of "chunked framing" according to RFC6242 for Netconf 1.1.
* First hello is 1.0 EOM framing, then successing rpc is chunked framing
  * See
    * [Netconf framing](https://github.com/clicon/clixon/issues/50), and
    * [Clixon does not switch to chunked framing after NETCONF 1.1 is negotiated](https://github.com/clicon/clixon/issues/314)
* C:
  * Moved netconf framing code from netconf application to clixon lib
* Test:
  * New expecteof_netconf and adjusted other expect scripts to handle NETCONF 1.1 framing
2022-03-28 12:36:44 +02:00
Olof hagsand
dd2ac56af8 Module state upgrade: RFC7895 to RFC 8525
Changes to clixon-config:
  * Added option: 'CLICON_YANG_LIBRARY`
  * Changed default value: `CLICON_MODULE_LIBRARY_RFC7895` to false
Datastore modstate is backward compatible
Tests upgraded to RFC8525
2022-03-07 10:56:54 +01:00
Olof hagsand
4102ca8a46 * configure --with-wwwdir=<dir> is removed
* test:
  * new test_helloworld.sh smoketest added for testing basic functions
  * stripped restconf fcgi default config
  * added yang variable
2022-02-07 13:31:39 +01:00
Olof hagsand
339b744835 * Optional yangs for testing have been removed from the Clixon repo
* These were included for testing
  * If you want to run the Clixon test suite you need to point `YANGMODELS`, see test/README.md
  * The following configure options have been removed:
    * `configure --with-opt-yang-installdir=DIR`
    * `configure   --enable-optyangs`
  * You may need to specify standard YANGs using configure option `--with-yang-standard-dir=DIR`
* Updated yang ietf models with fetures for tet
* Added option `CLICON_YANG_AUGMENT_ACCEPT_BROKEN` to accept broken yangmodels.
  * This is a debug option for CI testcases where standard YANG models are broken
2021-11-29 10:28:34 +01:00
Olof hagsand
9edf380f6a - Link utils statically if LINKAGE=static
- Fix event exit bug (set to new counter, dont increment)
- Changed default CI to be restconf=native instead of fcgi1~
- Tests:
  - Change to same wait backend and restconf commands
  - Change default restconf protocol from http to https
2021-06-18 22:50:30 +02:00
Olof hagsand
aaaeec92eb Internal RESTCONF changes:
- Kill running process directly, not schedul a kill
- fcgi: when kill, make exit more ordered, ensure fcgi-accept returns properly
2021-04-30 14:49:01 +02:00
Olof hagsand
5692072d36 * Fixed: [CLIXON is not waiting for the hello message #184](https://github.com/clicon/clixon/issues/184)
* Hello message semantics has been made stricter according to RFC 6241 Sec 8.1, for example:
  * A client MUST send a <hello> element.
  * Each peer MUST send at least the base NETCONF capability, "urn:ietf:params:netconf:base:1.1" (or 1.0 for RFC 4741)
  * The netconf client will terminate (close the socket) if the client does not comply
  * You can set `CLICON_NETCONF_HELLO_OPTIONAL` to true to use the old behavior of essentially ignoring hellos.
* New clixon-config@2020-03-08.yang revision
  * Added: `CLICON_NETCONF_HELLO_OPTIONAL`
* The base capability has been changed to "urn:ietf:params:netconf:base:1.1" following RFC6241.
2021-03-10 14:18:16 +01:00
Olof hagsand
1f0147f996 removed unnecessary strerror() in clixon_err() calls; added 'function' keyword in all function declarations in the test scripts 2021-02-04 12:04:48 +01:00
Olof hagsand
41e23865a1 revise testing: change expectfn -> extectpart etc 2020-12-29 13:49:49 +01:00
Jonathan Ben-Avraham
a1f54d71ac Update yang/mandatory files for rfc8527 compliance
Updated the yang/mandatory files and test scripts for rfc8527.
2020-11-07 22:06:05 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
25f67d1eb9 Fixed: [(CLI) the description of a used grouping is shown instead of the encapsulating container #124](https://github.com/clicon/clixon/issues/124) 2020-08-19 15:55:55 +02:00
Olof hagsand
918bf6223b added CR delimiters to if-feature 2020-07-27 13:00:56 +02:00
Olof hagsand
f7b640810b * Enhanced Clixon if-feature handling:
* If-feature now supports and/or lists, such as: `if-feature "a and b"` and `if-feature "a or b or c"`. However, full if-feature-expr including `not` and nested boolean experessions is still not supported.
  * Sanity check: if an `if-feature` statement exists, a corresponding `feature` statement must exists that declares that feature.
2020-07-27 12:06:36 +02:00
Olof hagsand
57b104bc25 * Checked yang binding validity which lead to:
* Changed and enhanced several `bad-element` error replies to `unknown-element` with more detailed error-message.
2020-07-25 16:09:24 +02:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof Hagsand
77b491c568 Demon->Daemon, FreeBSD grep changes 2019-10-14 07:47:08 -10:00
Olof hagsand
fe1aeda4f2 test env bash; freebsd config 2019-09-17 22:44:01 +02:00
Olof hagsand
27fd99e7cd * Implemented backend daemon drop privileges after initialization to
run as non-privileged user
2019-09-14 18:34:32 +02:00
Olof hagsand
3d5abb77f9 * Backend daemon drops privileges after initialization (to not run as root)
* New config option `CLICON_USER` with default value `clicon`
  * Can also be set with `-U <user>` clixon_backend command-line option
2019-09-11 21:24:14 +02:00
Olof hagsand
1f8c759f3d Merge branch 'master' of https://github.com/clicon/clixon.
Added new API function `xpath_parse()` to split parsing and xml evaluation.
2019-07-08 14:47:18 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
a2340c265c test scripts: replaced sleeps with wait functions 2019-07-01 10:32:48 +02:00
Olof hagsand
bc54f2d04c * Regexp improvements
* Added check for libxml in configure';
  * Added clixon_util_regexp utility function
* Yang state get improvements
  * Integrated state and config into same tree on retrieval, not separate trees
  * Added cli functions `cli_show_config_state()` and `cli_show_auto_state()` for showing combined config and state info.
  * Added integrated state in the main example: `interface/oper-state`.
  * Added performance tests for getting state, see [test/test_perf_state.sh].
2019-05-20 16:03:29 +02:00
Olof hagsand
6bf2a74e24 * Restconf with startup feature will now copy all edit changes to startup db (as it should according to RFC 8040)
* See [Restconf does not handle startup datastore according to the RFC](https://github.com/clicon/clixon/issues/74)
* Netconf Startup feature is no longer hardcoded, you need to explicitly enable it (See RFC 6241, Section 8.7)
  * Enable in config file with: `<CLICON_FEATURE>ietf-netconf:startup</CLICON_FEATURE>`, or use `*:*`
2019-04-26 12:12:55 +02:00
Olof hagsand
cf7675dfac Experimental customized error output strings, see [lib/clixon/clixon_err_string.h] 2019-04-11 12:45:31 +02:00
Olof hagsand
e29cd7cfb9 * Optimized validation by making xml_diff work on raw cache tree (not copies)
* xmldb_get() removed unnecessary config option
2019-04-07 15:55:53 +02:00
Olof hagsand
be59bd48d8 upgrade example and test 2019-03-27 18:04:01 +01:00
Olof hagsand
98fb6eceef * Clixon configuration file top-level symbols has changed to clixon-configand namespace check is enforced.
* clixon-config YAML file has new revision: 2019-03-05.
  * New URN and changed top-level symbol to `clixon-config`
* Removed obsolete `_CLICON_XML_NS_STRICT` variable and `CLICON_XML_NS_STRICT` config option.
2019-03-06 09:42:55 +01:00
Olof hagsand
4c0b412e9f Added valgrind memory leak tests for backend. Fixed some minor leaks and file descriptor closes. 2019-02-19 13:16:59 +01:00
Olof Hagsand
d64d433468 * Changed base OS in clixon-system container to Alpine
* Moved datastore/text/ code to datastore/ since there is only one type of datastore.
* Added "magic script line" to test script for sourced and scriped exits
2019-02-16 17:45:39 +01:00
Olof hagsand
02d725b2c0 * xml_cmp() respects 'ordered-by user' for state nodes, which violates RFC 7950 [https://github.com/clicon/clixon/issues/63. (Thanks JDL) 2019-02-03 16:19:33 +01:00
Olof hagsand
0267afcb21 * Added -o "<option>=<value>" command-line option to all programs: backend, cli, netconf, restconf.
* Added -p <dir> command-line option to all programs: backend, cli, netconf, restconf.
* Moved and updated all standard ietf and iana yang files from example and yang/ to `yang/standard`.
* Renamed example yang from example.yang -> clixon-example.yang
2019-01-13 17:34:15 +01:00
Olof hagsand
0baebc93fd * Strict namespace setting can be a problem when upgrading existing database files, such as startup-db or persistent running-db, or any other saved XML file.
* For backward compatibility, load of startup and running set CLICON_XML_NS_STRICT to false temporarily.
* Added three-valued return values for several validate functions where -1 is fatal error, 0 is validation failed and 1 is validation OK.
  * This includes: `xmldb_put`, `xml_yang_validate_all`, `xml_yang_validate_add`, `xml_yang_validate_rpc`, `api_path2xml`, `api_path2xpath`
* Added new xml functions for specific types: `xml_child_nr_notype`, `xml_child_nr_notype`, `xml_child_i_type`, `xml_find_type`.
2019-01-02 15:18:29 +01:00
Olof hagsand
f872c7e295 * More precise Yang validation and better error messages
* For Example, adding bad-, missing-, or unknown-element error messages, etc instead of operation-failed
* Removed delete-config support for candidate db since it is not supported in RFC6241.
* Switched the order of `error-type` and `error-tag` in all netconf and restconf error messages to comply to RFC order.
* Added example_rpc RPC to example backend
* Renamed xml_namespace[_set]() to xml_prefix[_set]()
* Some restconf error messages contained "rpc-reply" or "rpc-error" which have now been removed.
* Netconf/Restconf RPC extra input arguments are ignored (https://github.com/clicon/clixon/issues/47)
2018-12-21 01:33:41 +01:00
Olof hagsand
ae1af8da9e * NACM extension (RFC8341)
* NACM module support (RFC8341 A1+A2)
   * Recovery user "_nacm_recovery" added.
     * Example use is restconf PUT when NACM edit-config is permitted, then automatic commit and discard are permitted using recovery user.
   * Example user changed adm1 to andy to comply with RFC8341 example

 * Yang code upgrade (RFC7950)
   * RPC method input parameters validated
     * see https://github.com/clicon/clixon/issues/4
* Correct XML namespace handling
   * XML multiple modules was based on "loose" semantics so that yang modules were found by iterating thorugh namespaces until a match was made. This did not adhere to proper [XML namespace handling](https://www.w3.org/TR/2009/REC-xml-names-20091208), and causes problems with overlapping names and false positives. Below see XML accepted (but wrong), and correct namespace declaration:
```
      <rpc><my-own-method></rpc> # Wrong but accepted
      <rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> # Correct
        <my-own-method xmlns="http://example.net/me/my-own/1.0">
      </rpc>
```
   * To keep old loose semantics set config option CLICON_XML_NS_ITERATE (true by default)
   * XML to JSON translator support for mapping xmlns attribute to module name prefix.
   * Default namespace is still "urn:ietf:params:xml:ns:netconf:base:1.0"
   * See https://github.com/clicon/clixon/issues/49
* Changed all make tags --> make TAGS
* Keyvalue datastore removed (it has been disabled since 3.3.3)
* debug rpc added in example application (should be in clixon-config).
2018-12-16 19:49:19 +01:00
Olof hagsand
39a5086218 * Yang Support of submodule, include and belongs-to.
* Improved unknown handling
* Configure option `CLICON_YANG_DIR` is changed from a single directory to a path of directories
    * Note CLIXON_DATADIR (=/usr/local/share/clixon) need to be in the list
2018-12-01 18:17:42 +01:00
Olof hagsand
a8f0aad411 * Yang parser is stricter (see cardinality below) which may break parsing of slack yang specs.
* YANG parser cardinality checked (only modules level yet)
  * See https://github.com/clicon/clixon/issues/48
2018-11-18 20:55:57 +01:00
Olof hagsand
e343ad6454 Mem leakage and test update for valgrind 2018-11-06 14:41:43 +01:00
Olof hagsand
b477e22d1e schema resource conformance-type 2018-10-27 16:16:54 +02:00
Olof hagsand
a4e29bcdb7 * All yang modules are stored in the clicon_dbspec_yang() option.
* Except clixon-config module due to bug reported below
* Bug: Top-level Yang symbol cannot be called "config" in any imported yang file.
  * datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanit
2018-10-19 15:26:22 +02:00
Olof hagsand
40cda7b6a4 * ietf-netconf yang module added with candidate, validate, startup and xpath features enabled.
* Added urn:ietf:params:netconf:capability:yang-library:1.0
* Thanks SCadilhac for helping out, see https://github.com/clicon/clixon/issues/39
* uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters
2018-10-17 22:36:59 +02:00
Olof hagsand
65c44b41e8 RFC 7895:
* Changed Netconf hello to single capabilty urn:ietf:params:netconf:capability:yang-library:1.0 according to YANG 1.1 RFC7950 Sec 5.6.4.
  * Set by option: CLICON_MODULE_LIBRARY_RFC7895 - enabled by default
  * Option CLICON_MODULE_SET_ID is set and changed when modules change.
2018-10-12 18:41:53 +02:00
Olof hagsand
52021c3856 features 2018-10-10 20:38:34 +02:00