Commit graph

52 commits

Author SHA1 Message Date
Olof hagsand
9edf380f6a - Link utils statically if LINKAGE=static
- Fix event exit bug (set to new counter, dont increment)
- Changed default CI to be restconf=native instead of fcgi1~
- Tests:
  - Change to same wait backend and restconf commands
  - Change default restconf protocol from http to https
2021-06-18 22:50:30 +02:00
Olof hagsand
84f5762ab5 * Most tests work with HTTP/2 support using nghttp2
* except non-ssl http/1->/2 upgrade
* Restconf: ensure HEAD method works everywhere GET does.
2021-06-08 20:53:43 +02:00
Olof hagsand
aaaeec92eb Internal RESTCONF changes:
- Kill running process directly, not schedul a kill
- fcgi: when kill, make exit more ordered, ensure fcgi-accept returns properly
2021-04-30 14:49:01 +02:00
Olof hagsand
15d01c58d8 - Better restconf debug: when restconf debug flag set in datastore, ensure the process is started with -D set
- Fixed native http support for base container
- Changed test certs and restconf scripts to functions
2021-04-09 09:34:05 +02:00
Olof hagsand
5692072d36 * Fixed: [CLIXON is not waiting for the hello message #184](https://github.com/clicon/clixon/issues/184)
* Hello message semantics has been made stricter according to RFC 6241 Sec 8.1, for example:
  * A client MUST send a <hello> element.
  * Each peer MUST send at least the base NETCONF capability, "urn:ietf:params:netconf:base:1.1" (or 1.0 for RFC 4741)
  * The netconf client will terminate (close the socket) if the client does not comply
  * You can set `CLICON_NETCONF_HELLO_OPTIONAL` to true to use the old behavior of essentially ignoring hellos.
* New clixon-config@2020-03-08.yang revision
  * Added: `CLICON_NETCONF_HELLO_OPTIONAL`
* The base capability has been changed to "urn:ietf:params:netconf:base:1.1" following RFC6241.
2021-03-10 14:18:16 +01:00
Olof hagsand
b7991d9b39 * Made a separate Clixon datastore XML/JSON top-level symbol
* Replaces the hardcoded "config" keyword.
  * Implemented by a compile-time option called `DATASTORE_TOP_SYMBOL` option in clixon_custom.h
* Tests: added endtest to all tests. Removed all premature exits if BE=0
2021-03-05 14:15:15 +01:00
Olof hagsand
c64878fd4a * RESTCONF configuration is unified and moved from clixon-config.yang to clixon-restconf.yang
* Except `CLICON_RESTCONF_DIR` which remains in clixon-config.yang due to bootstrapping
    * -d <dir> option removed
  * This applies to both evhtp and fcgi RESTCONF
    * Both can also read config from backend, and be started from backend
  * You may need to move config as follows (from clixon-config.yang to clixon-restconf.yang)
    * CLICON_RESTCONF_PRETTY -> restconf/pretty
    * CLICON_RESTCONF_PATH -> restconf/fcgi-path
2021-02-11 13:46:34 +01:00
Olof hagsand
710fc76887 * Restconf authentication callback (ca_auth) signature changed
* Not backward compatible: All uses of the ca-auth callback in restconf plugins must be changed
  * New version is: `int ca_auth(h, req, auth_type, authp, userp)`
    * where `auth_type` is the requested authentication-type (none, client-cert or user-defined)
    * `authp` is the returned authentication flag
    * `userp` is the returned associated authenticated user
    * and the return value is three-valued: -1: Error, 0: ignored, 1: OK
  * For more info see [clixon-docs](https://clixon-docs.readthedocs.io/en/latest/restconf.html)
* New clixon-restconf@2020-12-30.yang revision
2021-02-09 21:26:35 +01:00
Olof hagsand
0b948248e4 * Reworked evhtp restconf config to only use clixon-restconf.yang and marked local restconf options as obsolete
* Extended clicon-config with clixon-restconf for local config
* Removed obsolete CLICON_TRANSACTION_MOD
2020-11-21 13:16:29 +01:00
Olof hagsand
9016dde319 Modify tests so they all work with evhtp and backend config, eg that CLICON_RESTCONF_CONFIG is default true.
Note that some tests are forced to CLICON_RESTCONF_CONFIG is false due to test complexities.
2020-11-18 13:57:25 +01:00
Olof hagsand
2c96cf71ea Changed main example: removed translate yang and replaced testcases w table/parameter 2020-09-08 11:39:55 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
794d51a365 * Top-level default leafs assigned.
* Enforcing RFC 7950 Sec 7.6.1 means unassigned top-level leafs (or leafs under non-presence containers) are assigned default values.
* NACM default behaviour is read-only (empty configs are dead-lockedd)
  * This applies if NACM is loaded and `CLICON_NACM_MODE` is `internal`
* Fixed: [default values don't show up in datastores #111](https://github.com/clicon/clixon/issues/111)
2020-08-06 15:20:57 +02:00
Olof hagsand
5dc2a6387d unified curl test options 2020-07-03 14:13:15 +02:00
Olof hagsand
3af184c655 Modify restconf tests to run for both evhtp and fcgi: , curl -sik, etc 2020-06-21 15:49:46 +02:00
Olof hagsand
ba59e22fd7 NACM RFC341 datanode paths, read operation 2020-04-14 11:38:29 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
662d6b0a3f Clixon 4.4 three compile-time options: MOVE_TRANS_END for changing transaction-end callbacks; STATE_ORDERED_BY_SYSTEM to make state data system ordered; XML_MERGE_TWO_ROUNDS to improve xml_merge performance. 2020-04-02 14:28:58 +02:00
Olof hagsand
adbb683329 C-API:
* All uses of `api_path2xpath_cvv()` should be replaced by `api_path2xpath()`
  * `api_path2xpath()` added an `xerr` argument.
2020-03-10 22:37:16 +01:00
Olof hagsand
0f54899ae4 * Added option: CLICON_CLI_BUF_START and CLICON_CLI_BUF_THRESHOLD so you can change the start and
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
  * Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
  * Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
  * Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
2020-03-04 11:27:43 +01:00
Olof hagsand
94c7be42b6 rpc-error error tests 2020-02-27 21:16:12 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
05ce6f38a3 test kill valgrind process after run 2019-12-09 21:12:39 +01:00
Olof hagsand
6bf8e49f3f CHANGELOG for 4.2.0 2019-10-27 19:47:59 +01:00
Olof Hagsand
82356b7450 test scripts: separated wait of backend and restconf starts 2019-10-27 15:54:16 +00:00
Olof hagsand
279614d64f * The backend socket has now support of credentials of peer clients
* Added: CLICON_NACM_CREDENTIALS and CLICON_NACM_RECOVERY_USER
2019-10-18 19:33:23 +02:00
Olof Hagsand
77b491c568 Demon->Daemon, FreeBSD grep changes 2019-10-14 07:47:08 -10:00
Olof Hagsand
734f65e25a freebsd test script modifications 2019-10-13 07:08:53 -10:00
Olof hagsand
fe1aeda4f2 test env bash; freebsd config 2019-09-17 22:44:01 +02:00
Olof hagsand
27fd99e7cd * Implemented backend daemon drop privileges after initialization to
run as non-privileged user
2019-09-14 18:34:32 +02:00
Olof hagsand
3d5abb77f9 * Backend daemon drops privileges after initialization (to not run as root)
* New config option `CLICON_USER` with default value `clicon`
  * Can also be set with `-U <user>` clixon_backend command-line option
2019-09-11 21:24:14 +02:00
Olof hagsand
aa14f8ac2c RESTCONF PUT/POST -d {} media is enforced 2019-08-06 15:47:46 +02:00
Olof hagsand
e7b60619da * Pushed tag to 4.0.1.PRE
* Restconf RFC 8040 increased feature compliance
  * Cache-Control: no-cache added in HTTP responses (RFC Section 5.5)
  * Restconf monitoring capabilities (RFC Section 9.1)
* Added support for Yang extensions
  * New plugin callback: ca_extension
  * Main backend example includes example code on how to implement a Yang extension in a plugin.
* JSON changes
  * Non-pretty-print output removed all extra spaces.
    * Example: `{"nacm-example:x": 42}` --> {"nacm-example:x":42}`
  * Empty JSON container changed from `null` to `{}`.
    * Empty list and leafs remain as `null`
* Removed unnecessary configure dependencies
  * libnsl, libcrypt, libm, if_vlan,...
* pseudo-plugin added, to enable callbacks also for main programs. Useful for extensions
* Yang Unique statements with multiple schema identifiers did not work on some platforms due to memory error.
2019-07-23 22:11:14 +02:00
Olof hagsand
1f8c759f3d Merge branch 'master' of https://github.com/clicon/clixon.
Added new API function `xpath_parse()` to split parsing and xml evaluation.
2019-07-08 14:47:18 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
41f7e44dec Return 404 Not found error if restconf GET does not return requested instance 2019-07-04 20:21:07 +02:00
Olof hagsand
a2340c265c test scripts: replaced sleeps with wait functions 2019-07-01 10:32:48 +02:00
Olof hagsand
de15b2bf80 * RESTCONF strict namespace validation of data in POST and PUT.
* Accepted:
  ```
    curl -X PUT http://localhost/restconf/data/mod:a -d {"mod:a":"x"}
  ```
  * Not accepted (must prefix "a" with module):
  ```
    curl -X PUT http://localhost/restconf/data/mod:a -d {"a":"x"}
  ```
  * Undefine `RESTCONF_NS_DATA_CHECK` in include/clixon_custom.h to disable strict check.
2019-06-10 16:15:02 +02:00
Olof hagsand
e29cd7cfb9 * Optimized validation by making xml_diff work on raw cache tree (not copies)
* xmldb_get() removed unnecessary config option
2019-04-07 15:55:53 +02:00
Olof hagsand
606245ef02 * Added flags to example backend to control its behaviour:
* Start with `-- -r` to run the reset plugin
  * Start with `-- -s` to run the state callback
* Rewrote yang dir load algorithm to follow the algorithm in [FAQ](FAQ(doc/FAQ.md#how-are-yang-files-found) with more precise timestamp checks, etc.
2019-03-18 16:31:34 +01:00
Olof hagsand
98fb6eceef * Clixon configuration file top-level symbols has changed to clixon-configand namespace check is enforced.
* clixon-config YAML file has new revision: 2019-03-05.
  * New URN and changed top-level symbol to `clixon-config`
* Removed obsolete `_CLICON_XML_NS_STRICT` variable and `CLICON_XML_NS_STRICT` config option.
2019-03-06 09:42:55 +01:00
Olof hagsand
83edd29678 Revert "Revert "* Added valgrind memory leak tests in testmem.sh for restconf""
This reverts commit a73d2bd242.
2019-02-20 16:25:42 +01:00
Olof hagsand
a73d2bd242 Revert "* Added valgrind memory leak tests in testmem.sh for restconf"
String error
This reverts commit 6e63f0a77b.
2019-02-20 16:19:07 +01:00
Olof Hagsand
6e63f0a77b * Added valgrind memory leak tests in testmem.sh for restconf
* Added clicon_socket_set() and clicon_socket_get() functions for cleaning up backend server and restconf FCGI socket on termination.
* Removed obsolete `-x` command-line option to clixon_cli.
2019-02-20 13:06:18 +01:00
Olof hagsand
4c0b412e9f Added valgrind memory leak tests for backend. Fixed some minor leaks and file descriptor closes. 2019-02-19 13:16:59 +01:00
Olof Hagsand
d64d433468 * Changed base OS in clixon-system container to Alpine
* Moved datastore/text/ code to datastore/ since there is only one type of datastore.
* Added "magic script line" to test script for sourced and scriped exits
2019-02-16 17:45:39 +01:00
Olof Hagsand
860c0c042d Added RCLOG for redirecting restconf logs at tests; with-restconf error in example 2019-02-12 13:19:52 +01:00
Olof Hagsand
2b596591f1 Added Clixon example full system docker container, see [docker/system]. 2019-02-07 22:23:58 +01:00
Olof hagsand
02d725b2c0 * xml_cmp() respects 'ordered-by user' for state nodes, which violates RFC 7950 [https://github.com/clicon/clixon/issues/63. (Thanks JDL) 2019-02-03 16:19:33 +01:00
Olof hagsand
936ae389e8 Test script change: merged new2+new functions 2019-02-02 12:20:47 +01:00