Commit graph

236 commits

Author SHA1 Message Date
Olof Hagsand
f7d4df01a6 restconf test fixes for freebsd and others 2021-02-12 16:30:30 +00:00
Olof hagsand
c64878fd4a * RESTCONF configuration is unified and moved from clixon-config.yang to clixon-restconf.yang
* Except `CLICON_RESTCONF_DIR` which remains in clixon-config.yang due to bootstrapping
    * -d <dir> option removed
  * This applies to both evhtp and fcgi RESTCONF
    * Both can also read config from backend, and be started from backend
  * You may need to move config as follows (from clixon-config.yang to clixon-restconf.yang)
    * CLICON_RESTCONF_PRETTY -> restconf/pretty
    * CLICON_RESTCONF_PATH -> restconf/fcgi-path
2021-02-11 13:46:34 +01:00
Olof hagsand
710fc76887 * Restconf authentication callback (ca_auth) signature changed
* Not backward compatible: All uses of the ca-auth callback in restconf plugins must be changed
  * New version is: `int ca_auth(h, req, auth_type, authp, userp)`
    * where `auth_type` is the requested authentication-type (none, client-cert or user-defined)
    * `authp` is the returned authentication flag
    * `userp` is the returned associated authenticated user
    * and the return value is three-valued: -1: Error, 0: ignored, 1: OK
  * For more info see [clixon-docs](https://clixon-docs.readthedocs.io/en/latest/restconf.html)
* New clixon-restconf@2020-12-30.yang revision
2021-02-09 21:26:35 +01:00
Olof hagsand
1f0147f996 removed unnecessary strerror() in clixon_err() calls; added 'function' keyword in all function declarations in the test scripts 2021-02-04 12:04:48 +01:00
Olof hagsand
b41f68b677 debug print of backend state, split test-stream into a netconf and restconf part, change restconf yang auth-type from leaf-list to leaf 2021-01-27 14:40:34 +01:00
Olof hagsand
41e23865a1 revise testing: change expectfn -> extectpart etc 2020-12-29 13:49:49 +01:00
Olof hagsand
8540820698 * New clixon-lib@2020-12-08.yang revision
* Added: autocli-op extension (see new features)
  * Added: rpc process-control for process/daemon management
* Added enable flag and removed presence in clixon-restconf
2020-12-13 22:19:25 +01:00
Olof Hagsand
99c570e278 test/vagrant fixes 2020-12-03 14:23:59 +00:00
Olof hagsand
c32950c8a9 * Added message-id attributes in error and hello replies
* See [namespace prefix nc is not supported in full #154](https://github.com/clicon/clixon/issues/154)
* Removed mandatory loading of clixon_restconf.yang
2020-12-01 12:14:30 +01:00
Olof hagsand
66e1d81609 travis again 2020-11-30 13:00:13 +01:00
Olof hagsand
cb10d4b552 travis test wait 2020-11-30 12:06:46 +01:00
Olof hagsand
cdc851e5ce travis again 2020-11-30 11:16:53 +01:00
Olof hagsand
993c5efe87 try native travis test #5 2020-11-29 23:03:13 +01:00
Olof hagsand
f69daf773e more test and vagrant for nginx checks 2020-11-25 19:48:42 +01:00
Olof hagsand
4f5b0178cc fix test sleep/loop problem 2020-11-24 22:29:48 +01:00
Olof hagsand
ccadd3f9ea test: sleep 0.5s; vagrant centos: ipv6 enable 2020-11-24 18:29:04 +01:00
Olof hagsand
8d8a37d18e test and vagrant fixes 2020-11-24 10:38:58 +01:00
Olof hagsand
0b948248e4 * Reworked evhtp restconf config to only use clixon-restconf.yang and marked local restconf options as obsolete
* Extended clicon-config with clixon-restconf for local config
* Removed obsolete CLICON_TRANSACTION_MOD
2020-11-21 13:16:29 +01:00
Olof Hagsand
eb549b4c21
Merge pull request #150 from benavrhm/topic_benavrhm_fixups_20201112
Topic benavrhm fixups 20201112
2020-11-17 15:45:08 +01:00
Olof hagsand
4eee00101c Disabling IPv6 per default due to failing docker tests 2020-11-15 18:16:06 +01:00
Olof hagsand
a5d56f1cb7 disable IPv6 restconf test by default 2020-11-15 17:00:35 +01:00
Olof hagsand
6eb18da5e9 * Multi-socket feature (eg IPv4/IPv6 http/https) to restconf evhtp
* Added by-ref parameter to `ys_cv_validate()` returning which sub-yang spec was validated in a union.
2020-11-15 12:34:40 +01:00
Jonathan Ben-Avraham
b1c742ff41 Test script documentation fixup
This commit provides a scriptable headline for the octal dump output.
When the received output of a test is empty or only whitespace, this
commit makes it clear that the octal dump putput is not the expected
output.
2020-11-12 19:02:40 +02:00
Olof hagsand
a0006075c3 Merge branch 'topic_benavrhm_rfc8527a_20201103' of https://github.com/benavrhm/clixon into benavrhm-topic_benavrhm_rfc8527a_20201103 2020-11-11 13:41:05 +01:00
Olof hagsand
4ecffa6885 test systemctl; better rpc namespace errormsg 2020-11-11 13:09:58 +01:00
Jonathan Ben-Avraham
e6899cc3f5 Additional documentation for test scripts
Added documentation in comments in test scripts, added example site.sh,
moved evaluation of site.sh later in lib.sh to allow site.sh to override
more lib.sh variables.
2020-11-07 21:44:59 +02:00
Olof hagsand
5088f23e6a disable nginx systemd check 2020-11-06 20:11:01 +01:00
Olof hagsand
6ecb0d5232 tests for restconf backend configure
rm client-cert-ca
2020-11-06 11:41:59 +01:00
Olof hagsand
a77cd27abb * Fixed: [namespace prefix nc is not supported](https://github.com/clicon/clixon/issues/143)
* Added more sanity checks on incoming top-level rpc and hello messages, including verifying top-level namespace
2020-10-17 18:04:46 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
41ac192362 Fixed: [Cannot create or modify NACM data node access rule with path using JSON encoding #129](https://github.com/clicon/clixon/issues/129). The evaluation of NACM datanode rule path is assumed to be canonical namespace and cannot be overruled with xmlns rules. 2020-08-30 17:03:49 +02:00
Olof hagsand
25f67d1eb9 Fixed: [(CLI) the description of a used grouping is shown instead of the encapsulating container #124](https://github.com/clicon/clixon/issues/124) 2020-08-19 15:55:55 +02:00
Olof hagsand
cbfc34fa6d wwwuser/wwwdir config doc 2020-08-17 14:50:31 +02:00
Olof hagsand
3d33335f4b Last minute documentation and testing changes befoire 4.6 2020-08-13 14:58:42 +02:00
Olof hagsand
57b104bc25 * Checked yang binding validity which lead to:
* Changed and enhanced several `bad-element` error replies to `unknown-element` with more detailed error-message.
2020-07-25 16:09:24 +02:00
Olof hagsand
5dc2a6387d unified curl test options 2020-07-03 14:13:15 +02:00
Olof hagsand
c049a397b0 Fixed ssl client certs for evhtp.
* Added SSL cert info as options: CLICON_SSL_SERVER_CERT, CLICON_SSL_SERVER_KEY, CLICON_SSL_CA_CERT
Added config.sh for testing for autotools
2020-07-01 15:11:22 +02:00
Olof hagsand
9c82e97072 Added http support for evhtp (not only https) 2020-06-29 14:41:08 +02:00
Olof hagsand
3af184c655 Modify restconf tests to run for both evhtp and fcgi: , curl -sik, etc 2020-06-21 15:49:46 +02:00
Olof hagsand
6e714beea5 * Restconf FCGI (eg via nginx) have changed reply message syntax slightly
* native http: new restconf_err files, generic data input, restconf_methods generalized.
* test: expecteq removed.
2020-06-16 21:51:00 +02:00
Olof hagsand
e898dda016 * Auto-CLI enhancements
* A generated clispec including state (default @datanodestate) also generated along with the config clispec tree (default @datanode)
  * New mode `GT_HIDE` set by option `CLICON_CLI_GENMODEL_TYPE` to collapse non-presence containers that only contain a single list
  * Added a prfix for cli_show_config/cli_show_auto so that it can produce parseable output
  * Thanks dcornejo@netgate.com for trying it out and suggestions
2020-06-13 12:05:26 +02:00
Olof hagsand
e2d9c046af further restconf refactoring 2020-06-11 21:54:09 +02:00
Olof hagsand
14b03f63a3 all clixon event functions prepended with clixon_
Replaced fcgi getparam code with clixon-data parameters
2020-06-05 16:19:03 +02:00
Olof hagsand
fdf335fb2f new function ; Added valgrind mem check in leak tests; pattern CDATA tests 2020-05-29 12:39:07 +02:00
Olof hagsand
bf04131e54 Added decriptive error message when plugins produce invalid state XML. 2020-05-10 17:10:07 +02:00
Olof hagsand
dafc6d10e0 Added option to treat unknown XML (wrt YANG) as anydata. 2020-05-03 22:03:33 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00