Commit graph

238 commits

Author SHA1 Message Date
Olof hagsand
26d37352f4 NETCONF hello error handling, changed YANG PATCH enabling, refactored static linking
NETCONF hello errors, such as wrong session-id, prefix, namespace terminates session
  * Instead of returning an rpc-error reply
Changed YANG PATCH enabling:
  * Now: `./configure --enable-yang-patch`
  * Before: set YANG_PATCH constant in `include/clixon_custom.h`
Refactored Makefile for static linking
2021-10-05 12:04:53 +02:00
Olof hagsand
8f55b8490d Refactored Makefile for static linking, removed netconf soap encoding 2021-10-04 20:06:11 +02:00
Olof hagsand
42e61f461c - Memleaks in xpath parse error
- Static linkage cli fix (worked in 5.3.0 but restructuring broke it)
- Renamed all cligen_ph_active_get to cligen_pt_active_get
- Added AFL deferred init
- test_install.sh for static .a suffix
2021-10-03 18:49:25 +02:00
Olof hagsand
d175720b7e Cleaned up Makefiles, especially LINKAGE=static vs dynamic
Added LIBSTATIC_SUFFIX instead of hardcoded .a
Changed building of build.c not to be triggered by install
2021-09-30 21:40:46 +02:00
Simon Bauer
bdda35d95b
Fixed SH_SUFFIX 2021-09-30 10:37:19 +02:00
Olof hagsand
51278d5901 Fixed static linking for coverage and fuzzing
Preparations for Clixon 5.3 release
2021-09-27 12:26:53 +02:00
Olof hagsand
5a875e3152 Fix some LINKAGE=static compile issues:
- Makefile circularity by fixing .so suffix
- simplify example/main Makefile
2021-09-26 11:22:48 +02:00
Olof hagsand
c00162aec1 - Modified linkage constant in makefile to test for dynamic only
- CLIXON_STATIC_PLUGIN to support statically linked plugins
- Added -H option to clixon_netconf: Do not require hello before request
2021-09-25 16:36:37 +02:00
Olof hagsand
0c7f2043f3 * Pagination according to new draft
* count/skip -> limit/offset
* ietf-yang-metadata RFC 7952 support, placeholder parsing and extension
2021-09-21 11:22:41 +02:00
Phil Heller
38f4cd8237 Testing dependency fixes, Netconf XML declaration and filter logic fix
Install libnghttp2-devel in ubuntu and centos per required dependencies
Ignore case when checking XML declaration encoding value per W3C recommendations
Fix filter logic to follow RFC6241 (7.1, 7.7) and default to subtree
2021-07-29 22:48:07 -06:00
Olof hagsand
448b56ca0c - Removed sha1 code
- ifdef unused SOAP encoding
2021-04-24 22:06:26 +02:00
Olof hagsand
b88722fa25 - Restructure plugin module using a "module struct" rather than global variables.
This include plugin module init and exit functions
  New type: clixon_plugin_t exposed via public API while struct clixon_plugin is private
- Makefile changes for static linkage using "LINKAGE=static"
- Moved nacm external init to later stage in backend_main
2021-04-19 11:06:40 +02:00
Olof hagsand
7412bb7b3d Fixed [changing interface name not support with openconfig module #195](https://github.com/clicon/clixon/issues/195) 2021-04-06 22:36:46 +02:00
Olof hagsand
2def5d2367 * Fixed Yang parsing of comments in (extension) unknown statements, to allow multiple white space
* Fixed cli_start_shell: comments and cli_error
* Fixed [making cli_show_options's output more human readable #199](https://github.com/clicon/clixon/issues/199)
* Fixed SSL/evhtp limited read buffer problem
2021-04-01 15:56:56 +02:00
Olof hagsand
477059f33d Fixed again: [backend start resconf failed due to path string truncated #192](https://github.com/clicon/clixon/issues/192) 2021-03-24 12:23:00 +01:00
Olof hagsand
43db8dd5c5 Fixed: [state showing error in cli with CLICON_STREAM_DISCOVERY_RFC8040 #191](https://github.com/clicon/clixon/issues/191) 2021-03-18 10:41:08 +01:00
Olof hagsand
1f5e71512a mem leaks in netconf hello 2021-03-13 17:50:43 +01:00
Olof hagsand
7762b10cbb * Changed signal handling
* Moved clixon-proc sigchild handling	from handler to clixon_events
2021-03-11 18:12:43 +01:00
Olof hagsand
5692072d36 * Fixed: [CLIXON is not waiting for the hello message #184](https://github.com/clicon/clixon/issues/184)
* Hello message semantics has been made stricter according to RFC 6241 Sec 8.1, for example:
  * A client MUST send a <hello> element.
  * Each peer MUST send at least the base NETCONF capability, "urn:ietf:params:netconf:base:1.1" (or 1.0 for RFC 4741)
  * The netconf client will terminate (close the socket) if the client does not comply
  * You can set `CLICON_NETCONF_HELLO_OPTIONAL` to true to use the old behavior of essentially ignoring hellos.
* New clixon-config@2020-03-08.yang revision
  * Added: `CLICON_NETCONF_HELLO_OPTIONAL`
* The base capability has been changed to "urn:ietf:params:netconf:base:1.1" following RFC6241.
2021-03-10 14:18:16 +01:00
Olof hagsand
699115736d NETCONF README: https://github.com/clicon/clixon/issues/185 2021-03-07 15:29:33 +01:00
Olof hagsand
72dd31395e fixed tests after updates 2021-02-10 20:48:57 +01:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
8cde7a4ded * Handling empty netconf XML messages "]]>]]>" is changed from being accepted to return an error.
* Netconf split lines input (input fragments) fixed
  * Netconf input split on several lines, eg using stdin: "<a>\nfoo</a>]]>]]>" could under some circumstances be split so that only "</a>]]>]]>" be properly processed. This could also happen to a socket receiving a sub-string and then after a delay receive the rest.
  * Fixed by storing residue and add that to the input string if later input is received on the same socket.
* Use [https://github.com/clicon/libevhtp](https://github.com/clicon/libevhtp) instead of [https://github.com/criticalstack/libevhtp](https://github.com/criticalstack/libevhtp) as a source of the evhtp source
2021-01-07 16:57:47 +01:00
Simon Bauer
638658c63d
Update netconf_rpc.c 2020-12-13 18:07:28 +01:00
Simon Bauer
be329dac6d
Fixed filter namespace in netconf_rpc.o
The "filter" node was not detected when an a namespace was used. The fix is similar to d045e8abce and related to bug #154 #143
2020-12-13 18:05:49 +01:00
Olof hagsand
19fb063f82 Support for building static lib: 2020-11-29 17:00:35 +01:00
Olof hagsand
a77cd27abb * Fixed: [namespace prefix nc is not supported](https://github.com/clicon/clixon/issues/143)
* Added more sanity checks on incoming top-level rpc and hello messages, including verifying top-level namespace
2020-10-17 18:04:46 +02:00
Olof hagsand
6f2c4a076d * Configuration directory
* A new configuration option `CLICON_CONFIGDIR` has been added for loading of extra config files
  * If not given, only the main configfile is loaded.
  * If given, and if the directory exists, the files in this directory will be loaded alphabetically AFTER the main config file in the following way:
    * leaf values are overwritten
    * leaf-list values are appended
  * You can override file setting with `-E <dir>` command-line option.
* New clixon-config@2020-10-01.yang revision
  * Added option for configuration directory: `CLICON_CONFIGDIR`
2020-10-01 11:04:25 +02:00
Olof hagsand
5b5dce7e39 Fixed: [clixon_netconf does not respond to hello #136](https://github.com/clicon/clixon/issues/136)
* The error showed only when CLICON_MODULE_LIBRARY_RFC7895 was disabled.
2020-09-08 15:59:12 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
57b104bc25 * Checked yang binding validity which lead to:
* Changed and enhanced several `bad-element` error replies to `unknown-element` with more detailed error-message.
2020-07-25 16:09:24 +02:00
Olof hagsand
ff5462ecac Replaced the global variable debug with access function: clicon_debug_get(). 2020-06-09 16:04:49 +02:00
Olof hagsand
14b03f63a3 all clixon event functions prepended with clixon_
Replaced fcgi getparam code with clixon-data parameters
2020-06-05 16:19:03 +02:00
Olof hagsand
ac1f36009f Makefile copyright 2020-05-12 13:59:45 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
ba59e22fd7 NACM RFC341 datanode paths, read operation 2020-04-14 11:38:29 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
8131bd1834 2020 copyright 2020-03-27 15:42:03 +01:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
b726c5008e renamed xml_spec_populate*() to xml_bind_yang*() 2020-03-17 16:11:58 +01:00
Olof hagsand
c5e2039cac Removed prefix parameter to xml_new
Added xml_new_body() by mgsmith
2020-03-17 11:18:22 +01:00
Olof hagsand
7425a3b520 * C-API change: xml_new() changed from xml_new(name, xp, ys) to xml_new(name, prefix, xp, type)
* If you have used, `ys`, add `xml_spec_set(x, ys)` after the statement
  * If you have `xml_type_set(x, TYPE)` or `xml_prefix_set(x, PREFIX)` immediately after the statement, you can remove those and set them directly as: `xml_new(name, PREFIX, xp, TYPE)`
2020-03-12 22:29:28 +01:00
Olof hagsand
0f54899ae4 * Added option: CLICON_CLI_BUF_START and CLICON_CLI_BUF_THRESHOLD so you can change the start and
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
  * Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
  * Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
  * Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
2020-03-04 11:27:43 +01:00
Olof hagsand
a71c256898 * New clixon-config@2020-02-22.yang revision
* C-code changes:
  - Replaced stream uri:s w constants
  - Replaced large debug print code with single clicon_log_xml
  - Restconf put and post handling refactored using new parse API
2020-02-24 15:25:06 +01:00
Olof hagsand
2eae2b6000 extension search index 2020-02-20 15:08:25 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
7ad16bd84b * C-API: Added xpath_first_localonly() as an xpath function that skips prefix and namespace checks.
* Added experimental code for optizing XPath search using binary search.
  * Enable with XPATH_LIST_OPTIMIZE
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
2019-12-20 18:21:46 +01:00