Commit graph

45 commits

Author SHA1 Message Date
Olof hagsand
d84c529ff1 [Code formatting: Change indentation style to space](https://github.com/clicon/clixon/issues/379)
* Applies to all c/h/y/l/sh files and .editorconfig
2022-10-27 14:21:17 +02:00
Olof hagsand
0c79298e76 Changed C-API for xml translation/print the internal cxobj tree data structure to other formats
New API is as follows:
  * `clixon_xml2file()` - Print internal tree as XML to file
  * `clixon_xml2cbuf()` - Print internal tree as XML to buffer
  * `clixon_json2file()` - Print internal tree as JSON to file
  * `clixon_json2cbuf()` - Print internal tree as JSON to buffer
  * `clixon_cli2file()` - Print internal tree as CLI format to file
  * `clixon_txt2file()` - Print internal tree as text format to file
2022-06-01 20:02:27 +02:00
Olof hagsand
43a57dad79 Changed C-API for xml translation/print to other formats.
* Added `skiptop` parameter, if set only apply to children of a node, skip top node
    * default is 0
* Functions are merged, ie removed and replaced with more generic functions
* `xml2json_cbuf()`: Added `skiptop` parameter: `xml2json_cbuf(..., int skiptop)`
* `xml2json()` and `xml2json_cb()` merged into `xml2json_file()` with `skiptop`
    * Replace `xml2json(...)` with `xml2json_file(..., stdout, 0)`
    * Replace `xml2json_cb(...)` with `xml2json_file(..., 0)`
* `clicon_xml2cbuf()`: Added `skiptop` parameter: `clicon_xml2cbuf(..., int skiptop)`
* `xml2cli()`: Added `skiptop` parameter: `xml2cli(..., int skiptop)`
2022-06-01 10:48:39 +02:00
Olof hagsand
614c927343 * Added nsc parameter to xml2xpath() and ensured the xpath uses prefixes.
* Old code: add `NULL` as second parameter
2022-05-09 13:48:07 +02:00
Olof hagsand
580396d32f - Misc documentation
- Test: move pid file from tmp to global storage to prevent multiple instances
2022-04-14 15:21:25 +02:00
Olof hagsand
03f667e2ea Fix part 2 for [Xpath API do not support filter data by wildcard](https://github.com/clicon/clixon/issues/313) 2022-03-15 19:59:54 +01:00
Olof hagsand
77b4468eb3 Updated copyright statements to 2022 2022-01-18 16:36:00 +01:00
Olof hagsand
5388aace12 Refactoring for better performance of large auto-cli specs
* Fixed: [very slow execution of load_set_file #288](https://github.com/clicon/clixon/issues/288)
* New `clixon-lib@2021-11-11.yang` revision
  * Modified option: RPC stats extended with YANG stats
* Modified `clixon-config@2021-11-11.yang` revision
  * Added option:
    * CLICON_PLUGIN_CALLBACK_CHECK
    * Enable to make plugin context check before and after all callbacks.
* Added statistics for YANG: number of objects and memory used
* Use the treeref no-copy option of CLIgen to reduce memory
* Refactored cli-generation/autocli-start code
* Refactored cligen glue functions to use cligen_eval directly (remove clicon_eval,clixon_cligen_eval)
2021-11-25 12:04:05 +01:00
Olof hagsand
448aa4c994 include signal.h 2021-10-21 16:29:43 +02:00
Olof hagsand
93d1149925 Dispatcher test additions 2021-10-14 08:58:35 +02:00
Olof hagsand
a046306270 - New netconf-specific uint32 parse functions
- Added failure handling to xpath traverse_canonical
- Started pagination cli code
2021-09-21 11:24:53 +02:00
Olof hagsand
fb9917e81e * Removed endtag parameter of clixon_xml_parse_file() 2021-03-15 13:24:27 +01:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
e1b94d94d2 * Ensure all fopen check for NULL (not -1)
* Improve test-perf cycles
2020-12-07 13:33:21 +01:00
Olof hagsand
c31b1c471b * Changed first parameter from int fd to FILE *f in the following functions:
* clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](https://github.com/clicon/clixon/issues/146)
2020-11-04 22:21:42 +01:00
Olof hagsand
9984cfa1c8 Fixed: [default state data returned with get-config](https://github.com/clicon/clixon/issues/140)
* Generalized default code for both config and state
2020-09-27 17:14:28 +02:00
Olof hagsand
21ac47915b More XPath function support
* `count`, `name`, `contains`, `not`, as defined in [xpath 1.0](https://www.w3.org/TR/xpath-10)
  * `deref`, `derived-from` and `derived-from-or-self` from RFC7950 Section 10.
    * in particular in augment/when statements
  * Improved error handling
    * Verification of XPath functions is done at startup when yang modules are loaded, not when XPaths are evaluated.
    * Separation of "not found" and "not implemented" XPath functions
    * Both give a fatal error (backend does not start).
2020-09-25 11:52:06 +02:00
Olof hagsand
57b104bc25 * Checked yang binding validity which lead to:
* Changed and enhanced several `bad-element` error replies to `unknown-element` with more detailed error-message.
2020-07-25 16:09:24 +02:00
Olof hagsand
ff5462ecac Replaced the global variable debug with access function: clicon_debug_get(). 2020-06-09 16:04:49 +02:00
Olof hagsand
82529a2f16 * Experimental optimzations of yang-bind and sort for large lists
* Enabled by compile-time options: `OPTIMIZE_45_BIND` and `OPTIMIZE_45_SORT`
2020-05-01 17:08:32 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
b726c5008e renamed xml_spec_populate*() to xml_bind_yang*() 2020-03-17 16:11:58 +01:00
Olof hagsand
c5e2039cac Removed prefix parameter to xml_new
Added xml_new_body() by mgsmith
2020-03-17 11:18:22 +01:00
Olof hagsand
7425a3b520 * C-API change: xml_new() changed from xml_new(name, xp, ys) to xml_new(name, prefix, xp, type)
* If you have used, `ys`, add `xml_spec_set(x, ys)` after the statement
  * If you have `xml_type_set(x, TYPE)` or `xml_prefix_set(x, PREFIX)` immediately after the statement, you can remove those and set them directly as: `xml_new(name, PREFIX, xp, TYPE)`
2020-03-12 22:29:28 +01:00
Olof hagsand
2d521d52c8 * Added clixon-stats in clixon-config.yang for clixon XML and memory statistics. 2020-03-01 10:59:24 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
5831c053fa clixon_util_path did not sort XML properly 2020-02-08 17:00:47 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
b340c36f79 Experimental optimized XPath, multiple keys 2019-12-31 13:29:25 +01:00
Olof hagsand
7ad16bd84b * C-API: Added xpath_first_localonly() as an xpath function that skips prefix and namespace checks.
* Added experimental code for optizing XPath search using binary search.
  * Enable with XPATH_LIST_OPTIMIZE
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
2019-12-20 18:21:46 +01:00
Olof hagsand
d68a829862 * Added namespace-context parameter nsc to xpath_first and xpath_vec, (xpath_vec_nsc and xpath_first_nsc` are removed). 2019-12-07 20:46:13 +01:00
Olof hagsand
039e08f86a [xml_parse_string() is slow for a long XML string #96](https://github.com/clicon/clixon/issues/96) 2019-11-14 20:57:20 +01:00
Olof hagsand
728fe9c6ac * Bumped version to 4.3.0.PRE
* Added wildcard `*` as a mode to `CLICON_MODE` in clispec files
* [Add missing includes](https://github.com/clicon/clixon/pulls)
2019-11-03 13:05:50 +01:00
Olof hagsand
03acf8e19c * XPATH canonical form implemented for NETCONF get and get-config. 2019-10-09 22:13:04 +02:00
Olof hagsand
ee329ee382 * RESTCONF "depth" query parameter supported
* C API change: Added `depth` parameter to function `clicon_xml2cbuf`, default is -1.
2019-08-17 10:54:13 +02:00
Olof hagsand
40d5b99d3b Reverted some of the nsc xpath API changes. In the revert, xpath_first() and xpath_vec() retain their old syntax with nsc==NULL.
The reason is to be conservative with the API. However, less used functions, such as xpath_vec_bool(), xpath_vec_ctx() and xpath_vec_flag()  are changed with a new `nsc`parameter, which should be set to NULL in most cases.
2019-07-09 12:19:03 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
1e1eabbc06 Non linear str2int 2019-04-23 12:53:29 +02:00
Olof hagsand
0103d58994 2019 2019-01-02 15:48:30 +01:00
Olof hagsand
d09a8c08aa getopt return value changed from char to int (https://github.com/clicon/clixon/issues/58) 2018-12-01 18:34:52 +01:00
Olof hagsand
9c57902b96 * XML parser conformance to W3 spec
* Names lexically correct (NCName)
  * Syntactically Correct handling of '<?' (processing instructions) and '<?xml' (XML declaration)
  * XML prolog syntax for 'well-formed' XML
  * <!DOCTYPE (ie DTD) is not supported.
2018-11-18 13:22:08 +01:00
Olof Hagsand
9245ef222a util warnings 2018-07-19 08:03:07 +00:00
Olof hagsand
c71791f168 Added util directory containing test applications 2018-07-19 09:57:38 +02:00
Renamed from lib/src/clixon_util_xpath.c (Browse further)