Commit graph

324 commits

Author SHA1 Message Date
Olof hagsand
dafc6d10e0 Added option to treat unknown XML (wrt YANG) as anydata. 2020-05-03 22:03:33 +02:00
Olof hagsand
861f78f588 fix state path kludge 2020-05-02 17:08:54 +02:00
Olof hagsand
703d1a2b08 Added CLICON_STATE_PATH_KLUDGE 2020-05-01 21:39:18 +02:00
Olof hagsand
82529a2f16 * Experimental optimzations of yang-bind and sort for large lists
* Enabled by compile-time options: `OPTIMIZE_45_BIND` and `OPTIMIZE_45_SORT`
2020-05-01 17:08:32 +02:00
Olof hagsand
b8ec6a4957 * Optimized prefix checks at xml parse time: using many prefixes slowed down parsing considerably
* Cleared startup-db cache after restart
2020-04-29 23:11:48 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
1c99bd6a9b * xml-stats moved from clixon-config.yang as state data to an rpc datastatsin clixon-lib.yang
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
2020-04-23 22:58:57 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
476f1d84d3 Fix a bind rpc assert on unknown namespace 2020-04-23 11:06:25 +02:00
Olof hagsand
04017c97ba Length of xml vector in many structs changed from to since it is a vector size, not byte size. This includes transaction_data_t 2020-04-21 10:17:09 +02:00
Olof hagsand
ba59e22fd7 NACM RFC341 datanode paths, read operation 2020-04-14 11:38:29 +02:00
Olof hagsand
ac5637a46a Stricter incoming RPC sanity checking, error messages may have changed 2020-04-07 20:54:24 +02:00
Olof hagsand
d2531421f2 remove move end transaction 2020-04-03 08:47:48 +02:00
Olof hagsand
662d6b0a3f Clixon 4.4 three compile-time options: MOVE_TRANS_END for changing transaction-end callbacks; STATE_ORDERED_BY_SYSTEM to make state data system ordered; XML_MERGE_TWO_ROUNDS to improve xml_merge performance. 2020-04-02 14:28:58 +02:00
Olof hagsand
8131bd1834 2020 copyright 2020-03-27 15:42:03 +01:00
Olof hagsand
916c1aabd7 NULL->/ for state callbacks. Clarified socket close error 2020-03-26 20:42:42 +01:00
Olof hagsand
1079fcad47 Added / to api-path to xpath conversion 2020-03-25 22:11:07 +01:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
b726c5008e renamed xml_spec_populate*() to xml_bind_yang*() 2020-03-17 16:11:58 +01:00
Olof hagsand
c5e2039cac Removed prefix parameter to xml_new
Added xml_new_body() by mgsmith
2020-03-17 11:18:22 +01:00
Olof hagsand
7425a3b520 * C-API change: xml_new() changed from xml_new(name, xp, ys) to xml_new(name, prefix, xp, type)
* If you have used, `ys`, add `xml_spec_set(x, ys)` after the statement
  * If you have `xml_type_set(x, TYPE)` or `xml_prefix_set(x, PREFIX)` immediately after the statement, you can remove those and set them directly as: `xml_new(name, PREFIX, xp, TYPE)`
2020-03-12 22:29:28 +01:00
Olof hagsand
71efe18f94 NACM datanode write rules have been changed from looking at datastore being chekend (eg running/candidate/startup) to *only* look at running.
Fixed: NACM datanode write problem: read/write/exec default rules did not work.
2020-03-12 14:58:30 +01:00
Olof hagsand
cdcffa768f * Replaced compile option VALIDATE_STATE_XML with runtime option CLICON_VALIDATE_STATE_XML.
* XML childvec Compile error
2020-03-04 11:46:30 +01:00
Olof hagsand
0f54899ae4 * Added option: CLICON_CLI_BUF_START and CLICON_CLI_BUF_THRESHOLD so you can change the start and
threshold of quadratic and linear growth of CLIgen buffers (cbuf:s)
* Memory footprint
  * Do not autopopulate namespace cache, instead use on-demand, see `xml2ns()`.
  * Set CBUF start level to 256 (`CLICON_CLI_BUF_START` option)
  * Reduced xml child vector default size from 4 to 1 with quadratic growoth to 64K then linear
2020-03-04 11:27:43 +01:00
Olof hagsand
2d521d52c8 * Added clixon-stats in clixon-config.yang for clixon XML and memory statistics. 2020-03-01 10:59:24 +01:00
Olof hagsand
a71c256898 * New clixon-config@2020-02-22.yang revision
* C-code changes:
  - Replaced stream uri:s w constants
  - Replaced large debug print code with single clicon_log_xml
  - Restconf put and post handling refactored using new parse API
2020-02-24 15:25:06 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
a674af6f2c Cleaning up code for xml insignificant whitespace removal
Experimenal explicit index search code
2020-02-07 14:59:57 +01:00
Olof hagsand
c7d6f69a85 * Session-id CLI functionality delayed: "lazy evaluation"
* From a cli perspective this is a revert to 4.1 behaviour, where the cli does not immediately exit on start if the backend is not running, but with the new session-id function
2020-02-05 15:37:25 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof Hagsand
a87e9cb246 clang warnings 2019-12-23 14:03:42 +00:00
Olof hagsand
7fb452f96e nacm external mem leak 2019-12-22 18:00:46 +01:00
Olof hagsand
b78054e7cf State callback error message changed to include namespace and use proper <bad-element> 2019-12-21 20:50:55 +01:00
Olof hagsand
7ad16bd84b * C-API: Added xpath_first_localonly() as an xpath function that skips prefix and namespace checks.
* Added experimental code for optizing XPath search using binary search.
  * Enable with XPATH_LIST_OPTIMIZE
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
2019-12-20 18:21:46 +01:00
Olof hagsand
d68a829862 * Added namespace-context parameter nsc to xpath_first and xpath_vec, (xpath_vec_nsc and xpath_first_nsc` are removed). 2019-12-07 20:46:13 +01:00
Olof hagsand
46552066d1 Added nsc parameter to xmldb_get() 2019-12-02 20:55:28 +01:00
Olof hagsand
08b128f4d7 moved config yang spec from main functions to options_main function 2019-12-01 17:42:34 +01:00
Olof hagsand
c4954f5c43 memerror in uid-get. reinsert save config yang spec (removed for wrong reasons previously causing memleaks) 2019-12-01 16:48:45 +01:00
Olof hagsand
0d22a8b6d0 Yang files reorganized into three classes: clixon, mandatory, optional 2019-11-26 21:51:28 +01:00
Olof hagsand
d0e97ee338 Removed obsolete config yang handler
Added base netconf as default namespace in canonical nsctx
2019-11-16 16:49:40 +01:00
Olof hagsand
52629d7b35 Added "canonical" global namespace context: nsctx_global 2019-11-15 17:04:13 +01:00
Olof hagsand
43eb9fe9d2 state callback xml error 2019-11-14 22:16:45 +01:00
Olof hagsand
6b9a9d46f9 changed netconf_err2cb 2019-11-12 20:47:09 +01:00
Olof hagsand
835f9030d2 * State callbacks provided by user are validated. If they are invalid an internal error is returned.
* Fixed multi-namespace for augmented state which was not covered in 4.2.0.
* The multi-namespace augment state may rearrange the XML namespace attributes.
* Mandatory variables can no longer be deleted.
2019-11-11 21:03:11 +01:00
Olof hagsand
728fe9c6ac * Bumped version to 4.3.0.PRE
* Added wildcard `*` as a mode to `CLICON_MODE` in clispec files
* [Add missing includes](https://github.com/clicon/clixon/pulls)
2019-11-03 13:05:50 +01:00
Olof hagsand
84c94b2c0e recover user as config option instead of constant 2019-10-24 20:43:03 +02:00
Olof hagsand
44138c0071 internal netconf hello mechanism to obtain session-id 2019-10-24 11:46:06 +02:00
Olof Hagsand
9a6478e267 Merge branch 'master' of https://github.com/clicon/clixon 2019-10-19 20:58:07 +00:00
Olof Hagsand
28a62ecb11 memleak 2019-10-19 20:58:03 +00:00
Olof hagsand
1fa45067a7 typo 2019-10-19 22:56:25 +02:00