Commit graph

58 commits

Author SHA1 Message Date
Olof hagsand
3916fa919c Netconf monitoring RFC 6022 Sessions (https://github.com/clicon/clixon/issues/370)
- statistics and transport/source-host parameters
  - extended internal NETCONF hello with transport and source-host attributes
clixon-lib,yang
  - Moved all extended internal NETCONF attributes to the clicon-lib namespace
C-API:
  - wrapped most attribute creation into new fn xml_add_attr()
2023-01-15 12:36:59 +01:00
Olof hagsand
8ebab16c4c Add proper error message if namespace not found in module-set header 2022-11-28 13:50:41 +01:00
Olof hagsand
d84c529ff1 [Code formatting: Change indentation style to space](https://github.com/clicon/clixon/issues/379)
* Applies to all c/h/y/l/sh files and .editorconfig
2022-10-27 14:21:17 +02:00
Olof hagsand
3bf8a7dca3 Assigned meta-data for the ietf-netconf-with-defaults:default attribute for JSON (RFC8040 Sec 5.3.2) 2022-09-16 14:49:20 +02:00
Olof hagsand
580396d32f - Misc documentation
- Test: move pid file from tmp to global storage to prevent multiple instances
2022-04-14 15:21:25 +02:00
Olof hagsand
dd2ac56af8 Module state upgrade: RFC7895 to RFC 8525
Changes to clixon-config:
  * Added option: 'CLICON_YANG_LIBRARY`
  * Changed default value: `CLICON_MODULE_LIBRARY_RFC7895` to false
Datastore modstate is backward compatible
Tests upgraded to RFC8525
2022-03-07 10:56:54 +01:00
Olof hagsand
77b4468eb3 Updated copyright statements to 2022 2022-01-18 16:36:00 +01:00
Olof hagsand
081e6871b3 * Renamed clixon-clispec.yang to clixon-autocli.yang
* First version of clixon-autocli.yang semantics
   * Default rules for module exclusion, list-keywords, completion, treeref-state
   * Specialized rules for compress and exclusion of modules
   * See [autocli documentation](https://clixon-docs.readthedocs.io/en/latest/cli.html#autocli)
* Obsoleted and moved autocli config options from clixon-config.yang to clixon-autocli.yang as follows:
   * `CLICON_CLI_GENMODEL`, use: `autocli/module-default=false` instead
      * Removed `clicon_cli_genmodel()`
   * `CLICON_CLI_GENMODEL_TYPE`, use `autocli/list-keyword-default` and compress rules instead)
      * Removed `clicon_cli_genmodel_type()`
   * `CLICON_CLI_GENMODEL_COMPLETION`, use `autocli/completion-default` instead
      * Removed `clicon_cli_genmodel_completion()`
   * `CLICON_CLI_AUTOCLI_EXCLUDE`, use `autocli/rule/operation=exclude` instead
   * `CLICON_CLI_MODEL_TREENAME`, use constant `AUTOCLI_TREENAME` instead
     * Removed `clicon_cli_model_treename()`
* New YANG functions: yang_single_child_type, yang_find_namespace_by_prefix, yang_str2key
* Changed return values of yang_find_prefix_by_namespace
* Merged `cli_cli2xml()` into `cli2xml()`
2022-01-07 12:50:07 +01:00
Olof hagsand
47141089c2 - Added netconf to fuzz
- Restconf internal start more robust
- Restconf fcgi fix
2021-09-22 18:55:32 +02:00
Olof hagsand
6bf3112fe7 * JSON encoding of YANG metadata according to RFC 7952
* XML -> JSON translation
2021-09-21 11:23:40 +02:00
Olof hagsand
76e59873c2 - Changed media name: yang.collection+xml/yang to application-collection+xml/json
- Modified ietf-netconf-list-pagination.yang:
     - changed get-pagable ->  get-pageable
     - renamed count -> limit
     - renamed skip -> offset
     - added import ietf-yang-metadata
     - added md:annotation remaining
2021-09-21 11:22:41 +02:00
Olof hagsand
0c7f2043f3 * Pagination according to new draft
* count/skip -> limit/offset
* ietf-yang-metadata RFC 7952 support, placeholder parsing and extension
2021-09-21 11:22:41 +02:00
Olof hagsand
dee081646c * Fixed: SEGV in clixon_netconf_lib functions from internal errors including validation.
* Check xerr argument both before and after call on netconf lib functions
2021-07-27 10:31:16 +02:00
Olof hagsand
df6f26c0de Misc cleanups, remove old yangs, typos, etc 2021-07-19 22:08:23 +02:00
Olof hagsand
5af7ea9b38 * Added check of old config in upgrade scenarios.
* Added to clixon-config.yang `CLICON_XMLDB_UPGRADE_CHECKOLD`, set to false to keep less strict check
2021-04-12 21:03:20 +02:00
Olof hagsand
8469a0962e * Add multiple yang support also for obsolete versions
* This means that files and datastores supporting modstate also looks for deleted or updated yang modules.
  * A stricter binding which gives error if loading outdated YANG file does not exist.
* Stricter yang checks: you cannot do get-config on datastores that have obsolete YANG
* Added xerr output parameter to `xmldb_get0()`
2021-04-12 12:42:26 +02:00
Olof hagsand
f8830dc27e Fixed: [yang submodule show error in modules-state #190](yang submodule show error in modules-state #190) 2021-03-17 15:44:45 +01:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
4d91bcd810 GET xpath on modstate was broken 2020-10-01 22:22:36 +02:00
Olof hagsand
576533ed82 Changed singnature of ys_real_module and improved error handling 2020-08-27 11:01:09 +02:00
Vladimir Ratnikov
f5d57ae50f fix clauses 2020-07-28 06:03:22 -04:00
Vladimir Ratnikov
8b5c77838b Don't call upgrade callbacks for module with no revision defined 2020-07-28 05:52:07 -04:00
shmuels
58ea4eba27 Changed code so that it will be compatible with c++.
1. Added #ifdef extern "C" to public header files.
2. Changed variables that are key words in c++, to different names (like ns instead of namespace).
3. Changed char* to const char* for constant strings.
2020-07-02 15:00:53 +03:00
Olof hagsand
c2e459e8bb Three errors in new upgrade code: DEL didnt work, empty config did now work, neither did no yang files in a dir 2020-06-27 13:02:04 +02:00
Olof hagsand
8f1de15ad3 Changed module-specific upgrade API, not backward compatible. The API has been simplified which means more has to be done by the programmer.
* In summary, a user registers an upgrade callback per module. The callback is called at startup if the module is added, has been removed or if the revision on file is different from the one in the system.
  * The register function has removed `from` and `rev` parameters: `upgrade_callback_register(h, cb, namespace, arg)`
  * The callback function has a new `op` parameter with possible values: `XML_FLAG_ADD`, `XML_FLAG_CHANGE` or `XML_FLAG_CHANGE`: `clicon_upgrade_cb(h, xn, ns, op, from, to, arg, cbret)`
2020-06-26 14:39:04 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
75b20936d3 Moved code from clixon_xml.c to namespace and io functions to reduce size of core xml file 2020-03-02 09:43:57 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
b340c36f79 Experimental optimized XPath, multiple keys 2019-12-31 13:29:25 +01:00
Olof hagsand
7ad16bd84b * C-API: Added xpath_first_localonly() as an xpath function that skips prefix and namespace checks.
* Added experimental code for optizing XPath search using binary search.
  * Enable with XPATH_LIST_OPTIMIZE
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
2019-12-20 18:21:46 +01:00
Olof hagsand
d68a829862 * Added namespace-context parameter nsc to xpath_first and xpath_vec, (xpath_vec_nsc and xpath_first_nsc` are removed). 2019-12-07 20:46:13 +01:00
Olof hagsand
44138c0071 internal netconf hello mechanism to obtain session-id 2019-10-24 11:46:06 +02:00
User Olof
8c18f2a86d Configure and test modification for better Freebsd port 2019-09-20 06:09:48 -10:00
Olof hagsand
40d5b99d3b Reverted some of the nsc xpath API changes. In the revert, xpath_first() and xpath_vec() retain their old syntax with nsc==NULL.
The reason is to be conservative with the API. However, less used functions, such as xpath_vec_bool(), xpath_vec_ctx() and xpath_vec_flag()  are changed with a new `nsc`parameter, which should be set to NULL in most cases.
2019-07-09 12:19:03 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof Hagsand
a7af7e7593 memleaks 2019-05-22 18:27:53 +00:00
Olof hagsand
bc54f2d04c * Regexp improvements
* Added check for libxml in configure';
  * Added clixon_util_regexp utility function
* Yang state get improvements
  * Integrated state and config into same tree on retrieval, not separate trees
  * Added cli functions `cli_show_config_state()` and `cli_show_auto_state()` for showing combined config and state info.
  * Added integrated state in the main example: `interface/oper-state`.
  * Added performance tests for getting state, see [test/test_perf_state.sh].
2019-05-20 16:03:29 +02:00
Olof hagsand
1cfb190ea9 * Changed return values in internal functions: netconf_trymerge, startup_module_state, yang_modules_state_get 2019-05-07 13:26:34 +02:00
Olof hagsand
16de5f47ba Merge branch 'master' of https://github.com/clicon/clixon 2019-04-24 15:35:57 +02:00
Olof hagsand
65c809b1c3 Removed external direct access to the yang_stmt struct. 2019-04-24 15:13:19 +02:00
Olof hagsand
d46ca41c8b Further optimizions and bugfixing of that 2019-04-16 12:09:21 +02:00
Olof hagsand
b5f1f7346e Structural C-code change: Merged yang_spec and yang_node types into yang_stmt 2019-04-02 12:27:33 +02:00
Olof hagsand
98a5ebc76e * Structural change: removed datastore plugin and directory, and merged into regulat clixon lib code.
* Moved out code from clixon_options.[ch] into a new file: clixon_data.[ch]
2019-03-31 18:17:40 +02:00
Olof hagsand
a0abf8436e XML chanelog revision 2019-03-25 10:42:27 +01:00
Olof hagsand
3f68cca06c * New yang changelog experimental feature for automatic upgrade
* Added modules-state diff parameter to xmldb_get datastore function for startup scenarios.
* Allowed Yang extended Xpath functions (syntax only):
  * re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
* XSD regular expression handling of dash(`-`)
  *: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`.
* YANG Anydata treated same as Anyxml
2019-03-21 18:05:55 +01:00
Olof hagsand
2394c6f46e Merge branch 'modules-state' 2019-02-26 16:53:59 +01:00
Olof hagsand
560110b4e8 * New backend startup and upgrade support, see [doc/startup.md] for details
* Datastore files contain RFC7895 module-state information
2019-02-26 15:48:15 +01:00
Olof hagsand
3c332b689f Added cache for modules-state RFC7895 to avoid building new XML every get call 2019-02-21 15:06:20 +01:00