Commit graph

172 commits

Author SHA1 Message Date
Olof hagsand
c31b1c471b * Changed first parameter from int fd to FILE *f in the following functions:
* clixon_xml_parse_file(), clixon_json_parse_file(), yang_parse_file()
  * See [Bytewise read() of files is slow #146](https://github.com/clicon/clixon/issues/146)
2020-11-04 22:21:42 +01:00
Olof hagsand
1614bcd19c Cleanup master for 4.9.0.PRE 2020-10-19 20:06:20 +02:00
Olof hagsand
f07f291bdd Main examples uses new auto-mode 2020-10-18 00:02:38 +02:00
Olof hagsand
2d56c9674a * New YANG generated auto-cli feature with syntax modes
* The existing autocli does not support modes, complete paths must be given, eg: `set a b c d 42`.
  * In the new auto-cli, automatic modes are present at each YANG syntax node level, eg the above can be given as: `edit a b c; set d 4; top`
  * The existing CLI API remains, the new API is as follows: `cli_auto_edit()`, `cli_auto_up()`, `cli_auto_top()`, `cli_auto_show()`, `cli_auto_set()`, `cli_auto_merge()`, `cli_auto_create()`, `cli_auto_del()`.
  * See `test/test_cli_auto.sh` for an example of the new API, and `apps/cli/cli_auto.c` for the source code of the new callback API.
  * Documentation will be appear and full integration with the main example.
* Added inline state field to clixon-example.yang
* Added new clicon_data_cvec_*() API for generic cvec structs
2020-10-14 10:54:10 +02:00
Olof hagsand
09d33acd45 cli show options. doc and testing 2020-10-01 11:07:57 +02:00
Olof hagsand
101a8cb6e0 minor fixes 2020-09-27 17:15:36 +02:00
Olof hagsand
27bb0a1d2b * Fixed: Restconf failed put/post could leave residue in candidate causing errors in next put/post
* Added -v option for backend plugins to generate validation error
2020-09-10 18:09:17 +02:00
Olof hagsand
2c96cf71ea Changed main example: removed translate yang and replaced testcases w table/parameter 2020-09-08 11:39:55 +02:00
Olof hagsand
1db969d3a7 Main example: remove translate cli line, changed 'startup' to 'copy running startup' 2020-09-07 15:12:20 +02:00
Olof hagsand
81fc7f742b * Netconf as default namespace has been disabled by default.
* Only requests on the form: `<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"><edit-config>...` will be accepted
  * All replies will be on the form: `<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">...`
  * Requests such as: `<rpc><edit-config>...` will not  be accepted.
  * You can revert this behaviour (to clixon pre-4.6 behaviour) by enabling `CLICON_NAMESPACE_NETCONF_DEFAULT`
  * This API change is a consequence of: [copy-config's RPC cxobj parameter does not contain namespace #131](https://github.com/clicon/clixon/issues/131)
2020-09-02 15:44:10 +02:00
Olof hagsand
0e4df0e8fc Added options for Restconf evhtp setting default bind socket address and ports CLICON_RESTCONF_IPV4_ADDR, CLICON_RESTCONF_IPV6_ADDR, CLICON_RESTCONF_HTTP_PORT, CLICON_RESTCONF_HTTPS_PORT 2020-08-21 13:13:20 +02:00
Olof hagsand
65733ffe69 * Added yang-binding yb parameter to xmldb_get0() and all xmldb get functions. 2020-08-03 13:57:04 +02:00
Olof hagsand
daeab85446 removed obsolete example yang, mod cli_dbxml to only append leaf in cli_set 2020-07-30 22:52:42 +02:00
Olof hagsand
593ed2ef3b Added const char * in multiple places 2020-07-02 19:25:48 +02:00
Olof hagsand
c049a397b0 Fixed ssl client certs for evhtp.
* Added SSL cert info as options: CLICON_SSL_SERVER_CERT, CLICON_SSL_SERVER_KEY, CLICON_SSL_CA_CERT
Added config.sh for testing for autotools
2020-07-01 15:11:22 +02:00
Olof hagsand
9c82e97072 Added http support for evhtp (not only https) 2020-06-29 14:41:08 +02:00
Olof hagsand
c2e459e8bb Three errors in new upgrade code: DEL didnt work, empty config did now work, neither did no yang files in a dir 2020-06-27 13:02:04 +02:00
Olof hagsand
8f1de15ad3 Changed module-specific upgrade API, not backward compatible. The API has been simplified which means more has to be done by the programmer.
* In summary, a user registers an upgrade callback per module. The callback is called at startup if the module is added, has been removed or if the revision on file is different from the one in the system.
  * The register function has removed `from` and `rev` parameters: `upgrade_callback_register(h, cb, namespace, arg)`
  * The callback function has a new `op` parameter with possible values: `XML_FLAG_ADD`, `XML_FLAG_CHANGE` or `XML_FLAG_CHANGE`: `clicon_upgrade_cb(h, xn, ns, op, from, to, arg, cbret)`
2020-06-26 14:39:04 +02:00
Olof hagsand
e2b3cdb3f6 Moved restconf_stream.c -> restconf_stream_fcgi.c, made some generaizations and documented what was hardcoded to FCGI 2020-06-23 15:00:41 +02:00
Olof hagsand
73bbcded87 Added restconf handle specialization to implement restconf parameters 2020-06-22 12:47:43 +02:00
Olof hagsand
0adcd94f3f * Added new cli show functions to work with cligen_output for cligen pageing to work. To acheive this, add a callback function as follows:
* xml2txt(...) --> xml2txt_cb(..., cligen_output)
  * xml2cli(...) --> xml2cli_cb(..., cligen_output)
  * clicon_xml2file(...) --> clicon_xml2file_cb(..., cligen_output)
  * xml2json(...) --> xml2json_cb(..., cligen_output)
  * yang_print(...) --> yang_print_cb(..., cligen_output)
2020-06-17 14:04:19 +02:00
Olof hagsand
5120409a56 Auto-cli updates and sync with clixon-docs 2020-06-15 16:07:04 +02:00
Olof hagsand
ff5462ecac Replaced the global variable debug with access function: clicon_debug_get(). 2020-06-09 16:04:49 +02:00
Olof hagsand
14b03f63a3 all clixon event functions prepended with clixon_
Replaced fcgi getparam code with clixon-data parameters
2020-06-05 16:19:03 +02:00
Olof hagsand
8c1372d26a libhttp compile 2020-05-28 17:19:15 +02:00
Olof hagsand
bf04131e54 Added decriptive error message when plugins produce invalid state XML. 2020-05-10 17:10:07 +02:00
Olof hagsand
bc0eb921d0 Added functionality to restart an individual plugin.: New clixon-lib:restart-plugin RPC 2020-05-04 15:04:02 +02:00
Olof hagsand
592c513b83 Fix up state bypass get code 2020-05-04 13:10:43 +02:00
Olof hagsand
94cf4a88b3 * Optimizations
* Reduced memory for attribute and body objects, see `XML_NEW_DIFFERENTIATE` compile-time option.
  * Optimized cbuf handling in parsing and xml2cbuf functions.
  * Optimized xml scanner to read strings rather than single chars
  * Optimized xml_merge for the case of disjunct trees.
2020-04-28 22:31:58 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
1c99bd6a9b * xml-stats moved from clixon-config.yang as state data to an rpc datastatsin clixon-lib.yang
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
2020-04-23 22:58:57 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
04017c97ba Length of xml vector in many structs changed from to since it is a vector size, not byte size. This includes transaction_data_t 2020-04-21 10:17:09 +02:00
Olof hagsand
0d4263e324 * xml_merge() changed to use 3-value return: 1:OK, 0:Yang failed, -1: Error
* `clixon_netconf_error(category, xerr, msg, arg)` removed first argument -> `clixon_netconf_error(xerr, msg, arg)`
* Renamed utility function `clixon_util_insert()` to `clixon_util_xml_mod()` and added merge functionality.
* Fixed: Insertion of subtree leaf nodes were not made in the crrect place, always ended up last regardless of yang spec (if ordered-by system).
2020-04-17 15:47:37 +02:00
Olof hagsand
662d6b0a3f Clixon 4.4 three compile-time options: MOVE_TRANS_END for changing transaction-end callbacks; STATE_ORDERED_BY_SYSTEM to make state data system ordered; XML_MERGE_TWO_ROUNDS to improve xml_merge performance. 2020-04-02 14:28:58 +02:00
Olof hagsand
8131bd1834 2020 copyright 2020-03-27 15:42:03 +01:00
Olof hagsand
09a2e09848 * New XML parsing API:
* `clixon_xml_parse_string()`
   * `clixon_xml_parse_file()`
* New JSON parsing API, with same signature as XML parsing:
   * `clixon_json_parse_string()`
   * `clixon_xml_parse_file()`
* XML YANG binding API have been rearranged as follows:
   * `xml_bind_yang_rpc()`
   * `xml_bind_yang_rpc_reply()`
   * `xml_bind_yang()`
   * `xml_bind_yang0()`
2020-03-19 21:32:27 +01:00
Olof hagsand
d7972ff311 Moved hello example to [clixon-examples](https://github.com/clicon/clixon-examples) 2020-03-11 11:41:24 +01:00
Olof hagsand
adbb683329 C-API:
* All uses of `api_path2xpath_cvv()` should be replaced by `api_path2xpath()`
  * `api_path2xpath()` added an `xerr` argument.
2020-03-10 22:37:16 +01:00
Olof hagsand
0e94937ccf Sanity check of mandatory key statement for Yang LISTs.
* If fails, exit with error message, eg: `Yang error: Sanity check failed: LIST vsDataContainer lacks key statement which MUST be present (See RFC 7950 Sec 7.8.2)`
  * Can be disabled by setting `CLICON_CLICON_YANG_LIST_CHECK` to `false`
2020-03-09 15:48:33 +01:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
Olof hagsand
19e21be0bc Very large commit for upcoming 4.4 release
Major New features

* New and updated search functions using xpath, api-path and instance-id
  * New search functions using api-path and instance_id:
    * C search functions: `clixon_find_instance_id()` and `clixon_find_api_path()`
  * Binary search optimization in lists for indexed leafs in all three formats.
    * This improves search performance to O(logN) which is drastical improvements for large lists.
  * You can also register explicit indexes for making binary search (not only list keys)
  * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)

API changes on existing features (you may need to change your code)
* On failed validation of leafrefs, error message changed from: `No such leaf` to `No leaf <name> matching path <path>`.
* CLI Error message (clicon_rpc_generate_error()) changed when backend returns netconf error to be more descriptive:
  * Original: `Config error: Validate failed. Edit and try again or discard changes: Invalid argument`
  * New (example): `Netconf error: application operation-failed Identityref validation failed, undefined not derived from acl-base . Validate failed. Edit and try again or discard changes"

Minor changes

* Test framework
  * Added `-- -S <file>` command-line to main example to be able to return any state to main example.
  * Added `test/cicd` test scripts for running on a set of other hosts
* C-code restructuring
  * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
    * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
2020-02-02 15:52:30 +01:00
Olof hagsand
7ad16bd84b * C-API: Added xpath_first_localonly() as an xpath function that skips prefix and namespace checks.
* Added experimental code for optizing XPath search using binary search.
  * Enable with XPATH_LIST_OPTIMIZE
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
2019-12-20 18:21:46 +01:00
Olof hagsand
d68a829862 * Added namespace-context parameter nsc to xpath_first and xpath_vec, (xpath_vec_nsc and xpath_first_nsc` are removed). 2019-12-07 20:46:13 +01:00
Olof hagsand
0d22a8b6d0 Yang files reorganized into three classes: clixon, mandatory, optional 2019-11-26 21:51:28 +01:00
Olof hagsand
835f9030d2 * State callbacks provided by user are validated. If they are invalid an internal error is returned.
* Fixed multi-namespace for augmented state which was not covered in 4.2.0.
* The multi-namespace augment state may rearrange the XML namespace attributes.
* Mandatory variables can no longer be deleted.
2019-11-11 21:03:11 +01:00
Olof hagsand
728fe9c6ac * Bumped version to 4.3.0.PRE
* Added wildcard `*` as a mode to `CLICON_MODE` in clispec files
* [Add missing includes](https://github.com/clicon/clixon/pulls)
2019-11-03 13:05:50 +01:00
Olof Hagsand
d7bb96b9cf FreeBSD modifications: Configure, makefiles and test scripts modification for Freebsd 2019-10-20 09:03:31 -10:00
Olof hagsand
6e41592aec * Changed clicon_rpc_get and clicon_rpc_get_config functions: added username and replaced namespace with namespace context
* Fixed several issues with multiple namespaces.
2019-10-03 22:01:16 +02:00
Olof hagsand
6df434093e * Restconf top-level operations GET root resource modified to comply with
RFC 8040 Sec 3.1
  * non-pretty print remove all spaces, eg `{"operations":{"clixon-example:client-rpc":[null]`
  * Replaced JSON `null` with `[null]` as proper empty JSON leaf/leaf-list encoding.
* [Cannot write to config using restconf example #91](https://github.com/clicon/clixon/issues/91)
  * Updated restconf documentation (the example was wrong)
* [clixon-lib yang revision file name update #92](https://github.com/clicon/clixon/issues/92)
  * Clixon-lib yang file had conflicting filename and internal yang revision.
  * This was only detected in the use-case when a whole dir was loaded.
  * Inserted sanity check in all yang parse routines.
  * Committed updated clixon-lib yang file that triggered the error
2019-08-24 15:30:43 +02:00