Commit graph

69 commits

Author SHA1 Message Date
Olof hagsand
e0f5472161 Tests: added expect script for pagination stdio 2022-03-14 14:02:21 +01:00
Olof hagsand
77b4468eb3 Updated copyright statements to 2022 2022-01-18 16:36:00 +01:00
Olof hagsand
5388aace12 Refactoring for better performance of large auto-cli specs
* Fixed: [very slow execution of load_set_file #288](https://github.com/clicon/clixon/issues/288)
* New `clixon-lib@2021-11-11.yang` revision
  * Modified option: RPC stats extended with YANG stats
* Modified `clixon-config@2021-11-11.yang` revision
  * Added option:
    * CLICON_PLUGIN_CALLBACK_CHECK
    * Enable to make plugin context check before and after all callbacks.
* Added statistics for YANG: number of objects and memory used
* Use the treeref no-copy option of CLIgen to reduce memory
* Refactored cli-generation/autocli-start code
* Refactored cligen glue functions to use cligen_eval directly (remove clicon_eval,clixon_cligen_eval)
2021-11-25 12:04:05 +01:00
Olof hagsand
0626de9431 * RPC replies now verified with YANG
* Stricter checking of outgoing RPC replies from server
  * See [RPC output not verified by yang](https://github.com/clicon/clixon/issues/283)
  * This lead to some corrections of RPC replies in system code
2021-11-16 22:12:14 +01:00
Olof hagsand
c93348d8d5 Hide plugin check handler struct with a handler 2021-10-19 17:36:23 +02:00
Olof hagsand
764e9c628c Added checks of changed handlers or blocked signal after plugin function calls 2021-10-19 16:42:34 +02:00
Olof hagsand
93d1149925 Dispatcher test additions 2021-10-14 08:58:35 +02:00
Olof hagsand
8469664075 Updated contribution and statedata callback docs 2021-10-10 18:48:10 +02:00
Olof hagsand
8ede8a87b2 Moved fuzz dir to test/fuzz
Extended contributing document to testing and licensing
2021-10-10 18:05:45 +02:00
Olof hagsand
edbbb43e1f Removed remaining and replaced pagination-mode with locked parameter
Dispatcher: Added dispatcher_free(), fixed mem-leaks and malloc return
checks
2021-10-10 11:40:47 +02:00
Olof hagsand
ce06f25be7 Merge branch dispatcher and broke out pagination callbacks to use it
* Merge branch 'dcornejo-master'
* Broke out pagination callback API from state data callbacks
  * New pagination callback API uses new dispatcher from netgate, thanks @dcornejo
   * Register callback with: `clixon_pagination_cb_register()`
   * Use accessor functions `pagination_offset()`, `pagination_limit()`, etc
 * Reverted state data callback API to pre-5.3 (see C/CLI API changes below)
2021-10-08 15:19:37 +02:00
Olof hagsand
a7a699a8b5 New state plugin callback for pagination.
* You need to change all statedata plugin callback for the new pagination feature
  * If you dont use pagination you can ignore the values of the new parameters
  * See [User manual pagination](https://clixon-docs.readthedocs.io/en/latest/misc.html#pagination)
  * The updated callback signature is as follows:
  ```
  int statedata(clicon_handle     h,
                cvec             *nsc,
	        char             *xpath,
	        pagination_mode_t pagmode,   // NEW
	        uint32_t          offset,    // NEW
	        uint32_t          limit,     // NEW
	        uint32_t         *remaining, // NEW
	        cxobj            *xstate)
  ```
2021-09-21 15:02:46 +02:00
Olof hagsand
ff3f600920 Renamed paging to pagination 2021-09-21 11:28:29 +02:00
Olof hagsand
60193cca0d - Add remaining argument to state callback 2021-09-21 11:27:58 +02:00
Olof hagsand
caabfd464e * New state callback signature (ca_statedata2)
* The new callback contains parameters for paging
   * Goal is to replace ca_statedata callback
* New plugin callback when lock/unlock occurs
  * Add `ca_lockdb` tro plugin init to use it.
* Fixed: Typing 'q' in CLI more paging did not properly quit output
  * Output continued but was not shown, for a very large file this could cause considerable delay
* Fixed: Lock was broken in first get get access
  * if the first netconf operation to a backend was lock;get;unlock, the lock was broken in the first get access.
2021-09-21 11:27:58 +02:00
Olof hagsand
aaf9a89183 - Added an extended state plugin callback: ca_statedata2 with offset and limit parameters
- Fixed memory errors
2021-09-21 11:26:28 +02:00
Olof hagsand
392e6679c5 * JSON errors are now labelled with JSON and not XML
* Fixed: [Performance issue when parsing large JSON param](https://github.com/clicon/clixon/issues/266)
* Moved strlen() from for end condition
* Fixed debugging of xpath parser
2021-09-20 21:35:01 +02:00
Olof hagsand
b88722fa25 - Restructure plugin module using a "module struct" rather than global variables.
This include plugin module init and exit functions
  New type: clixon_plugin_t exposed via public API while struct clixon_plugin is private
- Makefile changes for static linkage using "LINKAGE=static"
- Moved nacm external init to later stage in backend_main
2021-04-19 11:06:40 +02:00
Olof hagsand
244060fddc - Added specific WITH_RESTCONF compile contants for _NATIVE and _FCGI for C code
- Restconf auth=none changes
  - Load clixon-restconf and resolve features earlier so that config features work
  - Removed auth=none code from example (this was non-std half-baked basic auth)
  - Changed tests that used auth-type=none to enable feature clixon-restconf:allow-auth-none
- Moved cert creation from sub-shell to servercert function
- Fixed typos for dockerfile rename of restconf evhtp to native
2021-04-07 20:54:41 +02:00
Olof hagsand
a8d29b1e7e * Restconf authentication callback (ca_auth) signature changed
* Not backward compatible: All uses of the ca-auth callback in restconf plugins must be changed
  * New version is: `int ca_auth(h, req, auth_type, authp, userp)`
    * where `auth_type` is the requested authentication-type (none, client-cert or user-defined)
    * `authp` is the returned authentication flag
    * `userp` is the returned associated authenticated user
    * and the return value is three-valued: -1: Error, 0: not handled, 1: OK
  * For more info see [clixon-docs/restconf](https://clixon-docs.readthedocs.io/en/latest/restconf.html)
2021-03-07 15:34:26 +01:00
Olof hagsand
b7991d9b39 * Made a separate Clixon datastore XML/JSON top-level symbol
* Replaces the hardcoded "config" keyword.
  * Implemented by a compile-time option called `DATASTORE_TOP_SYMBOL` option in clixon_custom.h
* Tests: added endtest to all tests. Removed all premature exits if BE=0
2021-03-05 14:15:15 +01:00
Olof hagsand
b2f9c59a34 * Reverted blocked signal behavior introduced in 5.0.
* Introduced a delay before making process start/stop/restart processes for race conditions when configuring eg restconf
* For restconf `CLICON_BACKEND_RESTCONF_PROCESS`, restart restconf if restconf is edited.
2021-03-02 12:24:07 +01:00
Olof hagsand
710fc76887 * Restconf authentication callback (ca_auth) signature changed
* Not backward compatible: All uses of the ca-auth callback in restconf plugins must be changed
  * New version is: `int ca_auth(h, req, auth_type, authp, userp)`
    * where `auth_type` is the requested authentication-type (none, client-cert or user-defined)
    * `authp` is the returned authentication flag
    * `userp` is the returned associated authenticated user
    * and the return value is three-valued: -1: Error, 0: ignored, 1: OK
  * For more info see [clixon-docs](https://clixon-docs.readthedocs.io/en/latest/restconf.html)
* New clixon-restconf@2020-12-30.yang revision
2021-02-09 21:26:35 +01:00
Olof hagsand
1f0147f996 removed unnecessary strerror() in clixon_err() calls; added 'function' keyword in all function declarations in the test scripts 2021-02-04 12:04:48 +01:00
Olof hagsand
96b50b88e8 Copyright 2021 2021-01-13 14:40:34 +01:00
Olof hagsand
40da4421e6 pre-daemon callbaclk macro typo 2020-11-01 13:52:20 +01:00
Olof hagsand
5c5e45d10a Added new backend plugin: ca_pre-demon called if backend is daemonized just prior to forking. 2020-10-31 17:08:59 +01:00
Olof hagsand
3d33335f4b Last minute documentation and testing changes befoire 4.6 2020-08-13 14:58:42 +02:00
Olof hagsand
593ed2ef3b Added const char * in multiple places 2020-07-02 19:25:48 +02:00
shmuels
58ea4eba27 Changed code so that it will be compatible with c++.
1. Added #ifdef extern "C" to public header files.
2. Changed variables that are key words in c++, to different names (like ns instead of namespace).
3. Changed char* to const char* for constant strings.
2020-07-02 15:00:53 +03:00
Olof hagsand
8f1de15ad3 Changed module-specific upgrade API, not backward compatible. The API has been simplified which means more has to be done by the programmer.
* In summary, a user registers an upgrade callback per module. The callback is called at startup if the module is added, has been removed or if the revision on file is different from the one in the system.
  * The register function has removed `from` and `rev` parameters: `upgrade_callback_register(h, cb, namespace, arg)`
  * The callback function has a new `op` parameter with possible values: `XML_FLAG_ADD`, `XML_FLAG_CHANGE` or `XML_FLAG_CHANGE`: `clicon_upgrade_cb(h, xn, ns, op, from, to, arg, cbret)`
2020-06-26 14:39:04 +02:00
Olof hagsand
9a8c6cf3e6 * New clixon-lib@2020-04-23.yang revision
* Added: stats RPC for clixon XML and memory statistics.
  * Added: restart-plugin RPC for restarting individual plugins without restarting backend.
* xml-stats moved from clixon-config.yang as state data to an rpc `datastats` in clixon-lib.yang
* Experimental: restart_plugin
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
    * Note: If you have used "end" callback and usign transaction data, you should probably use this instead.
2020-04-28 22:10:06 +02:00
Olof hagsand
1c99bd6a9b * xml-stats moved from clixon-config.yang as state data to an rpc datastatsin clixon-lib.yang
* Two new plugin callbacks added
  * ca_daemon: Called just after a server has "daemonized", ie put in background.
  * ca_trans_commit_done: Called when all plugin commits have been done.
2020-04-23 22:58:57 +02:00
Olof hagsand
65806f1ef2 Copyright change: LLC -> LLC(Netgate) 2020-04-23 16:55:56 +02:00
Olof hagsand
9fa5e216c4 * New "general-purpose" datastore upgrade callback added which i called once on startup, intended for lo
w-level general upgrades and as a complement to module-specific upgrade.
  * Called on startup after initial XML parsing, but before module-specific upgrades
  * Enabled by definign the `.ca_datastore_upgrade`
  * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#ge
neral-purpose)
* JSON parse error messages change from ` on line x: syntax error,..` to `json_parse: line x: syntax err
or`
* Unknown-element error message is more descriptive, eg from `namespace is: urn:example:clixon` to: `Fai
led to find YANG spec of XML node: x with parent: xp in namespace urn:example:clixon`.
* C-API parse and validation API more capable
  * `xml_spec_populate` family of functions extended with three-value return values
    * -1: error, 0: parse OK, 1: parse and YANG binding OK.
  * `xml_parse` and `json_parse` API changes
    * Three value returns: -1: error, 0: parse OK, 1: parse and YANG binding OK.
    * Extended `xml_parse_file2` and `xml_parse_string2` extended API functions with all options available.
      * New concept called `yang_bind` that defines how XML symbols are bound to YANG after parsing
    * Existing API same except `xml_parse_file` `endtag` argument moved to `xml_parse_file2`
* C-API: Added instrumentation: `xml_size` and `xml_stats_get`.
* Fixed: Enabling modstate (CLICON_XMLDB_MODSTATE), changing a revision on a yang, and restarting made the backend daemon exit at start (thanks Matt)
  * Also: ensure to load `ietf-yang-library.yang ` if CLICON_XMLDB_MODSTATE is set
2020-02-20 14:00:01 +01:00
User Olof
8c18f2a86d Configure and test modification for better Freebsd port 2019-09-20 06:09:48 -10:00
Olof hagsand
fe1aeda4f2 test env bash; freebsd config 2019-09-17 22:44:01 +02:00
Olof hagsand
e7b60619da * Pushed tag to 4.0.1.PRE
* Restconf RFC 8040 increased feature compliance
  * Cache-Control: no-cache added in HTTP responses (RFC Section 5.5)
  * Restconf monitoring capabilities (RFC Section 9.1)
* Added support for Yang extensions
  * New plugin callback: ca_extension
  * Main backend example includes example code on how to implement a Yang extension in a plugin.
* JSON changes
  * Non-pretty-print output removed all extra spaces.
    * Example: `{"nacm-example:x": 42}` --> {"nacm-example:x":42}`
  * Empty JSON container changed from `null` to `{}`.
    * Empty list and leafs remain as `null`
* Removed unnecessary configure dependencies
  * libnsl, libcrypt, libm, if_vlan,...
* pseudo-plugin added, to enable callbacks also for main programs. Useful for extensions
* Yang Unique statements with multiple schema identifiers did not work on some platforms due to memory error.
2019-07-23 22:11:14 +02:00
Olof hagsand
67b8685bab The Clixon API has been extended with namespaces, or namespace contexts in the following cases:
* CLIspec functions have added namespace parameter:
    * `cli_show_config <db> <format> <xpath>` --> `cli_show_config <db> <format> <xpath> <namespace>`
    * `cli_copy_config <db> <xpath> ...` --> `cli_copy_config <db> <xpath> <namespace> ...`
  * Xpath API
    * `xpath_first(x, format, ...)` --> `xpath_first(x, nsc, format, ...)`
    * `xpath_vec(x, format, vec, veclen, ...)` --> `xpath_vec(x, nsc, format, vec, veclen, ...)`
    * `xpath_vec_flag(x, format, flags, vec, veclen, ...)` --> `xpath_vec_flag(x, format, flags, vec, veclen, ...)`
    * `xpath_vec_bool(x, format, ...)` --> `xpath_vec_bool(x, nsc, format, ...)`
    * `xpath_vec_ctx(x, xpath, xp)` --> `xpath_vec_ctx(x, nsc, xpath, xp)`
  * xmldb_get0 has an added `nsc` parameter:
    * `xmldb_get0(h, db, xpath, copy, xret, msd)` --> `xmldb_get0(h, db, nsc, xpath, copy, xret, msd)`
  * The plugin statedata callback (ca_statedata) has been extended with an nsc parameter:
    * `int example_statedata(clicon_handle h, cvec *nsc, char *xpath, cxobj *xstate);`
  * rpc get and get-config api function has an added namespace argument:
    * `clicon_rpc_get_config(clicon_handle h, char *db, char *xpath, char *namespace, cxobj **xt);`
    * `int clicon_rpc_get(clicon_handle h, char *xpath, char *namespace, cxobj **xt);`
2019-07-08 10:36:37 +02:00
Olof hagsand
b103599a1a * Clixon transaction mechanism has changed which may affect your backend plugin callbacks:
* Validate-only transactions are terminated by an `end` or `abort` callback.
* If a commit user callback fails, a new `revert` callback will be made to plugins that have made a succesful commit.
2019-05-10 14:45:23 +02:00
Olof hagsand
60cb87f998 Removed argc/argv parameters from ca_start plugin API function. You may need to change signatures of your startup in your plugins 2019-03-28 17:11:55 +01:00
Olof hagsand
be59bd48d8 upgrade example and test 2019-03-27 18:04:01 +01:00
Olof hagsand
4902f7cf1d Merge branch 'develop' 2019-03-27 16:32:01 +01:00
Olof hagsand
8624be0a67 xml changelog next iteration 2019-03-26 12:04:51 +01:00
Olof hagsand
a0abf8436e XML chanelog revision 2019-03-25 10:42:27 +01:00
Olof Hagsand
10d45bcee3 Memory leaks and errors 2019-03-21 20:36:47 +01:00
Olof hagsand
3f68cca06c * New yang changelog experimental feature for automatic upgrade
* Added modules-state diff parameter to xmldb_get datastore function for startup scenarios.
* Allowed Yang extended Xpath functions (syntax only):
  * re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
* XSD regular expression handling of dash(`-`)
  *: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`.
* YANG Anydata treated same as Anyxml
2019-03-21 18:05:55 +01:00
Olof hagsand
b1c74b5f1f * Ensured you can add multiple callbacks for any RPC, including basic ones.
* Extra RPC:s will be called _after_ the basic ones.
  * One specific usecase is hook for `copy-config` (see [doc/ROADMAP.md] that can be implemented thus way.
* `rpc_callback_register` added a namespace parameter. Example:
   ```
     rpc_callback_register(h, empty_rpc, NULL, "urn:example:clixon", "empty");
   ```
2019-03-10 17:27:52 +01:00
Olof hagsand
77387e9c65 Merge branch 'master' of https://github.com/dcornejo/clixon into dcornejo-master 2019-03-06 10:03:58 +01:00
Dave Cornejo
e2e465eb7b remove redundant typedef that caused problems with some compilers
thanks to report from Piotr Kubaj against FreeBSD port:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236195
2019-03-04 09:24:42 -10:00