experiments/clixon
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

All clixon test utilities in util/ moved to separate repo: clicon/clixon-util

Browse source
This commit is contained in:
Olof hagsand 2023-12-01 14:30:14 +01:00
parent 631ebaa759
commit 80a10b694c
29 changed files with 166 additions and 17293 deletions
Download patch file Download diff file Expand all files Collapse all files

265
CHANGELOG.md
View file

@ -48,6 +48,8 @@ Expected: December 2023
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
* All clixon test utilities in util/ have been moved to a separate repo: clicon/clixon-util
* To run tests you need to clone, build and install them separately
* Moved and split install of main example config file * Moved and split install of main example config file
* From `/usr/local/etc/example.xml` to `/usr/local/etc/clixon/example.xml` * From `/usr/local/etc/example.xml` to `/usr/local/etc/clixon/example.xml`
* Added `/usr/local/etc/clixon/example/autocli.xml` and `/usr/local/etc/clixon/example/restconf.xml` * Added `/usr/local/etc/clixon/example/autocli.xml` and `/usr/local/etc/clixon/example/restconf.xml`
@ -156,7 +158,7 @@ Clixon 6.3 introduces CLI output pipes and multiple updates and optimizations, p
* CLI output pipes * CLI output pipes
* Building on a new CLIgen feature * Building on a new CLIgen feature
* See https://clixon-docs.readthedocs.io/en/latest/cli.html#output-pipes * See https://clixon-docs.readthedocs.io/en/latest/cli.html#output-pipes
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -169,7 +171,7 @@ Users may have to change how they access the system
* New `clixon-autocli@2023-05-01.yang` revision * New `clixon-autocli@2023-05-01.yang` revision
* New `alias` and `skip` extensions (NOTE: just added in YANG, not implemented) * New `alias` and `skip` extensions (NOTE: just added in YANG, not implemented)
* New `grouping-treeref` option * New `grouping-treeref` option
### C/CLI-API changes on existing features ### C/CLI-API changes on existing features
Developers may need to change their code Developers may need to change their code
@ -231,17 +233,17 @@ Developers may need to change their code
* Example change: * Example change:
* `clixon_xml2file(f,x,p,f,s,a)` -> `clixon_xml2file(f,x,p,NULL,f,s,a)` * `clixon_xml2file(f,x,p,f,s,a)` -> `clixon_xml2file(f,x,p,NULL,f,s,a)`
* `clixon_xml2cbuf(c,x,l,p,d,s)` -> `clixon_xml2cbuf(c,x,l,p,NULL,d,s)` * `clixon_xml2cbuf(c,x,l,p,d,s)` -> `clixon_xml2cbuf(c,x,l,p,NULL,d,s)`
* `xmldb_validate` is removed. Yang checks should be enough, remnant of time before YANG checks. * `xmldb_validate` is removed. Yang checks should be enough, remnant of time before YANG checks.
* `xml_diff`: removed 1st `yspec` parameter * `xml_diff`: removed 1st `yspec` parameter
* `xml2xpath()`: Added `int apostrophe` as 4th parameter, default 0 * `xml2xpath()`: Added `int apostrophe` as 4th parameter, default 0
* This is for being able to choose single or double quote as xpath literal quotes * This is for being able to choose single or double quote as xpath literal quotes
* `clicon_msg_rcv`: Added `intr` parameter for interrupting on `^C` (default 0) * `clicon_msg_rcv`: Added `intr` parameter for interrupting on `^C` (default 0)
* Renamed include file: `clixon_backend_handle.h`to `clixon_backend_client.h` * Renamed include file: `clixon_backend_handle.h`to `clixon_backend_client.h`
* `candidate_commit()`: validate_level (added in 6.1) marked obsolete * `candidate_commit()`: validate_level (added in 6.1) marked obsolete
### Minor features ### Minor features
* Adjusted to Openssl 3.0 * Adjusted to Openssl 3.0
* Unified netconf input function * Unified netconf input function
* Three different implementations were used in external, internal and controller code * Three different implementations were used in external, internal and controller code
* Internal netconf still not moved to unified * Internal netconf still not moved to unified
@ -287,7 +289,7 @@ The Clixon 6.1 release completes Network monitoring (RFC 6022) and introduces a
* Hello statistics is based on backend statistics, hellos from RESTCONF, SNMP and CLI clients are included and dropped external NETCONF sessions are not * Hello statistics is based on backend statistics, hellos from RESTCONF, SNMP and CLI clients are included and dropped external NETCONF sessions are not
* RFC 6022 "YANG Module for NETCONF Monitoring" * RFC 6022 "YANG Module for NETCONF Monitoring"
* See [Feature Request: Support RFC 6022 (NETCONF Monitoring)](https://github.com/clicon/clixon/issues/370) * See [Feature Request: Support RFC 6022 (NETCONF Monitoring)](https://github.com/clicon/clixon/issues/370)
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -305,7 +307,7 @@ Users may have to change how they access the system
return implicit default values, only explicitly set values. return implicit default values, only explicitly set values.
* Applies to NETCONF `<get>`, `<get-config>` and RESTCONF `GET` * Applies to NETCONF `<get>`, `<get-config>` and RESTCONF `GET`
* To keep backward-compatible behavior, define option `NETCONF_DEFAULT_RETRIEVAL_REPORT_ALL` in `include/clixon_custom.h` * To keep backward-compatible behavior, define option `NETCONF_DEFAULT_RETRIEVAL_REPORT_ALL` in `include/clixon_custom.h`
* Alternatively, change all get operation to include with-defaults parameter `report-all` * Alternatively, change all get operation to include with-defaults parameter `report-all`
### C/CLI-API changes on existing features ### C/CLI-API changes on existing features
Developers may need to change their code Developers may need to change their code
@ -321,9 +323,9 @@ Developers may need to change their code
* This was for making an xpath to a yang-mount point (only for yang-mount) * This was for making an xpath to a yang-mount point (only for yang-mount)
* Example change: * Example change:
* `xml2xpath(x, n, xp)` -> `xml2xpath(x, n, 0, xp)` * `xml2xpath(x, n, xp)` -> `xml2xpath(x, n, 0, xp)`
* `xml_bind_*()` functions: Added `clicon_handle h` as first parameter * `xml_bind_*()` functions: Added `clicon_handle h` as first parameter
* Example change: * Example change:
* `xml_bind_yang(x, y, yp, xe)` -> `xml_bind_yang(h, x, y, yp, xe)` -> * `xml_bind_yang(x, y, yp, xe)` -> `xml_bind_yang(h, x, y, yp, xe)` ->
* `xmldb_get0()`: Added `with-defaults` parameter, default 0 * `xmldb_get0()`: Added `with-defaults` parameter, default 0
* Example change: * Example change:
* `xmldb_get0(0, db, yb, n, xp, c, x, m, x)` -> `xmldb_get0(0, db, yb, n, xp, c, WITHDEFAULTS_REPORT_ALL, x, m, x)` * `xmldb_get0(0, db, yb, n, xp, c, x, m, x)` -> `xmldb_get0(0, db, yb, n, xp, c, WITHDEFAULTS_REPORT_ALL, x, m, x)`
@ -333,7 +335,7 @@ Developers may need to change their code
* `xpath_vec_flag()`: Changed type of sixth `veclen` parameter to `size_t *` * `xpath_vec_flag()`: Changed type of sixth `veclen` parameter to `size_t *`
* `clicon_log_xml()`: All calls changed to new function `clicon_debug_xml()` * `clicon_log_xml()`: All calls changed to new function `clicon_debug_xml()`
* `clixon_proc_socket()`: Added `sock_flags` parameter * `clixon_proc_socket()`: Added `sock_flags` parameter
### Minor features ### Minor features
* Misc. build fixes encountered when cross-compiling by @troglobit in https://github.com/clicon/clixon/pull/418 * Misc. build fixes encountered when cross-compiling by @troglobit in https://github.com/clicon/clixon/pull/418
@ -414,7 +416,7 @@ Users may have to change how they access the system
* Default value is `3` * Default value is `3`
* NETCONF: Removed `message-id` from hello protocol following RFC 6241 * NETCONF: Removed `message-id` from hello protocol following RFC 6241
* See [message-id present on netconf app "hello"](https://github.com/clicon/clixon/issues/369) * See [message-id present on netconf app "hello"](https://github.com/clicon/clixon/issues/369)
### C/CLI-API changes on existing features ### C/CLI-API changes on existing features
Developers may need to change their code Developers may need to change their code
@ -434,7 +436,7 @@ Developers may need to change their code
* `myid` : Client-id of incoming message * `myid` : Client-id of incoming message
* `vlev` : validate level * `vlev` : validate level
* Both parameters are default `0` for backward-compatibility * Both parameters are default `0` for backward-compatibility
### Minor features ### Minor features
* Removed obsoleted compile-time options since 5.4: * Removed obsoleted compile-time options since 5.4:
@ -531,12 +533,12 @@ Developers may need to change their code
* For upgrade, add new sixth parameter and set it to `NULL`. * For upgrade, add new sixth parameter and set it to `NULL`.
* CLISPEC changes of cli show functions * CLISPEC changes of cli show functions
* For details of updated API, see https://clixon-docs.readthedocs.io/en/latest/cli.html#show-commands * For details of updated API, see https://clixon-docs.readthedocs.io/en/latest/cli.html#show-commands
* Changed `cli_show_auto()` * Changed `cli_show_auto()`
* Added parameters for pretty-print, state and with-default * Added parameters for pretty-print, state and with-default
* If the `<prefix>`is used, you need to change the call as follows: * If the `<prefix>`is used, you need to change the call as follows:
* `cli_show_auto(<db>, <format>, <prefix>)` -> `cli_show_auto(<db>, <format>, true, false, NULL, <prefix>)` * `cli_show_auto(<db>, <format>, <prefix>)` -> `cli_show_auto(<db>, <format>, true, false, NULL, <prefix>)`
* Otherwise the API is backward-compatible * Otherwise the API is backward-compatible
* Changed `cli_show_config()` * Changed `cli_show_config()`
* Added parameters for pretty-print, state and with-default * Added parameters for pretty-print, state and with-default
* If the `<prefix>` parameter is used, you need to change the call as follows: * If the `<prefix>` parameter is used, you need to change the call as follows:
* `cli_show_config(<db>, <format>, <xpath>, <ns>, <prefix>)` -> `cli_show_auto(<db>, <format>, <xpath>, <ns>, true, false, NULL, <prefix>)` * `cli_show_config(<db>, <format>, <xpath>, <ns>, <prefix>)` -> `cli_show_auto(<db>, <format>, <xpath>, <ns>, true, false, NULL, <prefix>)`
@ -547,7 +549,7 @@ Developers may need to change their code
* The first argument is removed. You need to change all calls as follows: * The first argument is removed. You need to change all calls as follows:
* `cli_show_config(<treename>, <db>, ...` -> `cli_show_auto_menu(<db>, ...)` * `cli_show_config(<treename>, <db>, ...` -> `cli_show_auto_menu(<db>, ...)`
* The `cli_auto_show()` callback remains in 5.9.0 for backward compatible reasons, but will be removed in later releaes. * The `cli_auto_show()` callback remains in 5.9.0 for backward compatible reasons, but will be removed in later releaes.
### Minor features ### Minor features
* Restconf: * Restconf:
@ -556,7 +558,7 @@ Developers may need to change their code
### Corrected Bugs ### Corrected Bugs
* Fixed: Leak in restconf http/1 data path: when multiple packets in same connection. * Fixed: Leak in restconf http/1 data path: when multiple packets in same connection.
* Fixed: [Replace operation](https://github.com/clicon/clixon/issues/350) * Fixed: [Replace operation](https://github.com/clicon/clixon/issues/350)
* Fixed: [When multiple lists have same key name, need more elaborate error message in case of configuration having duplicate keys](https://github.com/clicon/clixon/issues/362) * Fixed: [When multiple lists have same key name, need more elaborate error message in case of configuration having duplicate keys](https://github.com/clicon/clixon/issues/362)
* Solved by implementing RFC7950 Sec 5.1 correctly * Solved by implementing RFC7950 Sec 5.1 correctly
@ -602,7 +604,7 @@ New features in Clixon 5.8.0 include a new SNMP frontend, YANG action and parsea
* TEXT output format changed (see API changes) * TEXT output format changed (see API changes)
* FOr more info, see [user manual](https://clixon-docs.readthedocs.io/en/latest/datastore.html#other-formats) * FOr more info, see [user manual](https://clixon-docs.readthedocs.io/en/latest/datastore.html#other-formats)
* See [Support performant load_config_file(...) for TEXT format](https://github.com/clicon/clixon/issues/324) * See [Support performant load_config_file(...) for TEXT format](https://github.com/clicon/clixon/issues/324)
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -619,7 +621,7 @@ Users may have to change how they access the system
* This does not change the semantics, but changes the way XML prefixes are used * This does not change the semantics, but changes the way XML prefixes are used
* Example augmented ipv4 into interface: * Example augmented ipv4 into interface:
* Previously: `<interface><ip:ipv4 xmlns:ip="urn:...:ietf-ip"><ip:enabled>...` * Previously: `<interface><ip:ipv4 xmlns:ip="urn:...:ietf-ip"><ip:enabled>...`
* Now: `<interface><ipv4 xmlns="urn:...:ietf-ip"><enabled>...` * Now: `<interface><ipv4 xmlns="urn:...:ietf-ip"><enabled>...`
### C/CLI-API changes on existing features ### C/CLI-API changes on existing features
@ -714,7 +716,7 @@ Users may have to change how they access the system
* `<if>a</if><if>b</if><if>c</if> * `<if>a</if><if>b</if><if>c</if>
<ifref>b</ifref>` <ifref>b</ifref>`
* Existing behavior: expand to: `a, b, c` * Existing behavior: expand to: `a, b, c`
* New default behavior: expand to: `b` * New default behavior: expand to: `b`
* To keep existing behavior, set `<CLICON_CLI_EXPAND_LEAFREF>true<CLICON_CLI_EXPAND_LEAFREF>` * To keep existing behavior, set `<CLICON_CLI_EXPAND_LEAFREF>true<CLICON_CLI_EXPAND_LEAFREF>`
* Restconf * Restconf
@ -724,7 +726,7 @@ Users may have to change how they access the system
* To keep existing end-of-message encoding, set `CLICON_NETCONF_BASE_CAPABILITY` to `0` * To keep existing end-of-message encoding, set `CLICON_NETCONF_BASE_CAPABILITY` to `0`
* Added `clixon_netconf` command-line option `-0` and changed `-H` to `-1` * Added `clixon_netconf` command-line option `-0` and changed `-H` to `-1`
* `-0` means dont send hello, but fix netconf base version to 0 and use EOM framing * `-0` means dont send hello, but fix netconf base version to 0 and use EOM framing
* `-1` means dont send hello, but fix netconf base version to 1 and use chunked framing * `-1` means dont send hello, but fix netconf base version to 1 and use chunked framing
* Error message `data-not-unique` changed to return schema nodes instead of XML for RFC7950 compliance * Error message `data-not-unique` changed to return schema nodes instead of XML for RFC7950 compliance
* YANG * YANG
* Instead of removing YANG which is disabled by `if-feature`, replace it with an yang `anydata` node. * Instead of removing YANG which is disabled by `if-feature`, replace it with an yang `anydata` node.
@ -832,7 +834,7 @@ Users may have to change how they access the system
* `CLICON_LOG_STRING_LIMIT` * `CLICON_LOG_STRING_LIMIT`
* `CLICON_YANG_LIBRARY` * `CLICON_YANG_LIBRARY`
* Changed default value: * Changed default value:
* `CLICON_MODULE_LIBRARY_RFC7895` to false * `CLICON_MODULE_LIBRARY_RFC7895` to false
* Removed (previosly marked) obsolete options: * Removed (previosly marked) obsolete options:
* `CLICON_RESTCONF_PATH` * `CLICON_RESTCONF_PATH`
* `CLICON_RESTCONF_PRETTY` * `CLICON_RESTCONF_PRETTY`
@ -856,7 +858,7 @@ Users may have to change how they access the system
* Added RFC7951 parameter to `clixon_json_parse_string()` and `clixon_json_parse_file()` * Added RFC7951 parameter to `clixon_json_parse_string()` and `clixon_json_parse_file()`
* If set, honor RFC 7951: JSON Encoding of Data Modeled with YANG, eg it requires module name prefixes * If set, honor RFC 7951: JSON Encoding of Data Modeled with YANG, eg it requires module name prefixes
* If not set, parse as regular JSON * If not set, parse as regular JSON
### Minor features ### Minor features
* Added: [Strict auto completion for CLI argument expansion #163](https://github.com/clicon/clixon/issues/163) * Added: [Strict auto completion for CLI argument expansion #163](https://github.com/clicon/clixon/issues/163)
@ -899,7 +901,7 @@ This release introduces a new autocli design with a clixon-autocli YANG file
* Default is create edit-mode for all containers and list entries * Default is create edit-mode for all containers and list entries
* New edit-mode tree: `@datamodelmode` * New edit-mode tree: `@datamodelmode`
* Moved hide extensions from `clixon-lib` to `clixon-autocli` * Moved hide extensions from `clixon-lib` to `clixon-autocli`
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -941,7 +943,7 @@ Developers may need to change their code
* Remove dependency of IETF YANGs on most tests * Remove dependency of IETF YANGs on most tests
* Remove dependency of example/main in most tests, instead make local copy of example yang * Remove dependency of example/main in most tests, instead make local copy of example yang
* Changed `configure --with-yang-standard-installdir` to `configure --with-yang-standard-dir` * Changed `configure --with-yang-standard-installdir` to `configure --with-yang-standard-dir`
### Corrected Bugs ### Corrected Bugs
* Fixed: Autocli YANG patterns including `"` were not properly escaped: `\"` * Fixed: Autocli YANG patterns including `"` were not properly escaped: `\"`
@ -970,7 +972,7 @@ This release features lots of minor updates and bugfixes, an updated list pagina
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
* Optional yangs for testing have been removed from the Clixon repo * Optional yangs for testing have been removed from the Clixon repo
* As a consequence, the following configure options have been removed: * As a consequence, the following configure options have been removed:
* `configure --with-opt-yang-installdir=DIR` * `configure --with-opt-yang-installdir=DIR`
@ -983,11 +985,11 @@ Users may have to change how they access the system
* Stricter checking of outgoing RPC replies from server * Stricter checking of outgoing RPC replies from server
* See [RPC output not verified by yang](https://github.com/clicon/clixon/issues/283) * See [RPC output not verified by yang](https://github.com/clicon/clixon/issues/283)
* XML to JSON CDATA translation is NOT stripped * XML to JSON CDATA translation is NOT stripped
* Example, assume XML: `<s><![CDATA[ z > x & x < y ]]></s>` * Example, assume XML: `<s><![CDATA[ z > x & x < y ]]></s>`
* Previous bevavior: * Previous bevavior:
* JSON: {"s":" z > x & x < y "} * JSON: {"s":" z > x & x < y "}
* New behavior: * New behavior:
* JSON: `{"s":"<![CDATA[ z > x & x < y ]]>"}` * JSON: `{"s":"<![CDATA[ z > x & x < y ]]>"}`
* To keep old behavior, set `JSON_CDATA_STRIP` in clixon_custom.h * To keep old behavior, set `JSON_CDATA_STRIP` in clixon_custom.h
* New `clixon-lib@2021-11-11.yang` revision * New `clixon-lib@2021-11-11.yang` revision
* Modified option: RPC stats extended with YANG stats * Modified option: RPC stats extended with YANG stats
@ -1054,7 +1056,7 @@ Developers may need to change their code
* PR: [OpenConfig path compression](https://github.com/clicon/clixon/pull/276) * PR: [OpenConfig path compression](https://github.com/clicon/clixon/pull/276)
* C API: Added set/get pointer API to clixon_data: * C API: Added set/get pointer API to clixon_data:
* Added json/cli support for cli save/load * Added json/cli support for cli save/load
* clicon_ptr_get(), clicon_ptr_set(), * clicon_ptr_get(), clicon_ptr_set(),
* Restconf YANG PATCH according to RFC 8072 * Restconf YANG PATCH according to RFC 8072
* Changed YANG PATCH enabling: * Changed YANG PATCH enabling:
* Now: `./configure --enable-yang-patch` * Now: `./configure --enable-yang-patch`
@ -1148,7 +1150,7 @@ Developers may need to change their code
uint32_t *remaining, // NEW uint32_t *remaining, // NEW
cxobj *xstate) cxobj *xstate)
``` ```
### Minor features ### Minor features
* Fuzzing: * Fuzzing:
@ -1243,7 +1245,7 @@ Users may have to change how they access the system
* Changed config and install options for Restconf * Changed config and install options for Restconf
* clixon_restconf daemon is installed in `/usr/local/sbin` (as clixon_backend), instead of /www-data * clixon_restconf daemon is installed in `/usr/local/sbin` (as clixon_backend), instead of /www-data
* `configure --with-wwwdir=<dir>` remains but only applies to fcgi socket and log * `configure --with-wwwdir=<dir>` remains but only applies to fcgi socket and log
* New option `CLICON_RESTCONF_INSTALLDIR` is set to where clixon_restconf is installed, with default `/usr/local/sbin/` * New option `CLICON_RESTCONF_INSTALLDIR` is set to where clixon_restconf is installed, with default `/usr/local/sbin/`
* Restconf drop privileges user is defined by `CLICON_RESTCONF_USER` * Restconf drop privileges user is defined by `CLICON_RESTCONF_USER`
* `configure --with-wwwuser=<user>` is removed * `configure --with-wwwuser=<user>` is removed
* clixon_restconf drop of privileges is defined by `CLICON_RESTCONF_PRIVILEGES` option * clixon_restconf drop of privileges is defined by `CLICON_RESTCONF_PRIVILEGES` option
@ -1257,7 +1259,7 @@ Users may have to change how they access the system
* Added: restconf `log-destination` (syslog or file:`/var/log/clixon_restconf.log`) * Added: restconf `log-destination` (syslog or file:`/var/log/clixon_restconf.log`)
* RESTCONF error replies have changed * RESTCONF error replies have changed
* Added Restconf-style xml/json message bodies everywhere * Added Restconf-style xml/json message bodies everywhere
* Clixon removed the message body from many errors in the 4.6 version since they used html encoding. * Clixon removed the message body from many errors in the 4.6 version since they used html encoding.
* However, the RFC Section 7.1 mandates to use RESTCONF-style message bodies. * However, the RFC Section 7.1 mandates to use RESTCONF-style message bodies.
* RESTCONF in Clixon used empty key as "wildchar". But according to RFC 8040 it should mean the "empty string". * RESTCONF in Clixon used empty key as "wildchar". But according to RFC 8040 it should mean the "empty string".
* Example: `GET restconf/data/x:a=` * Example: `GET restconf/data/x:a=`
@ -1286,13 +1288,13 @@ Developers may need to change their code
* Added new startup-mode: `running-startup`: First try running db, if it is empty try startup db. * Added new startup-mode: `running-startup`: First try running db, if it is empty try startup db.
* See [Can startup mode to be extended to support running-startup mode? #234](https://github.com/clicon/clixon/issues/234) * See [Can startup mode to be extended to support running-startup mode? #234](https://github.com/clicon/clixon/issues/234)
* Restconf: added inline configuration using `-R <xml>` command line as an alternative to making advanced restconf configuration * Restconf: added inline configuration using `-R <xml>` command line as an alternative to making advanced restconf configuration
* New option `CLICON_RESTCONF_STARTUP_DONTUPDATE` added to disable RFC 8040 mandatory copy of running to startup after commit* * New option `CLICON_RESTCONF_STARTUP_DONTUPDATE` added to disable RFC 8040 mandatory copy of running to startup after commit*
* See [Need an option to disable restconf mandatory action of overwriting startup_db #230](https://github.com/clicon/clixon/issues/230) * See [Need an option to disable restconf mandatory action of overwriting startup_db #230](https://github.com/clicon/clixon/issues/230)
* Add default network namespace constant: `RESTCONF_NETNS_DEFAULT` with default value "default". * Add default network namespace constant: `RESTCONF_NETNS_DEFAULT` with default value "default".
* CLI: Two new hide variables added (thanks: shmuelnatan) * CLI: Two new hide variables added (thanks: shmuelnatan)
* hide-database : specifies that a command is not visible in database. This can be useful for setting passwords and not exposing them to users. * hide-database : specifies that a command is not visible in database. This can be useful for setting passwords and not exposing them to users.
* hide-database-auto-completion : specifies that a command is not visible in database and in auto completion. This can be useful for a password that was put in device by super user, not be changed. * hide-database-auto-completion : specifies that a command is not visible in database and in auto completion. This can be useful for a password that was put in device by super user, not be changed.
### Corrected Bugs ### Corrected Bugs
* Fixed: [uses oc-if:interface-ref error with openconfig #233](https://github.com/clicon/clixon/issues/233) * Fixed: [uses oc-if:interface-ref error with openconfig #233](https://github.com/clicon/clixon/issues/233)
@ -1330,7 +1332,7 @@ The 5.1 release contains more RESTCONF native mode restructuring, new multi-yang
* Moved out event handling to clixon event handling * Moved out event handling to clixon event handling
* Moved out all ssl calls to clixon * Moved out all ssl calls to clixon
* Plan is to remove reliance on libevhtp and libevent altogether * Plan is to remove reliance on libevhtp and libevent altogether
* Extended status restconf process message with: * Extended status restconf process message with:
* If `CLICON_BACKEND_RESTCONF_PROCESS` is set, RESTCONF is started as internal process from backend * If `CLICON_BACKEND_RESTCONF_PROCESS` is set, RESTCONF is started as internal process from backend
* Otherwise restconf daemon must be started externally by user, such as by systemd. * Otherwise restconf daemon must be started externally by user, such as by systemd.
* Netconf HELLO made mandatory * Netconf HELLO made mandatory
@ -1343,7 +1345,7 @@ The 5.1 release contains more RESTCONF native mode restructuring, new multi-yang
* Files and datastores supporting modstate also look for deleted or updated yang modules. * Files and datastores supporting modstate also look for deleted or updated yang modules.
* Stricter binding which gives error if loading outdated YANG file does not exist. * Stricter binding which gives error if loading outdated YANG file does not exist.
* Keep old behavior: disable `CLICON_XMLDB_UPGRADE_CHECKOLD`. * Keep old behavior: disable `CLICON_XMLDB_UPGRADE_CHECKOLD`.
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -1370,7 +1372,7 @@ Developers may need to change their code
* To keep existing semantics: `str2cvec(...,cvp) -> str2cvec(...,1, cvp)` * To keep existing semantics: `str2cvec(...,cvp) -> str2cvec(...,1, cvp)`
* Removed `cli_debug()`. Use `cli_debug_backend()` or `cli_debug_restconf()` instead. * Removed `cli_debug()`. Use `cli_debug_backend()` or `cli_debug_restconf()` instead.
* Removed `yspec_free()` - replace with `ys_free()` * Removed `yspec_free()` - replace with `ys_free()`
* `xmldb_get0()`: Added xerr output parameter to * `xmldb_get0()`: Added xerr output parameter to
* `clixon_xml_parse_file()`: Removed `endtag` parameter. * `clixon_xml_parse_file()`: Removed `endtag` parameter.
* Restconf authentication callback (ca_auth) signature changed (again) * Restconf authentication callback (ca_auth) signature changed (again)
* Minor modification to 5.0 change: userp removed. * Minor modification to 5.0 change: userp removed.
@ -1754,7 +1756,7 @@ Users may have to change how they access the system
* Fixed: [Type / Endianism problem in yang_parse_file #128](https://github.com/clicon/clixon/issues/128) * Fixed: [Type / Endianism problem in yang_parse_file #128](https://github.com/clicon/clixon/issues/128)
* Fixed: [(CLI) the description of a used grouping is shown instead of the encapsulating container #124](https://github.com/clicon/clixon/issues/124) * Fixed: [(CLI) the description of a used grouping is shown instead of the encapsulating container #124](https://github.com/clicon/clixon/issues/124)
* Uses/group and augments only copies *schemanodes*. This means reference/description/.. etc are not copied, the original is kept. Also, as a side-effect of the bugfix, a final cardinality sanity check is now made after all yang modifications, not only at the time the file is loaded. * Uses/group and augments only copies *schemanodes*. This means reference/description/.. etc are not copied, the original is kept. Also, as a side-effect of the bugfix, a final cardinality sanity check is now made after all yang modifications, not only at the time the file is loaded.
## 4.6.0 ## 4.6.0
14 August 2020 14 August 2020
@ -1773,7 +1775,7 @@ Thanks Netgate for making this possible.
* `--without-restconf Disable restconf altogether` * `--without-restconf Disable restconf altogether`
* SSL server and client certificates are supported. * SSL server and client certificates are supported.
* SSE Stream notification is not yet supported. * SSE Stream notification is not yet supported.
### API changes on existing protocol/config features ### API changes on existing protocol/config features
Users may have to change how they access the system Users may have to change how they access the system
@ -1845,7 +1847,7 @@ Developers may need to change their code
* `event_poll()` -> `clixon_event_poll()` * `event_poll()` -> `clixon_event_poll()`
* `event_loop()` -> `clixon_event_loop()` * `event_loop()` -> `clixon_event_loop()`
* `event_exit()` -> `clixon_event_exit()` * `event_exit()` -> `clixon_event_exit()`
### Minor changes ### Minor changes
These are new features that did not quite make it to the "Major features" list These are new features that did not quite make it to the "Major features" list
@ -1894,7 +1896,7 @@ optimizations. Note that this version must be run with CLIgen 4.5, it
cannot run with CLIgen 4.4 or earlier. cannot run with CLIgen 4.4 or earlier.
Thanks to everyone at Netgate for making this possible Thanks to everyone at Netgate for making this possible
### Major New features ### Major New features
* NACM RFC8341 datanode read and write paths * NACM RFC8341 datanode read and write paths
@ -1905,7 +1907,7 @@ Thanks to everyone at Netgate for making this possible
* ca_daemon: Called just after a server has "daemonized", ie put in background. * ca_daemon: Called just after a server has "daemonized", ie put in background.
* ca_trans_commit_done: Called when all plugin commits have been done. * ca_trans_commit_done: Called when all plugin commits have been done.
* Note: If you have used "end" callback and using transaction data, you should probably use this instead. * Note: If you have used "end" callback and using transaction data, you should probably use this instead.
### API changes on existing protocol/config features (For users) ### API changes on existing protocol/config features (For users)
* Stricter validation detecting duplicate container or leaf in XML. * Stricter validation detecting duplicate container or leaf in XML.
@ -1926,11 +1928,11 @@ Thanks to everyone at Netgate for making this possible
* Example: `transaction_data_t` * Example: `transaction_data_t`
* `xml_merge()` changed to use 3-value return: 1:OK, 0:Yang failed, -1: Error * `xml_merge()` changed to use 3-value return: 1:OK, 0:Yang failed, -1: Error
* Some function changed to use 3-value return: 1:OK, 0:Yang failed, -1: Error. * Some function changed to use 3-value return: 1:OK, 0:Yang failed, -1: Error.
* Example: `xml_merge()` * Example: `xml_merge()`
* `clixon_netconf_error(category, xerr, msg, arg)` removed first argument -> `clixon_netconf_error(xerr, msg, arg)` * `clixon_netconf_error(category, xerr, msg, arg)` removed first argument -> `clixon_netconf_error(xerr, msg, arg)`
* CLI * CLI
* `clicon_parse()`: Changed signature due to new cligen error and result handling: * `clicon_parse()`: Changed signature due to new cligen error and result handling:
* Removed: `cli_nomatch()` * Removed: `cli_nomatch()`
### Optimzations ### Optimzations
* Optimized namespace prefix checks at xml parse time: using many prefixes slowed down parsing considerably * Optimized namespace prefix checks at xml parse time: using many prefixes slowed down parsing considerably
@ -1975,14 +1977,14 @@ features include optimized search functions and a repair callback.
* Binary search optimization in lists for indexed leafs in all three formats. * Binary search optimization in lists for indexed leafs in all three formats.
* This improves search performance to O(logN) which is drastical improvements for large lists. * This improves search performance to O(logN) which is drastical improvements for large lists.
* You can also register explicit indexes for making binary search (not only list keys) * You can also register explicit indexes for making binary search (not only list keys)
* For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and * For more info, see docs at [paths](https://clixon-docs.readthedocs.io/en/latest/paths.html) and
[search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml) [search](https://clixon-docs.readthedocs.io/en/latest/xml.html#searching-in-xml)
### API changes on existing protocol/config features (You may have have to change how you use Clixon) ### API changes on existing protocol/config features (You may have have to change how you use Clixon)
* In the bbuild system, you dont need to do `make install-include` for installing include files for compiling. This is now included in the actions done by `make install`. * In the bbuild system, you dont need to do `make install-include` for installing include files for compiling. This is now included in the actions done by `make install`.
* State data is now ordered-by system for performance reasons. For example, alphabetically for strings and numeric for integers * State data is now ordered-by system for performance reasons. For example, alphabetically for strings and numeric for integers
* Controlled by compile-time option `STATE_ORDERED_BY_SYSTEM` * Controlled by compile-time option `STATE_ORDERED_BY_SYSTEM`
* Obsolete configuration options present in clixon configuration file will cause clixon application to exit at startup. * Obsolete configuration options present in clixon configuration file will cause clixon application to exit at startup.
* JSON * JSON
* Empty values in JSON has changed to comply to RFC 7951 * Empty values in JSON has changed to comply to RFC 7951
* empty values of yang type `empty` are encoded as: `{"x":[null]}` * empty values of yang type `empty` are encoded as: `{"x":[null]}`
@ -2045,7 +2047,7 @@ features include optimized search functions and a repair callback.
* If fails, exit with error message, eg: `Yang error: Sanity check failed: LIST vsDataContainer lacks key statement which MUST be present (See RFC 7950 Sec 7.8.2)` * If fails, exit with error message, eg: `Yang error: Sanity check failed: LIST vsDataContainer lacks key statement which MUST be present (See RFC 7950 Sec 7.8.2)`
* Can be disabled by setting `CLICON_CLICON_YANG_LIST_CHECK` to `false` * Can be disabled by setting `CLICON_CLICON_YANG_LIST_CHECK` to `false`
* Replaced compile option `VALIDATE_STATE_XML` with runtime option `CLICON_VALIDATE_STATE_XML`. * Replaced compile option `VALIDATE_STATE_XML` with runtime option `CLICON_VALIDATE_STATE_XML`.
* Memory footprint * Memory footprint
* Namespace cache is populated on-demand, see `xml2ns()`. * Namespace cache is populated on-demand, see `xml2ns()`.
* CBUF start level is set to 256 (`CLICON_CLI_BUF_START` option) * CBUF start level is set to 256 (`CLICON_CLI_BUF_START` option)
* Reduced xml child vector default size from 4 to 1 with quadratic growth to 64K then linear * Reduced xml child vector default size from 4 to 1 with quadratic growth to 64K then linear
@ -2059,7 +2061,7 @@ features include optimized search functions and a repair callback.
* C-code restructuring * C-code restructuring
* clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c. * clixon_yang.c partitioned and moved code into clixon_yang_parse_lib.c and clixon_yang_module.c and move back some code from clixon_yang_type.c.
* partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now. * partly to reduce size, but most important to limit code that accesses internal yang structures, only clixon_yang.c does this now.
### Corrected Bugs ### Corrected Bugs
* Fixed: Datastore read on startup got fixed default values. * Fixed: Datastore read on startup got fixed default values.
@ -2076,7 +2078,7 @@ features include optimized search functions and a repair callback.
* Fixed: Pretty-printed XML using prefixes not parsed correctly. * Fixed: Pretty-printed XML using prefixes not parsed correctly.
* eg `<a:x> <y/></a:x>` could lead to errors, wheras (`<x> <y/></x>`) works fine. * eg `<a:x> <y/></a:x>` could lead to errors, wheras (`<x> <y/></x>`) works fine.
* XML namespace merge bug fixed. Example: two xmlns attributes could both survive a merge whereas one should replace the other. * XML namespace merge bug fixed. Example: two xmlns attributes could both survive a merge whereas one should replace the other.
* Compile option `VALIDATE_STATE_XML` introduced in `include/custom.h` to control whether code for state data validation is compiled or not. * Compile option `VALIDATE_STATE_XML` introduced in `include/custom.h` to control whether code for state data validation is compiled or not.
* Fixed: Validation of user state data led to wrong validation, if state relied on config data, eg leafref/must/when etc. * Fixed: Validation of user state data led to wrong validation, if state relied on config data, eg leafref/must/when etc.
* Fixed: No revision in yang module led to errors in validation of state data * Fixed: No revision in yang module led to errors in validation of state data
* Fixed: Leafref validation did not cover case of when the "path" statement is declared within a typedef, only if it was declared in the data part directly under leaf. * Fixed: Leafref validation did not cover case of when the "path" statement is declared within a typedef, only if it was declared in the data part directly under leaf.
@ -2097,7 +2099,7 @@ There were several issues with multiple namespaces with augmented yangs in 4.2 t
* C-API * C-API
* Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)` * Changed `clicon_rpc_generate_error(msg, xerr)` to `clicon_rpc_generate_error(xerr, msg, arg)`
* If you pass NULL as arg it produces the same message as before. * If you pass NULL as arg it produces the same message as before.
* Added namespace-context parameter `nsc` to `xpath_first` and `xpath_vec`, (`xpath_vec_nsc` and * Added namespace-context parameter `nsc` to `xpath_first` and `xpath_vec`, (`xpath_vec_nsc` and
xpath_first_nsc` are removed). xpath_first_nsc` are removed).
* Added clicon_handle as parameter to all `clicon_connect_` functions to get better error message * Added clicon_handle as parameter to all `clicon_connect_` functions to get better error message
* Added nsc parameter to `xmldb_get()` * Added nsc parameter to `xmldb_get()`
@ -2127,7 +2129,6 @@ xpath_first_nsc` are removed).
* [xml_parse_string() is slow for a long XML string #96](https://github.com/clicon/clixon/issues/96) * [xml_parse_string() is slow for a long XML string #96](https://github.com/clicon/clixon/issues/96)
* Mandatory variables can no longer be deleted. * Mandatory variables can no longer be deleted.
* [Add missing includes](https://github.com/clicon/clixon/pulls) * [Add missing includes](https://github.com/clicon/clixon/pulls)
## 4.3.1 ## 4.3.1
2 February 2020 2 February 2020
@ -2136,7 +2137,7 @@ Patch release based on testing by Dave Cornejo, Netgate
### Corrected Bugs ### Corrected Bugs
* XML namespace merge bug fixed. Example: two xmlns attributes could both survive a merge whereas one should replace the other. * XML namespace merge bug fixed. Example: two xmlns attributes could both survive a merge whereas one should replace the other.
* Compile option `VALIDATE_STATE_XML` introduced in `include/custom.h` to control whether code for state data validation is compiled or not. * Compile option `VALIDATE_STATE_XML` introduced in `include/custom.h` to control whether code for state data validation is compiled or not.
* Fixed: Validation of user state data led to wrong validation, if state relied on config data, eg leafref/must/when etc. * Fixed: Validation of user state data led to wrong validation, if state relied on config data, eg leafref/must/when etc.
* Fixed: No revision in yang module led to errors in validation of state data * Fixed: No revision in yang module led to errors in validation of state data
* Fixed: Leafref validation did not cover case of when the "path" statement is declared within a typedef, only if it was declared in the data part directly under leaf. * Fixed: Leafref validation did not cover case of when the "path" statement is declared within a typedef, only if it was declared in the data part directly under leaf.
@ -2150,7 +2151,7 @@ Patch release based on testing by Dave Cornejo, Netgate
* Called on startup after initial XML parsing, but before module-specific upgrades * Called on startup after initial XML parsing, but before module-specific upgrades
* Enabled by definign the `.ca_datastore_upgrade` * Enabled by definign the `.ca_datastore_upgrade`
* [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#general-purpose) * [General-purpose upgrade documentation](https://clixon-docs.readthedocs.io/en/latest/backend.html#general-purpose)
### API changes on existing features (you may need to change your code) ### API changes on existing features (you may need to change your code)
* Session-id CLI functionality delayed: "lazy evaluation" * Session-id CLI functionality delayed: "lazy evaluation"
* C-api: Changed `clicon_session_id_get(clicon_handle h, uint32_t *id)` * C-api: Changed `clicon_session_id_get(clicon_handle h, uint32_t *id)`
@ -2189,7 +2190,7 @@ The main improvement in this release concerns security in terms of privileges an
* `drop_perm` After initialization, drop privileges permanently * `drop_perm` After initialization, drop privileges permanently
* `drop_perm` After initialization, drop privileges temporarily (to a euid) * `drop_perm` After initialization, drop privileges temporarily (to a euid)
* If dropped temporary, you can restore privileges with `restore_priv()` * If dropped temporary, you can restore privileges with `restore_priv()`
* The backend socket has now support of credentials of peer clients * The backend socket has now support of credentials of peer clients
* NACM users are cross-checked with client credentials (cli/netconf/restconf) * NACM users are cross-checked with client credentials (cli/netconf/restconf)
* Only UNIX domain socket supports client credential checks (IP sockets do not). * Only UNIX domain socket supports client credential checks (IP sockets do not).
* Controlled by option CLICON_NACM_CREDENTIALS * Controlled by option CLICON_NACM_CREDENTIALS
@ -2218,7 +2219,7 @@ a="urn:example:a" xmlns:b="urn:example:b"/>`
* Added: CLICON_BACKEND_PRIVILEGES: If and how to drop privileges * Added: CLICON_BACKEND_PRIVILEGES: If and how to drop privileges
* Added: CLICON_NACM_CREDENTIALS: If and how to check backend socket privileges with NACM * Added: CLICON_NACM_CREDENTIALS: If and how to check backend socket privileges with NACM
* Added: CLICON_NACM_RECOVERY_USER: Name of NACM recovery user. * Added: CLICON_NACM_RECOVERY_USER: Name of NACM recovery user.
* Restconf top-level operations GET root resource modified to comply with RFC 8040 Sec 3.1 * Restconf top-level operations GET root resource modified to comply with RFC 8040 Sec 3.1
* non-pretty print remove all spaces, eg `{"operations":{"clixon-example:client-rpc":[null]` * non-pretty print remove all spaces, eg `{"operations":{"clixon-example:client-rpc":[null]`
* Replaced JSON `null` with `[null]` as proper empty JSON leaf/leaf-list encoding. * Replaced JSON `null` with `[null]` as proper empty JSON leaf/leaf-list encoding.
* C-code change * C-code change
@ -2242,7 +2243,7 @@ a="urn:example:a" xmlns:b="urn:example:b"/>`
* XPATH canonical form implemented for NETCONF get and get-config. This means that all callbacks (including state callbacks) will have the prefixes that corresponds to module prefixes. If an xpath have other prefixes (or null as default), they will be translated to canonical form before any callbacks. * XPATH canonical form implemented for NETCONF get and get-config. This means that all callbacks (including state callbacks) will have the prefixes that corresponds to module prefixes. If an xpath have other prefixes (or null as default), they will be translated to canonical form before any callbacks.
* Example of a canonical form: `/a:x/a:y`, then symbols must belong to a yang module with prefix `a`. * Example of a canonical form: `/a:x/a:y`, then symbols must belong to a yang module with prefix `a`.
* FreeBSD modifications: Configure, makefiles and test scripts modification for Freebsd * FreeBSD modifications: Configure, makefiles and test scripts modification for Freebsd
### Corrected Bugs ### Corrected Bugs
* Fixed CLI error messages on wrong cli_set/merge xml-key to eg: * Fixed CLI error messages on wrong cli_set/merge xml-key to eg:
* `Config error: api-path syntax error \"/example:x/m1=%s\": rpc malformed-message List key m1 length mismatch : Invalid argument"` * `Config error: api-path syntax error \"/example:x/m1=%s\": rpc malformed-message List key m1 length mismatch : Invalid argument"`
@ -2263,7 +2264,7 @@ a="urn:example:a" xmlns:b="urn:example:b"/>`
4.1.0 is focussed on RFC 8040 RESTCONF features. Highlights include: 4.1.0 is focussed on RFC 8040 RESTCONF features. Highlights include:
- RFC8040 plain PATCH, - RFC8040 plain PATCH,
- Query parameters: content, depth, insert, position - Query parameters: content, depth, insert, position
- Standard return codes - Standard return codes
### Major New features ### Major New features
* Restconf RFC 8040 increased feature compliance * Restconf RFC 8040 increased feature compliance
@ -2392,7 +2393,7 @@ Olof Hagsand
* Support for multiple patterns as described in RFC7950 Section 9.4.7 * Support for multiple patterns as described in RFC7950 Section 9.4.7
* Support for inverted patterns as described in RFC7950 Section 9.4.6 * Support for inverted patterns as described in RFC7950 Section 9.4.6
* Libxml2 support for full XSD matching as alternative to Posix translation * Libxml2 support for full XSD matching as alternative to Posix translation
* Configure with: `./configure --with-libxml2` * Configure with: `./configure --with-libxml2`
* Set `CLICON_YANG_REGEXP` to libxml2 (default is posix) * Set `CLICON_YANG_REGEXP` to libxml2 (default is posix)
* Note you need to configure cligen as well with `--with-libxml2` * Note you need to configure cligen as well with `--with-libxml2`
* Better compliance with XSD regexps in the default Posix translation regex mode * Better compliance with XSD regexps in the default Posix translation regex mode
@ -2442,7 +2443,7 @@ Olof Hagsand
* Two config options control: * Two config options control:
* CLICON_XML_CHANGELOG enables the yang changelog feature * CLICON_XML_CHANGELOG enables the yang changelog feature
* CLICON_XML_CHANGELOG_FILE where the changelog resides * CLICON_XML_CHANGELOG_FILE where the changelog resides
### API changes on existing features (you may need to change your code) ### API changes on existing features (you may need to change your code)
* RESTCONF strict namespace validation of data in POST and PUT. * RESTCONF strict namespace validation of data in POST and PUT.
@ -2483,7 +2484,7 @@ Olof Hagsand
* All calls to the following functions must be changed: `hash_init`, `hash_free`, `hash_lookup`, `hash_value`, `hash_add`, `hash_del`, `hash_dump`, and `hash_keys`. * All calls to the following functions must be changed: `hash_init`, `hash_free`, `hash_lookup`, `hash_value`, `hash_add`, `hash_del`, `hash_dump`, and `hash_keys`.
* Replaced `CLIXON_DATADIR` with two configurable options defining where Clixon installs Yang files. * Replaced `CLIXON_DATADIR` with two configurable options defining where Clixon installs Yang files.
* use `--with-yang-installdir=DIR` to install Clixon yang files in DIR * use `--with-yang-installdir=DIR` to install Clixon yang files in DIR
* use `--with-std-yang-installdir=DIR` to install standard yang files that Clixon may use in DIR * use `--with-std-yang-installdir=DIR` to install standard yang files that Clixon may use in DIR
* Default is (as before) `/usr/local/share/clixon` * Default is (as before) `/usr/local/share/clixon`
* New clixon-config@2019-06-05.yang revision * New clixon-config@2019-06-05.yang revision
* Added: `CLICON_YANG_REGEXP, CLICON_CLI_TAB_MODE, CLICON_CLI_HIST_FILE, CLICON_CLI_HIST_SIZE, CLICON_XML_CHANGELOG, CLICON_XML_CHANGELOG_FILE`. * Added: `CLICON_YANG_REGEXP, CLICON_CLI_TAB_MODE, CLICON_CLI_HIST_FILE, CLICON_CLI_HIST_SIZE, CLICON_XML_CHANGELOG, CLICON_XML_CHANGELOG_FILE`.
@ -2499,9 +2500,9 @@ Olof Hagsand
* Clixon transaction mechanism has changed which may affect your backend plugin callbacks: * Clixon transaction mechanism has changed which may affect your backend plugin callbacks:
* Validate-only transactions are terminated by an `end` or `abort` callback. Now all started transactions are terminated either by an `end` or `abort` without exceptions * Validate-only transactions are terminated by an `end` or `abort` callback. Now all started transactions are terminated either by an `end` or `abort` without exceptions
* Validate-only transactions used to be terminated by `complete` * Validate-only transactions used to be terminated by `complete`
* If a commit user callback fails, a new `revert` callback will be made to plugins that have made a succesful commit. * If a commit user callback fails, a new `revert` callback will be made to plugins that have made a succesful commit.
* Clixon used to play the (already made) commit callbacks in reverse order * Clixon used to play the (already made) commit callbacks in reverse order
* Many validation functions have changed error parameter from cbuf to xml tree. * Many validation functions have changed error parameter from cbuf to xml tree.
* XML trees are more flexible for utility tools * XML trees are more flexible for utility tools
* If you use these(mostly internal), you need to change the error function: `generic_validate, from_validate_common, xml_yang_validate_all_top, xml_yang_validate_all, xml_yang_validate_add, xml_yang_validate_pprpc, xml_yang_validate_list_key_only` * If you use these(mostly internal), you need to change the error function: `generic_validate, from_validate_common, xml_yang_validate_all_top, xml_yang_validate_all, xml_yang_validate_add, xml_yang_validate_pprpc, xml_yang_validate_list_key_only`
* Datastore cache and xmldb_get() changes: * Datastore cache and xmldb_get() changes:
@ -2510,8 +2511,8 @@ Olof Hagsand
* New suite of API functions enabling zero-copy: `xmldb_get0`. You can still use `xmldb_get()`. The call sequence of zero-copy is (see reference for more info): * New suite of API functions enabling zero-copy: `xmldb_get0`. You can still use `xmldb_get()`. The call sequence of zero-copy is (see reference for more info):
``` ```
xmldb_get0(xh, "running", "/interfaces/interface[name="eth"]", 0, &xt, NULL); xmldb_get0(xh, "running", "/interfaces/interface[name="eth"]", 0, &xt, NULL);
xmldb_get0_clear(h, xt); # Clear tree from default values and flags xmldb_get0_clear(h, xt); # Clear tree from default values and flags
xmldb_get0_free(h, &xt); # Free tree xmldb_get0_free(h, &xt); # Free tree
``` ```
* Clixon config option `CLICON_XMLDB_CACHE` renamed to `CLICON_DATASTORE_CACHE` and changed type from `boolean` to `datastore_cache` * Clixon config option `CLICON_XMLDB_CACHE` renamed to `CLICON_DATASTORE_CACHE` and changed type from `boolean` to `datastore_cache`
* Type `datastore_cache` have values: nocache, cache, or cache-zerocopy * Type `datastore_cache` have values: nocache, cache, or cache-zerocopy
@ -2596,11 +2597,11 @@ Olof Hagsand
* Documented bug [Yang identityref XML encoding is not general #90](https://github.com/clicon/clixon/issues/90) * Documented bug [Yang identityref XML encoding is not general #90](https://github.com/clicon/clixon/issues/90)
* Added new API function `xpath_parse()` to split parsing and xml evaluation. * Added new API function `xpath_parse()` to split parsing and xml evaluation.
* Rewrote `api_path2xpath` to handle namespaces. * Rewrote `api_path2xpath` to handle namespaces.
* `api_path2xml_vec` strict mode check added if list key length mismatch * `api_path2xml_vec` strict mode check added if list key length mismatch
* `startup_extraxml` triggers unnecessary validation * `startup_extraxml` triggers unnecessary validation
* Renamed startup_db_reset -> xmldb_db_reset (its a general function) * Renamed startup_db_reset -> xmldb_db_reset (its a general function)
* In startup_extraxml(), check if reset callbacks or extraxml file actually makes and changes to the tmp db. * In startup_extraxml(), check if reset callbacks or extraxml file actually makes and changes to the tmp db.
* Print CLICON_YANG_DIR and CLICON_FEATURE lists on startup with debug flag * Print CLICON_YANG_DIR and CLICON_FEATURE lists on startup with debug flag
* Extended `util/clixon_util_xml` with yang and validate functionality so it can be used as a stand-alone utility for validating XML/JSON files * Extended `util/clixon_util_xml` with yang and validate functionality so it can be used as a stand-alone utility for validating XML/JSON files
* JSON parse and print improvements * JSON parse and print improvements
* Integrated parsing with namespace translation and yang spec lookup * Integrated parsing with namespace translation and yang spec lookup
@ -2613,10 +2614,10 @@ Olof Hagsand
* Improved submodule implementation (as part of [Yang submodule import prefix restrictions #60](https://github.com/clicon/clixon/issues/60)). * Improved submodule implementation (as part of [Yang submodule import prefix restrictions #60](https://github.com/clicon/clixon/issues/60)).
* Submodules share same namespace as modules, which means that functions looking for symbols under a module were extended to also look in that module's included submodules, also recursively (submodules can include submodules in Yang 1.0). * Submodules share same namespace as modules, which means that functions looking for symbols under a module were extended to also look in that module's included submodules, also recursively (submodules can include submodules in Yang 1.0).
* Submodules are no longer merged with modules in the code. This is necessary to have separate local import prefixes, for example. * Submodules are no longer merged with modules in the code. This is necessary to have separate local import prefixes, for example.
* New function `ys_real_module()` complements `ys_module()`. The latter gets the top module or submodule, whereas the former gets the ultimate module that a submodule belongs to. * New function `ys_real_module()` complements `ys_module()`. The latter gets the top module or submodule, whereas the former gets the ultimate module that a submodule belongs to.
* See [test/test_submodule.sh] * See [test/test_submodule.sh]
* New XMLDB_FORMAT added: `tree`. An experimental record-based tree database for * New XMLDB_FORMAT added: `tree`. An experimental record-based tree database for
direct access of records. direct access of records.
* Netconf error handling modified * Netconf error handling modified
* New option -e added. If set, the netconf client returns -1 on error. * New option -e added. If set, the netconf client returns -1 on error.
* A new minimal "hello world" example has been added * A new minimal "hello world" example has been added
@ -2644,7 +2645,7 @@ Olof Hagsand
* Added specific clixon_suberrno code: XMLPARSE_ERRNO to identify XML parse errors. * Added specific clixon_suberrno code: XMLPARSE_ERRNO to identify XML parse errors.
* Removed all dependency on strverscmp * Removed all dependency on strverscmp
* Added libgen.h for baseline() * Added libgen.h for baseline()
### Corrected Bugs ### Corrected Bugs
* Fixed: Return 404 Not found error if restconf GET does not return requested instance * Fixed: Return 404 Not found error if restconf GET does not return requested instance
@ -2668,14 +2669,14 @@ Olof Hagsand
* [Parent list key is not validated if not provided via RESTCONF #83](https://github.com/clicon/clixon/issues/83), thanks achernavin22. * [Parent list key is not validated if not provided via RESTCONF #83](https://github.com/clicon/clixon/issues/83), thanks achernavin22.
* [Invalid JSON if GET /operations via RESTCONF #82](https://github.com/clicon/clixon/issues/82), thanks achernavin22 * [Invalid JSON if GET /operations via RESTCONF #82](https://github.com/clicon/clixon/issues/82), thanks achernavin22
* List ordering bug - lists with ints as keys behaved wrongly and slow. * List ordering bug - lists with ints as keys behaved wrongly and slow.
* NACM read default rule did not work properly if nacm was enabled AND no groups were defined * NACM read default rule did not work properly if nacm was enabled AND no groups were defined
* Re-inserted `cli_output_reset` for what was erroneuos thought to be an obsolete function * Re-inserted `cli_output_reset` for what was erroneuos thought to be an obsolete function
* See in 3.9.0 minor changes: Replaced all calls to (obsolete) `cli_output` with `fprintf` * See in 3.9.0 minor changes: Replaced all calls to (obsolete) `cli_output` with `fprintf`
* Allowed Yang extended Xpath functions (syntax only): * Allowed Yang extended Xpath functions (syntax only):
* re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set * re-match, deref, derived-from, derived-from-or-self, enum-value, bit-is-set
* XSD regular expression handling of dash(`-`) * XSD regular expression handling of dash(`-`)
*: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`. *: Translate XDS `[xxx\-yyy]` to POSIX `[xxxyyy-]`.
* YANG Anydata treated same as Anyxml * YANG Anydata treated same as Anyxml
* Bugfix: [Nodes from more than one of the choice's branches exist at the same time](https://github.com/clicon/clixon/issues/81) * Bugfix: [Nodes from more than one of the choice's branches exist at the same time](https://github.com/clicon/clixon/issues/81)
* Note it may still be possible to load a file with multiple choice elements via netconf, but it will not pass validate. * Note it may still be possible to load a file with multiple choice elements via netconf, but it will not pass validate.
* Bugfix: Default NACM policies applied even if NACM is disabled * Bugfix: Default NACM policies applied even if NACM is disabled
@ -2717,13 +2718,13 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Netconf `edit-config xpath select` statement, and all xpath statements * Netconf `edit-config xpath select` statement, and all xpath statements
* Notifications * Notifications
* CLI syntax (ie generated commands) * CLI syntax (ie generated commands)
* The default namespace is ietf-netconf base syntax with uri: `urn:ietf:params:xml:ns:netconf:base:1.0` and need not be explicitly given. * The default namespace is ietf-netconf base syntax with uri: `urn:ietf:params:xml:ns:netconf:base:1.0` and need not be explicitly given.
* The following example shows changes in netconf and restconf: * The following example shows changes in netconf and restconf:
* Accepted pre 3.9 (now not valid): * Accepted pre 3.9 (now not valid):
``` ```
<rpc> <rpc>
<my-own-method/> <my-own-method/>
</rpc> </rpc>
<rpc-reply> <rpc-reply>
<route> <route>
<address-family>ipv4</address-family> <address-family>ipv4</address-family>
@ -2793,7 +2794,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* [Clixon base container](docker/base). * [Clixon base container](docker/base).
* [Clixon system and test container](docker/system) used in Travis CI. * [Clixon system and test container](docker/system) used in Travis CI.
* See also: [Clixon docker hub](https://hub.docker.com/u/clixon) * See also: [Clixon docker hub](https://hub.docker.com/u/clixon)
### API changes on existing features (you may need to change your code) ### API changes on existing features (you may need to change your code)
* XML namespace handling is corrected (see [#major-changes]) * XML namespace handling is corrected (see [#major-changes])
* For backward compatibility set config option CLICON_XML_NS_LOOSE * For backward compatibility set config option CLICON_XML_NS_LOOSE
@ -2875,7 +2876,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Some restconf error messages contained `rpc-reply` or `rpc-error` which have now been removed. * Some restconf error messages contained `rpc-reply` or `rpc-error` which have now been removed.
* getopt return value changed from char to int (https://github.com/clicon/clixon/issues/58) * getopt return value changed from char to int (https://github.com/clicon/clixon/issues/58)
* Netconf/Restconf RPC extra input arguments are ignored (https://github.com/clicon/clixon/issues/47) * Netconf/Restconf RPC extra input arguments are ignored (https://github.com/clicon/clixon/issues/47)
## 3.8.0 ## 3.8.0
6 Nov 2018 6 Nov 2018
@ -2891,7 +2892,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
``` ```
* logical combination of features not implemented, eg if-feature "not foo or bar and baz"; * logical combination of features not implemented, eg if-feature "not foo or bar and baz";
* ietf-netconf yang module added with candidate, validate, startup and xpath features enabled. * ietf-netconf yang module added with candidate, validate, startup and xpath features enabled.
* YANG module library * YANG module library
* YANG modules according to RFC 7895 and implemented by ietf-yang-library.yang * YANG modules according to RFC 7895 and implemented by ietf-yang-library.yang
* Enabled by configuration option CLICON_MODULE_LIBRARY_RFC7895 - enabled by default * Enabled by configuration option CLICON_MODULE_LIBRARY_RFC7895 - enabled by default
* RFC 7895 defines a module-set-id. Configure option CLICON_MODULE_SET_ID is set and changed when modules change. * RFC 7895 defines a module-set-id. Configure option CLICON_MODULE_SET_ID is set and changed when modules change.
@ -2924,7 +2925,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Old streams API which needs to be modified: * Old streams API which needs to be modified:
* clicon_log_register_callback() removed * clicon_log_register_callback() removed
* subscription_add() --> stream_add() * subscription_add() --> stream_add()
* stream_cb_add() --> stream_ss_add() * stream_cb_add() --> stream_ss_add()
* stream_cb_delete() --> stream_ss_delete() * stream_cb_delete() --> stream_ss_delete()
* backend_notify() and backend_notify_xml() - use stream_notify() instead * backend_notify() and backend_notify_xml() - use stream_notify() instead
* Example uses "NETCONF" stream instead of "ROUTING" * Example uses "NETCONF" stream instead of "ROUTING"
@ -2936,12 +2937,12 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* clixon_netconf -S is obsolete. Use `clixon_netconf -l s` instead. * clixon_netconf -S is obsolete. Use `clixon_netconf -l s` instead.
* Comply to RFC 8040 3.5.3.1 rule: api-identifier = [module-name ":"] identifier * Comply to RFC 8040 3.5.3.1 rule: api-identifier = [module-name ":"] identifier
* The "module-name" was a no-op before. * The "module-name" was a no-op before.
* This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/foobar:modules-state * This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/foobar:modules-state
* Generilized top-level yang parsing functions * Generilized top-level yang parsing functions
* Clarified semantics of main yang module: * Clarified semantics of main yang module:
* Command-line option -y MUST specify a filename * Command-line option -y MUST specify a filename
* Configure option CLICON_YANG_MODULE_MAIN MUST specify a module name * Configure option CLICON_YANG_MODULE_MAIN MUST specify a module name
* yang_parse() changed to take either filename or module name and revision. * yang_parse() changed to take either filename or module name and revision.
* Removed clicon_dbspec_name() and clicon_dbspec_name_set(). * Removed clicon_dbspec_name() and clicon_dbspec_name_set().
* Replace calls to yang_spec_main() with yang_spec_parse_module(). See for * Replace calls to yang_spec_main() with yang_spec_parse_module(). See for
example backend_main() and others if you need details. example backend_main() and others if you need details.
@ -2951,7 +2952,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Renamed test/test_auth*.sh tests to test/test_nacm*.sh * Renamed test/test_auth*.sh tests to test/test_nacm*.sh
* YANG keywords "action" and "belongs-to" implemented by syntactically by parser (but not proper semantics). * YANG keywords "action" and "belongs-to" implemented by syntactically by parser (but not proper semantics).
* clixon-config YAML file has new revision: 2018-10-21. * clixon-config YAML file has new revision: 2018-10-21.
* Allow new lines in CLI prompts * Allow new lines in CLI prompts
* uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters * uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters
* Added Configure option CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option). * Added Configure option CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option).
* New function: clicon_conf_xml() returns configuration tree * New function: clicon_conf_xml() returns configuration tree
@ -2964,17 +2965,17 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Single quotes for XML attributes https://github.com/clicon/clixon/issues/51 * Single quotes for XML attributes https://github.com/clicon/clixon/issues/51
* Thanks @SCadilhac * Thanks @SCadilhac
* Fixed https://github.com/clicon/clixon/issues/46 Issue with empty values in leaf-list * Fixed https://github.com/clicon/clixon/issues/46 Issue with empty values in leaf-list
* Thanks @achernavin22 * Thanks @achernavin22
* Identity without any identityref:s caused SEGV * Identity without any identityref:s caused SEGV
* Memory error in backend transaction revert * Memory error in backend transaction revert
* Set dir /www-data with www-data as owner, see https://github.com/clicon/clixon/issues/37 * Set dir /www-data with www-data as owner, see https://github.com/clicon/clixon/issues/37
### Known issues ### Known issues
* Netconf/Restconf RPC extra input arguments are ignored * Netconf/Restconf RPC extra input arguments are ignored
* https://github.com/clicon/clixon/issues/47 * https://github.com/clicon/clixon/issues/47
* Yang sub-command cardinality not checked. * Yang sub-command cardinality not checked.
* https://github.com/clicon/clixon/issues/48 * https://github.com/clicon/clixon/issues/48
* Issue with bare axis names (XPath 1.0) * Issue with bare axis names (XPath 1.0)
* https://github.com/clicon/clixon/issues/54 * https://github.com/clicon/clixon/issues/54
* Top-level Yang symbol cannot be called "config" in any imported yang file. * Top-level Yang symbol cannot be called "config" in any imported yang file.
* datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanity. * datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanity.
@ -2999,7 +3000,7 @@ Thanks for all bug reports, feature requests and support! Thanks to [Netgate](ht
* Full suport of boolean constraints for "when"/"must", not only nodesets. * Full suport of boolean constraints for "when"/"must", not only nodesets.
* See also API changes below. * See also API changes below.
* CDATA XML support (patch by David Cornejo, Netgate) * CDATA XML support (patch by David Cornejo, Netgate)
* Encode and decode (parsing) support * Encode and decode (parsing) support
* Added cligen variable translation. Useful for processing input such as hashing, encryption. * Added cligen variable translation. Useful for processing input such as hashing, encryption.
* More info in example and FAQ. * More info in example and FAQ.
* Example: * Example:
@ -3067,7 +3068,7 @@ translate {
* The example contains a more elaborate example. * The example contains a more elaborate example.
* Thanks ngashok for request, see https://github.com/clicon/clixon/issues/24 * Thanks ngashok for request, see https://github.com/clicon/clixon/issues/24
* Added XML namespace (xmlns) validation * Added XML namespace (xmlns) validation
* for eg <a xmlns:x="uri"><x:b/></a> * for eg <a xmlns:x="uri"><x:b/></a>
* ./configure extended with --enable-debug flag * ./configure extended with --enable-debug flag
* CFLAGS=-g ./configure deprecated * CFLAGS=-g ./configure deprecated
@ -3114,7 +3115,7 @@ translate {
* CLICON_FIND_PLUGIN * CLICON_FIND_PLUGIN
* clicon_valcb() * clicon_valcb()
* CLIXON_BACKEND_SYSDIR * CLIXON_BACKEND_SYSDIR
* CLIXON_CLI_SYSDIR * CLIXON_CLI_SYSDIR
* CLICON_MASTER_PLUGIN config variable * CLICON_MASTER_PLUGIN config variable
* Example of migrating a backend plugin module: * Example of migrating a backend plugin module:
* Add all callbacks in a clixon_plugin_api struct * Add all callbacks in a clixon_plugin_api struct
@ -3178,7 +3179,7 @@ clixon_plugin_api *clixon_plugin_init(clicon_handle h)
* Experimental: Added CLICON_TRANSACTION_MOD configuration option. If set, * Experimental: Added CLICON_TRANSACTION_MOD configuration option. If set,
modifications in validation and commit callbacks are written back modifications in validation and commit callbacks are written back
into the datastore. Requested by Stephen Jones, Netgate. into the datastore. Requested by Stephen Jones, Netgate.
* Invalid key to api_path2xml gives warning instead of error and quit. * Invalid key to api_path2xml gives warning instead of error and quit.
* Added restconf/operations get, see RFC8040 Sec 3.3.2: * Added restconf/operations get, see RFC8040 Sec 3.3.2:
* yang_find_topnode() and api_path2xml() schemanode parameter replaced with yang_class. Replace as follows: 0 -> YC_DATANODE, 1 -> YC_SCHEMANODE * yang_find_topnode() and api_path2xml() schemanode parameter replaced with yang_class. Replace as follows: 0 -> YC_DATANODE, 1 -> YC_SCHEMANODE
@ -3198,7 +3199,7 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
* Showing syntax using CLI commands was broekn and is fixed. * Showing syntax using CLI commands was broekn and is fixed.
* Fixed issue https://github.com/clicon/clixon/issues/18 RPC response issues reported by Stephen Jones at Netgate * Fixed issue https://github.com/clicon/clixon/issues/18 RPC response issues reported by Stephen Jones at Netgate
* Fixed issue https://github.com/clicon/clixon/issues/17 special character in strings can break RPCs reported by David Cornejo at Netgate. * Fixed issue https://github.com/clicon/clixon/issues/17 special character in strings can break RPCs reported by David Cornejo at Netgate.
* This was a large rewrite of XML parsing and output due to CharData not correctly encoded according to https://www.w3.org/TR/2008/REC-xml-20081126. * This was a large rewrite of XML parsing and output due to CharData not correctly encoded according to https://www.w3.org/TR/2008/REC-xml-20081126.
* Fixed three-key list entry problem (reported by jdl@netgate) * Fixed three-key list entry problem (reported by jdl@netgate)
* Translate xml->json \n correctly * Translate xml->json \n correctly
* Fix issue: https://github.com/clicon/clixon/issues/15 Replace whole config * Fix issue: https://github.com/clicon/clixon/issues/15 Replace whole config
@ -3216,11 +3217,11 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
### Major changes: ### Major changes:
* Major Restconf feature update to comply to RFC 8040. Thanks Stephen Jones of Netgate for getting right. * Major Restconf feature update to comply to RFC 8040. Thanks Stephen Jones of Netgate for getting right.
* GET: Always return object referenced (and nothing else). ie, GET /restconf/data/X returns X. * GET: Always return object referenced (and nothing else). ie, GET /restconf/data/X returns X.
* GET Added support for the following resources: Well-known, top-level resource, and yang library version, * GET Added support for the following resources: Well-known, top-level resource, and yang library version,
* GET Single element JSON lists use {list:[element]}, not {list:element}. * GET Single element JSON lists use {list:[element]}, not {list:element}.
* PUT Whole datastore * PUT Whole datastore
### Minor changes: ### Minor changes:
* Changed signature of plugin_credentials() restconf callback. Added a "user" parameter. To enable authentication and in preparation for access control a la RFC 6536. * Changed signature of plugin_credentials() restconf callback. Added a "user" parameter. To enable authentication and in preparation for access control a la RFC 6536.
@ -3229,7 +3230,7 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
* `configure --with-startup-compat`. Configure option CLICON_USE_STARTUP_CONFIG is also obsoleted. * `configure --with-startup-compat`. Configure option CLICON_USE_STARTUP_CONFIG is also obsoleted.
* `configure --with-config-compat`. The template clicon.conf.cpp files are also removed. * `configure --with-config-compat`. The template clicon.conf.cpp files are also removed.
* `configure --with-xml-compat` * `configure --with-xml-compat`
* New configuration option: CLICON_RESTCONF_PRETTY. Default true. Set to false to get more compact Restconf output. * New configuration option: CLICON_RESTCONF_PRETTY. Default true. Set to false to get more compact Restconf output.
* Default configure file handling generalized by Renato Botelho/Matt Smith. Config file FILE is selected in the following priority order: * Default configure file handling generalized by Renato Botelho/Matt Smith. Config file FILE is selected in the following priority order:
@ -3238,12 +3239,12 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
* Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml * Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml
* Provide --sysconfig=<dir> when configuring then FILE is <dir>/etc/clixon.xml * Provide --sysconfig=<dir> when configuring then FILE is <dir>/etc/clixon.xml
* FILE is /usr/local/etc/clixon.xml * FILE is /usr/local/etc/clixon.xml
### Corrected Bugs ### Corrected Bugs
* yang max keyword was not supported for string type. Corrected by setting "max" to MAXPATHLEN * yang max keyword was not supported for string type. Corrected by setting "max" to MAXPATHLEN
* Corrected "No yang spec" printed on tty when using leafref in CLI. * Corrected "No yang spec" printed on tty when using leafref in CLI.
* Fixed error in xml2cvec. If a (for example) int8 value has range error (eg 1000), it was treated as an error and the program terminated. Now this is just logged and skipped. Reported by Fredrik Pettai. * Fixed error in xml2cvec. If a (for example) int8 value has range error (eg 1000), it was treated as an error and the program terminated. Now this is just logged and skipped. Reported by Fredrik Pettai.
### Known issues ### Known issues
## 3.4.0 ## 3.4.0
@ -3255,7 +3256,7 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
* Added yang ordered-by user. The default (ordered-by system) will now sort lists and leaf-lists alphabetically to increase search performance. Note that this may change outputs. * Added yang ordered-by user. The default (ordered-by system) will now sort lists and leaf-lists alphabetically to increase search performance. Note that this may change outputs.
* If you need legacy order, either set CLICON_XML_SORT to false, or set that list to "ordered-by user". * If you need legacy order, either set CLICON_XML_SORT to false, or set that list to "ordered-by user".
* This replaces XML hash experimental code, ie xml_child_hash variables and all xmlv_hash_ functions have been removed. * This replaces XML hash experimental code, ie xml_child_hash variables and all xmlv_hash_ functions have been removed.
* Implementation detail: Cached keys are stored in in yang Y_LIST nodes as cligen vector, see ys_populate_list() * Implementation detail: Cached keys are stored in in yang Y_LIST nodes as cligen vector, see ys_populate_list()
* Datastore cache introduced: cache XML tree in memory for faster get access. * Datastore cache introduced: cache XML tree in memory for faster get access.
* Reads are cached. Writes are written to disk. * Reads are cached. Writes are written to disk.
@ -3276,7 +3277,7 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
``` ```
configure --with-xml-compat configure --with-xml-compat
``` ```
### Minor changes: ### Minor changes:
* Better semantic versioning, eg MAJOR/MINOR/PATCH, where increment in PATCH does not change API. * Better semantic versioning, eg MAJOR/MINOR/PATCH, where increment in PATCH does not change API.
* Added CLICON_XMLDB_PRETTY option. If set to false, XML database files will be more compact. * Added CLICON_XMLDB_PRETTY option. If set to false, XML database files will be more compact.
@ -3289,11 +3290,11 @@ enables saved files to be used as datastore without any editing. Thanks Matt, Ne
* Fixed bug that deletes running on startup if backup started with -m running. * Fixed bug that deletes running on startup if backup started with -m running.
When clixon starts again, running is lost. When clixon starts again, running is lost.
The error was that the running (or startup) configuration may fail when The error was that the running (or startup) configuration may fail when
clixon backend starts. clixon backend starts.
The fix now makes a copy of running and copies it back on failure. The fix now makes a copy of running and copies it back on failure.
* datastore/keyvalue/Makefile was left behind on make distclean. Fixed by conditional configure. Thanks renato@netgate.com. * datastore/keyvalue/Makefile was left behind on make distclean. Fixed by conditional configure. Thanks renato@netgate.com.
* Escape " in JSON names and strings and values * Escape " in JSON names and strings and values
### Known issues ### Known issues
* Please use text datastore, key-value datastore no up-to-date * Please use text datastore, key-value datastore no up-to-date
@ -3323,13 +3324,13 @@ SUNET for support, requests, debugging, bugfixes and proposed solutions.
... ...
</config> </config>
``` ```
* Simplified backend daemon startup modes. * Simplified backend daemon startup modes.
* The flags -IRCr are replaced with command-line option -s <mode> * The flags -IRCr are replaced with command-line option -s <mode>
* You use the -s to select the mode. Example: `clixon_backend -s running` * You use the -s to select the mode. Example: `clixon_backend -s running`
* You may also add a default method in the configuration file: `<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>` * You may also add a default method in the configuration file: `<CLICON_STARTUP_MODE>init</CLICON_STARTUP_MODE>`
* The configuration option CLICON_USE_STARTUP_CONFIG is obsolete * The configuration option CLICON_USE_STARTUP_CONFIG is obsolete
* Command-ine option `-I` is replaced with `-s init` * Command-ine option `-I` is replaced with `-s init`
* `-CIr` is replaced with `-s running` * `-CIr` is replaced with `-s running`
* Use `-s none` if you request no action on startu * Use `-s none` if you request no action on startu
* Backward compatibility is enabled by: * Backward compatibility is enabled by:
@ -3356,7 +3357,7 @@ plugin. The example application shows how.
* Restconf: http cookie sent as attribute in rpc restconf_post operations to backend as "id" XML attribute. * Restconf: http cookie sent as attribute in rpc restconf_post operations to backend as "id" XML attribute.
* Added option CLICON_CLISPEC_FILE as complement to CLICON_CLISPEC_DIR to * Added option CLICON_CLISPEC_FILE as complement to CLICON_CLISPEC_DIR to
specify single CLI specification file, not only directory containing files. specify single CLI specification file, not only directory containing files.
* Replaced the following cli_ functions with their original cligen_functions: * Replaced the following cli_ functions with their original cligen_functions:
cli_exiting, cli_set_exiting, cli_comment, cli_exiting, cli_set_exiting, cli_comment,
cli_set_comment, cli_tree_add, cli_tree_active, cli_set_comment, cli_tree_add, cli_tree_active,
@ -3391,24 +3392,24 @@ Aug 27 2017
* leafref occuring within lists: cli expansion does not work * leafref occuring within lists: cli expansion does not work
### Major changes: ### Major changes:
* Added support for YANG anyxml. * Added support for YANG anyxml.
* Changed top-level netconf get-config and get to return `<data>` instead of `<data><config>` to comply to the RFC. * Changed top-level netconf get-config and get to return `<data>` instead of `<data><config>` to comply to the RFC.
* If you use direct netconf get or get-config calls, you may need to handle the return XML differently. * If you use direct netconf get or get-config calls, you may need to handle the return XML differently.
* RESTCONF and CLI is not affected. * RESTCONF and CLI is not affected.
* Example: * Example:
``` ```
Query: Query:
<rpc><get/></rpc> <rpc><get/></rpc>
New reply: New reply:
<rpc-reply> <rpc-reply>
<data> <data>
<a/> # Example data model <a/> # Example data model
</data> </data>
</rpc-reply> </rpc-reply>
Old reply: Old reply:
<rpc-reply> <rpc-reply>
<data> <data>
<config> # Removed <config> # Removed
@ -3420,14 +3421,14 @@ Aug 27 2017
* Added support for yang presence and no-presence containers. Previous default was "presence". * Added support for yang presence and no-presence containers. Previous default was "presence".
* Empty containers will be removed unless you have used the "presence" yang declaration. * Empty containers will be removed unless you have used the "presence" yang declaration.
* Example YANG without presence: * Example YANG without presence:
``` ```
container container
nopresence { nopresence {
leaf j { leaf j {
type string; type string;
} }
} }
``` ```
@ -3459,8 +3460,8 @@ If you submit "nopresence" without a leaf, it will automatically be removed:
curl -sS -X POST -d {"input":{"name":"hello"}} http://localhost/restconf/operations/myroute' curl -sS -X POST -d {"input":{"name":"hello"}} http://localhost/restconf/operations/myroute'
``` ```
* Enhanced leafref functionality: * Enhanced leafref functionality:
* Validation for leafref forward and backward references; * Validation for leafref forward and backward references;
* CLI completion for generated cli leafrefs for both absolute and relative paths. * CLI completion for generated cli leafrefs for both absolute and relative paths.
* Example, relative path: * Example, relative path:
@ -3471,7 +3472,7 @@ If you submit "nopresence" without a leaf, it will automatically be removed:
} }
} }
``` ```
* Added state data: Netconf `<get>` operation, new backend plugin callback: "plugin_statedata()" for retreiving state data. * Added state data: Netconf `<get>` operation, new backend plugin callback: "plugin_statedata()" for retreiving state data.
* You can use netconf: `<rpc><get/></rpc>` and it will return both config and state data. * You can use netconf: `<rpc><get/></rpc>` and it will return both config and state data.
* Restconf GET will return state data also, if defined. * Restconf GET will return state data also, if defined.
@ -3492,7 +3493,7 @@ If you submit "nopresence" without a leaf, it will automatically be removed:
* Generic map_str2int generic mapping tables * Generic map_str2int generic mapping tables
* Removed vector return values from xmldb_get() * Removed vector return values from xmldb_get()
* Generalized yang type resolution to all included (sub)modules not just the topmost * Generalized yang type resolution to all included (sub)modules not just the topmost
## 3.3.1 ## 3.3.1
June 7 2017 June 7 2017
@ -3501,9 +3502,9 @@ June 7 2017
* Removed non-standard api_path extension from the internal netconf protocol so that the internal netconf is now fully standard. * Removed non-standard api_path extension from the internal netconf protocol so that the internal netconf is now fully standard.
* Strings in xmldb_put not properly encoded, eg eth/0 became eth.00000 * Strings in xmldb_put not properly encoded, eg eth/0 became eth.00000
## 3.3.0 (May 2017) ## 3.3.0 (May 2017)
* Datastore text module is now default. * Datastore text module is now default.
* Refined netconf "none" semantics in tests and text datastore * Refined netconf "none" semantics in tests and text datastore
@ -3522,7 +3523,7 @@ June 7 2017
* Created xmldb plugin api * Created xmldb plugin api
Moved qdbm, chunk and xmldb to datastore keyvalue directories Moved qdbm, chunk and xmldb to datastore keyvalue directories
Removed all other clixon dependency on chunk code Removed all other clixon dependency on chunk code
* cli_copy_config added as generic cli command * cli_copy_config added as generic cli command
* cli_show_config added as generic cli command * cli_show_config added as generic cli command
Replace all show_confv*() and show_conf*() with cli_show_config() Replace all show_confv*() and show_conf*() with cli_show_config()
@ -3534,14 +3535,14 @@ June 7 2017
* Many clicon special string functions have been removed * Many clicon special string functions have been removed
* The netconf support has been extended with lock/unlock * The netconf support has been extended with lock/unlock
* clicon_rpc_call() has been removed and should be replaced by extending the * clicon_rpc_call() has been removed and should be replaced by extending the
internal netconf protocol. internal netconf protocol.
See downcall() function in example/routing_cli.c and See downcall() function in example/routing_cli.c and
routing_downcall() in example/routing_backend.c routing_downcall() in example/routing_backend.c
* Replace clicon_rpc_xmlput with clicon_rpc_edit_config * Replace clicon_rpc_xmlput with clicon_rpc_edit_config
* Removed xmldb daemon. All xmldb acceses is made backend daemon. * Removed xmldb daemon. All xmldb acceses is made backend daemon.
No direct accesses by clients to xmldb API. No direct accesses by clients to xmldb API.
Instead use the rpc calls in clixon_proto_client.[ch] Instead use the rpc calls in clixon_proto_client.[ch]
In clients (eg cli/netconf) replace xmldb_get() in client code with In clients (eg cli/netconf) replace xmldb_get() in client code with
clicon_rpc_get_config(). clicon_rpc_get_config().
If you use the vector arguments of xmldb_get(), replace as follows: If you use the vector arguments of xmldb_get(), replace as follows:
xmldb_get(h, db, api_path, &xt, &xvec, &xlen); xmldb_get(h, db, api_path, &xt, &xvec, &xlen);
@ -3551,9 +3552,9 @@ June 7 2017
* clicon_rpc_change() is replaced with clicon_rpc_edit_config(). * clicon_rpc_change() is replaced with clicon_rpc_edit_config().
Note modify argument 5: Note modify argument 5:
clicon_rpc_change(h, db, op, apipath, "value") clicon_rpc_change(h, db, op, apipath, "value")
to: to:
clicon_rpc_edit_config(h, db, op, apipath, `"<config>value</config>"`) clicon_rpc_edit_config(h, db, op, apipath, `"<config>value</config>"`)
* xmdlb_put_xkey() and xmldb_put_tree() have been folded into xmldb_put() * xmdlb_put_xkey() and xmldb_put_tree() have been folded into xmldb_put()
Replace xmldb_put_xkey with xmldb_put as follows: Replace xmldb_put_xkey with xmldb_put as follows:
@ -3574,7 +3575,7 @@ June 7 2017
* Use restconf format for internal xmldb keys. Eg /a/b=3,4 * Use restconf format for internal xmldb keys. Eg /a/b=3,4
* List keys with special characters RFC 3986 encoded. * List keys with special characters RFC 3986 encoded.
* Replaced cli expand functions with single to multiple args * Replaced cli expand functions with single to multiple args
This change is _not_ backward compatible This change is _not_ backward compatible
@ -3583,7 +3584,7 @@ June 7 2017
* Replaced cli callback functions with single arg to multiple args * Replaced cli callback functions with single arg to multiple args
This change is _not_ backward compatible. This change is _not_ backward compatible.
You are affected if you You are affected if you
(1) use system callbacks (i.e. in clixon_cli_api.h) (1) use system callbacks (i.e. in clixon_cli_api.h)
(2) write your own cli callbacks (2) write your own cli callbacks
@ -3602,18 +3603,18 @@ June 7 2017
``` ```
and rewrite the code to handle argv instead of arg. and rewrite the code to handle argv instead of arg.
These are the system functions affected: These are the system functions affected:
cli_set, cli_merge, cli_del, cli_debug_backend, cli_set_mode, cli_set, cli_merge, cli_del, cli_debug_backend, cli_set_mode,
cli_start_shell, cli_quit, cli_commit, cli_validate, compare_dbs, cli_start_shell, cli_quit, cli_commit, cli_validate, compare_dbs,
load_config_file, save_config_file, delete_all, discard_changes, cli_notify, load_config_file, save_config_file, delete_all, discard_changes, cli_notify,
show_yang, show_conf_xpath show_yang, show_conf_xpath
* Added --with-cligen and --with-qdbm configure options * Added --with-cligen and --with-qdbm configure options
* Added union type check for non-cli (eg xml) input * Added union type check for non-cli (eg xml) input
* Empty yang type. Relaxed yang types for unions, eg two strings with different length. * Empty yang type. Relaxed yang types for unions, eg two strings with different length.
## (Dec 2016) ## (Dec 2016)
* Dual license: both GPLv3 and APLv2 * Dual license: both GPLv3 and APLv2
## (Feb 2016) ## (Feb 2016)
* Forked new clixon repository from clicon * Forked new clixon repository from clicon

38
Makefile.in
View file

@ -59,11 +59,11 @@ SUBDIRS2 = apps etc yang # without include lib for circular dependency
SUBDIRS= $(SUBDIRS1) $(SUBDIRS2) SUBDIRS= $(SUBDIRS1) $(SUBDIRS2)
.PHONY: doc example install-example clean-example all clean depend $(SUBDIRS) \ .PHONY: doc example install-example clean-example all clean depend $(SUBDIRS) \
install loc TAGS .config.status docker test util checkroot mrproper \ install loc TAGS .config.status docker test checkroot mrproper \
checkinstall warnroot install-util clean-util checkinstall warnroot
all: $(SUBDIRS2) warnroot all: $(SUBDIRS2) warnroot
@echo "\e[32mAfter 'make install' as euid root, build example app and test utils: 'make example'\e[0m" @echo "\e[32mAfter 'make install' as euid root, build example app: 'make example'\e[0m"
checkroot: checkroot:
@if command -v id &> /dev/null; then \ @if command -v id &> /dev/null; then \
@ -84,8 +84,6 @@ checkinstall:
echo "\e[31mclixon must be installed first to build this target. "\ echo "\e[31mclixon must be installed first to build this target. "\
"Run 'make'. Then run 'make install' as root.\e[0m"; exit 1; fi; "Run 'make'. Then run 'make install' as root.\e[0m"; exit 1; fi;
util: apps
# May cause circular include->include,lib # May cause circular include->include,lib
$(SUBDIRS2): $(SUBDIRS1) # Cannot build app before lib (for parallel make -j) $(SUBDIRS2): $(SUBDIRS1) # Cannot build app before lib (for parallel make -j)
(cd $@ && $(MAKE) $(MFLAGS) all) (cd $@ && $(MAKE) $(MFLAGS) all)
@ -106,37 +104,25 @@ install: checkroot
install-include: install-include:
for i in $(SUBDIRS) doc; \ for i in $(SUBDIRS) doc; \
do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done; do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done;
@echo "\e[32mTo install example app and test utils: make install-example\e[0m" @echo "\e[32mTo install example app: make install-example\e[0m"
uninstall: checkroot uninstall: checkroot
for i in $(SUBDIRS) doc example util docker; \ for i in $(SUBDIRS) doc example docker; \
do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done; do (cd $$i && $(MAKE) $(MFLAGS) $@)||exit 1; done;
doc: warnroot doc: warnroot
cd $@; $(MAKE) $(MFLAGS) $@ cd $@; $(MAKE) $(MFLAGS) $@
util:
cd $@; $(MAKE) $(MFLAGS)
clean-util:
cd util; $(MAKE) $(MFLAGS) clean
install-util: checkroot install-include
cd util; $(MAKE) $(MFLAGS) install
uninstall-util:
cd util; $(MAKE) $(MFLAGS) uninstall
clean-example: clean-example:
for i in example util; \ for i in example; \
do (cd $$i && $(MAKE) $(MFLAGS) clean) || exit 1; done; do (cd $$i && $(MAKE) $(MFLAGS) clean) || exit 1; done;
install-example: checkroot install-example: checkroot
for i in example util; \ for i in example; \
do (cd $$i && $(MAKE) $(MFLAGS) install) || exit 1; done; do (cd $$i && $(MAKE) $(MFLAGS) install) || exit 1; done;
uninstall-example: checkroot uninstall-example: checkroot
for i in example util; \ for i in example; \
do (cd $$i && $(MAKE) $(MFLAGS) uninstall) || exit 1; done; do (cd $$i && $(MAKE) $(MFLAGS) uninstall) || exit 1; done;
config.status: configure config.status: configure
@ -146,23 +132,23 @@ configure: configure.ac
cd $(srcdir) && autoconf cd $(srcdir) && autoconf
clean: clean:
for i in $(SUBDIRS) doc example util docker; \ for i in $(SUBDIRS) doc example docker; \
do (cd $$i && $(MAKE) $(MFLAGS) $@); done; do (cd $$i && $(MAKE) $(MFLAGS) $@); done;
rm -f *.gcov test/*.gcov rm -f *.gcov test/*.gcov
# Uninstall and clean all the targets used for testing, but without cloning or # Uninstall and clean all the targets used for testing, but without cloning or
# checking-out from git. Provides a reliabily clean slate for testing changes # checking-out from git. Provides a reliabily clean slate for testing changes
# before commit. # before commit.
mrproper: uninstall uninstall-example uninstall-util clean clean-example clean-util mrproper: uninstall uninstall-example clean clean-example
distclean: distclean:
rm -f Makefile TAGS config.status config.log *~ .depend rm -f Makefile TAGS config.status config.log *~ .depend
rm -rf autom4te.cache rm -rf autom4te.cache
for i in $(SUBDIRS) doc example util docker; \ for i in $(SUBDIRS) doc example docker; \
do (cd $$i && $(MAKE) $(MFLAGS) $@); done do (cd $$i && $(MAKE) $(MFLAGS) $@); done
# To make the example you need to run the "install-include" target first # To make the example you need to run the "install-include" target first
example: checkinstall util warnroot example: checkinstall warnroot
(cd $@ && $(MAKE) $(MFLAGS) all) (cd $@ && $(MAKE) $(MFLAGS) all)
@echo "\e[36mRemember to run 'make install-example' as euid root\e[0m" @echo "\e[36mRemember to run 'make install-example' as euid root\e[0m"

11156
config-aux/ltmain.sh
View file

File diff suppressed because it is too large Load diff

1
config-aux/ltmain.sh Symbolic link
View file

@ -0,0 +1 @@
/usr/share/libtool/build-aux/ltmain.sh

54
configure vendored
View file

@ -654,7 +654,6 @@ CLIGEN_DIR
WC_BIN WC_BIN
TAIL_BIN TAIL_BIN
GREP GREP
SSHD_BIN
SSH_BIN SSH_BIN
LEXLIB LEXLIB
LEX_OUTPUT_ROOT LEX_OUTPUT_ROOT
@ -5736,56 +5735,6 @@ fi
printf "%s\n" "#define SSH_BIN \"${SSH_BIN}\"" >>confdefs.h printf "%s\n" "#define SSH_BIN \"${SSH_BIN}\"" >>confdefs.h
# SSHD binary path for test
# Extract the first word of "sshd", so it can be a program name with args.
set dummy sshd; ac_word=$2
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
printf %s "checking for $ac_word... " >&6; }
if test ${ac_cv_path_SSHD_BIN+y}
then :
printf %s "(cached) " >&6
else $as_nop
case $SSHD_BIN in
[\\/]* | ?:[\\/]*)
ac_cv_path_SSHD_BIN="$SSHD_BIN" # Let the user override the test with a path.
;;
*)
as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
for as_dir in $PATH
do
IFS=$as_save_IFS
case $as_dir in #(((
'') as_dir=./ ;;
*/) ;;
*) as_dir=$as_dir/ ;;
esac
for ac_exec_ext in '' $ac_executable_extensions; do
if as_fn_executable_p "$as_dir$ac_word$ac_exec_ext"; then
ac_cv_path_SSHD_BIN="$as_dir$ac_word$ac_exec_ext"
printf "%s\n" "$as_me:${as_lineno-$LINENO}: found $as_dir$ac_word$ac_exec_ext" >&5
break 2
fi
done
done
IFS=$as_save_IFS
;;
esac
fi
SSHD_BIN=$ac_cv_path_SSHD_BIN
if test -n "$SSHD_BIN"; then
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: $SSHD_BIN" >&5
printf "%s\n" "$SSHD_BIN" >&6; }
else
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: no" >&5
printf "%s\n" "no" >&6; }
fi
printf "%s\n" "#define SSHD_BIN \"${SSHD_BIN}\"" >>confdefs.h
# For cli pipe output functions # For cli pipe output functions
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e" >&5 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e" >&5
printf %s "checking for grep that handles long lines and -e... " >&6; } printf %s "checking for grep that handles long lines and -e... " >&6; }
@ -7012,7 +6961,7 @@ fi
test "x$prefix" = xNONE && prefix=$ac_default_prefix test "x$prefix" = xNONE && prefix=$ac_default_prefix
ac_config_files="$ac_config_files Makefile lib/Makefile lib/src/Makefile lib/clixon/Makefile apps/Makefile apps/cli/Makefile apps/backend/Makefile apps/netconf/Makefile apps/restconf/Makefile apps/snmp/Makefile include/Makefile etc/Makefile etc/clixonrc example/Makefile example/main/Makefile example/main/example.xml docker/Makefile docker/clixon-dev/Makefile docker/example/Makefile docker/test/Makefile util/Makefile yang/Makefile yang/clixon/Makefile yang/mandatory/Makefile doc/Makefile test/Makefile test/config.sh test/cicd/Makefile test/vagrant/Makefile" ac_config_files="$ac_config_files Makefile lib/Makefile lib/src/Makefile lib/clixon/Makefile apps/Makefile apps/cli/Makefile apps/backend/Makefile apps/netconf/Makefile apps/restconf/Makefile apps/snmp/Makefile include/Makefile etc/Makefile etc/clixonrc example/Makefile example/main/Makefile example/main/example.xml docker/Makefile docker/clixon-dev/Makefile docker/example/Makefile docker/test/Makefile yang/Makefile yang/clixon/Makefile yang/mandatory/Makefile doc/Makefile test/Makefile test/config.sh test/cicd/Makefile test/vagrant/Makefile"
cat >confcache <<\_ACEOF cat >confcache <<\_ACEOF
# This file is a shell script that caches the results of configure # This file is a shell script that caches the results of configure
@ -7721,7 +7670,6 @@ do
"docker/clixon-dev/Makefile") CONFIG_FILES="$CONFIG_FILES docker/clixon-dev/Makefile" ;; "docker/clixon-dev/Makefile") CONFIG_FILES="$CONFIG_FILES docker/clixon-dev/Makefile" ;;
"docker/example/Makefile") CONFIG_FILES="$CONFIG_FILES docker/example/Makefile" ;; "docker/example/Makefile") CONFIG_FILES="$CONFIG_FILES docker/example/Makefile" ;;
"docker/test/Makefile") CONFIG_FILES="$CONFIG_FILES docker/test/Makefile" ;; "docker/test/Makefile") CONFIG_FILES="$CONFIG_FILES docker/test/Makefile" ;;
"util/Makefile") CONFIG_FILES="$CONFIG_FILES util/Makefile" ;;
"yang/Makefile") CONFIG_FILES="$CONFIG_FILES yang/Makefile" ;; "yang/Makefile") CONFIG_FILES="$CONFIG_FILES yang/Makefile" ;;
"yang/clixon/Makefile") CONFIG_FILES="$CONFIG_FILES yang/clixon/Makefile" ;; "yang/clixon/Makefile") CONFIG_FILES="$CONFIG_FILES yang/clixon/Makefile" ;;
"yang/mandatory/Makefile") CONFIG_FILES="$CONFIG_FILES yang/mandatory/Makefile" ;; "yang/mandatory/Makefile") CONFIG_FILES="$CONFIG_FILES yang/mandatory/Makefile" ;;

5
configure.ac
View file

@ -155,10 +155,6 @@ fi
AC_PATH_PROG(SSH_BIN, ssh) AC_PATH_PROG(SSH_BIN, ssh)
AC_DEFINE_UNQUOTED(SSH_BIN, "${SSH_BIN}", [SSH binary]) AC_DEFINE_UNQUOTED(SSH_BIN, "${SSH_BIN}", [SSH binary])
# SSHD binary path for test
AC_PATH_PROG(SSHD_BIN, sshd)
AC_DEFINE_UNQUOTED(SSHD_BIN, "${SSHD_BIN}", [SSHD binary])
# For cli pipe output functions # For cli pipe output functions
AC_PROG_GREP AC_PROG_GREP
AC_DEFINE_UNQUOTED(GREP_BIN, "$GREP", [Grep binary]) AC_DEFINE_UNQUOTED(GREP_BIN, "$GREP", [Grep binary])
@ -482,7 +478,6 @@ AC_CONFIG_FILES([Makefile
docker/clixon-dev/Makefile docker/clixon-dev/Makefile
docker/example/Makefile docker/example/Makefile
docker/test/Makefile docker/test/Makefile
util/Makefile
yang/Makefile yang/Makefile
yang/clixon/Makefile yang/clixon/Makefile
yang/mandatory/Makefile yang/mandatory/Makefile

7
docker/test/Dockerfile
View file

@ -81,8 +81,11 @@ RUN ./configure --prefix=/usr/local --sysconfdir=/etc --with-cligen=/clixon/buil
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install
# Install utils (for tests) # Configure, build and install clixon utilities (for tests)
WORKDIR /clixon/clixon/util WORKDIR /clixon
RUN git clone https://github.com/clicon/clixon-util.git
WORKDIR /clixon/clixon-util
RUN ./configure --prefix=/usr/local --with-cligen=/clixon/build/usr/local --with-clixon=/clixon/build/usr/local
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install

7
docker/test/Dockerfile.fcgi
View file

@ -89,8 +89,11 @@ RUN ./configure --prefix=/usr/local --sysconfdir=/etc --with-cligen=/clixon/buil
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install
# Install utils (for tests) # Configure, build and install clixon utilities (for tests)
WORKDIR /clixon/clixon/util WORKDIR /clixon
RUN git clone https://github.com/clicon/clixon-util.git
WORKDIR /clixon/clixon-util
RUN ./configure --prefix=/usr/local --with-cligen=/clixon/build/usr/local --with-clixon=/clixon/build/usr/local
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install

7
docker/test/Dockerfile.native
View file

@ -87,8 +87,11 @@ RUN ./configure --prefix=/usr/local --sysconfdir=/etc --with-cligen=/clixon/buil
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install
# Install utils (for tests) # Configure, build and install clixon utilities (for tests)
WORKDIR /clixon/clixon/util WORKDIR /clixon
RUN git clone https://github.com/clicon/clixon-util.git
WORKDIR /clixon/clixon-util
RUN ./configure --prefix=/usr/local --with-cligen=/clixon/build/usr/local --with-clixon=/clixon/build/usr/local
RUN make RUN make
RUN make DESTDIR=/clixon/build install RUN make DESTDIR=/clixon/build install

3
include/clixon_config.h.in
View file

@ -171,9 +171,6 @@
/* Default restconf network namespace */ /* Default restconf network namespace */
#undef RESTCONF_NETNS_DEFAULT #undef RESTCONF_NETNS_DEFAULT
/* SSHD binary */
#undef SSHD_BIN
/* SSH binary */ /* SSH binary */
#undef SSH_BIN #undef SSH_BIN

6
test/README.md
View file

@ -55,9 +55,11 @@ There are also [manual cicd scripts here](cicd/README.md)
## Getting started ## Getting started
You need to build and install the clixon utility programs before running the tests as some of the tests rely on them: You need to build and install the clixon utility programs from a separate repo
before running the tests as some of the tests rely on them:
``` ```
cd util git clone https://github.com/clicon/clixon-util.git
./configure
make make
sudo make install sudo make install
``` ```

201
util/Makefile.in
View file

@ -1,201 +0,0 @@
#
# ***** BEGIN LICENSE BLOCK *****
#
# Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
# Copyright (C) 2017-2019 Olof Hagsand
# Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
#
# This file is part of CLIXON
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Alternatively, the contents of this file may be used under the terms of
# the GNU General Public License Version 3 or later (the "GPL"),
# in which case the provisions of the GPL are applicable instead
# of those above. If you wish to allow use of your version of this file only
# under the terms of the GPL, and not to allow others to
# use your version of this file under the terms of Apache License version 2,
# indicate your decision by deleting the provisions above and replace them with
# the notice and other provisions required by the GPL. If you do not delete
# the provisions above, a recipient may use your version of this file under
# the terms of any one of the Apache License version 2 or the GPL.
#
# ***** END LICENSE BLOCK *****
#
prefix = @prefix@
datarootdir = @datarootdir@
srcdir = @srcdir@
top_srcdir = @top_srcdir@
exec_prefix = @exec_prefix@
bindir = @bindir@
libdir = @libdir@
dbdir = @prefix@/db
mandir = @mandir@
libexecdir = @libexecdir@
localstatedir = @localstatedir@
sysconfdir = @sysconfdir@
HOST_VENDOR = @host_vendor@
with_restconf = @with_restconf@
with_libcurl = @with_libcurl@
LIBXML2_CFLAGS = @LIBXML2_CFLAGS@
SH_SUFFIX = @SH_SUFFIX@
CLIXON_VERSION = @CLIXON_VERSION@
CLIXON_MAJOR = @CLIXON_VERSION_MAJOR@
CLIXON_MINOR = @CLIXON_VERSION_MINOR@
VPATH = @srcdir@
CC = @CC@
CFLAGS = @CFLAGS@
INSTALL = @INSTALL@
INSTALL_LIB = @INSTALL@
ifeq ($(HOST_VENDOR),apple)
INSTALLFLAGS =
else
INSTALLFLAGS = @INSTALLFLAGS@
endif
LDFLAGS = @LDFLAGS@
LIBS = @LIBS@
CPPFLAGS = @CPPFLAGS@
LINKAGE = @LINKAGE@
INCLUDES = -I. @INCLUDES@ -I$(top_srcdir)/lib -I$(top_srcdir)/include
CPPFLAGS += $(INCLUDES)
ifeq ($(LINKAGE),dynamic)
CLIXON_LIB = libclixon$(SH_SUFFIX).$(CLIXON_MAJOR).$(CLIXON_MINOR)
CLIXON_BACKEND_LIB = libclixon_backend$(SH_SUFFIX).$(CLIXON_MAJOR).$(CLIXON_MINOR)
else
CLIXON_LIB = libclixon.a
CLIXON_BACKEND_LIB = libclixon_backend.a # for util_validate
endif
# For dependency. A little strange that we rely on it being built in the src dir
# even though it may exist in $(libdir). But the new version may not have been installed yet.
LIBDEPS = $(top_srcdir)/lib/src/$(CLIXON_LIB)
BELIBDEPS = $(top_srcdir)/apps/backend/$(CLIXON_BACKEND_LIB)
# Utilities, unit testings.
APPSRC = clixon_util_xml.c
APPSRC += clixon_util_xml_mod.c
APPSRC += clixon_util_json.c
APPSRC += clixon_util_yang.c
APPSRC += clixon_util_xpath.c
APPSRC += clixon_util_path.c
APPSRC += clixon_util_datastore.c
APPSRC += clixon_util_regexp.c
APPSRC += clixon_util_socket.c
APPSRC += clixon_util_validate.c
APPSRC += clixon_util_dispatcher.c
APPSRC += clixon_netconf_ssh_callhome.c
APPSRC += clixon_netconf_ssh_callhome_client.c
ifdef with_restconf
ifdef with_libcurl
APPSRC += clixon_util_stream.c
endif
ifeq ($(with_restconf), native)
APPSRC += clixon_restconf_callhome_client.c
endif
endif
ifdef with_http2
APPSRC += clixon_util_ssl.c # requires http/2
#APPSRC += clixon_util_grpc.c # work in progress
endif
APPS = $(APPSRC:.c=)
all: $(APPS)
# Dependency of clixon library
$(top_srcdir)/lib/src/$(CLIXON_LIB):
(cd $(top_srcdir)/lib/src && $(MAKE) $(MFLAGS) $(CLIXON_LIB))
$(top_srcdir)/lib/src/$(CLIXON_BACKEND_LIB):
(cd $(top_srcdir)/apps/backend && $(MAKE) $(MFLAGS) $(CLIXON_BACKEND_LIB))
clean:
rm -f $(APPS) clixon_util_stream *.core
rm -f *.gcda *.gcno *.gcov # coverage
# APPS
clixon_util_xml: clixon_util_xml.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_json: clixon_util_json.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_yang: clixon_util_yang.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_xpath: clixon_util_xpath.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_path: clixon_util_path.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_datastore: clixon_util_datastore.c $(LIBDEPS)
$(CC) $(CPPFLAGS) $(CFLAGS) -D__PROGRAM__=\"$@\" $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_xml_mod: clixon_util_xml_mod.c $(LIBDEPS)
$(CC) $(CPPFLAGS) $(CFLAGS) -D__PROGRAM__=\"$@\" $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_regexp: clixon_util_regexp.c $(LIBDEPS)
$(CC) $(LIBXML2_CFLAGS) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_socket: clixon_util_socket.c $(LIBDEPS)
$(CC) $(CPPFLAGS) $(CFLAGS) -D__PROGRAM__=\"$@\" $(LDFLAGS) $^ $(LIBS) -o $@
clixon_util_validate: clixon_util_validate.c $(BELIBDEPS) $(LIBDEPS)
$(CC) $(CPPFLAGS) $(CFLAGS) -D__PROGRAM__=\"$@\" $(LDFLAGS) $^ -l clixon_backend -o $@ $(LIBS) $(BELIBS)
clixon_util_dispatcher: clixon_util_dispatcher.c $(BELIBDEPS) $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ -l clixon_backend -o $@ $(LIBS) $(BELIBS)
ifdef with_restconf
clixon_util_stream: clixon_util_stream.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -lcurl -o $@
clixon_restconf_callhome_client: clixon_restconf_callhome_client.c $(LIBDEPS)
$(CC) $(CPPFLAGS) -D__PROGRAM__=\"$@\" $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
endif
#clixon_util_grpc: clixon_util_grpc.c $(LIBDEPS)
# $(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $^ $(LIBS) -o $@
distclean: clean
rm -f Makefile *~ .depend
install: $(APPS)
install -d -m 0755 $(DESTDIR)$(bindir)
install -m 0755 $(INSTALLFLAGS) $(APPS) $(DESTDIR)$(bindir)
install-include:
install-lib:
uninstall:
rm -f $(DESTDIR)$(bindir)/$(APPS)
TAGS:
find . -name '*.[ch]' -print | etags -
depend:
$(CC) $(DEPENDFLAGS) @DEFS@ $(CPPFLAGS) $(CFLAGS) -MM $(APPSRC) > .depend
#include .depend

9
util/README.md
View file

@ -1,9 +0,0 @@
# Clixon utils
This directory contains Clixon utility programs, ie, programs that are
good to have for testing, analysis, etc, but not an actual part of
delivered code.
Look inside C-code for documentation
Note, streams utility may need: libcurl4-openssl-dev or corresponding.

325
util/clixon_netconf_ssh_callhome.c
View file

@ -1,325 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Create stream socket, connect to remote address, then exec sshd -e that takes over the
* tcp connection.
device/server client
+-----------------+ 2) tcp connect +-----------------+
| callhome | ----------------> | callhome-client |
+-----------------+ +-----------------+
| 3) c ^
v 1) | 4)
+-----------------+ ssh +-----------------+ 5) stdio
| sshd -i | <----------------> | ssh | <------ <rpc>...</rpc>]]>]]>"
+-----------------+ |-----------------+
| stdio
+-----------------+
| clixon_netconf |
+-----------------+
|
+-----------------+
| clixon_backend |
+-----------------+
1) Start ssh client using -o ProxyUseFdpass=yes -o ProxyCommand="callhome-client".
Callhome-client listens on port 4334 for incoming TCP connections.
2) Start callhome on server making tcp connect to client on port 4334 establishing a tcp stream
3) Callhome starts sshd -i using the established stream socket (stdio)
4) Callhome-client returns with an open stream socket to the ssh client establishing an SSH stream
to server
5) Client request sent on stdin to ssh client on established SSH stream using netconf subsystem
to clixon_netconf client
ssh -s -v -o ProxyUseFdpass=yes -o ProxyCommand="clixon_netconf_ssh_callhome_client -a 127.0.0.1" . netconf
sudo clixon_netconf_ssh_callhome -a 127.0.0.1 -c /var/tmp/./test_netconf_ssh_callhome.sh/conf_yang.xml
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <assert.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/socket.h>
#define NETCONF_CH_SSH 4334
#define SSHDBIN_DEFAULT SSHD_BIN
#define UTIL_OPTS "hD:f:a:p:s:c:C:"
static int
callhome_connect(struct sockaddr *sa,
size_t sa_len,
int *sp)
{
int retval = -1;
int s;
if ((s = socket(sa->sa_family, SOCK_STREAM, 0)) < 0) {
perror("socket");
goto done;
}
if (connect(s, sa, sa_len) < 0){
perror("connect");
close(s);
goto done;
}
*sp = s;
retval = 0;
done:
return retval;
}
/* @see clixon_inet2sin */
static int
inet2sin(const char *addrtype,
const char *addrstr,
uint16_t port,
struct sockaddr *sa,
size_t *sa_len)
{
struct sockaddr_in6 *sin6;
struct sockaddr_in *sin;
if (strcmp(addrtype, "inet:ipv6-address") == 0) {
sin6 = (struct sockaddr_in6 *)sa;
*sa_len = sizeof(struct sockaddr_in6);
sin6->sin6_port = htons(port);
sin6->sin6_family = AF_INET6;
inet_pton(AF_INET6, addrstr, &sin6->sin6_addr);
}
else if (strcmp(addrtype, "inet:ipv4-address") == 0) {
sin = (struct sockaddr_in *)sa;
*sa_len = sizeof(struct sockaddr_in);
sin->sin_family = AF_INET;
sin->sin_port = htons(port);
sin->sin_addr.s_addr = inet_addr(addrstr);
}
else{
fprintf(stderr, "Unexpected addrtype: %s\n", addrtype);
return -1;
}
return 0;
}
static int
ssh_server_exec(int s,
char *sshdbin,
char *sshdconfigfile,
char *clixonconfigfile,
int dbg)
{
int retval = -1;
char **argv = NULL;
int i;
int nr;
char *optstr = NULL;
size_t len;
const char *formatstr = "Subsystem netconf " CLIXON_CONFIG_BINDIR "/clixon_netconf -f %s";
if (s < 0){
errno = EINVAL;
perror("socket s");
goto done;
}
if (sshdbin == NULL){
errno = EINVAL;
perror("sshdbin");
goto done;
}
if (sshdconfigfile == NULL){
errno = EINVAL;
perror("sshdconfigfile");
goto done;
}
if (clixonconfigfile == NULL){
errno = EINVAL;
perror("clixonconfigfile");
goto done;
}
/* Construct subsystem string */
len = strlen(formatstr)+strlen(clixonconfigfile)+1;
if ((optstr = malloc(len)) == NULL){
perror("malloc");
goto done;
}
snprintf(optstr, len, formatstr, clixonconfigfile);
nr = 9; /* See below */
if (dbg)
nr++;
if ((argv = calloc(nr, sizeof(char *))) == NULL){
perror("calloc");
goto done;
}
i = 0;
/* Note if you change here, also change in nr = above */
argv[i++] = sshdbin;
argv[i++] = "-i"; /* Specifies that sshd is being run from inetd(8) */
argv[i++] = "-D"; /* Foreground ? */
if (dbg)
argv[i++] = "-d"; /* Debug mode */
argv[i++] = "-e"; /* write debug logs to stderr */
argv[i++] = "-o"; /* option */
argv[i++] = optstr;
argv[i++] = "-f"; /* config file */
argv[i++] = sshdconfigfile;
argv[i++] = NULL;
assert(i==nr);
if (setreuid(0, 0) < 0){
perror("setreuid");
goto done;
}
close(0);
close(1);
if (dup2(s, STDIN_FILENO) < 0){
perror("dup2");
return -1;
}
if (dup2(s, STDOUT_FILENO) < 0){
perror("dup2");
return -1;
}
if (execv(argv[0], argv) < 0) {
perror("execv");
exit(1);
}
/* Should reach here */
retval = 0;
done:
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f ipv4|ipv6 \tSocket address family(inet:ipv4-address default)\n"
"\t-a <addrstr> \tIP address (eg 1.2.3.4) - mandatory\n"
"\t-p <port> \tPort (default 4334)\n"
"\t-c <file> \tClixon config file - (default " CLIXON_DEFAULT_CONFIG ")\n"
"\t-C <file> \tSSHD config file - (default /dev/null)\n"
"\t-s <sshd> \tPath to sshd binary, default %s\n"
,
argv0, SSHDBIN_DEFAULT);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
int c;
char *family = "inet:ipv4-address";
char *addr = NULL;
struct sockaddr_in6 sin6 = {0, };
struct sockaddr *sa = (struct sockaddr *)&sin6;
size_t sa_len;
int dbg = 0;
uint16_t port = NETCONF_CH_SSH;
int s = -1;
char *sshdbin = SSHDBIN_DEFAULT;
char *sshdconfigfile = "/dev/null";
char *clixonconfigfile = CLIXON_CONFIG_SYSCONFDIR "/clixon.xml";
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
dbg++;
break;
case 'f':
family = optarg;
break;
case 'a':
addr = optarg;
break;
case 'p':
port = atoi(optarg);
break;
case 'C':
sshdconfigfile = optarg;
break;
case 'c':
clixonconfigfile = optarg;
break;
case 's':
sshdbin = optarg;
break;
default:
usage(argv[0]);
break;
}
if (port == 0){
fprintf(stderr, "-p <port> is invalid\n");
usage(argv[0]);
goto done;
}
if (addr == NULL){
fprintf(stderr, "-a <addr> is NULL\n");
usage(argv[0]);
goto done;
}
if (inet2sin(family, addr, port, sa, &sa_len) < 0)
goto done;
if (callhome_connect(sa, sa_len, &s) < 0)
goto done;
/* For some reason this sshd returns -1 which is unclear why */
if (ssh_server_exec(s, sshdbin, sshdconfigfile, clixonconfigfile, dbg) < 0)
goto done;
/* Should not reach here */
if (s >= 0)
close(s);
retval = 0;
done:
return retval;
}

310
util/clixon_netconf_ssh_callhome_client.c
View file

@ -1,310 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
device/server client
+-----------------+ 2) tcp connect +-----------------+
| callhome | ----------------> | callhome-client |
+-----------------+ +-----------------+
| 3) c ^
v 1) | 4)
+-----------------+ ssh +-----------------+ 5) stdio
| sshd -i | <----------------> | ssh | <------ <rpc>...</rpc>]]>]]>"
+-----------------+ |-----------------+
| stdio
+-----------------+
| clixon_netconf |
+-----------------+
|
+-----------------+
| clixon_backend |
+-----------------+
1) Start ssh client using -o ProxyUseFdpass=yes -o ProxyCommand="callhome-client".
Callhome-client listens on port 4334 for incoming TCP connections.
2) Start callhome on server making tcp connect to client on port 4334 establishing a tcp stream
3) Callhome starts sshd -i using the established stream socket (stdio)
4) Callhome-client returns with an open stream socket to the ssh client establishing an SSH stream
to server
5) Client request sent on stdin to ssh client on established SSH stream using netconf subsystem
to clixon_netconf client
Example sshd-config (-c option):n
ssh -s -v -o ProxyUseFdpass=yes -o ProxyCommand="clixon_netconf_ssh_callhome_client -a 127.0.0.1" . netconf
sudo clixon_netconf_ssh_callhome -a 127.0.0.1
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <poll.h>
#include <errno.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/socket.h>
#define NETCONF_CH_SSH 4334
#define UTIL_OPTS "hD:f:a:p:"
/*
* fdpass()
* Pass the connected file descriptor to stdout and exit.
* This is taken from:
* https://github.com/openbsd/src/blob/master/usr.bin/nc/netcat.c
* Copyright (c) 2001 Eric Jackson <ericj@monkey.org>
* Copyright (c) 2015 Bob Beck. All rights reserved.
*/
static int
fdpass(int nfd)
{
struct msghdr mh;
union {
struct cmsghdr hdr;
char buf[CMSG_SPACE(sizeof(int))];
} cmsgbuf;
struct cmsghdr *cmsg;
struct iovec iov;
char c = '\0';
ssize_t r;
struct pollfd pfd;
/* Avoid obvious stupidity */
if (isatty(STDOUT_FILENO)){
perror("Cannot pass file descriptor to tty");
return -1;
}
memset(&mh, 0, sizeof(mh));
memset(&cmsgbuf, 0, sizeof(cmsgbuf));
memset(&iov, 0, sizeof(iov));
mh.msg_control = (char*)&cmsgbuf.buf;
mh.msg_controllen = sizeof(cmsgbuf.buf);
cmsg = CMSG_FIRSTHDR(&mh);
cmsg->cmsg_len = CMSG_LEN(sizeof(int));
cmsg->cmsg_level = SOL_SOCKET;
cmsg->cmsg_type = SCM_RIGHTS;
*(int *)CMSG_DATA(cmsg) = nfd;
iov.iov_base = &c;
iov.iov_len = 1;
mh.msg_iov = &iov;
mh.msg_iovlen = 1;
memset(&pfd, 0, sizeof(pfd));
pfd.fd = STDOUT_FILENO;
pfd.events = POLLOUT;
for (;;) {
r = sendmsg(STDOUT_FILENO, &mh, 0);
if (r == -1) {
if (errno == EAGAIN || errno == EINTR) {
if (poll(&pfd, 1, -1) == -1){
perror("poll");
return -1;
}
continue;
}
perror("sendmsg");
return -1;
} else if (r != 1){
perror("sendmsg: unexpected return value");
return -1;
}
else
break;
}
// exit(0);
return 0;
}
/*! Create and bind stream socket
*
* @param[in] sa Socketaddress
* @param[in] sa_len Length of sa. Tecynicaliyu to be independent of sockaddr sa_len
* @param[in] backlog Listen backlog, queie of pending connections
* @param[out] sock Server socket (bound for accept)
* @retval 0 OK
* @retval -1 Error
*/
int
callhome_bind(struct sockaddr *sa,
size_t sin_len,
int backlog,
int *sock)
{
int retval = -1;
int s = -1;
int on = 1;
if (sock == NULL){
errno = EINVAL;
perror("sock");
goto done;
}
/* create inet socket */
if ((s = socket(sa->sa_family, SOCK_STREAM, 0)) < 0) {
perror("socket");
goto done;
}
if (setsockopt(s, SOL_SOCKET, SO_KEEPALIVE, (void *)&on, sizeof(on)) == -1) {
perror("setsockopt SO_KEEPALIVE");
goto done;
}
if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (void *)&on, sizeof(on)) == -1) {
perror("setsockopt SO_REUSEADDR");
goto done;
}
/* only bind ipv6, otherwise it may bind to ipv4 as well which is strange but seems default */
if (sa->sa_family == AF_INET6 &&
setsockopt(s, IPPROTO_IPV6, IPV6_V6ONLY, &on, sizeof(on)) == -1) {
perror("setsockopt IPPROTO_IPV6");
goto done;
}
if (bind(s, sa, sin_len) == -1) {
perror("bind");
goto done;
}
if (listen(s, backlog) < 0){
perror("listen");
goto done;
}
if (sock)
*sock = s;
retval = 0;
done:
if (retval != 0 && s != -1)
close(s);
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f ipv4|ipv6 \tSocket address family(ipv4 default)\n"
"\t-a <addrstr> \tIP address (eg 1.2.3.4) - mandatory\n"
"\t-p <port> \tPort (default 4334)\n"
,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
int c;
char *family = "ipv4";
char *addr = NULL;
struct sockaddr *sa;
struct sockaddr_in6 sin6 = { 0 };
struct sockaddr_in sin = { 0 };
struct sockaddr from = {0,};
socklen_t len;
size_t sin_len;
uint16_t port = NETCONF_CH_SSH;
int ss = -1; /* server socket */
int s = -1; /* accepted session socket */
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'f':
family = optarg;
break;
case 'a':
addr = optarg;
break;
case 'p':
port = atoi(optarg);
break;
default:
usage(argv[0]);
break;
}
if (port == 0){
fprintf(stderr, "-p <port> is invalid\n");
usage(argv[0]);
goto done;
}
if (addr == NULL){
fprintf(stderr, "-a <addr> is NULL\n");
usage(argv[0]);
goto done;
}
if (strcmp(family, "ipv6") == 0){
sin_len = sizeof(struct sockaddr_in6);
sin6.sin6_port = htons(port);
sin6.sin6_family = AF_INET6;
inet_pton(AF_INET6, addr, &sin6.sin6_addr);
sa = (struct sockaddr *)&sin6;
}
else if (strcmp(family, "ipv4") == 0){
sin_len = sizeof(struct sockaddr_in);
sin.sin_family = AF_INET;
sin.sin_port = htons(port);
sin.sin_addr.s_addr = inet_addr(addr);
sa = (struct sockaddr *)&sin;
}
else{
fprintf(stderr, "-f <%s> is invalid family\n", family);
goto done;
}
/* Bind port */
if (callhome_bind(sa, sin_len, 1, &ss) < 0)
goto done;
/* Wait until connect */
len = sizeof(from);
if ((s = accept(ss, &from, &len)) < 0){
perror("accept");
goto done;
}
/* s Pass the first connected socket using sendmsg(2) to stdout and exit. */
if (fdpass(s) < 0)
goto done;
retval = 0;
done:
return retval;
}

749
util/clixon_restconf_callhome_client.c
View file

@ -1,749 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
See RFC 8071 NETCONF Call Home and RESTCONF Call Home
device/server client
+-----------------+ 1) tcp connect +-----------------+
| clixon_restconf | ----------------> | callhome-client | <------ 3) HTTP
| | 2) tls | |
+-----------------+ <--------------- +-----------------+
The callhome-client listens on accept, when connect comes in, creates data socket and sends
RESTCONF GET to server, then re-waits for new accepts.
When accepting a connection, send HTTP data from input or -f <file> tehn wait for reply
Reply is matched with -e <expectfile> or printed on stdout
Tracing events on stdout using:
Accept:<n> at t=<sec> # where <n> is connection nr, <sec> is time since start of program
Close: <n> <where> <sec> at t=<sec> # where <n> is connection nr, <where> is local or remote, <sec> is time since start of connection
Reply: <n> t=<sec> [\n<msg>\n] # where <n> is nr data reply from start, <sec> is time since start of connection
Exit: <function> # where <reason> is which exit point (for debugging)
Timeline:
w
<-------------->
a0 d0 d1 a1 d0 d1
----------|----|----|------------------|----|----|------------------|
ai Accepted connect from server
di Reply from server
n Number of ai:s, 0 means no limit (_accepts)
D Timeout of di:s (1st request sent at ai, sent back-to-back or with 1sec interval) (_data_timeout_s)
idle? If set do not close after D timeout
t Wait for accept, exit if no accepts (default: 60s), just a safety for deadlocks (_accept_timeout_s)
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <syslog.h>
#include <signal.h>
#include <netdb.h> /* gethostbyname */
#include <arpa/inet.h> /* inet_pton */
#include <netinet/tcp.h> /* TCP_NODELAY */
#include <openssl/ssl.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
#define UTIL_TLS_OPTS "hD:f:F:a:p:c:C:k:n:N:it:d:e:"
#define RESTCONF_CH_TLS 4336
/* User struct for context / accept */
typedef struct {
int ta_ss; /* Accept socket */
SSL_CTX *ta_ctx; /* SSL context */
struct timeval ta_t0; /* Program start */
} tls_accept_handle;
/* User connection-specific data handle */
typedef struct {
int sd_s; /* data socket */
SSL *sd_ssl; /* SSL connection data */
struct timeval sd_t0; /* Start of connection, eg accept call*/
} tls_session_data;
/* Lots of global variables here, alt pass them between ta and sd structs
*/
/* Input data file for HTTP request data */
static FILE *_input_file = NULL;
/* Expected accepts */
static int _accepts = 1;
/* Number of accepts */
static int _n_accepts = 0;
/* After accepting a socket, a request is sent to the server. The handle the data socket as follows:
* 0: close after first reply
* -1: dont close after reply, (remote side may close)
* s>0: send new requests during <s> seconds after accept, then dont close
*/
static int _idle = 0;
/* Timeout in seconds after each accept, if fired just exit */
static int _accept_timeout_s = 60;
/* Timeout of data requests (1st request sent at accept, sent back-to-back / 1sec interval)
* Note: uses blockling timeout 100ms
*/
static int _data_timeout_s = 0;
/* Event trace, 1: terse (Accept:/Reply:/Close:) 2: full (data payload) */
static int _event_trace = 0;
static FILE *_event_f = NULL; /* set to stdout in main */
/*! Create and bind stream socket
*
* @param[in] sa Socketaddress
* @param[in] sa_len Length of sa. Tecynicaliyu to be independent of sockaddr sa_len
* @param[in] backlog Listen backlog, queie of pending connections
* @param[out] sock Server socket (bound for accept)
* @retval 0 OK
* @retval -1 Error
*/
int
callhome_bind(struct sockaddr *sa,
size_t sin_len,
int backlog,
int *sock)
{
int retval = -1;
int s = -1;
int on = 1;
if (sock == NULL){
errno = EINVAL;
perror("sock");
goto done;
}
/* create inet socket */
if ((s = socket(sa->sa_family, SOCK_STREAM, 0)) < 0) {
perror("socket");
goto done;
}
if (setsockopt(s, SOL_SOCKET, SO_KEEPALIVE, (void *)&on, sizeof(on)) == -1) {
perror("setsockopt SO_KEEPALIVE");
goto done;
}
if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, (void *)&on, sizeof(on)) == -1) {
perror("setsockopt SO_REUSEADDR");
goto done;
}
/* only bind ipv6, otherwise it may bind to ipv4 as well which is strange but seems default */
if (sa->sa_family == AF_INET6 &&
setsockopt(s, IPPROTO_IPV6, IPV6_V6ONLY, &on, sizeof(on)) == -1) {
perror("setsockopt IPPROTO_IPV6");
goto done;
}
if (bind(s, sa, sin_len) == -1) {
perror("bind");
goto done;
}
if (listen(s, backlog) < 0){
perror("listen");
goto done;
}
if (sock)
*sock = s;
retval = 0;
done:
if (retval != 0 && s != -1)
close(s);
return retval;
}
/*! read data from file return a malloced buffer
*
* Note same file is reread multiple times: same request/reply is made each iteration
* Also, the file read is limited to 1024 bytes
*/
static int
read_data_file(FILE *fe,
char **bufp,
size_t *lenp)
{
int retval = -1;
char *buf = NULL;
int buflen = 1024; /* start size */
char ch;
size_t len = 0;
int ret;
if ((buf = malloc(buflen)) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
goto done;
}
memset(buf, 0, buflen);
/* Start file form beginning */
rewind(fe);
while (1){
if ((ret = fread(&ch, 1, 1, fe)) < 0){
clicon_err(OE_JSON, errno, "fread");
goto done;
}
if (ret == 0)
break;
buf[len++] = ch;
// XXX No realloc, can overflow
}
*bufp = buf;
*lenp = len;
retval = 0;
done:
return retval;
}
/*! Read data from file/stdin and write to TLS data socket
*/
static int
tls_write_file(FILE *fp,
SSL *ssl)
{
int retval = -1;
char *buf = NULL;
size_t len = 0;
int ret;
int sslerr;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
if (read_data_file(fp, &buf, &len) < 0)
goto done;
if ((ret = SSL_write(ssl, buf, len)) < 1){
sslerr = SSL_get_error(ssl, ret);
clixon_debug(CLIXON_DBG_DEFAULT, "%s SSL_write() n:%d errno:%d sslerr:%d", __FUNCTION__, ret, errno, sslerr);
}
retval = 0;
done:
if (buf)
free(buf);
return retval;
}
/*! Client data socket, receive reply from server
*
* Print info on stdout
* If keep_open = 0, then close socket directly after 1st reply (client close)
* If keep_open = 1, then keep socket open (server close)
*/
static int
tls_server_reply_cb(int s,
void *arg)
{
int retval = -1;
tls_session_data *sd = (tls_session_data *)arg;
SSL *ssl;
char buf[1024];
int n;
char *expbuf = NULL;
struct timeval now;
struct timeval td;
static int seq = 0; // from start
// clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
ssl = sd->sd_ssl;
/* get reply & decrypt */
if ((n = SSL_read(ssl, buf, sizeof(buf))) < 0){
clicon_err(OE_XML, errno, "SSL_read");
goto done;
}
clixon_debug(CLIXON_DBG_DEFAULT, "%s n:%d", __FUNCTION__, n);
gettimeofday(&now, NULL);
timersub(&now, &sd->sd_t0, &td); /* from start of connection */
if (n == 0){ /* Server closed socket */
SSL_free(ssl);
clixon_event_unreg_fd(s, tls_server_reply_cb);
if (_event_trace)
fprintf(_event_f, "Close: %d remote at t=%lu\n", _n_accepts, td.tv_sec);
close(s);
free(sd);
if (_accepts == 0)
;
else if (_accepts == 1){
clixon_exit_set(1); /* XXX more elaborate logic: 1) continue request, 2) close and accept new */
fprintf(_event_f, "Exit: %s remote\n", __FUNCTION__);
}
else
_accepts--;
goto ok;
}
seq++;
buf[n] = 0;
if (_event_trace){
fprintf(_event_f, "Reply: %d t=%lu\n", seq, td.tv_sec);
if (_event_trace > 1)
fprintf(_event_f, "%s\n", buf);
}
/* See if we should send more requests on this socket */
if (sd->sd_t0.tv_sec + _data_timeout_s > now.tv_sec){
/* Send another packet */
usleep(100000); /* XXX This is a blocking timeout */
/* Write HTTP request on socket */
if (tls_write_file(_input_file, sd->sd_ssl) < 0)
goto done;
}
else if (!_idle){
clixon_debug(CLIXON_DBG_DEFAULT, "%s idle", __FUNCTION__);
SSL_shutdown(ssl);
SSL_free(ssl);
clixon_event_unreg_fd(s, tls_server_reply_cb);
if (_event_trace)
fprintf(_event_f, "Close: %d local at t=%lu\n", _n_accepts, td.tv_sec);
close(s);
if (_accepts == 0)
;
else if (_accepts == 1){
clixon_exit_set(1); /* XXX more elaborate logic: 1) continue request, 2) close and accept new */
fprintf(_event_f, "Exit: %s idle\n", __FUNCTION__);
}
else
_accepts--;
free(sd);
}
ok:
retval = 0;
done:
if (expbuf)
free(expbuf);
clixon_debug(CLIXON_DBG_DEFAULT, "%s ret:%d", __FUNCTION__, retval);
return retval;
}
/*! Create ssl connection, select alpn, connect and verify
*/
static int
tls_ssl_init_connect(SSL_CTX *ctx,
int s,
SSL **sslp)
{
int retval = -1;
SSL *ssl = NULL;
unsigned char protos[10];
int ret;
int verify;
int sslerr;
/* create new SSL connection state */
if ((ssl = SSL_new(ctx)) == NULL){
clicon_err(OE_SSL, 0, "SSL_new.");
goto done;
}
SSL_set_fd(ssl, s); /* attach the socket descriptor */
SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);
protos[0] = 8;
strncpy((char*)&protos[1], "http/1.1", 9);
if ((retval = SSL_set_alpn_protos(ssl, protos, 9)) != 0){
clicon_err(OE_SSL, retval, "SSL_set_alpn_protos.");
goto done;
}
#if 0
SSL_get0_next_proto_negotiated(conn_.tls.ssl, &next_proto, &next_proto_len);
SSL_get0_alpn_selected(conn_.tls.ssl, &next_proto, &next_proto_len);
#endif
/* perform the connection
TLSEXT_TYPE_application_layer_protocol_negotiation
int SSL_set_alpn_protos(SSL *ssl, const unsigned char *protos,
unsigned int protos_len);
see
https://www.openssl.org/docs/man3.0/man3/SSL_CTX_set_alpn_select_cb.html
*/
if ((ret = SSL_connect(ssl)) < 1){
sslerr = SSL_get_error(ssl, ret);
clixon_debug(CLIXON_DBG_DEFAULT, "%s SSL_read() n:%d errno:%d sslerr:%d", __FUNCTION__, ret, errno, sslerr);
switch (sslerr){
case SSL_ERROR_SSL: /* 1 */
goto done;
break;
default:
clicon_err(OE_XML, errno, "SSL_connect");
goto done;
break;
}
}
/* check certificate verification result */
verify = SSL_get_verify_result(ssl);
switch (verify) {
case X509_V_OK:
break;
default:
clicon_err(OE_SSL, errno, "verify problems: %d", verify);
goto done;
}
*sslp = ssl;
retval = 0;
done:
return retval;
}
static int
tls_timeout_cb(int fd,
void *arg)
{
fprintf(_event_f, "Exit: %s\n", __FUNCTION__);
exit(200);
}
/*! Timeout in seconds after each accept, if fired just exit
*/
static int
tls_client_timeout(void *arg)
{
int retval = -1;
struct timeval now;
struct timeval t;
struct timeval t1 = {0, 0};
/* Unregister existing timeout */
clixon_event_unreg_timeout(tls_timeout_cb, arg);
/* Set timeout */
gettimeofday(&now, NULL);
t1.tv_sec = _accept_timeout_s;
timeradd(&now, &t1, &t);
if (clixon_event_reg_timeout(t,
tls_timeout_cb,
arg,
"tls client timeout") < 0)
goto done;
retval = 0;
done:
return retval;
}
/*! Callhome-server accept socket
*/
static int
tls_server_accept_cb(int ss,
void *arg)
{
int retval = -1;
tls_accept_handle *ta = (tls_accept_handle *)arg;
tls_session_data *sd = NULL;
int s;
struct sockaddr from = {0,};
socklen_t len;
SSL *ssl = NULL;
struct timeval td;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
len = sizeof(from);
if ((s = accept(ss, &from, &len)) < 0){
perror("accept");
goto done;
}
clixon_debug(CLIXON_DBG_DEFAULT, "accepted");
if (tls_ssl_init_connect(ta->ta_ctx, s, &ssl) < 0)
goto done;
clixon_debug(CLIXON_DBG_DEFAULT, "connected");
if ((sd = malloc(sizeof(*sd))) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
goto done;
}
memset(sd, 0, sizeof(*sd));
sd->sd_s = s;
sd->sd_ssl = ssl;
gettimeofday(&sd->sd_t0, NULL);
timersub(&sd->sd_t0, &ta->ta_t0, &td); /* from start of connection */
_n_accepts++;
if (_event_trace)
fprintf(_event_f, "Accept: %d at t=%lu\n", _n_accepts, td.tv_sec);
/* Always write one HTTP request on socket, maybe more if _data_timeout_s > 0 */
if (tls_write_file(_input_file, ssl) < 0)
goto done;
/* register callback for reply */
if (clixon_event_reg_fd(s, tls_server_reply_cb, sd, "tls server reply") < 0)
goto done;
/* Unregister old + register new timeout */
if (tls_client_timeout(ta) < 0)
goto done;
retval = 0;
done:
return retval;
}
/*! Out must be set to point to the selected protocol (which may be within in).
*/
static int
tls_proto_select_cb(SSL *s,
unsigned char **out,
unsigned char *outlen,
const unsigned char *in,
unsigned int inlen,
void *arg)
{
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
return 0;
}
/*! Verify tls auth
*
* @see tlsauth_verify_callback
* This code needs a "X509 store", see X509_STORE_new()
* crl_file / crl_dir
*/
static int
tls_auth_verify_callback(int preverify_ok,
X509_STORE_CTX *x509_ctx)
{
return 1; /* success */
}
static SSL_CTX *
tls_ctx_init(const char *cert_path,
const char *key_path,
const char *ca_cert_path)
{
SSL_CTX *ctx = NULL;
if ((ctx = SSL_CTX_new(TLS_client_method())) == NULL) {
clicon_err(OE_SSL, 0, "SSL_CTX_new");
goto done;
}
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, tls_auth_verify_callback);
/* get peer certificate
nc_client_tls_update_opts */
if (SSL_CTX_use_certificate_file(ctx, cert_path, SSL_FILETYPE_PEM) != 1) {
clicon_err(OE_SSL, 0, "SSL_CTX_use_certificate_file");
goto done;
}
if (SSL_CTX_use_PrivateKey_file(ctx, key_path, SSL_FILETYPE_PEM) != 1) {
clicon_err(OE_SSL, 0, "SSL_CTX_use_PrivateKey_file");
goto done;
}
if (SSL_CTX_load_verify_locations(ctx, ca_cert_path, NULL) != 1) {
clicon_err(OE_SSL, 0, "SSL_CTX_load_verify_locations");
goto done;
}
(void)SSL_CTX_set_next_proto_select_cb(ctx, tls_proto_select_cb, NULL);
return ctx;
done:
return NULL;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f <file> \tHTTP input file (overrides stdin)\n"
"\t-F ipv4|ipv6 \tSocket address family(ipv4 default)\n"
"\t-a <addrstr> \tIP address (eg 1.2.3.4) - mandatory\n"
"\t-p <port> \tPort (default %d)\n"
"\t-c <path> \tcert\n"
"\t-C <path> \tcacert\n"
"\t-k <path> \tkey\n"
"\t-n <nr> \tQuit after this many incoming connections, 0 means no limit. Default: 1\n"
"\t-t <sec> \tTimeout in seconds after each accept, if fired just exit. Default: %ds\n"
"\t-d <sec> \tTimeout of data requests on a connection in seconds after each accept, if fired either close or keep idle (see -i). Default: 0s\n"
"\t-i \tIdle after receiving last reply. Otherwise close directly after receiving last reply\n"
"\t-e <nr> \tEvent trace on stdout, 1: terse, 2: full\n"
,
argv0,
RESTCONF_CH_TLS,
_accept_timeout_s);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
clicon_handle h;
int c;
uint16_t port = RESTCONF_CH_TLS;
SSL_CTX *ctx = NULL;
int ss = -1;
int dbg = 0;
tls_accept_handle *ta = NULL;
char *input_filename = NULL;
char *ca_cert_path = NULL;
char *cert_path = NULL;
char *key_path = NULL;
FILE *fp = stdin; /* base file, stdin, can be overridden with -f */
struct sockaddr_in6 sin6 = {0,}; // because its larger than sin and sa
struct sockaddr *sa = (struct sockaddr *)&sin6;
size_t sa_len;
char *addr = "127.0.0.1";
char *family = "inet:ipv4-address";
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
if ((h = clicon_handle_init()) == NULL)
goto done;
while ((c = getopt(argc, argv, UTIL_TLS_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'f':
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
input_filename = optarg;
break;
case 'F':
family = optarg;
break;
case 'a':
addr = optarg;
break;
case 'p':
if (sscanf(optarg, "%hu", &port) != 1)
usage(argv[0]);
break;
case 'c':
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
cert_path = optarg;
break;
case 'C':
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
ca_cert_path = optarg;
break;
case 'k':
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
key_path = optarg;
break;
case 'n':
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
_accepts = atoi(optarg);
break;
case 'i': /* keep open, do not close after first reply */
_idle = 1;
break;
case 't': /* accept timeout */
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
_accept_timeout_s = atoi(optarg);
break;
case 'd': /* data timeout */
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
_data_timeout_s = atoi(optarg);
break;
case 'e': /* Event trace */
if (optarg == NULL || *optarg == '-')
usage(argv[0]);
_event_trace = atoi(optarg);
_event_f = stdout;
break;
default:
usage(argv[0]);
break;
}
if (cert_path == NULL || key_path == NULL || ca_cert_path == NULL){
fprintf(stderr, "-c <cert path> and -k <key path> -C <ca-cert> are mandatory\n");
usage(argv[0]);
}
clixon_debug_init(dbg, NULL);
if (input_filename){
if ((_input_file = fopen(input_filename, "r")) == NULL){
clicon_err(OE_YANG, errno, "open(%s)", input_filename);
goto done;
}
}
if ((ctx = tls_ctx_init(cert_path, key_path, ca_cert_path)) == NULL)
goto done;
if (port == 0){
fprintf(stderr, "-p <port> is invalid\n");
usage(argv[0]);
goto done;
}
if (addr == NULL){
fprintf(stderr, "-a <addr> is NULL\n");
usage(argv[0]);
goto done;
}
if (clixon_inet2sin(family, addr, port, sa, &sa_len) < 0)
goto done;
/* Bind port */
if (callhome_bind(sa, sa_len, 1, &ss) < 0)
goto done;
clixon_debug(CLIXON_DBG_DEFAULT, "callhome_bind %s:%hu", addr, port);
if ((ta = malloc(sizeof(*ta))) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
goto done;
}
memset(ta, 0, sizeof(*ta));
ta->ta_ctx = ctx;
ta->ta_ss = ss;
gettimeofday(&ta->ta_t0, NULL);
if (clixon_event_reg_fd(ss, tls_server_accept_cb, ta, "tls server accept") < 0)
goto done;
if (tls_client_timeout(ta) < 0)
goto done;
if (clixon_event_loop(h) < 0)
goto done;
retval = 0;
done:
if (ss != -1)
clixon_event_unreg_fd(ss, tls_server_accept_cb);
if (ta)
free(ta);
if (fp)
fclose(fp);
if (ss != -1)
close(ss);
if (ctx)
SSL_CTX_free(ctx); /* release context */
clicon_handle_exit(h); /* frees h and options (and streams) */
clixon_err_exit();
clixon_debug(CLIXON_DBG_DEFAULT, "clixon_restconf_callhome_client pid:%u done", getpid());
clicon_log_exit(); /* Must be after last clixon_debug */
return retval;
}

366
util/clixon_util_datastore.c
View file

@ -1,366 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdarg.h>
#include <errno.h>
#include <signal.h>
#include <fcntl.h>
#include <time.h>
#include <signal.h>
#include <syslog.h>
#include <sys/time.h>
#include <sys/socket.h>
#include <sys/param.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netinet/in.h>
/* cligen */
#include <cligen/cligen.h>
/* clicon */
#include <clixon/clixon.h>
/* Command line options to be passed to getopt(3) */
#define DATASTORE_OPTS "hDd:b:f:x:y:Y:"
/*! usage
*/
static void
usage(char *argv0)
{
fprintf(stderr, "usage:%s <options>* [<command>]\n"
"where options are\n"
"\t-h\t\tHelp\n"
"\t-D\t\tDebug\n"
"\t-d <db>\t\tDatabase name. Default: running. Alt: candidate,startup\n"
"\t-b <dir>\tDatabase directory. Mandatory\n"
"\t-f <fmt>\tDatabase format: xml or json\n"
"\t-x <xml>\tXML file. Alternative to put <xml> argument\n"
"\t-y <file>\tYang file. Mandatory\n"
"\t-Y <dir> \tYang dirs (can be several)\n"
"and command is either:\n"
"\tget [<xpath>]\n"
"\tmget <nr> [<xpath>]\n"
"\tput (merge|replace|create|delete|remove) [<xml>]\n"
"\tcopy <todb>\n"
"\tlock <pid>\n"
"\tunlock\n"
"\tunlock_all <pid>\n"
"\tislocked\n"
"\texists\n"
"\tdelete\n"
"\tinit\n"
,
argv0
);
exit(0);
}
int
main(int argc, char **argv)
{
int retval = -1;
int c;
clicon_handle h;
char *argv0;
char *db = "running";
char *cmd = NULL;
yang_stmt *yspec = NULL;
char *yangfilename = NULL;
char *xmlfilename = NULL;
char *dbdir = NULL;
int ret;
uint32_t id;
enum operation_type op;
cxobj *xt = NULL;
int i;
char *xpath;
cbuf *cbret = NULL;
int dbg = 0;
cxobj *xerr = NULL;
cxobj *xcfg = NULL;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
argv0 = argv[0];
/* Defaults */
if ((h = clicon_handle_init()) == NULL)
goto done;
if ((xcfg = xml_new("clixon-config", NULL, CX_ELMNT)) == NULL)
goto done;
if (clicon_conf_xml_set(h, xcfg) < 0)
goto done;
/* getopt in two steps, first find config-file before over-riding options. */
clicon_option_str_set(h, "CLICON_XMLDB_FORMAT", "xml"); /* default */
while ((c = getopt(argc, argv, DATASTORE_OPTS)) != -1)
switch (c) {
case '?' :
case 'h' : /* help */
usage(argv0);
break;
case 'D' : /* debug */
dbg = 1;
break;
case 'd': /* db symbolic: running|candidate|startup */
if (!optarg)
usage(argv0);
db = optarg;
break;
case 'b': /* db directory */
if (!optarg)
usage(argv0);
dbdir = optarg;
break;
case 'f': /* db format */
if (!optarg)
usage(argv0);
clicon_option_str_set(h, "CLICON_XMLDB_FORMAT", optarg);
break;
case 'x': /* XML file */
if (!optarg)
usage(argv0);
xmlfilename = optarg;
break;
case 'y': /* Yang file */
if (!optarg)
usage(argv0);
yangfilename = optarg;
break;
case 'Y':
if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0)
goto done;
break;
}
/*
* Logs, error and debug to stderr, set debug level
*/
clicon_log_init(__FILE__, dbg?LOG_DEBUG:LOG_INFO, CLICON_LOG_STDERR);
clixon_debug_init(dbg, NULL);
argc -= optind;
argv += optind;
if (argc < 1)
usage(argv0);
cmd = argv[0];
if (dbdir == NULL){
clicon_err(OE_DB, 0, "Missing dbdir -b option");
goto done;
}
if (yangfilename == NULL){
clicon_err(OE_YANG, 0, "Missing yang filename -y option");
goto done;
}
/* Connect to plugin to get a handle */
if (xmldb_connect(h) < 0)
goto done;
/* Create yang spec */
if ((yspec = yspec_new()) == NULL)
goto done;
/* Parse yang spec from given file */
if (yang_spec_parse_file(h, yangfilename, yspec) < 0)
goto done;
clicon_option_str_set(h, "CLICON_XMLDB_DIR", dbdir);
clicon_dbspec_yang_set(h, yspec);
if (strcmp(cmd, "get")==0){
if (argc != 1 && argc != 2)
usage(argv0);
if (argc==2)
xpath = argv[1];
else
xpath = "/";
if (xmldb_get(h, db, NULL, xpath, &xt) < 0)
goto done;
if (clixon_xml2file(stdout, xt, 0, 0, NULL, fprintf, 0, 0) < 0)
goto done;
fprintf(stdout, "\n");
if (xt){
xml_free(xt);
xt = NULL;
}
}
else if (strcmp(cmd, "mget")==0){
int nr;
if (argc != 2 && argc != 3)
usage(argv0);
nr = atoi(argv[1]);
if (argc==3)
xpath = argv[2];
else
xpath = "/";
for (i=0;i<nr;i++){
if (xmldb_get(h, db, NULL, xpath, &xt) < 0)
goto done;
if (xt == NULL){
clicon_err(OE_DB, 0, "xt is NULL");
goto done;
}
if (clixon_xml2file(stdout, xt, 0, 0, NULL, fprintf, 0, 0) < 0)
goto done;
if (xt){
xml_free(xt);
xt = NULL;
}
}
fprintf(stdout, "\n");
}
else if (strcmp(cmd, "put")==0){
if (argc == 2){
if (xmlfilename == NULL){
clicon_err(OE_DB, 0, "XML filename expected");
usage(argv0);
}
}
else if (argc != 3){
clicon_err(OE_DB, 0, "Unexpected nr of args: %d", argc);
usage(argv0);
}
if (xml_operation(argv[1], &op) < 0){
clicon_err(OE_DB, 0, "Unrecognized operation: %s", argv[1]);
usage(argv0);
}
if (argc == 2){
FILE *fp;
if ((fp = fopen(xmlfilename, "r")) == NULL){
clicon_err(OE_UNIX, errno, "fopen(%s)", xmlfilename);
goto done;
}
if (clixon_xml_parse_file(fp, YB_MODULE, yspec, &xt, NULL) < 0)
goto done;
fclose(fp);
}
else{
if ((ret = clixon_xml_parse_string(argv[2], YB_MODULE, yspec, &xt, &xerr)) < 0)
goto done;
if (ret == 0){
xml_print(stderr, xerr);
goto done;
}
}
if (xml_name_set(xt, NETCONF_INPUT_CONFIG) < 0)
goto done;
if ((cbret = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
goto done;
}
if ((ret = xmldb_put(h, db, op, xt, NULL, cbret)) < 0)
goto done;
}
else if (strcmp(cmd, "copy")==0){
if (argc != 2)
usage(argv0);
if (xmldb_copy(h, db, argv[1]) < 0)
goto done;
}
else if (strcmp(cmd, "lock")==0){
if (argc != 2)
usage(argv0);
id = atoi(argv[1]);
if (xmldb_lock(h, db, id) < 0)
goto done;
}
else if (strcmp(cmd, "unlock")==0){
if (argc != 1)
usage(argv0);
if (xmldb_unlock(h, db) < 0)
goto done;
}
else if (strcmp(cmd, "unlock_all")==0){
if (argc != 2)
usage(argv0);
id = atoi(argv[1]);
if (xmldb_unlock_all(h, id) < 0)
goto done;
}
else if (strcmp(cmd, "islocked")==0){
if (argc != 1)
usage(argv0);
if ((ret = xmldb_islocked(h, db)) < 0)
goto done;
fprintf(stdout, "islocked: %d\n", ret);
}
else if (strcmp(cmd, "exists")==0){
if (argc != 1)
usage(argv0);
if ((ret = xmldb_exists(h, db)) < 0)
goto done;
fprintf(stdout, "exists: %d\n", ret);
}
else if (strcmp(cmd, "delete")==0){
if (argc != 1)
usage(argv0);
if (xmldb_delete(h, db) < 0)
goto done;
}
else if (strcmp(cmd, "init")==0){
if (argc != 1)
usage(argv0);
if (xmldb_create(h, db) < 0)
goto done;
}
else{
clicon_err(OE_DB, 0, "Unrecognized command: %s", cmd);
usage(argv0);
}
if (xmldb_disconnect(h) < 0)
goto done;
retval = 0;
done:
if (xcfg)
xml_free(xcfg);
if (cbret)
cbuf_free(cbret);
if (xt)
xml_free(xt);
if (h)
clicon_handle_exit(h);
if (yspec)
ys_free(yspec);
return retval;
}

189
util/clixon_util_dispatcher.c
View file

@ -1,189 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2021-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Utility for testing path dispatcher
* Everything is run by options and order is significant which makes it a little special.
* For example:
* clixon_util_dispatcher -r -c / :
* Register cb1 with default path "/" and arg NULL, call with path /
* clixon_util_dispatcher -i 2 -p /foo -a bar -r -c /bar -c /fie
* Register cb2 with path "/foo" and arg bar, call with path /bar then /fie
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
#include "clixon/clixon_backend.h"
/* Command line options to be passed to getopt(3) */
#define DISPATCHER_OPTS "hD:a:i:p:rc:"
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \t Debug - print dispatch tree\n"
"\t-a <string>\t Argument to callback (default: NULL)\n"
"\t-i <int> \t Function index: 1..3 (default: 1)\n"
"\t-p <path> \t Registered path (default: /)\n"
"\t-r \t Register callback (based on -a/-i/-p setting)\n"
"\t-c <path> \t Call dispatcher with path\n",
argv0
);
exit(0);
}
/*! Function to handle a path
*
* @param[in] h Generic handler
* @param[in] xpath Registered XPath using canonical prefixes
* @param[in] userargs Per-call user arguments
* @param[in] arg Per-path user argument
*(
/ * Make a CB() macro to generate simple callbacks that just prints the path and arg
*/
#define CB(i) static int cb##i(void *h0, char *xpath, void *userargs, void *arg) { fprintf(stdout, "%s %s\n", __FUNCTION__, (char*)arg); return 0; }
CB(1)
CB(2)
int
main(int argc,
char **argv)
{
int retval = -1;
char *argv0 = argv[0];
int logdst = CLICON_LOG_STDERR;
int dbg = 0;
int c;
char *arg = NULL;
handler_function fn = cb1;
dispatcher_entry_t *htable = NULL;
int ret;
char *regpath = "/"; /* Register path */
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init("dispatcher", LOG_DEBUG, logdst);
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, DISPATCHER_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv0);
break;
case 'a' :
case 'i' :
case 'p' :
case 'r' :
case 'c' :
break;
default:
usage(argv[0]);
break;
}
/*
* Logs, error and debug to stderr or syslog, set debug level
*/
clicon_log_init("xpath", dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
/* Now rest of options */
opterr = 0;
optind = 1;
while ((c = getopt(argc, argv, DISPATCHER_OPTS)) != -1){
switch (c) {
case 'D' : /* debug */
break; /* see above */
case 'a' : /* arg string */
arg = optarg;
break;
case 'i' : /* dispatcher function: 1..3 */
switch (atoi(optarg)){
case 1: fn = cb1; break;
case 2: fn = cb2; break;
// case 3: fn = cb3; break;
}
break;
case 'p' : /* register path */
regpath = optarg;
break;
case 'r' :{ /* register callback based on -a/-i/-p*/
dispatcher_definition x = {regpath, fn, arg};
if (dispatcher_register_handler(&htable, &x) < 0)
goto done;
break;
}
case 'c':{ /* Execute a call using path */
char *path = optarg;
if ((ret = dispatcher_call_handlers(htable, NULL, path, NULL)) < 0)
goto done;
fprintf(stderr, "path:%s ret:%d\n", path, ret);
break;
}
default:
usage(argv[0]);
break;
}
}
if (dbg)
dispatcher_print(stderr, 0, htable);
dispatcher_free(htable);
retval = 0;
done:
return retval;
}

551
util/clixon_util_grpc.c
View file

@ -1,551 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
*
* HTTP2 + OPENSSL client integrated with clixon events
* Ubuntu package:
* apt install libnghttp2-dev
* Example run: clixon_util_ssl -H nghttp2.org
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <syslog.h>
#include <netdb.h> /* gethostbyname */
#include <arpa/inet.h> /* inet_pton */
#include <netinet/tcp.h> /* TCP_NODELAY */
#include <openssl/ssl.h>
#include <nghttp2/nghttp2.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
#define UTIL_GRPC_OPTS "hD:H:"
#define ARRLEN(x) (sizeof(x) / sizeof(x[0]))
/* User data handle to nghttp2 lib */
typedef struct {
int sd_s;
SSL *sd_ssl;
nghttp2_session *sd_session;
int32_t sd_stream_id;
} session_data;
#define MAKE_NV(NAME, VALUE, VALUELEN) \
{ \
(uint8_t *)NAME, (uint8_t *)VALUE, sizeof(NAME) - 1, VALUELEN, \
NGHTTP2_NV_FLAG_NONE \
}
#define MAKE_NV2(NAME, VALUE) \
{ \
(uint8_t *)NAME, (uint8_t *)VALUE, sizeof(NAME) - 1, sizeof(VALUE) - 1, \
NGHTTP2_NV_FLAG_NONE \
}
#if 1 /* DEBUG */
static void
print_header(const uint8_t *name,
size_t namelen,
const uint8_t *value,
size_t valuelen)
{
clixon_debug(CLIXON_DBG_DEFAULT, "%s %s", name, value);
}
/* Print HTTP headers to |f|. Please note that this function does not
take into account that header name and value are sequence of
octets, therefore they may contain non-printable characters. */
static void
print_headers(nghttp2_nv *nva,
size_t nvlen)
{
size_t i;
for (i = 0; i < nvlen; ++i)
print_header(nva[i].name, nva[i].namelen, nva[i].value, nva[i].valuelen);
}
#endif /* DEBUG */
/* Transmit the |data|, |length| bytes, to the network.
* type is: nghttp2_on_header_callback
*/
static ssize_t
send_callback(nghttp2_session *session,
const uint8_t *data,
size_t length,
int flags,
void *user_data)
{
session_data *sd = (session_data*)user_data;
int ret;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %zu:", __FUNCTION__, length);
#if 0
{
int i;
for (i=0; i<length; i++)
fprintf(stderr, "%02x", data[i]&255);
fprintf(stderr, "\n");
}
#endif
/* encrypt & send message */
if ((ret = SSL_write(sd->sd_ssl, data, length)) < 0)
return ret;
return ret;
}
/*!
*/
static int
on_frame_recv_callback(nghttp2_session *session,
const nghttp2_frame *frame,
void *user_data)
{
//session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d", __FUNCTION__, frame->hd.stream_id);
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE)
clixon_debug(CLIXON_DBG_DEFAULT, "All headers received %d", frame->hd.stream_id);
return 0;
}
/*!
*/
static int
on_data_chunk_recv_callback(nghttp2_session *session,
uint8_t flags,
int32_t stream_id,
const uint8_t *data,
size_t len,
void *user_data)
{
session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d", __FUNCTION__, stream_id);
if (sd->sd_session == session &&
sd->sd_stream_id == stream_id)
fwrite(data, 1, len, stdout); /* This is where data is printed */
return 0;
}
/*!
*/
static int
on_stream_close_callback(nghttp2_session *session,
int32_t stream_id,
nghttp2_error_code error_code,
void *user_data)
{
//session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
return 0;
}
/*!
*/
static int
on_header_callback(nghttp2_session *session,
const nghttp2_frame *frame,
const uint8_t *name,
size_t namelen,
const uint8_t *value,
size_t valuelen,
uint8_t flags,
void *user_data)
{
// session_data *sd = (session_data*)user_data;
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE){
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d:", __FUNCTION__, frame->hd.stream_id);
print_header(name, namelen, value, valuelen);
}
return 0;
}
/*!
*/
static int
on_begin_headers_callback(nghttp2_session *session,
const nghttp2_frame *frame,
void *user_data)
{
// session_data *sd = (session_data*)user_data;
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE)
clixon_debug(CLIXON_DBG_DEFAULT, "%s Response headers %d",
__FUNCTION__, frame->hd.stream_id);
return 0;
}
/*! Initialize callbacks
*/
static int
session_init(nghttp2_session **session,
session_data *sd)
{
nghttp2_session_callbacks *callbacks = NULL;
nghttp2_session_callbacks_new(&callbacks);
nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks,
on_frame_recv_callback);
nghttp2_session_callbacks_set_on_data_chunk_recv_callback(
callbacks, on_data_chunk_recv_callback);
nghttp2_session_callbacks_set_on_stream_close_callback(
callbacks, on_stream_close_callback);
nghttp2_session_callbacks_set_on_header_callback(callbacks,
on_header_callback);
nghttp2_session_callbacks_set_on_begin_headers_callback(
callbacks, on_begin_headers_callback);
nghttp2_session_client_new(session, callbacks, sd);
nghttp2_session_callbacks_del(callbacks);
return 0;
}
static int
send_client_connection_header(nghttp2_session *session)
{
int retval = -1;
nghttp2_settings_entry iv[1] = {
{NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}};
int rv;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
/* client 24 bytes magic string will be sent by nghttp2 library */
rv = nghttp2_submit_settings(session, NGHTTP2_FLAG_NONE, iv, ARRLEN(iv));
if (rv != 0) {
clicon_err(OE_XML, 0, "Could not submit SETTINGS: %s", nghttp2_strerror(rv));
goto done;
}
retval = 0;
done:
return retval;
}
/*! Sets sd->sd_stream_id
*/
static int
submit_request(session_data *sd,
char *schema,
char *hostname,
char *path)
{
int retval = -1;
nghttp2_nv hdrs[] = {
MAKE_NV2(":method", "GET"),
MAKE_NV(":scheme", schema, strlen(schema)),
MAKE_NV(":authority", hostname, strlen(hostname)),
MAKE_NV(":path", path, strlen(path))
};
clixon_debug(CLIXON_DBG_DEFAULT, "%s Request headers:", __FUNCTION__);
print_headers(hdrs, ARRLEN(hdrs));
if ((sd->sd_stream_id = nghttp2_submit_request(sd->sd_session,
NULL,
hdrs,
ARRLEN(hdrs),
NULL,
NULL)) < 0){
clicon_err(OE_XML, 0, "Could not submit HTTP request: %s",
nghttp2_strerror(sd->sd_stream_id));
goto done;
}
retval = 0;
done:
return retval;
}
static int
socket_connect_inet(char *hostname,
uint16_t port,
int *sock0)
{
int retval = -1;
int s = -1;
struct sockaddr_in addr;
int ret;
struct hostent *host;
int one = 1;
clixon_debug(CLIXON_DBG_DEFAULT, "%s to %s:%hu", __FUNCTION__, hostname, port);
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
if ((ret = inet_pton(addr.sin_family, hostname, &addr.sin_addr)) < 0){
clicon_err(OE_UNIX, errno, "inet_pton");
goto done;
}
if (ret == 0){ /* Try DNS NOTE OBSOLETE */
if ((host = gethostbyname(hostname)) == NULL){
clicon_err(OE_UNIX, errno, "gethostbyname");
goto done;
}
addr.sin_addr.s_addr = *(long*)(host->h_addr); /* XXX Just to get it to work */
}
/* special error handling to get understandable messages (otherwise ENOENT) */
if ((s = socket(addr.sin_family, SOCK_STREAM, 0)) < 0) {
clicon_err(OE_CFG, errno, "socket");
return -1;
}
if (connect(s, (struct sockaddr*)&addr, sizeof(addr)) < 0){
clicon_err(OE_CFG, errno, "connecting socket inet4");
close(s);
goto done;
}
/* libev requires this */
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, (char *)&one, sizeof(one));
if (sock0 != NULL)
*sock0 = s;
retval = 0;
done:
if (sock0 == NULL && s >= 0)
close(s);
return retval;
}
/* NPN TLS extension client callback. We check that server advertised
the HTTP/2 protocol the nghttp2 library supports. If not, exit
the program. */
static int
select_next_proto_cb(SSL *ssl,
unsigned char **out,
unsigned char *outlen,
const unsigned char *in,
unsigned int inlen,
void *arg)
{
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
if (nghttp2_select_next_protocol(out, outlen, in, inlen) <= 0)
return -1;
clixon_debug(CLIXON_DBG_DEFAULT, "%s out: %s in:%s", __FUNCTION__, *out, in);
return SSL_TLSEXT_ERR_OK;
}
static SSL_CTX*
InitCTX(void)
{
const SSL_METHOD *method;
SSL_CTX *ctx;
#if 1
method = SSLv23_client_method();
#else
OpenSSL_add_all_algorithms(); /* Load cryptos, et.al. */
SSL_load_error_strings(); /* Bring in and register error messages */
method = TLSv1_2_client_method(); /* Create new client-method instance */
#endif
/* Create new context */
if ((ctx = SSL_CTX_new(method)) == NULL){
clicon_err(OE_XML, errno, "SSL_CTX_new");
goto done;
}
SSL_CTX_set_options(ctx,
SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 |
SSL_OP_NO_COMPRESSION |
SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
#ifndef OPENSSL_NO_NEXTPROTONEG
SSL_CTX_set_next_proto_select_cb(ctx, select_next_proto_cb, NULL);
#endif
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
SSL_CTX_set_alpn_protos(ctx, (const unsigned char *)"\x02h2", 3);
#endif
done:
return ctx;
}
static int
ssl_input_cb(int s,
void *arg)
{
int retval = -1;
session_data *sd = (session_data *)arg;
SSL *ssl;
char buf[1024];
int n;
nghttp2_session *session;
int readlen;
ssl = sd->sd_ssl;
session = sd->sd_session;
/* get reply & decrypt */
if ((n = SSL_read(ssl, buf, sizeof(buf))) < 0){
clicon_err(OE_XML, errno, "SSL_read");
goto done;
}
if (n == 0){
fprintf(stdout, "%s closed\n", __FUNCTION__);
goto done;
}
if ((readlen = nghttp2_session_mem_recv(session, (unsigned char*)buf, n)) < 0){
clicon_err(OE_XML, errno, "nghttp2_session_mem_recv");
goto done;
}
nghttp2_session_send(session);
#if 0
buf[n] = 0;
fprintf(stdout, "%s\n", buf);
#endif
retval = 0;
done:
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] with xml on stdin\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-H <hostname> \tURI hostname\n"
,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
clicon_handle h;
int c;
char *hostname = NULL;
uint16_t port = 443;
SSL_CTX *ctx = NULL;
int ss = -1;
SSL *ssl;
int ret;
nghttp2_session *session = NULL;
session_data *sd = NULL;
int dbg = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
if ((h = clicon_handle_init()) == NULL)
goto done;
while ((c = getopt(argc, argv, UTIL_GRPC_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'H': /* hostname */
hostname = optarg;
break;
default:
usage(argv[0]);
break;
}
if (hostname == NULL){
fprintf(stderr, "-H <hostname> is mandatory\n");
usage(argv[0]);
}
clixon_debug_init(dbg, NULL);
SSL_library_init();
if ((ctx = InitCTX()) == NULL)
goto done;
if (socket_connect_inet(hostname, port, &ss) < 0)
goto done;
ssl = SSL_new(ctx); /* create new SSL connection state */
SSL_set_fd(ssl, ss); /* attach the socket descriptor */
/* perform the connection */
if ((ret = SSL_connect(ssl)) < 0){
clicon_err(OE_XML, errno, "SSL_connect");
goto done;
}
/* In the nghttp2 code, there is an asynchronous step for
* a connected socket, here I just assume it is connected. */
if ((sd = malloc(sizeof(*sd))) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
goto done;
}
memset(sd, 0, sizeof(*sd));
sd->sd_s = ss;
sd->sd_ssl = ssl;
if (session_init(&session, sd) < 0)
goto done;
sd->sd_session = session;
if (send_client_connection_header(session) < 0)
goto done;
if (submit_request(sd, "https", hostname, "/") < 0)
goto done;
if (nghttp2_session_send(session) != 0){
clicon_err(OE_XML, errno, "nghttp2_session_send");
goto done;
}
if (clixon_event_reg_fd(ss, ssl_input_cb, sd, "ssl socket") < 0)
goto done;
if (clixon_event_loop(h) < 0)
goto done;
retval = 0;
done:
if (ss != -1)
close(ss);
if (ctx)
SSL_CTX_free(ctx); /* release context */
return retval;
}

165
util/clixon_util_json.c
View file

@ -1,165 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* JSON utility command
* @see http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
* and RFC 7951 JSON Encoding of Data Modeled with YANG
* and RFC 8259 The JavaScript Object Notation (JSON) Data Interchange Format
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <ctype.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <signal.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/*
* JSON parse and pretty print test program
* Usage: xpath
* read json from input
* Example compile:
gcc -g -o json -I. -I../clixon ./clixon_json.c -lclixon -lcligen
* Example run:
echo '{"foo": -23}' | ./json
*/
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] JSON as input on stdin\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-j \t\tOutput as JSON (default is as XML)\n"
"\t-l <s|e|o> \tLog on (s)yslog, std(e)rr, std(o)ut (stderr is default)\n"
"\t-p \t\tPretty-print output\n"
"\t-y <filename> \tyang filename to parse (must be stand-alone)\n" ,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
cxobj *xt = NULL;
cbuf *cb = cbuf_new();
int c;
int logdst = CLICON_LOG_STDERR;
int json = 0;
char *yang_filename = NULL;
yang_stmt *yspec = NULL;
cxobj *xerr = NULL; /* malloced must be freed */
int ret;
int pretty = 0;
int dbg = 0;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, "hD:jl:py:")) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'j':
json++;
break;
case 'l': /* Log destination: s|e|o|f */
if ((logdst = clicon_log_opt(optarg[0])) < 0)
usage(argv[0]);
break;
case 'p':
pretty++;
break;
case 'y':
yang_filename = optarg;
break;
default:
usage(argv[0]);
break;
}
clicon_log_init(__FILE__, dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
if (yang_filename){
if ((yspec = yspec_new()) == NULL)
goto done;
if (yang_parse_filename(NULL, yang_filename, yspec) == NULL){
fprintf(stderr, "yang parse error %s\n", clicon_err_reason);
return -1;
}
}
if ((ret = clixon_json_parse_file(stdin, yspec?1:0, yspec?YB_MODULE:YB_NONE, yspec, &xt, &xerr)) < 0)
goto done;
if (ret == 0){
xml_print(stderr, xerr);
goto done;
}
if (json){
if (clixon_json2cbuf(cb, xt, pretty, 1, 0) < 0)
goto done;
}
else if (clixon_xml2cbuf(cb, xt, 0, pretty, NULL, -1, 1) < 0)
goto done;
fprintf(stdout, "%s", cbuf_get(cb));
fflush(stdout);
retval = 0;
done:
if (yspec)
ys_free(yspec);
if (xt)
xml_free(xt);
if (cb)
cbuf_free(cb);
return retval;
}

306
util/clixon_util_path.c
View file

@ -1,306 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* "api-path" is "URI-encoded path expression" definition in RFC8040 3.5.3
* "Instance-identifier" is a subset of XML Xpaths and defined in Yang, used in NACM for example.
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/* Command line options to be passed to getopt(3) */
#define UTIL_PATH_OPTS "hD:f:ap:y:Y:n:"
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f <file> \tXML file\n"
"\t-a \t\tUse API-PATH (default INSTANCE-ID)\n"
"\t-p <xpath> \tPATH string\n"
"\t-y <filename> \tYang filename or dir (load all files)\n"
"\t-Y <dir> \tYang dirs (can be several)\n"
"\t-n <n> \tRepeat the call n times(for profiling)\n"
"and the following extra rules:\n"
"\tif -f is not given, XML input is expected on stdin\n"
"\tif -p is not given, <path> is expected as the first line on stdin\n"
"This means that with no arguments, <api-path> and XML is expected on stdin.\n",
argv0
);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
char *argv0 = argv[0];
int i;
cxobj *x = NULL;
cxobj *xc;
cxobj **xvec = NULL;
int xlen = 0;
int c;
int len;
char *buf = NULL;
int ret;
FILE *fp = stdin; /* unless overriden by argv[1] */
char *yang_file_dir = NULL;
yang_stmt *yspec = NULL;
char *path = NULL;
char *filename;
cbuf *cb = NULL;
int api_path_p = 0; /* api-path or instance-id */
clicon_handle h;
struct stat st;
cxobj *xcfg = NULL;
cxobj *xerr = NULL; /* malloced must be freed */
int nr = 1;
int dbg = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init("api-path", LOG_DEBUG, CLICON_LOG_STDERR);
/* Initialize clixon handle */
if ((h = clicon_handle_init()) == NULL)
goto done;
/* Initialize config tree (needed for -Y below) */
if ((xcfg = xml_new("clixon-config", NULL, CX_ELMNT)) == NULL)
goto done;
if (clicon_conf_xml_set(h, xcfg) < 0)
goto done;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_PATH_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv0);
break;
case 'f': /* XML file */
filename = optarg;
if ((fp = fopen(filename, "r")) == NULL){
clicon_err(OE_UNIX, errno, "fopen(%s)", optarg);
goto done;
}
break;
case 'a': /* API-PATH instead of INSTANCE-ID */
api_path_p++;
break;
case 'p': /* API-PATH string */
path = optarg;
break;
case 'y':
yang_file_dir = optarg;
break;
case 'Y':
if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0)
goto done;
break;
case 'n':
nr = atoi(optarg);
break;
default:
usage(argv[0]);
break;
}
clixon_debug_init(dbg, NULL);
yang_init(h);
/* Parse yang */
if (yang_file_dir){
if ((yspec = yspec_new()) == NULL)
goto done;
if (stat(yang_file_dir, &st) < 0){
clicon_err(OE_YANG, errno, "%s not found", yang_file_dir);
goto done;
}
if (S_ISDIR(st.st_mode)){
if (yang_spec_load_dir(h, yang_file_dir, yspec) < 0)
goto done;
}
else{
if (yang_spec_parse_file(h, yang_file_dir, yspec) < 0)
goto done;
}
}
if (path==NULL){
/* First read api-path from file */
len = 1024; /* any number is fine */
if ((buf = malloc(len)) == NULL){
perror("pt_file malloc");
return -1;
}
memset(buf, 0, len);
i = 0;
while (1){
if ((ret = read(0, &c, 1)) < 0){
perror("read");
goto done;
}
if (ret == 0)
break;
if (c == '\n')
break;
if (len==i){
if ((buf = realloc(buf, 2*len)) == NULL){
fprintf(stderr, "%s: realloc: %s\n", __FUNCTION__, strerror(errno));
return -1;
}
memset(buf+len, 0, len);
len *= 2;
}
buf[i++] = (char)(c&0xff);
}
path = buf;
}
/*
* If fp=stdin, then continue reading from stdin (after CR)
* XXX Note 0 above, stdin here
*/
if (clixon_xml_parse_file(fp, YB_NONE, NULL, &x, NULL) < 0){
fprintf(stderr, "Error: parsing: %s\n", clicon_err_reason);
return -1;
}
/* Validate XML as well */
if (yang_file_dir){
/* Populate */
if ((ret = xml_bind_yang(h, x, YB_MODULE, yspec, &xerr)) < 0)
goto done;
if (ret == 0){
if ((cb = cbuf_new()) ==NULL){
clicon_err(OE_XML, errno, "cbuf_new");
goto done;
}
if (netconf_err2cb(h, xerr, cb) < 0)
goto done;
fprintf(stderr, "xml validation error: %s\n", cbuf_get(cb));
goto done;
}
/* sort */
if (xml_sort_recurse(x) < 0)
goto done;
if (xml_apply0(x, -1, xml_sort_verify, h) < 0)
clicon_log(LOG_NOTICE, "%s: sort verify failed", __FUNCTION__);
/* Add default values */
if (xml_default_recurse(x, 0) < 0)
goto done;
if ((ret = xml_yang_validate_all_top(h, x, &xerr)) < 0)
goto done;
if (ret > 0 && (ret = xml_yang_validate_add(h, x, &xerr)) < 0)
goto done;
if (ret == 0){
if ((cb = cbuf_new()) ==NULL){
clicon_err(OE_XML, errno, "cbuf_new");
goto done;
}
if (netconf_err2cb(h, xerr, cb) < 0)
goto done;
fprintf(stderr, "xml validation error: %s\n", cbuf_get(cb));
goto done;
}
}
/* Repeat for performance profiling (default is nr = 1) */
xvec = NULL;
for (i=0; i<nr; i++){
if (api_path_p){
if ((ret = clixon_xml_find_api_path(x, yspec, &xvec, &xlen, "%s", path)) < 0)
goto done;
}
else{
if ((ret = clixon_xml_find_instance_id(x, yspec, &xvec, &xlen, "%s", path)) < 0)
goto done;
}
if (ret == 0){
fprintf(stderr, "Fail %d %s\n", clicon_errno, clicon_err_reason);
goto done;
}
}
/* Print results */
for (i = 0; i < xlen; i++){
xc = xvec[i];
fprintf(stdout, "%d: ", i);
clixon_xml2file(stdout, xc, 0, 0, NULL, fprintf, 0, 0);
fputc('\n', stdout);
fflush(stdout);
}
retval = 0;
done:
if (yspec != NULL)
ys_free(yspec);
if (cb)
cbuf_free(cb);
if (xvec)
free(xvec);
if (buf)
free(buf);
if (x)
xml_free(x);
if (xcfg)
xml_free(xcfg);
if (fp)
fclose(fp);
if (h)
clicon_handle_exit(h);
return retval;
}

237
util/clixon_util_regexp.c
View file

@ -1,237 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Utility for compiling regexp and checking validity
* gcc -I /usr/include/libxml2 regex.c -o regex -lxml2
* @see http://www.w3.org/TR/2004/REC-xmlschema-2-20041028
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <unistd.h> /* unistd */
#include <string.h>
#include <regex.h> /* posix regex */
#include <syslog.h>
#include <stdlib.h>
#include <limits.h>
#include <signal.h>
#ifdef HAVE_LIBXML2 /* Actually it should check for a header file */
#include <libxml/xmlregexp.h>
#endif
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/*! libxml2 regex implementation
*
* @see http://www.w3.org/TR/2004/REC-xmlschema-2-20041028
* @retval 1 Match
* @retval 0 Not match
* @retval -1 Error
*/
static int
regex_libxml2(char *regexp0,
char *content0,
int nr,
int debug)
{
int retval = -1;
#ifdef HAVE_LIBXML2
xmlChar *regexp = (xmlChar*)regexp0;
xmlChar *content = (xmlChar*)content0;
xmlRegexp *xrp = NULL;
int ret;
int i;
if ((xrp = xmlRegexpCompile(regexp)) == NULL)
goto done;
if (nr==0)
return 1;
for (i=0; i<nr; i++)
if ((ret = xmlRegexpExec(xrp, content)) < 0)
goto done;
return ret;
done:
#endif
return retval;
}
static int
regex_posix(char *regexp,
char *content,
int nr,
int debug)
{
int retval = -1;
char *posix = NULL;
char pattern[1024];
int status = 0;
regex_t re;
char errbuf[1024];
int len0;
int i;
if (regexp_xsd2posix(regexp, &posix) < 0)
goto done;
clixon_debug(CLIXON_DBG_DEFAULT, "posix: %s", posix);
len0 = strlen(posix);
if (len0 > sizeof(pattern)-5){
fprintf(stderr, "pattern too long\n");
return -1;
}
/* note following two lines trigger [-Wstringop-truncation] warnings, but see no actual error */
strncpy(pattern, "^(", 3);
strncpy(pattern+2, posix, sizeof(pattern)-3);
strncat(pattern, ")$", sizeof(pattern)-len0-1);
if (regcomp(&re, pattern, REG_NOSUB|REG_EXTENDED) != 0)
return(0); /* report error */
if (nr==0)
return 1;
for (i=0; i<nr; i++)
status = regexec(&re, content, (size_t) 0, NULL, 0);
regfree(&re);
if (status != 0) {
regerror(status, &re, errbuf, sizeof(errbuf)); /* XXX error is ignored */
return(0); /* report error */
}
return(1);
done:
if (posix)
free(posix);
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level>\tDebug\n"
"\t-p \txsd->posix translation regexp (default)\n"
"\t-x \tlibxml2 regexp (alternative to -p)\n"
"\t-n <nr> \tIterate content match (default: 1, 0: no match only compile)\n"
"\t-r <regexp> \tregexp (mandatory)\n"
"\t-c <string> \tValue content string(mandatory if -n > 0)\n",
argv0
);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
char *argv0 = argv[0];
int c;
char *regexp = NULL;
char *content = NULL;
int ret = 0;
int nr = 1;
int mode = 0; /* 0 is posix, 1 is libxml */
int dbg = 0;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, "hD:pxn:r:c:")) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv0);
break;
case 'p': /* xsd->posix */
mode = 0;
break;
case 'n': /* Number of iterations */
if ((nr = atoi(optarg)) < 0)
usage(argv0);
break;
case 'x': /* libxml2 */
mode = 1;
break;
case 'r': /* regexp */
regexp = optarg;
break;
case 'c': /* value content string */
content = optarg;
break;
default:
usage(argv[0]);
break;
}
clicon_log_init(__FILE__, dbg?LOG_DEBUG:LOG_INFO, CLICON_LOG_STDERR);
clixon_debug_init(dbg, NULL);
if (regexp == NULL){
fprintf(stderr, "-r mandatory\n");
usage(argv0);
}
if (nr > 0 && content == NULL){
fprintf(stderr, "-c mandatory (if -n > 0)\n");
usage(argv0);
}
if (mode != 0 && mode != 1){
fprintf(stderr, "Neither posix or libxml2 set\n");
usage(argv0);
}
clixon_debug(CLIXON_DBG_DEFAULT, "regexp:%s", regexp);
clixon_debug(CLIXON_DBG_DEFAULT, "content:%s", content);
if (mode == 0){
if ((ret = regex_posix(regexp, content, nr, dbg)) < 0)
goto done;
}
else if (mode == 1){
if ((ret = regex_libxml2(regexp, content, nr, dbg)) < 0)
goto done;
}
else
usage(argv0);
fprintf(stdout, "%d\n", ret);
exit(ret);
retval = 0;
done:
return retval;
}

198
util/clixon_util_socket.c
View file

@ -1,198 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Unit test for testting the backend socket, ie simulating a client by
* directly sending XML to the backend.
* Precondition:
* The backend must have been started using socket path goven as -s
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/time.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] with xml on stdin (unless -f)\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-a <family>\tSocket address family (default UNIX)\n"
"\t-s <sockpath> \tPath to unix domain socket (or IP addr)\n"
"\t-f <file>\tXML input file (overrides stdin)\n"
"\t-J \t\tInput as JSON (instead of XML)\n"
,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
int c;
int logdst = CLICON_LOG_STDERR;
struct clicon_msg *msg = NULL;
char *sockpath = NULL;
char *retdata = NULL;
int jsonin = 0;
char *input_filename = NULL;
FILE *fp = stdin;
cxobj *xt = NULL;
cxobj *xc;
cxobj *xerr = NULL;
char *family = "UNIX";
int ret;
cbuf *cb = cbuf_new();
clicon_handle h;
int dbg = 0;
int s;
int eof = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
if ((h = clicon_handle_init()) == NULL)
goto done;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, "hD:s:f:Ja:")) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 's':
sockpath = optarg;
break;
case 'f':
input_filename = optarg;
break;
case 'J':
jsonin++;
break;
case 'a':
family = optarg;
break;
default:
usage(argv[0]);
break;
}
clicon_log_init(__FILE__, dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
if (sockpath == NULL){
fprintf(stderr, "Mandatory option missing: -s <sockpath>\n");
usage(argv[0]);
}
if (input_filename){
if ((fp = fopen(input_filename, "r")) == NULL){
clicon_err(OE_YANG, errno, "open(%s)", input_filename);
goto done;
}
}
/* 2. Parse data (xml/json) */
if (jsonin){
if ((ret = clixon_json_parse_file(fp, 0, YB_NONE, NULL, &xt, &xerr)) < 0)
goto done;
if (ret == 0){
fprintf(stderr, "Invalid JSON\n");
goto done;
}
}
else{
if (clixon_xml_parse_file(fp, YB_NONE, NULL, &xt, NULL) < 0){
fprintf(stderr, "xml parse error: %s\n", clicon_err_reason);
goto done;
}
}
if ((xc = xml_child_i(xt, 0)) == NULL){
fprintf(stderr, "No xml\n");
goto done;
}
if (clixon_xml2cbuf(cb, xc, 0, 0, NULL, -1, 0) < 0)
goto done;
if ((msg = clicon_msg_encode(getpid(), "%s", cbuf_get(cb))) < 0)
goto done;
if (strcmp(family, "UNIX")==0){
if (clicon_rpc_connect_unix(h, sockpath, &s) < 0)
goto done;
}
else
if (clicon_rpc_connect_inet(h, sockpath, 4535, &s) < 0)
goto done;
if (clicon_rpc(s, NULL, msg, &retdata, &eof) < 0)
goto done;
close(s);
fprintf(stdout, "%s\n", retdata);
retval = 0;
done:
if (fp)
fclose(fp);
if (xerr)
xml_free(xerr);
if (xt)
xml_free(xt);
if (msg)
free(msg);
if (cb)
cbuf_free(cb);
return retval;
}

551
util/clixon_util_ssl.c
View file

@ -1,551 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Actually HTTP2 + OPENSSL client integrated with clixon events
* Ubuntu package:
* apt install libnghttp2-dev
* Example run: clixon_util_ssl -H nghttp2.org
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <syslog.h>
#include <signal.h>
#include <netdb.h> /* gethostbyname */
#include <arpa/inet.h> /* inet_pton */
#include <netinet/tcp.h> /* TCP_NODELAY */
#include <openssl/ssl.h>
#ifdef HAVE_LIBNGHTTP2
#include <nghttp2/nghttp2.h>
#endif
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
#define UTIL_SSL_OPTS "hD:H:"
#define ARRLEN(x) (sizeof(x) / sizeof(x[0]))
/* User data handle to nghttp2 lib */
typedef struct {
int sd_s;
SSL *sd_ssl;
nghttp2_session *sd_session;
int32_t sd_stream_id;
} session_data;
#define MAKE_NV(NAME, VALUE, VALUELEN) \
{ \
(uint8_t *)NAME, (uint8_t *)VALUE, sizeof(NAME) - 1, VALUELEN, \
NGHTTP2_NV_FLAG_NONE \
}
#define MAKE_NV2(NAME, VALUE) \
{ \
(uint8_t *)NAME, (uint8_t *)VALUE, sizeof(NAME) - 1, sizeof(VALUE) - 1, \
NGHTTP2_NV_FLAG_NONE \
}
#if 1 /* DEBUG */
static void
print_header(const uint8_t *name,
size_t namelen,
const uint8_t *value,
size_t valuelen)
{
clixon_debug(CLIXON_DBG_DEFAULT, "%s %s", name, value);
}
/* Print HTTP headers to |f|. Please note that this function does not
take into account that header name and value are sequence of
octets, therefore they may contain non-printable characters. */
static void
print_headers(nghttp2_nv *nva,
size_t nvlen)
{
size_t i;
for (i = 0; i < nvlen; ++i)
print_header(nva[i].name, nva[i].namelen, nva[i].value, nva[i].valuelen);
}
#endif /* DEBUG */
/* Transmit the |data|, |length| bytes, to the network.
* type is: nghttp2_on_header_callback
*/
static ssize_t
send_callback(nghttp2_session *session,
const uint8_t *data,
size_t length,
int flags,
void *user_data)
{
session_data *sd = (session_data*)user_data;
int ret;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %zu:", __FUNCTION__, length);
#if 0
{
int i;
for (i=0; i<length; i++)
fprintf(stderr, "%02x", data[i]&255);
fprintf(stderr, "\n");
}
#endif
/* encrypt & send message */
if ((ret = SSL_write(sd->sd_ssl, data, length)) < 0)
return ret;
return ret;
}
/*!
*/
static int
on_frame_recv_callback(nghttp2_session *session,
const nghttp2_frame *frame,
void *user_data)
{
//session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d", __FUNCTION__, frame->hd.stream_id);
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE)
clixon_debug(CLIXON_DBG_DEFAULT, "All headers received %d", frame->hd.stream_id);
return 0;
}
/*!
*/
static int
on_data_chunk_recv_callback(nghttp2_session *session,
uint8_t flags,
int32_t stream_id,
const uint8_t *data,
size_t len,
void *user_data)
{
session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d", __FUNCTION__, stream_id);
if (sd->sd_session == session &&
sd->sd_stream_id == stream_id)
fwrite(data, 1, len, stdout); /* This is where data is printed */
return 0;
}
/*!
*/
static int
on_stream_close_callback(nghttp2_session *session,
int32_t stream_id,
nghttp2_error_code error_code,
void *user_data)
{
//session_data *sd = (session_data*)user_data;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
return 0;
}
/*!
*/
static int
on_header_callback(nghttp2_session *session,
const nghttp2_frame *frame,
const uint8_t *name,
size_t namelen,
const uint8_t *value,
size_t valuelen,
uint8_t flags,
void *user_data)
{
// session_data *sd = (session_data*)user_data;
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE){
clixon_debug(CLIXON_DBG_DEFAULT, "%s %d:", __FUNCTION__, frame->hd.stream_id);
print_header(name, namelen, value, valuelen);
}
return 0;
}
/*!
*/
static int
on_begin_headers_callback(nghttp2_session *session,
const nghttp2_frame *frame,
void *user_data)
{
// session_data *sd = (session_data*)user_data;
if (frame->hd.type == NGHTTP2_HEADERS &&
frame->headers.cat == NGHTTP2_HCAT_RESPONSE)
clixon_debug(CLIXON_DBG_DEFAULT, "%s Response headers %d",
__FUNCTION__, frame->hd.stream_id);
return 0;
}
/*! Initialize callbacks
*/
static int
session_init(nghttp2_session **session,
session_data *sd)
{
nghttp2_session_callbacks *callbacks = NULL;
nghttp2_session_callbacks_new(&callbacks);
nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks,
on_frame_recv_callback);
nghttp2_session_callbacks_set_on_data_chunk_recv_callback(
callbacks, on_data_chunk_recv_callback);
nghttp2_session_callbacks_set_on_stream_close_callback(
callbacks, on_stream_close_callback);
nghttp2_session_callbacks_set_on_header_callback(callbacks,
on_header_callback);
nghttp2_session_callbacks_set_on_begin_headers_callback(
callbacks, on_begin_headers_callback);
nghttp2_session_client_new(session, callbacks, sd);
nghttp2_session_callbacks_del(callbacks);
return 0;
}
static int
send_client_connection_header(nghttp2_session *session)
{
int retval = -1;
nghttp2_settings_entry iv[1] = {
{NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}};
int rv;
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
/* client 24 bytes magic string will be sent by nghttp2 library */
rv = nghttp2_submit_settings(session, NGHTTP2_FLAG_NONE, iv, ARRLEN(iv));
if (rv != 0) {
clicon_err(OE_XML, 0, "Could not submit SETTINGS: %s", nghttp2_strerror(rv));
goto done;
}
retval = 0;
done:
return retval;
}
/*! Sets sd->sd_stream_id
*/
static int
submit_request(session_data *sd,
char *schema,
char *hostname,
char *path)
{
int retval = -1;
nghttp2_nv hdrs[] = {
MAKE_NV2(":method", "GET"),
MAKE_NV(":scheme", schema, strlen(schema)),
MAKE_NV(":authority", hostname, strlen(hostname)),
MAKE_NV(":path", path, strlen(path))
};
clixon_debug(CLIXON_DBG_DEFAULT, "%s Request headers:", __FUNCTION__);
print_headers(hdrs, ARRLEN(hdrs));
if ((sd->sd_stream_id = nghttp2_submit_request(sd->sd_session,
NULL,
hdrs,
ARRLEN(hdrs),
NULL,
NULL)) < 0){
clicon_err(OE_XML, 0, "Could not submit HTTP request: %s",
nghttp2_strerror(sd->sd_stream_id));
goto done;
}
retval = 0;
done:
return retval;
}
static int
socket_connect_inet(char *hostname,
uint16_t port,
int *sock0)
{
int retval = -1;
int s = -1;
struct sockaddr_in addr;
int ret;
struct hostent *host;
int one = 1;
clixon_debug(CLIXON_DBG_DEFAULT, "%s to %s:%hu", __FUNCTION__, hostname, port);
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
if ((ret = inet_pton(addr.sin_family, hostname, &addr.sin_addr)) < 0){
clicon_err(OE_UNIX, errno, "inet_pton");
goto done;
}
if (ret == 0){ /* Try DNS NOTE OBSOLETE */
if ((host = gethostbyname(hostname)) == NULL){
clicon_err(OE_UNIX, errno, "gethostbyname");
goto done;
}
addr.sin_addr.s_addr = *(long*)(host->h_addr); /* XXX Just to get it to work */
}
/* special error handling to get understandable messages (otherwise ENOENT) */
if ((s = socket(addr.sin_family, SOCK_STREAM, 0)) < 0) {
clicon_err(OE_CFG, errno, "socket");
return -1;
}
if (connect(s, (struct sockaddr*)&addr, sizeof(addr)) < 0){
clicon_err(OE_CFG, errno, "connecting socket inet4");
close(s);
goto done;
}
/* libev requires this */
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, (char *)&one, sizeof(one));
if (sock0 != NULL)
*sock0 = s;
retval = 0;
done:
if (sock0 == NULL && s >= 0)
close(s);
return retval;
}
/* NPN TLS extension client callback. We check that server advertised
the HTTP/2 protocol the nghttp2 library supports. If not, exit
the program. */
static int
select_next_proto_cb(SSL *ssl,
unsigned char **out,
unsigned char *outlen,
const unsigned char *in,
unsigned int inlen,
void *arg)
{
clixon_debug(CLIXON_DBG_DEFAULT, "%s", __FUNCTION__);
if (nghttp2_select_next_protocol(out, outlen, in, inlen) <= 0)
return -1;
clixon_debug(CLIXON_DBG_DEFAULT, "%s out: %s in:%s", __FUNCTION__, *out, in);
return SSL_TLSEXT_ERR_OK;
}
static SSL_CTX*
InitCTX(void)
{
const SSL_METHOD *method;
SSL_CTX *ctx;
#if 1
method = SSLv23_client_method();
#else
OpenSSL_add_all_algorithms(); /* Load cryptos, et.al. */
SSL_load_error_strings(); /* Bring in and register error messages */
method = TLSv1_2_client_method(); /* Create new client-method instance */
#endif
/* Create new context */
if ((ctx = SSL_CTX_new(method)) == NULL){
clicon_err(OE_XML, errno, "SSL_CTX_new");
goto done;
}
SSL_CTX_set_options(ctx,
SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 |
SSL_OP_NO_COMPRESSION |
SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
#ifndef OPENSSL_NO_NEXTPROTONEG
SSL_CTX_set_next_proto_select_cb(ctx, select_next_proto_cb, NULL);
#endif
#if OPENSSL_VERSION_NUMBER >= 0x10002000L
SSL_CTX_set_alpn_protos(ctx, (const unsigned char *)"\x02h2", 3);
#endif
done:
return ctx;
}
static int
ssl_input_cb(int s,
void *arg)
{
int retval = -1;
session_data *sd = (session_data *)arg;
SSL *ssl;
char buf[1024];
int n;
nghttp2_session *session;
int readlen;
ssl = sd->sd_ssl;
session = sd->sd_session;
/* get reply & decrypt */
if ((n = SSL_read(ssl, buf, sizeof(buf))) < 0){
clicon_err(OE_XML, errno, "SSL_read");
goto done;
}
if (n == 0){
fprintf(stdout, "%s closed\n", __FUNCTION__);
goto done;
}
if ((readlen = nghttp2_session_mem_recv(session, (unsigned char*)buf, n)) < 0){
clicon_err(OE_XML, errno, "nghttp2_session_mem_recv");
goto done;
}
nghttp2_session_send(session);
#if 0
buf[n] = 0;
fprintf(stdout, "%s\n", buf);
#endif
retval = 0;
done:
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] with xml on stdin\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-H <hostname> \tURI hostname\n"
,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
clicon_handle h;
int c;
char *hostname = NULL;
uint16_t port = 443;
SSL_CTX *ctx = NULL;
int ss = -1;
SSL *ssl;
int ret;
nghttp2_session *session = NULL;
session_data *sd = NULL;
int dbg = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
if ((h = clicon_handle_init()) == NULL)
goto done;
while ((c = getopt(argc, argv, UTIL_SSL_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'H': /* hostname */
hostname = optarg;
break;
default:
usage(argv[0]);
break;
}
if (hostname == NULL){
fprintf(stderr, "-H <hostname> is mandatory\n");
usage(argv[0]);
}
clixon_debug_init(dbg, NULL);
SSL_library_init();
if ((ctx = InitCTX()) == NULL)
goto done;
if (socket_connect_inet(hostname, port, &ss) < 0)
goto done;
ssl = SSL_new(ctx); /* create new SSL connection state */
SSL_set_fd(ssl, ss); /* attach the socket descriptor */
/* perform the connection */
if ((ret = SSL_connect(ssl)) < 0){
clicon_err(OE_XML, errno, "SSL_connect");
goto done;
}
/* In the nghttp2 code, there is an asynchronous step for
* a connected socket, here I just assume it is connected. */
if ((sd = malloc(sizeof(*sd))) == NULL){
clicon_err(OE_UNIX, errno, "malloc");
goto done;
}
memset(sd, 0, sizeof(*sd));
sd->sd_s = ss;
sd->sd_ssl = ssl;
if (session_init(&session, sd) < 0)
goto done;
sd->sd_session = session;
if (send_client_connection_header(session) < 0)
goto done;
if (submit_request(sd, "https", hostname, "/") < 0)
goto done;
if (nghttp2_session_send(session) != 0){
clicon_err(OE_XML, errno, "nghttp2_session_send");
goto done;
}
if (clixon_event_reg_fd(ss, ssl_input_cb, sd, "ssl socket") < 0)
goto done;
if (clixon_event_loop(h) < 0)
goto done;
retval = 0;
done:
if (ss != -1)
close(ss);
if (ctx)
SSL_CTX_free(ctx); /* release context */
return retval;
}

286
util/clixon_util_stream.c
View file

@ -1,286 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Stream restconf support functions.
* (Original in grideye)
* Example: clixon_util_stream -u http://localhost/streams/EXAMPLE -s 2018-10-21T19:22:16
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <signal.h>
#include <curl/curl.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/*
* Types (curl)
*/
struct curlbuf{
size_t b_len;
char *b_buf;
};
/*
* For the asynchronous case. I think we must handle the case where of many of these
* come in before we can handle them in the upper-level polling routine.
* realloc. Therefore, we append new data to the userdata buffer.
*/
static size_t
curl_get_cb(void *ptr,
size_t size,
size_t nmemb,
void *userdata)
{
struct curlbuf *buf = (struct curlbuf *)userdata;
int len;
len = size*nmemb;
if ((buf->b_buf = realloc(buf->b_buf, buf->b_len+len+1)) == NULL)
return 0;
memcpy(buf->b_buf+buf->b_len, ptr, len);
buf->b_len += len;
buf->b_buf[buf->b_len] = '\0';
// fprintf(stderr, "%s\n", buf->b_buf);
return len;
}
/*! Given an URL and data to post, do a (curl) get request with data.
*
* If getdata is set, return the (malloced) data (which should be freed).
*
* @param[in] start 'start-time' parameter that will be URL-encoded
* @retval 1 ok
* @retval -1 fatal error
*
* @note curl_easy_perform blocks
* @note New handle is created every time, the handle can be re-used for
* better TCP performance
*/
int
stream_url_get(char *url,
char *start,
char *stop,
int timeout,
char **getdata)
{
int retval = -1;
CURL *curl;
char *err = NULL;
char *encoded = NULL;
struct curlbuf cb = {0, };
cbuf *cbf = NULL;
struct curl_slist *list = NULL;
int ret;
clixon_debug(CLIXON_DBG_DEFAULT, "%s: curl -G %s start-time=%s stop-time=%s",
__FUNCTION__, url, start?start:"", stop?stop:"");
/* Set up curl for doing the communication with the controller */
if ((curl = curl_easy_init()) == NULL) {
clicon_err(OE_PLUGIN, errno, "curl_easy_init");
goto done;
}
if ((cbf = cbuf_new()) == NULL)
goto done;
if ((err = malloc(CURL_ERROR_SIZE)) == NULL) {
clixon_debug(CLIXON_DBG_DEFAULT, "%s: malloc", __FUNCTION__);
goto done;
}
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1L);
/* HEADERS */
list = curl_slist_append(list, "Accept: text/event-stream");
// list = curl_slist_append(list, "Cache-Control: no-cache");
// list = curl_slist_append(list, "Connection: keep-alive");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
/* specify URL to get */
cprintf(cbf, "%s", url);
if (strlen(start)||strlen(stop))
cprintf(cbf, "?");
if (strlen(start)){
if ((encoded = curl_easy_escape(curl, start, 0)) == NULL)
goto done;
cprintf(cbf, "start-time=%s", encoded);
curl_free(encoded);
encoded = NULL;
}
if (strlen(stop)){
if (strlen(start))
cprintf(cbf, "&");
if ((encoded = curl_easy_escape(curl, stop, 0)) == NULL)
goto done;
cprintf(cbf, "stop-time=%s", encoded);
curl_free(encoded);
encoded = NULL;
}
clixon_debug(CLIXON_DBG_DEFAULT, "url: %s\n", cbuf_get(cbf));
curl_easy_setopt(curl, CURLOPT_URL, cbuf_get(cbf));
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_get_cb);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &cb);
/* some servers don't like requests that are made without a user-agent
field, so we provide one */
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, err);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L);
ret = curl_easy_perform(curl);
if (ret != CURLE_OPERATION_TIMEDOUT && ret != CURLE_OK){
fprintf(stderr, "curl: %s %d", err, ret);
goto done;
}
if (getdata && cb.b_buf){
*getdata = cb.b_buf;
cb.b_buf = NULL;
}
retval = 1;
done:
if (cbf)
cbuf_free(cbf);
if (err)
free(err);
if (encoded)
curl_free(encoded);
if (cb.b_buf)
free(cb.b_buf);
if (curl)
curl_easy_cleanup(curl); /* cleanup */
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s <options>*\n"
"where options are:\n"
"\t-h\t\tHelp\n"
"\t-D <level>\tDebug level\n"
"\t-u <url>\tURL (mandatory)\n"
"\t-s <start>\tStart-time (format: 2018-10-21T19:22:16 OR +/-<x>s\n"
"\t-e <end>\tStop-time (same format as start)\n"
"\t-t <timeout>\tTimeout (default: 10)\n"
, argv0);
exit(0);
}
int
main(int argc, char **argv)
{
cbuf *cb = cbuf_new();
char *url = NULL;
char *getdata = NULL;
int timeout = 10;
char start[28] = {0,}; /* strlen = 0 */
char stop[28] = {0,};
int c;
char *argv0 = argv[0];
struct timeval now;
int dbg = 0;
clicon_log_init("xpath", LOG_DEBUG, CLICON_LOG_STDERR);
gettimeofday(&now, NULL);
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, "hDu:s:e:t:")) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
dbg = 1;
break;
case 'u': /* URL */
url = optarg;
break;
case 's': /* start-time */
if (*optarg == '+' || *optarg == '-'){
struct timeval t = now;
t.tv_sec += atoi(optarg);
if (time2str(&t, start, sizeof(start)) < 0)
goto done;
}
else
strcpy(start, optarg);
break;
case 'e': /* stop-time */
if (*optarg == '+' || *optarg == '-'){
struct timeval t = now;
t.tv_sec += atoi(optarg);
if (time2str(&t, stop, sizeof(stop)) < 0)
goto done;
}
else
strcpy(stop, optarg);
break;
case 't': /* timeout */
timeout = atoi(optarg);
break;
default:
usage(argv[0]);
break;
}
clixon_debug_init(dbg, NULL);
if (url == NULL)
usage(argv[0]);
curl_global_init(0);
if (stream_url_get(url, start, stop, timeout, &getdata) < 0)
goto done;
if (getdata)
fprintf(stdout, "%s", getdata);
fflush(stdout);
done:
curl_global_cleanup();
if (getdata)
free(getdata);
if (cb)
cbuf_free(cb);
return 0;
}

248
util/clixon_util_validate.c
View file

@ -1,248 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Utility to validate and/or commit as a single utility, to be used in eg shell scripts
* Does much of what backend_main.c does, only less so
* Example:
* 1) validate foo_db using a tmp dbdir
* ./clixon_util_validate -f /usr/local/etc/example.xml -d foo -o CLICON_XMLDB_DIR=/tmp
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/time.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/* For validate and commit commands. */
#include "clixon/clixon_backend.h"
/* Command line options passed to getopt(3) */
#define UTIL_COMMIT_OPTS "hD:f:cd:o:"
static int
usage(char *argv0)
{
fprintf(stderr, "Tool to validate a database\nusage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f <file>\tClixon config file\n"
"\t-d <file>\tDatabase name (if not candidate, must be in XMLDBDIR)\n"
"\t-c \t\tValidate + commit, otherwise only validate\n"
"\t-o \"<option>=<value>\"\tGive configuration option overriding config file (see clixon-config.yang)\n",
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
int c;
yang_stmt *yspec = NULL;
int commit = 0;
char *database = NULL;
clicon_handle h;
int dbg = 0;
char *dir;
char *str;
int ret;
cbuf *cb = NULL;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
/* Initialize clixon handle */
if ((h = clicon_handle_init()) == NULL)
goto done;
/*
* Command-line options for help, debug, and config-file
*/
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_COMMIT_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'f': /* config file */
if (!strlen(optarg))
usage(argv[0]);
clicon_option_str_set(h, "CLICON_CONFIGFILE", optarg);
break;
case 'c': /* commit (otherwise only validate) */
case 'd': /* candidate database (if not candidate) */
case 'o': /* Configuration option */
break; /* see next getopt */
default:
usage(argv[0]);
break;
}
clixon_debug_init(dbg, NULL);
yang_init(h);
/* Find and read configfile */
if (clicon_options_main(h) < 0)
goto done;
/* Initialize plugin module by creating a handle holding plugin and callback lists */
if (clixon_plugin_module_init(h) < 0)
goto done;
/* Now run through the operational args */
opterr = 1;
optind = 1;
while ((c = getopt(argc, argv, UTIL_COMMIT_OPTS)) != -1)
switch (c) {
case 'h' : /* help */
case 'D' : /* debug */
case 'f': /* config file */
break;
case 'c': /* commit (otherwise only validate) */
commit++;
break;
case 'd': /* candidate database (if not candidate) */
database = optarg;
break;
case 'o':{ /* Configuration option */
char *val;
if ((val = index(optarg, '=')) == NULL)
usage(argv[0]);
*val++ = '\0';
if (clicon_option_add(h, optarg, val) < 0)
goto done;
break;
}
default:
usage(argv[0]);
break;
}
/* Set default namespace according to CLICON_NAMESPACE_NETCONF_DEFAULT */
xml_nsctx_namespace_netconf_default(h);
/* Add (hardcoded) netconf features in case ietf-netconf loaded here
* Otherwise it is loaded in netconf_module_load below
*/
if (netconf_module_features(h) < 0)
goto done;
/* Create top-level yang spec and store as option */
if ((yspec = yspec_new()) == NULL)
goto done;
clicon_dbspec_yang_set(h, yspec);
/* Load backend plugins before yangs are loaded (eg extension callbacks) */
if ((dir = clicon_backend_dir(h)) != NULL &&
clixon_plugins_load(h, CLIXON_PLUGIN_INIT, dir,
clicon_option_str(h, "CLICON_BACKEND_REGEXP")) < 0)
goto done;
/* Load Yang modules
* 1. Load a yang module as a specific absolute filename */
if ((str = clicon_yang_main_file(h)) != NULL)
if (yang_spec_parse_file(h, str, yspec) < 0)
goto done;
/* 2. Load a (single) main module */
if ((str = clicon_yang_module_main(h)) != NULL)
if (yang_spec_parse_module(h, str, clicon_yang_module_revision(h),
yspec) < 0)
goto done;
/* 3. Load all modules in a directory (will not overwrite file loaded ^) */
if ((str = clicon_yang_main_dir(h)) != NULL)
if (yang_spec_load_dir(h, str, yspec) < 0)
goto done;
/* Load clixon lib yang module */
if (yang_spec_parse_module(h, "clixon-lib", NULL, yspec) < 0)
goto done;
/* Load yang module library, RFC7895 */
if (yang_modules_init(h) < 0)
goto done;
/* Add generic yang specs, used by netconf client and as internal protocol
*/
if (netconf_module_load(h) < 0)
goto done;
/* Load yang restconf module */
if (yang_spec_parse_module(h, "ietf-restconf", NULL, yspec)< 0)
goto done;
/* Load yang YANG module state */
if (clicon_option_bool(h, "CLICON_XMLDB_MODSTATE") &&
yang_spec_parse_module(h, "ietf-yang-library", NULL, yspec)< 0)
goto done;
/* Here all modules are loaded */
if (database == NULL)
database = "candidate";
if ((cb = cbuf_new()) == NULL){
clicon_err(OE_UNIX, errno, "cbuf_new");
goto done;
}
if (commit){
if ((ret = candidate_commit(h, NULL, database, 0, 0, cb)) < 0)
goto done;
}
else{
if ((ret = candidate_validate(h, database, cb)) < 0)
goto done;
}
if (ret == 0){
clicon_err(OE_DB, 0, " Failed: %s", cbuf_get(cb));
goto done;
}
fprintf(stdout, "OK\n");
retval = 0;
done:
if (cb)
cbuf_free(cb);
return retval;
}

371
util/clixon_util_xml.c
View file

@ -1,371 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* XML support functions.
* @see https://www.w3.org/TR/2008/REC-xml-20081126
* https://www.w3.org/TR/2009/REC-xml-names-20091208
* The function can do yang validation, process xml and json, etc.
* On success, nothing is printed and exitcode 0
* On failure, an error is printed on stderr and exitcode != 0
* Failure error prints are different, it would be nice to make them more
* uniform. (see clixon_netconf_error)
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/time.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/* Command line options passed to getopt(3) */
#define UTIL_XML_OPTS "hD:f:JjXl:pvoy:Y:t:T:u"
static int
validate_tree(clicon_handle h,
cxobj *xt,
yang_stmt *yspec)
{
int retval = -1;
int ret;
cxobj *xerr = NULL; /* malloced must be freed */
cbuf *cbret = NULL;
/* should already be populated */
/* Add default values */
if (xml_default_recurse(xt, 0) < 0)
goto done;
if (xml_apply(xt, -1, xml_sort_verify, h) < 0)
clicon_log(LOG_NOTICE, "%s: sort verify failed", __FUNCTION__);
if ((ret = xml_yang_validate_all_top(h, xt, &xerr)) < 0)
goto done;
if (ret > 0 && (ret = xml_yang_validate_add(h, xt, &xerr)) < 0)
goto done;
if (ret == 0){
if ((cbret = cbuf_new()) ==NULL){
clicon_err(OE_XML, errno, "cbuf_new");
goto done;
}
if (netconf_err2cb(h, xerr, cbret) < 0)
goto done;
fprintf(stderr, "xml validation error: %s\n", cbuf_get(cbret));
goto done;
}
retval = 0;
done:
if (cbret)
cbuf_free(cbret);
if (xerr)
xml_free(xerr);
return retval;
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] with xml on stdin (unless -f)\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f <file>\tXML input file (overrides stdin)\n"
"\t-J \t\tInput as JSON\n"
"\t-j \t\tOutput as JSON\n"
"\t-X \t\tOutput as TEXT \n"
"\t-l <s|e|o> \tLog on (s)yslog, std(e)rr, std(o)ut (stderr is default)\n"
"\t-o \t\tOutput the file\n"
"\t-v \t\tValidate the result in terms of Yang model (requires -y)\n"
"\t-p \t\tPretty-print output\n"
"\t-y <filename> \tYang filename or dir (load all files)\n"
"\t-Y <dir> \tYang dirs (can be several)\n"
"\t-t <file>\tXML top input file (where base tree is pasted to)\n"
"\t-T <path>\tXPath to where in top input file base should be pasted\n"
"\t-u \t\tTreat unknown XML as anydata\n"
,
argv0);
exit(0);
}
int
main(int argc,
char **argv)
{
int retval = -1;
int ret;
cxobj *xt = NULL; /* Base cxobj tree parsed from xml or json */
cbuf *cb = cbuf_new();
int c;
int logdst = CLICON_LOG_STDERR;
int jsonin = 0;
int jsonout = 0;
int textout = 0;
char *input_filename = NULL;
char *top_input_filename = NULL;
char *yang_file_dir = NULL;
yang_stmt *yspec = NULL;
cxobj *xerr = NULL; /* malloced must be freed */
int pretty = 0;
int validate = 0;
int output = 0;
clicon_handle h;
struct stat st;
FILE *fp = stdin; /* base file, stdin */
FILE *tfp = NULL; /* top file */
cxobj *xcfg = NULL;
cbuf *cbret = NULL;
cxobj *xtop = NULL; /* Top tree if any */
char *top_path = NULL;
cxobj *xbot; /* Place in xtop where base cxobj is parsed */
cvec *nsc = NULL;
yang_bind yb;
int dbg = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR);
/* Initialize clixon handle */
if ((h = clicon_handle_init()) == NULL)
goto done;
if ((xcfg = xml_new("clixon-config", NULL, CX_ELMNT)) == NULL)
goto done;
if (clicon_conf_xml_set(h, xcfg) < 0)
goto done;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_XML_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'f':
input_filename = optarg;
break;
case 'J':
jsonin++;
break;
case 'j':
jsonout++;
break;
case 'X':
textout++;
break;
case 'l': /* Log destination: s|e|o|f */
if ((logdst = clicon_log_opt(optarg[0])) < 0)
usage(argv[0]);
break;
case 'o':
output++;
break;
case 'v':
validate++;
break;
case 'p':
pretty++;
break;
case 'y':
yang_file_dir = optarg;
break;
case 'Y':
if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0)
goto done;
break;
case 't':
top_input_filename = optarg;
break;
case 'T': /* top file xpath */
top_path = optarg;
break;
case 'u':
if (clicon_option_bool_set(h, "CLICON_YANG_UNKNOWN_ANYDATA", 1) < 0)
goto done;
xml_bind_yang_unknown_anydata(1);
break;
default:
usage(argv[0]);
break;
}
if (validate && !yang_file_dir){
fprintf(stderr, "-v requires -y\n");
usage(argv[0]);
}
if (top_input_filename && top_path == NULL){
fprintf(stderr, "-t requires -T\n");
usage(argv[0]);
}
clicon_log_init(__FILE__, dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
yang_init(h);
/* 1. Parse yang */
if (yang_file_dir){
if ((yspec = yspec_new()) == NULL)
goto done;
if (stat(yang_file_dir, &st) < 0){
clicon_err(OE_YANG, errno, "%s not found", yang_file_dir);
goto done;
}
if (S_ISDIR(st.st_mode)){
if (yang_spec_load_dir(h, yang_file_dir, yspec) < 0)
goto done;
}
else{
if (yang_spec_parse_file(h, yang_file_dir, yspec) < 0)
goto done;
}
}
/* If top file is declared, the base XML/JSON is pasted as child to the top-file.
* This is to emulate sub-tress, not just top-level parsing.
* Always validated
*/
if (top_input_filename){
if ((tfp = fopen(top_input_filename, "r")) == NULL){
clicon_err(OE_YANG, errno, "fopen(%s)", top_input_filename);
goto done;
}
if ((ret = clixon_xml_parse_file(tfp, YB_MODULE, yspec, &xtop, &xerr)) < 0){
fprintf(stderr, "xml parse error: %s\n", clicon_err_reason);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "Parse top file", NULL);
goto done;
}
if (validate_tree(h, xtop, yspec) < 0)
goto done;
/* Compute canonical namespace context */
if (xml_nsctx_yangspec(yspec, &nsc) < 0)
goto done;
if ((xbot = xpath_first(xtop, nsc, "%s", top_path)) == NULL){
fprintf(stderr, "Path not found in top tree: %s\n", top_path);
goto done;
}
xt = xbot;
}
if (input_filename){
if ((fp = fopen(input_filename, "r")) == NULL){
clicon_err(OE_YANG, errno, "open(%s)", input_filename);
goto done;
}
}
/* 2. Parse data (xml/json) */
if (jsonin){
if ((ret = clixon_json_parse_file(fp, 1, top_input_filename?YB_PARENT:YB_MODULE, yspec, &xt, &xerr)) < 0)
goto done;
if (ret == 0){
clixon_netconf_error(h, xerr, "util_xml", NULL);
goto done;
}
}
else{ /* XML */
if (!yang_file_dir)
yb = YB_NONE;
else if (xt == NULL)
yb = YB_MODULE;
else
yb = YB_PARENT;
if ((ret = clixon_xml_parse_file(fp, yb, yspec, &xt, &xerr)) < 0){
fprintf(stderr, "xml parse error: %s\n", clicon_err_reason);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "util_xml", NULL);
goto done;
}
}
/* 3. Validate data (if yspec) */
if (validate){
if (validate_tree(h, xt, yspec) < 0)
goto done;
}
/* 4. Output data (xml/json/text) */
if (output){
if (textout){
if (clixon_text2file(stdout, xt, 0, fprintf, 1, 0) < 0)
goto done;
}
else if (jsonout){
if (clixon_json2cbuf(cb, xt, pretty, 1, 0) < 0)
goto done;
}
else if (clixon_xml2cbuf(cb, xt, 0, pretty, NULL, -1, 1) < 0)
goto done;
fprintf(stdout, "%s", cbuf_get(cb));
fflush(stdout);
}
retval = 0;
done:
if (tfp)
fclose(tfp);
if (fp)
fclose(fp);
if (nsc)
cvec_free(nsc);
if (cbret)
cbuf_free(cbret);
if (xcfg)
xml_free(xcfg);
if (xerr)
xml_free(xerr);
if (xtop)
xml_free(xtop);
else if (xt)
xml_free(xt);
if (cb)
cbuf_free(cb);
return retval;
}

310
util/clixon_util_xml_mod.c
View file

@ -1,310 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2016 Olof Hagsand and Benny Holmgren
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Utility for manipulating XML trees. In all operations, there is a primary base tree x0/xb) and a
* secondary tree (x1/xs). There are several operations on how to modify the base tree using the
* secondary tree. Both x0 and x1 are root trees, whereas xb/xs are subytrees of x0/x1 respectively
* after path has been applied.
* This includes:
* - Insert subtree (last) in list: -b <x0> -x <x1> -p <path>
* which gives xb and xs. The first element of xs is inserted under xb
* Example: xb := <b><c/></b>; xs := <b><d/></b>
* Result is : xb = <b><c/><d/></b>
* - Merging trees: -o merge -b <base> -x <2nd> -p <path>
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/* Command line options passed to getopt(3) */
#define UTIL_XML_MOD_OPTS "hD:o:y:Y:b:x:p:s"
enum opx{
OPX_ERROR = -1,
OPX_INSERT,
OPX_MERGE,
OPX_PARENT
};
static const map_str2int opx_map[] = {
{"insert", OPX_INSERT},
{"merge", OPX_MERGE},
{"parent", OPX_PARENT},
{NULL, -1}
};
const enum opx
opx_str2int(char *opstr)
{
return clicon_str2int(opx_map, opstr);
}
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level>\tDebug\n"
"\t-o <op> \tOperation: parent, insert or merge\n"
"\t-y <file> \tYANG spec file\n"
"\t-Y <dir> \tYang dirs (can be several)\n"
"\t-b <base> \tXML base expression\n"
"\t-x <xml> \tXML to insert\n"
"\t-p <xpath>\tXpath to where in base and XML\n"
"\t-s \tSort output after operation\n",
argv0
);
exit(0);
}
int
main(int argc, char **argv)
{
int retval = -1;
char *argv0 = argv[0];
int c;
char *yangfile = NULL;
int fd = 0; /* unless overriden by argv[1] */
char *x0str = NULL;
char *x1str = NULL;
char *xpath = NULL;
yang_stmt *yspec = NULL;
cxobj *x0 = NULL;
cxobj *x1 = NULL;
cxobj *xb = NULL;
cxobj *xi = NULL;
cxobj *xi1 = NULL;
cxobj *xerr = NULL;
int sort = 0;
int ret;
clicon_handle h;
enum opx opx = OPX_ERROR;
char *reason = NULL;
int dbg = 0;
cxobj *xcfg = NULL;
clicon_log_init("clixon_insert", LOG_DEBUG, CLICON_LOG_STDERR);
if ((h = clicon_handle_init()) == NULL)
goto done;
if ((xcfg = xml_new("clixon-config", NULL, CX_ELMNT)) == NULL)
goto done;
if (clicon_conf_xml_set(h, xcfg) < 0)
goto done;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, UTIL_XML_MOD_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv0);
break;
case 'o': /* Operation */
opx = opx_str2int(optarg);
break;
case 'y': /* YANG spec file */
yangfile = optarg;
break;
case 'Y':
if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0)
goto done;
break;
case 'b': /* Base XML expression */
x0str = optarg;
break;
case 'x': /* XML to insert */
x1str = optarg;
break;
case 'p': /* XPath base */
xpath = optarg;
break;
case 's': /* sort output after insert */
sort++;
break;
default:
usage(argv[0]);
break;
}
/* Sanity check: check mandatory arguments */
if (x1str == NULL || x0str == NULL || yangfile == NULL)
usage(argv0);
if (opx == OPX_ERROR)
usage(argv0);
clixon_debug_init(dbg, NULL);
if ((yspec = yspec_new()) == NULL)
goto done;
if (yang_spec_parse_file(h, yangfile, yspec) < 0)
goto done;
/* Parse base XML */
if ((ret = clixon_xml_parse_string(x0str, YB_MODULE, yspec, &x0, &xerr)) < 0){
clicon_err(OE_XML, 0, "Parsing base xml: %s", x0str);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "Parsing base xml", NULL);
goto done;
}
/* Get base subtree by xpath */
if (xpath == NULL)
xb = x0;
else if ((xb = xpath_first(x0, NULL, "%s", xpath)) == NULL){
clicon_err(OE_XML, 0, "xpath: %s not found in x0", xpath);
goto done;
}
if (clixon_debug_get()){
clixon_debug(CLIXON_DBG_DEFAULT, "xb:");
xml_print(stderr, xb);
}
switch (opx){
case OPX_PARENT:
/* Parse insert XML */
if ((ret = clixon_xml_parse_string(x1str, YB_PARENT, yspec, &xb, &xerr)) < 0){
clicon_err(OE_XML, 0, "Parsing insert xml: %s", x1str);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "Parsing secondary xml", NULL);
goto done;
}
break;
case OPX_MERGE:
/* Parse merge XML */
if ((ret = clixon_xml_parse_string(x1str, YB_MODULE, yspec, &x1, &xerr)) < 0){
clicon_err(OE_XML, 0, "Parsing insert xml: %s", x1str);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "Parsing secondary xml", NULL);
goto done;
}
if (xpath == NULL)
xi = x1;
else if ((xi = xpath_first(x1, NULL, "%s", xpath)) == NULL){
clicon_err(OE_XML, 0, "xpath: %s not found in xi", xpath);
goto done;
}
if ((ret = xml_merge(xb, xi, yspec, &reason)) < 0)
goto done;
if (ret == 0){
clicon_err(OE_XML, 0, "%s", reason);
goto done;
}
break;
case OPX_INSERT:
/* Parse insert XML */
if ((ret = clixon_xml_parse_string(x1str, YB_MODULE, yspec, &x1, &xerr)) < 0){
clicon_err(OE_XML, 0, "Parsing insert xml: %s", x1str);
goto done;
}
if (ret == 0){
clixon_netconf_error(h, xerr, "Parsing secondary xml", NULL);
goto done;
}
/* Get secondary subtree by xpath */
if (xpath == NULL)
xi = x1;
else if ((xi = xpath_first(x1, NULL, "%s", xpath)) == NULL){
clicon_err(OE_XML, 0, "xpath: %s not found in xi", xpath);
goto done;
}
/* Find first element child of secondary */
if ((xi1 = xml_child_i_type(xi, 0, CX_ELMNT)) == NULL){
clicon_err(OE_XML, 0, "xi has no element child");
goto done;
}
/* Remove it from parent */
if (xml_rm(xi1) < 0)
goto done;
if (xml_insert(xb, xi1, INS_LAST, NULL, NULL) < 0)
goto done;
break;
default:
usage(argv0);
}
if (clixon_debug_get()){
clixon_debug(CLIXON_DBG_DEFAULT, "x0:");
xml_print(stderr, x0);
}
if (sort)
xml_sort_recurse(xb);
if (strcmp(xml_name(xb),"top")==0){
if (clixon_xml2file(stdout, xb, 0, 0, NULL, fprintf, 1, 0) < 0)
goto done;
}
else{
if (clixon_xml2file(stdout, xb, 0, 0, NULL, fprintf, 0, 0) < 0)
goto done;
}
fprintf(stdout, "\n");
retval = 0;
done:
if (x0)
xml_free(x0);
if (x1)
xml_free(x1);
if (xcfg)
xml_free(xcfg);
if (xerr)
xml_free(xerr);
if (reason)
free(reason);
if (yspec)
ys_free(yspec);
if (fd > 0)
close(fd);
return retval;
}

418
util/clixon_util_xpath.c
View file

@ -1,418 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2009-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC(Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
See https://www.w3.org/TR/xpath/
*
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <limits.h>
#include <stdint.h>
#include <syslog.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/stat.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/* Command line options to be passed to getopt(3) */
#define XPATH_OPTS "hD:f:p:i:In:cl:y:Y:"
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options]\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-f <file> \tXML file\n"
"\t-p <xpath> \tPrimary XPath string\n"
"\t-i <xpath0>\t(optional) Initial XPath string\n"
"\t-I \t\tCheck inverse, map back xml result to xpath and check if equal\n"
"\t-n <pfx:id>\tNamespace binding (pfx=NULL for default)\n"
"\t-c \t\tMap xpath to canonical form\n"
"\t-l <s|e|o|f<file>> \tLog on (s)yslog, std(e)rr, std(o)ut or (f)ile (stderr is default)\n"
"\t-y <filename> \tYang filename or dir (load all files)\n"
"\t-Y <dir> \tYang dirs (can be several)\n"
"and the following extra rules:\n"
"\tif -f is not given, XML input is expected on stdin\n"
"\tif -p is not given, <xpath> is expected as the first line on stdin\n"
"This means that with no arguments, <xpath> and XML is expected on stdin.\n",
argv0
);
exit(0);
}
static int
ctx_print2(cbuf *cb,
xp_ctx *xc)
{
int retval = -1;
int i;
cprintf(cb, "%s:", (char*)clicon_int2str(ctxmap, xc->xc_type));
switch (xc->xc_type){
case XT_NODESET:
for (i=0; i<xc->xc_size; i++){
cprintf(cb, "%d:", i);
if (clixon_xml2cbuf(cb, xc->xc_nodeset[i], 0, 0, NULL, -1, 0) < 0)
goto done;
}
break;
case XT_BOOL:
cprintf(cb, "%s", xc->xc_bool?"true":"false");
break;
case XT_NUMBER:
cprintf(cb, "%lf", xc->xc_number);
break;
case XT_STRING:
cprintf(cb, "%s", xc->xc_string);
break;
}
retval = 0;
done:
return retval;
}
int
main(int argc,
char **argv)
{
int retval = -1;
char *argv0 = argv[0];
int i;
cxobj *x0 = NULL;
cxobj *x;
int c;
int len;
char *buf = NULL;
int ret;
FILE *fp = stdin; /* unless overriden by -f */
char *yang_file_dir = NULL;
yang_stmt *yspec = NULL;
char *xpath = NULL;
char *xpath0 = NULL;
char *filename;
xp_ctx *xc = NULL;
cbuf *cb = NULL;
clicon_handle h;
struct stat st;
cvec *nsc = NULL;
int canonical = 0;
cxobj *xcfg = NULL;
cbuf *cbret = NULL;
cxobj *xerr = NULL; /* malloced must be freed */
int logdst = CLICON_LOG_STDERR;
int dbg = 0;
int xpath_inverse = 0;
/* In the startup, logs to stderr & debug flag set later */
clicon_log_init("xpath", LOG_DEBUG, logdst);
/* Initialize clixon handle */
if ((h = clicon_handle_init()) == NULL)
goto done;
/* Initialize config tree (needed for -Y below) */
if ((xcfg = xml_new("clixon-config", NULL, CX_ELMNT)) == NULL)
goto done;
if (clicon_conf_xml_set(h, xcfg) < 0)
goto done;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, XPATH_OPTS)) != -1)
switch (c) {
case 'h':
usage(argv0);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv0);
break;
case 'f': /* XML file */
filename = optarg;
if ((fp = fopen(filename, "r")) == NULL){
clicon_err(OE_UNIX, errno, "fopen(%s)", optarg);
goto done;
}
break;
case 'p': /* Primary XPath string */
xpath = optarg;
break;
case 'i': /* Optional initial XPath string */
xpath0 = optarg;
break;
case 'I': /* Check inverse */
xpath_inverse++;
break;
case 'n':{ /* Namespace binding */
char *prefix;
char *id;
if (nsc == NULL &&
(nsc = xml_nsctx_init(NULL, NULL)) == NULL)
goto done;
if (nodeid_split(optarg, &prefix, &id) < 0)
goto done;
if (prefix && strcmp(prefix, "null")==0){
free(prefix);
prefix = NULL;
}
if (xml_nsctx_add(nsc, prefix, id) < 0)
goto done;
if (prefix)
free(prefix);
if (id)
free(id);
break;
}
case 'c': /* Map namespace to canonical form */
canonical = 1;
break;
case 'l': /* Log destination: s|e|o|f */
if ((logdst = clicon_log_opt(optarg[0])) < 0)
usage(argv[0]);
if (logdst == CLICON_LOG_FILE &&
strlen(optarg)>1 &&
clicon_log_file(optarg+1) < 0)
goto done;
break;
case 'y':
yang_file_dir = optarg;
break;
case 'Y':
if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0)
goto done;
break;
default:
usage(argv[0]);
break;
}
/*
* Logs, error and debug to stderr or syslog, set debug level
*/
clicon_log_init("xpath", dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
yang_init(h);
/* Parse yang */
if (yang_file_dir){
if ((yspec = yspec_new()) == NULL)
goto done;
if (stat(yang_file_dir, &st) < 0){
clicon_err(OE_YANG, errno, "%s not found", yang_file_dir);
goto done;
}
if (S_ISDIR(st.st_mode)){
if (yang_spec_load_dir(h, yang_file_dir, yspec) < 0)
goto done;
}
else{
if (yang_spec_parse_file(h, yang_file_dir, yspec) < 0)
goto done;
}
}
if (xpath==NULL){
/* First read xpath */
len = 1024; /* any number is fine */
if ((buf = malloc(len)) == NULL){
perror("pt_file malloc");
return -1;
}
memset(buf, 0, len);
i = 0;
while (1){
if ((ret = read(0, &c, 1)) < 0){
perror("read");
goto done;
}
if (ret == 0)
break;
if (c == '\n')
break;
if (len==i){
if ((buf = realloc(buf, 2*len)) == NULL){
fprintf(stderr, "%s: realloc: %s\n", __FUNCTION__, strerror(errno));
return -1;
}
memset(buf+len, 0, len);
len *= 2;
}
buf[i++] = (char)(c&0xff);
}
xpath = buf;
}
/* If canonical, translate nsc and xpath to canonical form */
if (canonical){
char *xpath1 = NULL;
cvec *nsc1 = NULL;
cbuf *cbreason = NULL;
if ((ret = xpath2canonical(xpath, nsc, yspec, &xpath1, &nsc1, &cbreason)) < 0)
goto done;
if (ret == 0){
fprintf(stderr, "Error with %s: %s", xpath, cbuf_get(cbreason));
goto ok;
}
xpath = xpath1;
if (xpath)
fprintf(stdout, "%s\n", xpath);
if (nsc)
xml_nsctx_free(nsc);
nsc = nsc1;
if (nsc)
cvec_print(stdout, nsc);
goto ok; /* need a switch to continue, now just print and quit */
}
/*
* If fp=stdin, then continue reading from stdin (after CR)
* XXX Note 0 above, stdin here
*/
if (clixon_xml_parse_file(fp, YB_NONE, NULL, &x0, NULL) < 0){
fprintf(stderr, "Error: parsing: %s\n", clicon_err_reason);
goto done;
}
/* Validate XML as well */
if (yang_file_dir){
/* Populate */
if ((ret = xml_bind_yang(h, x0, YB_MODULE, yspec, &xerr)) < 0)
goto done;
if (ret == 0){
if ((cbret = cbuf_new()) ==NULL){
clicon_err(OE_XML, errno, "cbuf_new");
goto done;
}
if (netconf_err2cb(h, xerr, cbret) < 0)
goto done;
fprintf(stderr, "xml validation error: %s\n", cbuf_get(cbret));
goto done;
}
/* Sort */
if (xml_sort_recurse(x0) < 0)
goto done;
/* Add default values */
if (xml_default_recurse(x0, 0) < 0)
goto done;
if (xml_apply0(x0, -1, xml_sort_verify, h) < 0)
clicon_log(LOG_NOTICE, "%s: sort verify failed", __FUNCTION__);
if ((ret = xml_yang_validate_all_top(h, x0, &xerr)) < 0)
goto done;
if (ret > 0 && (ret = xml_yang_validate_add(h, x0, &xerr)) < 0)
goto done;
if (ret == 0){
if ((cbret = cbuf_new()) ==NULL){
clicon_err(OE_XML, errno, "cbuf_new");
goto done;
}
if (netconf_err2cb(h, xerr, cbret) < 0)
goto done;
fprintf(stderr, "xml validation error: %s\n", cbuf_get(cbret));
goto done;
}
}
/* If xpath0 given, position current x (ie somewhere else than root) */
if (xpath0){
if ((x = xpath_first(x0, NULL, "%s", xpath0)) == NULL){
fprintf(stderr, "Error: xpath0 returned NULL\n");
return -1;
}
}
else
x = x0;
#if 0 // filter syntax errors
{
xpath_tree *xptree = NULL;
if (xpath_parse(xpath, &xptree) < 0)
goto ok; // Parse errors returns OK
}
#endif
if (xpath_vec_ctx(x, nsc, xpath, 0, &xc) < 0)
return -1;
/* Check inverse, eg XML back to xpath and compare with original, only if nodes */
if (xpath_inverse && xc->xc_type == XT_NODESET){
cxobj *xi;
char *xpathi = NULL;
for (i=0; i<xc->xc_size; i++){
xi = xc->xc_nodeset[i];
if (xml2xpath(xi, nsc, 0, 0, &xpathi) < 0)
goto done;
fprintf(stdout, "Inverse: %s\n", xpathi);
if (xpathi){
free(xpathi);
xpathi = NULL;
}
}
goto ok;
}
/* Print results */
cb = cbuf_new();
ctx_print2(cb, xc);
fprintf(stdout, "%s\n", cbuf_get(cb));
ok:
retval = 0;
done:
if (cb)
cbuf_free(cb);
if (nsc)
xml_nsctx_free(nsc);
if (xc)
ctx_free(xc);
if (xcfg)
xml_free(xcfg);
if (buf)
free(buf);
if (x0)
xml_free(x0);
if (fp)
fclose(fp);
if (h)
clicon_handle_exit(h);
return retval;
}

120
util/clixon_util_yang.c
View file

@ -1,120 +0,0 @@
/*
*
***** BEGIN LICENSE BLOCK *****
Copyright (C) 2017-2019 Olof Hagsand
Copyright (C) 2020-2022 Olof Hagsand and Rubicon Communications, LLC (Netgate)
This file is part of CLIXON.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Alternatively, the contents of this file may be used under the terms of
the GNU General Public License Version 3 or later (the "GPL"),
in which case the provisions of the GPL are applicable instead
of those above. If you wish to allow use of your version of this file only
under the terms of the GPL, and not to allow others to
use your version of this file under the terms of Apache License version 2,
indicate your decision by deleting the provisions above and replace them with
the notice and other provisions required by the GPL. If you do not delete
the provisions above, a recipient may use your version of this file under
the terms of any one of the Apache License version 2 or the GPL.
***** END LICENSE BLOCK *****
* Parse a SINGLE yang file - no dependencies - utility function only useful
* for basic syntactic checks.
*/
#ifdef HAVE_CONFIG_H
#include "clixon_config.h" /* generated by config & autoconf */
#endif
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <limits.h>
#include <ctype.h>
#include <string.h>
#include <arpa/inet.h>
#include <regex.h>
#include <dirent.h>
#include <syslog.h>
#include <signal.h>
#include <sys/stat.h>
#include <libgen.h>
#include <netinet/in.h>
/* cligen */
#include <cligen/cligen.h>
/* clixon */
#include "clixon/clixon.h"
/*
*/
static int
usage(char *argv0)
{
fprintf(stderr, "usage:%s [options] # input yang spec on stdin\n"
"where options are\n"
"\t-h \t\tHelp\n"
"\t-D <level> \tDebug\n"
"\t-l <s|e|o> \tLog on (s)yslog, std(e)rr, std(o)ut (stderr is default)\n",
argv0);
exit(0);
}
int
main(int argc, char **argv)
{
yang_stmt *yspec = NULL;
int c;
int logdst = CLICON_LOG_STDERR;
int dbg = 0;
optind = 1;
opterr = 0;
while ((c = getopt(argc, argv, "hD:l:")) != -1)
switch (c) {
case 'h':
usage(argv[0]);
break;
case 'D':
if (sscanf(optarg, "%d", &dbg) != 1)
usage(argv[0]);
break;
case 'l': /* Log destination: s|e|o|f */
if ((logdst = clicon_log_opt(optarg[0])) < 0)
usage(argv[0]);
break;
default:
usage(argv[0]);
break;
}
clicon_log_init("clixon_util_yang", dbg?LOG_DEBUG:LOG_INFO, logdst);
clixon_debug_init(dbg, NULL);
if ((yspec = yspec_new()) == NULL)
goto done;
if (yang_parse_file(stdin, "yang test", yspec) == NULL){
fprintf(stderr, "yang parse error %s\n", clicon_err_reason);
return -1;
}
yang_print(stdout, yspec);
done:
if (yspec)
ys_free(yspec);
return 0;
}