diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ab298e1..9769c04a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,8 @@ * Added clixon_util_regexp utility function * Added extensive regexp test [test/test_pattern.sh] for both posix and libxml2 * Added regex cache to type resolution +* Yang "refine" feature supported + * According to RFC 7950 7.13.2 * Yang "min-element" and "max-element" feature supported * According to RFC 7950 7.7.4 and 7.7.5 * See (tests)[test/test_minmax.sh] @@ -68,6 +70,29 @@ ### API changes on existing features (you may need to change your code) * All hash_ functions have been prefixed with `clicon_` to avoid name collision with other packages (frr) +* RESTCONF strict namespace validation of data in POST and PUT. + * Accepted: + ``` + curl -X PUT http://localhost/restconf/data/mod:a -d {"mod:a":"x"} + ``` + * Not accepted (must prefix "a" with module): + ``` + curl -X PUT http://localhost/restconf/data/mod:a -d {"a":"x"} + ``` + * Undefine `RESTCONF_NS_DATA_CHECK` in include/clixon_custom.h to disable strict check. +* Many validation functions have changed error parameter from cbuf to xml tree. + * XML trees are more flexible for utility tools + * If you use these(mostly internal), you need to change the error function: `generic_validate, from_validate_common, xml_yang_validate_all_top, xml_yang_validate_all, xml_yang_validate_add, xml_yang_validate_rpc, xml_yang_validate_list_key_only` +* Replaced `CLIXON_DATADIR` with two configurable options defining where Clixon installs Yang files. + * use `--with-yang-installdir=DIR` to install Clixon yang files in DIR + * use `--with-std-yang-installdir=DIR` to install standard yang files that Clixon may use in DIR + * Default is (as before) `/usr/local/share/clixon` +* New clixon-config@2019-06-05.yang revision + * Added: `CLICON_YANG_REGEXP, CLICON_CLI_TAB_MODE, CLICON_CLI_HIST_FILE, CLICON_CLI_HIST_SIZE, CLICON_XML_CHANGELOG, CLICON_XML_CHANGELOG_FILE`. + * Renamed: `CLICON_XMLDB_CACHE` to `CLICON_DATASTORE_CACHE` and type changed. + * Deleted: `CLICON_XMLDB_PLUGIN, CLICON_USE_STARTUP_CONFIG`; +* New clixon-lib@2019-06-05.yang revision + * Added: ping rpc added (for liveness) * Added compiled regexp parameter as part of internal yang type resolution functions * `yang_type_resolve()`, `yang_type_get()` * All internal `ys_populate_*()` functions (except ys_populate()) have switched parameters: `clicon_handle, yang_stmt *)` @@ -167,6 +192,11 @@ ### Minor changes +* `startup_extraxml` triggers unnecessary validation + * Renamed startup_db_reset -> xmldb_db_reset (its a general function) + * In startup_extraxml(), check if reset callbacks or extraxml file actually makes and changes to the tmp db. +* Print CLICON_YANG_DIR and CLICON_FEATURE lists on startup with debug flag +* Extended `util/clixon_util_xml` with yang and validate functionality so it can be used as a stand-alone utility for validating XML/JSON files * JSON parse and print improvements * Integrated parsing with namespace translation and yang spec lookup * Added CLIgen tab-modes in config option CLICON_CLI_TAB_MODE, which means you can control the behaviour of `` in the CLI. @@ -212,6 +242,8 @@ ### Corrected Bugs +* Fixed a problem with some netconf error messages caused restconf daemon to exit due to no XML encoding +* Check cligen tab mode, dont start if CLICON_CLI_TAB_MODE is undefined * Startup transactions did not mark added tree with XML_FLAG_ADD as it should. * Restconf PUT different keys detected (thanks @dcornejo) and fixed * This was accepted but shouldn't be: `PUT http://restconf/data/A=hello/B -d '{"B":"goodbye"}'` diff --git a/README.md b/README.md index 0a6a90db..e2354c15 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ support. * [Frequently asked questions (FAQ)](doc/FAQ.md) * [Hello world](example/hello/README.md) * [Changelog](CHANGELOG.md) - * [Installation](#installation) + * [Installation](doc/INSTALL.md) * [Licenses](#licenses) * [Support](#support) * [Dependencies](#dependencies) @@ -47,21 +47,6 @@ Users of Clixon currently include: See also [Clicon project page](http://clicon.org). -Clixon runs on Linux, [FreeBSD port](https://www.freshports.org/devel/clixon) and Mac/Apple. CPU architecures include x86_64, i686, ARM32. - -## Installation - -A typical installation is as follows: -``` - configure # Configure clixon to platform - make # Compile - sudo make install # Install libs, binaries, and config-files - sudo make install-include # Install include files (for compiling) -``` - -One [example application](example/README.md) is provided, a IETF IP YANG datamodel with -generated CLI, Netconf and restconf interface. - ## Licenses Clixon is open-source and dual licensed. Either Apache License, Version 2.0 or GNU @@ -113,7 +98,7 @@ Clixon follows: However, the following YANG syntax modules are not implemented (reference to RFC7950 in parenthesis): - deviation (7.20.3) - action (7.15) -- refine (7.13.2) +- augment in a uses sub-clause (7.17) (module-level augment is implemented) - status (7.21.2) - extension (7.19) - YIN (13) @@ -149,23 +134,22 @@ The standards covered include: Not supported: - !DOCTYPE (ie DTD) -Historically, Clixon has not until 3.9 made strict namespace -enforcing. For example, the following non-strict netconf was -previously accepted: -``` - -``` -In 3.9, the same statement should be, for example: -``` - -``` -Note that base netconf syntax is still not enforced but recommended: +The following xpath axes are supported: +- CHILD, DESCENDANT, DESCENDANT_OR_SELF, SELF, and PARENT + +The following xpath axes are _not_ supported: +- PRECEEDING, PRECEEDING_SIBLING, NAMESPACE, FOLLOWING_SIBLING, FOLLOWING, ANCESTOR,ANCESTOR_OR_SELF, ATTRIBUTE + +Note that base netconf namespace syntax is not enforced but recommended, which means that the following two expressions are treated equivalently: ``` + + + ``` - +All other namespaces are enforced. ## Netconf diff --git a/apps/backend/backend_client.c b/apps/backend/backend_client.c index 6805dc70..b35975d5 100644 --- a/apps/backend/backend_client.c +++ b/apps/backend/backend_client.c @@ -411,6 +411,7 @@ from_client_edit_config(clicon_handle h, cbuf *cbx = NULL; /* Assist cbuf */ int ret; char *username; + cxobj *xret = NULL; username = clicon_username_get(h); if ((yspec = clicon_dbspec_yang(h)) == NULL){ @@ -470,10 +471,13 @@ from_client_edit_config(clicon_handle h, goto ok; } /* xmldb_put (difflist handling) requires list keys */ - if ((ret = xml_yang_validate_list_key_only(h, xc, cbret)) < 0) + if ((ret = xml_yang_validate_list_key_only(h, xc, &xret)) < 0) goto done; - if (ret == 0) + if (ret == 0){ + if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0) + goto done; goto ok; + } /* Cant do this earlier since we dont have a yang spec to * the upper part of the tree, until we get the "config" tree. */ @@ -493,6 +497,8 @@ from_client_edit_config(clicon_handle h, ok: retval = 0; done: + if (xret) + xml_free(xret); if (cbx) cbuf_free(cbx); clicon_debug(1, "%s done cbret:%s", __FUNCTION__, cbuf_get(cbret)); @@ -1121,6 +1127,7 @@ from_client_msg(clicon_handle h, yang_stmt *ye; yang_stmt *ymod; cxobj *xnacm = NULL; + cxobj *xret = NULL; clicon_debug(1, "%s", __FUNCTION__); yspec = clicon_dbspec_yang(h); @@ -1147,10 +1154,13 @@ from_client_msg(clicon_handle h, * maybe not necessary since it should be */ if (xml_spec_populate_rpc(h, x, yspec) < 0) goto done; - if ((ret = xml_yang_validate_rpc(h, x, cbret)) < 0) + if ((ret = xml_yang_validate_rpc(h, x, &xret)) < 0) goto done; - if (ret == 0) + if (ret == 0){ + if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0) + goto done; goto reply; + } xe = NULL; username = xml_find_value(x, "username"); /* May be used by callbacks, etc */ @@ -1222,6 +1232,8 @@ from_client_msg(clicon_handle h, retval = 0; done: clicon_debug(1, "%s retval:%d", __FUNCTION__, retval); + if (xret) + xml_free(xret); if (xt) xml_free(xt); if (cbret) diff --git a/apps/backend/backend_commit.c b/apps/backend/backend_commit.c index 07de3da7..61e7193f 100644 --- a/apps/backend/backend_commit.c +++ b/apps/backend/backend_commit.c @@ -81,7 +81,7 @@ * are if code comes via XML/NETCONF. * @param[in] yspec Yang spec * @param[in] td Transaction data - * @param[out] cbret Cligen buffer containing netconf error (if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval -1 Error * @retval 0 Validation failed (with cbret set) * @retval 1 Validation OK @@ -90,7 +90,7 @@ static int generic_validate(clicon_handle h, yang_stmt *yspec, transaction_data_t *td, - cbuf *cbret) + cxobj **xret) { int retval = -1; cxobj *x1; @@ -100,7 +100,7 @@ generic_validate(clicon_handle h, int ret; /* All entries */ - if ((ret = xml_yang_validate_all_top(h, td->td_target, cbret)) < 0) + if ((ret = xml_yang_validate_all_top(h, td->td_target, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -109,7 +109,7 @@ generic_validate(clicon_handle h, x1 = td->td_scvec[i]; /* source changed */ x2 = td->td_tcvec[i]; /* target changed */ /* Should this be recursive? */ - if ((ret = xml_yang_validate_add(h, x2, cbret)) < 0) + if ((ret = xml_yang_validate_add(h, x2, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -119,7 +119,7 @@ generic_validate(clicon_handle h, x1 = td->td_dvec[i]; ys = xml_spec(x1); if (ys && yang_mandatory(ys) && yang_config(ys)==0){ - if (netconf_missing_element(cbret, "protocol", xml_name(x1), "Missing mandatory variable") < 0) + if (netconf_missing_element_xml(xret, "protocol", xml_name(x1), "Missing mandatory variable") < 0) goto done; goto fail; } @@ -127,7 +127,7 @@ generic_validate(clicon_handle h, /* added entries */ for (i=0; itd_alen; i++){ x2 = td->td_avec[i]; - if ((ret = xml_yang_validate_add(h, x2, cbret)) < 0) + if ((ret = xml_yang_validate_add(h, x2, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -175,6 +175,7 @@ startup_common(clicon_handle h, modstate_diff_t *msd = NULL; cxobj *xt = NULL; cxobj *x; + cxobj *xret = NULL; /* If CLICON_XMLDB_MODSTATE is enabled, then get the db XML with * potentially non-matching module-state in msd @@ -225,11 +226,13 @@ startup_common(clicon_handle h, /* 5. Make generic validation on all new or changed data. Note this is only call that uses 3-values */ clicon_debug(1, "Validating startup %s", db); - if ((ret = generic_validate(h, yspec, td, cbret)) < 0) + if ((ret = generic_validate(h, yspec, td, &xret)) < 0) goto done; - if (ret == 0) + if (ret == 0){ + if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0) + goto done; goto fail; /* STARTUP_INVALID */ - + } /* 6. Call plugin transaction validate callbacks */ if (plugin_transaction_validate(h, td) < 0) goto done; @@ -240,6 +243,8 @@ startup_common(clicon_handle h, ok: retval = 1; done: + if (xret) + xml_free(xret); if (xt) xml_free(xt); if (msd) @@ -374,6 +379,7 @@ startup_commit(clicon_handle h, * and call application callback validations. * @param[in] h Clicon handle * @param[in] candidate The candidate database. The wanted backend state + * @param[out] xret Error XML tree. Free with xml_free after use * @retval -1 Error - or validation failed (but cbret not set) * @retval 0 Validation failed (with cbret set) * @retval 1 Validation OK @@ -385,7 +391,7 @@ static int from_validate_common(clicon_handle h, char *candidate, transaction_data_t *td, - cbuf *cbret) + cxobj **xret) { int retval = -1; yang_stmt *yspec; @@ -409,7 +415,7 @@ from_validate_common(clicon_handle h, * But xml_diff requires some basic validation, at least check that yang-specs * have been assigned */ - if ((ret = xml_yang_validate_all_top(h, td->td_target, cbret)) < 0) + if ((ret = xml_yang_validate_all_top(h, td->td_target, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -462,7 +468,7 @@ from_validate_common(clicon_handle h, /* 5. Make generic validation on all new or changed data. Note this is only call that uses 3-values */ - if ((ret = generic_validate(h, yspec, td, cbret)) < 0) + if ((ret = generic_validate(h, yspec, td, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -503,6 +509,7 @@ candidate_commit(clicon_handle h, int retval = -1; transaction_data_t *td = NULL; int ret; + cxobj *xret = NULL; /* 1. Start transaction */ if ((td = transaction_new()) == NULL) @@ -511,10 +518,13 @@ candidate_commit(clicon_handle h, /* Common steps (with validate). Load candidate and running and compute diffs * Note this is only call that uses 3-values */ - if ((ret = from_validate_common(h, candidate, td, cbret)) < 0) + if ((ret = from_validate_common(h, candidate, td, &xret)) < 0) goto done; - if (ret == 0) + if (ret == 0){ + if (clicon_xml2cbuf(cbret, xret, 0, 0) < 0) + goto done; goto fail; + } /* 7. Call plugin transaction commit callbacks */ if (plugin_transaction_commit(h, td) < 0) @@ -563,7 +573,9 @@ candidate_commit(clicon_handle h, xmldb_get0_free(h, &td->td_src); transaction_free(td); } - return retval; + if (xret) + xml_free(xret); + return retval; fail: retval = 0; goto done; @@ -725,6 +737,7 @@ from_client_validate(clicon_handle h, transaction_data_t *td = NULL; int ret; char *db; + cxobj *xret = NULL; if ((db = netconf_db_find(xe, "source")) == NULL){ if (netconf_missing_element(cbret, "protocol", "source", NULL) < 0) @@ -737,9 +750,15 @@ from_client_validate(clicon_handle h, if ((td = transaction_new()) == NULL) goto done; /* Common steps (with commit) */ - if ((ret = from_validate_common(h, db, td, cbret)) < 1){ + if ((ret = from_validate_common(h, db, td, &xret)) < 1){ + /* A little complex due to several sources of validation fails or errors. + * (1) xerr is set -> translate to cbret; (2) cbret set use that; otherwise + * use clicon_err. */ + if (xret && clicon_xml2cbuf(cbret, xret, 0, 0) < 0) + goto done; plugin_transaction_abort(h, td); - if (netconf_operation_failed(cbret, "application", clicon_err_reason)< 0) + if (!cbuf_len(cbret) && + netconf_operation_failed(cbret, "application", clicon_err_reason)< 0) goto done; goto ok; } @@ -773,6 +792,8 @@ from_client_validate(clicon_handle h, xmldb_get0_free(h, &td->td_src); transaction_free(td); } + if (xret) + xml_free(xret); return retval; } /* from_client_validate */ diff --git a/apps/backend/backend_main.c b/apps/backend/backend_main.c index dae76e94..5e9a5354 100644 --- a/apps/backend/backend_main.c +++ b/apps/backend/backend_main.c @@ -642,7 +642,7 @@ main(int argc, switch (startup_mode){ case SM_INIT: /* Scratch running and start from empty */ /* [Delete and] create running db */ - if (startup_db_reset(h, "running") < 0) + if (xmldb_db_reset(h, "running") < 0) goto done; case SM_NONE: /* Fall through * * Load plugins and call plugin_init() */ diff --git a/apps/backend/backend_startup.c b/apps/backend/backend_startup.c index 2e5342b3..8fd12e0e 100644 --- a/apps/backend/backend_startup.c +++ b/apps/backend/backend_startup.c @@ -70,22 +70,6 @@ #include "backend_commit.h" #include "backend_startup.h" -/*! Create an XML database. If it exists already, delete it before creating - * @param[in] h Clixon handle - * @param[in] db Symbolic database name, eg "candidate", "running" - */ -int -startup_db_reset(clicon_handle h, - char *db) -{ - if (xmldb_exists(h, db) == 1){ - if (xmldb_delete(h, db) != 0 && errno != ENOENT) - return -1; - } - if (xmldb_create(h, db) < 0) - return -1; - return 0; -} /*! Merge db1 into db2 without commit * @retval -1 Error @@ -235,34 +219,45 @@ startup_extraxml(clicon_handle h, cbuf *cbret) { int retval = -1; - char *db = "tmp"; + char *tmp_db = "tmp"; int ret; cxobj *xt = NULL; /* Potentially upgraded XML */ /* Clear tmp db */ - if (startup_db_reset(h, db) < 0) + if (xmldb_db_reset(h, tmp_db) < 0) goto done; /* Application may define extra xml in its reset function*/ - if (clixon_plugin_reset(h, db) < 0) + if (clixon_plugin_reset(h, tmp_db) < 0) goto done; /* Extra XML can also be added via file */ if (file){ /* Parse and load file into tmp db */ - if ((ret = load_extraxml(h, file, db, cbret)) < 0) + if ((ret = load_extraxml(h, file, tmp_db, cbret)) < 0) goto done; if (ret == 0) goto fail; } + /* + * Check if tmp db is empty. + * It should be empty if extra-xml is null and reset plugins did nothing + * then skip validation. + */ + if (xmldb_get(h, tmp_db, NULL, &xt) < 0) + goto done; + if (xt==NULL || xml_child_nr(xt)==0) + goto ok; + xml_free(xt); + xt = NULL; /* Validate the tmp db and return possibly upgraded xml in xt */ - if ((ret = startup_validate(h, db, &xt, cbret)) < 0) + if ((ret = startup_validate(h, tmp_db, &xt, cbret)) < 0) goto done; if (ret == 0) goto fail; if (xt==NULL || xml_child_nr(xt)==0) goto ok; /* Merge tmp into running (no commit) */ - if ((ret = db_merge(h, db, "running", cbret)) < 0) + if ((ret = db_merge(h, tmp_db, "running", cbret)) < 0) goto fail; if (ret == 0) goto fail; @@ -270,7 +265,7 @@ startup_extraxml(clicon_handle h, retval = 1; done: xmldb_get0_free(h, &xt); - if (xmldb_delete(h, db) != 0 && errno != ENOENT) + if (xmldb_delete(h, tmp_db) != 0 && errno != ENOENT) return -1; return retval; fail: @@ -306,7 +301,7 @@ startup_failsafe(clicon_handle h) /* Copy original running to tmp as backup (restore if error) */ if (xmldb_copy(h, "running", "tmp") < 0) goto done; - if (startup_db_reset(h, "running") < 0) + if (xmldb_db_reset(h, "running") < 0) goto done; ret = candidate_commit(h, db, cbret); if (ret != 1) diff --git a/apps/backend/backend_startup.h b/apps/backend/backend_startup.h index 7eed20a6..52d1ef06 100644 --- a/apps/backend/backend_startup.h +++ b/apps/backend/backend_startup.h @@ -40,7 +40,6 @@ /* * Prototypes */ -int startup_db_reset(clicon_handle h, char *db); int startup_mode_startup(clicon_handle h, char *db, cbuf *cbret); int startup_extraxml(clicon_handle h, char *file, cbuf *cbret); int startup_failsafe(clicon_handle h); diff --git a/apps/cli/cli_main.c b/apps/cli/cli_main.c index f1ce87b2..3ca2c904 100644 --- a/apps/cli/cli_main.c +++ b/apps/cli/cli_main.c @@ -283,6 +283,7 @@ main(int argc, char **argv) yang_stmt *yspecfg = NULL; /* For config XXX clixon bug */ struct passwd *pw; char *str; + int tabmode; /* Defaults */ once = 0; @@ -542,7 +543,11 @@ main(int argc, char **argv) clicon_log(LOG_WARNING, "No such cli mode: %s (Specify cli mode with CLICON_CLI_MODE in config file or -m on command line", cli_syntax_mode(h)); /* CLIgen tab mode, ie how s behave */ - cligen_tabmode_set(cli_cligen(h), clicon_cli_tab_mode(h)); + if ((tabmode = clicon_cli_tab_mode(h)) < 0){ + fprintf(stderr, "FATAL: CLICON_CLI_TAB_MODE not set\n"); + goto done; + } + cligen_tabmode_set(cli_cligen(h), tabmode); if (logclisyntax) cli_logsyntax_set(h, logclisyntax); diff --git a/apps/netconf/netconf_main.c b/apps/netconf/netconf_main.c index f1432ec0..a2c90486 100644 --- a/apps/netconf/netconf_main.c +++ b/apps/netconf/netconf_main.c @@ -84,7 +84,7 @@ static int ignore_packet_errors = 1; */ static int netconf_input_packet(clicon_handle h, - cbuf *cb) + cbuf *cb) { int retval = -1; char *str; @@ -125,9 +125,10 @@ netconf_input_packet(clicon_handle h, isrpc++; if (xml_spec_populate_rpc(h, xrpc, yspec) < 0) goto done; - if ((ret = xml_yang_validate_rpc(h, xrpc, cbret)) < 0) + if ((ret = xml_yang_validate_rpc(h, xrpc, &xret)) < 0) goto done; if (ret == 0){ + clicon_xml2cbuf(cbret, xret, 0, 0); netconf_output_encap(1, cbret, "rpc-error"); goto ok; } @@ -155,7 +156,7 @@ netconf_input_packet(clicon_handle h, netconf_output_encap(1, cbret, "rpc-error"); goto done; } - if ((xc = xml_child_i(xret,0))!=NULL){ + if ((xc = xml_child_i(xret, 0))!=NULL){ xa=NULL; /* Copy message-id attribute from incoming to reply. * RFC 6241: diff --git a/apps/netconf/netconf_rpc.c b/apps/netconf/netconf_rpc.c index 3204de9a..6668c018 100644 --- a/apps/netconf/netconf_rpc.c +++ b/apps/netconf/netconf_rpc.c @@ -547,6 +547,7 @@ netconf_application_rpc(clicon_handle h, yang_stmt *yinput; yang_stmt *youtput; cxobj *xoutput; + cxobj *xerr = NULL; cbuf *cb = NULL; cbuf *cbret = NULL; int ret; @@ -587,15 +588,13 @@ netconf_application_rpc(clicon_handle h, xml_spec_set(xn, yinput); /* needed for xml_spec_populate */ if (xml_apply(xn, CX_ELMNT, xml_spec_populate, yspec) < 0) goto done; - if ((ret = xml_yang_validate_all_top(h, xn, cbret)) < 0) - goto done; - if (ret == 0){ - netconf_output_encap(1, cbret, "rpc-error"); - goto ok; - } - if ((ret = xml_yang_validate_add(h, xn, cbret)) < 0) + if ((ret = xml_yang_validate_all_top(h, xn, &xerr)) < 0) + goto done; + if (ret > 0 && (ret = xml_yang_validate_add(h, xn, &xerr)) < 0) goto done; if (ret == 0){ + if (clicon_xml2cbuf(cbret, xerr, 0, 0) < 0) + goto done; netconf_output_encap(1, cbret, "rpc-error"); goto ok; } @@ -622,15 +621,13 @@ netconf_application_rpc(clicon_handle h, if (xml_apply(xoutput, CX_ELMNT, xml_spec_populate, yspec) < 0) goto done; - if ((ret = xml_yang_validate_all_top(h, xoutput, cbret)) < 0) - goto done; - if (ret == 0){ - clicon_log(LOG_WARNING, "Errors in output netconf %s", cbuf_get(cbret)); - goto ok; - } - if ((ret = xml_yang_validate_add(h, xoutput, cbret)) < 0) + if ((ret = xml_yang_validate_all_top(h, xoutput, &xerr)) < 0) + goto done; + if (ret > 0 && (ret = xml_yang_validate_add(h, xoutput, &xerr)) < 0) goto done; if (ret == 0){ + if (clicon_xml2cbuf(cbret, xerr, 0, 0) < 0) + goto done; clicon_log(LOG_WARNING, "Errors in output netconf %s", cbuf_get(cbret)); goto ok; } @@ -641,6 +638,8 @@ netconf_application_rpc(clicon_handle h, ok: retval = 0; done: + if (xerr) + xml_free(xerr); if (cb) cbuf_free(cb); if (cbret) diff --git a/apps/restconf/restconf_methods.c b/apps/restconf/restconf_methods.c index 756d2cce..79571982 100644 --- a/apps/restconf/restconf_methods.c +++ b/apps/restconf/restconf_methods.c @@ -437,13 +437,17 @@ api_data_post(clicon_handle h, { int retval = -1; enum operation_type op = OP_CREATE; + cxobj *xdata0 = NULL; /* Original -d data struct (including top symbol) */ + cxobj *xdata; /* -d data (without top symbol)*/ int i; - cxobj *xdata = NULL; cbuf *cbx = NULL; - cxobj *xtop = NULL; /* xpath root */ - cxobj *xbot = NULL; - cxobj *x; - yang_stmt *y = NULL; + cxobj *xtop = NULL; /* top of api-path */ + cxobj *xbot = NULL; /* bottom of api-path */ + yang_stmt *ybot = NULL; /* yang of xbot */ +#ifdef RESTCONF_NS_DATA_CHECK + yang_stmt *ymodapi = NULL; /* yang module of api-path (if any) */ + yang_stmt *ymoddata = NULL; /* yang module of data (-d) */ +#endif yang_stmt *yspec; cxobj *xa; cxobj *xret = NULL; @@ -469,8 +473,12 @@ api_data_post(clicon_handle h, /* Translate api_path to xtop/xbot */ xbot = xtop; if (api_path){ - if ((ret = api_path2xml(api_path, yspec, xtop, YC_DATANODE, 1, &xbot, &y)) < 0) + if ((ret = api_path2xml(api_path, yspec, xtop, YC_DATANODE, 1, &xbot, &ybot)) < 0) goto done; +#ifdef RESTCONF_NS_DATA_CHECK + if (ybot) + ymodapi=ys_module(ybot); +#endif if (ret == 0){ /* validation failed */ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; @@ -486,7 +494,7 @@ api_data_post(clicon_handle h, } /* Parse input data as json or xml into xml */ if (parse_xml){ - if (xml_parse_string(data, NULL, &xdata) < 0){ + if (xml_parse_string(data, NULL, &xdata0) < 0){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -499,7 +507,7 @@ api_data_post(clicon_handle h, } } else { - if ((ret = json_parse_str(data, yspec, &xdata, &xerr)) < 0){ + if ((ret = json_parse_str(data, yspec, &xdata0, &xerr)) < 0){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -523,7 +531,7 @@ api_data_post(clicon_handle h, /* 4.4.1: The message-body MUST contain exactly one instance of the * expected data resource. */ - if (xml_child_nr(xdata) != 1){ + if (xml_child_nr(xdata0) != 1){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -534,15 +542,46 @@ api_data_post(clicon_handle h, goto done; goto ok; } - x = xml_child_i(xdata,0); + xdata = xml_child_i(xdata0,0); +#ifdef RESTCONF_NS_DATA_CHECK + /* If the api-path (above) defines a module, then xdata must have a prefix + * and it match the module defined in api-path. + * In a POST, maybe there are cornercases where xdata (which is a child) and + * xbot (which is the parent) may have non-matching namespaces? + * This does not apply if api-path is / (no module) + */ + if (ys_module_by_xml(yspec, xdata, &ymoddata) < 0) + goto done; + if (ymoddata && ymodapi){ + if (ymoddata != ymodapi){ + if (netconf_malformed_message_xml(&xerr, "Data is not prefixed with matching namespace") < 0) + goto done; + if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ + clicon_err(OE_XML, EINVAL, "rpc-error not found (internal error)"); + goto done; + } + if (debug){ + cbuf *ccc=cbuf_new(); + if (clicon_xml2cbuf(ccc, xe, 0, 0) < 0) + goto done; + clicon_debug(1, "%s XE:%s", __FUNCTION__, cbuf_get(ccc)); + } + + if (api_return_err(h, r, xe, pretty, use_xml) < 0) + goto done; + goto ok; + } + } +#endif /* RESTCONF_NS_DATA_CHECK */ + /* Add operation (create/replace) as attribute */ - if ((xa = xml_new("operation", x, NULL)) == NULL) + if ((xa = xml_new("operation", xdata, NULL)) == NULL) goto done; xml_type_set(xa, CX_ATTR); if (xml_value_set(xa, xml_operation2str(op)) < 0) goto done; /* Replace xbot with x, ie bottom of api-path with data */ - if (xml_addsub(xbot, x) < 0) + if (xml_addsub(xbot, xdata) < 0) goto done; /* Create text buffer for transfer to backend */ if ((cbx = cbuf_new()) == NULL) @@ -626,8 +665,8 @@ api_data_post(clicon_handle h, xml_free(xretdis); if (xtop) xml_free(xtop); - if (xdata) - xml_free(xdata); + if (xdata0) + xml_free(xdata0); if (cbx) cbuf_free(cbx); return retval; @@ -636,11 +675,14 @@ api_data_post(clicon_handle h, /*! Check matching keys * + * Check that x1 and x2 are of type list/leaf-list and share the same key statements + * I.e that if x1=b then x2 = b as + * well. Otherwise return -1. * @param[in] y Yang statement, should be list or leaf-list - * @param[in] xdata XML data tree - * @param[in] xapipath XML api-path tree + * @param[in] x1 First XML tree (eg data) + * @param[in] x2 Second XML tree (eg api-path) * @retval 0 Yes, keys match - * @retval -1 No keys do not match + * @retval -1 No, keys do not match * If the target resource represents a YANG leaf-list, then the PUT * method MUST NOT change the value of the leaf-list instance. * @@ -651,17 +693,17 @@ api_data_post(clicon_handle h, */ static int match_list_keys(yang_stmt *y, - cxobj *xdata, - cxobj *xapipath) + cxobj *x1, + cxobj *x2) { int retval = -1; cvec *cvk = NULL; /* vector of index keys */ cg_var *cvi; char *keyname; - cxobj *xkeya; /* xml key object in api-path */ - cxobj *xkeyd; /* xml key object in data */ - char *keya; - char *keyd; + cxobj *xkey1; /* xml key object of x1 */ + cxobj *xkey2; /* xml key object of x2 */ + char *key1; + char *key2; clicon_debug(1, "%s", __FUNCTION__); switch (yang_keyword_get(y)){ @@ -670,29 +712,30 @@ match_list_keys(yang_stmt *y, cvi = NULL; while ((cvi = cvec_each(cvk, cvi)) != NULL) { keyname = cv_string_get(cvi); - if ((xkeya = xml_find(xapipath, keyname)) == NULL) + if ((xkey2 = xml_find(x2, keyname)) == NULL) goto done; /* No key in api-path */ - if ((keya = xml_body(xkeya)) == NULL) + if ((key2 = xml_body(xkey2)) == NULL) goto done; - if ((xkeyd = xml_find(xdata, keyname)) == NULL) + if ((xkey1 = xml_find(x1, keyname)) == NULL) goto done; /* No key in data */ - if ((keyd = xml_body(xkeyd)) == NULL) + if ((key1 = xml_body(xkey1)) == NULL) goto done; - if (strcmp(keya, keyd) != 0) + if (strcmp(key2, key1) != 0) goto done; /* keys dont match */ } break; case Y_LEAF_LIST: - if ((keya = xml_body(xapipath)) == NULL) + if ((key2 = xml_body(x2)) == NULL) goto done; /* No key in api-path */ - if ((keyd = xml_body(xdata)) == NULL) + if ((key1 = xml_body(x1)) == NULL) goto done; /* No key in data */ - if (strcmp(keya, keyd) != 0) + if (strcmp(key2, key1) != 0) goto done; /* keys dont match */ break; default: - goto done; + goto ok; } + ok: retval = 0; done: clicon_debug(1, "%s retval:%d", __FUNCTION__, retval); @@ -738,13 +781,17 @@ api_data_put(clicon_handle h, int retval = -1; enum operation_type op = OP_REPLACE; int i; - cxobj *xdata = NULL; + cxobj *xdata0 = NULL; /* Original -d data struct (including top symbol) */ + cxobj *xdata; /* -d data (without top symbol)*/ cbuf *cbx = NULL; - cxobj *xtop = NULL; /* xpath root */ - cxobj *xbot = NULL; + cxobj *xtop = NULL; /* top of api-path */ + cxobj *xbot = NULL; /* bottom of api-path */ + yang_stmt *ybot = NULL; /* yang of xbot */ +#ifdef RESTCONF_NS_DATA_CHECK + yang_stmt *ymodapi = NULL; /* yang module of api-path (if any) */ + yang_stmt *ymoddata = NULL; /* yang module of data (-d) */ +#endif cxobj *xparent; - cxobj *x; - yang_stmt *y = NULL; yang_stmt *yp; /* yang parent */ yang_stmt *yspec; cxobj *xa; @@ -757,6 +804,7 @@ api_data_put(clicon_handle h, char *username; int ret; char *namespace0; + char *dname; clicon_debug(1, "%s api_path:\"%s\" json:\"%s\"", __FUNCTION__, api_path0, data); @@ -773,8 +821,12 @@ api_data_put(clicon_handle h, /* Translate api_path to xtop/xbot */ xbot = xtop; if (api_path){ - if ((ret = api_path2xml(api_path, yspec, xtop, YC_DATANODE, 1, &xbot, &y)) < 0) + if ((ret = api_path2xml(api_path, yspec, xtop, YC_DATANODE, 1, &xbot, &ybot)) < 0) goto done; +#ifdef RESTCONF_NS_DATA_CHECK + if (ybot) + ymodapi=ys_module(ybot); +#endif if (ret == 0){ /* validation failed */ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; @@ -788,9 +840,10 @@ api_data_put(clicon_handle h, goto ok; } } + /* Parse input data as json or xml into xml */ if (parse_xml){ - if (xml_parse_string(data, NULL, &xdata) < 0){ + if (xml_parse_string(data, yspec, &xdata0) < 0){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -803,7 +856,7 @@ api_data_put(clicon_handle h, } } else{ - if ((ret = json_parse_str(data, yspec, &xdata, &xerr)) < 0){ + if ((ret = json_parse_str(data, yspec, &xdata0, &xerr)) < 0){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -827,7 +880,7 @@ api_data_put(clicon_handle h, /* The message-body MUST contain exactly one instance of the * expected data resource. */ - if (xml_child_nr(xdata) != 1){ + if (xml_child_nr(xdata0) != 1){ if (netconf_malformed_message_xml(&xerr, clicon_err_reason) < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -838,34 +891,56 @@ api_data_put(clicon_handle h, goto done; goto ok; } - x = xml_child_i(xdata,0); + xdata = xml_child_i(xdata0,0); +#ifdef RESTCONF_NS_DATA_CHECK + /* If the api-path (above) defines a module, then xdata must have a prefix + * and it match the module defined in api-path + * This does not apply if api-path is / (no module) + */ + if (ys_module_by_xml(yspec, xdata, &ymoddata) < 0) + goto done; + if (ymoddata && ymodapi){ + if (ymoddata != ymodapi){ + if (netconf_malformed_message_xml(&xerr, "Data is not prefixed with matching namespace") < 0) + goto done; + if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ + clicon_err(OE_XML, EINVAL, "rpc-error not found (internal error)"); + goto done; + } + if (api_return_err(h, r, xe, pretty, use_xml) < 0) + goto done; + goto ok; + } + } +#endif /* RESTCONF_NS_DATA_CHECK */ + /* Add operation (create/replace) as attribute */ - if ((xa = xml_new("operation", x, NULL)) == NULL) + if ((xa = xml_new("operation", xdata, NULL)) == NULL) goto done; xml_type_set(xa, CX_ATTR); if (xml_value_set(xa, xml_operation2str(op)) < 0) goto done; -#if 0 - if (debug){ - cbuf *ccc=cbuf_new(); - if (clicon_xml2cbuf(ccc, xdata, 0, 0) < 0) - goto done; - clicon_debug(1, "%s DATA:%s", __FUNCTION__, cbuf_get(ccc)); - } -#endif - /* Replace xparent with x, ie bottom of api-path with data */ - if (api_path==NULL && strcmp(xml_name(x),"data")==0){ - if (xml_addsub(NULL, x) < 0) + + /* Top-of tree, no api-path + * Replace xparent with x, ie bottom of api-path with data + */ + dname = xml_name(xdata); + if (api_path==NULL && strcmp(dname,"data")==0){ + if (xml_addsub(NULL, xdata) < 0) goto done; if (xtop) xml_free(xtop); - xtop = x; + xtop = xdata; xml_name_set(xtop, "config"); } else { - clicon_debug(1, "%s x:%s xbot:%s",__FUNCTION__, xml_name(x), xml_name(xbot)); + /* There is an api-path that defines an element in the datastore tree. + * Not top-of-tree. + */ + clicon_debug(1, "%s x:%s xbot:%s",__FUNCTION__, dname, xml_name(xbot)); + /* Check same symbol in api-path as data */ - if (strcmp(xml_name(x), xml_name(xbot))){ + if (strcmp(dname, xml_name(xbot))){ if (netconf_operation_failed_xml(&xerr, "protocol", "Not same symbol in api-path as data") < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -884,27 +959,13 @@ api_data_put(clicon_handle h, * That is why the conditional is somewhat hairy */ xparent = xml_parent(xbot); -#if 1 - if (debug){ - cbuf *ccc=cbuf_new(); - if (clicon_xml2cbuf(ccc, xtop, 0, 0) < 0) - goto done; - clicon_debug(1, "%s AAA XPATH:%s", __FUNCTION__, cbuf_get(ccc)); - } - if (debug){ - cbuf *ccc=cbuf_new(); - if (clicon_xml2cbuf(ccc, xdata, 0, 0) < 0) - goto done; - clicon_debug(1, "%s DATA:%s", __FUNCTION__, cbuf_get(ccc)); - } -#endif - - if (y){ - yp = yang_parent_get(y); - if (((yang_keyword_get(y) == Y_LIST || yang_keyword_get(y) == Y_LEAF_LIST) && - match_list_keys(y, x, xbot) < 0) || - (yp && yang_keyword_get(yp) == Y_LIST && - match_list_keys(yp, xml_parent(x), xparent) < 0)){ + if (ybot){ + /* Ensure list keys match between uri and data. That is: + * If data is on the form: -d {"a":{"k":1}} where a is list or leaf-list + * then uri-path must be ../a=1 + * match_list_key() checks if this is true + */ + if (match_list_keys(ybot, xdata, xbot) < 0){ if (netconf_operation_failed_xml(&xerr, "protocol", "api-path keys do not match data keys") < 0) goto done; if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ @@ -914,15 +975,40 @@ api_data_put(clicon_handle h, if (api_return_err(h, r, xe, pretty, use_xml) < 0) goto done; goto ok; - + } + /* Ensure keys in lists are not changed. That is: + * If data is on the form: -d {"k":1} and its parent is a list "a" + * then the uri-path must be "../a=1 (you cannot change a's key)" + */ + if ((yp = yang_parent_get(ybot)) != NULL && + yang_keyword_get(yp) == Y_LIST){ + if ((ret = yang_key_match(yp, dname)) < 0) + goto done; + if (ret == 1){ /* Match: xdata is a key */ + char *parbod = xml_find_body(xparent, dname); + /* Check if the key is different from the one in uri-path, + * or does not exist + */ + if (parbod == NULL || strcmp(parbod, xml_body(xdata))){ + if (netconf_operation_failed_xml(&xerr, "protocol", "api-path keys do not match data keys") < 0) + goto done; + if ((xe = xpath_first(xerr, "rpc-error")) == NULL){ + clicon_err(OE_XML, EINVAL, "rpc-error not found (internal error)"); + goto done; + } + if (api_return_err(h, r, xe, pretty, use_xml) < 0) + goto done; + goto ok; + } + } } } xml_purge(xbot); - if (xml_addsub(xparent, x) < 0) + if (xml_addsub(xparent, xdata) < 0) goto done; /* If we already have that default namespace, remove it in child */ - if ((xa = xml_find_type(x, NULL, "xmlns", CX_ATTR)) != NULL){ + if ((xa = xml_find_type(xdata, NULL, "xmlns", CX_ATTR)) != NULL){ if (xml2ns(xparent, NULL, &namespace0) < 0) goto done; /* Set xmlns="" default namespace attribute (if diff from default) */ @@ -945,6 +1031,7 @@ api_data_put(clicon_handle h, clicon_debug(1, "%s xml: %s api_path:%s",__FUNCTION__, cbuf_get(cbx), api_path); if (clicon_rpc_netconf(h, cbuf_get(cbx), &xret, NULL) < 0) goto done; + if ((xe = xpath_first(xret, "//rpc-error")) != NULL){ if (api_return_err(h, r, xe, pretty, use_xml) < 0) goto done; @@ -1010,8 +1097,8 @@ api_data_put(clicon_handle h, xml_free(xretdis); if (xtop) xml_free(xtop); - if (xdata) - xml_free(xdata); + if (xdata0) + xml_free(xdata0); if (cbx) cbuf_free(cbx); return retval; @@ -1454,14 +1541,9 @@ api_operations_post_output(clicon_handle h, cxobj *xa; /* xml attribute (xmlns) */ cxobj *x; cxobj *xok; - cbuf *cbret = NULL; int isempty; // clicon_debug(1, "%s", __FUNCTION__); - if ((cbret = cbuf_new()) == NULL){ - clicon_err(OE_UNIX, 0, "cbuf_new"); - goto done; - } /* Validate that exactly only tag */ if ((xoutput = xml_child_i_type(xret, 0, CX_ELMNT)) == NULL || strcmp(xml_name(xoutput),"rpc-reply") != 0 || @@ -1499,14 +1581,12 @@ api_operations_post_output(clicon_handle h, #if 0 if (xml_apply(xoutput, CX_ELMNT, xml_spec_populate, yspec) < 0) goto done; - if ((ret = xml_yang_validate_all(xoutput, cbret)) < 0) + if ((ret = xml_yang_validate_all(xoutput, &xerr)) < 0) goto done; if (ret == 1 && - (ret = xml_yang_validate_add(h, xoutput, cbret)) < 0) + (ret = xml_yang_validate_add(h, xoutput, &xerr)) < 0) goto done; if (ret == 0){ /* validation failed */ - if (xml_parse_string(cbuf_get(cbret), yspec, &xerr) < 0) - goto done; if ((xe = xpath_first(xerr, "rpc-reply/rpc-error")) == NULL){ clicon_err(OE_XML, EINVAL, "rpc-error not found (internal error)"); goto done; @@ -1551,8 +1631,6 @@ api_operations_post_output(clicon_handle h, retval = 1; done: clicon_debug(1, "%s retval: %d", __FUNCTION__, retval); - if (cbret) - cbuf_free(cbret); if (xerr) xml_free(xerr); return retval; @@ -1738,14 +1816,12 @@ api_operations_post(clicon_handle h, /* 6. Validate incoming RPC and fill in defaults */ if (xml_spec_populate_rpc(h, xtop, yspec) < 0) /* */ goto done; - if ((ret = xml_yang_validate_rpc(h, xtop, cbret)) < 0) + if ((ret = xml_yang_validate_rpc(h, xtop, &xret)) < 0) goto done; if (ret == 0){ - if (xml_parse_string(cbuf_get(cbret), NULL, &xret) < 0) - goto done; - if ((xe = xpath_first(xret, "rpc-reply/rpc-error")) == NULL){ + if ((xe = xpath_first(xret, "rpc-error")) == NULL){ clicon_err(OE_XML, EINVAL, "rpc-error not found (internal error)"); - goto done; + goto ok; } if (api_return_err(h, r, xe, pretty, use_xml) < 0) goto done; diff --git a/configure b/configure index 79ba3200..a308e81c 100755 --- a/configure +++ b/configure @@ -621,7 +621,8 @@ ac_includes_default="\ ac_subst_vars='LTLIBOBJS LIBOBJS -CLIXON_DATADIR +STD_YANG_INSTALLDIR +YANG_INSTALLDIR EGREP GREP LEXLIB @@ -637,7 +638,6 @@ wwwuser wwwdir enable_stdyangs with_restconf -RANLIB SH_SUFFIX CLIXON_DEFAULT_CONFIG INSTALLFLAGS @@ -718,6 +718,8 @@ with_restconf with_wwwuser with_configfile with_libxml2 +with_yang_installdir +with_std_yang_installdir ' ac_precious_vars='build_alias host_alias @@ -1367,6 +1369,8 @@ Optional Packages: --with-wwwuser= Set www user different from www-data --with-configfile=FILE set default path to config file --with-libxml2 use gnome/libxml2 regex engine + --with-yang-installdir=DIR Install Clixon yang files here (default: ${prefix}/share/clixon) + --with-std-yang-installdir=DIR Install standard yang files here (default: ${prefix}/share/clixon) Some influential environment variables: CC C compiler command @@ -2463,7 +2467,6 @@ test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644' - # If yes, compile apps/restconf wwwdir=/www-data @@ -4493,13 +4496,39 @@ fi done -# CLIXON_DATADIR is where clixon installs the "system" yang files in yang/Makfile -# This directory should most probably be included in each application, -# so each application designer may need to place CLIXON_DATADIR in their config -# (last in yang dir list): -# $CLIXON_DATADIR +# YANG_INSTALLDIR is where clixon installs the Clixon yang files +# (the files in in yang/clixon) +# Each application designer may need to place CLIXON_YANG_DIR in their config: +# $YANG_INSTALLDIR -CLIXON_DATADIR="${prefix}/share/clixon" +# Check whether --with-yang-installdir was given. +if test "${with_yang_installdir+set}" = set; then : + withval=$with_yang_installdir; YANG_INSTALLDIR="$withval" +else + YANG_INSTALLDIR="${prefix}/share/clixon" + +fi + + +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Clixon yang files are installed in ${YANG_INSTALLDIR}" >&5 +$as_echo "Clixon yang files are installed in ${YANG_INSTALLDIR}" >&6; } + +# STD_YANG_INSTALLDIR is where clixon installs standard yang files +# (the files in in yang/standard) +# that Clixon needs to run (or examples rely on). These may be retreived from +# elsewhere (eg yangmodels repo) + +# Check whether --with-std-yang-installdir was given. +if test "${with_std_yang_installdir+set}" = set; then : + withval=$with_std_yang_installdir; STD_YANG_INSTALLDIR="$withval" +else + STD_YANG_INSTALLDIR="${prefix}/share/clixon" + +fi + + +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Standard yang files are installed in ${STD_YANG_INSTALLDIR}" >&5 +$as_echo "Standard yang files are installed in ${STD_YANG_INSTALLDIR}" >&6; } # Default location for config file diff --git a/configure.ac b/configure.ac index ee1946d2..13885d43 100644 --- a/configure.ac +++ b/configure.ac @@ -89,7 +89,6 @@ AC_SUBST(INSTALLFLAGS) AC_SUBST(CLIXON_DEFAULT_CONFIG) AC_SUBST(LIBS) AC_SUBST(SH_SUFFIX) -AC_SUBST(RANLIB) AC_SUBST(with_restconf) # If yes, compile apps/restconf AC_SUBST(enable_stdyangs) AC_SUBST(wwwdir,/www-data) @@ -232,13 +231,29 @@ fi # AC_CHECK_FUNCS(inet_aton sigaction sigvec strlcpy strsep strndup alphasort versionsort) -# CLIXON_DATADIR is where clixon installs the "system" yang files in yang/Makfile -# This directory should most probably be included in each application, -# so each application designer may need to place CLIXON_DATADIR in their config -# (last in yang dir list): -# $CLIXON_DATADIR -AC_SUBST(CLIXON_DATADIR) -CLIXON_DATADIR="${prefix}/share/clixon" +# YANG_INSTALLDIR is where clixon installs the Clixon yang files +# (the files in in yang/clixon) +# Each application designer may need to place YANG_INSTALLDIR in their config: +# $YANG_INSTALLDIR +AC_ARG_WITH(yang-installdir, + [ --with-yang-installdir=DIR Install Clixon yang files here (default: ${prefix}/share/clixon) ], + [YANG_INSTALLDIR="$withval"], + [YANG_INSTALLDIR="${prefix}/share/clixon"] + ) +AC_SUBST(YANG_INSTALLDIR) +AC_MSG_RESULT(Clixon yang files are installed in ${YANG_INSTALLDIR}) + +# STD_YANG_INSTALLDIR is where clixon installs standard yang files +# (the files in in yang/standard) +# that Clixon needs to run (or examples rely on). These may be retreived from +# elsewhere (eg yangmodels repo) +AC_ARG_WITH(std-yang-installdir, + [ --with-std-yang-installdir=DIR Install standard yang files here (default: ${prefix}/share/clixon) ], + [STD_YANG_INSTALLDIR="$withval"], + [STD_YANG_INSTALLDIR="${prefix}/share/clixon"] + ) +AC_SUBST(STD_YANG_INSTALLDIR) +AC_MSG_RESULT(Standard yang files are installed in ${STD_YANG_INSTALLDIR}) # Default location for config file AC_DEFINE_UNQUOTED(CLIXON_DEFAULT_CONFIG,"${CLIXON_DEFAULT_CONFIG}",[Location for apps to find default config file]) diff --git a/doc/FAQ.md b/doc/FAQ.md index c052e9ae..37af03a0 100644 --- a/doc/FAQ.md +++ b/doc/FAQ.md @@ -305,7 +305,7 @@ The following configuration file options control the loading of Yang files: - `CLICON_YANG_MODULE_REVISION` : Specifies a revision to the main module. - `CLICON_YANG_MAIN_DIR` - Load all yang modules in this directory. -Note that the special `CLIXON_DATADIR`, by default `/usr/local/share/clixon` should be included in the yang dir path for Clixon system files to be found. +Note that the special `YANG_INSTALLDIR`, by default `/usr/local/share/clixon` should be included in the yang dir path for Clixon system files to be found. You can combine the options, however, if there are different variants of the same module, more specific options override less diff --git a/doc/INSTALL.md b/doc/INSTALL.md new file mode 100644 index 00000000..6efca64f --- /dev/null +++ b/doc/INSTALL.md @@ -0,0 +1,47 @@ +# Building Clixon + +Clixon runs on Linux, [FreeBSD port](https://www.freshports.org/devel/clixon) and Mac/Apple. CPU architecures include x86_64, i686, ARM32. + +## Ubuntu Linux + +### Installing dependencies + +Install packages +``` +sudo apt-get update +sudo apt-get install flex bison fcgi-dev curl-dev +``` + +Install and build CLIgen +``` + git clone https://github.com/olofhagsand/cligen.git + cd cligen; + configure; + make; + make install +``` + +Add a user group, using groupadd and usermod: +``` + sudo groupadd clicon # + sudo usermod -a -G clicon + sudo usermod -a -G clicon www-data +``` + + +### Build from source +``` + configure # Configure clixon to platform + make # Compile + sudo make install # Install libs, binaries, and config-files + sudo make install-include # Install include files (for compiling) +``` + +## Alpine Linux +Docker is used to build Alpine Linux +### Build docker image + +## FreeBSD +### Package install +### Build from source + diff --git a/example/hello/Makefile.in b/example/hello/Makefile.in index 7ba400aa..f29d11ef 100644 --- a/example/hello/Makefile.in +++ b/example/hello/Makefile.in @@ -45,7 +45,7 @@ libdir = @exec_prefix@/lib APPNAME = hello # Here is where example yang appears -CLIXON_DATADIR = @CLIXON_DATADIR@ +YANG_INSTALLDIR = @YANG_INSTALLDIR@ # Install here if you want default clixon location: CLIXON_DEFAULT_CONFIG = @CLIXON_DEFAULT_CONFIG@ @@ -125,7 +125,7 @@ install: $(YANGSPECS) $(CLISPECS) $(PLUGINS) $(APPNAME).xml install -d -m 0755 $(DESTDIR)$(libdir)/$(APPNAME)/clispec install -m 0644 $(CLISPECS) $(DESTDIR)$(libdir)/$(APPNAME)/clispec install -d -m 0755 $(DESTDIR)$(datarootdir)/$(APPNAME)/yang - install -m 0644 $(YANGSPECS) $(DESTDIR)$(DESTDIR)$(CLIXON_DATADIR) + install -m 0644 $(YANGSPECS) $(DESTDIR)$(DESTDIR)$(YANG_INSTALLDIR) install -d -m 0755 $(DESTDIR)$(localstatedir)/$(APPNAME) # Uncomment for installing config file in /usr/local/etc instead diff --git a/example/main/Makefile.in b/example/main/Makefile.in index d0e502b5..363d460b 100644 --- a/example/main/Makefile.in +++ b/example/main/Makefile.in @@ -44,7 +44,7 @@ libdir = @exec_prefix@/lib APPNAME = example # Here is where example yang appears -CLIXON_DATADIR = @CLIXON_DATADIR@ +YANG_INSTALLDIR = @YANG_INSTALLDIR@ # Install here if you want default clixon location: CLIXON_DEFAULT_CONFIG = @CLIXON_DEFAULT_CONFIG@ @@ -129,7 +129,7 @@ install: $(YANGSPECS) $(CLISPECS) $(PLUGINS) $(APPNAME).xml install -m 0644 $(APPNAME).xml $(DESTDIR)$(sysconfdir) # install -m 0644 $(APPNAME).xml $(DESTDIR)$(CLIXON_DEFAULT_CONFIG) install -d -m 0755 $(DESTDIR)$(datarootdir)/$(APPNAME)/yang - install -m 0644 $(YANGSPECS) $(DESTDIR)$(DESTDIR)$(CLIXON_DATADIR) + install -m 0644 $(YANGSPECS) $(DESTDIR)$(DESTDIR)$(YANG_INSTALLDIR) install -d -m 0755 $(DESTDIR)$(libdir)/$(APPNAME)/cli install -m 0644 $(INSTALLFLAGS) $(CLI_PLUGIN) $(DESTDIR)$(libdir)/$(APPNAME)/cli install -d -m 0755 $(DESTDIR)$(libdir)/$(APPNAME)/backend diff --git a/include/clixon_custom.h b/include/clixon_custom.h index c302d0ed..69ffe850 100644 --- a/include/clixon_custom.h +++ b/include/clixon_custom.h @@ -41,6 +41,6 @@ */ #undef RPC_USERNAME_ASSERT -/* Use new xml_insert code on sorted xml lists +/* Make namespace check on RESTCONF PUT and POST -d data */ -#define USE_XML_INSERT +#define RESTCONF_NS_DATA_CHECK diff --git a/lib/clixon/clixon_datastore.h b/lib/clixon/clixon_datastore.h index f29903bf..a600c30d 100644 --- a/lib/clixon/clixon_datastore.h +++ b/lib/clixon/clixon_datastore.h @@ -62,5 +62,7 @@ int xmldb_islocked(clicon_handle h, const char *db); int xmldb_exists(clicon_handle h, const char *db); int xmldb_delete(clicon_handle h, const char *db); int xmldb_create(clicon_handle h, const char *db); +/* utility functions */ +int xmldb_db_reset(clicon_handle h, char *db); #endif /* _CLIXON_DATASTORE_H */ diff --git a/lib/clixon/clixon_netconf_lib.h b/lib/clixon/clixon_netconf_lib.h index 2953ab5f..f3651da5 100644 --- a/lib/clixon/clixon_netconf_lib.h +++ b/lib/clixon/clixon_netconf_lib.h @@ -32,7 +32,8 @@ ***** END LICENSE BLOCK ***** * Netconf library functions. See RFC6241 - * + * Functions to generate a netconf error message come in two forms: xml-tree and + * cbuf. XML tree is preferred. */ #ifndef _CLIXON_NETCONF_LIB_H #define _CLIXON_NETCONF_LIB_H @@ -60,16 +61,18 @@ int netconf_lock_denied(cbuf *cb, char *info, char *message); int netconf_resource_denied(cbuf *cb, char *type, char *message); int netconf_rollback_failed(cbuf *cb, char *type, char *message); int netconf_data_exists(cbuf *cb, char *message); -int netconf_data_missing(cbuf *cb, char *message); +int netconf_data_missing(cbuf *cb, char *missing_choice, char *message); +int netconf_data_missing_xml(cxobj **xret, char *missing_choice, char *message); int netconf_operation_not_supported(cbuf *cb, char *type, char *message); int netconf_operation_failed(cbuf *cb, char *type, char *message); int netconf_operation_failed_xml(cxobj **xret, char *type, char *message); int netconf_malformed_message(cbuf *cb, char *message); int netconf_malformed_message_xml(cxobj **xret, char *message); -int netconf_data_not_unique(cbuf *cb, cxobj *x, cvec *cvk); -int netconf_minmax_elements(cbuf *cb, cxobj *x, int max); +int netconf_data_not_unique_xml(cxobj **xret, cxobj *x, cvec *cvk); +int netconf_minmax_elements_xml(cxobj **xret, cxobj *x, int max); int netconf_trymerge(cxobj *x, yang_stmt *yspec, cxobj **xret); int netconf_module_load(clicon_handle h); char *netconf_db_find(cxobj *xn, char *name); +int netconf_err2cb(cxobj *xerr, cbuf **cberr); #endif /* _CLIXON_NETCONF_LIB_H */ diff --git a/lib/clixon/clixon_xml_map.h b/lib/clixon/clixon_xml_map.h index a15e68b8..b6a09ccc 100644 --- a/lib/clixon/clixon_xml_map.h +++ b/lib/clixon/clixon_xml_map.h @@ -48,11 +48,11 @@ int xml2txt(FILE *f, cxobj *x, int level); int xml2cli(FILE *f, cxobj *x, char *prepend, enum genmodel_type gt); int xml_yang_root(cxobj *x, cxobj **xr); int xmlns_assign(cxobj *x); -int xml_yang_validate_rpc(clicon_handle h, cxobj *xrpc, cbuf *cbret); -int xml_yang_validate_list_key_only(clicon_handle h, cxobj *xt, cbuf *cbret); -int xml_yang_validate_add(clicon_handle h, cxobj *xt, cbuf *cbret); -int xml_yang_validate_all(clicon_handle h, cxobj *xt, cbuf *cbret); -int xml_yang_validate_all_top(clicon_handle h, cxobj *xt, cbuf *cbret); +int xml_yang_validate_rpc(clicon_handle h, cxobj *xrpc, cxobj **xret); +int xml_yang_validate_list_key_only(clicon_handle h, cxobj *xt, cxobj **xret); +int xml_yang_validate_add(clicon_handle h, cxobj *xt, cxobj **xret); +int xml_yang_validate_all(clicon_handle h, cxobj *xt, cxobj **xret); +int xml_yang_validate_all_top(clicon_handle h, cxobj *xt, cxobj **xret); int xml2cvec(cxobj *xt, yang_stmt *ys, cvec **cvv0); int cvec2xml_1(cvec *cvv, char *toptag, cxobj *xp, cxobj **xt0); diff --git a/lib/src/clixon_datastore.c b/lib/src/clixon_datastore.c index 78e5c300..a7becb1c 100644 --- a/lib/src/clixon_datastore.c +++ b/lib/src/clixon_datastore.c @@ -78,6 +78,7 @@ #include "clixon_datastore_write.h" #include "clixon_datastore_read.h" + /*! Translate from symbolic database name to actual filename in file-system * @param[in] th text handle handle * @param[in] db Symbolic database name, eg "candidate", "running" @@ -446,3 +447,21 @@ xmldb_create(clicon_handle h, close(fd); return retval; } + +/*! Create an XML database. If it exists already, delete it before creating + * Utility function. + * @param[in] h Clixon handle + * @param[in] db Symbolic database name, eg "candidate", "running" + */ +int +xmldb_db_reset(clicon_handle h, + char *db) +{ + if (xmldb_exists(h, db) == 1){ + if (xmldb_delete(h, db) != 0 && errno != ENOENT) + return -1; + } + if (xmldb_create(h, db) < 0) + return -1; + return 0; +} diff --git a/lib/src/clixon_datastore_read.c b/lib/src/clixon_datastore_read.c index eb099521..b4358ae7 100644 --- a/lib/src/clixon_datastore_read.c +++ b/lib/src/clixon_datastore_read.c @@ -641,7 +641,8 @@ xmldb_get_zerocopy(clicon_handle h, * @param[in] db Name of database to search in (filename including dir path * @param[in] xpath String with XPATH syntax. or NULL for all * @param[out] xret Single return XML tree. Free with xml_free() - + * @retval 0 OK + * @retval -1 Error * @code * if (xmldb_get(xh, "running", "/interfaces/interface[name="eth"]", &xt) < 0) * err; diff --git a/lib/src/clixon_datastore_write.c b/lib/src/clixon_datastore_write.c index 3989e816..fff7558d 100644 --- a/lib/src/clixon_datastore_write.c +++ b/lib/src/clixon_datastore_write.c @@ -154,17 +154,10 @@ text_modify(clicon_handle h, goto fail; permit = 1; } - // int iamkey=0; - -#ifdef USE_XML_INSERT /* Add new xml node but without parent - insert when node fully copied (see changed conditional below) */ if ((x0 = xml_new(x1name, NULL, (yang_stmt*)y0)) == NULL) goto done; -#else - if ((x0 = xml_new(x1name, x0p, (yang_stmt*)y0)) == NULL) - goto done; -#endif changed++; /* Copy xmlns attributes */ @@ -210,16 +203,14 @@ text_modify(clicon_handle h, } } } -#ifdef USE_XML_INSERT if (changed){ if (xml_insert(x0p, x0) < 0) goto done; } -#endif break; case OP_DELETE: if (x0==NULL){ - if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0) + if (netconf_data_missing(cbret, NULL, "Data does not exist; cannot delete resource") < 0) goto done; goto fail; } @@ -295,17 +286,12 @@ text_modify(clicon_handle h, goto fail; permit = 1; } -#ifdef USE_XML_INSERT /* Add new xml node but without parent - insert when node fully * copied (see changed conditional below) * Note x0 may dangle cases if exit before changed conditional */ if ((x0 = xml_new(x1name, NULL, (yang_stmt*)y0)) == NULL) goto done; -#else - if ((x0 = xml_new(x1name, x0p, (yang_stmt*)y0)) == NULL) - goto done; -#endif changed++; /* Copy xmlns attributes */ x1a = NULL; @@ -367,16 +353,14 @@ text_modify(clicon_handle h, if (ret == 0) goto fail; } -#ifdef USE_XML_INSERT if (changed){ if (xml_insert(x0p, x0) < 0) goto done; } -#endif break; case OP_DELETE: if (x0==NULL){ - if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0) + if (netconf_data_missing(cbret, NULL, "Data does not exist; cannot delete resource") < 0) goto done; goto fail; } @@ -396,17 +380,11 @@ text_modify(clicon_handle h, break; } /* CONTAINER switch op */ } /* else Y_CONTAINER */ -#ifndef USE_XML_INSERT - if (changed) - xml_sort(x0p, h); -#endif retval = 1; done: -#ifdef USE_XML_INSERT /* Remove dangling added objects */ if (changed && x0 && xml_parent(x0)==NULL) xml_purge(x0); -#endif if (x0vec) free(x0vec); return retval; @@ -488,7 +466,7 @@ text_modify_top(clicon_handle h, I.e., curl -u andy:bar -sS -X DELETE http://localhost/restconf/data */ case OP_DELETE: - if (netconf_data_missing(cbret, "Data does not exist; cannot delete resource") < 0) + if (netconf_data_missing(cbret, NULL, "Data does not exist; cannot delete resource") < 0) goto done; goto fail; break; diff --git a/lib/src/clixon_netconf_lib.c b/lib/src/clixon_netconf_lib.c index b02fa04d..bf59a9bb 100644 --- a/lib/src/clixon_netconf_lib.c +++ b/lib/src/clixon_netconf_lib.c @@ -32,6 +32,8 @@ ***** END LICENSE BLOCK ***** * Netconf library functions. See RFC6241 + * Functions to generate a netconf error message come in two forms: xml-tree and + * cbuf. XML tree is preferred. */ #ifdef HAVE_CONFIG_H @@ -63,6 +65,8 @@ #include "clixon_options.h" #include "clixon_data.h" #include "clixon_xml_map.h" +#include "clixon_xpath_ctx.h" +#include "clixon_xpath.h" #include "clixon_netconf_lib.h" /*! Create Netconf in-use error XML tree according to RFC 6241 Appendix A @@ -70,7 +74,7 @@ * The request requires a resource that already is in use. * @param[out] cb CLIgen buf. Error XML is written in this buffer * @param[in] type Error type: "application" or "protocol" - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ int netconf_in_use(cbuf *cb, @@ -109,7 +113,7 @@ netconf_in_use(cbuf *cb, * The request specifies an unacceptable value for one or more parameters. * @param[out] cb CLIgen buf. Error XML is written in this buffer * @param[in] type Error type: "application" or "protocol" - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ int netconf_invalid_value(cbuf *cb, @@ -149,7 +153,7 @@ netconf_invalid_value(cbuf *cb, * too large for the implementation to handle. * @param[out] cb CLIgen buf. Error XML is written in this buffer * @param[in] type Error type: "transport", "rpc", "application", "protocol" - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ int netconf_too_big(cbuf *cb, @@ -189,7 +193,7 @@ netconf_too_big(cbuf *cb, * @param[out] cb CLIgen buf. Error XML is written in this buffer * @param[in] type Error type: "rpc", "application" or "protocol" * @param[in] info bad-attribute or bad-element xml - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ int netconf_missing_attribute(cbuf *cb, @@ -230,7 +234,7 @@ netconf_missing_attribute(cbuf *cb, * @param[out] cb CLIgen buf. Error XML is written in this buffer * @param[in] type Error type: "rpc", "application" or "protocol" * @param[in] info bad-attribute or bad-element xml - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ int netconf_bad_attribute(cbuf *cb, @@ -313,7 +317,7 @@ netconf_unknown_attribute(cbuf *cb, * @param[in] type Error type: "application" or "protocol" * @param[in] tag Error tag * @param[in] element bad-element xml - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) */ static int netconf_common_xml(cxobj **xret, @@ -323,8 +327,9 @@ netconf_common_xml(cxobj **xret, char *element, char *message) { - int retval =-1; + int retval =-1; cxobj *xerr; + char *encstr = NULL; if (*xret == NULL){ if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) @@ -340,11 +345,17 @@ netconf_common_xml(cxobj **xret, "error", type, tag, infotag, element, infotag) < 0) goto done; - if (message && xml_parse_va(&xerr, NULL, "%s", - message) < 0) - goto done; + if (message){ + if (xml_chardata_encode(&encstr, "%s", message) < 0) + goto done; + if (xml_parse_va(&xerr, NULL, "%s", + encstr) < 0) + goto done; + } retval = 0; done: + if (encstr) + free(encstr); return retval; } @@ -551,7 +562,7 @@ netconf_access_denied(cbuf *cb, * authorization failed. * @param[out] xret Error XML tree. Free with xml_free after use * @param[in] type Error type: "application" or "protocol" - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) * @code * cxobj *xret = NULL; * if (netconf_access_denied_xml(&xret, "protocol", "Unauthorized") < 0) @@ -565,9 +576,10 @@ netconf_access_denied_xml(cxobj **xret, char *type, char *message) { - int retval =-1; + int retval =-1; cxobj *xerr; - + char *encstr = NULL; + if (*xret == NULL){ if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) goto done; @@ -580,11 +592,17 @@ netconf_access_denied_xml(cxobj **xret, "access-denied" "error", type) < 0) goto done; - if (message && xml_parse_va(&xerr, NULL, "%s", - message) < 0) - goto done; + if (message){ + if (xml_chardata_encode(&encstr, "%s", message) < 0) + goto done; + if (xml_parse_va(&xerr, NULL, "%s", + encstr) < 0) + goto done; + } retval = 0; done: + if (encstr) + free(encstr); return retval; } @@ -752,38 +770,81 @@ netconf_data_exists(cbuf *cb, * does not exist. For example, a "delete" operation was attempted on * data that does not exist. * @param[out] cb CLIgen buf. Error XML is written in this buffer + * @param[in] missing_choice If set, see RFC7950: 15.6 violates mandatiry choice * @param[in] message Error message */ int netconf_data_missing(cbuf *cb, + char *missing_choice, char *message) { int retval = -1; - char *encstr = NULL; + cxobj *xret = NULL; - if (cprintf(cb, "" - "application" - "data-missing" - "error") <0) - goto err; + if (netconf_data_missing_xml(&xret, missing_choice, message) < 0) + goto done; + if (clicon_xml2cbuf(cb, xret, 0, 0) < 0) + goto done; + retval = 0; + done: + if (xret) + xml_free(xret); + return retval; +} + +/*! Create Netconf data-missing error XML tree according to RFC 6241 App A + * + * Request could not be completed because the relevant data model content + * does not exist. For example, a "delete" operation was attempted on + * data that does not exist. + * @param[out] xret Error XML tree. Free with xml_free after use + * @param[in] missing_choice If set, see RFC7950: 15.6 violates mandatiry choice + * @param[in] message Error message + */ +int +netconf_data_missing_xml(cxobj **xret, + char *missing_choice, + char *message) +{ + int retval = -1; + char *encstr = NULL; + cxobj *xerr; + + if (*xret == NULL){ + if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) + goto done; + } + else if (xml_name_set(*xret, "rpc-reply") < 0) + goto done; + if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL) + goto done; + if (xml_parse_va(&xerr, NULL, + "application" + "data-missing") < 0) + goto done; + if (missing_choice) /* NYI: RFC7950: 15.6 */ + if (xml_parse_va(&xerr, NULL, + "missing-choice" + "%s", + missing_choice) < 0) + goto done; + if (xml_parse_va(&xerr, NULL, + "error") < 0) + goto done; if (message){ if (xml_chardata_encode(&encstr, "%s", message) < 0) goto done; - if (cprintf(cb, "%s", encstr) < 0) - goto err; + if (xml_parse_va(&xerr, NULL, + "%s", encstr) < 0) + goto done; } - if (cprintf(cb, "") <0) - goto err; retval = 0; done: if (encstr) free(encstr); return retval; - err: - clicon_err(OE_XML, errno, "cprintf"); - goto done; } - + /*! Create Netconf operation-not-supported error XML according to RFC 6241 App A * * Request could not be completed because the requested operation is not @@ -858,7 +919,7 @@ netconf_operation_failed(cbuf *cb, * some reason not covered by any other error condition. * @param[out] xret Error XML tree * @param[in] type Error type: "rpc", "application" or "protocol" - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) * @code * cxobj *xret = NULL; * if (netconf_operation_failed_xml(&xret, "protocol", "Unauthorized") < 0) @@ -874,6 +935,7 @@ netconf_operation_failed_xml(cxobj **xret, { int retval =-1; cxobj *xerr; + char *encstr = NULL; if (*xret == NULL){ if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) @@ -887,11 +949,17 @@ netconf_operation_failed_xml(cxobj **xret, "operation-failed" "error", type) < 0) goto done; - if (message && xml_parse_va(&xerr, NULL, "%s", - message) < 0) + if (message){ + if (xml_chardata_encode(&encstr, "%s", message) < 0) + goto done; + if (xml_parse_va(&xerr, NULL, "%s", + encstr) < 0) goto done; + } retval = 0; done: + if (encstr) + free(encstr); return retval; } @@ -929,7 +997,7 @@ netconf_malformed_message(cbuf *cb, * For example, the message is not well-formed XML or it uses an * invalid character set. * @param[out] xret Error XML tree - * @param[in] message Error message + * @param[in] message Error message (will be XML encoded) * @note New in :base:1.1 * @code * cxobj *xret = NULL; @@ -943,8 +1011,9 @@ int netconf_malformed_message_xml(cxobj **xret, char *message) { - int retval =-1; + int retval =-1; cxobj *xerr; + char *encstr = NULL; if (*xret == NULL){ if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) @@ -958,11 +1027,17 @@ netconf_malformed_message_xml(cxobj **xret, "malformed-message" "error") < 0) goto done; - if (message && xml_parse_va(&xerr, NULL, "%s", - message) < 0) - goto done; + if (message){ + if (xml_chardata_encode(&encstr, "%s", message) < 0) + goto done; + if (xml_parse_va(&xerr, NULL, "%s", + encstr) < 0) + goto done; + } retval = 0; done: + if (encstr) + free(encstr); return retval; } @@ -970,79 +1045,97 @@ netconf_malformed_message_xml(cxobj **xret, * * A NETCONF operation would result in configuration data where a * "unique" constraint is invalidated. - * @param[out] cb CLIgen buf. Error XML is written in this buffer - * @param[in] x List element containing duplicate - * @param[in] cvk List of comonents in x that are non-unique + * @param[out] xret Error XML tree. Free with xml_free after use + * @param[in] x List element containing duplicate + * @param[in] cvk List of comonents in x that are non-unique * @see RFC7950 Sec 15.1 */ int -netconf_data_not_unique(cbuf *cb, - cxobj *x, - cvec *cvk) +netconf_data_not_unique_xml(cxobj **xret, + cxobj *x, + cvec *cvk) { int retval = -1; cg_var *cvi = NULL; cxobj *xi; + cxobj *xerr; + cxobj *xinfo; + cbuf *cb = NULL; - if (cprintf(cb, "" - "protocol" - "operation-failed" - "data-not-unique" - "error" - "") < 0) - goto err; - while ((cvi = cvec_each(cvk, cvi)) != NULL){ - if ((xi = xml_find(x, cv_string_get(cvi))) == NULL) - continue; /* ignore, shouldnt happen */ - cprintf(cb, ""); - clicon_xml2cbuf(cb, xi, 0, 0); - cprintf(cb, ""); + if (*xret == NULL){ + if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) + goto done; + } + else if (xml_name_set(*xret, "rpc-reply") < 0) + goto done; + if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL) + goto done; + if (xml_parse_va(&xerr, NULL, "protocol" + "operation-failed" + "data-not-unique" + "error") < 0) + goto done; + if (cvec_len(cvk)){ + if ((xinfo = xml_new("error-info", xerr, NULL)) == NULL) + goto done; + if ((cb = cbuf_new()) == NULL){ + clicon_err(OE_UNIX, errno, "cbuf_new"); + goto done; + } + while ((cvi = cvec_each(cvk, cvi)) != NULL){ + if ((xi = xml_find(x, cv_string_get(cvi))) == NULL) + continue; /* ignore, shouldnt happen */ + clicon_xml2cbuf(cb, xi, 0, 0); + if (xml_parse_va(&xinfo, NULL, "%s", cbuf_get(cb)) < 0) + goto done; + cbuf_reset(cb); + } } - if (cprintf(cb, "") <0) - goto err; retval = 0; done: + if (cb) + cbuf_free(cb); return retval; - err: - clicon_err(OE_XML, errno, "cprintf"); - goto done; } /*! Create Netconf too-many/few-elements err msg according to RFC 7950 15.2/15.3 * * A NETCONF operation would result in configuration data where a list or a leaf-list would have too many entries, the following error - * @param[out] cb CLIgen buf. Error XML is written in this buffer + * @param[out] xret Error XML tree. Free with xml_free after use * @param[in] x List element containing duplicate * @param[in] max If set, return too-many, otherwise too-few * @see RFC7950 Sec 15.1 */ int -netconf_minmax_elements(cbuf *cb, - cxobj *x, - int max) +netconf_minmax_elements_xml(cxobj **xret, + cxobj *x, + int max) { - int retval = -1; + int retval = -1; + cxobj *xerr; - if (cprintf(cb, "" - "protocol" - "operation-failed" - "too-%s-elements" - "error" - "%s" - "", - max?"many":"few", - xml_name(x)) < 0) /* XXX should be xml2xpath */ - goto err; + if (*xret == NULL){ + if ((*xret = xml_new("rpc-reply", NULL, NULL)) == NULL) + goto done; + } + else if (xml_name_set(*xret, "rpc-reply") < 0) + goto done; + if ((xerr = xml_new("rpc-error", *xret, NULL)) == NULL) + goto done; + if (xml_parse_va(&xerr, NULL, "protocol" + "operation-failed" + "too-%s-elements" + "error" + "%s", + max?"many":"few", + xml_name(x)) < 0) /* XXX should be xml2xpath */ + goto done; retval = 0; done: return retval; - err: - clicon_err(OE_XML, errno, "cprintf"); - goto done; } - /*! Help function: merge - check yang - if error make netconf errmsg * @param[in] x XML tree * @param[in] yspec Yang spec @@ -1156,3 +1249,41 @@ netconf_db_find(cxobj *xn, return db; } +/*! Generate netconf error msg to cbuf to use in string printout or logs + * @param[in] xerr Netconf error message on the level: + * @param[out] cberr Translation from netconf err to cbuf. Free with cbuf_free. + * @retval 0 OK, with cberr set + * @retval -1 Error + * @code + * cbuf *cb = NULL; + * if (netconf_err2cb(xerr, &cb) < 0) + * err; + * printf("%s", cbuf_get(cb)); + * @endcode + * @see clicon_rpc_generate_error + */ +int +netconf_err2cb(cxobj *xerr, + cbuf **cberr) +{ + int retval = -1; + cbuf *cb = NULL; + cxobj *x; + + if ((cb = cbuf_new()) ==NULL){ + clicon_err(OE_XML, errno, "cbuf_new"); + goto done; + } + if ((x=xpath_first(xerr, "error-type"))!=NULL) + cprintf(cb, "%s ", xml_body(x)); + if ((x=xpath_first(xerr, "error-tag"))!=NULL) + cprintf(cb, "%s ", xml_body(x)); + if ((x=xpath_first(xerr, "error-message"))!=NULL) + cprintf(cb, "%s ", xml_body(x)); + if ((x=xpath_first(xerr, "error-info"))!=NULL) + clicon_xml2cbuf(cb, xml_child_i(x,0), 0, 0); + *cberr = cb; + retval = 0; + done: + return retval; +} diff --git a/lib/src/clixon_options.c b/lib/src/clixon_options.c index a3fbbdf1..008dc61d 100644 --- a/lib/src/clixon_options.c +++ b/lib/src/clixon_options.c @@ -72,6 +72,7 @@ #include "clixon_data.h" #include "clixon_xpath_ctx.h" #include "clixon_xpath.h" +#include "clixon_netconf_lib.h" #include "clixon_xml_map.h" /* Mapping between Clicon startup modes string <--> constants, @@ -89,6 +90,7 @@ static const map_str2int startup_mode_map[] = { * @param[in] dbglevel Debug level * @retval 0 OK * @retval -1 Error + * @note CLICON_FEATURE and CLICON_YANG_DIR are treated specially since they are lists */ int clicon_option_dump(clicon_handle h, @@ -101,6 +103,7 @@ clicon_option_dump(clicon_handle h, void *val; size_t klen; size_t vlen; + cxobj *x = NULL; if (clicon_hash_keys(hash, &keys, &klen) < 0) goto done; @@ -115,7 +118,22 @@ clicon_option_dump(clicon_handle h, else clicon_debug(dbglevel, "%s = NULL", keys[i]); } - retval = 0; + /* Next print CLICON_FEATURE and CLICON_YANG_DIR from config tree + * Since they are lists they are placed in the config tree. + */ + x = NULL; + while ((x = xml_child_each(clicon_conf_xml(h), x, CX_ELMNT)) != NULL) { + if (strcmp(xml_name(x), "CLICON_YANG_DIR") != 0) + continue; + clicon_debug(dbglevel, "%s =\t \"%s\"", xml_name(x), xml_body(x)); + } + x = NULL; + while ((x = xml_child_each(clicon_conf_xml(h), x, CX_ELMNT)) != NULL) { + if (strcmp(xml_name(x), "CLICON_FEATURE") != 0) + continue; + clicon_debug(dbglevel, "%s =\t \"%s\"", xml_name(x), xml_body(x)); + } + retval = 0; done: if (keys) free(keys); @@ -145,6 +163,7 @@ parse_configfile(clicon_handle h, char *body; clicon_hash_t *copt = clicon_options(h); cbuf *cbret = NULL; + cxobj *xret = NULL; int ret; if (filename == NULL || !strlen(filename)){ @@ -194,13 +213,11 @@ parse_configfile(clicon_handle h, } if (xml_apply0(xc, CX_ELMNT, xml_default, h) < 0) goto done; - if ((cbret = cbuf_new()) == NULL){ - clicon_err(OE_XML, errno, "cbuf_new"); - goto done; - } - if ((ret = xml_yang_validate_add(h, xc, cbret)) < 0) + if ((ret = xml_yang_validate_add(h, xc, &xret)) < 0) goto done; if (ret == 0){ + if (netconf_err2cb(xret, &cbret) < 0) + goto done; clicon_err(OE_CFG, 0, "Config file validation: %s", cbuf_get(cbret)); goto done; } @@ -234,6 +251,8 @@ parse_configfile(clicon_handle h, done: if (cbret) cbuf_free(cbret); + if (xret) + xml_free(xret); if (xt) xml_free(xt); if (f) @@ -243,6 +262,7 @@ parse_configfile(clicon_handle h, /*! Add configuration option overriding file setting * Add to clicon_options hash, and to clicon_conf_xml tree + * Assumes clicon_conf_xml_set has been called * @param[in] h Clicon handle * @param[in] name Name of configuration option (see clixon-config.yang) * @param[in] value String value diff --git a/lib/src/clixon_proto_client.c b/lib/src/clixon_proto_client.c index 9502dd64..03a1bf7d 100644 --- a/lib/src/clixon_proto_client.c +++ b/lib/src/clixon_proto_client.c @@ -71,6 +71,7 @@ #include "clixon_proto.h" #include "clixon_err.h" #include "clixon_err_string.h" +#include "clixon_netconf_lib.h" #include "clixon_proto_client.h" /*! Send internal netconf rpc from client to backend @@ -224,29 +225,22 @@ clicon_rpc_netconf_xml(clicon_handle h, } /*! Generate and log clicon error function call from Netconf error message + * @param[in] prefix Print this string (if given) before: ": " * @param[in] xerr Netconf error message on the level: */ int -clicon_rpc_generate_error(char *format, +clicon_rpc_generate_error(char *prefix, cxobj *xerr) { int retval = -1; cbuf *cb = NULL; - cxobj *x; - if ((cb = cbuf_new()) ==NULL){ - clicon_err(OE_XML, errno, "cbuf_new"); + if (netconf_err2cb(xerr, &cb) < 0) goto done; - } - if ((x=xpath_first(xerr, "error-type"))!=NULL) - cprintf(cb, "%s ", xml_body(x)); - if ((x=xpath_first(xerr, "error-tag"))!=NULL) - cprintf(cb, "%s ", xml_body(x)); - if ((x=xpath_first(xerr, "error-message"))!=NULL) - cprintf(cb, "%s ", xml_body(x)); - if ((x=xpath_first(xerr, "error-info"))!=NULL) - clicon_xml2cbuf(cb, xml_child_i(x,0), 0, 0); - clicon_log(LOG_ERR, "%s: %s", format, cbuf_get(cb)); + if (prefix) + clicon_log(LOG_ERR, "%s: %s", prefix, cbuf_get(cb)); + else + clicon_log(LOG_ERR, "%s", cbuf_get(cb)); retval = 0; done: if (cb) diff --git a/lib/src/clixon_xml_changelog.c b/lib/src/clixon_xml_changelog.c index 024c02c6..850a383c 100644 --- a/lib/src/clixon_xml_changelog.c +++ b/lib/src/clixon_xml_changelog.c @@ -67,6 +67,7 @@ #include "clixon_options.h" #include "clixon_data.h" #include "clixon_yang_module.h" +#include "clixon_netconf_lib.h" #include "clixon_xml_map.h" #include "clixon_xml_changelog.h" #include "clixon_xpath_ctx.h" @@ -424,8 +425,9 @@ clixon_xml_changelog_init(clicon_handle h) int fd = -1; cxobj *xt = NULL; yang_stmt *yspec; - cbuf *cbret = NULL; int ret; + cxobj *xret = NULL; + cbuf *cbret = NULL; yspec = clicon_dbspec_yang(h); if ((filename = clicon_option_str(h, "CLICON_XML_CHANGELOG_FILE")) != NULL){ @@ -437,15 +439,13 @@ clixon_xml_changelog_init(clicon_handle h) goto done; if (xml_rootchild(xt, 0, &xt) < 0) goto done; - if ((cbret = cbuf_new()) == NULL){ - clicon_err(OE_UNIX, errno, "cbuf_new"); + if ((ret = xml_yang_validate_all(h, xt, &xret)) < 0) goto done; - } - if ((ret = xml_yang_validate_all(h, xt, cbret)) < 0) - goto done; - if (ret==1 && (ret = xml_yang_validate_add(h, xt, cbret)) < 0) + if (ret==1 && (ret = xml_yang_validate_add(h, xt, &xret)) < 0) goto done; if (ret == 0){ /* validation failed */ + if (netconf_err2cb(xret, &cbret) < 0) + goto done; clicon_err(OE_YANG, 0, "validation failed: %s", cbuf_get(cbret)); goto done; } @@ -455,12 +455,14 @@ clixon_xml_changelog_init(clicon_handle h) } retval = 0; done: + if (cbret) + cbuf_free(cbret); + if (xret) + xml_free(xret); if (fd != -1) close(fd); if (xt) xml_free(xt); - if (cbret) - cbuf_free(cbret); return retval; } diff --git a/lib/src/clixon_xml_map.c b/lib/src/clixon_xml_map.c index 2e82e923..973c8698 100644 --- a/lib/src/clixon_xml_map.c +++ b/lib/src/clixon_xml_map.c @@ -248,7 +248,7 @@ xml2cli(FILE *f, /*! Validate xml node of type leafref, ensure the value is one of that path's reference * @param[in] xt XML leaf node of type leafref * @param[in] ytype Yang type statement belonging to the XML node - * @param[out] cbret Error buffer + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed * @retval -1 Error @@ -256,7 +256,7 @@ xml2cli(FILE *f, static int validate_leafref(cxobj *xt, yang_stmt *ytype, - cbuf *cbret) + cxobj **xret) { int retval = -1; yang_stmt *ypath; @@ -270,7 +270,7 @@ validate_leafref(cxobj *xt, if ((leafrefbody = xml_body(xt)) == NULL) goto ok; if ((ypath = yang_find(ytype, Y_PATH, NULL)) == NULL){ - if (netconf_missing_element(cbret, "application", yang_argument_get(ytype), "Leafref requires path statement") < 0) + if (netconf_missing_element_xml(xret, "application", yang_argument_get(ytype), "Leafref requires path statement") < 0) goto done; goto fail; } @@ -284,12 +284,12 @@ validate_leafref(cxobj *xt, break; } if (i==xlen){ - if (netconf_bad_element(cbret, "application", leafrefbody, "Leafref validation failed: No such leaf") < 0) + if (netconf_bad_element_xml(xret, "application", leafrefbody, "Leafref validation failed: No such leaf") < 0) goto done; goto fail; } ok: - retval = 0; + retval = 1; done: if (xvec) free(xvec); @@ -312,7 +312,7 @@ validate_leafref(cxobj *xt, * @param[in] xt XML leaf node of type identityref * @param[in] ys Yang spec of leaf * @param[in] ytype Yang type field of type identityref - * @param[out] cbret Error buffer + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed * @retval -1 Error @@ -324,7 +324,8 @@ static int validate_identityref(cxobj *xt, yang_stmt *ys, yang_stmt *ytype, - cbuf *cbret) + cxobj **xret) + { int retval = -1; char *node; @@ -350,13 +351,13 @@ validate_identityref(cxobj *xt, } /* This is the type's base reference */ if ((ybaseref = yang_find(ytype, Y_BASE, NULL)) == NULL){ - if (netconf_missing_element(cbret, "application", yang_argument_get(ytype), "Identityref validation failed, no base") < 0) + if (netconf_missing_element_xml(xret, "application", yang_argument_get(ytype), "Identityref validation failed, no base") < 0) goto done; goto fail; } /* This is the actual base identity */ if ((ybaseid = yang_find_identity(ybaseref, yang_argument_get(ybaseref))) == NULL){ - if (netconf_missing_element(cbret, "application", yang_argument_get(ybaseref), "Identityref validation failed, no base identity") < 0) + if (netconf_missing_element_xml(xret, "application", yang_argument_get(ybaseref), "Identityref validation failed, no base identity") < 0) goto done; goto fail; } @@ -367,7 +368,7 @@ validate_identityref(cxobj *xt, cbuf_reset(cb); cprintf(cb, "Identityref validation failed, %s not derived from %s", node, yang_argument_get(ybaseid)); - if (netconf_operation_failed(cbret, "application", cbuf_get(cb)) < 0) + if (netconf_operation_failed_xml(xret, "application", cbuf_get(cb)) < 0) goto done; goto fail; } @@ -413,10 +414,12 @@ xml_yang_root(cxobj *x, } /*! Validate an RPC node - * @param[in] xt XML node to be validated - * @retval 1 Validation OK - * @retval 0 Validation failed - * @retval -1 Error + * @param[in] h Clicon handle + * @param[in] xrpc XML node to be validated + * @param[out] xret Error XML tree. Free with xml_free after use + * @retval 1 Validation OK + * @retval 0 Validation failed + * @retval -1 Error * rfc7950 * 7.14.2 * If a leaf in the input tree has a "mandatory" statement with the @@ -454,8 +457,8 @@ xml_yang_root(cxobj *x, */ int xml_yang_validate_rpc(clicon_handle h, - cxobj *xrpc, - cbuf *cbret) + cxobj *xrpc, + cxobj **xret) { int retval = -1; yang_stmt *yn=NULL; /* rpc name */ @@ -469,15 +472,15 @@ xml_yang_validate_rpc(clicon_handle h, /* xn is name of rpc, ie */ while ((xn = xml_child_each(xrpc, xn, CX_ELMNT)) != NULL) { if ((yn = xml_spec(xn)) == NULL){ - if (netconf_unknown_element(cbret, "application", xml_name(xn), NULL) < 0) + if (netconf_unknown_element_xml(xret, "application", xml_name(xn), NULL) < 0) goto done; goto fail; } - if ((retval = xml_yang_validate_all(h, xn, cbret)) < 1) + if ((retval = xml_yang_validate_all(h, xn, xret)) < 1) goto done; /* error or validation fail */ - if ((retval = xml_yang_validate_add(h, xn, cbret)) < 1) + if ((retval = xml_yang_validate_add(h, xn, xret)) < 1) goto done; /* error or validation fail */ - if (xml_apply0(xn, CX_ELMNT, xml_default, NULL) < 0) + if (xml_apply0(xn, CX_ELMNT, xml_default, h) < 0) goto done; } // ok: /* pass validation */ @@ -492,7 +495,7 @@ xml_yang_validate_rpc(clicon_handle h, /*! Check if an xml node is a part of a choice and have >1 siblings * @param[in] xt XML node to be validated * @param[in] yt xt:s yang statement - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error @@ -501,7 +504,7 @@ xml_yang_validate_rpc(clicon_handle h, static int check_choice(cxobj *xt, yang_stmt *yt, - cbuf *cbret) + cxobj **xret) { int retval = -1; yang_stmt *y; @@ -552,7 +555,7 @@ check_choice(cxobj *xt, continue; /* not choice */ break; } - if (netconf_bad_element(cbret, "application", xml_name(x), "Element in choice statement already exists") < 0) + if (netconf_bad_element_xml(xret, "application", xml_name(x), "Element in choice statement already exists") < 0) goto done; goto fail; } /* while */ @@ -569,7 +572,7 @@ check_choice(cxobj *xt, /*! Check if an xml node lacks mandatory children * @param[in] xt XML node to be validated * @param[in] yt xt:s yang statement - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error @@ -577,7 +580,8 @@ check_choice(cxobj *xt, static int check_mandatory(cxobj *xt, yang_stmt *yt, - cbuf *cbret) + cxobj **xret) + { int retval = -1; int i; @@ -600,7 +604,7 @@ check_mandatory(cxobj *xt, while ((cvi = cvec_each(cvk, cvi)) != NULL) { keyname = cv_string_get(cvi); if (xml_find_type(xt, NULL, keyname, CX_ELMNT) == NULL){ - if (netconf_missing_element(cbret, "application", keyname, "Mandatory key") < 0) + if (netconf_missing_element_xml(xret, "application", keyname, "Mandatory key") < 0) goto done; goto fail; } @@ -623,7 +627,7 @@ check_mandatory(cxobj *xt, break; /* got it */ } if (x == NULL){ - if (netconf_missing_element(cbret, "application", yc->ys_argument, "Mandatory variable") < 0) + if (netconf_missing_element_xml(xret, "application", yc->ys_argument, "Mandatory variable") < 0) goto done; goto fail; } @@ -640,17 +644,7 @@ check_mandatory(cxobj *xt, if (x == NULL){ /* @see RFC7950: 15.6 Error Message for Data That Violates * a Mandatory "choice" Statement */ - if (cprintf(cbret, "" - "application" - "data-missing" - "missing-choice" -#ifdef NYI - // "" -#endif - "%s" - "error" - "", - yc->ys_argument) <0) + if (netconf_data_missing_xml(xret, yc->ys_argument, NULL) < 0) goto done; goto fail; } @@ -667,10 +661,14 @@ check_mandatory(cxobj *xt, goto done; } +/*! + * @param[out] xret Error XML tree. Free with xml_free after use + */ static int check_list_key(cxobj *xt, yang_stmt *yt, - cbuf *cbret) + cxobj **xret) + { int retval = -1; int i; @@ -690,7 +688,7 @@ check_list_key(cxobj *xt, while ((cvi = cvec_each(cvk, cvi)) != NULL) { keyname = cv_string_get(cvi); if (xml_find_type(xt, NULL, keyname, CX_ELMNT) == NULL){ - if (netconf_missing_element(cbret, "application", keyname, "Mandatory key") < 0) + if (netconf_missing_element_xml(xret, "application", keyname, "Mandatory key") < 0) goto done; goto fail; } @@ -739,7 +737,7 @@ check_insert_duplicate(char **vec, * @param[in] xt The parent of x * @param[in] y Its yang spec (Y_LIST) * @param[in] yu A yang unique spec (Y_UNIQUE) - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error @@ -750,7 +748,8 @@ check_unique_list(cxobj *x, cxobj *xt, yang_stmt *y, yang_stmt *yu, - cbuf *cbret) + cxobj **xret) + { int retval = -1; cvec *cvk; /* unique vector */ @@ -784,7 +783,7 @@ check_unique_list(cxobj *x, if (cvi==NULL){ /* Last element (i) is newly inserted, see if it is already there */ if (check_insert_duplicate(vec, i, vlen) < 0){ - if (netconf_data_not_unique(cbret, x, cvk) < 0) + if (netconf_data_not_unique_xml(xret, x, cvk) < 0) goto done; goto fail; } @@ -807,7 +806,7 @@ check_unique_list(cxobj *x, * @param[in] x One x (the last) of a specific lis * @param[in] y Yang spec of x * @param[in] nr Number of elements (like x) in thlist - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error @@ -817,7 +816,7 @@ static int check_min_max(cxobj *x, yang_stmt *y, int nr, - cbuf *cbret) + cxobj **xret) { int retval = -1; yang_stmt *ymin; /* yang min */ @@ -827,7 +826,7 @@ check_min_max(cxobj *x, if ((ymin = yang_find(y, Y_MIN_ELEMENTS, NULL)) != NULL){ cv = yang_cv_get(ymin); if (nr < cv_uint32_get(cv)){ - if (netconf_minmax_elements(cbret, x, 0) < 0) + if (netconf_minmax_elements_xml(xret, x, 0) < 0) goto done; goto fail; } @@ -836,7 +835,7 @@ check_min_max(cxobj *x, cv = yang_cv_get(ymax); if (cv_uint32_get(cv) > 0 && /* 0 means unbounded */ nr > cv_uint32_get(cv)){ - if (netconf_minmax_elements(cbret, x, 1) < 0) + if (netconf_minmax_elements_xml(xret, x, 1) < 0) goto done; goto fail; } @@ -851,9 +850,9 @@ check_min_max(cxobj *x, /*! Detect unique constraint for duplicates from parent node and minmax * @param[in] xt XML parent (may have lists w unique constraints as child) - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK - * @retval 0 Validation failed (cbret set) + * @retval 0 Validation failed (xret set) * @retval -1 Error * Assume xt:s children are sorted and yang populated. * The function does two different things of the children of an XML node: @@ -889,8 +888,8 @@ check_min_max(cxobj *x, * are not allowed. */ static int -check_list_unique_minmax(cxobj *xt, - cbuf *cbret) +check_list_unique_minmax(cxobj *xt, + cxobj **xret) { int retval = -1; cxobj *x = NULL; @@ -932,7 +931,7 @@ check_list_unique_minmax(cxobj *xt, } else { /* Check if the list length violates min/max */ - if ((ret = check_min_max(xp, yp, nr, cbret)) < 0) + if ((ret = check_min_max(xp, yp, nr, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -953,7 +952,7 @@ check_list_unique_minmax(cxobj *xt, do { if (yang_keyword_get(ye) == Y_LIST || yang_keyword_get(ye) == Y_LEAF_LIST){ /* Check if the list length violates min/max */ - if ((ret = check_min_max(xt, ye, 0, cbret)) < 0) + if ((ret = check_min_max(xt, ye, 0, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -973,7 +972,7 @@ check_list_unique_minmax(cxobj *xt, * its first element x, its yang spec y, its parent xt, and * a unique yang spec yu, */ - if ((ret = check_unique_list(x, xt, y, yu, cbret)) < 0) + if ((ret = check_unique_list(x, xt, y, yu, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -984,7 +983,7 @@ check_list_unique_minmax(cxobj *xt, */ if (yp){ /* Check if the list length violates min/max */ - if ((ret = check_min_max(xp, yp, nr, cbret)) < 0) + if ((ret = check_min_max(xp, yp, nr, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -996,7 +995,7 @@ check_list_unique_minmax(cxobj *xt, do { if (yang_keyword_get(ye) == Y_LIST || yang_keyword_get(ye) == Y_LEAF_LIST){ /* Check if the list length violates min/max */ - if ((ret = check_min_max(xt, ye, 0, cbret)) < 0) + if ((ret = check_min_max(xt, ye, 0, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1014,14 +1013,14 @@ check_list_unique_minmax(cxobj *xt, * 1. Check if mandatory leafs present as subs. * 2. Check leaf values, eg int ranges and string regexps. * @param[in] xt XML node to be validated - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree. Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error * @code * cxobj *x; - * cbuf *cbret = cbuf_new(); - * if ((ret = xml_yang_validate_add(h, x, cbret)) < 0) + * cbuf *xret = NULL; + * if ((ret = xml_yang_validate_add(h, x, &xret)) < 0) * err; * if (ret == 0) * fail; @@ -1032,8 +1031,8 @@ check_list_unique_minmax(cxobj *xt, */ int xml_yang_validate_add(clicon_handle h, - cxobj *xt, - cbuf *cbret) + cxobj *xt, + cxobj **xret) { int retval = -1; cg_var *cv = NULL; @@ -1047,11 +1046,11 @@ xml_yang_validate_add(clicon_handle h, /* if not given by argument (overide) use default link and !Node has a config sub-statement and it is false */ if ((yt = xml_spec(xt)) != NULL && yang_config(yt) != 0){ - if ((ret = check_choice(xt, yt, cbret)) < 0) + if ((ret = check_choice(xt, yt, xret)) < 0) goto done; if (ret == 0) goto fail; - if ((ret = check_mandatory(xt, yt, cbret)) < 0) + if ((ret = check_mandatory(xt, yt, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1073,21 +1072,21 @@ xml_yang_validate_add(clicon_handle h, * are considered as "" */ cvtype = cv_type_get(cv); if (cv_isint(cvtype) || cvtype == CGV_BOOL || cvtype == CGV_DEC64){ - if (netconf_bad_element(cbret, "application", yt->ys_argument, "Invalid NULL value") < 0) + if (netconf_bad_element_xml(xret, "application", yt->ys_argument, "Invalid NULL value") < 0) goto done; goto fail; } } else{ if (cv_parse1(body, cv, &reason) != 1){ - if (netconf_bad_element(cbret, "application", yt->ys_argument, reason) < 0) + if (netconf_bad_element_xml(xret, "application", yt->ys_argument, reason) < 0) goto done; goto fail; } } if ((ys_cv_validate(h, cv, yt, &reason)) != 1){ - if (netconf_bad_element(cbret, "application", yt->ys_argument, reason) < 0) + if (netconf_bad_element_xml(xret, "application", yt->ys_argument, reason) < 0) goto done; goto fail; } @@ -1098,7 +1097,7 @@ xml_yang_validate_add(clicon_handle h, } x = NULL; while ((x = xml_child_each(xt, x, CX_ELMNT)) != NULL) { - if ((ret = xml_yang_validate_add(h, x, cbret)) < 0) + if ((ret = xml_yang_validate_add(h, x, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1116,11 +1115,12 @@ xml_yang_validate_add(clicon_handle h, } /*! Some checks done only at edit_config, eg keys in lists + * @param[out] xret Error XML tree. Free with xml_free after use */ int xml_yang_validate_list_key_only(clicon_handle h, - cxobj *xt, - cbuf *cbret) + cxobj *xt, + cxobj **xret) { int retval = -1; yang_stmt *yt; /* yang spec of xt going in */ @@ -1130,14 +1130,14 @@ xml_yang_validate_list_key_only(clicon_handle h, /* if not given by argument (overide) use default link and !Node has a config sub-statement and it is false */ if ((yt = xml_spec(xt)) != NULL && yang_config(yt) != 0){ - if ((ret = check_list_key(xt, yt, cbret)) < 0) + if ((ret = check_list_key(xt, yt, xret)) < 0) goto done; if (ret == 0) goto fail; } x = NULL; while ((x = xml_child_each(xt, x, CX_ELMNT)) != NULL) { - if ((ret = xml_yang_validate_list_key_only(h, x, cbret)) < 0) + if ((ret = xml_yang_validate_list_key_only(h, x, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1154,17 +1154,18 @@ xml_yang_validate_list_key_only(clicon_handle h, /*! Validate a single XML node with yang specification for all (not only added) entries * 1. Check leafrefs. Eg you delete a leaf and a leafref references it. * @param[in] xt XML node to be validated - * @param[out] cbret Error buffer (set w netconf error if retval == 0) + * @param[out] xret Error XML tree (if retval=0). Free with xml_free after use * @retval 1 Validation OK * @retval 0 Validation failed (cbret set) * @retval -1 Error * @code * cxobj *x; - * cbuf *cbret = cbuf_new(); - * if ((ret = xml_yang_validate_all(x, cbret)) < 0) + * cbuf *xret = NULL; + * if ((ret = xml_yang_validate_all(h, x, &xret)) < 0) * err; * if (ret == 0) * fail; + * xml_free(xret); * @endcode * @see xml_yang_validate_add * @see xml_yang_validate_rpc @@ -1172,8 +1173,8 @@ xml_yang_validate_list_key_only(clicon_handle h, */ int xml_yang_validate_all(clicon_handle h, - cxobj *xt, - cbuf *cbret) + cxobj *xt, + cxobj **xret) { int retval = -1; yang_stmt *ys; /* yang node */ @@ -1188,7 +1189,7 @@ xml_yang_validate_all(clicon_handle h, and !Node has a config sub-statement and it is false */ ys=xml_spec(xt); if (ys==NULL){ - if (netconf_unknown_element(cbret, "application", xml_name(xt), NULL) < 0) + if (netconf_unknown_element_xml(xret, "application", xml_name(xt), NULL) < 0) goto done; goto fail; } @@ -1207,12 +1208,16 @@ xml_yang_validate_all(clicon_handle h, */ if ((yc = yang_find(ys, Y_TYPE, NULL)) != NULL){ if (strcmp(yc->ys_argument, "leafref") == 0){ - if (validate_leafref(xt, yc, cbret) < 0) + if ((ret = validate_leafref(xt, yc, xret)) < 0) goto done; + if (ret == 0) + goto fail; } else if (strcmp(yc->ys_argument, "identityref") == 0){ - if (validate_identityref(xt, ys, yc, cbret) < 0) + if ((ret = validate_identityref(xt, ys, yc, xret)) < 0) goto done; + if (ret == 0) + goto fail; } } break; @@ -1230,7 +1235,7 @@ xml_yang_validate_all(clicon_handle h, goto done; if (!nr){ ye = yang_find(yc, Y_ERROR_MESSAGE, NULL); - if (netconf_operation_failed(cbret, "application", + if (netconf_operation_failed_xml(xret, "application", ye?ye->ys_argument:"must xpath validation failed") < 0) goto done; goto fail; @@ -1242,7 +1247,7 @@ xml_yang_validate_all(clicon_handle h, if ((nr = xpath_vec_bool(xt, "%s", xpath)) < 0) goto done; if (!nr){ - if (netconf_operation_failed(cbret, "application", + if (netconf_operation_failed_xml(xret, "application", "when xpath validation failed") < 0) goto done; goto fail; @@ -1251,7 +1256,7 @@ xml_yang_validate_all(clicon_handle h, } x = NULL; while ((x = xml_child_each(xt, x, CX_ELMNT)) != NULL) { - if ((ret = xml_yang_validate_all(h, x, cbret)) < 0) + if ((ret = xml_yang_validate_all(h, x, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1259,7 +1264,7 @@ xml_yang_validate_all(clicon_handle h, /* Check unique and min-max after choice test for example*/ if (yang_config(ys) != 0){ /* Checks if next level contains any unique list constraints */ - if ((ret = check_list_unique_minmax(xt, cbret)) < 0) + if ((ret = check_list_unique_minmax(xt, xret)) < 0) goto done; if (ret == 0) goto fail; @@ -1274,24 +1279,25 @@ xml_yang_validate_all(clicon_handle h, } /*! Translate a single xml node to a cligen variable vector. Note not recursive + * @param[out] xret Error XML tree (if ret == 0). Free with xml_free after use * @retval 1 Validation OK - * @retval 0 Validation failed (cbret set) + * @retval 0 Validation failed (xret set) * @retval -1 Error */ int xml_yang_validate_all_top(clicon_handle h, - cxobj *xt, - cbuf *cbret) + cxobj *xt, + cxobj **xret) { int ret; cxobj *x; x = NULL; while ((x = xml_child_each(xt, x, CX_ELMNT)) != NULL) { - if ((ret = xml_yang_validate_all(h, x, cbret)) < 1) + if ((ret = xml_yang_validate_all(h, x, xret)) < 1) return ret; } - if ((ret = check_list_unique_minmax(xt, cbret)) < 1) + if ((ret = check_list_unique_minmax(xt, xret)) < 1) return ret; return 1; } @@ -2053,7 +2059,7 @@ xml_tree_prune_flagged(cxobj *xt, */ int xml_default(cxobj *xt, - void *arg) + void *arg) { int retval = -1; yang_stmt *ys; @@ -2079,13 +2085,8 @@ xml_default(cxobj *xt, if (!cv_flag(y->ys_cv, V_UNSET)){ /* Default value exists */ if (!xml_find(xt, y->ys_argument)){ -#ifdef USE_XML_INSERT if ((xc = xml_new(y->ys_argument, NULL, y)) == NULL) goto done; -#else - if ((xc = xml_new(y->ys_argument, xt, y)) == NULL) - goto done; -#endif xml_flag_set(xc, XML_FLAG_DEFAULT); if ((xb = xml_new("body", xc, NULL)) == NULL) goto done; @@ -2098,18 +2099,12 @@ xml_default(cxobj *xt, goto done; free(str); added++; -#ifdef USE_XML_INSERT if (xml_insert(xt, xc) < 0) goto done; -#endif } } } } -#ifndef USE_XML_INSERT - if (added) - xml_sort(xt, NULL); -#endif retval = 0; done: return retval; @@ -2206,7 +2201,9 @@ xml_spec_populate_rpc(clicon_handle h, if (yrpc){ xml_spec_set(x, yrpc); if ((yi = yang_find(yrpc, Y_INPUT, NULL)) != NULL){ - /* kludge rpc -> input XXX THIS HIDES AN ERROR IN xml_spec_populate */ + /* xml_spec_populate need to have parent with yang spec for + * recursive population to work. Therefore, assign input yang + * to rpc level although not 100% intuitive */ xml_spec_set(x, yi); if (xml_apply(x, CX_ELMNT, xml_spec_populate, yspec) < 0) goto done; @@ -2497,7 +2494,7 @@ api_path2xml_vec(char **vec, else{ if ((valvec = clicon_strsep(restval, ",", &nvalvec)) == NULL) goto done; - if (nvalvec > cvec_len(cvk)){ + if (nvalvec != cvec_len(cvk)){ clicon_err(OE_XML, EINVAL, "List key %s length mismatch", name); goto fail; } diff --git a/lib/src/clixon_yang.c b/lib/src/clixon_yang.c index 3c16852c..e535451b 100644 --- a/lib/src/clixon_yang.c +++ b/lib/src/clixon_yang.c @@ -301,7 +301,38 @@ ys_free1(yang_stmt *ys) return 0; } -/*! Free a tree of yang statements recursively */ +/*! Remove child i from parent yp (dont free) + * @param[in] yp Parent node + * @param[in] i Order of child to remove + * @retval NULL No such node, nothing done + * @retval yc returned orphaned yang node + * @see ys_free Deallocate yang node + * @note Do not call this in a loop of yang children (unless you know what you are doing) + */ +static yang_stmt * +ys_prune(yang_stmt *yp, + int i) +{ + size_t size; + yang_stmt *yc = NULL; + + if (i >= yp->ys_len) + goto done; + size = (yp->ys_len - i - 1)*sizeof(struct yang_stmt *); + yc = yp->ys_stmt[i]; + memmove(&yp->ys_stmt[i], + &yp->ys_stmt[i+1], + size); + yc = yp->ys_stmt[yp->ys_len--] = NULL;; + done: + return yc; +} + +/*! Free a yang statement tree recursively + * @param[in] ys Yang node to remove and all its children recursively + * @note does not remove yang node from tree + * @see ys_prune Remove from parent + */ int ys_free(yang_stmt *ys) { @@ -1363,7 +1394,6 @@ ys_populate_leaf(clicon_handle h, /* 3b. If not default value, indicate empty cv. */ cv_flag_set(cv, V_UNSET); /* no value (no default) */ } - /* 4. Check if leaf is part of list, if key exists mark leaf as key/unique */ if (yparent && yparent->ys_keyword == Y_LIST){ if ((ret = yang_key_match(yparent, ys->ys_argument)) < 0) @@ -1822,23 +1852,25 @@ ys_populate_unknown(clicon_handle h, /*! Populate with cligen-variables, default values, etc. Sanity checks on complete tree. * - * We do this in 2nd pass after complete parsing to be sure to have a complete parse-tree - * See ys_parse_sub for first pass and what can be assumed + * @param[in] ys Yang statement + * @param[in] h Clicon handle + * Preferably run this command using yang_apply + * Done in 2nd pass after complete parsing to be sure to have a complete + * parse-tree + * After this pass, cv:s are set for LEAFs and LEAF-LISTs + * @see ys_parse_sub for first pass and what can be assumed + * @see ys_populate2 for after grouping expand and augment + * (there may be more functions (all?) that may be moved to ys_populate2) */ int ys_populate(yang_stmt *ys, void *arg) { - int retval = -1; + int retval = -1; clicon_handle h = (clicon_handle)arg; switch(ys->ys_keyword){ - case Y_LEAF: - case Y_LEAF_LIST: - if (ys_populate_leaf(h, ys) < 0) - goto done; - break; case Y_LIST: if (ys_populate_list(h, ys) < 0) goto done; @@ -1851,11 +1883,6 @@ ys_populate(yang_stmt *ys, if (ys_populate_length(h, ys) < 0) goto done; break; - case Y_MANDATORY: /* call yang_mandatory() to check if set */ - case Y_CONFIG: - if (ys_parse(ys, CGV_BOOL) == NULL) - goto done; - break; case Y_TYPE: if (ys_populate_type(h, ys) < 0) goto done; @@ -1880,9 +1907,42 @@ ys_populate(yang_stmt *ys, return retval; } +/*! Run after grouping expand and augment + * @see ys_populate run before grouping expand and augment + */ +static int +ys_populate2(yang_stmt *ys, + void *arg) +{ + int retval = -1; + clicon_handle h = (clicon_handle)arg; + + switch(ys->ys_keyword){ + case Y_LEAF: + case Y_LEAF_LIST: + if (ys_populate_leaf(h, ys) < 0) + goto done; + break; + case Y_MANDATORY: /* call yang_mandatory() to check if set */ + case Y_CONFIG: + if (ys_parse(ys, CGV_BOOL) == NULL) + goto done; + break; + default: + break; + } + retval = 0; + done: + return retval; +} + /*! Resolve a grouping name from a point in the yang tree - * @retval 0 OK, but ygrouping determines if a grouping was resolved or not - * @retval -1 Error, with clicon_err called + * @param[in] ys Yang statement of "uses" statement doing the lookup + * @param[in] prefix Prefix of grouping to look for + * @param[in] name Name of grouping to look for + * @param[out] ygrouping0 A found grouping yang structure as result + * @retval 0 OK, ygrouping may be NULL + * @retval -1 Error, with clicon_err called */ static int ys_grouping_resolve(yang_stmt *ys, @@ -1999,23 +2059,94 @@ yang_augment_spec(yang_stmt *ysp, return retval; } -/*! Macro expansion of grouping/uses done in step 2 of yang parsing - NOTE - RFC6020 says this: - Identifiers appearing inside the grouping are resolved relative to the scope in which the - grouping is defined, not where it is used. Prefix mappings, type names, grouping - names, and extension usage are evaluated in the hierarchy where the - "grouping" statement appears. - But it will be very difficult to generate keys etc with this semantics. So for now I - macro-expand them -*/ +/*! Given a refine node, perform the refinement action on the target refine node + * The RFC is somewhat complicate in the rules for refine. + * Most nodes will be replaced, but some are added + * @param[in] yr Refine node + * @param[in] yt Refine target node (will be modified) + * @see RFC7950 Sec 7.13.2 + * There may be some missed cornercases + */ static int -yang_expand(yang_stmt *yn) +ys_do_refine(yang_stmt *yr, + yang_stmt *yt) +{ + int retval = -1; + yang_stmt *yrc; /* refine child */ + yang_stmt *yrc1; + yang_stmt *ytc; /* target child */ + enum rfc_6020 keyw; + int i; + + /* Loop through refine node children. First if remove do that first + * In some cases remove a set of nodes. + */ + yrc = NULL; + while ((yrc = yn_each(yr, yrc)) != NULL) { + keyw = yang_keyword_get(yrc); + switch (keyw){ + case Y_DEFAULT: /* remove old, add new */ + case Y_DESCRIPTION: + case Y_REFERENCE: + case Y_CONFIG: + case Y_MANDATORY: + case Y_PRESENCE: + case Y_MIN_ELEMENTS: + case Y_MAX_ELEMENTS: + case Y_EXTENSION: + /* Remove old matching, dont increment due to prune in loop */ + for (i=0; iys_len; ){ + ytc = yt->ys_stmt[i]; + if (keyw != yang_keyword_get(ytc)){ + i++; + continue; + } + ys_prune(yt, i); + ys_free(ytc); + } + /* fall through and add if not found */ + case Y_MUST: /* keep old, add new */ + case Y_IF_FEATURE: + break; + default: + break; + } + } + /* Second, add the node(s) */ + yrc = NULL; + while ((yrc = yn_each(yr, yrc)) != NULL) { + keyw = yang_keyword_get(yrc); + /* Make copy */ + if ((yrc1 = ys_dup(yrc)) == NULL) + goto done; + if (yn_insert(yt, yrc1) < 0) + goto done; + } + retval = 0; + done: + return retval; +} + +/*! Macro expansion of grouping/uses done in step 2 of yang parsing + * RFC7950: + * Identifiers appearing inside the grouping are resolved + * relative to the scope in which the grouping is defined, not where it is + * used. Prefix mappings, type names, grouping names, and extension usage are + * evaluated in the hierarchy where the "grouping" statement appears. + * The identifiers defined in the grouping are not bound to a namespace + * until the contents of the grouping are added to the schema tree via a + * "uses" statement that does not appear inside a "grouping" statement, + * at which point they are bound to the namespace of the current module. + */ +static int +yang_expand_grouping(yang_stmt *yn) { int retval = -1; yang_stmt *ys = NULL; - yang_stmt *ygrouping; - yang_stmt *yg; + yang_stmt *ygrouping; /* grouping original */ + yang_stmt *ygrouping2; /* grouping copy */ + yang_stmt *yg; /* grouping child */ + yang_stmt *yr; /* refinement */ int glen; int i; int j; @@ -2034,27 +2165,32 @@ yang_expand(yang_stmt *yn) prefix = yarg_prefix(ys); /* And this its prefix */ if (ys_grouping_resolve(ys, prefix, name, &ygrouping) < 0) goto done; - + if (prefix){ + free(prefix); + prefix = NULL; + } if (ygrouping == NULL){ clicon_log(LOG_NOTICE, "%s: Yang error : grouping \"%s\" not found in module \"%s\"", __FUNCTION__, ys->ys_argument, ys_module(ys)->ys_argument); goto done; break; } - if (prefix) - free(prefix); /* XXX move up */ /* Check mark flag to see if this grouping (itself) has been expanded If not, this needs to be done before we can insert it into the 'uses' place */ if ((ygrouping->ys_flags & YANG_FLAG_MARK) == 0){ - if (yang_expand(ygrouping) < 0) + if (yang_expand_grouping(ygrouping) < 0) goto done; ygrouping->ys_flags |= YANG_FLAG_MARK; /* Mark as expanded */ } + /* Make a copy of the grouping, then make refinements to this copy + */ + if ((ygrouping2 = ys_dup(ygrouping)) == NULL) + goto done; /* Replace ys with ygrouping,... * First enlarge parent vector */ - glen = ygrouping->ys_len; + glen = ygrouping2->ys_len; /* * yn is parent: the children of ygrouping replaces ys. * Is there a case when glen == 0? YES AND THIS BREAKS @@ -2064,7 +2200,7 @@ yang_expand(yang_stmt *yn) yn->ys_len += glen - 1; if (glen && (yn->ys_stmt = realloc(yn->ys_stmt, (yn->ys_len)*sizeof(yang_stmt *))) == 0){ clicon_err(OE_YANG, errno, "realloc"); - return -1; + goto done; } /* Then move all existing elements up from i+1 (not uses-stmt) */ if (size) @@ -2072,16 +2208,42 @@ yang_expand(yang_stmt *yn) &yn->ys_stmt[i+1], size); } + + /* Iterate through refinements and modify grouping copy + * See RFC 7950 7.13.2 yrt is the refine target node + */ + yr = NULL; + while ((yr = yn_each(ys, yr)) != NULL) { + yang_stmt *yrt; /* refine target node */ + if (yang_keyword_get(yr) != Y_REFINE) + continue; + /* Find a node */ + if (yang_desc_schema_nodeid(ygrouping2, + yang_argument_get(yr), + -1, + &yrt) < 0) + goto done; + /* Not found, try next */ + if (yrt == NULL) + continue; + /* Do the actual refinement */ + if (ys_do_refine(yr, yrt) < 0) + goto done; + /* RFC: The argument is a string that identifies a node in the + * grouping. I interpret that as only one node --> break */ + break; + } /* Then copy and insert each child element */ for (j=0; jys_stmt[j])) == NULL) - goto done; + yg = ygrouping2->ys_stmt[j]; /* Child of refined copy */ yn->ys_stmt[i+j] = yg; yg->ys_parent = yn; } - /* XXX: refine */ /* Remove 'uses' node */ ys_free(ys); + /* Remove the grouping copy */ + ygrouping2->ys_len = 0; + ys_free(ygrouping2); break; /* Note same child is re-iterated since it may be changed */ default: i++; @@ -2091,7 +2253,7 @@ yang_expand(yang_stmt *yn) /* Second pass since length may have changed */ for (i=0; iys_len; i++){ ys = yn->ys_stmt[i]; - if (yang_expand(ys) < 0) + if (yang_expand_grouping(ys) < 0) goto done; } retval = 0; @@ -2441,8 +2603,8 @@ ys_schemanode_check(yang_stmt *ys, if (yang_desc_schema_nodeid(yp, ys->ys_argument, -1, &yres) < 0) goto done; if (yres == NULL){ - clicon_err(OE_YANG, 0, "schemanode sanity check of %d %s", - ys->ys_keyword, + clicon_err(OE_YANG, 0, "schemanode sanity check of %s %s", + yang_key2str(ys->ys_keyword), ys->ys_argument); goto done; } @@ -2609,15 +2771,20 @@ yang_parse_post(clicon_handle h, /* 6: Macro expansion of all grouping/uses pairs. Expansion needs marking */ for (i=modnr; iys_len; i++){ - if (yang_expand(yspec->ys_stmt[i]) < 0) + if (yang_expand_grouping(yspec->ys_stmt[i]) < 0) goto done; yang_apply(yspec->ys_stmt[i], -1, ys_flag_reset, (void*)YANG_FLAG_MARK); } - /* 7: Top-level augmentation of all modules XXX: only new modules? */ + /* 7: Top-level augmentation of all modules. (Augment also in uses) */ if (yang_augment_spec(yspec, modnr) < 0) goto done; + /* 4: Go through parse tree and do 2nd step populate (eg default) */ + for (i=modnr; iys_len; i++) + if (yang_apply(yspec->ys_stmt[i], -1, ys_populate2, (void*)h) < 0) + goto done; + /* 8: sanity check of schemanode references, need more here */ for (i=modnr; iys_len; i++) if (yang_apply(yspec->ys_stmt[i], -1, ys_schemanode_check, NULL) < 0) @@ -2829,19 +2996,13 @@ yang_spec_load_dir(clicon_handle h, * This is a failsafe in case anything else fails */ if (revm && rev0){ - int size; if (revm > rev0) /* Loaded module is older or eq -> remove ym */ ym = ym0; for (j=0; jys_len; j++) if (yspec->ys_stmt[j] == ym) break; - size = (yspec->ys_len - j - 1)*sizeof(struct yang_stmt *); - memmove(&yspec->ys_stmt[j], - &yspec->ys_stmt[j+1], - size); + ys_prune(yspec, j); ys_free(ym); - yspec->ys_len--; - yspec->ys_stmt[yspec->ys_len] = NULL; } } if (yang_parse_post(h, yspec, modnr) < 0) @@ -3101,7 +3262,7 @@ yang_abs_schema_nodeid(yang_stmt *yspec, * @param[out] yres First yang node matching schema nodeid * @retval 0 OK * @retval -1 Error, with clicon_err called - * @see yang_schema_nodeid + * @see yang_abs_schema_nodeid * Used in yang: unique, refine, uses augment */ int @@ -3429,6 +3590,7 @@ yang_arg2cvec(yang_stmt *ys, /*! Check if yang node yn has key-stmt as child which matches name * + * The function looks at the LIST argument string (not actual children) * @param[in] yn Yang node with sub-statements (look for a key child) * @param[in] name Check if this name (eg "b") is a key in the yang key statement * diff --git a/test/all.sh b/test/all.sh index e2ff3809..6711558f 100755 --- a/test/all.sh +++ b/test/all.sh @@ -2,6 +2,9 @@ # Run, eg as: # ./all.sh 2>&1 | tee test.log # break on first test +# Pattern to run tests, default is all, but you may want to narrow it down +: ${pattern:=test_*.sh} + if [ $# -gt 0 ]; then echo "usage: $0 # detailed logs and stopon first error" exit -1 @@ -9,7 +12,7 @@ fi err=0 testnr=0 -for test in test_*.sh; do +for test in $pattern; do if [ $testnr != 0 ]; then echo; fi testfile=$test . ./$test diff --git a/test/mem.sh b/test/mem.sh index a35193d3..d9c0e1dd 100755 --- a/test/mem.sh +++ b/test/mem.sh @@ -2,6 +2,9 @@ # Run valgrind leak test for cli, restconf, netconf or background. # Stop on first error +# Pattern to run tests, default is all, but you may want to narrow it down +: ${pattern:=test_*.sh} + # Run valgrindtest once, args: # what: (cli|netconf|restconf|backend)* # no args means all memonce(){ @@ -45,7 +48,7 @@ memonce(){ esac err=0 - for test in test_*.sh; do + for test in $pattern; do if [ $testnr != 0 ]; then echo; fi testfile=$test . ./$test diff --git a/test/test_augment.sh b/test/test_augment.sh index b7302270..a3a9d7ef 100755 --- a/test/test_augment.sh +++ b/test/test_augment.sh @@ -22,6 +22,7 @@ cat < $cfg a:test $dir /usr/local/share/clixon + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/cli $APPNAME @@ -66,14 +67,14 @@ module ietf-interfaces { } } grouping endpoint { - description "A reusable endpoint group."; - leaf mip { + description "A reusable endpoint group. From rf7950 Sec 7.12.2"; + leaf ip { type string; } - leaf mport { + leaf port { type uint16; } - } + } } EOF @@ -98,12 +99,14 @@ module example-augment { identity you { base my-type; } - grouping mypoint { - description "A reusable endpoint group."; - leaf ip { - type string; + grouping localgroup { + description "Local grouping defining lid and lport"; + leaf lid { + description "this will be kept as-is"; + type string; } - leaf port { + leaf lport { + description "this will be refined"; type uint16; } } @@ -124,12 +127,14 @@ module example-augment { } } uses if:endpoint { + description "Use an external grouping defining ip and port"; refine port { default 80; } } - uses mypoint { - refine mport { + uses localgroup { + description "Use a local grouping defining lip and lport"; + refine lport { default 8080; } } @@ -137,37 +142,35 @@ module example-augment { } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" - sudo clixon_backend -zf $cfg -y $fyang + sudo clixon_backend -zf $cfg if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg new "waiting" sleep $RCWAIT fi # mandatory-leaf See RFC7950 Sec 7.17 -# Error1: the xml should have xmlns for "mymod" -# XMLNS_YANG_ONLY must be undeffed new "netconf set interface with augmented type and mandatory leaf" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' e1 mymod:some-new-iftype true ]]>]]>' "^]]>]]>$" -new "netconf validate ok" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^]]>]]>$' +new "netconf verify get with refined ports" +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^e1mymod:some-new-iftypetrue808080]]>]]>$' new "netconf set identity defined in other" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' e2 fddi @@ -176,10 +179,10 @@ expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" new "netconf validate ok" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^]]>]]>$' new "netconf set identity defined in main" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' e3 fddi @@ -188,9 +191,11 @@ expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" new "netconf validate ok" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^]]>]]>$' + +new "discard" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" if [ $BE -eq 0 ]; then exit # BE diff --git a/test/test_cli.sh b/test/test_cli.sh index 66f98394..5816296f 100755 --- a/test/test_cli.sh +++ b/test/test_cli.sh @@ -46,7 +46,7 @@ if [ $BE -ne 0 ]; then start_backend -s init -f $cfg new "waiting" - sleep $RCWAIT + wait_backend fi new "cli configure top" diff --git a/test/test_identity.sh b/test/test_identity.sh index d2ae40cf..a31b1bbb 100755 --- a/test/test_identity.sh +++ b/test/test_identity.sh @@ -15,6 +15,7 @@ cat < $cfg $dir /usr/local/share/clixon $IETFRFC + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/backend example_backend.so$ @@ -108,80 +109,82 @@ cat < $fyang } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" sudo clixon_backend -zf $cfg if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg new "waiting" sleep $RCWAIT fi new "Set crypto to aes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'aes]]>]]>' '^]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 'aes]]>]]>' '^]]>]]>$' new "netconf validate " -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "Set crypto to mc:aes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'mc:aes]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'mc:aes]]>]]>' "^]]>]]>$" new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "Set crypto to des:des3" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'des:des3]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'des:des3]]>]]>' "^]]>]]>$" new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "Set crypto to mc:foo" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'mc:foo]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'mc:foo]]>]]>' "^]]>]]>$" new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "Set crypto to des:des3 using xmlns" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'des:des3]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'des:des3]]>]]>' "^]]>]]>$" new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" +if false; then # XXX this is not supported -#new "Set crypto to x:des3 using xmlns" -#expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'x:des3]]>]]>' "^]]>]]>$" +new "Set crypto to x:des3 using xmlns" +expecteof "$clixon_netconf -qf $cfg" 0 'x:des3]]>]]>' "^]]>]]>$" new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" +fi # not supported new "Set crypto to foo:bar" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'foo:bar]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'foo:bar]]>]]>' "^]]>]]>$" -new "netconf validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^applicationoperation-failederrorIdentityref validation failed, foo:bar not derived from crypto-alg]]>]]>$" +new "netconf validate (expect fail)" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^applicationoperation-failederrorIdentityref validation failed, foo:bar not derived from crypto-alg]]>]]>$" new "cli set crypto to mc:aes" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o set crypto mc:aes" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o set crypto mc:aes" 0 "^$" new "cli validate" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o validate" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o validate" 0 "^$" new "cli set crypto to aes" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o set crypto aes" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o set crypto aes" 0 "^$" new "cli validate" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o validate" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o validate" 0 "^$" new "cli set crypto to des:des3" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o set crypto des:des3" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o set crypto des:des3" 0 "^$" new "cli validate" -expectfn "$clixon_cli -1 -f $cfg -y $fyang -l o validate" 0 "^$" +expectfn "$clixon_cli -1 -f $cfg -l o validate" 0 "^$" if [ $BE -eq 0 ]; then exit # BE diff --git a/test/test_nacm_ext.sh b/test/test_nacm_ext.sh index cd66b01c..9453978a 100755 --- a/test/test_nacm_ext.sh +++ b/test/test_nacm_ext.sh @@ -149,7 +149,8 @@ new "start restconf daemon (-a is enable http basic auth)" start_restconf -f $cfg -- -a new "waiting" -sleep $RCWAIT +wait_backend +wait_restconf new "auth get" expecteq "$(curl -u andy:bar -sS -X GET http://localhost/restconf/data)" 0 '{"data": {"clixon-example:state": {"op": ["42","41","43"]}}} @@ -183,10 +184,10 @@ new "admin edit nacm" expecteq "$(curl -u andy:bar -sS -X PUT -d '{"nacm-example:x": 1}' http://localhost/restconf/data/nacm-example:x)" 0 "" new "limited edit nacm" -expecteq "$(curl -u wilma:bar -sS -X PUT -d '{"x": 2}' http://localhost/restconf/data/nacm-example:x)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "access-denied","error-severity": "error","error-message": "default deny"}}} ' +expecteq "$(curl -u wilma:bar -sS -X PUT -d '{"nacm-example:x": 2}' http://localhost/restconf/data/nacm-example:x)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "access-denied","error-severity": "error","error-message": "default deny"}}} ' new "guest edit nacm" -expecteq "$(curl -u guest:bar -sS -X PUT -d '{"x": 3}' http://localhost/restconf/data/nacm-example:x)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "access-denied","error-severity": "error","error-message": "access denied"}}} ' +expecteq "$(curl -u guest:bar -sS -X PUT -d '{"nacm-example:x": 3}' http://localhost/restconf/data/nacm-example:x)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "access-denied","error-severity": "error","error-message": "access denied"}}} ' new "cli show conf as admin" expectfn "$clixon_cli -1 -U andy -l o -f $cfg show conf" 0 "^x 1;$" diff --git a/test/test_nacm_module_read.sh b/test/test_nacm_module_read.sh index 8204b6a4..16755175 100755 --- a/test/test_nacm_module_read.sh +++ b/test/test_nacm_module_read.sh @@ -153,7 +153,7 @@ new "commit it" expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "enable nacm" -expecteq "$(curl -u andy:bar -sS -X PUT -d '{"enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" +expecteq "$(curl -u andy:bar -sS -X PUT -d '{"ietf-netconf-acm:enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" #--------------- nacm enabled @@ -256,7 +256,7 @@ expecteof "$clixon_netconf -U guest -qf $cfg" 0 ']]>]]>" "^]]>]]>$" new "enable nacm" - expecteq "$(curl -u andy:bar -sS -X PUT -d '{"enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" + expecteq "$(curl -u andy:bar -sS -X PUT -d '{"ietf-netconf-acm:enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" } #--------------- enable nacm diff --git a/test/test_nacm_protocol.sh b/test/test_nacm_protocol.sh index 3e6089ce..c9092ac9 100755 --- a/test/test_nacm_protocol.sh +++ b/test/test_nacm_protocol.sh @@ -162,7 +162,7 @@ new "commit it" expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "enable nacm" -expecteq "$(curl -u andy:bar -sS -X PUT -d '{"enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" +expecteq "$(curl -u andy:bar -sS -X PUT -d '{"ietf-netconf-acm:enable-nacm": true}' http://localhost/restconf/data/ietf-netconf-acm:nacm/enable-nacm)" 0 "" #--------------- nacm enabled diff --git a/test/test_netconf.sh b/test/test_netconf.sh index 6ec9e127..05cfdfe5 100755 --- a/test/test_netconf.sh +++ b/test/test_netconf.sh @@ -46,7 +46,7 @@ if [ $BE -ne 0 ]; then start_backend -s init -f $cfg -- -s new "waiting" - sleep $RCWAIT + wait_backend fi new "netconf hello" diff --git a/test/test_order.sh b/test/test_order.sh index 375c537c..20393f18 100755 --- a/test/test_order.sh +++ b/test/test_order.sh @@ -33,6 +33,7 @@ cat < $cfg /tmp/conf_yang.xml /usr/local/share/clixon $IETFRFC + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/cli $APPNAME @@ -149,16 +150,16 @@ cat < $dbdir/running_db EOF -new "test params: -s running -f $cfg -y $fyang -- -s" +new "test params: -s running -f $cfg -- -s" if [ $BE -ne 0 ]; then new "kill old backend" - sudo clixon_backend -zf $cfg -y $fyang + sudo clixon_backend -zf $cfg if [ $? -ne 0 ]; then err fi new "start backend" - start_backend -s running -f $cfg -y $fyang -- -s + start_backend -s running -f $cfg -- -s new "waiting" sleep $RCWAIT @@ -169,75 +170,75 @@ new "state data (should be unordered: 42,41,43)" cat < $tmp ]]>]]> EOF -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "$(cat $tmp)" '424143]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 "$(cat $tmp)" '424143]]>]]>' # Check as file new "verify running from start, should be: c,l,y0,y1,y2,y3; y1 and y3 sorted." -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^hejhoppdbcaabcddbarabarcbarbbarabarbbarcbardbar]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^hejhoppdbcaabcddbarabarcbarbbarabarbbarcbardbar]]>]]>$' new "get each ordered-by user leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^abar]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^abar]]>]]>$' new "get each ordered-by user leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^abar]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^abar]]>]]>$' new "get each ordered-by user leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^bbar]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^bbar]]>]]>$' new "get each ordered-by user leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^bbar]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^bbar]]>]]>$' new "delete candidate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'none]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'none]]>]]>' "^]]>]]>$" # LEAF_LISTS new "add two entries (c,b) to leaf-list user order" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'cb]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'cb]]>]]>' "^]]>]]>$" new "add one entry (a) to leaf-list user order" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'a]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'a]]>]]>' "^]]>]]>$" new "netconf commit" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "add one entry (0) to leaf-list user order after commit" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '0]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '0]]>]]>' "^]]>]]>$" new "netconf commit" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "verify leaf-list user order in running (as entered: c,b,a,0)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^cba0]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^cba0]]>]]>$' # LISTS new "add two entries to list user order" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'cbarbfoo]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'cbarbfoo]]>]]>' "^]]>]]>$" new "add one entry to list user order" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'afie]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'afie]]>]]>' "^]]>]]>$" new "verify list user order (as entered)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^cbarbfooafie]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^cbarbfooafie]]>]]>$' new "Overwrite existing ordered-by user y2->c" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' cnewc ]]>]]>' new "Overwrite existing ordered-by user y2->b" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' bnewb ]]>]]>' new "Overwrite existing ordered-by user y2->a" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' anewa ]]>]]>' new "Tests for no duplicates." -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^cnewcbnewbanewa]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^cnewcbnewbanewa]]>]]>$' #-- order by type rather than strings. # there are three leaf-lists:strings, ints, and decimal64, and two lists: @@ -246,44 +247,44 @@ expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' 1021 ]]>]]>' "^]]>]]>$" new "check string order (1,10,2)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^1102]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^1102]]>]]>$' new "put leaf-list int (10,2,1)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' 1021 ]]>]]>' "^]]>]]>$" new "check leaf-list int order (1,2,10)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^1210]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^1210]]>]]>$' new "put list int (10,2,1)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' 1021 ]]>]]>' "^]]>]]>$" new "check list int order (1,2,10)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^1210]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^1210]]>]]>$' new "put leaf-list decimal64 (10,2,1)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' 10.02.01.0 ]]>]]>' "^]]>]]>$" new "check leaf-list decimal64 order (1,2,10)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^1.02.010.0]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^1.02.010.0]]>]]>$' new "put list decimal64 (10,2,1)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ' +expecteof "$clixon_netconf -qf $cfg" 0 ' 10.02.01.0 ]]>]]>' "^]]>]]>$" new "check list decimal64 order (1,2,10)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^1.02.010.0]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^1.02.010.0]]>]]>$' if [ $BE -eq 0 ]; then exit # BE diff --git a/test/test_restconf.sh b/test/test_restconf.sh index 61659a88..fe86cc65 100755 --- a/test/test_restconf.sh +++ b/test/test_restconf.sh @@ -43,6 +43,7 @@ if [ $BE -ne 0 ]; then if [ $? -ne 0 ]; then err fi + sudo pkill clixon_backend # to be sure new "start backend -s init -f $cfg -- -s" start_backend -s init -f $cfg -- -s fi @@ -57,8 +58,6 @@ new "waiting" wait_backend wait_restconf -new "restconf tests" - new "restconf root discovery. RFC 8040 3.1 (xml+xrd)" expecteq "$(curl -s -X GET http://localhost/.well-known/host-meta)" 0 " diff --git a/test/test_restconf2.sh b/test/test_restconf2.sh index 52b6d2a5..84aba720 100755 --- a/test/test_restconf2.sh +++ b/test/test_restconf2.sh @@ -14,9 +14,9 @@ fyang=$dir/restconf.yang cat < $cfg $cfg - /usr/local/var /usr/local/share/clixon $IETFRFC + $fyang false /usr/local/var/$APPNAME/$APPNAME.sock $dir/restconf.pidfile @@ -66,7 +66,7 @@ module example{ } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" @@ -74,28 +74,30 @@ if [ $BE -ne 0 ]; then if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + sudo pkill clixon_backend # to be sure + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg fi new "kill old restconf daemon" sudo pkill -u www-data -f "/www-data/clixon_restconf" new "start restconf daemon" -start_restconf -f $cfg -y $fyang +start_restconf -f $cfg new "waiting" wait_backend wait_restconf -new "restconf tests" - new "restconf POST tree without key" expectfn 'curl -s -X POST -d {"example:cont1":{"interface":{"type":"regular"}}} http://localhost/restconf/data' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "missing-element","error-info": {"bad-element": "name"},"error-severity": "error","error-message": "Mandatory key"}}} ' new "restconf POST initial tree" expectfn 'curl -s -X POST -d {"example:cont1":{"interface":{"name":"local0","type":"regular"}}} http://localhost/restconf/data' 0 "" +new "restconf POST top without namespace" +expectfn 'curl -s -X POST -d {"cont1":{"interface":{"name":"local0","type":"regular"}}} http://localhost/restconf/data' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "unknown-element","error-info": {"bad-element": "cont1"},"error-severity": "error","error-message": "Unassigned yang spec"}}}' + new "restconf GET datastore initial" expectfn "curl -s -X GET http://localhost/restconf/data/example:cont1" 0 '{"example:cont1": {"interface": \[{"name": "local0","type": "regular"}\]}}' @@ -114,17 +116,19 @@ new "restconf GET if-type" expectfn "curl -s -X GET http://localhost/restconf/data/example:cont1/interface=local0/type" 0 '{"example:type": "regular"}' new "restconf POST interface without mandatory type" -expectfn 'curl -s -X POST http://localhost/restconf/data/example:cont1 -d {"interface":{"name":"TEST"}} http://localhost/restconf/data/example:cont1' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "missing-element","error-info": {"bad-element": "type"},"error-severity": "error","error-message": "Mandatory variable"}}} ' +expectfn 'curl -s -X POST http://localhost/restconf/data/example:cont1 -d {"example:interface":{"name":"TEST"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "missing-element","error-info": {"bad-element": "type"},"error-severity": "error","error-message": "Mandatory variable"}}} ' new "restconf POST interface without mandatory key" -expectfn 'curl -s -X POST http://localhost/restconf/data/example:cont1 -d {"interface":{"type":"regular"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "missing-element","error-info": {"bad-element": "name"},"error-severity": "error","error-message": "Mandatory key"}}} ' +expectfn 'curl -s -X POST http://localhost/restconf/data/example:cont1 -d {"example:interface":{"type":"regular"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "missing-element","error-info": {"bad-element": "name"},"error-severity": "error","error-message": "Mandatory key"}}} ' new "restconf POST interface" expectfn 'curl -s -X POST -d {"example:interface":{"name":"TEST","type":"eth0"}} http://localhost/restconf/data/example:cont1' 0 "" -# XXX should it be example:interface? +new "restconf POST interface without namespace" +expectfn 'curl -s -X POST -d {"interface":{"name":"TEST2","type":"eth0"}} http://localhost/restconf/data/example:cont1' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "rpc","error-tag": "malformed-message","error-severity": "error","error-message": "Data is not prefixed with matching namespace"}}}' + new "restconf POST again" -expecteq "$(curl -s -X POST -d '{"interface":{"name":"TEST","type":"eth0"}}' http://localhost/restconf/data/example:cont1)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "data-exists","error-severity": "error","error-message": "Data already exists; cannot create new resource"}}} ' +expecteq "$(curl -s -X POST -d '{"example:interface":{"name":"TEST","type":"eth0"}}' http://localhost/restconf/data/example:cont1)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "data-exists","error-severity": "error","error-message": "Data already exists; cannot create new resource"}}} ' new "restconf POST from top" expecteq "$(curl -s -X POST -d '{"example:cont1":{"interface":{"name":"TEST","type":"eth0"}}}' http://localhost/restconf/data)" 0 '{"ietf-restconf:errors" : {"error": {"error-type": "application","error-tag": "data-exists","error-severity": "error","error-message": "Data already exists; cannot create new resource"}}} ' @@ -163,16 +167,22 @@ new "restconf PUT initial datastore again" expectfn 'curl -s -X PUT -d {"data":{"example:cont1":{"interface":{"name":"local0","type":"regular"}}}} http://localhost/restconf/data' 0 "" new "restconf PUT change interface" -expectfn 'curl -s -X PUT -d {"interface":{"name":"local0","type":"atm0"}} http://localhost/restconf/data/example:cont1/interface=local0' 0 "" +expectfn 'curl -s -X PUT -d {"example:interface":{"name":"local0","type":"atm0"}} http://localhost/restconf/data/example:cont1/interface=local0' 0 "" new "restconf GET datastore atm" expectfn "curl -s -X GET http://localhost/restconf/data/example:cont1" 0 '{"example:cont1": {"interface": \[{"name": "local0","type": "atm0"}\]}}' new "restconf PUT add interface" -expectfn 'curl -s -X PUT -d {"interface":{"name":"TEST","type":"eth0"}} http://localhost/restconf/data/example:cont1/interface=TEST' 0 "" +expectfn 'curl -s -X PUT -d {"example:interface":{"name":"TEST","type":"eth0"}} http://localhost/restconf/data/example:cont1/interface=TEST' 0 "" new "restconf PUT change key error" -expectfn 'curl -is -X PUT -d {"interface":{"name":"ALPHA","type":"eth0"}} http://localhost/restconf/data/example:cont1/interface=TEST' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' +expectfn 'curl -is -X PUT -d {"example:interface":{"name":"ALPHA","type":"eth0"}} http://localhost/restconf/data/example:cont1/interface=TEST' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' + +new "restconf PUT change type to eth0 (non-key sub-element to list)" +expectfn 'curl -s -X PUT -d {"example:type":"eth0"} http://localhost/restconf/data/example:cont1/interface=local0/type' 0 "" + +new "restconf GET datastore eth" +expectfn "curl -s -X GET http://localhost/restconf/data/example:cont1/interface=local0" 0 '{"example:interface": \[{"name": "local0","type": "eth0"}\]}' #--------------- json type tests new "restconf POST type x3" diff --git a/test/test_restconf_listkey.sh b/test/test_restconf_listkey.sh index 473ac4a9..cb0546e4 100755 --- a/test/test_restconf_listkey.sh +++ b/test/test_restconf_listkey.sh @@ -14,7 +14,6 @@ fyang=$dir/list.yang cat < $cfg $cfg - /usr/local/var /usr/local/share/clixon $IETFRFC $fyang @@ -32,13 +31,30 @@ module list{ prefix ex; container c{ list a{ - key b; + key "b c"; leaf b{ type string; } leaf c{ type string; } + leaf nonkey{ + description "non-key element"; + type string; + } + list e{ + description "A list in a list"; + key "f"; + leaf f{ + type string; + } + leaf nonkey{ + type string; + } + } + leaf-list f{ + type string; + } } leaf-list d{ type string; @@ -55,6 +71,8 @@ if [ $BE -ne 0 ]; then if [ $? -ne 0 ]; then err fi + sudo pkill clixon_backend # to be sure + new "start backend -s init -f $cfg" start_backend -s init -f $cfg fi @@ -69,17 +87,36 @@ new "waiting" wait_backend wait_restconf -new "restconf PUT add list entry" -expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x -d {"list:a":{"b":"x","c":"0"}}' 0 '' +new "restconf PUT add whole list entry" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y -d {"list:a":{"b":"x","c":"y","nonkey":"0"}}' 0 '' -new "restconf PUT change regular list entry" -expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x -d {"list:a":{"b":"x","c":"z"}}' 0 '' +new "restconf PUT add whole list entry XML" +expecteq "$(curl -s -X PUT -H 'Content-Type: application/yang-data+xml' -H 'Accept: application/yang-data+xml' -d 'xxxy0' http://localhost/restconf/data/list:c/a=xx,xy)" 0 '' -new "restconf PUT change list key entry (expect fail)" -expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x -d {"list:a":{"b":"y"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' -new "restconf PUT change actual list key entry (expect fail)" -expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x/b -d {"b":"y"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' +new "restconf PUT change whole list entry (same keys)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y -d {"list:a":{"b":"x","c":"y","nonkey":"z"}}' 0 '' + +new "restconf PUT change whole list entry (no namespace)(expect fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y -d {"a":{"b":"x","c":"y","nonkey":"z"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "rpc","error-tag": "malformed-message","error-severity": "error","error-message": "Data is not prefixed with matching namespace"}}}' + +new "restconf PUT change list entry (wrong keys)(expect fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y -d {"list:a":{"b":"y","c":"x"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' + +new "restconf PUT change list entry (wrong keys)(expect fail) XML" +expecteq "$(curl -s -X PUT -H 'Content-Type: application/yang-data+xml' -H 'Accept: application/yang-data+xml' -d 'xyxz0' http://localhost/restconf/data/list:c/a=xx,xy)" 0 'protocoloperation-failederrorapi-path keys do not match data keys ' + +new "restconf PUT change list entry (just one key)(expect fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x -d {"list:a":{"b":"x"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "rpc","error-tag": "malformed-message","error-severity": "error","error-message": "List key a length mismatch"}}} ' + +new "restconf PUT sub non-key" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/nonkey -d {"list:nonkey":"u"}' 0 '' + +new "restconf PUT sub key same value" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/b -d {"list:b":"x"}' 0 '' + +new "restconf PUT just key other value (should fail)ZX" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x/b -d {"b":"y"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "rpc","error-tag": "malformed-message","error-severity": "error","error-message": "List key a length mismatch"}}}' new "restconf PUT add leaf-list entry" expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/d=x -d {"list:d":"x"}' 0 '' @@ -87,6 +124,31 @@ expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/d=x -d {"list:d": new "restconf PUT change leaf-list entry (expect fail)" expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/d=x -d {"list:d":"y"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' +new "restconf PUT list-list" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/e=z -d {"list:e":{"f":"z","nonkey":"0"}}' 0 '' + +new "restconf PUT change list-lst entry (wrong keys)(expect fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/e=z -d {"list:e":{"f":"wrong","nonley":"0"}}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' + +new "restconf PUT list-list sub non-key" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/e=z/nonkey -d {"list:nonkey":"u"}' 0 '' + +new "restconf PUT list-list single first key" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x/e=z/f -d {"f":"z"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "rpc","error-tag": "malformed-message","error-severity": "error","error-message": "List key a length mismatch"}}}' + +new "restconf PUT list-list just key ok" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/e=z/f -d {"list:f":"z"}' 0 '' + +new "restconf PUT list-list just key just key wrong value (should fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/e=z/f -d {"list:f":"wrong"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' + +new "restconf PUT add list+leaf-list entry" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/f=u -d {"list:f":"u"}' 0 '' + +new "restconf PUT change list+leaf-list entry (expect fail)" +expectfn 'curl -s -X PUT http://localhost/restconf/data/list:c/a=x,y/f=u -d {"list:f":"w"}' 0 '{"ietf-restconf:errors" : {"error": {"error-type": "protocol","error-tag": "operation-failed","error-severity": "error","error-message": "api-path keys do not match data keys"}}}' + + new "Kill restconf daemon" stop_restconf diff --git a/test/test_rpc.sh b/test/test_rpc.sh index d78ec54e..64ed55c3 100755 --- a/test/test_rpc.sh +++ b/test/test_rpc.sh @@ -77,11 +77,11 @@ expecteq "$(curl -s -X POST -d '{"clixon-example:input":{"x":"0","y":"99"}}' htt new "restconf example rpc json/json" # XXX example:input example:output -expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+json' -H 'Content-Type: application/yang-data+json' -d '{"clixon-example:input":{"x":"0"}}' http://localhost/restconf/operations/clixon-example:example)" 0 '{"clixon-example:output": {"x": "0","y": "42"}} +expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+json' -H 'Accept: application/yang-data+json' -d '{"clixon-example:input":{"x":"0"}}' http://localhost/restconf/operations/clixon-example:example)" 0 '{"clixon-example:output": {"x": "0","y": "42"}} ' new "restconf example rpc xml/json" -expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+xml' -H 'Content-Type: application/yang-data+json' -d '0' http://localhost/restconf/operations/clixon-example:example)" 0 '{"clixon-example:output": {"x": "0","y": "42"}} +expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+xml' -H 'Accept: application/yang-data+json' -d '0' http://localhost/restconf/operations/clixon-example:example)" 0 '{"clixon-example:output": {"x": "0","y": "42"}} ' new "restconf example rpc json/xml" @@ -92,6 +92,13 @@ new "restconf example rpc xml/xml" expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+xml' -H 'Accept: application/yang-data+xml' -d '0' http://localhost/restconf/operations/clixon-example:example)" 0 '042 ' +new "restconf example rpc xml in w json encoding (expect fail)" +expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+json' -H 'Accept: application/yang-data+xml' -d '0' http://localhost/restconf/operations/clixon-example:example)" 0 "rpcmalformed-messageerror on line 1: syntax error at or before: '<' " + + +new "restconf example rpc json in xml encoding (expect fail)" +expecteq "$(curl -s -X POST -H 'Content-Type: application/yang-data+xml' -H 'Accept: application/yang-data+xml' -d '{"clixon-example:input":{"x":"0"}}' http://localhost/restconf/operations/clixon-example:example)" 0 'rpcmalformed-messageerrorxml_parse: line 0: syntax error: at or before: " ' + new "netconf example rpc" expecteof "$clixon_netconf -qf $cfg" 0 '0]]>]]>' '^042]]>]]>$' diff --git a/test/test_stream.sh b/test/test_stream.sh index 01a5697c..806aa9b8 100755 --- a/test/test_stream.sh +++ b/test/test_stream.sh @@ -39,6 +39,7 @@ cat < $cfg $cfg /usr/local/share/clixon $IETFRFC + $fyang false /usr/local/var/$APPNAME/$APPNAME.sock /usr/local/lib/$APPNAME/backend @@ -101,7 +102,7 @@ cat < $fyang } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" @@ -109,15 +110,15 @@ if [ $BE -ne 0 ]; then if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg fi new "kill old restconf daemon" sudo pkill -u www-data -f "/www-data/clixon_restconf" new "start restconf daemon" -start_restconf -f $cfg -y $fyang +start_restconf -f $cfg new "waiting" wait_backend @@ -128,35 +129,35 @@ wait_restconf new "1. Netconf RFC5277 stream testing" # 1.1 Stream discovery new "netconf event stream discovery RFC5277 Sec 3.2.5" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' 'EXAMPLEExample event streamtrue]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' 'EXAMPLEExample event streamtrue]]>]]>' new "netconf event stream discovery RFC8040 Sec 6.2" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' 'EXAMPLEExample event streamtruexmlhttps://localhost/streams/EXAMPLE]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' 'EXAMPLEExample event streamtruexmlhttps://localhost/streams/EXAMPLE]]>]]>' # # 1.2 Netconf stream subscription new "netconf EXAMPLE subscription" -expectwait "$clixon_netconf -qf $cfg -y $fyang" 'EXAMPLE]]>]]>' '^]]>]]>20' $NCWAIT +expectwait "$clixon_netconf -qf $cfg" 'EXAMPLE]]>]]>' '^]]>]]>20' $NCWAIT new "netconf subscription with empty startTime" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'EXAMPLE]]>]]>' '^applicationbad-elementstartTimeerrorregexp match fail:' +expecteof "$clixon_netconf -qf $cfg" 0 'EXAMPLE]]>]]>' '^applicationbad-elementstartTimeerrorregexp match fail:' new "netconf EXAMPLE subscription with simple filter" -expectwait "$clixon_netconf -qf $cfg -y $fyang" 'EXAMPLE]]>]]>' '^]]>]]>20' $NCWAIT +expectwait "$clixon_netconf -qf $cfg" 'EXAMPLE]]>]]>' '^]]>]]>20' $NCWAIT new "netconf EXAMPLE subscription with filter classifier" -expectwait "$clixon_netconf -qf $cfg -y $fyang" "EXAMPLE]]>]]>" '^]]>]]>20' $NCWAIT +expectwait "$clixon_netconf -qf $cfg" "EXAMPLE]]>]]>" '^]]>]]>20' $NCWAIT new "netconf NONEXIST subscription" -expectwait "$clixon_netconf -qf $cfg -y $fyang" 'NONEXIST]]>]]>' '^applicationinvalid-valueerrorNo such stream]]>]]>$' $NCWAIT +expectwait "$clixon_netconf -qf $cfg" 'NONEXIST]]>]]>' '^applicationinvalid-valueerrorNo such stream]]>]]>$' $NCWAIT new "netconf EXAMPLE subscription with wrong date" -expectwait "$clixon_netconf -qf $cfg -y $fyang" 'EXAMPLEkallekaka]]>]]>' '^applicationbad-elementstartTimeerrorregexp match fail:' 0 +expectwait "$clixon_netconf -qf $cfg" 'EXAMPLEkallekaka]]>]]>' '^applicationbad-elementstartTimeerrorregexp match fail:' 0 #new "netconf EXAMPLE subscription with replay" #NOW=$(date +"%Y-%m-%dT%H:%M:%S") #sleep 10 -#expectwait "$clixon_netconf -qf $cfg -y $fyang" "EXAMPLE$NOW]]>]]>" '^]]>]]>20' 10 +#expectwait "$clixon_netconf -qf $cfg" "EXAMPLE$NOW]]>]]>" '^]]>]]>20' 10 sleep 2 # diff --git a/test/test_transaction.sh b/test/test_transaction.sh index 3699a9b5..69bddea0 100755 --- a/test/test_transaction.sh +++ b/test/test_transaction.sh @@ -87,6 +87,7 @@ checklog(){ s=$1 # statement l0=$2 # linenr new "Check $s in log" +# echo "grep \"transaction_log $s\" $flog" t=$(grep -n "transaction_log $s" $flog) if [ -z "$t" ]; then echo -e "\e[31m\nError in Test$testnr [$testname]:" @@ -124,7 +125,7 @@ if [ $BE -ne 0 ]; then sleep $RCWAIT fi -let nr=1 +let nr=0 new "Basic transaction to add top-level x" expecteof "$clixon_netconf -qf $cfg" 0 "$nr]]>]]>" '^]]>]]>$' diff --git a/test/test_union.sh b/test/test_union.sh index 6326c5ab..8fa52a7b 100755 --- a/test/test_union.sh +++ b/test/test_union.sh @@ -18,6 +18,7 @@ cat < $cfg $dir /usr/local/share/clixon $IETFRFC + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/cli $APPNAME @@ -76,7 +77,7 @@ module example{ } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" @@ -84,21 +85,21 @@ if [ $BE -ne 0 ]; then if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg new "waiting" sleep $RCWAIT fi new "cli set transitive string" -expectfn "$clixon_cli -1f $cfg -l o -y $fyang set c talle x" 0 "^$" +expectfn "$clixon_cli -1f $cfg -l o set c talle x" 0 "^$" new "cli set transitive union" -expectfn "$clixon_cli -1f $cfg -l o -y $fyang set c ulle 33" 0 "^$" +expectfn "$clixon_cli -1f $cfg -l o set c ulle 33" 0 "^$" new "cli set transitive union error" -expectfn "$clixon_cli -1f $cfg -l o -y $fyang set c ulle kalle" 255 '^CLI syntax error: "set c ulle kalle": Unknown command$' +expectfn "$clixon_cli -1f $cfg -l o set c ulle kalle" 255 '^CLI syntax error: "set c ulle kalle": Unknown command$' if [ $BE -eq 0 ]; then exit # BE diff --git a/test/test_when_must.sh b/test/test_when_must.sh index d4cbc0ea..a57c4fe4 100755 --- a/test/test_when_must.sh +++ b/test/test_when_must.sh @@ -15,6 +15,7 @@ cat < $cfg $cfg /usr/local/share/clixon $IETFRFC + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/cli $APPNAME @@ -89,56 +90,56 @@ module $APPNAME{ } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" - sudo clixon_backend -zf $cfg -y $fyang + sudo clixon_backend -zf $cfg if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg new "waiting" sleep $RCWAIT fi new "when: add static route" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'staticr1]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'staticr1]]>]]>' "^]]>]]>$" new "when: validate ok" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "when: add direct route" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'directr2]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'directr2]]>]]>' "^]]>]]>$" new "when get config" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^directr2staticr1]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^directr2staticr1]]>]]>$' new "when: validate fail" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^applicationoperation-failederrorwhen xpath validation failed]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^applicationoperation-failederrorwhen xpath validation failed]]>]]>$" new "when: discard-changes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "must: add interface" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'ethernet1500]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'ethernet1500]]>]]>' "^]]>]]>$" new "must: validate ok" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "must: add atm interface" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'atm32]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'atm32]]>]]>' "^]]>]]>$" new "must: atm validate fail" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^applicationoperation-failederrorAn ATM MTU must be 64 .. 17966]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^applicationoperation-failederrorAn ATM MTU must be 64 .. 17966]]>]]>$" new "must: add eth interface" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'ethernet989]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'ethernet989]]>]]>' "^]]>]]>$" new "must: eth validate fail" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^applicationoperation-failederrorAn Ethernet MTU must be 1500]]>]]>" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^applicationoperation-failederrorAn Ethernet MTU must be 1500]]>]]>" if [ $BE -eq 0 ]; then exit # BE diff --git a/test/test_xml.sh b/test/test_xml.sh index 52784a12..f24997ac 100755 --- a/test/test_xml.sh +++ b/test/test_xml.sh @@ -6,7 +6,7 @@ # Magic line must be first in script (see README.md) s="$_" ; . ./lib.sh || if [ "$s" = $0 ]; then exit 0; else return 0; fi -: ${clixon_util_xml:=clixon_util_xml} +: ${clixon_util_xml:=clixon_util_xml -o} # -o is output new "xml parse" expecteof "$clixon_util_xml" 0 "" "^$" diff --git a/test/test_yang.sh b/test/test_yang.sh index 6ce8ed3d..06161c00 100755 --- a/test/test_yang.sh +++ b/test/test_yang.sh @@ -17,6 +17,7 @@ cat < $cfg /usr/local/share/clixon $dir $IETFRFC + $fyang /usr/local/lib/$APPNAME/clispec /usr/local/lib/$APPNAME/cli $APPNAME @@ -141,35 +142,35 @@ module $APPNAME{ } EOF -new "test params: -f $cfg -y $fyang" +new "test params: -f $cfg" if [ $BE -ne 0 ]; then new "kill old backend" - sudo clixon_backend -zf $cfg -y $fyang + sudo clixon_backend -zf $cfg if [ $? -ne 0 ]; then err fi - new "start backend -s init -f $cfg -y $fyang" - start_backend -s init -f $cfg -y $fyang + new "start backend -s init -f $cfg" + start_backend -s init -f $cfg new "waiting" wait_backend fi new "cli defined extension" -expectfn "$clixon_cli -1f $cfg -y $fyang show version" 0 "3." +expectfn "$clixon_cli -1f $cfg show version" 0 "3." new "empty values in leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'a]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'a]]>]]>' "^]]>]]>$" new "empty values in leaf-list2" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' "^]]>]]>$" new "netconf get config" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^a]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^a]]>]]>$' new "netconf discard-changes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" #new "cli not defined extension" @@ -179,106 +180,106 @@ expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "] #expectfn "$clixon_cli -1f $cfg -y $fyangerr show version" 0 "Yang error: Extension ex:not-defined not found" new "netconf schema resource, RFC 7895" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' 'ietf-yang-types2013-07-15urn:ietf:params:xml:ns:yang:ietf-yang-typesimplement' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' 'ietf-yang-types2013-07-15urn:ietf:params:xml:ns:yang:ietf-yang-typesimplement' new "netconf edit config" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '125one]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '125one]]>]]>' "^]]>]]>$" new "netconf commit" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" # text empty type in running new "netconf commit 2nd" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "netconf get config xpath" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^125one]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^125one]]>]]>$' new "netconf edit leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'hejhopp]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'hejhopp]]>]]>' "^]]>]]>$" new "netconf get leaf-list" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^hejhopp]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^hejhopp]]>]]>$' new "netconf get leaf-list path" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^hejhopp]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^hejhopp]]>]]>$" new "netconf get (should be some)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" '^125one' +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" '^125one' new "cli set leaf-list" -expectfn "$clixon_cli -1f $cfg -y $fyang set x f e foo" 0 "" +expectfn "$clixon_cli -1f $cfg set x f e foo" 0 "" new "cli show leaf-list" -expectfn "$clixon_cli -1f $cfg -y $fyang show xpath /x/f/e" 0 "foo" +expectfn "$clixon_cli -1f $cfg show xpath /x/f/e" 0 "foo" new "netconf set state data (not allowed)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '42]]>]]>' '^protocolinvalid-valueerrorState data not allowed]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 '42]]>]]>' '^protocolinvalid-valueerrorState data not allowed]]>]]>$' new "netconf set presence and not present" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' "^]]>]]>$" new "netconf get presence only" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^]]>]]>$' new "netconf get presence only" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' "^]]>]]>$" new "netconf discard-changes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "netconf anyxml" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' "^]]>]]>$" new "netconf validate anyxml" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "netconf delete candidate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'none]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'none]]>]]>' "^]]>]]>$" # Check 3-keys new "netconf add one 3-key entry" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '111one]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '111one]]>]]>' "^]]>]]>$" new "netconf check add one 3-key" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '111one]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '111one]]>]]>' new "netconf add another (with same 1st key)" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '121two]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '121two]]>]]>' "^]]>]]>$" new "netconf check add another" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '111one121two]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '111one121two]]>]]>' new "netconf replace first" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '111replace]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '111replace]]>]]>' "^]]>]]>$" new "netconf check replace" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '111replace121two]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '111replace121two]]>]]>' new "netconf delete first" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 '111]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 '111]]>]]>' "^]]>]]>$" new "netconf check delete" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '121two]]>]]>' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '121two]]>]]>' # clear db for next test new "netconf delete candidate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'none]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'none]]>]]>' "^]]>]]>$" new "netconf commit empty candidate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "netconfig config submodule" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 'afoo]]>]]>' "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 'afoo]]>]]>' "^]]>]]>$" new "netconf submodule get config" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 ']]>]]>' '^afoo]]>]]>$' +expecteof "$clixon_netconf -qf $cfg" 0 ']]>]]>' '^afoo]]>]]>$' new "netconf submodule validate" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" new "netconf submodule discard-changes" -expecteof "$clixon_netconf -qf $cfg -y $fyang" 0 "]]>]]>" "^]]>]]>$" +expecteof "$clixon_netconf -qf $cfg" 0 "]]>]]>" "^]]>]]>$" if [ $BE -eq 0 ]; then exit # BE diff --git a/util/clixon_util_json.c b/util/clixon_util_json.c index 2a3a1c58..2ae5bed3 100644 --- a/util/clixon_util_json.c +++ b/util/clixon_util_json.c @@ -59,7 +59,7 @@ #include "clixon/clixon.h" /* - * Turn this on to get a json parse and pretty print test program + * JSON parse and pretty print test program * Usage: xpath * read json from input * Example compile: diff --git a/util/clixon_util_xml.c b/util/clixon_util_xml.c index 001c08a2..06504753 100644 --- a/util/clixon_util_xml.c +++ b/util/clixon_util_xml.c @@ -34,6 +34,11 @@ * XML support functions. * @see https://www.w3.org/TR/2008/REC-xml-20081126 * https://www.w3.org/TR/2009/REC-xml-names-20091208 + * The function can do yang validation, process xml and json, etc. + * On success, nothing is printed and exitcode 0 + * On failure, an error is printed on stderr and exitcode != 0 + * Failure error prints are different, it would be nice to make them more + * uniform. (see clicon_rpc_generate_error) */ #ifdef HAVE_CONFIG_H @@ -49,7 +54,9 @@ #include #include #include +#include #include +#include /* cligen */ #include @@ -69,12 +76,20 @@ static int usage(char *argv0) { - fprintf(stderr, "usage:%s [options]\n" + fprintf(stderr, "usage:%s [options] with xml on stdin\n" "where options are\n" "\t-h \t\tHelp\n" "\t-D \tDebug\n" + "\t-f \tXML input file (overrides stdin)\n" + "\t-J \t\tInput as JSON\n" "\t-j \t\tOutput as JSON\n" - "\t-l \tLog on (s)yslog, std(e)rr, std(o)ut (stderr is default)\n", + "\t-l \tLog on (s)yslog, std(e)rr, std(o)ut (stderr is default)\n" + "\t-o \t\tOutput the file\n" + "\t-v \t\tValidate the result in terms of Yang model (requires -y)\n" + "\t-p \t\tPretty-print output\n" + "\t-y \tYang filename or dir (load all files)\n" + "\t-Y \tYang dirs (can be several)\n" + , argv0); exit(0); } @@ -83,17 +98,39 @@ int main(int argc, char **argv) { - int retval = -1; - cxobj *xt = NULL; - cxobj *xc; - cbuf *cb = cbuf_new(); - int c; - int logdst = CLICON_LOG_STDERR; - int json = 0; + int retval = -1; + cxobj *xt = NULL; + cxobj *xc; + cbuf *cb = cbuf_new(); + int c; + int logdst = CLICON_LOG_STDERR; + int jsonin = 0; + int jsonout = 0; + char *input_filename = NULL; + char *yang_file_dir = NULL; + yang_stmt *yspec = NULL; + cxobj *xerr = NULL; /* malloced must be freed */ + int ret; + int pretty = 0; + int validate = 0; + int output = 0; + clicon_handle h; + struct stat st; + int fd = 0; /* stdin */ + cxobj *xcfg = NULL; + cbuf *cbret = NULL; + /* In the startup, logs to stderr & debug flag set later */ + clicon_log_init(__FILE__, LOG_INFO, CLICON_LOG_STDERR); + + if ((h = clicon_handle_init()) == NULL) + goto done; + + xcfg = xml_new("clixon-config", NULL, NULL); + clicon_conf_xml_set(h, xcfg); optind = 1; opterr = 0; - while ((c = getopt(argc, argv, "hD:jl:")) != -1) + while ((c = getopt(argc, argv, "hD:f:Jjl:pvoy:Y:")) != -1) switch (c) { case 'h': usage(argv[0]); @@ -102,37 +139,131 @@ main(int argc, if (sscanf(optarg, "%d", &debug) != 1) usage(argv[0]); break; + case 'f': + input_filename = optarg; + break; + case 'J': + jsonin++; + break; case 'j': - json++; + jsonout++; break; case 'l': /* Log destination: s|e|o|f */ if ((logdst = clicon_log_opt(optarg[0])) < 0) usage(argv[0]); break; + case 'o': + output++; + break; + case 'v': + validate++; + break; + case 'p': + pretty++; + break; + case 'y': + yang_file_dir = optarg; + break; + case 'Y': + if (clicon_option_add(h, "CLICON_YANG_DIR", optarg) < 0) + goto done; + break; default: usage(argv[0]); break; } - clicon_log_init(__FILE__, debug?LOG_DEBUG:LOG_INFO, logdst); - if (xml_parse_file(0, "", NULL, &xt) < 0){ - fprintf(stderr, "xml parse error %s\n", clicon_err_reason); - goto done; + if (validate && !yang_file_dir){ + fprintf(stderr, "-v requires -y\n"); + usage(argv[0]); + } + clicon_log_init(__FILE__, debug?LOG_DEBUG:LOG_INFO, logdst); + /* 1. Parse yang */ + if (yang_file_dir){ + if ((yspec = yspec_new()) == NULL) + goto done; + if (stat(yang_file_dir, &st) < 0){ + clicon_err(OE_YANG, errno, "%s not found", yang_file_dir); + goto done; + } + if (S_ISDIR(st.st_mode)){ + if (yang_spec_load_dir(h, yang_file_dir, yspec) < 0) + goto done; + } + else{ + if (yang_spec_parse_file(h, yang_file_dir, yspec) < 0) + goto done; + } + } + if (input_filename){ + if ((fd = open(input_filename, O_RDONLY)) < 0){ + clicon_err(OE_YANG, errno, "open(%s)", input_filename); + goto done; + } + } + /* 2. Parse data (xml/json) */ + if (jsonin){ + if ((ret = json_parse_file(fd, yspec, &xt, &xerr)) < 0) + goto done; + if (ret == 0){ + clicon_rpc_generate_error("util_xml", xerr); + goto done; + } + } + else{ + if (xml_parse_file(fd, "", NULL, &xt) < 0){ + fprintf(stderr, "xml parse error: %s\n", clicon_err_reason); + goto done; + } + } + + /* Dump data structures (for debug) */ + if (debug){ + cbuf_reset(cb); + xmltree2cbuf(cb, xt, 0); + fprintf(stderr, "%s\n", cbuf_get(cb)); + cbuf_reset(cb); + } + + /* 3. Validate data (if yspec) */ + if (validate){ + xc = xml_child_i(xt, 0); + /* Populate */ + if (xml_apply0(xc, CX_ELMNT, xml_spec_populate, yspec) < 0) + goto done; + /* Sort */ + if (xml_apply0(xc, CX_ELMNT, xml_sort, h) < 0) + goto done; + /* Add default values */ + if (xml_apply(xc, CX_ELMNT, xml_default, h) < 0) + goto done; + if (xml_apply0(xc, -1, xml_sort_verify, h) < 0) + clicon_log(LOG_NOTICE, "%s: sort verify failed", __FUNCTION__); + if ((ret = xml_yang_validate_all_top(h, xc, &xerr)) < 0) + goto done; + if (ret > 0 && (ret = xml_yang_validate_add(h, xc, &xerr)) < 0) + goto done; + if (ret == 0){ + if (netconf_err2cb(xerr, &cbret) < 0) + goto done; + fprintf(stderr, "xml validation error: %s\n", cbuf_get(cbret)); + goto done; + } + } + /* 4. Output data (xml/json) */ + if (output){ + xc = NULL; + while ((xc = xml_child_each(xt, xc, -1)) != NULL) + if (jsonout) + xml2json_cbuf(cb, xc, pretty); /* print xml */ + else + clicon_xml2cbuf(cb, xc, 0, pretty); /* print xml */ + fprintf(stdout, "%s", cbuf_get(cb)); + fflush(stdout); } - xc = NULL; - while ((xc = xml_child_each(xt, xc, -1)) != NULL) - if (json) - xml2json_cbuf(cb, xc, 0); /* print xml */ - else - clicon_xml2cbuf(cb, xc, 0, 0); /* print xml */ - fprintf(stdout, "%s", cbuf_get(cb)); - fflush(stdout); -#if 0 - cbuf_reset(cb); - xmltree2cbuf(cb, xt, 0); /* dump data structures */ - fprintf(stderr, "%s\n", cbuf_get(cb)); -#endif retval = 0; done: + if (cbret) + cbuf_free(cbret); if (xt) xml_free(xt); if (cb) diff --git a/yang/Makefile.in b/yang/Makefile.in index aa7ceb39..d9ee3301 100644 --- a/yang/Makefile.in +++ b/yang/Makefile.in @@ -39,8 +39,6 @@ includedir = @includedir@ datarootdir = @datarootdir@ enable_stdyangs = @enable_stdyangs@ -CLIXON_DATADIR = @CLIXON_DATADIR@ - SUBDIRS = clixon # See configure.ac ifeq ($(enable_stdyangs),yes) diff --git a/yang/clixon/Makefile.in b/yang/clixon/Makefile.in index bd541876..c982ddc2 100644 --- a/yang/clixon/Makefile.in +++ b/yang/clixon/Makefile.in @@ -38,10 +38,11 @@ bindir = @bindir@ includedir = @includedir@ datarootdir = @datarootdir@ -CLIXON_DATADIR = @CLIXON_DATADIR@ +# See also STD_YANG_INSTALLDIR for the standard yang files +YANG_INSTALLDIR = @YANG_INSTALLDIR@ -YANGSPECS = clixon-config@2019-03-05.yang -YANGSPECS += clixon-lib@2019-01-02.yang +YANGSPECS = clixon-config@2019-06-05.yang +YANGSPECS += clixon-lib@2019-06-05.yang YANGSPECS += clixon-rfc5277@2008-07-01.yang YANGSPECS += clixon-xml-changelog@2019-03-21.yang @@ -55,11 +56,11 @@ distclean: clean rm -f Makefile *~ .depend install: $(YANGSPECS) - install -d -m 0755 $(DESTDIR)$(CLIXON_DATADIR) - install -m 0644 $(YANGSPECS) $(DESTDIR)$(CLIXON_DATADIR) + install -d -m 0755 $(DESTDIR)$(YANG_INSTALLDIR) + install -m 0644 $(YANGSPECS) $(DESTDIR)$(YANG_INSTALLDIR) uninstall: - (cd $(DESTDIR)$(CLIXON_DATADIR); rm -rf *.yang) + (cd $(DESTDIR)$(YANG_INSTALLDIR); rm -rf *.yang) install-include: diff --git a/yang/clixon/clixon-config@2019-03-05.yang b/yang/clixon/clixon-config@2019-06-05.yang similarity index 97% rename from yang/clixon/clixon-config@2019-03-05.yang rename to yang/clixon/clixon-config@2019-06-05.yang index 92bf3be5..6a1112d8 100644 --- a/yang/clixon/clixon-config@2019-03-05.yang +++ b/yang/clixon/clixon-config@2019-06-05.yang @@ -39,7 +39,15 @@ module clixon-config { ***** END LICENSE BLOCK *****"; - revision 2019-03-05 { + revision 2019-06-05 { + description + "Added: CLICON_YANG_REGEXP, CLICON_CLI_TAB_MODE, + CLICON_CLI_HIST_FILE, CLICON_CLI_HIST_SIZE, + CLICON_XML_CHANGELOG, CLICON_XML_CHANGELOG_FILE; + Renamed CLICON_XMLDB_CACHE to CLICON_DATASTORE_CACHE (changed type) + Deleted: CLICON_XMLDB_PLUGIN, CLICON_USE_STARTUP_CONFIG"; + } + revision 2019-03-05{ description "Changed URN. Changed top-level symbol to clixon-config. Released in Clixon 3.10"; @@ -197,7 +205,7 @@ module clixon-config { "Yang directory path for finding module and submodule files. A list of these options should be in the configuration. When loading a Yang module, Clixon searches this list in the order - they appear. Ensure that CLIXON_DATADIR(default + they appear. Ensure that YANG_INSTALLDIR(default /usr/local/share/clixon) is present in the path"; } leaf CLICON_YANG_MAIN_FILE { @@ -427,14 +435,6 @@ module clixon-config { description "Directory where \"running\", \"candidate\" and \"startup\" are placed."; } - leaf CLICON_XMLDB_PLUGIN { - type string; - status obsolete; - description - "XMLDB datastore plugin filename - (see datastore/ and clixon_xml_db.[ch]) - Obsolete: Merged with libclixon in 3.10"; - } leaf CLICON_DATASTORE_CACHE { type datastore_cache; default cache; diff --git a/yang/clixon/clixon-lib@2019-01-02.yang b/yang/clixon/clixon-lib@2019-06-05.yang similarity index 93% rename from yang/clixon/clixon-lib@2019-01-02.yang rename to yang/clixon/clixon-lib@2019-06-05.yang index e0e3b03d..370819c8 100644 --- a/yang/clixon/clixon-lib@2019-01-02.yang +++ b/yang/clixon/clixon-lib@2019-06-05.yang @@ -40,6 +40,10 @@ module clixon-lib { ***** END LICENSE BLOCK *****"; + revision 2019-06-05 { + description + "ping rpc added for liveness"; + } revision 2019-01-02 { description "Released in Clixon 3.9"; @@ -53,6 +57,6 @@ module clixon-lib { } } rpc ping { - description "Check aliveness of backend daemon."; + description "Check aliveness of backend daemon."; } } diff --git a/yang/standard/Makefile.in b/yang/standard/Makefile.in index 6da576ae..47061637 100644 --- a/yang/standard/Makefile.in +++ b/yang/standard/Makefile.in @@ -38,8 +38,8 @@ bindir = @bindir@ includedir = @includedir@ datarootdir = @datarootdir@ -# Could place them in separate standards dir? -CLIXON_DATADIR = @CLIXON_DATADIR@ +# See also YANG_INSTALLDIR for the clixon-specific yang files +STD_YANG_INSTALLDIR = @STD_YANG_INSTALLDIR@ YANGSPECS = iana-if-type@2014-05-08.yang YANGSPECS += ietf-interfaces@2018-02-20.yang @@ -61,11 +61,11 @@ distclean: clean rm -f Makefile *~ .depend install: $(YANGSPECS) - install -d -m 0755 $(DESTDIR)$(CLIXON_DATADIR) - install -m 0644 $(YANGSPECS) $(DESTDIR)$(CLIXON_DATADIR) + install -d -m 0755 $(DESTDIR)$(STD_YANG_INSTALLDIR) + install -m 0644 $(YANGSPECS) $(DESTDIR)$(STD_YANG_INSTALLDIR) uninstall: - (cd $(DESTDIR)$(CLIXON_DATADIR); rm -rf *.yang) + (cd $(DESTDIR)$(STD_YANG_INSTALLDIR); rm -rf *.yang) install-include: