diff --git a/CHANGELOG.md b/CHANGELOG.md index 5dd3e2d2..b8291dcd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,98 +1,81 @@ # Clixon Changelog -## 3.8.0 (Upcoming) +## 3.8.0 (Expected: Nov 4) ### Major New features * YANG Features * Yang 1.1 feature and if-feature according to RFC 7950 7.20.1 and 7.20.2. * See https://github.com/clicon/clixon/issues/41 - * Features are declared via CLICON_FEATURE in the configuration file. Examples showing enabling (1) a specific feature; (2) all features in a module; (3) all features in all modules: -``` + * Features are declared via CLICON_FEATURE in the configuration file. Example below shows enabling (1) a specific feature; (2) all features in a module; (3) all features in all modules: + ``` ietf-routing:router-id ietf-routing:* *:* -``` + ``` * logical combination of features not implemented, eg if-feature "not foo or bar and baz"; * ietf-netconf yang module added with candidate, validate, startup and xpath features enabled. -* YANG Module Library support - * According to RFC 7895 and implemented by ietf-yang-library.yang - * Changed Netconf hello to single capabilty urn:ietf:params:netconf:capability:yang-library:1.0 according to YANG 1.1 RFC7950 Sec 5.6.4. - * Set by option: CLICON_MODULE_LIBRARY_RFC7895 - enabled by default - * Option CLICON_MODULE_SET_ID is set and changed when modules change. - * Notification not supported +* YANG module library + * YANG modules according to RFC 7895 and implemented by ietf-yang-library.yang + * Enabled by configuration option CLICON_MODULE_LIBRARY_RFC7895 - enabled by default + * RFC 7895 defines a module-set-id. COnfigure option CLICON_MODULE_SET_ID is set and changed when modules change. * Yang 1.1 notification support (RFC 7950: Sec 7.16) -* Restconf stream notification support - two variants. - * Both a "native" stream support and one using nginx/nchan pub/sub. - * See (apps/restconf/README.md) for details. -* New event streams implementation +* New event streams implementation with replay * See clicon_stream.[ch] for details * Added stream discovery according to RFC 5277 for netconf and RFC 8040 for restconf - * Enabled by CLICON_STREAM_DISCOVERY_RFC5277 and CLICON_STREAM_DISCOVERY_RFC8040. + * Enabled by CLICON_STREAM_DISCOVERY_RFC5277 and CLICON_STREAM_DISCOVERY_RFC8040 + * Configure option CLICON_STREAM_RETENTION is default number of seconds before dropping replay buffers +8040. +* Restconf stream notification support according to RFC8040 + * See (apps/restconf/README.md) for more details. + * start-time and stop-time query parameters * Set access/subscribe base URL with: CLICON_STREAM_URL (default "https://localhost") and CLICON_STREAM_PATH (default "streams") * Example: new stream "foo" will get access URL: https://localhost/streams/foo - * Optional pub/sub support enabled by ./configure --enable-publish + * Alternative variant using pub/sub support enabled by ./configure --enable-publish * Set publish URL base with: CLICON_STREAM_PUB (default http://localhost/pub) - * Example: new stream "foo" will get pub URL: https://localhost/pub/foo -* Stream replay support + * Example: new stream "foo" will publish event streams on URL: https://localhost/pub/foo * RFC8040 Restconf replay support: start-time and stop-time query parameter support * This only applies to "native" restconf stream support, Nchan mode has different replay functionality - * RFC5277 Netconf replay support using and + * RFC5277 Netconf replay supported * Replay support is only in-memory and not persistent. External time-series DB could be added. ### API changes on existing features (you may need to change your code) -* clixon-config YAML file has new revision: 2018-10-21. * Netconf hello capability updated to YANG 1.1 RFC7950 Sec 5.6.4 * Added urn:ietf:params:netconf:capability:yang-library:1.0 * Thanks @SCadilhac for helping out, see https://github.com/clicon/clixon/issues/39 -* Major rewrite of event streams +* Major rewrite of event streams (as described above) * If you used old event callbacks API, you need to switch to the streams API * See clixon_stream.[ch] * Old streams API which needs to be removed include: * clicon_log_register_callback() - * subscription_add() --> stream_register() + * subscription_add() --> stream_add() * backend_notify() and backend_notify_xml() - use stream_notify() instead * Example uses "NETCONF" stream instead of "ROUTING" -* clixon_restconf and clixon_netconf now take -D as command-line option instead of just -D +* clixon_restconf and clixon_netconf changed to take -D `` as command-line option instead of just -D (without debig level) * This aligns to clixon_cli and clixon_backend * Application command option -S to clixon_netconf is obsolete. Use `clixon_netconf -l s` instead. -* Unified log handling for all clicon applications using -l e|o|s|f. +* Unified log handling for all clicon applications using command-line option: `-l e|o|s|f`. * The options stand for e:stderr, o:stdout, s: syslog, f:file * Added file logging (`-l f` or `-l f`) for cases where neither syslog nor stderr is useful. * Comply to RFC 8040 3.5.3.1 rule: api-identifier = [module-name ":"] identifier * The "module-name" was a no-op before. - * This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/XXXX:modules-state + * This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/foobar:modules-state * Generilized top-level yang parsing functions - * All yang modules are stored in the clicon_dbspec_yang() option. - * Except clixon-config module due to bug reported below * Clarified semantics of main yang module: - * -y option to commands MUST specify filename - * CLICON_YANG_MODULE_MAIN MUST specify a module + * Command-line option -y MUST specify a filename + * Configure option CLICON_YANG_MODULE_MAIN MUST specify a module name * yang_parse() changed to take either filename or module name and revision. - * Removed clicon_dbspec_name[_set](). - * Replace code for initializing the main yang module - * Replace yang_spec_main() with yang_spec_parse_module() as follows: -``` - /* old code */ - if ((yspec = yang_spec_main(ch)) == NULL) - goto done; - /* new code */ - if ((yspec = yspec_new()) == NULL) - goto done; - clicon_dbspec_yang_set(h, yspec); - if (yang_spec_parse_module(h, clicon_yang_module_main(h), - clicon_yang_dir(h), clicon_yang_module_revision(h), yspec) < 0) - goto done; -``` + * Removed clicon_dbspec_name() and clicon_dbspec_name_set(). + * Replace calls to yang_spec_main() with yang_spec_parse_module(). See for + example backend_main() and others if you need details. ### Minor changes +* clixon-config YAML file has new revision: 2018-10-21. * Allow new lines in CLI prompts * uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters -* Added CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option). -* Yang 1.1 action syntax added (but function is not supported) +* Added Configure option CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option). * New function: clicon_conf_xml() returns configuration tree * Obsoleted COMPAT_CLIV and COMPAT_XSL that were optional in 3.7 -* Added timeout option -t for clixon_netconf - quit after max time. -* Added -l option for clixon_backend for directing syslog to stderr or stdout if running in foreground +* Added command-line option `-t ` for clixon_netconf - quit after max time. ### Corrected Bugs * Single quotes for XML attributes https://github.com/clicon/clixon/issues/51 @@ -104,12 +87,12 @@ * Set dir /www-data with www-data as owner, see https://github.com/clicon/clixon/issues/37 ### Known issues -* netconf rpc input is not sanity checked for wrong symbols (just ignored). +* Netconf RPC input is not sanity checked for wrong symbols (just ignored). * Yang sub-command order and cardinality not checked. * Top-level Yang symbol cannot be called "config" in any imported yang file. * datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanity. * Namespace name relabeling is not supported. - * Eg: if "des" is defined as prefix for an imported module, then a relabeling using xmlfns is not supported, such as: + * Eg: if "des" is defined as prefix for an imported module, then a relabeling using xmlns is not supported, such as: ``` x:des3 ``` diff --git a/README.md b/README.md index 440f57af..c06986ad 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # Clixon -Clixon is an automatic configuration manager where you generate -interactive CLI, NETCONF, RESTCONF and embedded databases with -transaction support from a YANG specification. +Clixon is a YANG-based configuration manager, with interactive CLI, +NETCONF and RESTCONF interfaces, an embedded database and transaction +support. * [Background](#background) * [Frequently asked questions](doc/FAQ.md) @@ -107,23 +107,21 @@ specification for handling XML configuration data. The YANG spec is used to generate an interactive CLI, netconf and restconf clients. It also manages an XML datastore. -Clixon mainly follows [YANG 1.0 RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) with some exceptions: -- conformance: deviation -- list features: min/max-elements, unique -- action statements +Clixon follows: +- [YANG 1.0 RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) +- [YANG 1.1 RFC 7950](https://www.rfc-editor.org/rfc/rfc7950.txt). +- [RFC 7895: YANG module library](http://www.rfc-base.org/txt/rfc-7895.txt) -The aim is also to cover new features in YANG 1.1 [YANG RFC 7950](https://www.rfc-editor.org/rfc/rfc7950.txt) - -Clixon has its own XML library designed for performance. +However, the following YANG syntax modules are not implemented: +`deviation`, `min/max-elements`, `unique`, and `action`. Netconf ======= Clixon implements the following NETCONF proposals or standards: -- [NETCONF Configuration Protocol](http://www.rfc-base.org/txt/rfc-4741.txt) -- [Using the NETCONF Configuration Protocol over Secure Shell (SSH)](http://www.rfc-base.org/txt/rfc-4742.txt) -- [NETCONF Event Notifications](http://www.rfc-base.org/txt/rfc-5277.txt) - -Some updates are being made to RFC 6241 and RFC 6242. +- [RFC 6241: NETCONF Configuration Protocol](http://www.rfc-base.org/txt/rfc-6241.txt) +- [RFC 6242: Using the NETCONF Configuration Protocol over Secure Shell (SSH)](http://www.rfc-base.org/txt/rfc-6242.txt) +- [RFC 5277: NETCONF Event Notifications](http://www.rfc-base.org/txt/rfc-5277.txt) +- [RFC 8341: Network Configuration Access Control Model](http://www.rfc-base.org/txt/rfc-8341.txt) Clixon does not yet support the following netconf features: @@ -154,8 +152,6 @@ The Clixon datastore is a stand-alone XML based datastore. The idea is to be able to use different datastores backends with the same API. -Update: There used to be a key-value plugin based on qdbm but isnow obsoleted. Only a text datastore is implemented. - The datastore is primarily designed to be used by Clixon but can be used separately. @@ -163,7 +159,6 @@ See [more detailed instructions](datastore/README.md). Auth ==== - Authentication is managed outside Clixon using SSH, SSL, Oauth2, etc. For CLI, login is typically made via SSH. For netconf, SSH netconf diff --git a/apps/restconf/restconf_stream.c b/apps/restconf/restconf_stream.c index 2231d4a4..04edd129 100644 --- a/apps/restconf/restconf_stream.c +++ b/apps/restconf/restconf_stream.c @@ -172,15 +172,19 @@ static int restconf_stream(clicon_handle h, FCGX_Request *r, char *name, + cvec *qvec, int pretty, int use_xml, int *sp) { - int retval = -1; - cxobj *xret = NULL; - cxobj *xe; - cbuf *cb = NULL; - int s; /* socket */ + int retval = -1; + cxobj *xret = NULL; + cxobj *xe; + cbuf *cb = NULL; + int s; /* socket */ + int i; + cg_var *cv; + char *vname; *sp = -1; clicon_debug(1, "%s", __FUNCTION__); @@ -188,7 +192,23 @@ restconf_stream(clicon_handle h, clicon_err(OE_XML, errno, "cbuf_new"); goto done; } - cprintf(cb, "%s]]>]]>", name); + cprintf(cb, "%s", name); + /* Print all fields */ + for (i=0; i"); + cv2cbuf(cv, cb); + cprintf(cb, ""); + } + else if (strcmp(vname, "stop-time") == 0){ + cprintf(cb, ""); + cv2cbuf(cv, cb); + cprintf(cb, ""); + } + } + cprintf(cb, "]]>]]>"); if (clicon_rpc_netconf(h, cbuf_get(cb), &xret, &s) < 0) goto done; if ((xe = xpath_first(xret, "rpc-reply/rpc-error")) != NULL){ @@ -267,7 +287,7 @@ api_stream(clicon_handle h, int s=-1; clicon_debug(1, "%s", __FUNCTION__); - path = FCGX_GetParam("REQUEST_URI", r->envp); + path = FCGX_GetParam("DOCUMENT_URI", r->envp); query = FCGX_GetParam("QUERY_STRING", r->envp); pretty = clicon_option_bool(h, "CLICON_RESTCONF_PRETTY"); test(r, 1); @@ -294,7 +314,6 @@ api_stream(clicon_handle h, clicon_debug(1, "%s: method=%s", __FUNCTION__, method); if (str2cvec(query, '&', '=', &qvec) < 0) goto done; - if (str2cvec(path, '/', '=', &pcvec) < 0) /* rest url eg /album=ricky/foo */ goto done; /* data */ @@ -327,7 +346,7 @@ api_stream(clicon_handle h, goto ok; } clicon_debug(1, "%s auth2:%d %s", __FUNCTION__, authenticated, clicon_username_get(h)); - if (restconf_stream(h, r, method, pretty, use_xml, &s) < 0) + if (restconf_stream(h, r, method, qvec, pretty, use_xml, &s) < 0) goto done; if (s != -1){ /* Listen to backend socket */ diff --git a/doc/FAQ.md b/doc/FAQ.md index b21a623f..25727e98 100644 --- a/doc/FAQ.md +++ b/doc/FAQ.md @@ -2,17 +2,18 @@ ## What is Clixon? -Clixon is a configuration management tool including a generated CLI , -Yang parser, netconf and restconf interface and an embedded databases. +Clixon is a YANG-based configuration manager, with interactive CLI, +NETCONF and RESTCONF interfaces, an embedded database and transaction +support. ## Why should I use Clixon? -If you want an easy-to-use configuration frontend based on yang with an +If you want an easy-to-use configuration toolkit based on yang with an open-source license. Typically for embedded devices requiring a config interface such as routers and switches. ## What license is available? -CLIXON is dual license. Either Apache License, Version 2.0 or GNU +Clixon is dual license. Either Apache License, Version 2.0 or GNU General Public License Version 2. ## Is Clixon extendible? @@ -41,6 +42,16 @@ The example: sudo make install ``` +## How do you run Clixon example commands? + +- Start a backend server: `clixon_backend -Ff /usr/local/etc/example.xml` +- Start a cli session: `clixon_cli -f /usr/local/etc/example.xml` +- Start a netconf session: `clixon_netconf -f /usr/local/etc/example.xml` +- Start a restconf daemon: `sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data` +- Send a restconf command: `curl -G http://127.0.0.1/restconf/data` + +More info in the [example](../example) directory. + ## Do I need to setup anything? (IMPORTANT) The config demon requires a valid group to create a server UNIX domain socket. @@ -63,18 +74,11 @@ clicon:x:1001:,www-data ## What about reference documentation? Clixon uses Doxygen for reference documentation. -Build using 'make doc' and aim your browser at doc/html/index.html or -use the web resource: http://clicon.org/ref/index.html - -## How do you run the example? -- Start a backend server: 'clixon_backend -Ff /usr/local/etc/example.xml' -- Start a cli session: clixon_cli -f /usr/local/etc/example.xml -- Start a netconf session: clixon_netconf -f /usr/local/etc/example.xml +Build using 'make doc' and aim your browser at doc/html/index.html. ## How is configuration data stored? -Configuration data is stored in an XML datastore. The default is a -text-based datastore. In the example the datastore are regular files found in -/usr/local/var/example/. +Configuration data is stored in an XML datastore. In the example the +datastore are regular files found in /usr/local/var/example/. ## What is validate and commit? Clixon follows netconf in its validate and commit semantics. @@ -93,7 +97,7 @@ configuration file is /usr/local/etc/clixon.xml. The example configuration file is installed at /usr/local/etc/example.xml. The YANG specification for the configuration file is clixon-config.yang. -You can change where CLixon looks for the configuration FILE as follows: +You can change where Clixon looks for the configuration FILE as follows: - Provide -f FILE option when starting a program (eg clixon_backend -f FILE) - Provide --with-configfile=FILE when configuring - Provide --with-sysconfig= when configuring, then FILE is /clixon.xml @@ -130,7 +134,9 @@ Look in the example documentation for more info. As an alternative to cli configuration, you can use netconf. Easiest is to just pipe netconf commands to the clixon_netconf application. Example: +``` echo "]]>]]>" | clixon_netconf -f /usr/local/etc/example.xml +``` However, more useful is to run clixon_netconf as an SSH subsystem. Register the subsystem in /etc/sshd_config: @@ -155,7 +161,6 @@ For example, using nginx, install, and edit config file: /etc/nginx/sites-availa server { ... location /restconf { - root /usr/share/nginx/html/restconf; fastcgi_pass unix:/www-data/fastcgi_restconf.sock; include fastcgi_params; } @@ -166,8 +171,6 @@ Start nginx daemon sudo /etc/init.d/nginx start ``` -Read more in the restconf docs. - Example: ``` curl -G http://127.0.0.1/restconf/data/interfaces/interface/name=eth9/type @@ -177,14 +180,24 @@ Example: } ] ``` +Read more in the (restconf)[../apps/restconf] docs. -## How do I use notifications? + +## Does Clixon support event streams? + +Yes, Clixon supports event notification streams in the CLI, Netconf and Restconf API:s. The example has a prebuilt notification stream called "EXAMPLE" that triggers every 5s. You enable the notification via the CLI: ``` cli> notify cli> +event-class fault; +reportingEntity { + card Ethernet0; +} +severity major; +... ``` or via NETCONF: ``` diff --git a/example/README.md b/example/README.md index a9f782c3..6fdc8641 100644 --- a/example/README.md +++ b/example/README.md @@ -41,7 +41,7 @@ Send netconf command: ``` Start clixon restconf daemon ``` -> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data + sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data ``` Send restconf command ``` @@ -81,7 +81,7 @@ Send restconf command The example has an EXAMPLE stream notification triggering every 5s. To start a notification stream in the session using netconf, create a subscription: ``` -ROUTING]]>]]> +EXAMPLE]]>]]> ]]>]]> Routing notification]]>]]> Routing notification]]>]]> diff --git a/example/example_backend.c b/example/example_backend.c index 16a8d6bb..084863f9 100644 --- a/example/example_backend.c +++ b/example/example_backend.c @@ -294,14 +294,18 @@ static clixon_plugin_api api = { clixon_plugin_api * clixon_plugin_init(clicon_handle h) { - clicon_debug(1, "%s backend", __FUNCTION__); + struct timeval retention = {0,0}; + clicon_debug(1, "%s backend", __FUNCTION__); /* Example stream initialization: * 1) Register EXAMPLE stream * 2) setup timer for notifications, so something happens on stream * 3) setup stream callbacks for notification to push channel */ - if (stream_register(h, "EXAMPLE", "Example event stream", 1) < 0) + if (clicon_option_exists(h, "CLICON_STREAM_RETENTION")) + retention.tv_sec = clicon_option_int(h, "CLICON_STREAM_RETENTION"); + if (stream_add(h, "EXAMPLE", "Example event stream", 1, + retention.tv_sec?&retention:NULL) < 0) goto done; /* assumes: CLIXON_PUBLISH_STREAMS, eg configure --enable-publish */ diff --git a/example/example_cli.cli b/example/example_cli.cli index c4ccc4d1..1856c8c7 100644 --- a/example/example_cli.cli +++ b/example/example_cli.cli @@ -64,7 +64,7 @@ load("Load configuration from XML file") ("Filename (local file } example("This is a comment") ("Just a random number"), mycallback("myarg"); rpc("ex:fib-route rpc") ("routing instance"), fib_route_rpc("myarg"); -notify("Get notifications from backend"), cli_notify("NETCONF", "1", "text"); -no("Negate") notify("Get notifications from backend"), cli_notify("NETCONF", "0", "xml"); +notify("Get notifications from backend"), cli_notify("EXAMPLE", "1", "text"); +no("Negate") notify("Get notifications from backend"), cli_notify("EXAMPLE", "0", "xml"); lock,cli_lock("candidate"); unlock,cli_unlock("candidate"); \ No newline at end of file diff --git a/lib/clixon/clixon_stream.h b/lib/clixon/clixon_stream.h index 3ceb0dac..98be4fb4 100644 --- a/lib/clixon/clixon_stream.h +++ b/lib/clixon/clixon_stream.h @@ -72,7 +72,9 @@ struct event_stream{ char *es_description; struct stream_subscription *es_subscription; int es_replay_enabled; /* set if replay is enables */ + struct timeval es_retention; /* replay retention - how much to save */ struct stream_replay *es_replay; + }; typedef struct event_stream event_stream_t; @@ -80,7 +82,7 @@ typedef struct event_stream event_stream_t; * Prototypes */ event_stream_t *stream_find(clicon_handle h, const char *name); -int stream_register(clicon_handle h, const char *name, const char *description, int replay_enabled); +int stream_add(clicon_handle h, const char *name, const char *description, int replay_enabled, struct timeval *retention); int stream_delete_all(clicon_handle h); int stream_get_xml(clicon_handle h, int access, cbuf *cb); int stream_timer_setup(int fd, void *arg); diff --git a/lib/src/clixon_stream.c b/lib/src/clixon_stream.c index d216076c..3bf3dea1 100644 --- a/lib/src/clixon_stream.c +++ b/lib/src/clixon_stream.c @@ -104,13 +104,18 @@ stream_find(clicon_handle h, } /*! Add notification event stream - * + * @param[in] h Clicon handle + * @param[in] name Name of stream + * @param[in] description Description of stream + * @param[in] replay_enabled Set if replay possible in stream + * @param[in] retention For replay buffer how much relative to save */ int -stream_register(clicon_handle h, - const char *name, - const char *description, - const int replay_enabled) +stream_add(clicon_handle h, + const char *name, + const char *description, + const int replay_enabled, + struct timeval *retention) { int retval = -1; event_stream_t *es; @@ -131,6 +136,8 @@ stream_register(clicon_handle h, goto done; } es->es_replay_enabled = replay_enabled; + if (retention) + es->es_retention = *retention; clicon_stream_append(h, es); ok: retval = 0; @@ -159,9 +166,9 @@ stream_delete_all(clicon_handle h) while ((ss = es->es_subscription) != NULL) stream_ss_rm(es, ss); while ((r = es->es_replay) != NULL){ + DELQ(r, es->es_replay, struct stream_replay *); if (r->r_xml) xml_free(r->r_xml); - DELQ(r, es->es_replay, struct stream_replay *); free(r); } free(es); @@ -224,20 +231,25 @@ stream_timer_setup(int fd, struct timeval now; struct timeval t; struct timeval t1 = {STREAM_TIMER_TIMEOUT_S, 0}; + struct timeval tret; event_stream_t *es; struct stream_subscription *ss; struct stream_subscription *ss1; + struct stream_replay *r; + struct stream_replay *r1; /* Go thru callbacks and see if any have timed out, if so remove them * Could also be done by a separate timer. */ gettimeofday(&now, NULL); - /* for all event streams, remove subscription if past stop time */ - - - + /* For all event streams: + * 1) Go through subscriptions, if stop-time and its past, remove it + * XXX: but client may not be closed + * 2) Go throughreplay buffer and remove entries with passed retention time + */ if ((es = clicon_stream(h)) != NULL){ do { + /* 1) Go through subscriptions, if stop-time and its past, remove it */ if ((ss = es->es_subscription) != NULL) do { if (timerisset(&ss->ss_stoptime) && timercmp(&ss->ss_stoptime, &now, <)){ @@ -249,6 +261,23 @@ stream_timer_setup(int fd, else ss = NEXTQ(struct stream_subscription *, ss); } while (ss && ss != es->es_subscription); + /* 2) Go throughreplay buffer and remove entries with passed retention time */ + if (timerisset(&es->es_retention) && + (r = es->es_replay) != NULL){ + timersub(&now, &es->es_retention, &tret); + do { + if (timercmp(&r->r_tv, &tret, <)){ + r1 = NEXTQ(struct stream_replay *, r); + DELQ(r, es->es_replay, struct stream_replay *); + if (r->r_xml) + xml_free(r->r_xml); + free(r); + r = r1; + } + else + r = NEXTQ(struct stream_replay *, r); + } while (r && r!=es->es_replay); + } es = NEXTQ(struct event_stream *, es); } while (es && es != clicon_stream(h)); } @@ -550,8 +579,7 @@ stream_replay_notify(clicon_handle h, event_stream_t *es, struct stream_subscription *ss) { - int retval = -1; - + int retval = -1; struct stream_replay *r; /* If is not present, this is not a replay */ diff --git a/test/test_stream.sh b/test/test_stream.sh index 9f7ab600..f1f84bc6 100755 --- a/test/test_stream.sh +++ b/test/test_stream.sh @@ -122,8 +122,14 @@ if false; then # data: 2018-10-21T19:22:16.387228faultEthernet0major new "restconf monitor event ok stream" -expectwait 'curl -s -X GET -H "Accept: text/event-stream" -H "Cache-Control: no-cache" -H "Connection: keep-alive" http://localhost/streams/EXAMPLE' 0 'foo' 2 +expectwait 'curl -s -X GET -H "Accept: text/event-stream" -H "Cache-Control: no-cache" -H "Connection: keep-alive" http://localhost/streams/EXAMPLE' 0 'foo' 5 + +new "restconf monitor event starttime" +NOW=$(date +"%Y-%m-%dT%H%%3A%M%%3A%S") +sleep 10 +expectwait "curl -s -X GET -H \"Accept: text/event-stream\" -H \"Cache-Control: no-cache\" -H \"Connection: keep-alive\" http://localhost/streams/EXAMPLE?start-time=$NOW" 0 'foo' 2 fi + # Restconf stream subscription RFC8040 Sec 6.3 - Nginx nchan solution # Need manual testing new "restconf monitor streams nchan NEEDS manual testing" diff --git a/yang/clixon-config@2018-10-21.yang b/yang/clixon-config@2018-10-21.yang index 6ee83d29..6f7bf38a 100644 --- a/yang/clixon-config@2018-10-21.yang +++ b/yang/clixon-config@2018-10-21.yang @@ -370,11 +370,11 @@ module clixon-config { type string; default "0"; description "If RFC 7895 YANG Module library enabled: - Contains a server-specific identifier representing - the current set of modules and submodules. The - server MUST change the value of this leaf if the - information represented by the 'module' list instances - has changed."; + Contains a server-specific identifier representing + the current set of modules and submodules. The + server MUST change the value of this leaf if the + information represented by the 'module' list instances + has changed."; } leaf CLICON_STREAM_DISCOVERY_RFC5277 { type boolean; @@ -418,5 +418,13 @@ module clixon-config { Note this may be a local/provate URL behind reverse-proxy. If not given, do NOT enable stream publishing using NCHAN."; } + leaf CLICON_STREAM_RETENTION { + type uint32; + default 0; + units s; + description "Retention for stream replay buffers in seconds, ie how much + data to store before dropping. 0 means no retention"; + + } } }