3.8 docs

CHANGELOG.md

@@ -1,50 +1,46 @@
 # Clixon Changelog
 
-## 3.8.0 (Upcoming)
+## 3.8.0 (Expected: Nov 4)
 
 ### Major New features
 * YANG Features
   * Yang 1.1 feature and if-feature according to RFC 7950 7.20.1 and 7.20.2.
   * See https://github.com/clicon/clixon/issues/41
-  * Features are declared via CLICON_FEATURE in the configuration file. Examples showing enabling (1) a specific feature; (2) all features in a module; (3) all features in all modules:
+  * Features are declared via CLICON_FEATURE in the configuration file. Example below shows enabling (1) a specific feature; (2) all features in a module; (3) all features in all modules:
-```
+   ```
       <CLICON_FEATURE>ietf-routing:router-id</CLICON_FEATURE>
       <CLICON_FEATURE>ietf-routing:*</CLICON_FEATURE>
       <CLICON_FEATURE>*:*</CLICON_FEATURE>
-```
+   ```
   * logical combination of features not implemented, eg if-feature "not foo or bar and baz";
   * ietf-netconf yang module added with candidate, validate, startup and xpath features enabled.
-* YANG Module Library support
+* YANG module library 
-  * According to RFC 7895 and implemented by ietf-yang-library.yang
+  * YANG modules according to RFC 7895 and implemented by ietf-yang-library.yang
-  * Changed Netconf hello to single capabilty urn:ietf:params:netconf:capability:yang-library:1.0 according to YANG 1.1 RFC7950 Sec 5.6.4.
+  * Enabled by configuration option CLICON_MODULE_LIBRARY_RFC7895 - which is enabled by default
-  * Set by option: CLICON_MODULE_LIBRARY_RFC7895 - enabled by default
+  * RFC 7895 defines a module-set-id. COnfigure option CLICON_MODULE_SET_ID is set and changed when modules change.
-  * Option CLICON_MODULE_SET_ID is set and changed when modules change. 
+  * Notification yang-library-change not supported
-  * Notification not supported
 * Yang 1.1 notification support (RFC 7950: Sec 7.16)
-* Restconf stream notification support - two variants.
-  * Both a "native" stream support and one using nginx/nchan pub/sub.
-  * See (apps/restconf/README.md) for details.
 * New event streams implementation
   * See clicon_stream.[ch] for details
   * Added stream discovery according to RFC 5277 for netconf and RFC 8040 for restconf
     * Enabled by CLICON_STREAM_DISCOVERY_RFC5277 and CLICON_STREAM_DISCOVERY_RFC8040.
+* Restconf stream notification support according to RFC8040
+  * See (apps/restconf/README.md) for more details.
   * Set access/subscribe base URL with: CLICON_STREAM_URL (default "https://localhost") and CLICON_STREAM_PATH (default "streams")
     * Example: new stream "foo" will get access URL: https://localhost/streams/foo
-  * Optional pub/sub support enabled by ./configure --enable-publish
+  * Alternative variant using pub/sub support enabled by ./configure --enable-publish
     * Set publish URL base with: CLICON_STREAM_PUB (default http://localhost/pub)
-    * Example: new stream "foo" will get pub URL: https://localhost/pub/foo
+    * Example: new stream "foo" will publish event streams on URL: https://localhost/pub/foo
-* Stream replay support
   * RFC8040 Restconf replay support: start-time and stop-time query parameter support
     * This only applies to "native" restconf stream support, Nchan mode has different replay functionality
-  * RFC5277 Netconf replay support using <startTime> and
+  * RFC5277 Netconf replay supported
   * Replay support is only in-memory and not persistent. External time-series DB could be added.
 
 ### API changes on existing features (you may need to change your code)
-* clixon-config YAML file has new revision: 2018-10-21.
 * Netconf hello capability updated to YANG 1.1 RFC7950 Sec 5.6.4
   * Added urn:ietf:params:netconf:capability:yang-library:1.0
   * Thanks @SCadilhac for helping out, see https://github.com/clicon/clixon/issues/39
-* Major rewrite of event streams
+* Major rewrite of event streams (as described above)
   * If you used old event callbacks API, you need to switch to the streams API
     * See clixon_stream.[ch]
   * Old streams API which needs to be removed include:
@@ -52,47 +48,32 @@
     * subscription_add() --> stream_register()
     * backend_notify() and backend_notify_xml() - use stream_notify() instead
   * Example uses "NETCONF" stream instead of "ROUTING"
-* clixon_restconf and clixon_netconf now take -D <level> as command-line option instead of just -D
+* clixon_restconf and clixon_netconf changed to take -D `<level>` as command-line option instead of just -D (without debig level)
   * This aligns to clixon_cli and clixon_backend
 * Application command option -S to clixon_netconf is obsolete. Use `clixon_netconf -l s` instead.
-* Unified log handling for all clicon applications using -l e|o|s|f<file>.
+* Unified log handling for all clicon applications using command-line option: `-l e|o|s|f<file>`.
   * The options stand for e:stderr, o:stdout, s: syslog, f:file
   * Added file logging (`-l f` or `-l f<file>`) for cases where neither syslog nor stderr is useful.
 * Comply to RFC 8040 3.5.3.1 rule: api-identifier = [module-name ":"] identifier
   * The "module-name" was a no-op before.
-  * This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/XXXX:modules-state 
+  * This means that there was no difference between eg: GET /restconf/data/ietf-yang-library:modules-state and GET /restconf/data/foobar:modules-state 
 * Generilized top-level yang parsing functions
-  * All yang modules are stored in the clicon_dbspec_yang() option.
-    * Except clixon-config module due to bug reported below
   * Clarified semantics of main yang module:
-    * -y option to commands MUST specify filename
+    * Command-line option -y MUST specify a filename
-    * CLICON_YANG_MODULE_MAIN MUST specify a module
+    * Configure option CLICON_YANG_MODULE_MAIN MUST specify a module name
     * yang_parse() changed to take either filename or module name and revision. 
-  * Removed clicon_dbspec_name[_set]().
+  * Removed clicon_dbspec_name() and clicon_dbspec_name_set().
-  * Replace code for initializing the main yang module
+  * Replace calls to yang_spec_main() with yang_spec_parse_module(). See for
-    * Replace yang_spec_main() with yang_spec_parse_module() as follows:
+    example backend_main() and others if you need details.
-```
-    /* old code */
-    if ((yspec = yang_spec_main(ch)) == NULL)
-	goto done;
-    /* new code */
-    if ((yspec = yspec_new()) == NULL)
-	goto done;
-    clicon_dbspec_yang_set(h, yspec);
-    if (yang_spec_parse_module(h, clicon_yang_module_main(h),
-               clicon_yang_dir(h), clicon_yang_module_revision(h), yspec) < 0)
-	goto done;
-```    
 
 ### Minor changes
+* clixon-config YAML file has new revision: 2018-10-21.
 * Allow new lines in CLI prompts 
 * uri_percent_encode() and xml_chardata_encode() changed to use stdarg parameters
-* Added CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option).
+* Added Configure option CLIXON_DEFAULT_CONFIG=/usr/local/etc/clixon.xml as option and in example (so you dont need to provide -f command-line option).
-* Yang 1.1 action syntax added (but function is not supported)
 * New function: clicon_conf_xml() returns configuration tree
 * Obsoleted COMPAT_CLIV and COMPAT_XSL that were optional in 3.7
-* Added timeout option -t for clixon_netconf - quit after max time.
+* Added command-line option `-t <timeout>` for clixon_netconf - quit after max time.
-* Added -l option for clixon_backend for directing syslog to stderr or stdout if running in foreground
 
 ### Corrected Bugs
 * Single quotes for XML attributes https://github.com/clicon/clixon/issues/51
@@ -104,7 +85,7 @@
 * Set dir /www-data with www-data as owner, see https://github.com/clicon/clixon/issues/37
 	
 ### Known issues
-* netconf rpc input is not sanity checked for wrong symbols (just ignored).
+* Netconf RPC input is not sanity checked for wrong symbols (just ignored).
 * Yang sub-command order and cardinality not checked.
 * Top-level Yang symbol cannot be called "config" in any imported yang file.
   * datastore uses "config" as reserved keyword for storing any XML whoich collides with code for detecting Yang sanity.

README.md

@@ -1,8 +1,8 @@
 # Clixon
 
-Clixon is an automatic configuration manager where you generate
+Clixon is a YANG-based configuration manager, with interactive CLI,
-interactive CLI, NETCONF, RESTCONF and embedded databases with
+NETCONF and RESTCONF interfaces, an embedded database and transaction
-transaction support from a YANG specification.
+support.
 
   * [Background](#background)
   * [Frequently asked questions](doc/FAQ.md)
@@ -107,23 +107,21 @@ specification for handling XML configuration data. The YANG spec is
 used to generate an interactive CLI, netconf and restconf clients. It
 also manages an XML datastore.
 
-Clixon mainly follows [YANG 1.0 RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt) with some exceptions:
+Clixon follows:
-- conformance: deviation
+- [YANG 1.0 RFC 6020](https://www.rfc-editor.org/rfc/rfc6020.txt)
-- list features: min/max-elements, unique
+- [YANG 1.1 RFC 7950](https://www.rfc-editor.org/rfc/rfc7950.txt).
-- action statements
+- [RFC 7895: YANG module library](http://www.rfc-base.org/txt/rfc-7895.txt)
 
-The aim is also to cover new features in YANG 1.1 [YANG RFC 7950](https://www.rfc-editor.org/rfc/rfc7950.txt)
+However, the following YANG syntax modules are not implemented:
-
+`deviation`, `min/max-elements`, `unique`, and `action`.
-Clixon has its own XML library designed for performance.
 
 Netconf
 =======
 Clixon implements the following NETCONF proposals or standards:
-- [NETCONF Configuration Protocol](http://www.rfc-base.org/txt/rfc-4741.txt)
+- [RFC 6241: NETCONF Configuration Protocol](http://www.rfc-base.org/txt/rfc-6241.txt)
-- [Using the NETCONF Configuration Protocol over Secure Shell (SSH)](http://www.rfc-base.org/txt/rfc-4742.txt)
+- [RFC 6242: Using the NETCONF Configuration Protocol over Secure Shell (SSH)](http://www.rfc-base.org/txt/rfc-6242.txt)
-- [NETCONF Event Notifications](http://www.rfc-base.org/txt/rfc-5277.txt)
+- [RFC 5277: NETCONF Event Notifications](http://www.rfc-base.org/txt/rfc-5277.txt)
-
+- [RFC 8341: Network Configuration Access Control Model](http://www.rfc-base.org/txt/rfc-8341.txt)
-Some updates are being made to RFC 6241 and RFC 6242. 
 
 Clixon does not yet support the following netconf features:
 
@@ -154,8 +152,6 @@ The Clixon datastore is a stand-alone XML based datastore. The idea is
 to be able to use different datastores backends with the same
 API.
 
-Update: There used to be a key-value plugin based on qdbm but isnow obsoleted. Only a text datastore is implemented.
-
 The datastore is primarily designed to be used by Clixon but can be used
 separately.
 
@@ -163,7 +159,6 @@ See [more detailed instructions](datastore/README.md).
 
 Auth
 ====
-
 Authentication is managed outside Clixon using SSH, SSL, Oauth2, etc.
 
 For CLI, login is typically made via SSH. For netconf, SSH netconf

doc/FAQ.md

@@ -2,17 +2,18 @@
 
 ## What is Clixon?
 
-Clixon is a configuration management tool including a generated CLI ,
+Clixon is a YANG-based configuration manager, with interactive CLI,
-Yang parser, netconf and restconf interface and an embedded databases.
+NETCONF and RESTCONF interfaces, an embedded database and transaction
+support.
 
 ## Why should I use Clixon?
 
-If you want an easy-to-use configuration frontend based on yang with an
+If you want an easy-to-use configuration toolkit based on yang with an
 open-source license.  Typically for embedded devices requiring a
 config interface such as routers and switches. 
 
 ## What license is available?
-CLIXON is dual license. Either Apache License, Version 2.0 or GNU
+Clixon is dual license. Either Apache License, Version 2.0 or GNU
 General Public License Version 2. 
 
 ## Is Clixon extendible?
@@ -41,6 +42,16 @@ The example:
 	 sudo make install
 ```
 
+## How do you run Clixon example commands?
+
+- Start a backend server: `clixon_backend -Ff /usr/local/etc/example.xml`
+- Start a cli session: `clixon_cli -f /usr/local/etc/example.xml`
+- Start a netconf session: `clixon_netconf -f /usr/local/etc/example.xml`
+- Start a restconf daemon: `sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data`
+- Send a restconf command: `curl -G http://127.0.0.1/restconf/data`
+
+More info in the [example](../example) directory.
+
 ## Do I need to setup anything? (IMPORTANT)
 
 The config demon requires a valid group to create a server UNIX domain socket.
@@ -63,18 +74,11 @@ clicon:x:1001:<user>,www-data
 
 ## What about reference documentation?
 Clixon uses Doxygen for reference documentation.
-Build using 'make doc' and aim your browser at doc/html/index.html or
+Build using 'make doc' and aim your browser at doc/html/index.html.
-use the web resource: http://clicon.org/ref/index.html
-
-## How do you run the example?
-- Start a backend server: 'clixon_backend -Ff /usr/local/etc/example.xml'
-- Start a cli session: clixon_cli -f /usr/local/etc/example.xml
-- Start a netconf session: clixon_netconf -f /usr/local/etc/example.xml
 
 ## How is configuration data stored?
-Configuration data is stored in an XML datastore. The default is a
+Configuration data is stored in an XML datastore. In the example the
-text-based datastore. In the example the datastore are regular files found in
+datastore are regular files found in /usr/local/var/example/.
-/usr/local/var/example/.
 
 ## What is validate and commit?
 Clixon follows netconf in its validate and commit semantics.
@@ -93,7 +97,7 @@ configuration file is /usr/local/etc/clixon.xml. The example
 configuration file is installed at /usr/local/etc/example.xml. The
 YANG specification for the configuration file is clixon-config.yang.
 
-You can change where CLixon looks for the configuration FILE as follows:
+You can change where Clixon looks for the configuration FILE as follows:
   - Provide -f FILE option when starting a program (eg clixon_backend -f FILE)
   - Provide --with-configfile=FILE when configuring
   - Provide --with-sysconfig=<dir> when configuring, then FILE is <dir>/clixon.xml
@@ -130,7 +134,9 @@ Look in the example documentation for more info.
 
 As an alternative to cli configuration, you can use netconf. Easiest is to just pipe netconf commands to the clixon_netconf application.
 Example:
+```
 	echo "<rpc><get-config><source><candidate/></source><configuration/></get-config></rpc>]]>]]>" | clixon_netconf -f /usr/local/etc/example.xml
+```
 
 However, more useful is to run clixon_netconf as an SSH
 subsystem. Register the subsystem in /etc/sshd_config:
@@ -155,7 +161,6 @@ For example, using nginx, install, and edit config file: /etc/nginx/sites-availa
 server {
   ...
   location /restconf {
-    root /usr/share/nginx/html/restconf;
     fastcgi_pass unix:/www-data/fastcgi_restconf.sock;
     include fastcgi_params;
   }
@@ -166,8 +171,6 @@ Start nginx daemon
 sudo /etc/init.d/nginx start
 ```
 
-Read more in the restconf docs.
-
 Example:
 ```
    curl -G http://127.0.0.1/restconf/data/interfaces/interface/name=eth9/type
@@ -177,14 +180,24 @@ Example:
      }
    ]
 ```
+Read more in the (restconf)[../apps/restconf] docs.
 
-## How do I use notifications?
+
+## Does Clixon support event streams? 
+
+Yes, Clixon supports event notification streams in the CLI, Netconf and Restconf API:s.
 
 The example has a prebuilt notification stream called "EXAMPLE" that triggers every 5s.
 You enable the notification via the CLI:
 ```
 cli> notify 
 cli>
+event-class fault;
+reportingEntity {
+    card Ethernet0;
+}
+severity major;
+...
 ```
 or via NETCONF:
 ```

example/README.md

@@ -41,7 +41,7 @@ Send netconf command:
 ```
 Start clixon restconf daemon
 ```
-> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data
+    sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/example.xml " -s /bin/sh www-data
 ```
 Send restconf command
 ```

example/example_cli.cli

@@ -64,7 +64,7 @@ load("Load configuration from XML file") <filename:string>("Filename (local file
 }
 example("This is a comment") <var:int32>("Just a random number"), mycallback("myarg");
 rpc("ex:fib-route rpc") <instance:string>("routing instance"), fib_route_rpc("myarg");
-notify("Get notifications from backend"), cli_notify("NETCONF", "1", "text");
+notify("Get notifications from backend"), cli_notify("EXAMPLE", "1", "text");
-no("Negate") notify("Get notifications from backend"), cli_notify("NETCONF", "0", "xml");
+no("Negate") notify("Get notifications from backend"), cli_notify("EXAMPLE", "0", "xml");
 lock,cli_lock("candidate");
 unlock,cli_unlock("candidate");

yang/clixon-config@2018-10-21.yang

@@ -370,11 +370,11 @@ module clixon-config {
 	    type string;
 	    default "0";
 	    description "If RFC 7895 YANG Module library enabled:
-                          Contains a server-specific identifier representing
+                         Contains a server-specific identifier representing
-                          the current set of modules and submodules.  The
+                         the current set of modules and submodules.  The
-                          server MUST change the value of this leaf if the
+                         server MUST change the value of this leaf if the
-                          information represented by the 'module' list instances
+                         information represented by the 'module' list instances
-                          has changed.";
+                         has changed.";
 	}
 	leaf CLICON_STREAM_DISCOVERY_RFC5277 {
 	    type boolean;