Starting 3.10 release cycle

Moved docs to doc/
2019-02-22 14:45:37 +01:00 · 2019-02-22 14:45:37 +01:00 · 7590395fe3
commit 7590395fe3
parent 701ef1dead
7 changed files with 104 additions and 13 deletions
--- a/doc/DEVELOP.md
+++ b/doc/DEVELOP.md
@ -0,0 +1,121 @@
+# README for Clixon developers
+
+  * [Code documentation](#documentation)
+  * [How to work in git (branching)](#branching)
+  * [How the meta-configure stuff works](#meta-configure)
+  * [How to debug](#debug)
+  * [New release](#new-release)
+
+## Documentation
+How to document the code
+
+```
+/*! This is a small comment on one line
+ *
+ * This is a detailed description
+ * spanning several lines.
+ *
+ * Example usage:
+ * @code
+ *   fn(a, &b);
+ * @endcode
+ *
+ * @param[in] src         This is a description of the first parameter
+ * @param[in,out] dest    This is a description of the second parameter
+ * @retval TRUE           This is a description of the return value
+ * @retval FALSE          This is a description of another return value
+ * @see                   See also this function
+ */
+```
+
+## Branching
+How to work in git (branching)
+
+Try to keep a single master branch always working. Currently testing is made using [Travis CI](https://travis-ci.org/clicon/clixon).
+
+However, releases are made periodically (ca every 3 months) which is more tested.
+
+## How the meta-configure stuff works
+```
+configure.ac --.
+                    |   .------> autoconf* -----> configure
+     [aclocal.m4] --+---+
+                    |   `-----> [autoheader*] --> [config.h.in]
+     [acsite.m4] ---'
+
+                           .-------------> [config.cache]
+     configure* ------------+-------------> config.log
+                            |
+     [config.h.in] -.       v            .-> [config.h] -.
+                    +--> config.status* -+               +--> make*
+     Makefile.in ---'                    `-> Makefile ---'
+```
+Note: remember to run autoheader sometimes (when?)
+And when you do note (https://github.com/olofhagsand/cligen/issues/17) which states that cligen_custom.h should be in quote. 
+
+## Debug
+How to debug
+
+### Configure in debug mode
+```
+   CFLAGS="-g -Wall" INSTALLFLAGS="" ./configure
+```
+
+### Make your own simplified yang and configuration file.
+```
+cat <<EOF > /tmp/my.yang
+module mymodule{
+   container x {
+    list y {
+      key "a";
+      leaf a {
+        type string;
+      }
+    }
+  }
+}
+EOF
+cat <<EOF > /tmp/myconf.xml
+<config>
+  <CLICON_CONFIGFILE>/tmp/myconf.xml</CLICON_CONFIGFILE>
+  <CLICON_YANG_DIR>/usr/local/share/example/yang</CLICON_YANG_DIR>
+  <CLICON_YANG_MODULE_MAIN>example</CLICON_YANG_MODULE_MAIN>
+  <CLICON_SOCK>/usr/local/var/example/example.sock</CLICON_SOCK>
+  <CLICON_BACKEND_PIDFILE>/usr/local/var/example/example.pidfile</CLICON_BACKEND_PIDFILE>
+  <CLICON_XMLDB_DIR>/usr/local/var/example</CLICON_XMLDB_DIR>
+  <CLICON_XMLDB_PLUGIN>/usr/local/lib/xmldb/text.so</CLICON_XMLDB_PLUGIN>
+</config>
+EOF
+ sudo clixon_backend -F -s init -f /tmp/myconf.xml -y /tmp/my.yang
+ ```
+
+### Run valgrind and callgrind
+ ```
+  valgrind --leak-check=full --show-leak-kinds=all clixon_netconf -qf /tmp/myconf.xml -y /tmp/my.yang
+  valgrind --tool=callgrind clixon_netconf -qf /tmp/myconf.xml -y /tmp/my.yang
+  sudo kcachegrind
+ ```
+
+## New release
+What to think about when doing a new release.
+* Ensure all tests run OK
+* New yang/clicon/clixon-config@XXX.yang revision?
+* In configure.ac, for minor releases change CLIXON_VERSION in configure.ac to eg: (minor should have been bumped):
+```
+  CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}\""
+```
+* For patch releases change CLIXON_VERSION_PATCH
+* Run autoconf
+* Git stuff:
+```
+  git tag -a <version"
+  git push origin <version>
+```
+
+After release:
+* Bump minor version and add a "PRE":
+```
+  CLIXON_VERSION_MINOR="10" ++
+  CLIXON_VERSION="\"${CLIXON_VERSION_MAJOR}.${CLIXON_VERSION_MINOR}.${CLIXON_VERSION_PATCH}.PRE\""
+```
+* Run autoconf
--- a/doc/ROADMAP.md
+++ b/doc/ROADMAP.md
@ -0,0 +1,48 @@
+# Clixon roadmap
+
+## High prio
+- Special handling of the initial startup transaction to avoid exit at startup
+  - Possibly - draft-wu-netconf-restconf-factory-restore-03
+  - See (startup.md)
+- Handle revisions to data model.
+  - Possibly draft-wang-netmod-module-revision-management-01
+  - See (startup.md)
+- (DONE) NACM (RFC 8341) 
+  - NACM support for create, read, update, delete operations 
+  - ACM support for specifying a module name other than '*' 
+- (DONE)XML [Namespace handling](https://github.com/clicon/clixon/issues/49) (DONE)
+
+## Medium prio:
+- Support a plugin callback that is invoked when copy-config is called.
+- Preserve CLI command history across sessions. The up/down arrows
+- (DONE)Support for XML regex's.
+  - Currently Posix extended regular expressions
+- (DONE) Input validation on custom RPCs/
+  - [Sanity checks](https://github.com/clicon/clixon/issues/47)
+  
+## Low prio:
+- Provide a client library to access netconf APIs provided by system services.
+  - Netconf backend (Clixon acts as netconf controller)
+- Support for restconf call-home (RFC 8071)
+
+Not prioritized:
+- Support for restconf PATCH method
+- NETCONF
+  - Support for additional Netconf [edit-config modes](https://github.com/clicon/clixon/issues/53)
+  - Netconf [framing](https://github.com/clicon/clixon/issues/50)
+  - [Child ordering](https://github.com/clicon/clixon/issues/22)
+- Restconf
+  - Query parameters
+- Streams (netconf and restconf)
+  - Extend native stream mode with external persistent timeseries database, eg influxdb.
+- (DONE)Jenkins/Travis CI/CD and webhooks
+- YANG
+  - RFC 6022 [NETCONF monitoring](https://github.com/clicon/clixon/issues/39)
+  - Deviation, min/max-elements, action, unique
+- Containers
+  - (DONE)[Docker improvements](https://github.com/clicon/clixon/issues/44)
+  - Kubernetes Helm chart definition
+- [gRPC](https://github.com/clicon/clixon/issues/43)
+
+
+
--- a/doc/startup.md
+++ b/doc/startup.md
@ -0,0 +1,62 @@
+# New Clixon Startup functionality
+
+## Objectives
+When Clixon 3.9 backend starts, it assumes a well-formed initial XML
+configuration which it parses and validates. Depending on starting
+mode (-s command-line) this is the "startup" or "running"
+configuration.
+
+If this initial configuration fails, clixon backend exits. This has
+the consequence that an operator cannot manage the system unless with
+out-of-band mechanisms.
+
+## Objectives
+This document describes a new startup mechanism with the following goals:
+* An operator should be notified of the startup status
+* The backend should remain up in case of errors but may enter a "failsafe" mode.
+* XML syntax errors should be detected and reported
+* Yang module info is added to (startup) datastore database
+* Yang module mismatch should be detected and reported
+* Validation failures should be detected and reported, specifically of mismatching modules.
+
+## Proposal
+
+A new user callback is introduced:
+```
+  int startup-cb(h, status, module-state-diff, *valid)
+```
+which is called once at startup to report startup state to application:
+- status is one of: OK, INVALID and ERROR.
+- module-state-diff contains a list of RFC7895 differences between the yang modules running in the system, and the ones in the startup config.
+- valid is a return value that if set to 0 forces the status to INVALID (if OK on entry).
+
+A new read-only datastore is introduced:
+```
+  CLICON_XMLDB_FAILSAFE If set, a failsafe read-only datastore is expected,
+                         in CLICON_XMLDB_DIR, called failsafe_db
+```
+
+Datastore databases are optionally extended with modules state according to
+RFC7895. A new config option is introduced to control this:
+```
+  CLICON_XMLDB_MODSTATE If set, tag datastores with RFC 7895 YANG Module Library 
+ info. When loaded at startup, a check is made if the system yang modules match
+```
+
+Proposed algoritm:
+0. Backend starts with a set of yang module revisions as of RFC7895.
+1. Parse startup XML (or JSON)
+2. If syntax failure, call startup-cb(ERROR), copy failsafe db to candidate and commit. Done
+3. Check yang module versions between backend and init config XML. (msdiff)
+4. Validate startup db. (valid)
+5. If valid fails, call startup-cb(Invalid, msdiff), keep startup in candidate and commit failsafe db. Done.
+6. Call startup-cb(OK, msdiff) and commit. 
+
+Note:
+
+1. If done in step 2) the failsafe db is in both candidate and running. The operator need to repair the XML file before reloading.
+2. If done in step 5) The operator has the non-valid database in candidate and can edit it, and when ready can commit it. During this time, the failsafe db is running.
+3. If done in steps 5 and 6, the module-state-diff contains the (potential) differences in the modules-state diff.
+
+## Thanks
+Thanks matt smith and dave cornejo for input