xmldb_init -> xmldb_create

README.md

@@ -6,39 +6,44 @@ databases with transaction support.
 
 Presentations and tutorial is found on the [CLICON project page](http://www.clicon.org)
 
-## Installation
+## 1. Installation
 
 A typical installation is as follows:
-
+```
-    > configure	       	        # Configure clixon to platform
+     configure	       	        # Configure clixon to platform
-    > make                      # Compile
+     make                      # Compile
-    > sudo make install         # Install libs, binaries, and config-files
+     sudo make install         # Install libs, binaries, and config-files
-    > sudo make install-include # Install include files (for compiling)
+     sudo make install-include # Install include files (for compiling)
-
+```
 One example applications is provided, a IETF IP YANG datamodel with generated CLI and configuration interface. 
 
-## More info
+## 2. Documentation
 
-- [Datastore](datastore).
+- [Frequently asked questions](http://www.clicon.org/FAQ.html)
-- [Restconf](apps/restconf).
+- [Reference manual(http://www.clicon.org/doxygen/index.html) (may not be 100%% synched)
-- [Netconf](apps/netconf).
 
-## Dependencies
+## 3. Dependencies
 
-[CLIgen](http://www.cligen.se) is required for building CLIXON. If you need 
+Clixon is dependend on the following packages
+- [CLIgen](http://www.cligen.se) is required for building CLIXON. If you need 
 to build and install CLIgen: 
-
+```
     git clone https://github.com/olofhagsand/cligen.git
     cd cligen; configure; make; make install
+```
+- Yacc/bison
+- Lex/Flex
+- Fcgi (if restconf is enabled)
+- Qdbm key-value store (if keyvalue datastore is enabled)
 
-## Licenses
+## 4. Licenses
 
 CLIXON is dual license. Either Apache License, Version 2.0 or GNU
 General Public License Version 2. You choose.
 
 See LICENSE.md for license, CHANGELOG for recent changes.
 
-## Related
+## 5. History
 
 CLIXON is a fork of CLICON where legacy key specification has been
 replaced completely by YANG. This means that legacy CLICON
@@ -46,5 +51,204 @@ applications such as CLICON/ROST does not run on CLIXON.
 
 Clixon origins from work at [KTH](http://www.csc.kth.se/~olofh/10G_OSR)
 
+## 6. Clixon Datastore
+The Clixon datastore is a stand-alone XML based datastore used by
+Clixon. The idea is to be able to use different datastores. There is
+currently a key-value plugin based on qdbm and a plain text-file
+datastore.
 
+The datastore is primarily designed to be used by Clixon but can be used
+separately.  
+
+A datastore is a dynamic plugin that is loaded at runtime with a
+well-defined API. This means it is possible to create your own
+datastore and plug it in a Clixon backend at runtime. 
+
+### The functional API
+```
+int xmldb_plugin_load(clicon_handle h, char *filename);
+int xmldb_plugin_unload(clicon_handle h);
+int xmldb_connect(clicon_handle h);
+int xmldb_disconnect(clicon_handle h);
+int xmldb_getopt(clicon_handle h, char *optname, void **value);
+int xmldb_setopt(clicon_handle h, char *optname, void *value);
+int xmldb_get(clicon_handle h, char *db, char *xpath,
+	      cxobj **xtop, cxobj ***xvec, size_t *xlen);
+int xmldb_put(clicon_handle h, char *db, enum operation_type op, 
+	      char *api_path,  cxobj *xt);
+int xmldb_copy(clicon_handle h, char *from, char *to);
+int xmldb_lock(clicon_handle h, char *db, int pid);
+int xmldb_unlock(clicon_handle h, char *db);
+int xmldb_unlock_all(clicon_handle h, int pid);
+int xmldb_islocked(clicon_handle h, char *db);
+int xmldb_exists(clicon_handle h, char *db);
+int xmldb_delete(clicon_handle h, char *db);
+int xmldb_create(clicon_handle h, char *db);
+```
+
+### Using the API
+
+To use the API, a client needs the following:
+- A clicon handle. 
+- A datastore plugin, such as a text.so or keyvalue.so. These are normally built and installed at Clixon make.
+- A directory where to store databases
+- A yang specification. This needs to be parsed using the Clixon yang_parse() method.
+
+A client calling the API needs to (1)load a plugin and (2)connect to a
+datastore. You can connect to several datastores, even concurrently,
+but in practice in Clixon, you connect to a single store. 
+
+After connecting to a datastore, you can create and modify databases
+within the datastore, and set and get options of the datastore itself.
+
+When done, you disconnect from the datastore and unload the plugin.
+
+Within a datastore, the following four databases may exist:
+- running
+- candidate
+- startup
+- tmp
+
+Initially, a database does not exist but is created by
+xmldb_create(). It is deleted by xmldb_delete(). You may check for
+existence with xmldb_exists(). You need to create a database before
+you can perform any data access on it.
+
+You may lock a database for exclusive modification according to
+Netconf semantics. You may also unlock a single dabase, unlock all frm
+a specific session.
+
+You can read a database with xmldb_get() and modify a database with
+xmldb_put(), and xmldb_copy().
+
+A typical datastore session can be as follows, see the source code of
+datastore_client.c for a more elaborate example.
+
+```
+  h = clicon_handle_init();
+  xmldb_plugin_load(h, plugin);
+  xmldb_connect(h);
+  xmldb_setopt(h, "dbdir", dbdir);
+  xmldb_setopt(h, "yangspec", yspec);
+  /* From here databases in the datastore may be accessed */
+  xmldb_create(h, "candidate");
+  xmldb_copy(h, "running", "candidate");
+  xmldb_lock(h, "candidate", 7878);
+  xmldb_put(h, "candidate", OP_CREATE, "/interfaces/interface=eth0", xml);
+  xmldb_unlock(h, "candidate");
+  xmldb_get(h, "candidate", "/", &xml, &xvec, &xlen);
+  xmldb_disconnect(h)
+  xmdlb_plugin_unload(h);
+```
+
+## 7. YANG
+
+Clixon implements YANG RFC 6020. Clixon generates an interactive CLI
+for YANG specifications. It also provides Restconf and Netconf clients.
+
+Clixon YANG currently does not provide the following support:
+- type object-references
+- if-feature
+- unique
+- rpc
+
+## 8. Netconf
+
+Clixon Netconf implements the following NETCONF standards:
+- RFC 4741 (NETCONF Configuration Protocol)
+- RFC 4742 (Using the NETCONF Configuration Protocol over Secure SHell (SSH))
+- RFC 5277 (NETCONF Event Notifications)
+
+It needs to be updated to RFC6241 and RFC 6242. 
+
+Clixon NETCONF currently does not support the following Netconf features:
+
+- :url capability
+- copy-config source config
+- edit-config testopts 
+- edit-config erropts
+- edit-config config-text
+
+## 9. Restconf
+
+### Features
+
+Clixon restconf is a daemon based on FASTCGI. Instructions are available to
+run with NGINX. 
+The implementatation supports plain OPTIONS, HEAD, GET, POST, PUT, PATCH, DELETE.
+and is based on draft-ietf-netconf-restconf-13. 
+There is currently (2017) a RFC 8040, many of those features are _not_ implemented,
+including:
+- query parameters (section 4.9)
+- notifications (sec 6)
+- only rudimentary error reporting exists (sec 7)
+
+### Installation using Nginx
+
+Define nginx config file/etc/nginx/sites-available/default
+```
+server {
+  ...
+  location /restconf {
+    root /usr/share/nginx/html/restconf;
+    fastcgi_pass unix:/www-data/fastcgi_restconf.sock;
+    include fastcgi_params;
+  }
+}
+```
+Start nginx daemon
+```
+sudo /etc/init.d nginx start
+```
+
+Start clixon restconf daemon
+```
+olof@vandal> sudo su -c "/www-data/clixon_restconf -f /usr/local/etc/routing.conf " -s /bin/sh www-data
+```
+
+Make restconf calls with curl
+```
+olof@vandal> curl -G http://127.0.0.1/restconf/data/interfaces
+[
+  {
+    "interfaces": {
+      "interface":[
+        {
+          "name": "eth0",
+          "type": "eth",
+          "enabled": "true",
+          "name": "eth9",
+          "type": "eth",
+          "enabled": "true"
+         }
+      ]
+    }
+  }
+]
+olof@vandal> curl -G http://127.0.0.1/restconf/data/interfaces/interface/name=eth9/type
+[
+  {
+    "type": "eth" 
+  }
+]
+
+curl -sX POST -d '{"clicon":{"interfaces":{"interface":{"name":"eth1","type":"eth","enabled":"true"}}}}' http://localhost/restconf/data
+```
+
+### Debugging
+
+Start the restconf fastcgi program with debug flag:
+```
+sudo su -c "/www-data/clixon_restconf -Df /usr/local/etc/routing.conf" -s /bin/sh www-
+data
+```
+Look at syslog:
+```
+tail -f /var/log/syslog | grep clixon_restconf
+```
+
+Send command:
+```
+curl -G http://127.0.0.1/restconf/data/*
+```
 

apps/backend/backend_client.c

@@ -632,7 +632,7 @@ from_client_delete_config(clicon_handle h,
 		"</rpc-error></rpc-reply>", clicon_err_reason);
 	goto ok;
     }
-    if (xmldb_init(h, target) < 0){
+    if (xmldb_create(h, target) < 0){
 	cprintf(cbret, "<rpc-reply><rpc-error>"
 		"<error-tag>operation-failed</error-tag>"
 		"<error-type>protocol</error-type>"

apps/backend/backend_main.c

@@ -159,7 +159,7 @@ db_reset(clicon_handle h,
 {
     if (xmldb_delete(h, db) != 0 && errno != ENOENT) 
 	return -1;
-    if (xmldb_init(h, db) < 0)
+    if (xmldb_create(h, db) < 0)
 	return -1;
     return 0;
 }
@@ -181,7 +181,7 @@ rundb_main(clicon_handle h,
     cxobj     *xt = NULL;
     cxobj     *xn;
 
-    if (xmldb_init(h, "tmp") < 0)
+    if (xmldb_create(h, "tmp") < 0)
 	goto done;
     if (xmldb_copy(h, "running", "tmp") < 0){
 	clicon_err(OE_UNIX, errno, "file copy");
@@ -538,7 +538,7 @@ main(int argc, char **argv)
 	else
 	    if (db_reset(h, "running") < 0)
 		goto done;
-	if (xmldb_init(h, "candidate") < 0)
+	if (xmldb_create(h, "candidate") < 0)
 	    goto done;
 	if (xmldb_copy(h, "running", "candidate") < 0)
 	    goto done;
@@ -562,7 +562,7 @@ main(int argc, char **argv)
     }
     /* If candidate does not exist, create it from running */
     if (xmldb_exists(h, "candidate") != 1){
-	if (xmldb_init(h, "candidate") < 0)
+	if (xmldb_create(h, "candidate") < 0)
 	    goto done;
 	if (xmldb_copy(h, "running", "candidate") < 0)
 	    goto done;

datastore/README.md

@@ -9,8 +9,59 @@ The datastore is primarily designed to be used by Clixon but can be used
 separately.  See datastore_client.c for an example of how to use a
 datastore plugin for other applications
 
-Can we equate a file that does not exist with an empty file?
+## The functional API
-Or is empty file same as <config/>?
+```
-Three states:
+int xmldb_plugin_load(clicon_handle h, char *filename);
-NULL <---> "" <---> "<config/>"
+int xmldb_plugin_unload(clicon_handle h);
-which are valid?
+int xmldb_connect(clicon_handle h);
+int xmldb_disconnect(clicon_handle h);
+int xmldb_getopt(clicon_handle h, char *optname, void **value);
+int xmldb_setopt(clicon_handle h, char *optname, void *value);
+int xmldb_get(clicon_handle h, char *db, char *xpath,
+	      cxobj **xtop, cxobj ***xvec, size_t *xlen);
+int xmldb_put(clicon_handle h, char *db, enum operation_type op, 
+	      char *api_path,  cxobj *xt);
+int xmldb_copy(clicon_handle h, char *from, char *to);
+int xmldb_lock(clicon_handle h, char *db, int pid);
+int xmldb_unlock(clicon_handle h, char *db);
+int xmldb_unlock_all(clicon_handle h, int pid);
+int xmldb_islocked(clicon_handle h, char *db);
+int xmldb_exists(clicon_handle h, char *db);
+int xmldb_delete(clicon_handle h, char *db);
+int xmldb_init(clicon_handle h, char *db);
+```
+
+## Using the API
+
+This section described how the API is used. Please refer to datastore/datastore_client.c for an example of an actual implementation.
+
+### Prerequisities
+To use the API, a client needs the following:
+- A clicon handle. 
+- A datastore plugin, such as a text.so or keyvalue.so. These are normally built and installed at Clixon make.
+- A directory where to store the datastores
+- A yang specification. This needs to be parsed using the Clixon yang_parse() method.
+
+### Dynamics
+
+A client calling the API needs to load a plugin and connect to a
+datastore. In principle, you cannot connect to several datastores,
+even concurrently, but in practice in Clixon, you connect to a single
+store.
+
+Within a datastore, there may be 
+
+```
+  h = clicon_handle_init();
+  xmldb_plugin_load(h, plugin);
+```
+The plugin is the complete path-name of a file.
+
+Thereafter, a connection is made to a specific plugin, such as a text plugin. It is possible to connect to several datastore at once, although this is not supported by CLixon:
+```
+xmldb_connect(h);
+xmldb_setopt(h, "dbdir", dbdir);
+xmldb_setopt(h, "yangspec", yspec);
+```
+
+

datastore/datastore_client.c

@@ -286,7 +286,7 @@ main(int argc, char **argv)
     else if (strcmp(cmd, "init")==0){
 	if (argc != 1)
 	    usage(argv0);
-	if (xmldb_init(h, db) < 0)
+	if (xmldb_create(h, db) < 0)
 	    goto done;
     }
     else{

lib/clixon/clixon_xml_db.h

@@ -104,7 +104,7 @@ typedef int (xmldb_exists_t)(xmldb_handle xh, char *db);
 typedef int (xmldb_delete_t)(xmldb_handle xh, char *db);
 
 /* Type of xmldb init function */
-typedef int (xmldb_init_t)(xmldb_handle xh, char *db);
+typedef int (xmldb_create_t)(xmldb_handle xh, char *db);
 
 /* plugin init struct for the api */
 struct xmldb_api{
@@ -125,7 +125,7 @@ struct xmldb_api{
     xmldb_islocked_t   *xa_islocked_fn;
     xmldb_exists_t     *xa_exists_fn;
     xmldb_delete_t     *xa_delete_fn;
-    xmldb_init_t       *xa_init_fn;
+    xmldb_create_t     *xa_create_fn;
 };
 
 /*
@@ -150,6 +150,6 @@ int xmldb_unlock_all(clicon_handle h, int pid);
 int xmldb_islocked(clicon_handle h, char *db);
 int xmldb_exists(clicon_handle h, char *db);
 int xmldb_delete(clicon_handle h, char *db);
-int xmldb_init(clicon_handle h, char *db);
+int xmldb_create(clicon_handle h, char *db);
 
 #endif /* _CLIXON_XML_DB_H */

lib/src/clixon_xml_db.c

@@ -652,15 +652,15 @@ xmldb_delete(clicon_handle h,
     return retval;
 }
 
-/*! Initialize database. Open database for writing.
+/*! Create a database. Open database for writing.
  * @param[in]  h   Clicon handle
  * @param[in]  db  Database
  * @retval  0  OK
  * @retval -1  Error
  */
 int 
-xmldb_init(clicon_handle h, 
+xmldb_create(clicon_handle h, 
-	   char         *db)
+	     char         *db)
 {
     int               retval = -1;
     xmldb_handle      xh;