CouchDB - Admin, Security and Design


#1

This topic is for issues around CouchDB, particularly issues of security and database organization.


#2

@rbaynes, @Webb.Peter
The Cloud CouchDB is running fine, at the level now of trying to figure out how to design the layers and automate administration.
Currently I envision two (or three layers) (using schools as the standard model). Each school could have multiple Food Computers, and these may be grouped by classroom. In CC (Cloud CouchDB) there would be a layer of databases, one for each ‘Farm’ (classroom) to which the individual Raspberries would replicate their data.
The second layer would be the master data database (currently mvp_sensor_data), to which the Farms would replicate. I think this will take care of our security needs. There may be some other databases (public read-only, back-up replication) at this layer.
I am not sure it is needed, but there could be a third layer of consolidated Farm data, but at this time I don’t see any reporting/aggregation being done at this level.
There will be the following groups of security.
Administrators - what Rob and I currently are. We can do anything
Public - read only access to the/a consolidated data (read only has to be set up via a security script). We may need to have a database that is ‘in the wild’ for anyone to access (no roles)


Farm_Admin - teachers with admin access to their Farm databases. This is set up via an admin role (farm specific) on their database. The user can have one or more roles (access multiple databases).
Farm Growers - students with read/write access to their Farm database. This is set up via a member role (farm specific) on their database. The user can have one or more roles.
Farm Helper - read only access to Farm database. This is a combination of role and security script.

I need to play with the read-only script, but I have some python code that will automate the building of Farm databases and Farm Admin. I envision the code being called via the UI (so it could run from its own user name).

We may want some additional users, such as what Peter and his friends are currently set up as, who may have broad (but limited) access for building reports and working on coding the UI.

Does this sound like a reasonable architecture?
Let me know if you want any test Farms created.


#3

Looks like a great first plan, thanks Howard @webbhm.

I like the idea of a python script to set things up.

From what I’ve seen, almost everything in couch can be controlled from the command line. For example, I added this to the brain setup to turn on automatic compaction in couch:

curl -X PUT http://localhost:5984/_config/compactions/_default -d '"[{db_fragmentation, \"40%\"}, {view_fragmentation, \"30%\"}, {from, \"23:00\"}, {to, \"06:00\"}]"'

#4

With the Ambassadors program we would place at least one MVP in at least 4 schools. The schools would participate in an experiment that would take place during a single semester. In the future I could envision a significant number of schools taking part in an experiment together. Do you think the architecture you are proposing would allow groups of schools to collaborate in this way?

My initial thought was to have an administrator that could start up the experiment and either setup or invite the schools to join.


#5

@pspeth
You are in the early stages of thinking this through, but the more details you give me the better I can answer your question. I think this will work, but that is not a promise.
Help me think this through from two perspectives: 1) Data entry and control, 2) Data sharing and reporting. I think you want the individual boxes to be managed locally, with few people (a classroom) being able to enter and change data. On the reporting side, I think you want to be able to create dashboards that include all the boxes, or at least all the ones running the same basic experiment.
The security model should work for this, the more complex issue would be designing queries that would pull back the desired data, some of these may need to be custom designed (get data for an ‘Enterprise’ (group of ‘Farms’), then limit to a particular ‘experiment’ or ‘recipe’)…
I like the idea of what you are trying to do.


#6

@rbaynes, @Webb.Peter Trying to get CouchDB up with the latest Raspberry Pi (stretch), and came across some interesting articles about using CouchDB as a web server, with some articles on user profile information. Looks like there may be some slick solutions that would not require a relational database. Have you seen any of these, or have any thoughts? I just found them, but will spend some time digging deeper.
https://blog.couchbase.com/creating-user-profile-store-with-node-js-nosql-database/
https://www.npmjs.com/package/couch-profile

There are also some hints that it is possible to run python from within CouchDB, and not just Javascript.
https://pythonhosted.org/CouchDB/views.html

Regards,
Howard

How is the data modeling and messaging coming?


#7

Hi Howard @webbhm,

I have to find some way of sharing the data dictionary with you and the
community to get feedback. I have been building a prototype backend based
on the google cloud platform. I have scripts to automatically create our
BigQuery schema and upload our existing research. I also have a few
AppEngine instances that handle the PubSub communications and add data to
the DB. All very early prototyping, mostly just to learn the platform and
if it has issues.

My first task is to get all our existing research data uploaded and write
some common queries that we will use, to know if the data design is correct.

Second is finish the PubSub to BigQuery AppEngine instance and data
validator.

Third is a device side python script that can publish data to the backend.
There is a bit of security work for this part too.

After that I plan to extract some PFC data from CouchDB and publish it.

Thanks,
Rob Baynes


#8

Thanks for the update.


#9

Hi,
I have been trying to install couchdb on Raspberri p3 with Raspbian jessie. I found many problems with dependencies specially with erlang. is there a clear resource on how to install couchdb?

I tried to install FoodComputer after installing ubuntu on the raspberry pi and after many tries I manged installing ROS and couchdb, but then found problems with installations of the brain saying it is recommended to be raspbian jessie. I then installed Raspbian jessie and installed ROS successfully but for more than 15 hours I could not mange overcomming the problems for installing couchdb. I tried the steps of many resources and found it always stuch at the erlang part!!

Is there a steps of how to install couchdb on Raspbian jessie??

Thank you very much in advance!!!


#10

The latest instructions for the MVP have some steps for installing CouchDB
(https://github.com/webbhm/OpenAg-MVP-II)
Under MVP/setup you would need ReleaseScript_DB.sh (which calls couchBuild.sh) and couchInit.sh (which builds the tables)

Most of this was taken from instructions at:
http://andyfelong.com/2017/09/couchdb-2-1-on-raspberry-pi-raspbian-stretch/


#11

Thank you very much for you kind help!!!