Running recipe fills SD card with data!


#1

Hello community,

While running our basil test grow, I noticed that the SD card was filling up pretty quickly, about 1.5G per day.

The reason this happens is because the brain writes every sensor value and actuation command to the environmental_data_point database about once a second.

I am using the command below to turn on automatic database and view compaction in couchDB. The percentages I selected are a bit aggressive, but they are based on measuring DB fragmentation before and after compaction.

curl -X PUT http://localhost:5984/_config/compactions/_default -d '"[{db_fragmentation, \"40%\"}, {view_fragmentation, \"30%\"}, {from, \"23:00\"}, {to, \"06:00\"}]"' 

How do those of you who are running a PFC deal with the disk running out of space?

Thanks!


#2

@rbaynes
From a debugging perspective, at times it is useful to track every action. I question whether such frequent data collection is needed for gathering research data. The MVP is collecting sensor readings every 20 minutes, which establishes trends. The actuators (exhaust fan) are run every minute, but only log if there is a change of state (turned on or turned off).
You might consider distinguishing long term research data and operational data. Send the operational data to a separate database, and purge it on a periodic basis (daily, weekly?). It would be available for debugging but not accumulate long term.


#3

Hi Howard @webbhm,
I agree with you. What is currently going on is that since ROS is an event driven system, every actuation and sensing event is simply written to the database.

I’m going to look into a change that would only write a particular variable to the database if its value has changed. That will greatly reduce the data load without losing any information. A short term fix for the existing system architecture.

[update] I just checked the code. In our sensor_persistence.py ROS node, we are only saving a value if it has a delta from the previous value greater than 0.01. I need to add the same logic to our recipe_persistence.py node.