Custom MVP Data Logging and Dashboard


#1

Hey everybody!

I wrote software to export the MVP data to an external web server (running on Heroku) to be saved in my own database. I then made a web page to view that data on my web site.

My stack is MongoDB, Express/Node, React.

I wrote an article about sending, storing, and receiving the images taken with the PI based off the work I did for this project.

My lettuce is currently very dead probably because I used too much fertilizer. But here is an image of the custom dashboard when the lettuce was flourishing.

Let me know if anybody finds this interesting or has any questions! Hopefully I can offer some contribution to the software side of things.


#2

Wow! I would love to talk further. This couldn’t be more relevant to the work we’re trying to do. Please shoot me an E-mail and we can set up a video conference: peter@marsfarm.io

Thanks for posting - I really appreciate you taking the time to contribute to the forums and look forward to working together. @hemulin @ferguman @webbhm


#3

Awesome! I’ll send an email over soon. I’m pumped to contribute.


#4

@colinrlly

Can you elaborate on why you chose these, and what you are using them for specifically? As much as possible I have pushed to remain with as few components as possible.

  1. Is any of your code on GitHub?
  2. Do you have any plans for data entry?
  3. Please provide critical feedback regarding below:

Right now our approach has been: CouchDB (AWS hosted), Mango queries into ECharts using a nuxt.js framework for a vue.js progressive web app. Essentially taking the MVP’s CouchDB and replicating it with then a hosted UI which will allow for remote access. We also intend to use CouchDB for data entry and even (if possible, discovering now…) actuation/changes to recipe from UI back to the local CouchDB instance. https://github.com/futureag/nuxt-ui


#5

Very interesting! Thanks for the post :slight_smile:

What kind of resolution are you using for the pictures? And since you store the actual Buffer (I only saw best practices with storing file paths) in the database, how fast is your db growing?


#6

What kind of resolution are you using for the pictures?

The picture is 1280 x 720 - the default resolution the Logitech camera from the MVP build took.

And since you store the actual Buffer (I only saw best practices with storing file paths) in the database, how fast is your db growing?

Haha great question. I have to say I didn’t really think through the whole photo storing idea when I built this. 653 images in my database is currently taking up 314MB. I take 1 image every hour between 6am and 10pm, so it’s growing pretty quickly.

The reason I stored the entire image buffer in the database is because I sent the image as a buffer from the Raspberry PI on the MVP to my server (NodeJS). Since I already had the image in buffer form after sending it over a POST request, the first thought that came to my mind was to store the buffer in my database directly. The space has to be taken up somewhere - either on the server file system or in the database so it’s not that big a deal in my opinion. If I were to do it again I’d write the code to convert the POSTed buffer to a .jpeg file, then store it on the file system, and store the path and meta data in the database. It would have been easier to serve the images to the client as a .jpeg rather than as a buffer since in my current situation I have to convert the buffer to a different format on the client in order to display the image.

At the very least I learned a lot about how image data can be manipulated and represented :slight_smile:.


#7

I mostly chose these components because I either wanted to learn about them or because the website I built into was already using them. Specifically I use Express/Node as the server - they accept the incoming POST request from the MVP PI which contains the image data, as well as the temperature and humidity data. I am running my Node server on a free tier Heroku server. It is working well, except after 30 minutes the server goes to sleep and it takes a few seconds for it to wake up the next time somebody accesses the website.

Node then stores the MVP data in MongoDB. I am using a database-as-a-service service called MLAB. They have a free tier as well. From there it’s just connecting to the MongoDB url in my Node code saving the data. They also have a web portal which is a convenient way to view my data through a GUI.

I use React as a front end which runs in the client. It’s a very nice way to develop websites - but has a steep learning curve as it kind of flips how websites are made around. I wouldn’t recommend it for people who don’t have a lot of time to devote to learning it. A template engine like Jinja would also be a good approach if one is trying to code a front end by hand.

It is! Here’s the server.js file which accepts the MVP data from the PI. It’s mixed in with code related to my portfolio website, but the (majority?) is MVP related code.

Here’s the MVPDataModel.js file which acts as the MongoDB model for storing the temperature and humidity data.

MVPImgModel.js is the model to store the image buffers.

The rest of the code in that repo is unrelated to the MVP project except for the React stuff which takes care of the dashboard UI. I’m pretty sure all my React stuff related to the MVP is in this folder.

I don’t currently have any plans for data entry through my dashboard.

I looked at / worked with CouchDB while building this project. It seems to be a similar NoSQL database like MongoDB with a good GUI like Mlab. I think it’s a good fit for the MVP project as it’s simple enough to grasp while still offering a robust, decoupled database solution - especially if it’s AWS hosted.

I like the idea of using ECharts as a charting software - from my brief look it seems to be a more complete solution to charting. If it can be avoided I would not suggest a more low level charting library like chartJS which I used. If a big chunk of data can be sent to the chart, and the library can handle things like day, month, year view that would be better than coding those things manually.

I haven’t used Vue or nuxt.js but they are popular and I am sure good solutions.

That tech stack seems like a good fit for the MVP needs. I agree the MVP project should aim to keep the tech as simple as possible so less components is definitely better.