Data Collection for my Keras/TensorFlow Reinforcement Learning Project.

This project was designed for a business in Arizona called Thrive and Grow Farms. The goal for the project is to develop a system that can maximize the growth of microgreen plants in a controlled environment. This post will describe the general setup for our microgreen project as well as issues I ran into and how I overcame them. This project spanned over 2.5 months and is now able to produce data for my project. This is phase one of the project and if time permits I will continue working on it after the semester ends.

Interview with the owner of Thrive and Grow Farms:

Thrive and grow farms info: https://www.thriveandgrowfarms.com/

There were some minor issues with Michaels mic in the above video. But we appreciate him taking the time to meet with us about the project.

As you can see I opted for quite a few relays for this project. They are used to control the lighting and watering system. Though there are more elegant solutions out there this did ultimately work nicely for me.

Working around limitations for the raspberry pi.

I built a new Raspberry pi 4 for this project. It featured 8 GB of ram and USB 3.0 ports. I tried several operating systems on this raspberry pi this semester. Recently they came out with a 64 bit OS version of ubuntu 20.04 for the raspberry pi but I quickly started running into errors. I at the time felt that this was because of the new software release but after some further troubleshooting I realized that the raspberry pi was overheating. This raspberry pi is much more powerful than the raspberry pi 3B+ so I eventually removed the top of the case I purchased to solve this issue.

There were confirmed reports that the new 64 bit OS had some issues so I switched to the more stable 32 bit OS. My goal was to see if it was possible to use TensorFlow on the Raspberry Pi. But after reading the documentation on the TensorFlow website I realized that not only did TensorFlow need a 64 bit OS as well as python 3.7 or 3.8. I went back to the 64 bit system but unfortunately there was a new operating system that came out for the raspberry pi ubuntu 21.04 this shipped with Python 3.9 instead of 3.8 which was tough to work around. I still had quite a bit of trouble getting TensorFlow installed on the Raspberry pi and ended up opting for a new strategy that would allow multiple computers to access the data on the raspberry pi. It was never guaranteed that the raspberry pi could build the network so instead of spending more time troubleshooting I implemented a new plan. I plan to revisit this issue in the future.

Here is the first draft of my solution. I needed to ensure that I had access to the data from multiple computers.

My plan was to send zipped files through the network ports. But ended up instead discovering how to setup a samba share in Linux which essentially allowed all of my computers to be able to access the shared data. There was a couple days of troubleshooting permission issues with my network but eventually it worked out nicely. This was important because I could not rely on my main computer to be on 100% of the time. Since the network will be designed to make new predictions on an hourly basis I need multiple options for building my neural network. My plan is to have multiple computers available to train the network as needed so they all need to be able to access the sensor readings. Here is my samba share setup below.

Ensuring accurate data using an automated watering system.

The watering system was an important feature for this project. We needed to collect data without ever reaching into the trays. The sensors would always be recording and we did not want to deal with entering bad data into our system. If we put our hand between the plants and the sensors while watering we could not guarantee accuracy. I worked with the owner of Thrive and Grow and we came up with a solution. Watering tables. Getting the tables ended up being quite the chore. We continually found watering tables that were about two inches too large for my project space. We were close to having to find a new solution for the watering system but we ended up finding some watering tables that fit the shelving units perfectly. In fact they were designed for my make and model shelves that I had purchased at Costco.

I found the watering tables in town but unfortunately the salesmen led me down the wrong path. They did not have the ebb and flow kits that I asked for and he talked me into a different setup that would ultimately fail. They sold me a way to push water into the watering tables but the drains did not work the way he said they would. The ebb and flow system has an interesting setup it has a drain that removes water at the level that you need for the plants. So the water level rises just past the bottom of the tray and there is an emergency overflow that stops the water from going higher. But the input for the water after the water pump shuts down is the real drain for the system. This was not obvious at first and the documentation online assumed that the end user had their full setup but my setup only had emergency drains. I eventually logically thought this all through and fixed the water draining issue.

So as you can see in the picture above I only had the emergency drain installed and no way to push water and remove it properly. I made two major alterations here. I noticed after some testing that the screen on the emergency drain actually blocked the flow of water which was dangerous if the system was left on it could have flooded so I removed the screen on the right and purchased the proper ebb and flow kits.

Here is a short video of the water working properly.

As you can see above my system is properly removing excess water while the intake pumps water into the tray. Problem solved!

Though there was one more issue that needed to be fixed. The water pump that we had purchased was not powerful enough to push water up to the top tray. So I ended up replacing it with a more powerful model. Featuring a 100 watt 1050 Gallon per hour pump that had the added advantage of pulling water from the bottom of the container I was holding the water in.

Gathering data using HC-SR04 Ultra Sonic Sensors.

The project uses two remotely controlled raspberry pi microcomputers to control lighting and water systems as well as take distance readings using HC-SR04 ultrasonic sensors.

The sensors were fitted into boards so they would lay flat.

My initial setup had a breadboard for connecting wires to the GPIO pins on the raspberry pi. Though this setup turned out to not be ideal. It worked for the relays but the sensors had so many wires that it became problematic.

For the wiring, I built a spreadsheet to make the installation easier and it did work on the first try though there was a slight issue with GPIO pin 12. I miss understood the label but found out later that the pin was used for other solutions. I figured this out pretty quickly. But you can see that I marked that pin with an 11?

The wiring worked on the first try. I hired my kid to help me put it all together. She did great and we were off to the races.

But there were issues lurking under the surface.

At first when I tested my sensors they all worked which was lucky and unlucky. Lucky because I knew right away that the wiring was correct. I got readings from all sensors quickly. I believed that they were all working which I found out later that it was not really the case.

Here is the first data that I got from my sensors. It looked really good though there were some strange issues. One of which happens when the plants have no leaves. I sometimes get sensors that give 12 meter readings. This may be a limitation of the sensors but could be eliminated using statistics by eliminating values that fall outside of specific z-scores.

I think the biggest issue I had here was that the errors I started getting with the sensors made it seem like the sensors were to blame when in fact the breadboard had some loose connections that were causing issues it took me a couple of weeks to figure out that the breadboard was to blame. Many error messages pointed me to solutions online that did not generate solutions for my project. This led to many attempts to resolve my issues with the sensors. Including an solution that I tested extensively. It was suggested that issues may arise when not using resistors to bring the voltage down on the echo pin from 5 volts to 3.3. But at least for my project this did not prove useful I tried several setups and even tried new sensors attached to resistors but generated the same kinds of errors. The following is a relay diagram that was used to step down the voltage for one of my pins.

Courtesy of https://thepihut.com/

Eventually I began to suspect that the issue was with the breadboard. I found a couple forums where people were pointing out that the breadboards could cause some unexpected connection issues. One issue I noticed was that a sensor would look like it was malfunctioning then later it would work fine but another sensor would take its place. My solution was to install a hat for the raspberry pi.

I removed the case around the pi to avoid overheating and removed the breadboard replacing it with the hat. I could now secure the wires with screws. I spent a little extra on one that showed led lights for the active connections that helped me later.

Here is an example of a wire that came undone and I was able to quickly see which one it was and replace it using the led lights on the hat. Troubleshooting these issues were never obvious but that purchase paid for itself quickly I doubt I would have noticed the wire it was hidden in one of the bottom trays.

The Lighting system and temporary prediction placeholder for our neural network.

I experienced some discomfort with working with dictionaries and json files recently and decided to incorporate them in my project for practice.

I built a python program that would randomly choose a number between 1 and 60 to serve as stand in neural network predictions. Every hour it updates a file. I check to see if the file size has changed and when it has I read the new predictions into the system. Then I adjust the lights accordingly.

My theory is that we can find optimal lighting settings using Reinforcement learning.

Currently I am gathering data for three microgreen trays. The first two are being fed random data and turning on and off the relays accordingly. The third group is a control group that has the lights turned on between the hours 8 am and 8 pm.

Here we can see my computer controlling the lights remotely using the predictions.

Everything looked like it was setup nicely but then I started running into problems. The toughest thing about this project was running into errors that mimicked other issues.

This was the common one. As you can see here "no echo received" looks like a sensor issue.

But would ultimately end with the program crashing with a memory error.

It even resulted in me at one point losing all of my data for the project. At the time I was having trouble making backups due to a permission issue with my samba share. I have since not only fixed the permission issue but built a script that backs up the system every hour using the crontab Linux scheduler.

The fix for the above issue was difficult to find. I kept thinking that there was an issue with the sensors.

But this was in fact a pesky logic error that I had not considered. I had modeled my code in a similar way to how I was saving and reading the prediction data. But that data had been opened and written to hourly. My sensors on the other hand fire every three seconds.

Despite having fully working code it was not very pythonic. In some cases I was duplicating code which was not acceptable. I decided that I needed to redo the python code and try out object oriented programming. I built several classes and put them in different files. I worked with inheritance between parent classes and child classes I even built class factories. Which was a good start for learning object oriented programming. But I made a mistake in logic since I was building classes for my different sensors I should have opened the file once in the initialization of the class but instead I was opening the file every time I took a reading. In the network prediction section this made sense the data was changing and we needed new data but in the sensor section this was causing issues. Hence the warning that was referencing not getting a echo from the sensor was really happening because of how I was handling my database.

After eliminating the error in logic I started getting some clean readings. The warning issue has been resolved.

Now everything is being stored separately and should be ready to be cleaned up for our Neural Network.

Building Log Files

When our system takes an action we are saving the results in log files. I started building log files at the beginning of the project so I could monitor the progress of the system as well as troubleshoot problems as they came up.

This is a basic one that monitors whether or not a sensor errored out or not. It documents the time and date.

Since then my log files have evolved allowing me to provide more detailed info. I believe there is still room for improvement but its a good start. Below you can see that Sensor 1 failed to write to the file. This is good information to have if things are going wrong.

Next Steps:

Trouble Shooting odd sensor readings.

The sensors are working much better now but they are still not perfect.

Some sensors will produce readings of 12 meters. I suspect that this could just be a faulty sensor. Though I have seen other sensors exhibit the same behavior from time to time. In this case sensor 4 was producing accurate readings during the blackout phase. In the black out phase there is a cover over the plants that allows the sound waves to bounce off a flat surface. But then instantly started producing 12 meter readings once the cover was removed.

This could be due to the lack of leaves possibly but its notable that not all sensors are getting readings of 12 so it may be a faulty sensor. Sensor 4 still continues to produce 12 meter readings from time to time.

One other issue I have noticed is the following.

Two days ago the plants were ready to be picked and the readings looked accurate for all sensors.

I wanted to see what the sensors do when the plants are overgrown. I have seen the plants spike quickly after this stage so it only took two days for the plants to grow wild as they chased the light.

The strange reading comes from sensor 6 where now we are getting readings for over 1 meter. This may very well be a sensor that needs to be replaced but I have noticed this behavior happens typically when the plants are over grown and moving toward the light on the left side of the tray. Note if you look below the next reading drops to .179 meters.

To eliminate the possibility of faulty sensors I think we should replace the sensors that have been providing false readings before doing any further trouble shooting.

Incorporating Object Oriented Programming

My first sensor reading python code was not very pythonic. There was quite a bit of redundancy built into it. It was difficult to make changes since any change I made had to be repeated over multiple sensors. The code was working but was just not efficient enough to work with.

(Warning! Inefficient code here)

I decided to re-write my code into their own class libraries. The top left code is the only code that I need now to gather my sensor data. Second to the left I have my distance sensor class that inherits from my factory class and relay class. I used inheritance to build class factories with reusable code. This allowed me to make quick alterations that effected all of my sensors at the same time. It was also much more efficient. My other code was working fine but I was happy that I chose to re-write it.

Automating the Process.

In the future we will need to ensure that there are no lapses in data collection. Many of the issues have been resolved in this project but there is still the possibility of crashes and service interruptions due to power outages. I have been investigating using lock files to ensure that our data collection scripts are always running. The raspberry pi is using a Linux OS which gives us access to crontab. Crontab can be used to run scheduled scripts at specific times. When a python file is active it produces a lock file that can be checked to ensure that the program is running. I would like to take advantage of this to have the system run the data collection scripts if they happen to close due to error or outage.

The data will be processed by a Tensor flow Neural network using Keras. The network has full control of the lighting system and will be using reinforcement learning to optimize the heights of microgreen plants by changing the duration that the plants are exposed to the lights. The tarp was used to reduce outside light sources for my project.

This project seeks to solve a real-world concern for Thrive and Grow Farms. If we can optimize the growth rate for microgreen plants giving them the perfect amount of lighting we can grow plants more efficiently. This could improve both the quality and quantity of the plants that are being grown.

This project was designed and implemented by Jason Ismail. A student at Bellevue University. All intellectual rights reserved.

Acknowledgements:

Dr. Catherine Williams for inspiring me to take on such a challenging project this semester. I have enjoyed working with her several times over the course of this masters program. She is a natural born leader that inspires people to push their limits.

Dr. Brett Werner who took the time to help me on my two real world data gathering projects. These types of projects were not a requirement for the program but my teachers inspired me to reach higher.

Professor Matthew Metzger helped me discover the raspberry pi and explained how to automate tasks in crontab. I was very fortunate that he was there to help me out with this. It opened my eyes to new and exiting technology that I have put to good use.

Michael Ismail the owner of Thrive and Grow Farms has always inspired me to work hard and test the boundaries. His anything is possible approach to business is an inspiration to everyone around him.

Looking to hire?

If you represent a company that is using science to change the world and are in need of a Data Scientist with a Bachelors in Mathematics. Feel free to contact me and tell me about your projects.