This summer, four high school students from Trinity Valley School interned at BRIT through our Junior Volunteer/Intern program. These students were given the task of applying their computer science background to the challenge of helping BRIT create a quick and easy way to determine the fullness of our herbarium cabinets. By better understanding the details of the capacity of the cabinets, BRIT will be able to strategically plan for future growth and management of the herbarium collections.
L to R: Ashia White, Kevin James, Jason Best, and Jacob Haydel
I worked with students Grace Beasley, Jacob Haydel, Kevin James, and Ashia White to explore the process of using computer vision technology to analyze images of the open cabinets. We set out to extract details of each cabinet’s structure and the height of each stack of specimens within the cubbies of the cabinet to determine how much space is available for use in the cabinets. The students also created a powerful graphical interface for visualizing the cabinet data once they are generated.
The students already had experience with Processing, a software platform which facilitates the creation of software with rich and dynamic user interfaces, so we chose this as our primary tool for the project. Fortunately, Processing also has a number of additional modules which helped the team manipulate and analyze the images. In particular we used two modules that are focused on computer vision technology: OpenCV and BoofCV.
The popular app FaceSwap involves similar technologies as those being used for this project. Photo credit: John Woods (Woodsie TV @ YouTube)
Computer vision technologies are used for many applications ranging from self-driving cars, to robotics, to entertainment, and much more. For our particular challenge, we used computer vision techniques to analyze the images of open cabinets to first find the boundaries of the cabinets and the cubbies, then to identify and measure the spaces that were filled with specimens. The computer vision modules provided all the tools we needed to do this, but we still had to experiment a lot to find the right combination of tools and methods to get consistent and accurate results. We haven’t yet found the perfect formula, but we are very close to a solution that we hope to start using soon.
Grace and Jacob
Jacob applies a QRcode
Ashia captures an image
To perform a full herbarium capacity analysis, we’ll need to capture an image of every cabinet in the herbarium (something we hope to recruit our loyal volunteers to help with), then process them with our computer vision solution. In the end, we’ll have a dataset with a record for each cubby and the percentage fullness of that cubby, as well as a total fullness percentage of each cabinet. With those data, we’ll have a much better idea of how much room we have to grow and how to strategically organize our collection to more easily adopt orphaned collections. To better understand this huge amount of data, the team also created a graphical interface that helps visualize the data in a more intuitive format. In particular, we created a herbarium “heat-map” using Processing which can read all the capacity data and superimpose color-coded cabinets on a herbarium map (pictured below). The interface also lets you click on a cabinet and see the details of each cubby and eventually we’ll combine this with the herbarium inventory data to let you see what species are in the cabinet.
A heat-map of the BRIT herbarium showing cabinet fullness. This was created in Processing and uses simulated data, but the interface itself is fully functional.
The project still has a ways to go before we have a final, polished tool ready for use by the herbarium, but several of the students are eager to continue their efforts beyond the end of their summer break. Below, Jacob and Kevin share their summer experiences.
Going into this project for BRIT, I didn’t really know what to expect. I hadn’t really done any work with computer vision, but I was eager to get started. The main thing I have learned from the project is that programing as a team is very different from working on an individual project. When working as a team you have to make sure everyone is on the same page and agrees on the direction the project should go. I also learned a lot about the problems that face BRIT in the digitization process. Taking a written format and converting it to a digital one can be a very tedious process; however there are huge benefits to digitization.
Working on the program, at first we had a really simple goal, to figure out how full the cabinets were. As we progressed we slowly started adding other goals to our original goal and expanded our idea of what this program could do. Despite this we were able to incorporate most of the ideas we came up with, and I think when the final program is done it will be much better than we initially imagined. I have really enjoyed working on what I would consider my first real world project in programing. It has definitely solidified my desire to pursue computer science as a career.
Going into this project, I wasn’t too sure how we were going to approach a project on such a large scale as this one. To begin, we learned the basics of computer vision and data visualization. Over the summer, we mainly worked on the data visualization aspect of the project, using fake data to test our program. We now have a heat map of the BRIT Herbarium, as well as a program that lets us view the fullness of individual cabinets. We hope to continue to expand this program, and we also hope to find a reliable method to obtain accurate data from the herbarium.
Our group faced the challenges that come with working on such large code as a team. This included changing specific parts of the code without unintentionally altering the entire program. We eventually learned how to work together more efficiently, and we managed to make a working program to display our herbarium data. This project was a very rewarding experience for me because I got an opportunity to use my computer skills in the real world.