BRIT’s Computer Vision(aries)

BRIT’s Computer Vision(aries)

August 31, 2016

This summer, four high school students from Trinity Valley School interned at BRIT through our Junior Volunteer/Intern program. These students were given the task of applying their computer science background to the challenge of helping BRIT create a quick and easy way to determine the fullness of our herbarium cabinets. By better understanding the details of the capacity of the cabinets, BRIT will be able to strategically plan for future growth and management of the herbarium collections.

Ashia White, Kevin James, Jason Best, and Jacob Haydel
L to R: Ashia White, Kevin James, Jason Best, and Jacob Haydel

I worked with students Grace Beasley, Jacob Haydel, Kevin James, and Ashia White to explore the process of using computer vision technology to analyze images of the open cabinets. We set out to extract details of each cabinet’s structure and the height of each stack of specimens within the cubbies of the cabinet to determine how much space is available for use in the cabinets. The students also created a powerful graphical interface for visualizing the cabinet data once they are generated.

The students already had experience with Processing, a software platform which facilitates the creation of software with rich and dynamic user interfaces, so we chose this as our primary tool for the project. Fortunately, Processing also has a number of additional modules which helped the team manipulate and analyze the images. In particular we used two modules that are focused on computer vision technology: OpenCV and BoofCV.

Faceswap app
The popular app FaceSwap involves similar technologies as those being used for this project. Photo credit: John Woods (Woodsie TV @ YouTube)

Computer vision technologies are used for many applications ranging from self-driving cars, to robotics, to entertainment, and much more. For our particular challenge, we used computer vision techniques to analyze the images of open cabinets to first find the boundaries of the cabinets and the cubbies, then to identify and measure the spaces that were filled with specimens. The computer vision modules provided all the tools we needed to do this, but we still had to experiment a lot to find the right combination of tools and methods to get consistent and accurate results. We haven’t yet found the perfect formula, but we are very close to a solution that we hope to start using soon.

To perform a full herbarium capacity analysis, we’ll need to capture an image of every cabinet in the herbarium (something we hope to recruit our loyal volunteers to help with), then process them with our computer vision solution. In the end, we’ll have a dataset with a record for each cubby and the percentage fullness of that cubby, as well as a total fullness percentage of each cabinet. With those data, we’ll have a much better idea of how much room we have to grow and how to strategically organize our collection to more easily adopt orphaned collections. To better understand this huge amount of data, the team also created a graphical interface that helps visualize the data in a more intuitive format. In particular, we created a herbarium “heat-map” using Processing which can read all the capacity data and superimpose color-coded cabinets on a herbarium map (pictured below). The interface also lets you click on a cabinet and see the details of each cubby and eventually we’ll combine this with the herbarium inventory data to let you see what species are in the cabinet.

herbarium heat map
A heat-map of the BRIT herbarium showing cabinet fullness. This was created in Processing and uses simulated data, but the interface itself is fully functional.

The project still has a ways to go before we have a final, polished tool ready for use by the herbarium, but several of the students are eager to continue their efforts beyond the end of their summer break. Below, Jacob and Kevin share their summer experiences.

Jacob Haydel

Going into this project for BRIT, I didn’t really know what to expect. I hadn’t really done any work with computer vision, but I was eager to get started. The main thing I have learned from the project is that programing as a team is very different from working on an individual project. When working as a team you have to make sure everyone is on the same page and agrees on the direction the project should go. I also learned a lot about the problems that face BRIT in the digitization process. Taking a written format and converting it to a digital one can be a very tedious process; however there are huge benefits to digitization.

Working on the program, at first we had a really simple goal, to figure out how full the cabinets were. As we progressed we slowly started adding other goals to our original goal and expanded our idea of what this program could do. Despite this we were able to incorporate most of the ideas we came up with, and I think when the final program is done it will be much better than we initially imagined. I have really enjoyed working on what I would consider my first real world project in programing. It has definitely solidified my desire to pursue computer science as a career.

Kevin James

Going into this project, I wasn’t too sure how we were going to approach a project on such a large scale as this one. To begin, we learned the basics of computer vision and data visualization. Over the summer, we mainly worked on the data visualization aspect of the project, using fake data to test our program. We now have a heat map of the BRIT Herbarium, as well as a program that lets us view the fullness of individual cabinets. We hope to continue to expand this program, and we also hope to find a reliable method to obtain accurate data from the herbarium.

Our group faced the challenges that come with working on such large code as a team. This included changing specific parts of the code without unintentionally altering the entire program. We eventually learned how to work together more efficiently, and we managed to make a working program to display our herbarium data. This project was a very rewarding experience for me because I got an opportunity to use my computer skills in the real world.

A few photos of the project:

Grace and Jacob
Grace and Jacob photographing a cabinet. We love seeing smiles in the herbarium!
Aisha
Aisha
Jacob
Jacob applying a QR code to a cabinet.
cabinet bounds
Cabinet bounds
cubby capacity
Cubby capacity

 

Leave A Response

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.

Related Articles

Survey of BRIT’s Tarrant County Bryophyte Collection

By Hanna Liebermann, Hendrix College biology student / BRIT intern, & Charles Gardner, bryophyte specialist / BRIT Research Associate A tiny world of wonder! (taken with an iPhone 6 through the microscope eyepiece) Bryophytes, defined by their lack of vascular tissue, are a category of smaller plants that include the mosses, liverworts, and hornworts. With the exception of the few specialists who hold these groups dear and recognize their miniature complexity, the “bryos” are generally an unsung category of botanical study. Nevertheless, they are an extremely valuable section of BRIT’s collection, representative of local and global vegetation. While the majority of BRIT’s bryophyte collection has been scanned and the packet label information entered into the Symbiota database, little had b...
Read More >

Data Entry, Insects, and Flying Cacti: My BRIT Internship

Haley Rylander looks back on her varied experiences during her BRIT internship.
Read More >

My Summer Education: The Microscopic World

BRIT has been amazing – a catalyst that has unlocked the doors to the natural world and shown me wonders that I had never fully appreciated. I have always loved hiking, climbing trees, being outside in general, but my connection to the surrounding plant-life was distant, similar to the relationship between a homeowner and the trees that form the hardwood floors. I appreciated plants’ beauty and enjoyed the shade, but now there is a definite connection between my (limited) scientific knowledge and the physical plants. It means so much more to know the scientific name of a tree you just ran past, or to recognize Vitis mustangensis and know that you can eat the wild grapes growing on the vines. Perhaps most exciting thing to me is this: the further realization that the microscopic world is bu...
Read More >