Meet Jonathan Jaramillo

One of the undeniable pleasures of life in the Finger Lakes region of New York is sitting on a deck at one of the many wineries that line the shores of Seneca and Cayuga Lakes, sipping a cold Riesling and listening to a local band as the late afternoon sun washes everyone and everything in a warm golden light. If your eyes wander from the band out to the lake and the viny green hillsides it is easy to understand why this part of New York is one of the state’s top tourism destinations. It truly is gorgeous.

When Jonathan Jaramillo sits on those same decks, soaking up the sun and the good feelings generated by the band, his eyes also pass over to the green hillsides, covered in grape vines. But his eyes are not aimlessly wandering; they are looking at those vines with a purpose.

Jaramillo grew up in the town of Interlaken—in the heart of the Finger Lakes region. He is a doctoral student in Cornell’s Collective Embodied Intelligence Lab, where he works with his advisor, Assistant Professor Kirstin Petersen, in the School of Electrical and Computer Engineering (ECE). Much of his work centers on developing computer vision pipelines and machine learning techniques that can help small grape growers manage their vineyards in ways that are maximally efficient.

“In high-value crops like grapes there is a thing called ‘crop load,’ which is the ratio of grapes to leaf area,” says Jaramillo. “It’s sort of this magic ratio, because if you have too many leaves and not enough grapes you’re going to have lower profits due to increased shadows, reduced airflow, and an increased prevalence of mildew, which is one of the biggest problems that any grape grower faces. But if you have too many grape clusters and not enough leaves you’re going to have lower quality fruit with low sugar and poor acid levels.”

Jaramillo’s goal is to use machine learning and neural networks to create a smartphone-based tool to enable farmers to figure out whether their leaf-to-grape ratio is in the range they want it to be. With roughly just 20% of the acreage devoted to grape cultivation as California’s Napa Valley and a smaller average size per vineyard, grape growers in the Finger Lakes are in a more precarious financial position than many of the larger growers elsewhere. Having a low-cost tool to help them manage their crop each year could be a real boon.

As a high school student in Interlaken, Jaramillo worked at Sunrise Hill Vineyard in the the Finger Lakes, pruning vines and trimming leaves. But he had no idea that he would someday help create high-tech tools small farmers could deploy to help increase their yields. As a child he liked to take things apart to see how they worked, and as a fifth grader he wanted to be an inventor, but had no real sense of what it was he would invent. In his undergraduate years at Houghton College he majored in physics, minored in computer science and mathematics, and had several research and internship experiences that brought him to engineering.

After graduating from Houghton, Jaramillo took a job with Lockheed Martin working as a systems engineer. Ironically, it was this decision to go into industry that led Jaramillo pretty quickly back into academia. “After a year at Lockheed Martin, they told me they wanted me to get a master’s degree, so they sent me to Cornell,” says Jaramillo. “I started my M.Eng. in Systems Engineering and after one semester I joined the Collective Embodied Intelligence Lab. I pretty quickly realized that I wanted to work much more with this lab than just a semester or two, so I left Lockheed Martin and applied to Cornell for my Ph.D. in Electrical and Computer Engineering.”

His timing could not have been better. Professor Petersen had just been awarded a grant to fund some work in the area of agricultural robotics and Jaramillo had the engineering background, industrial experience, and agricultural background to dive into the project. Now starting his third year as a doctoral student, Jaramillo says “People make decisions sometimes arbitrarily, hoping that they will open up doors but you never really know what doors are going to be on the other side of any given decision. In this case, it has worked out really well for me so far.”

In his first years in the lab Jaramillo has worked on several other projects as well, including a new platform for a service robot aimed for safe and intuitive human interaction in crowded, chaotic outdoor settings. As part of this project Jaramillo has been overseeing the work of M.Eng. students and says he has discovered that he very much enjoys mentoring and teaching in addition to hardware design and programming.

Another project Jaramillo has been working with an entomologist is the creation of a computer vision algorithm to detect and track honeybees engaged in pheromone triggered anemotaxis. Honeybees engage in a type of swarming behavior when they are trying to locate the hive’s queen. Entomologists are not sure if the bees are following a pheromone gradient to get to the queen or if they are simply directing themselves upwind. The computer vision pipelines Jaramillo and his collaborator create to identify and track honeybees leverage neural networks to help answer this question. Once perfected, he says the tool could be modified to track other animals and insects as well and become a useful tool to research biologists.

Moving forward, Jaramillo says that his focus in the remainder of his years of doctoral study will most likely be the further development of smartphone-based computer vision tools for grape farmers. Small growers often have small margins and cannot afford to tie a lot of money up in inventory that might not get used. Jaramillo’s tool will help greatly with resource allocation. If grape growers can have accurate predictions of their eventual yield early in a growing season they can have a better idea of how much tank space and other resources they might need. As mentioned earlier, they can also take what they learn from the tool and use it to manage their grape-to-leaf ratio for optimal production.

“My focus right now and for the foreseeable future is creating low-cost, simple to use automation tools that can help vineyard managers improve their crop production,” says Jaramillo. “When I tell people I work on agricultural robotics they sometimes give me dirty looks because they think I am trying to put farmers out of business and replace them with robots. But really it’s the opposite. My goal is to make farming at a small scale more profitable. With a high value crop like grapes, there is just something special about what the Finger Lakes has and I would hate to see the rest of the world’s large grape producers have access to the best and newest technology while the small producers in the Finger Lakes get left in the dust. I want to disseminate the technology down to the small grape growers.”

Listening to Jonathan Jaramillo talk about his research, it is easy to hear his commitment to the place he was born and raised. So if you see him one sunny day on the deck of a winery, staring intently at the vines as everyone else is watching the band, now you’ll know why.