Posted by the Google Dev Library Team
Introducing the Dev Library Contributor Spotlights - a blog series highlighting developers that are supporting the thriving development ecosystem by contributing their resources and tools to Google Dev Library.
We met with Doug Duhaime, Full Stack Developer in Yale University's Digital Humanities Lab, to discuss his passion for Machine Learning, his processes and what inspired him to release his PixPlot project as an Open Source.
What led you to explore the field of machine learning?
I was an English major in undergrad and in graduate school. I have a PhD in English literature. My dissertation was exploring copyright history and the ways that changes in copyright law affected the book market. How does the institution of fixed duration copyright influence the book market? To answer this question, I had to mine an enormous collection of data - half a million books, published before 1800 - to look at different patterns. That was one of the key projects that got me inspired to further explore the world of Machine Learning.
In fact, one of my projects - the PixPlot library - uses computer vision to analyze image collections, which was also partially used in my research. Part of my research looked at plagiarism detection and how readily people are inclined to copy images once it becomes legal to copy them from other texts. Computer vision helps us to answer these questions and identify key patterns.
I’ve seen machine learning and programming as a way to ask new questions in historical contexts. And there's a whole field of us - we're called digital humanists. Yale University, where I've been for the last five years, has a fantastic digital humanities program where researchers are asking questions like this and using fun machine learning platforms like TensorFlow to answer those questions.
Can you tell us more about the evolution of your PixPlot library project?
We started in Yale's digital humanities lab with a project called neural neighbors. And the idea here was to find patterns in the Meserve-Kunhardt Collection of images.
Meserve-Kunhardt is a collection of photographs largely from the 19th century that Yale recently acquired. After being acquired by the university, some curators were preparing to identify all this really rich metadata to describe these images. However, they had a backlog, and they needed help to try to make sense of what's in this collection. And so, Neural Neighbors was our initial attempt to answer this question.
As this project went on, we started running up against limitations and asking bigger questions. For example, instead of just looking at the pictures, what would it be like to look at the entire collection all at once? In order to answer this question, we needed a more performant rendering layer.
So we decided to utilize TensorFlow, which allowed us to extract vector representation of each image. We then compressed the dimensionality of those vectors down to 2D. But for PixPlot, we decided to use a different dimensionality reduction technique called umap. And that brought us to the first release of PixPlot.
The idea here was to take the whole collection, shoot it down into 2D, and then let you move through it and look at the images in the collection wherein we expect images with similar content to be placed close by one another.
And so it's just evolved from that early genesis and Neural Neighbors through to where it is today.
What inspired you to release PixPlot as an open source project?
In the case of PixPlot, I was working for Yale University, and we had a goal to make as much of our contributions to the software world as possible open and publicly accessible without any commercial terms.
It was a huge privilege to spend time with the lab and build software that others found useful. I would say even more generally, in my personal life, I really like building things that people find useful and, when possible, contributing back to the open source world because, I think, so many of us learn from open source.
Find out more content contributed and authored by Doug Duhaime and discover more unique tools and resources on the Google Dev Library website!
