POLO SOLOGUB
Polo Sologub is a data scientist, developer and designer with a focus on machine learning and human-computer interaction. Currently working as an AI Software Engineer for the Creative Computing Institute at UAL, they are especially interested in archives and collections and how to use creative technologies to open them up in engaging ways.
Email
Instagram
LinkedIn
Github
01 SKETCHY COLLECTIONS
Sketchy Collections is an AI-driven image search tool that makes it possible to explore select museum collections by drawing or uploading a picture. The tool analyses the query picture and images of museum objects, predicting text descriptions based on each museum's own terminology, and compares those to find the top match. It makes exploring both cultural heritage and AI technology more intuitive and fun!
The tool uses an AI model called CLIP to turn the images into long lists of terms with probabilities. In the app, you can see the five top terms for both the sketch and the matching images. Each museum has its own vocabulary, some of which are tags made for the public or terms used for cataloguing the museum objects.
When comparing the images and tags across different museums, we can see how the AI model sees images and how it describes them given different “languages” of museums. All of this is shaped by different factors: What was the model trained on initially? What objects are included in the museums? How do museums talk about their collections?