Columbia Center of AI Technology Announces Four New Faculty Research Awards

Columbia Engineering and Amazon have announced four new faculty research awards for the Columbia Center of AI Technology (CAIT). The latest round of projects will explore a range of challenges in artificial intelligence (AI), specifically algorithmic fairness, unified methods for interpreting artistic images found on the internet, the development of a scalable, differentially-private data market system, and CAIT’s first award focused on human-computer interaction (HCI).

Launched in September 2020, CAIT is a strategic collaboration between Columbia and Amazon to push the frontiers of AI. In addition to supporting research, CAIT also provides funding for PhD fellowships, a seminar series, and an annual research symposium.

The four faculty research projects being supported are:

Algorithmic Fairness through a Causal Lens, Elias Bareinboim, associate professor of computer science

“Despite the growing concern about issues of transparency and fairness and the high complexity entailed by this task, there is still not much understanding of basic properties of such [AI] systems,” Bareinboim wrote in his abstract. “To assist AI designers in developing systems that are ethical and fair, we will build on recent advances in causal inference to develop a principled and general causal framework for capturing and disentangling different causal mechanisms that may be simultaneously present. Besides providing a causal formalization for fairness analysis, we will investigate admissibility conditions, decomposability, and power of the proposed fine-grained causal measures of fairness. This will allow us to quantitatively explain the total observed disparity of decisions through different underlying causal mechanisms that are commonly found in real-world decision-making settings.”

Facially Expressive Robotics, Hod Lipson, the James and Sally Scapa Professor of Innovation in the Department of Mechanical Engineering

“The nonverbal portion of human-machine interaction is not keeping pace” with ongoing progress in language models,” Lipson’s abstract explained. “This growing chasm between the advancing verbal content and the poor nonverbal ability will prevent AI from reaching its potential in full human engagement. The goal of this research pilot is to explore architectures that will allow robots to begin to learn the subtle but critical art of physical facial expressions.” Lipson’s lab has developed a soft animatronic face platform containing 26 soft actuators, most of them around critical expression zones such as lips and eyes. “We aim to study two key communication pathways,” Lipson wrote. “The first is learning what facial expression to make (and when) based on conversational context, and the second is learning how to physically articulate these expressions on a given soft face.”

Neural Methods for Describing and Interpreting Works of Art, Kathleen McKeown, the Henry and Gertrude Rothschild Professor of Computer Science

“The ubiquity of art on the internet demands better ways of organizing and making sense of visual art,” stated McKeown in her abstract. “We propose an investigation into unified methods for representing and describing these artistic images. We propose beginning with an investigation of representations produced by large pretrained vision and language models to understand the kinds of aesthetic information they encode.” This information includes color, form, style, emotion, and subject matter. McKeown proposed a follow-up study of greater difficulty, generating descriptive and interpretative captions. “We believe this line of inquiry has the potential to drive social good and commercial value, expanding access to the visually impaired while simultaneously enabling better tools for a range of commercial scenarios,” McKeown wrote.

DataEx: A Data Market System for Modern Data Users, Eugene Wu, associate professor of computer science

“We propose to develop a scalable and differentially private data market system and to deploy a version for the Columbia campus,” Wu wrote in his abstract. “The data market system allows anyone that has a machine learning task to upload their training dataset in a differentially private [manner] and search for other datasets that could be used to augment their training data to produce a higher accuracy model. At the same time, data providers can upload differentially private summaries of their datasets to be indexed by the platform.” Differential privacy is the gold standard in data privacy, guaranteeing anonymity for individuals. Wu’s system would allow sharing of sensitive information without individually identifiable information. “In a Columbia deployment, researchers and teams throughout Columbia would be able to register the data they have available, and benefit from the collective capacity of the whole university,” wrote Wu.

Wu’s project is managed by Michael Kearns, Amazon Scholar and professor in the Department of Computer and Information Science at the University of Pennsylvania. The collaboration between Wu and Kearns demonstrates the cross-pollination among Amazon’s academic community and up-and-coming researchers, a goal of CAIT since its inception.