|
Email / Google Scholar / Twitter / Github |
|
|
My interests revolve around the convergence of natural language processing and computer vision, with a focus on gaining insights from human cognition. I am enthusiastic about exploring language grounding within multimodal contexts and investigating the linguistic and cognitive characteristics of models. |
|
|
TL;DR Our experiments show that LMs partially converge towards representations isomorphic to those of vision models, subject to dispersion, polysemy, and frequency, which has important implications for both multi-modal processing and the LM understanding debate. |
|
TL;DR In this work, we introduce FoodieQA, a manually curated, fine-grained image-text dataset capturing the intricate features of food cultures across various regions in China, and evaluates vision-language Models (VLMs) and large language models (LLMs) on newly collected, unseen food images and corresponding questions. |
|
TL;DR We analyze the robustness of a retrieval-augmented captioning model SmallCap and propose to train the model by sampling retrieved captions from more diverse sets, which decreases the chance that the model learns to copy majority tokens, and improves both in-domain and cross-domain performance. |
|
TL;DR We empirically show that GPT-4V excels at identifying cultural concepts but still exhibits weaker performance in low-resource languages, such as Tamil and Swahili, suggesting a promising solution for future visual cultural benchmark construction. |
|
TL;DR This work shows that the larger neural language models get, the more their representations are structurally similar to neural response measurements from brain imaging. |
|
TL;DR We explore the issue of copyright violations and large language models through the lens of verbatim memorization, focusing on possible redistribution of copyrighted text. |
|
TL;DR We probe ChatGPT for its conversational understanding and introduces a conversational framework (protocol) that can be adopted in future studies to assess ChatGPT's ability to generalize, combine features, and to acquire and reason over newly introduced knowledge from human feedback. |
|
|
Stolen from Jon Barron. Big thanks!
|
