Published in arXiv, 2024
In this work, we released Collaborative Gym (Co-Gym), a general framework enabling asynchronous, tripartite interaction among agents, humans, and task environments across a multitude of tasks.
Published in COLING, 2024
In this work we benchmark several of the top data contamination approaches on the state-of-the-art LLMs in order to provide insight to the strengths and limitations of each approach.
Published in arXiv, 2024
In this work, we released the first ever dynamic evaluation framework for persona agents in LLMs termed PersonaGym. Additionally we established the first autmatic human-sligned metric for quantifying the abilities of a given persona agent termed PersonaScore.
Published in ACL Student Research Workshop, 2024
In this work, we tackle the low resource issue in certain QA datasets by synthetically augmenting them using GPT-4 to improve downstream fine-tuned performance for LMs trained on the augmented dataset.
Published in ACL Findings, 2024
In this work we introduce the first ever publicly available dataset for implicit attribute value extraction. In addition to releasing the curated dataset we also benchmark several SOTA Multimodal Large Language Models on ImplicitAVE.