A1316
Title: Investigating finite-sample properties of vector embeddings of black-box LLMs
Authors: Aranyak Acharyya - Johns Hopkins University (United States) [presenting]
Joshua Agterberg - University of Illinois Urbana-Champaign (United States)
Youngser Park - Johns Hopkins University (United States)
Carey Priebe - Johns Hopkins University (United States)
Abstract: Large language models are becoming increasingly prevalent across multiple domains, yet statistical inference on black box LLMs remains challenging due to their opaque internal mechanisms. Analyzing responses to user-given queries enables statistical inference on black box LLMs. A response-based vector representation is constructed for each LLM to facilitate inference tasks. This study investigates the finite-sample properties of vector representations obtained through Data Kernel Perspective Space Embedding. Theoretical results are established and validated through simulations and real Data experiments.
