A new research paper titled 'Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs' has been published by Microsoft Research, Georgia Institute of Technology, and UC Berkeley. The paper introduces a method called PASTA, which downweights attention heads that are not emphasized by the user. The authors discuss the importance of textual emphases in guiding readers' attention and highlight the challenges in this area. The paper aims to provide a general formal framework for representational alignment and reviews the existing literature in the field of neuroscience.
[CL] Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs Q Zhang, C Singh, L Liu, X Liu, B Yu, J Gao, T Zhao [Microsoft Research & Georgia Institute of Technology & UC Berkeley] (2023) https://t.co/JNeIV4bNFD - The paper proposes PASTA, a method to steer the… https://t.co/zxyimFMk5B https://t.co/QFT6CRoB1O
Nice Paper - "Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs" In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for the readers to grasp… https://t.co/1AF4wHmWXG https://t.co/ATWFoJKgNR
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs paper page: https://t.co/kzGLjYbivO In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for… https://t.co/vXK8Sksrc9 https://t.co/arJgOTCwfy
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs abs: https://t.co/TE1Ru0V4CQ code: https://t.co/vkXZxJ4UNB This Microsoft paper introduces PASTA. PASTA downweights the attention heads that are not emphasized by the user, as determined through a model… https://t.co/x726DH8w6B https://t.co/hyQ794Z4j1
#PrePrint published, (v2): position & review paper on representational alignment. Outline: - find common language across research disciplines - provide a general formal framework - review the existing literature - discuss open challenges https://t.co/5p8jjN7stX #Neuroscience https://t.co/9YrQk0oQig