In Context Learning (10) 썸네일형 리스트형 [논문이해] Query2doc: Query Expansion with Large Language Models 논문명: Query2doc: Query Expansion with Large Language Models 논문링크: https://arxiv.org/abs/2303.07678 Query2doc: Query Expansion with Large Language Models This paper introduces a simple yet effective query expansion approach, denoted as query2doc, to improve both sparse and dense retrieval systems. The proposed method first generates pseudo-documents by few-shot prompting large language models (LLM.. [huggingface🤗] Prompting? PEFT? 총정리 계기 NLP 분야를 공부하다 보면, Prompting, prompt tuning, soft prompt, p-tuning, prefix-tuning, In-Context Learning 등 다양한 용어들 때문에 헷갈린다. 하지만 새로운 분야와 방법이 등장하며, 점차 자리를 잡아가는 과정이기 때문에 이런 혼란스러움은 필연적이다. 그래도 한번 정리할 필요가 있겠다 싶어서 나름 정리 해봤다. 아무리 검색해봐도 제대로 정리한 이미지는 찾지 못했다. 기준 NLP 분야에서 어느 정도 권위가 있는 huggingface 문서를 따랐다. 개인적인 의견이 담긴 블로그나 논문보다는 그나마 가장 객관적이라 판단하였다. https://huggingface.co/docs/peft/conceptual_guides/prompting.. [논문이해] What learning algorithm is in-context learning? Investigations with linear models 논문명: What learning algorithm is in-context learning? Investigations with linear models 논문링크: https://arxiv.org/abs/2211.15661 What learning algorithm is in-context learning? Investigations with linear models Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples $(x, f(x))$ prese.. [논문이해] Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers 논문명: Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers 논문링크: https://arxiv.org/abs/2212.10559 Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers Large pretrained language models have shown surprising in-context learning (ICL) ability. With a few demonstration input-label pairs, they can predict t.. [논문이해] Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations 논문명: Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations 논문링크: https://arxiv.org/abs/2205.12685 Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations Despite recent explosion of interests in in-context learning, the underlying mechanism and the precise impact of the quality of demonstrations remain elusive. Intuitively, ground-truth labels should have as.. [논문이해] Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations 논문명:Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations 논문링크: https://arxiv.org/abs/2212.09865 Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations Although large language models can be prompted for both zero- and few-shot learning, performance drops significantly when no demonstrations are available. In this paper, we introduce Z-ICL, a new zero-shot method that closes th.. [논문이해] Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? 논문명: Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?논문 링크: https://arxiv.org/abs/2202.12837 Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?Large language models (LMs) are able to in-context learn -- perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. Howev.. [논문이해] Noisy Channel Language Model Prompting for Few-Shot Text Classification 논문명: Noisy Channel Language Model Prompting for Few-Shot Text Classification https://arxiv.org/abs/2108.04106 Noisy Channel Language Model Prompting for Few-Shot Text Classification We introduce a noisy channel approach for language model prompting in few-shot text classification. Instead of computing the likelihood of the label given the input (referred as direct models), channel models compute.. 이전 1 2 다음
