ACL2023 (2) 썸네일형 리스트형 [논문이해] Should You Mask 15% in Masked Language Modeling? 논문명: Should You Mask 15% in Masked Language Modeling? 논문링크: https://arxiv.org/abs/2202.08005 Should You Mask 15% in Masked Language Modeling? Masked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rate has been widely used, regardless of model sizes or masking strategies. In .. [논문이해] Pre-Training to Learn in Context 논문명: Pre-Training to Learn in Context 논문링크: https://arxiv.org/abs/2305.09137 Pre-Training to Learn in Context In-context learning, where pre-trained language models learn to perform tasks from task examples and instructions in their contexts, has attracted much attention in the NLP community. However, the ability of in-context learning is not fully exploited becau arxiv.org 아이디어만 정리합니다. 아이디어 기존 .. 이전 1 다음
