Lora (4) 썸네일형 리스트형 [논문이해] LORA-FA: MEMORY-EFFICIENT LOW-RANK ADAPTATION FOR LARGE LANGUAGE MODELS FINE-TUNING 논문명: LORA-FA: MEMORY-EFFICIENT LOW-RANK ADAPTATION FOR LARGE LANGUAGE MODELS FINE-TUNING논문 링크: https://arxiv.org/abs/2308.03303 LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuningThe low-rank adaptation (LoRA) method can largely reduce the amount of trainable parameters for fine-tuning large language models (LLMs), however, it still requires expensive activation m.. [논문이해] LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models 논문명: LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models 논문링크: https://arxiv.org/abs/2309.12307 LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models We present LongLoRA, an efficient fine-tuning approach that extends the context sizes of pre-trained large language models (LLMs), with limited computation cost. Typically, training LLMs with long context sizes is .. [huggingface🤗] Prompting? PEFT? 총정리 계기 NLP 분야를 공부하다 보면, Prompting, prompt tuning, soft prompt, p-tuning, prefix-tuning, In-Context Learning 등 다양한 용어들 때문에 헷갈린다. 하지만 새로운 분야와 방법이 등장하며, 점차 자리를 잡아가는 과정이기 때문에 이런 혼란스러움은 필연적이다. 그래도 한번 정리할 필요가 있겠다 싶어서 나름 정리 해봤다. 아무리 검색해봐도 제대로 정리한 이미지는 찾지 못했다. 기준 NLP 분야에서 어느 정도 권위가 있는 huggingface 문서를 따랐다. 개인적인 의견이 담긴 블로그나 논문보다는 그나마 가장 객관적이라 판단하였다. https://huggingface.co/docs/peft/conceptual_guides/prompting.. [huggingface🤗] Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA 이 글은 huggingface blog 의 'Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA' 이라는 글을 의역한 것입니다. https://huggingface.co/blog/4bit-transformers-bitsandbytes Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA LLMs are known to be large, and running or training t.. 이전 1 다음
