Data Efficiency

SMART: Submodular Data Mixture Strategy for Instruction Tuning
Instruction Tuning involves finetuning a language model on a collection of instruction-formatted datasets in order to enhance the …
PDF Cite Poster Video DOI
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models
A salient characteristic of pre-trained language models (PTLMs) is a remarkable improvement in their generalization capability and …
PDF Cite Poster Video DOI