• January 5, 2025
• in deep-learning, pytorch, transformers
• 3 min read

Finetune Transformers

Learning how to finetune a BERT model using PyTorch/TensorFlow from HuggingFace for your usecase is a art in itself because there are so many ways and methods to do it that you will not able to figure out which is the best for my usecase. BTW, you can always refer to HuggingFace documentation.

For example!

1. Choose between PyTorch and TensorFlow. (let choose )
2. If you are importing your dataset with pandas or polars then need to create a custom class by inheriting torch.utils.data.Dataset class.
3. Then need to tokenize the data and need to use DataLoader and Data Collator.
4. Then use a for-loop to train and validate the model.

But there is a easy way to finetune, by using objects like transformers.TrainingArguments and transformers.Trainer which reduces the manual looping complexity.