I am working on a text classification problem and would like to use the BERT model to improve my results. I have read about the BERT model but I am unsure about how to implement it in Python for my specific use case.
Here’s the code I’m currently using with a simple logistic regression model:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
text_clf = Pipeline([
('tfidf', TfidfVectorizer()),
('clf', LogisticRegression()),
])
text_clf.fit(X_train, y_train)
predictions = text_clf.predict(X_test)
Can someone provide an example of how I might be able to replace this with a BERT model for text classification?
>Solution :
hi my friend welcome to stackoverflow use the transformers library in Python, Here’s a basic example of how you might use BERT for text classification
from transformers import BertTokenizer, BertForSequenceClassification
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
input_ids = tokenizer.encode("Here is some text to classify", add_special_tokens=True)
input_ids = torch.tensor(input_ids).unsqueeze(0) # Batch size 1
outputs = model(input_ids)
_, predicted_class = torch.max(outputs.logits, dim=1)
print(predicted_class)
checking out the transformers library documentation