LIME for text classification

This is an example of the LIME explainer on text classification. If using this explainer, please cite the original work: https://github.com/marcotcr/lime.

import numpy as np
import sklearn.ensemble
from sklearn.datasets import fetch_20newsgroups

from omnixai.data.text import Text
from omnixai.explainers.nlp import LimeText
from omnixai.preprocessing.text import Tfidf

We use a Text object to represent a batch of texts/sentences. The package omnixai.preprocessing.text provides some transforms related to text data such as Tfidf.

# Load the training and text datasets
categories = ['alt.atheism', 'soc.religion.christian']
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)

x_train = Text(newsgroups_train.data)
y_train = newsgroups_train.target
x_test = Text(newsgroups_test.data)
y_test = newsgroups_test.target
class_names = ['atheism', 'christian']
# A TFDIF transform
transform = Tfidf().fit(x_train)

For this classification task, we train a random forest classifier with TF-IDF feature vectors.

train_vectors = transform.transform(x_train)
test_vectors = transform.transform(x_test)
model = sklearn.ensemble.RandomForestClassifier(n_estimators=500)
model.fit(train_vectors, y_train)
predict_function = lambda x: model.predict_proba(transform.transform(x))

predictions = model.predict(test_vectors)
print('Test accuracy: {}'.format(
    sklearn.metrics.f1_score(y_test, predictions, average='binary')))

Test accuracy: 0.9230769230769231

To initialize LimeText, we need to set the following parameters:

• predict_function: The prediction function corresponding to the machine learning model to explain. For classification tasks, the outputs of the predict_function are the class probabilities.

idx = 83
explainer = LimeText(predict_function=predict_function)
explanations = explainer.explain(x_test[idx:idx+4])
explanations.ipython_plot(index=0, class_names=class_names)

Instance 0: Class atheism

Host NNTP Posting edu Organization an Lines post Subject anyone

explanations.ipython_plot(index=1, class_names=class_names)

Instance 1: Class christian

Christians sin murder of the writes Re said more nation