botiverse.preprocessors.TF_IDF package#

Submodules#

botiverse.preprocessors.TF_IDF.TF_IDF module#

class botiverse.preprocessors.TF_IDF.TF_IDF.TF_IDF[source]#

Bases: object

An interface for transforming sentences into tf-idf vectors.

transform_list(sentence_list, all_words)[source]#

Given a list of tokenized sentences, return a table of tf-idf vectors (one for each sentence) in the form of a numpy array.

Parameters:

  • sentence_list (list) – A list of tokenized sentences

  • all_words (list) – A list of all the words in the corpus

Returns:

A numpy array of tf-idf vectors

Return type:

numpy.ndarray

transform(sentence)[source]#

Given a sentence, return its tf-idf vector as a numpy array.

Parameters:

sentence (str) – A string of words

Returns:

A numpy array of the tf-idf vector

Return type:

numpy.ndarray

Module contents#