Method Detail

• incrementNumDocs

  void incrementNumDocs(double by)

  Increment the number of documents

  Parameters:

  by -

• numDocs

  double numDocs()

  Number of documents

  Returns:

  the number of documents

• wordAt

  String wordAt(int i)

  Returns a word in the vocab at a particular index

  Parameters:

  i - the index to get

  Returns:

  the word at that index in the vocab

• wordIndex

  int wordIndex(String word)

• initialize

  void initialize(Configuration conf)

  Configuration for initializing

  Parameters:

  conf - the configuration to initialize with

• wordFrequency

  double wordFrequency(String word)

  Get the word frequency for a word

  Parameters:

  word - the word to get frequency for

  Returns:

  the frequency for a given word

• minWordFrequency

  int minWordFrequency()

  The min word frequency needed to be included in the vocab (default 5)

  Returns:

  the min word frequency to be included in the vocab

• vocabWords

  Index vocabWords()

  All of the vocab words (ordered) note that these are not all the possible tokens

  Returns:

  the list of vocab words

• incrementDocCount

  void incrementDocCount(String word)

  Increment the doc count for a word by 1

  Parameters:

  word - the word to increment the count for

• incrementDocCount

  void incrementDocCount(String word,
                         double by)

  Increment the document count for a particular word

  Parameters:

  word - the word to increment the count for

  by - the amount to increment by

• incrementCount

  void incrementCount(String word)

  Increment a word count by 1

  Parameters:

  word - the word to increment the count for

• incrementCount

  void incrementCount(String word,
                      double by)

  Increment count for a word

  Parameters:

  word - the word to increment the count for

  by - the amount to increment by

• idf

  double idf(String word)

  Number of documents word has occurred in

  Parameters:

  word - the word to get the idf for

• tfidf

  double tfidf(String word,
               double frequency,
               boolean smoothIdf)

  Calculate the tfidf of the word given the document frequency

  Parameters:

  word - the word to get frequency for

  frequency - the frequency

  Returns:

  the tfidf for a word