| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<VocabWord> |
TfidfVectorizer.Builder.vocabCache |
protected VocabCache<VocabWord> |
BaseTextVectorizer.vocabCache |
protected VocabCache<VocabWord> |
BagOfWordsVectorizer.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<VocabWord> |
TextVectorizer.getVocabCache()
The vocab sorted in descending order
|
| Modifier and Type | Method and Description |
|---|---|
TfidfVectorizer.Builder |
TfidfVectorizer.Builder.setVocab(VocabCache<VocabWord> vocab) |
BagOfWordsVectorizer.Builder |
BagOfWordsVectorizer.Builder.setVocab(VocabCache<VocabWord> vocab) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<T> |
WeightLookupTable.getVocabCache()
Returns corresponding vocabulary
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
InMemoryLookupTable.vocab |
protected VocabCache<T> |
InMemoryLookupTable.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
InMemoryLookupTable.getVocab() |
VocabCache<T> |
InMemoryLookupTable.getVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
InMemoryLookupTable.Builder<T> |
InMemoryLookupTable.Builder.cache(VocabCache<T> vocab) |
void |
InMemoryLookupTable.setVocab(VocabCache vocab) |
| Constructor and Description |
|---|
InMemoryLookupTable(VocabCache<T> vocab,
int vectorLength,
boolean useAdaGrad,
double lr,
Random gen,
double negative) |
InMemoryLookupTable(VocabCache<T> vocab,
int vectorLength,
boolean useAdaGrad,
double lr,
Random gen,
double negative,
boolean useHS) |
| Modifier and Type | Method and Description |
|---|---|
void |
SequenceLearningAlgorithm.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
void |
ElementsLearningAlgorithm.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
SkipGram.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
SkipGram.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration)
SkipGram initialization over given vocabulary and WeightLookupTable
|
void |
GloVe.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
void |
CBOW.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
DBOW.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
DM.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
void |
DBOW.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Method and Description |
|---|---|
static VocabCache<VocabWord> |
WordVectorSerializer.readVocabCache(File file)
This method reads vocab cache from provided file.
|
static VocabCache<VocabWord> |
WordVectorSerializer.readVocabCache(InputStream stream)
This method reads vocab cache from provided InputStream.
|
| Modifier and Type | Method and Description |
|---|---|
static Pair<InMemoryLookupTable,VocabCache> |
WordVectorSerializer.loadTxt(File vectorsFile)
Loads an in memory cache from the given path (sets syn0 and the vocab)
|
| Modifier and Type | Method and Description |
|---|---|
static WordVectors |
WordVectorSerializer.fromTableAndVocab(WeightLookupTable table,
VocabCache vocab)
Load word vectors for the given vocab and table
|
static void |
WordVectorSerializer.writeVocabCache(VocabCache<VocabWord> vocabCache,
File file)
This method saves vocab cache to provided File.
|
static void |
WordVectorSerializer.writeVocabCache(VocabCache<VocabWord> vocabCache,
OutputStream stream)
This method saves vocab cache to provided OutputStream.
|
| Modifier and Type | Method and Description |
|---|---|
static Word2Vec |
WordVectorSerializer.fromPair(Pair<InMemoryLookupTable,VocabCache> pair)
Load word vectors from the given pair
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
BasicModelUtils.vocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
WordVectorsImpl.vocab |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<T> |
WordVectorsImpl.vocab() |
VocabCache |
WordVectors.vocab()
Vocab for the vectors
|
| Modifier and Type | Method and Description |
|---|---|
void |
WordVectorsImpl.setVocab(VocabCache vocab) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
FastText.vocab() |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
AbstractCoOccurrences.vocabCache |
protected VocabCache<T> |
AbstractCoOccurrences.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
GloveWeightLookupTable.Builder<T> |
GloveWeightLookupTable.Builder.cache(VocabCache<T> vocab) |
static GloveWeightLookupTable |
GloveWeightLookupTable.load(InputStream is,
VocabCache<? extends SequenceElement> vocab)
Deprecated.
Load a glove model from an input stream.
|
AbstractCoOccurrences.Builder<T> |
AbstractCoOccurrences.Builder.vocabCache(VocabCache<T> cache) |
Glove.Builder |
Glove.Builder.vocabCache(VocabCache<VocabWord> vocabCache) |
| Constructor and Description |
|---|
GloveWeightLookupTable(VocabCache<T> vocab,
int vectorLength,
boolean useAdaGrad,
double lr,
Random gen,
double negative,
double xMax,
double maxCount)
Deprecated.
|
| Constructor and Description |
|---|
ASCIICoOccurrenceReader(File file,
VocabCache<T> vocabCache) |
BinaryCoOccurrenceReader(File file,
VocabCache<T> vocabCache,
CountMap<T> map) |
| Modifier and Type | Method and Description |
|---|---|
Node2Vec.Builder<V,E> |
Node2Vec.Builder.vocabCache(VocabCache<V> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
ParagraphVectors.Builder |
ParagraphVectors.Builder.vocabCache(VocabCache<VocabWord> vocabCache)
This method allows to define external VocabCache to be used
|
| Constructor and Description |
|---|
BlindInferenceCallable(VocabCache<VocabWord> vocabCache,
TokenizerFactory tokenizerFactory,
String document) |
BlindInferenceCallable(VocabCache<VocabWord> vocabCache,
TokenizerFactory tokenizerFactory,
String document,
AtomicLong flag) |
InferenceCallable(VocabCache<VocabWord> vocabCache,
TokenizerFactory tokenizerFactory,
LabelledDocument document) |
InferenceCallable(VocabCache<VocabWord> vocabCache,
TokenizerFactory tokenizerFactory,
LabelledDocument document,
AtomicLong flag) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
SequenceVectors.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
SequenceVectors.Builder<T> |
SequenceVectors.Builder.vocabCache(VocabCache<T> vocabCache)
You can pass externally built vocabCache object, containing vocabulary
|
| Constructor and Description |
|---|
FilteredSequenceIterator(SequenceIterator<T> iterator,
VocabCache<T> vocabCache)
Creates Filtered SequenceIterator on top of another SequenceIterator and appropriate VocabCache instance
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<VocabWord> |
SentenceTransformer.Builder.vocabCache |
protected VocabCache<T> |
GraphTransformer.vocabCache |
protected VocabCache<T> |
GraphTransformer.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
GraphTransformer.Builder<T> |
GraphTransformer.Builder.setVocabCache(VocabCache<T> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
StaticWord2Vec.vocab()
Vocab for the vectors
|
| Modifier and Type | Method and Description |
|---|---|
void |
Huffman.applyIndexes(VocabCache<? extends SequenceElement> cache)
This method updates VocabCache and all it's elements with Huffman indexes
Please note: it should be the same VocabCache as was used for Huffman tree initialization
|
Word2Vec.Builder |
Word2Vec.Builder.vocabCache(VocabCache<VocabWord> vocabCache)
This method allows to define external VocabCache to be used
|
| Constructor and Description |
|---|
Builder(AbstractStorage<Integer> storage,
VocabCache<VocabWord> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<T> |
VocabConstructor.buildExtendedVocabulary()
Placeholder for future implementation
|
VocabCache<T> |
VocabConstructor.buildJointVocabulary(boolean resetCounters,
boolean buildHuffmanTree)
This method scans all sources passed through builder, and returns all words as vocab.
|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(VocabCache<T> vocabCache,
boolean fetchLabels)
This method transfers existing vocabulary into current one
Please note: this method expects source vocabulary has Huffman tree indexes applied
|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(WordVectors wordVectors,
boolean fetchLabels)
This method transfers existing WordVectors model into current one
|
VocabCache<T> |
VocabConstructor.transferVocabulary(VocabCache<T> vocabCache,
boolean buildHuffman) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(VocabCache<T> vocabCache,
boolean fetchLabels)
This method transfers existing vocabulary into current one
Please note: this method expects source vocabulary has Huffman tree indexes applied
|
VocabularyHolder.Builder |
VocabularyHolder.Builder.externalCache(VocabCache cache) |
void |
VocabCache.importVocabulary(VocabCache<T> vocabCache)
imports vocabulary
|
VocabConstructor.Builder<T> |
VocabConstructor.Builder.setTargetVocabCache(VocabCache<T> cache)
After temporary internal vocabulary is built, it will be transferred to target VocabCache you pass here
|
void |
VocabularyHolder.transferBackToVocabCache(VocabCache cache) |
void |
VocabularyHolder.transferBackToVocabCache(VocabCache cache,
boolean emptyHolder)
This method is required for compatibility purposes.
|
VocabCache<T> |
VocabConstructor.transferVocabulary(VocabCache<T> vocabCache,
boolean buildHuffman) |
| Constructor and Description |
|---|
VocabularyHolder(VocabCache<? extends SequenceElement> cache,
boolean markAsSpecial)
Builds VocabularyHolder from VocabCache.
|
| Modifier and Type | Class and Description |
|---|---|
class |
AbstractCache<T extends SequenceElement>
This is generic VocabCache implementation designed as abstract SequenceElements vocabulary
|
class |
InMemoryLookupCache
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
void |
AbstractCache.importVocabulary(VocabCache<T> vocabCache)
This method imports all elements from VocabCache passed as argument
If element already exists,
|
void |
InMemoryLookupCache.importVocabulary(VocabCache<VocabWord> vocabCache)
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
Pair<VocabCache<VocabWord>,GloveWeightLookupTable> |
Glove.train(org.apache.spark.api.java.JavaRDD<String> rdd)
Train on the corpus
|
| Constructor and Description |
|---|
VocabWordPairs(org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocab) |
| Constructor and Description |
|---|
CoOccurrenceCalculator(boolean symmetric,
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocab,
int windowSize) |
| Modifier and Type | Method and Description |
|---|---|
Iterable<Map.Entry<VocabWord,INDArray>> |
VocabHolder.getSplit(VocabCache<VocabWord> vocabCache) |
INDArray |
VocabHolder.getSyn0Vector(Integer wordIndex,
VocabCache<VocabWord> vocabCache) |
void |
NegativeHolder.initHolder(VocabCache<VocabWord> vocabCache,
double[] expTable,
int layerSize) |
| Constructor and Description |
|---|
FirstIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast,
org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast,
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
SecondIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast,
org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast,
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkParagraphVectors.getShallowVocabCache() |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.shallowVocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
SparkSequenceVectors.shallowVocabCacheBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<T>> |
SparkSequenceVectors.vocabCacheBroadcast |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.buildShallowVocabCache(Counter<Long> counter)
This method builds shadow vocabulary and huffman tree
|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.getShallowVocabCache() |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<VocabWord> |
VocabCacheExporter.vocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
TrainingFunction.shallowVocabCache |
protected VocabCache<ShallowSequenceElement> |
PartitionTrainingFunction.shallowVocabCache |
protected VocabCache<ShallowSequenceElement> |
DistributedFunction.shallowVocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
DistributedFunction.shallowVocabBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
TrainingFunction.vocabCacheBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
PartitionTrainingFunction.vocabCacheBroadcast |
| Constructor and Description |
|---|
DistributedFunction(org.apache.spark.broadcast.Broadcast<VoidConfiguration> configurationBroadcast,
org.apache.spark.broadcast.Broadcast<VectorsConfiguration> vectorsConfigurationBroadcast,
org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> shallowVocabBroadcast) |
ExportFunction(org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> vocabCacheBroadcast,
String hdfsFilePath) |
PartitionTrainingFunction(org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> vocabCacheBroadcast,
org.apache.spark.broadcast.Broadcast<VectorsConfiguration> vectorsConfigurationBroadcast,
org.apache.spark.broadcast.Broadcast<VoidConfiguration> paramServerConfigurationBroadcast) |
TrainingFunction(org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> vocabCacheBroadcast,
org.apache.spark.broadcast.Broadcast<VectorsConfiguration> vectorsConfigurationBroadcast,
org.apache.spark.broadcast.Broadcast<VoidConfiguration> paramServerConfigurationBroadcast) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
BaseSparkLearningAlgorithm.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
BaseSparkLearningAlgorithm.configure(VocabCache<ShallowSequenceElement> vocabCache,
WeightLookupTable<ShallowSequenceElement> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
BaseSparkSequenceLearningAlgorithm.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
BaseSparkSequenceLearningAlgorithm.configure(VocabCache<ShallowSequenceElement> vocabCache,
WeightLookupTable<ShallowSequenceElement> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkWord2Vec.getShallowVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<VocabWord> |
TextPipeline.getVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> |
TextPipeline.getBroadCastVocabCache() |
| Constructor and Description |
|---|
WordsListToVocabWordsFunction(org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
Copyright © 2020. All rights reserved.