| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<VocabWord> |
BagOfWordsVectorizer.Builder.vocabCache |
protected VocabCache<VocabWord> |
BaseTextVectorizer.vocabCache |
protected VocabCache<VocabWord> |
TfidfVectorizer.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<VocabWord> |
TextVectorizer.getVocabCache()
The vocab sorted in descending order
|
| Modifier and Type | Method and Description |
|---|---|
BagOfWordsVectorizer.Builder |
BagOfWordsVectorizer.Builder.setVocab(@NonNull VocabCache<VocabWord> vocab) |
TfidfVectorizer.Builder |
TfidfVectorizer.Builder.setVocab(@NonNull VocabCache<VocabWord> vocab) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<T> |
WeightLookupTable.getVocabCache()
Returns corresponding vocabulary
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
InMemoryLookupTable.vocab |
protected VocabCache<T> |
InMemoryLookupTable.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
InMemoryLookupTable.getVocab() |
VocabCache<T> |
InMemoryLookupTable.getVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
InMemoryLookupTable.Builder<T> |
InMemoryLookupTable.Builder.cache(@NonNull VocabCache<T> vocab) |
void |
InMemoryLookupTable.setVocab(VocabCache vocab) |
| Constructor and Description |
|---|
InMemoryLookupTable(VocabCache<T> vocab,
int vectorLength,
boolean useAdaGrad,
double lr,
Random gen,
double negative) |
InMemoryLookupTable(VocabCache<T> vocab,
int vectorLength,
boolean useAdaGrad,
double lr,
Random gen,
double negative,
boolean useHS) |
| Modifier and Type | Method and Description |
|---|---|
void |
ElementsLearningAlgorithm.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
void |
SequenceLearningAlgorithm.configure(VocabCache<T> vocabCache,
WeightLookupTable<T> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
SkipGram.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
CBOW.configure(@NonNull VocabCache<T> vocabCache,
@NonNull WeightLookupTable<T> lookupTable,
@NonNull VectorsConfiguration configuration) |
void |
SkipGram.configure(@NonNull VocabCache<T> vocabCache,
@NonNull WeightLookupTable<T> lookupTable,
@NonNull VectorsConfiguration configuration)
SkipGram initialization over given vocabulary and WeightLookupTable
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
DBOW.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
DBOW.configure(@NonNull VocabCache<T> vocabCache,
@NonNull WeightLookupTable<T> lookupTable,
@NonNull VectorsConfiguration configuration) |
void |
DM.configure(@NonNull VocabCache<T> vocabCache,
@NonNull WeightLookupTable<T> lookupTable,
@NonNull VectorsConfiguration configuration) |
| Modifier and Type | Method and Description |
|---|---|
static VocabCache<VocabWord> |
WordVectorSerializer.readVocabCache(@NonNull File file)
This method reads vocab cache from provided file.
|
static VocabCache<VocabWord> |
WordVectorSerializer.readVocabCache(@NonNull InputStream stream)
This method reads vocab cache from provided InputStream.
|
| Modifier and Type | Method and Description |
|---|---|
static Pair<InMemoryLookupTable,VocabCache> |
WordVectorSerializer.loadTxt(@NonNull File file) |
static Pair<InMemoryLookupTable,VocabCache> |
WordVectorSerializer.loadTxt(@NonNull InputStream inputStream)
Loads an in memory cache from the given input stream (sets syn0 and the vocab).
|
| Modifier and Type | Method and Description |
|---|---|
static WordVectors |
WordVectorSerializer.fromTableAndVocab(WeightLookupTable table,
VocabCache vocab)
Load word vectors for the given vocab and table
|
static void |
WordVectorSerializer.writeVocabCache(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull File file)
This method saves vocab cache to provided File.
|
static void |
WordVectorSerializer.writeVocabCache(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull OutputStream stream)
This method saves vocab cache to provided OutputStream.
|
| Modifier and Type | Method and Description |
|---|---|
static Word2Vec |
WordVectorSerializer.fromPair(Pair<InMemoryLookupTable,VocabCache> pair)
Load word vectors from the given pair
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
BasicModelUtils.vocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
WordVectorsImpl.vocab |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
WordVectors.vocab()
Vocab for the vectors
|
VocabCache<T> |
WordVectorsImpl.vocab() |
| Modifier and Type | Method and Description |
|---|---|
void |
WordVectorsImpl.setVocab(VocabCache vocab) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
FastText.vocab() |
| Modifier and Type | Method and Description |
|---|---|
Node2Vec.Builder<V,E> |
Node2Vec.Builder.vocabCache(@NonNull VocabCache<V> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
ParagraphVectors.Builder |
ParagraphVectors.Builder.vocabCache(@NonNull VocabCache<VocabWord> vocabCache)
This method allows to define external VocabCache to be used
|
| Constructor and Description |
|---|
BlindInferenceCallable(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull TokenizerFactory tokenizerFactory,
@NonNull String document) |
BlindInferenceCallable(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull TokenizerFactory tokenizerFactory,
@NonNull String document,
@NonNull AtomicLong flag) |
InferenceCallable(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull TokenizerFactory tokenizerFactory,
@NonNull LabelledDocument document) |
InferenceCallable(@NonNull VocabCache<VocabWord> vocabCache,
@NonNull TokenizerFactory tokenizerFactory,
@NonNull LabelledDocument document,
@NonNull AtomicLong flag) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
SequenceVectors.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
SequenceVectors.Builder<T> |
SequenceVectors.Builder.vocabCache(@NonNull VocabCache<T> vocabCache)
You can pass externally built vocabCache object, containing vocabulary
|
| Constructor and Description |
|---|
FilteredSequenceIterator(@NonNull SequenceIterator<T> iterator,
@NonNull VocabCache<T> vocabCache)
Creates Filtered SequenceIterator on top of another SequenceIterator and appropriate VocabCache instance
|
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<T> |
GraphTransformer.vocabCache |
protected VocabCache<T> |
GraphTransformer.Builder.vocabCache |
protected VocabCache<VocabWord> |
SentenceTransformer.Builder.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
GraphTransformer.Builder<T> |
GraphTransformer.Builder.setVocabCache(@NonNull VocabCache<T> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache |
StaticWord2Vec.vocab()
Vocab for the vectors
|
| Modifier and Type | Method and Description |
|---|---|
void |
Huffman.applyIndexes(VocabCache<? extends SequenceElement> cache)
This method updates VocabCache and all it's elements with Huffman indexes
Please note: it should be the same VocabCache as was used for Huffman tree initialization
|
Word2Vec.Builder |
Word2Vec.Builder.vocabCache(@NonNull VocabCache<VocabWord> vocabCache)
This method allows to define external VocabCache to be used
|
| Constructor and Description |
|---|
Builder(AbstractStorage<Integer> storage,
VocabCache<VocabWord> vocabCache) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<T> |
VocabConstructor.buildExtendedVocabulary()
Placeholder for future implementation
|
VocabCache<T> |
VocabConstructor.buildJointVocabulary(boolean resetCounters,
boolean buildHuffmanTree)
This method scans all sources passed through builder, and returns all words as vocab.
|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(@NonNull VocabCache<T> vocabCache,
boolean fetchLabels)
This method transfers existing vocabulary into current one
Please note: this method expects source vocabulary has Huffman tree indexes applied
|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(@NonNull WordVectors wordVectors,
boolean fetchLabels)
This method transfers existing WordVectors model into current one
|
VocabCache<T> |
VocabConstructor.transferVocabulary(@NonNull VocabCache<T> vocabCache,
boolean buildHuffman) |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<T> |
VocabConstructor.buildMergedVocabulary(@NonNull VocabCache<T> vocabCache,
boolean fetchLabels)
This method transfers existing vocabulary into current one
Please note: this method expects source vocabulary has Huffman tree indexes applied
|
VocabularyHolder.Builder |
VocabularyHolder.Builder.externalCache(@NonNull VocabCache cache) |
void |
VocabCache.importVocabulary(VocabCache<T> vocabCache)
imports vocabulary
|
VocabConstructor.Builder<T> |
VocabConstructor.Builder.setTargetVocabCache(@NonNull VocabCache<T> cache)
After temporary internal vocabulary is built, it will be transferred to target VocabCache you pass here
|
void |
VocabularyHolder.transferBackToVocabCache(VocabCache cache) |
void |
VocabularyHolder.transferBackToVocabCache(VocabCache cache,
boolean emptyHolder)
This method is required for compatibility purposes.
|
VocabCache<T> |
VocabConstructor.transferVocabulary(@NonNull VocabCache<T> vocabCache,
boolean buildHuffman) |
| Constructor and Description |
|---|
VocabularyHolder(@NonNull VocabCache<? extends SequenceElement> cache,
boolean markAsSpecial)
Builds VocabularyHolder from VocabCache.
|
| Modifier and Type | Class and Description |
|---|---|
class |
AbstractCache<T extends SequenceElement>
This is generic VocabCache implementation designed as abstract SequenceElements vocabulary
|
class |
InMemoryLookupCache
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
void |
AbstractCache.importVocabulary(@NonNull VocabCache<T> vocabCache)
This method imports all elements from VocabCache passed as argument
If element already exists,
|
void |
InMemoryLookupCache.importVocabulary(VocabCache<VocabWord> vocabCache)
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
Iterable<Map.Entry<VocabWord,INDArray>> |
VocabHolder.getSplit(VocabCache<VocabWord> vocabCache) |
INDArray |
VocabHolder.getSyn0Vector(Integer wordIndex,
VocabCache<VocabWord> vocabCache) |
void |
NegativeHolder.initHolder(@NonNull VocabCache<VocabWord> vocabCache,
double[] expTable,
int layerSize) |
| Constructor and Description |
|---|
FirstIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast,
org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast,
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
SecondIterationFunction(org.apache.spark.broadcast.Broadcast<Map<String,Object>> word2vecVarMapBroadcast,
org.apache.spark.broadcast.Broadcast<double[]> expTableBroadcast,
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkParagraphVectors.getShallowVocabCache() |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.shallowVocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
SparkSequenceVectors.shallowVocabCacheBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<T>> |
SparkSequenceVectors.vocabCacheBroadcast |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.buildShallowVocabCache(Counter<Long> counter)
This method builds shadow vocabulary and huffman tree
|
protected VocabCache<ShallowSequenceElement> |
SparkSequenceVectors.getShallowVocabCache() |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<VocabWord> |
VocabCacheExporter.vocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
DistributedFunction.shallowVocabCache |
protected VocabCache<ShallowSequenceElement> |
PartitionTrainingFunction.shallowVocabCache |
protected VocabCache<ShallowSequenceElement> |
TrainingFunction.shallowVocabCache |
| Modifier and Type | Field and Description |
|---|---|
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
DistributedFunction.shallowVocabBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
PartitionTrainingFunction.vocabCacheBroadcast |
protected org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> |
TrainingFunction.vocabCacheBroadcast |
| Constructor and Description |
|---|
ExportFunction(org.apache.spark.broadcast.Broadcast<VocabCache<ShallowSequenceElement>> vocabCacheBroadcast,
@NonNull String hdfsFilePath) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
BaseSparkLearningAlgorithm.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
BaseSparkLearningAlgorithm.configure(VocabCache<ShallowSequenceElement> vocabCache,
WeightLookupTable<ShallowSequenceElement> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Field and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
BaseSparkSequenceLearningAlgorithm.vocabCache |
| Modifier and Type | Method and Description |
|---|---|
void |
BaseSparkSequenceLearningAlgorithm.configure(VocabCache<ShallowSequenceElement> vocabCache,
WeightLookupTable<ShallowSequenceElement> lookupTable,
VectorsConfiguration configuration) |
| Modifier and Type | Method and Description |
|---|---|
protected VocabCache<ShallowSequenceElement> |
SparkWord2Vec.getShallowVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
VocabCache<VocabWord> |
TextPipeline.getVocabCache() |
| Modifier and Type | Method and Description |
|---|---|
org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> |
TextPipeline.getBroadCastVocabCache() |
| Constructor and Description |
|---|
WordsListToVocabWordsFunction(org.apache.spark.broadcast.Broadcast<VocabCache<VocabWord>> vocabCacheBroadcast) |
Copyright © 2020. All rights reserved.