TokenizerFunction

java.lang.Object
- org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
- - org.deeplearning4j.spark.models.sequencevectors.functions.TokenizerFunction

All Implemented Interfaces:: java.io.Serializable, org.apache.spark.api.java.function.Function<java.lang.String,Sequence<VocabWord>>

public class TokenizerFunction
extends BaseTokenizerFunction
implements org.apache.spark.api.java.function.Function<java.lang.String,Sequence<VocabWord>>

See Also:: Serialized Form

- Field Summary
  - Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
    configurationBroadcast, tokenizerFactory, tokenPreprocessor
- Constructor Summary
  
  Constructors
  Constructor and Description
  
  TokenizerFunction(org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)
- Method Summary
  
  All Methods Instance Methods Concrete Methods
  Modifier and Type Method and Description
  
  Sequence<VocabWord> call(java.lang.String s)
  - Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
    instantiateTokenizerFactory
  - Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructors
Constructor and Description
`TokenizerFunction(org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)`

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`Sequence<VocabWord>`	`call(java.lang.String s)`

Constructor Detail

TokenizerFunction

public TokenizerFunction(@NonNull
                         org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)

Method Detail
- call
```
public Sequence<VocabWord> call(java.lang.String s)
                         throws java.lang.Exception
```
  Specified by:
  
  call in interface org.apache.spark.api.java.function.Function<java.lang.String,Sequence<VocabWord>>
  
  Throws:
  
  java.lang.Exception

Class TokenizerFunction

Field Summary

Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction

Constructor Summary

Method Summary

Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction

Methods inherited from class java.lang.Object

Constructor Detail

TokenizerFunction

Method Detail

call