TokenizerFunction (deeplearning4j 1.0.0-beta7 API)

java.lang.Object
- org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
- - org.deeplearning4j.spark.models.sequencevectors.functions.TokenizerFunction

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function<String,Sequence<VocabWord>>
```
public class TokenizerFunction
extends BaseTokenizerFunction
implements org.apache.spark.api.java.function.Function<String,Sequence<VocabWord>>
```
Author:

raver119@gmail.com

See Also:

Serialized Form

- Field Summary
  - Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
    configurationBroadcast, tokenizerFactory, tokenPreprocessor
- Constructor Summary
  
  Constructors
  Constructor and Description
  
  TokenizerFunction(@NonNull org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)
- Method Summary
  
  All Methods Instance Methods Concrete Methods
  Modifier and Type Method and Description
  
  Sequence<VocabWord> call(String s)
  - Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction
    instantiateTokenizerFactory
  - Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructors
Constructor and Description
`TokenizerFunction(@NonNull org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)`

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`Sequence<VocabWord>`	`call(String s)`

Constructor Detail

TokenizerFunction

public TokenizerFunction(@NonNull
                         @NonNull org.apache.spark.broadcast.Broadcast<VectorsConfiguration> configurationBroadcast)

Method Detail
- call
```
public Sequence<VocabWord> call(String s)
                         throws Exception
```
  Specified by:
  
  call in interface org.apache.spark.api.java.function.Function<String,Sequence<VocabWord>>
  
  Throws:
  
  Exception

Class TokenizerFunction

Field Summary

Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction

Constructor Summary

Method Summary

Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.functions.BaseTokenizerFunction

Methods inherited from class java.lang.Object

Constructor Detail

TokenizerFunction

Method Detail

call