org.deeplearning4j.nlp.uima.tokenization.tokenizer.preprocessor

Class EmbeddedStemmingPreprocessor

java.lang.Object

org.deeplearning4j.nlp.uima.tokenization.tokenizer.preprocessor.EmbeddedStemmingPreprocessor

All Implemented Interfaces:

TokenPreProcess

public class EmbeddedStemmingPreprocessor
extends Object
implements TokenPreProcess

This tokenizer preprocessor uses given preprocessor + does english Porter stemming on tokens on top of it

Author:

[email protected]

Constructor Summary

Constructors

Constructor and Description
EmbeddedStemmingPreprocessor(@NonNull TokenPreProcess preProcess)

Method Summary

Modifier and Type	Method and Description
String	preProcess(String token) Pre process a token

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EmbeddedStemmingPreprocessor

public EmbeddedStemmingPreprocessor(@NonNull
                                    @NonNull TokenPreProcess preProcess)

Method Detail

preProcess

public String preProcess(String token)

Description copied from interface: TokenPreProcess

Pre process a token

Specified by:

preProcess in interface TokenPreProcess

Parameters:

token - the token to pre process

Returns:

the preprocessed token