| Package | Description |
|---|---|
| com.atilika.kuromoji.ipadic | |
| org.deeplearning4j.text.tokenization.tokenizer | |
| org.deeplearning4j.text.tokenization.tokenizerfactory |
| Class and Description |
|---|
| Token
IPADIC token produced by the IPADIC tokenizer with various morphological features
|
| Tokenizer
A tokenizer based on the IPADIC dictionary
See
Token for details on the morphological features produced by this tokenizer
The following code example demonstrates how to use the Kuromoji tokenizer:
{@code
package com.atilika.kuromoji.example;
import com.atilika.kuromoji.ipadic.Token;
import com.atilika.kuromoji.ipadic.Tokenizer;
import java.util.List;
public class KuromojiExample {
public static void main(String[] args) {
Tokenizer tokenizer = new Tokenizer() ;
List |
| Tokenizer.Builder
Builder class for creating a customized tokenizer instance
|
| Class and Description |
|---|
| Tokenizer
A tokenizer based on the IPADIC dictionary
See
Token for details on the morphological features produced by this tokenizer
The following code example demonstrates how to use the Kuromoji tokenizer:
{@code
package com.atilika.kuromoji.example;
import com.atilika.kuromoji.ipadic.Token;
import com.atilika.kuromoji.ipadic.Tokenizer;
import java.util.List;
public class KuromojiExample {
public static void main(String[] args) {
Tokenizer tokenizer = new Tokenizer() ;
List |
| Class and Description |
|---|
| Tokenizer.Builder
Builder class for creating a customized tokenizer instance
|
Copyright © 2020. All rights reserved.