public class TfidfRecordReader extends FileRecordReader
appendLabel, charset, conf, currentUri, labels, locationsIteratorinputSplit, listeners, streamCreatorFnAPPEND_LABEL, LABELS, NAME_SPACE| Constructor and Description |
|---|
TfidfRecordReader() |
| Modifier and Type | Method and Description |
|---|---|
void |
close() |
Configuration |
getConf()
Return the configuration used by this object.
|
int |
getNumFeatures() |
TfidfVectorizer |
getTfidfVectorizer() |
boolean |
hasNext()
Whether there are anymore records
|
void |
initialize(Configuration conf,
InputSplit split)
Called once at initialization.
|
void |
initialize(InputSplit split)
Called once at initialization.
|
List<Record> |
loadFromMetaData(List<RecordMetaData> recordMetaDatas)
Load multiple records from the given a list of
RecordMetaData instances |
Record |
loadFromMetaData(RecordMetaData recordMetaData)
Load a single record from the given
RecordMetaData instanceNote: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using RecordReader.loadFromMetaData(List) |
List<Writable> |
next()
Get the next record
|
Record |
nextRecord()
Similar to
RecordReader.next(), but returns a Record object, that may include metadata such as the source
of the data |
void |
reset()
Reset record reader iterator
|
void |
setConf(Configuration conf)
Set the configuration to be used by this object.
|
void |
setTfidfVectorizer(TfidfVectorizer tfidfVectorizer) |
void |
shuffle() |
void |
shuffle(Random random) |
doInitialize, getCurrentLabel, getLabel, getLabels, next, record, resetSupported, setLabelsbatchesSupported, getListeners, invokeListeners, setListeners, setListenerspublic void initialize(InputSplit split) throws IOException, InterruptedException
RecordReaderinitialize in interface RecordReaderinitialize in class FileRecordReadersplit - the split that defines the range of records to readIOExceptionInterruptedExceptionpublic void initialize(Configuration conf, InputSplit split) throws IOException, InterruptedException
RecordReaderinitialize in interface RecordReaderinitialize in class FileRecordReaderconf - a configuration for initializationsplit - the split that defines the range of records to readIOExceptionInterruptedExceptionpublic void reset()
RecordReaderreset in interface RecordReaderreset in class FileRecordReaderpublic Record nextRecord()
RecordReaderRecordReader.next(), but returns a Record object, that may include metadata such as the source
of the datanextRecord in interface RecordReadernextRecord in class FileRecordReaderpublic List<Writable> next()
RecordReadernext in interface RecordReadernext in class FileRecordReaderpublic boolean hasNext()
RecordReaderhasNext in interface RecordReaderhasNext in class FileRecordReaderpublic void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableclose in class FileRecordReaderIOExceptionpublic void setConf(Configuration conf)
ConfigurablesetConf in interface ConfigurablesetConf in class FileRecordReaderpublic Configuration getConf()
ConfigurablegetConf in interface ConfigurablegetConf in class FileRecordReaderpublic TfidfVectorizer getTfidfVectorizer()
public void setTfidfVectorizer(TfidfVectorizer tfidfVectorizer)
public int getNumFeatures()
public void shuffle()
public void shuffle(Random random)
public Record loadFromMetaData(RecordMetaData recordMetaData) throws IOException
RecordReaderRecordMetaData instanceRecordReader.loadFromMetaData(List)loadFromMetaData in interface RecordReaderloadFromMetaData in class FileRecordReaderrecordMetaData - Metadata for the record that we want to load fromIOException - If I/O error occurs during loadingpublic List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException
RecordReaderRecordMetaData instancesloadFromMetaData in interface RecordReaderloadFromMetaData in class FileRecordReaderrecordMetaDatas - Metadata for the records that we want to load fromIOException - If I/O error occurs during loadingCopyright © 2020. All rights reserved.