public class TransformProcessSequenceRecordReader extends Object implements SequenceRecordReader
SequenceRecordReader
with a TransformProcess
which will allow every Record
returned from the SequenceRecordReader
to be transformed before being returned.Modifier and Type | Field and Description |
---|---|
protected SequenceRecordReader |
sequenceRecordReader |
protected TransformProcess |
transformProcess |
APPEND_LABEL, LABELS, NAME_SPACE
Constructor and Description |
---|
TransformProcessSequenceRecordReader() |
Modifier and Type | Method and Description |
---|---|
boolean |
batchesSupported()
This method returns true, if next(int) signature is supported by this RecordReader implementation.
|
void |
close()
Closes this stream and releases any system resources associated
with it.
|
Configuration |
getConf()
Return the configuration used by this object.
|
List<String> |
getLabels()
List of label strings
|
List<RecordListener> |
getListeners()
Get the record listeners for this record reader.
|
boolean |
hasNext()
Whether there are anymore records
|
void |
initialize(Configuration conf,
InputSplit split)
Called once at initialization.
|
void |
initialize(InputSplit split)
Called once at initialization.
|
List<Record> |
loadFromMetaData(List<RecordMetaData> recordMetaDatas)
Load multiple records from the given a list of
RecordMetaData instances |
Record |
loadFromMetaData(RecordMetaData recordMetaData)
Load a single record from the given
RecordMetaData instanceNote: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using loadFromMetaData(List) |
List<SequenceRecord> |
loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas)
Load multiple sequence records from the given a list of
RecordMetaData instances |
SequenceRecord |
loadSequenceFromMetaData(RecordMetaData recordMetaData)
Load a single sequence record from the given
RecordMetaData instanceNote: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using loadSequenceFromMetaData(List) |
List<Writable> |
next()
Get the next record
|
List<List<Writable>> |
next(int num)
This method will be used, if batchesSupported() returns true.
|
Record |
nextRecord()
|
SequenceRecord |
nextSequence()
Similar to
sequenceRecord() , but returns a Record object, that may include metadata such as the source
of the data |
List<Writable> |
record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
void |
reset()
Reset record reader iterator
|
boolean |
resetSupported() |
List<List<Writable>> |
sequenceRecord()
Returns a sequence record.
|
List<List<Writable>> |
sequenceRecord(URI uri,
DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
void |
setConf(Configuration conf)
Set the configuration to be used by this object.
|
void |
setListeners(Collection<RecordListener> listeners)
Set the record listeners for this record reader.
|
void |
setListeners(RecordListener... listeners)
Set the record listeners for this record reader.
|
protected SequenceRecordReader sequenceRecordReader
protected TransformProcess transformProcess
public TransformProcessSequenceRecordReader()
public void setConf(Configuration conf)
setConf
in interface Configurable
conf
- public Configuration getConf()
getConf
in interface Configurable
public List<List<Writable>> sequenceRecord()
sequenceRecord
in interface SequenceRecordReader
public boolean batchesSupported()
RecordReader
batchesSupported
in interface RecordReader
public List<List<Writable>> next(int num)
RecordReader
next
in interface RecordReader
public List<List<Writable>> sequenceRecord(URI uri, DataInputStream dataInputStream) throws IOException
next()
the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStreamsequenceRecord
in interface SequenceRecordReader
uri
- dataInputStream
- IOException
- if error occurs during reading from the input streampublic SequenceRecord nextSequence()
sequenceRecord()
, but returns a Record
object, that may include metadata such as the source
of the datanextSequence
in interface SequenceRecordReader
public SequenceRecord loadSequenceFromMetaData(RecordMetaData recordMetaData) throws IOException
RecordMetaData
instanceloadSequenceFromMetaData(List)
loadSequenceFromMetaData
in interface SequenceRecordReader
recordMetaData
- Metadata for the sequence record that we want to load fromIOException
- If I/O error occurs during loadingpublic List<SequenceRecord> loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException
RecordMetaData
instancesloadSequenceFromMetaData
in interface SequenceRecordReader
recordMetaDatas
- Metadata for the records that we want to load fromIOException
- If I/O error occurs during loadingpublic void initialize(InputSplit split) throws IOException, InterruptedException
initialize
in interface RecordReader
split
- the split that defines the range of records to readIOException
InterruptedException
public void initialize(Configuration conf, InputSplit split) throws IOException, InterruptedException
initialize
in interface RecordReader
conf
- a configuration for initializationsplit
- the split that defines the range of records to readIOException
InterruptedException
public List<Writable> next()
next
in interface RecordReader
public boolean hasNext()
hasNext
in interface RecordReader
public List<String> getLabels()
getLabels
in interface RecordReader
public void reset()
reset
in interface RecordReader
public boolean resetSupported()
resetSupported
in interface RecordReader
public List<Writable> record(URI uri, DataInputStream dataInputStream) throws IOException
next()
the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStreamrecord
in interface RecordReader
uri
- dataInputStream
- IOException
- if error occurs during reading from the input streampublic Record nextRecord()
next()
, but returns a Record
object, that may include metadata such as the source
of the datanextRecord
in interface RecordReader
public Record loadFromMetaData(RecordMetaData recordMetaData) throws IOException
RecordMetaData
instanceloadFromMetaData(List)
loadFromMetaData
in interface RecordReader
recordMetaData
- Metadata for the record that we want to load fromIOException
- If I/O error occurs during loadingpublic List<Record> loadFromMetaData(List<RecordMetaData> recordMetaDatas) throws IOException
RecordMetaData
instancesloadFromMetaData
in interface RecordReader
recordMetaDatas
- Metadata for the records that we want to load fromIOException
- If I/O error occurs during loadingpublic List<RecordListener> getListeners()
getListeners
in interface RecordReader
public void setListeners(RecordListener... listeners)
setListeners
in interface RecordReader
listeners
- public void setListeners(Collection<RecordListener> listeners)
setListeners
in interface RecordReader
listeners
- public void close() throws IOException
As noted in AutoCloseable.close()
, cases where the
close may fail require careful attention. It is strongly advised
to relinquish the underlying resources and to internally
mark the Closeable
as closed, prior to throwing
the IOException
.
close
in interface Closeable
close
in interface AutoCloseable
IOException
- if an I/O error occursCopyright © 2020. All rights reserved.