public class CSVLineSequenceRecordReader extends CSVRecordReader implements SequenceRecordReader
a,b,c 1,2,3,4will produce two sequences, both with one value per time step; one of length 3 (values a, b, then c for the 3 time steps respectively) and one of length 4 (values 1, 2, 3, then 4 for each of the 4 time steps respectively)
DEFAULT_DELIMITER, DEFAULT_QUOTE, DELIMITER, QUOTE, SKIP_NUM_LINES, skipNumLines
conf, inputSplit, lineIndex, locations, splitIndex
listeners
APPEND_LABEL, LABELS, NAME_SPACE
Constructor and Description |
---|
CSVLineSequenceRecordReader()
Default settings: skip 0 lines, use ',' as the delimiter, and '"' for quotes
|
CSVLineSequenceRecordReader(int skipNumLines,
char delimiter)
Skip lines and use delimiter
|
CSVLineSequenceRecordReader(int skipNumLines,
char delimiter,
char quote)
Skip lines, use delimiter, and strip quotes
|
Modifier and Type | Method and Description |
---|---|
protected SequenceRecord |
convert(Record r) |
java.util.List<SequenceRecord> |
loadSequenceFromMetaData(java.util.List<RecordMetaData> recordMetaDatas)
Load multiple sequence records from the given a list of
RecordMetaData instances |
SequenceRecord |
loadSequenceFromMetaData(RecordMetaData recordMetaData)
Load a single sequence record from the given
RecordMetaData instanceNote: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using SequenceRecordReader.loadSequenceFromMetaData(List) |
SequenceRecord |
nextSequence()
Similar to
SequenceRecordReader.sequenceRecord() , but returns a Record object, that may include metadata such as the source
of the data |
java.util.List<java.util.List<Writable>> |
sequenceRecord()
Returns a sequence record.
|
java.util.List<java.util.List<Writable>> |
sequenceRecord(java.net.URI uri,
java.io.DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
RecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
batchesSupported, hasNext, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, onLocationOpen, parseLine, readStringLine, record, reset
close, closeIfRequired, getConf, getIterator, getLabels, initialize, resetSupported, setConf
getListeners, invokeListeners, setListeners, setListeners
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
batchesSupported, getLabels, getListeners, hasNext, initialize, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, record, reset, resetSupported, setListeners, setListeners
getConf, setConf
public CSVLineSequenceRecordReader()
public CSVLineSequenceRecordReader(int skipNumLines, char delimiter)
skipNumLines
- the number of lines to skipdelimiter
- the delimiterpublic CSVLineSequenceRecordReader(int skipNumLines, char delimiter, char quote)
skipNumLines
- the number of lines to skipdelimiter
- the delimiterquote
- the quote to strippublic java.util.List<java.util.List<Writable>> sequenceRecord()
SequenceRecordReader
sequenceRecord
in interface SequenceRecordReader
public java.util.List<java.util.List<Writable>> sequenceRecord(java.net.URI uri, java.io.DataInputStream dataInputStream) throws java.io.IOException
SequenceRecordReader
RecordReader.next()
the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStreamsequenceRecord
in interface SequenceRecordReader
java.io.IOException
- if error occurs during reading from the input streampublic SequenceRecord nextSequence()
SequenceRecordReader
SequenceRecordReader.sequenceRecord()
, but returns a Record
object, that may include metadata such as the source
of the datanextSequence
in interface SequenceRecordReader
public SequenceRecord loadSequenceFromMetaData(RecordMetaData recordMetaData) throws java.io.IOException
SequenceRecordReader
RecordMetaData
instanceSequenceRecordReader.loadSequenceFromMetaData(List)
loadSequenceFromMetaData
in interface SequenceRecordReader
recordMetaData
- Metadata for the sequence record that we want to load fromjava.io.IOException
- If I/O error occurs during loadingpublic java.util.List<SequenceRecord> loadSequenceFromMetaData(java.util.List<RecordMetaData> recordMetaDatas) throws java.io.IOException
SequenceRecordReader
RecordMetaData
instancesloadSequenceFromMetaData
in interface SequenceRecordReader
recordMetaDatas
- Metadata for the records that we want to load fromjava.io.IOException
- If I/O error occurs during loadingprotected SequenceRecord convert(Record r)