public class CSVLineSequenceRecordReader extends CSVRecordReader implements SequenceRecordReader
a,b,c 1,2,3,4will produce two sequences, both with one value per time step; one of length 3 (values a, b, then c for the 3 time steps respectively) and one of length 4 (values 1, 2, 3, then 4 for each of the 4 time steps respectively)
DEFAULT_DELIMITER, DEFAULT_QUOTE, DELIMITER, QUOTE, SKIP_NUM_LINES, skipNumLinesconf, inputSplit, lineIndex, locations, splitIndexlistenersAPPEND_LABEL, LABELS, NAME_SPACE| Constructor and Description |
|---|
CSVLineSequenceRecordReader()
Default settings: skip 0 lines, use ',' as the delimiter, and '"' for quotes
|
CSVLineSequenceRecordReader(int skipNumLines,
char delimiter)
Skip lines and use delimiter
|
CSVLineSequenceRecordReader(int skipNumLines,
char delimiter,
char quote)
Skip lines, use delimiter, and strip quotes
|
| Modifier and Type | Method and Description |
|---|---|
protected SequenceRecord |
convert(Record r) |
java.util.List<SequenceRecord> |
loadSequenceFromMetaData(java.util.List<RecordMetaData> recordMetaDatas)
Load multiple sequence records from the given a list of
RecordMetaData instances |
SequenceRecord |
loadSequenceFromMetaData(RecordMetaData recordMetaData)
Load a single sequence record from the given
RecordMetaData instanceNote: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using SequenceRecordReader.loadSequenceFromMetaData(List) |
SequenceRecord |
nextSequence()
Similar to
SequenceRecordReader.sequenceRecord(), but returns a Record object, that may include metadata such as the source
of the data |
java.util.List<java.util.List<Writable>> |
sequenceRecord()
Returns a sequence record.
|
java.util.List<java.util.List<Writable>> |
sequenceRecord(java.net.URI uri,
java.io.DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
RecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
batchesSupported, hasNext, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, onLocationOpen, parseLine, readStringLine, record, resetclose, closeIfRequired, getConf, getIterator, getLabels, initialize, resetSupported, setConfgetListeners, invokeListeners, setListeners, setListenersclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitbatchesSupported, getLabels, getListeners, hasNext, initialize, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, record, reset, resetSupported, setListeners, setListenersgetConf, setConfpublic CSVLineSequenceRecordReader()
public CSVLineSequenceRecordReader(int skipNumLines,
char delimiter)
skipNumLines - the number of lines to skipdelimiter - the delimiterpublic CSVLineSequenceRecordReader(int skipNumLines,
char delimiter,
char quote)
skipNumLines - the number of lines to skipdelimiter - the delimiterquote - the quote to strippublic java.util.List<java.util.List<Writable>> sequenceRecord()
SequenceRecordReadersequenceRecord in interface SequenceRecordReaderpublic java.util.List<java.util.List<Writable>> sequenceRecord(java.net.URI uri, java.io.DataInputStream dataInputStream) throws java.io.IOException
SequenceRecordReaderRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStreamsequenceRecord in interface SequenceRecordReaderjava.io.IOException - if error occurs during reading from the input streampublic SequenceRecord nextSequence()
SequenceRecordReaderSequenceRecordReader.sequenceRecord(), but returns a Record object, that may include metadata such as the source
of the datanextSequence in interface SequenceRecordReaderpublic SequenceRecord loadSequenceFromMetaData(RecordMetaData recordMetaData) throws java.io.IOException
SequenceRecordReaderRecordMetaData instanceSequenceRecordReader.loadSequenceFromMetaData(List)loadSequenceFromMetaData in interface SequenceRecordReaderrecordMetaData - Metadata for the sequence record that we want to load fromjava.io.IOException - If I/O error occurs during loadingpublic java.util.List<SequenceRecord> loadSequenceFromMetaData(java.util.List<RecordMetaData> recordMetaDatas) throws java.io.IOException
SequenceRecordReaderRecordMetaData instancesloadSequenceFromMetaData in interface SequenceRecordReaderrecordMetaDatas - Metadata for the records that we want to load fromjava.io.IOException - If I/O error occurs during loadingprotected SequenceRecord convert(Record r)