public class RecordMapper extends Object
RecordReaderand writes the data out with the specified
RecordWriter. The setup is as follows: Specify a
RecordReaderas the data source Specify a
RecordWriteras the destination. When setting up the locations, use 2 different
RecordReader.initialize(InputSplit)respectively to configure the locations of where the data will be read from and written to. When writing the data, you need to specify a link
Partitionerto determine how to slice up the data being written (say in to number of lines per record per file per
NumberOfRecordsPartitioneramong other implementations. Finally, you may specify a batch size for batch read and write if the record reader and writer support it. Of note, is you can also specify multiple readers. In which case, it will read from every stream jointly and write out the specified writer accordingly.
copy()will work the same with the following exceptions, you must specify
splitPerReader(one split per reader)
readersToConcatand the readers which will be read from writing to the same record writer. See
copy()for more information here.
|Constructor and Description|
public void copy() throws Exception
RecordReaderdata using the
RecordWriter. Note that unless batch is supported by both the
RecordWriterthen writes will happen one at a time. You can see if batch is enabled via
Copyright © 2020. All rights reserved.