public class NumberOfRecordsPartitioner extends Object implements Partitioner
recordsPerFile
number of records is written per
file when outputting to the various locations of the
InputSplit
locations.Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_RECORDS_PER_FILE |
static String |
RECORDS_PER_FILE_CONFIG |
Constructor and Description |
---|
NumberOfRecordsPartitioner() |
Modifier and Type | Method and Description |
---|---|
OutputStream |
currentOutputStream()
Get the current output stream
|
void |
init(Configuration configuration,
InputSplit split)
Initializes this partitioner with the given configuration
and input split
|
void |
init(InputSplit inputSplit)
Initializes this partitioner with the given configuration
and input split
|
boolean |
needsNewPartition()
Returns true if the partition needs to be moved to the next.
|
int |
numPartitions()
Returns the number of partitions
|
int |
numRecordsWritten()
Number of records written so far
|
OutputStream |
openNewStream()
"Increment" to the next stream
|
int |
totalRecordsWritten()
Returns the total records written
|
void |
updatePartitionInfo(PartitionMetaData metadata)
Updates the metadata for this partitioner
(to indicate whether the next partition is needed or not)
|
public static final int DEFAULT_RECORDS_PER_FILE
public static final String RECORDS_PER_FILE_CONFIG
public int totalRecordsWritten()
Partitioner
totalRecordsWritten
in interface Partitioner
public int numRecordsWritten()
Partitioner
numRecordsWritten
in interface Partitioner
public int numPartitions()
Partitioner
numPartitions
in interface Partitioner
public void init(InputSplit inputSplit)
Partitioner
init
in interface Partitioner
inputSplit
- the input split to use with this partitionerpublic void init(Configuration configuration, InputSplit split)
Partitioner
init
in interface Partitioner
configuration
- the configuration to configure
this partitioner withsplit
- the input split to use with this partitionerpublic void updatePartitionInfo(PartitionMetaData metadata)
Partitioner
updatePartitionInfo
in interface Partitioner
public boolean needsNewPartition()
Partitioner
Partitioner.updatePartitionInfo(PartitionMetaData)
which handles incrementing counters and the like
to determine whether the current partition has been exhausted.needsNewPartition
in interface Partitioner
public OutputStream openNewStream()
Partitioner
openNewStream
in interface Partitioner
public OutputStream currentOutputStream()
Partitioner
currentOutputStream
in interface Partitioner
Copyright © 2020. All rights reserved.