public class SplitPartitionsFunction<T> extends Object implements org.apache.spark.api.java.function.Function2<Integer,Iterator<T>,Iterator<T>>
AbstractJavaRDDLike.mapPartitionsWithIndex(Function2, boolean)via filtering.
JavaRDD.randomSplit(double)however it is less prone to producing imbalanced splits that method. Specifically,
JavaRDD.randomSplit(double)splits each element individually (i.e., randomly determine a new split for each element at random), whereas this method chooses one out of every numSplits objects per output split. Exactly which of these is done randomly.
|Constructor and Description|
Copyright © 2020. All rights reserved.