| Modifier and Type | Class and Description |
|---|---|
class |
SerializedMultiDataSetLoader
Loads DataSets using
MultiDataSet.load(InputStream) |
| Modifier and Type | Method and Description |
|---|---|
void |
TrainingMaster.executeTrainingPaths(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> trainingDataPaths,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader)
Fit the network using a list of paths for serialized DataSet objects.
|
| Constructor and Description |
|---|
ExecuteWorkerPathMDSFlatMap(TrainingWorker<R> worker,
MultiDataSetLoader loader,
org.apache.spark.broadcast.Broadcast<SerializableHadoopConfig> hadoopConfig) |
| Modifier and Type | Method and Description |
|---|---|
protected IEvaluation[] |
SparkComputationGraph.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
int evalNumWorkers,
int evalBatchSize,
DataSetLoader loader,
MultiDataSetLoader mdsLoader,
IEvaluation... emptyEvaluations) |
IEvaluation[] |
SparkComputationGraph.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
int evalNumWorkers,
int evalBatchSize,
MultiDataSetLoader loader,
IEvaluation... emptyEvaluations)
Perform evaluation on serialized MultiDataSet objects on disk, (potentially in any format), that are loaded using an
MultiDataSetLoader |
IEvaluation[] |
SparkComputationGraph.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
MultiDataSetLoader loader,
IEvaluation... emptyEvaluations)
Perform evaluation on serialized MultiDataSet objects on disk, (potentially in any format), that are loaded using an
MultiDataSetLoader. |
Evaluation |
SparkComputationGraph.evaluate(String path,
MultiDataSetLoader loader)
Evaluate the single-output network on a directory containing a set of MultiDataSet objects to be loaded with a
MultiDataSetLoader. |
ComputationGraph |
SparkComputationGraph.fitPaths(org.apache.spark.api.java.JavaRDD<String> paths,
MultiDataSetLoader loader) |
| Modifier and Type | Field and Description |
|---|---|
protected MultiDataSetLoader |
IEvaluateMDSPathsFlatMapFunction.mdsLoader |
| Constructor and Description |
|---|
IEvaluateMDSPathsFlatMapFunction(org.apache.spark.broadcast.Broadcast<String> json,
org.apache.spark.broadcast.Broadcast<byte[]> params,
int evalNumWorkers,
int evalBatchSize,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader,
org.apache.spark.broadcast.Broadcast<SerializableHadoopConfig> configuration,
IEvaluation[] evaluations) |
| Modifier and Type | Method and Description |
|---|---|
protected IEvaluation[] |
SparkDl4jMultiLayer.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
int evalNumWorkers,
int evalBatchSize,
DataSetLoader loader,
MultiDataSetLoader mdsLoader,
IEvaluation... emptyEvaluations) |
IEvaluation[] |
SparkDl4jMultiLayer.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
int evalNumWorkers,
int evalBatchSize,
MultiDataSetLoader loader,
IEvaluation... emptyEvaluations)
Perform evaluation on serialized MultiDataSet objects on disk, (potentially in any format), that are loaded using an
MultiDataSetLoader |
IEvaluation[] |
SparkDl4jMultiLayer.doEvaluation(org.apache.spark.api.java.JavaRDD<String> data,
MultiDataSetLoader loader,
IEvaluation... emptyEvaluations)
Perform evaluation on serialized MultiDataSet objects on disk, (potentially in any format), that are loaded using an
MultiDataSetLoader. |
| Modifier and Type | Method and Description |
|---|---|
protected void |
ParameterAveragingTrainingMaster.doIterationPaths(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> split,
int splitNum,
int numSplits,
int dataSetObjectNumExamples,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader) |
void |
ParameterAveragingTrainingMaster.executeTrainingPaths(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> trainingDataPaths,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader) |
protected void |
ParameterAveragingTrainingMaster.executeTrainingPathsHelper(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> trainingDataPaths,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader,
int dataSetObjectsNumExamples) |
| Constructor and Description |
|---|
PathSparkMultiDataSetIterator(Collection<String> dataSetStreams,
MultiDataSetLoader loader,
org.apache.spark.broadcast.Broadcast<SerializableHadoopConfig> hadoopConfig) |
PathSparkMultiDataSetIterator(Iterator<String> iter,
MultiDataSetLoader loader,
org.apache.spark.broadcast.Broadcast<SerializableHadoopConfig> hadoopConfig) |
| Modifier and Type | Field and Description |
|---|---|
protected MultiDataSetLoader |
SharedFlatMapPathsMDS.loader |
| Constructor and Description |
|---|
SharedFlatMapPathsMDS(TrainingWorker<R> worker,
MultiDataSetLoader loader,
org.apache.spark.broadcast.Broadcast<SerializableHadoopConfig> hadoopConfig) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
SharedTrainingMaster.doIterationPaths(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> data,
int splitNum,
int numSplits,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader,
int dataSetObjectNumExamples) |
void |
SharedTrainingMaster.executeTrainingPaths(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> trainingDataPaths,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader) |
protected void |
SharedTrainingMaster.executeTrainingPathsHelper(SparkDl4jMultiLayer network,
SparkComputationGraph graph,
org.apache.spark.api.java.JavaRDD<String> trainingDataPaths,
DataSetLoader dsLoader,
MultiDataSetLoader mdsLoader,
int dataSetObjectsNumExamples) |
Copyright © 2020. All rights reserved.