SharedTrainingWorker (deeplearning4j 1.0.0-beta7 API)

java.lang.Object
- org.deeplearning4j.spark.impl.paramavg.BaseTrainingWorker<SharedTrainingResult>
- - org.deeplearning4j.spark.parameterserver.training.SharedTrainingWorker

All Implemented Interfaces:: Serializable, TrainingWorker<SharedTrainingResult>

public class SharedTrainingWorker
extends BaseTrainingWorker<SharedTrainingResult>
implements TrainingWorker<SharedTrainingResult>

Author:: raver119@gmail.com
See Also:: Serialized Form

Constructor Summary

Constructors
Constructor and Description
`SharedTrainingWorker(long instanceId, org.apache.spark.broadcast.Broadcast<NetBroadcastTuple> broadcastModel, org.apache.spark.broadcast.Broadcast<SharedTrainingConfiguration> broadcastConfiguration, List<TrainingListener> listeners, StatsStorageRouter router, Boolean workerTogglePeriodicGC, Integer workerPeriodicGCFrequency)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addHook(TrainingHook trainingHook)` Add a training hook to be used during training of the worker
`WorkerConfiguration`	`getDataConfiguration()` Get the `WorkerConfiguration` that contains information such as minibatch sizes, etc
`SharedTrainingResult`	`getFinalResult(ComputationGraph network)` Get the final result to be returned to the driver
`SharedTrainingResult`	`getFinalResult(MultiLayerNetwork network)` Get the final result to be returned to the driver
`SharedTrainingResult`	`getFinalResultNoData()` Get the final result to be returned to the driver, if no data was available for this executor
`Pair<SharedTrainingResult,SparkTrainingStats>`	`getFinalResultNoDataWithStats()` As per `TrainingWorker.getFinalResultNoData()` but used when `SparkTrainingStats` are being collected
`Pair<SharedTrainingResult,SparkTrainingStats>`	`getFinalResultWithStats(ComputationGraph graph)` As per `TrainingWorker.getFinalResult(ComputationGraph)` but used when `SparkTrainingStats` are being collected
`Pair<SharedTrainingResult,SparkTrainingStats>`	`getFinalResultWithStats(MultiLayerNetwork network)` As per `TrainingWorker.getFinalResult(MultiLayerNetwork)` but used when `SparkTrainingStats` are being collected
`MultiLayerNetwork`	`getInitialModel()` Get the initial model when training a MultiLayerNetwork/SparkDl4jMultiLayer
`ComputationGraph`	`getInitialModelGraph()` Get the initial model when training a ComputationGraph/SparkComputationGraph
`SharedTrainingResult`	`processMinibatch(DataSet dataSet, ComputationGraph graph, boolean isLast)` Process (fit) a minibatch for a ComputationGraph
`SharedTrainingResult`	`processMinibatch(DataSet dataSet, MultiLayerNetwork network, boolean isLast)` Process (fit) a minibatch for a MultiLayerNetwork
`SharedTrainingResult`	`processMinibatch(MultiDataSet dataSet, ComputationGraph graph, boolean isLast)` Process (fit) a minibatch for a ComputationGraph using a MultiDataSet
`Pair<SharedTrainingResult,SparkTrainingStats>`	`processMinibatchWithStats(DataSet dataSet, ComputationGraph graph, boolean isLast)` As per `TrainingWorker.processMinibatch(DataSet, ComputationGraph, boolean)` but used when `SparkTrainingStats` are being collected
`Pair<SharedTrainingResult,SparkTrainingStats>`	`processMinibatchWithStats(DataSet dataSet, MultiLayerNetwork network, boolean isLast)` As per `TrainingWorker.processMinibatch(DataSet, MultiLayerNetwork, boolean)` but used when `SparkTrainingStats` are being collecte
`Pair<SharedTrainingResult,SparkTrainingStats>`	`processMinibatchWithStats(MultiDataSet dataSet, ComputationGraph graph, boolean isLast)` As per `TrainingWorker.processMinibatch(MultiDataSet, ComputationGraph, boolean)` but used when `SparkTrainingStats` are being collected
`void`	`removeHook(TrainingHook trainingHook)` Remove a training hook from the worker

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.spark.api.TrainingWorker
getInstanceId

- Constructor Detail
  - SharedTrainingWorker
```
public SharedTrainingWorker(long instanceId,
                            org.apache.spark.broadcast.Broadcast<NetBroadcastTuple> broadcastModel,
                            org.apache.spark.broadcast.Broadcast<SharedTrainingConfiguration> broadcastConfiguration,
                            List<TrainingListener> listeners,
                            StatsStorageRouter router,
                            Boolean workerTogglePeriodicGC,
                            Integer workerPeriodicGCFrequency)
```
- Method Detail
  - removeHook
```
public void removeHook(TrainingHook trainingHook)
```
    Description copied from interface: TrainingWorker
    
    Remove a training hook from the worker
    
    Specified by:
    
    removeHook in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    trainingHook - the training hook to remove
  - addHook
```
public void addHook(TrainingHook trainingHook)
```
    Description copied from interface: TrainingWorker
    
    Add a training hook to be used during training of the worker
    
    Specified by:
    
    addHook in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    trainingHook - the training hook to add
  - getInitialModel
```
public MultiLayerNetwork getInitialModel()
```
    Description copied from interface: TrainingWorker
    
    Get the initial model when training a MultiLayerNetwork/SparkDl4jMultiLayer
    
    Specified by:
    
    getInitialModel in interface TrainingWorker<SharedTrainingResult>
    
    Returns:
    
    Initial model for this worker
  - getInitialModelGraph
```
public ComputationGraph getInitialModelGraph()
```
    Description copied from interface: TrainingWorker
    
    Get the initial model when training a ComputationGraph/SparkComputationGraph
    
    Specified by:
    
    getInitialModelGraph in interface TrainingWorker<SharedTrainingResult>
    
    Returns:
    
    Initial model for this worker
  - processMinibatch
```
public SharedTrainingResult processMinibatch(DataSet dataSet,
                                             MultiLayerNetwork network,
                                             boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    Process (fit) a minibatch for a MultiLayerNetwork
    
    Specified by:
    
    processMinibatch in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    dataSet - Data set to train on
    
    network - Network to train
    
    isLast - If true: last data set currently available. If false: more data sets will be processed for this executor
    
    Returns:
    
    Null, or a training result if training should be terminated immediately.
  - processMinibatch
```
public SharedTrainingResult processMinibatch(DataSet dataSet,
                                             ComputationGraph graph,
                                             boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    Process (fit) a minibatch for a ComputationGraph
    
    Specified by:
    
    processMinibatch in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    dataSet - Data set to train on
    
    graph - Network to train
    
    isLast - If true: last data set currently available. If false: more data sets will be processed for this executor
    
    Returns:
    
    Null, or a training result if training should be terminated immediately.
  - processMinibatch
```
public SharedTrainingResult processMinibatch(MultiDataSet dataSet,
                                             ComputationGraph graph,
                                             boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    Process (fit) a minibatch for a ComputationGraph using a MultiDataSet
    
    Specified by:
    
    processMinibatch in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    dataSet - Data set to train on
    
    graph - Network to train
    
    isLast - If true: last data set currently available. If false: more data sets will be processed for this executor
    
    Returns:
    
    Null, or a training result if training should be terminated immediately.
  - processMinibatchWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> processMinibatchWithStats(DataSet dataSet,
                                                                               MultiLayerNetwork network,
                                                                               boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.processMinibatch(DataSet, MultiLayerNetwork, boolean) but used when SparkTrainingStats are being collecte
    
    Specified by:
    
    processMinibatchWithStats in interface TrainingWorker<SharedTrainingResult>
  - processMinibatchWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> processMinibatchWithStats(DataSet dataSet,
                                                                               ComputationGraph graph,
                                                                               boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.processMinibatch(DataSet, ComputationGraph, boolean) but used when SparkTrainingStats are being collected
    
    Specified by:
    
    processMinibatchWithStats in interface TrainingWorker<SharedTrainingResult>
  - processMinibatchWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> processMinibatchWithStats(MultiDataSet dataSet,
                                                                               ComputationGraph graph,
                                                                               boolean isLast)
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.processMinibatch(MultiDataSet, ComputationGraph, boolean) but used when SparkTrainingStats are being collected
    
    Specified by:
    
    processMinibatchWithStats in interface TrainingWorker<SharedTrainingResult>
  - getFinalResult
```
public SharedTrainingResult getFinalResult(MultiLayerNetwork network)
```
    Description copied from interface: TrainingWorker
    
    Get the final result to be returned to the driver
    
    Specified by:
    
    getFinalResult in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    network - Current state of the network
    
    Returns:
    
    Result to return to the driver
  - getFinalResult
```
public SharedTrainingResult getFinalResult(ComputationGraph network)
```
    Description copied from interface: TrainingWorker
    
    Get the final result to be returned to the driver
    
    Specified by:
    
    getFinalResult in interface TrainingWorker<SharedTrainingResult>
    
    Parameters:
    
    network - Current state of the network
    
    Returns:
    
    Result to return to the driver
  - getFinalResultNoData
```
public SharedTrainingResult getFinalResultNoData()
```
    Description copied from interface: TrainingWorker
    
    Get the final result to be returned to the driver, if no data was available for this executor
    
    Specified by:
    
    getFinalResultNoData in interface TrainingWorker<SharedTrainingResult>
    
    Returns:
    
    Result to return to the driver
  - getFinalResultNoDataWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> getFinalResultNoDataWithStats()
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.getFinalResultNoData() but used when SparkTrainingStats are being collected
    
    Specified by:
    
    getFinalResultNoDataWithStats in interface TrainingWorker<SharedTrainingResult>
  - getFinalResultWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> getFinalResultWithStats(MultiLayerNetwork network)
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.getFinalResult(MultiLayerNetwork) but used when SparkTrainingStats are being collected
    
    Specified by:
    
    getFinalResultWithStats in interface TrainingWorker<SharedTrainingResult>
  - getFinalResultWithStats
```
public Pair<SharedTrainingResult,SparkTrainingStats> getFinalResultWithStats(ComputationGraph graph)
```
    Description copied from interface: TrainingWorker
    
    As per TrainingWorker.getFinalResult(ComputationGraph) but used when SparkTrainingStats are being collected
    
    Specified by:
    
    getFinalResultWithStats in interface TrainingWorker<SharedTrainingResult>
  - getDataConfiguration
```
public WorkerConfiguration getDataConfiguration()
```
    Description copied from interface: TrainingWorker
    
    Get the WorkerConfiguration that contains information such as minibatch sizes, etc
    
    Specified by:
    
    getDataConfiguration in interface TrainingWorker<SharedTrainingResult>
    
    Returns:
    
    Worker configuration

Class SharedTrainingWorker

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface org.deeplearning4j.spark.api.TrainingWorker

Constructor Detail

SharedTrainingWorker

Method Detail

removeHook

addHook

getInitialModel

getInitialModelGraph

processMinibatch

processMinibatch

processMinibatch

processMinibatchWithStats

processMinibatchWithStats

processMinibatchWithStats

getFinalResult

getFinalResult

getFinalResultNoData

getFinalResultNoDataWithStats

getFinalResultWithStats

getFinalResultWithStats

getDataConfiguration