org.deeplearning4j.ui.model.storage

Class BaseCollectionStatsStorage

java.lang.Object

org.deeplearning4j.ui.model.storage.BaseCollectionStatsStorage

All Implemented Interfaces:

StatsStorage, StatsStorageRouter

Direct Known Subclasses:

InMemoryStatsStorage, MapDBStatsStorage

public abstract class BaseCollectionStatsStorage
extends Object
implements StatsStorage

An implementation of the StatsStorage interface, backed by MapDB

Author:

Alex Black

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseCollectionStatsStorage.SessionTypeId
static class	BaseCollectionStatsStorage.SessionTypeWorkerId

Field Summary

Fields

Modifier and Type	Field and Description
protected List<StatsStorageListener>	listeners
protected Set<String>	sessionIDs
protected Map<BaseCollectionStatsStorage.SessionTypeWorkerId,Persistable>	staticInfo
protected Map<BaseCollectionStatsStorage.SessionTypeId,StorageMetaData>	storageMetaData
protected Map<BaseCollectionStatsStorage.SessionTypeWorkerId,Map<Long,Persistable>>	updates

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseCollectionStatsStorage()

Method Summary

Modifier and Type	Method and Description
protected List<StatsStorageEvent>	checkStorageEvents(Persistable p)
void	deregisterStatsStorageListener(StatsStorageListener listener) Remove the specified listener, if it is present.
List<Persistable>	getAllStaticInfos(String sessionID, String typeID) Get all static informations for the given session and type ID
List<Persistable>	getAllUpdatesAfter(String sessionID, String typeID, long timestamp) Get all updates for the given session ID (all worker IDs), that occur after (not including) the given timestamp.
List<Persistable>	getAllUpdatesAfter(String sessionID, String typeID, String workerID, long timestamp) Get all updates for the given session and worker ID, that occur after (not including) the given timestamp.
long[]	getAllUpdateTimes(String sessionID, String typeID, String workerID) List the times of all updates for the specified sessionID, typeID and workerID
Persistable	getLatestUpdate(String sessionID, String typeID, String workerID) Get the latest update record (i.e., update record with the largest timestamp value) for the specified session and worker IDs
List<Persistable>	getLatestUpdateAllWorkers(String sessionID, String typeID) Get the latest update for all workers, for the given session ID
List<StatsStorageListener>	getListeners() Get a list (shallow copy) of all listeners currently present
int	getNumUpdateRecordsFor(String sessionID) Return the number of update records for the given session ID (all workers)
int	getNumUpdateRecordsFor(String sessionID, String typeID, String workerID) Return the number of update records for the given session ID and worker ID
Persistable	getStaticInfo(String sessionID, String typeID, String workerID) Get the static info for the given session and worker IDs, or null if no such static info has been reported
StorageMetaData	getStorageMetaData(String sessionID, String typeID) Get the session metadata, if any has been registered via StatsStorageRouter.putStorageMetaData(StorageMetaData)
Persistable	getUpdate(String sessionID, String typeID, String workerID, long timestamp) Get the specified update (or null, if none exists for the given session/worker ids and timestamp)
protected abstract Map<Long,Persistable>	getUpdateMap(String sessionID, String typeID, String workerID, boolean createIfRequired)
List<Persistable>	getUpdates(String sessionID, String typeID, String workerID, long[] timestamps) Get updates for the specified times only
List<String>	listSessionIDs() Get a list of all sessions stored by this storage backend
List<String>	listTypeIDsForSession(String sessionID) Get the list of type IDs for the given session ID
List<String>	listWorkerIDsForSession(String sessionID) For a given session ID, list all of the known worker IDs
List<String>	listWorkerIDsForSessionAndType(String sessionID, String typeID) For a given session ID and type ID, list all of the known worker IDs
protected void	notifyListeners(List<StatsStorageEvent> sses)
void	putStaticInfo(Collection<? extends Persistable> staticInfo) Static info: reported once per session, upon initialization
abstract void	putStaticInfo(Persistable staticInfo) Static info: reported once per session, upon initialization
void	putStorageMetaData(Collection<? extends StorageMetaData> storageMetaData)
abstract void	putStorageMetaData(StorageMetaData storageMetaData) Method to store some additional metadata for each session.
void	putUpdate(Collection<? extends Persistable> updates) Updates: stored multiple times per session (periodically, for example)
abstract void	putUpdate(Persistable update) Updates: stored multiple times per session (periodically, for example)
void	registerStatsStorageListener(StatsStorageListener listener) Add a new StatsStorageListener.
void	removeAllListeners() Remove all listeners from the StatsStorage instance
boolean	sessionExists(String sessionID) Check if the specified session ID exists or not

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.core.storage.StatsStorage

close, isClosed

Field Detail

sessionIDs

protected Set<String> sessionIDs

storageMetaData

protected Map<BaseCollectionStatsStorage.SessionTypeId,StorageMetaData> storageMetaData

staticInfo

protected Map<BaseCollectionStatsStorage.SessionTypeWorkerId,Persistable> staticInfo

updates

protected Map<BaseCollectionStatsStorage.SessionTypeWorkerId,Map<Long,Persistable>> updates

listeners

protected List<StatsStorageListener> listeners

Constructor Detail

BaseCollectionStatsStorage

protected BaseCollectionStatsStorage()

Method Detail

getUpdateMap

protected abstract Map<Long,Persistable> getUpdateMap(String sessionID,
                                                      String typeID,
                                                      String workerID,
                                                      boolean createIfRequired)

checkStorageEvents

protected List<StatsStorageEvent> checkStorageEvents(Persistable p)

notifyListeners

protected void notifyListeners(List<StatsStorageEvent> sses)

listSessionIDs

public List<String> listSessionIDs()

Description copied from interface: StatsStorage

Get a list of all sessions stored by this storage backend

Specified by:

listSessionIDs in interface StatsStorage

sessionExists

public boolean sessionExists(String sessionID)

Description copied from interface: StatsStorage

Check if the specified session ID exists or not

Specified by:

sessionExists in interface StatsStorage

Parameters:

sessionID - Session ID to check

Returns:

true if session exists, false otherwise

getStaticInfo

public Persistable getStaticInfo(String sessionID,
                                 String typeID,
                                 String workerID)

Description copied from interface: StatsStorage

Get the static info for the given session and worker IDs, or null if no such static info has been reported

Specified by:

getStaticInfo in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - worker ID

Returns:

Static info, or null if none has been reported

getAllStaticInfos

public List<Persistable> getAllStaticInfos(String sessionID,
                                           String typeID)

Description copied from interface: StatsStorage

Get all static informations for the given session and type ID

Specified by:

getAllStaticInfos in interface StatsStorage

Parameters:

sessionID - Session ID to get static info for

typeID - Type ID to get static info for

Returns:

All static info instances matching both the session and type IDs

listTypeIDsForSession

public List<String> listTypeIDsForSession(String sessionID)

Description copied from interface: StatsStorage

Get the list of type IDs for the given session ID

Specified by:

listTypeIDsForSession in interface StatsStorage

Parameters:

sessionID - Session ID to query

Returns:

List of type IDs

listWorkerIDsForSession

public List<String> listWorkerIDsForSession(String sessionID)

Description copied from interface: StatsStorage

For a given session ID, list all of the known worker IDs

Specified by:

listWorkerIDsForSession in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

List of worker IDs, or possibly null if session ID is unknown

listWorkerIDsForSessionAndType

public List<String> listWorkerIDsForSessionAndType(String sessionID,
                                                   String typeID)

Description copied from interface: StatsStorage

For a given session ID and type ID, list all of the known worker IDs

Specified by:

listWorkerIDsForSessionAndType in interface StatsStorage

Parameters:

sessionID - Session ID

typeID - Type ID

Returns:

List of worker IDs, or possibly null if session ID is unknown

getNumUpdateRecordsFor

public int getNumUpdateRecordsFor(String sessionID)

Description copied from interface: StatsStorage

Return the number of update records for the given session ID (all workers)

Specified by:

getNumUpdateRecordsFor in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

number of update records

getNumUpdateRecordsFor

public int getNumUpdateRecordsFor(String sessionID,
                                  String typeID,
                                  String workerID)

Description copied from interface: StatsStorage

Return the number of update records for the given session ID and worker ID

Specified by:

getNumUpdateRecordsFor in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker ID

Returns:

number of update records

getLatestUpdate

public Persistable getLatestUpdate(String sessionID,
                                   String typeID,
                                   String workerID)

Description copied from interface: StatsStorage

Get the latest update record (i.e., update record with the largest timestamp value) for the specified session and worker IDs

Specified by:

getLatestUpdate in interface StatsStorage

Parameters:

sessionID - session ID

workerID - worker ID

Returns:

UpdateRecord containing the session/worker IDs, timestamp and content for the most recent update

getUpdate

public Persistable getUpdate(String sessionID,
                             String typeID,
                             String workerID,
                             long timestamp)

Description copied from interface: StatsStorage

Get the specified update (or null, if none exists for the given session/worker ids and timestamp)

Specified by:

getUpdate in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker ID

timestamp - Timestamp

Returns:

Update

getLatestUpdateAllWorkers

public List<Persistable> getLatestUpdateAllWorkers(String sessionID,
                                                   String typeID)

Description copied from interface: StatsStorage

Get the latest update for all workers, for the given session ID

Specified by:

getLatestUpdateAllWorkers in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

List of updates for the given Session ID

getAllUpdatesAfter

public List<Persistable> getAllUpdatesAfter(String sessionID,
                                            String typeID,
                                            String workerID,
                                            long timestamp)

Description copied from interface: StatsStorage

Get all updates for the given session and worker ID, that occur after (not including) the given timestamp. Results should be sorted by time.

Specified by:

getAllUpdatesAfter in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker Id

timestamp - Timestamp

Returns:

List of records occurring after the given timestamp

getAllUpdatesAfter

public List<Persistable> getAllUpdatesAfter(String sessionID,
                                            String typeID,
                                            long timestamp)

Description copied from interface: StatsStorage

Get all updates for the given session ID (all worker IDs), that occur after (not including) the given timestamp. Results should be sorted by time.

Specified by:

getAllUpdatesAfter in interface StatsStorage

Parameters:

sessionID - Session ID

timestamp - Timestamp

Returns:

List of records occurring after the given timestamp

getStorageMetaData

public StorageMetaData getStorageMetaData(String sessionID,
                                          String typeID)

Description copied from interface: StatsStorage

Get the session metadata, if any has been registered via StatsStorageRouter.putStorageMetaData(StorageMetaData)

Specified by:

getStorageMetaData in interface StatsStorage

Parameters:

sessionID - Session ID to get metadat

Returns:

Session metadata, or null if none is available

getAllUpdateTimes

public long[] getAllUpdateTimes(String sessionID,
                                String typeID,
                                String workerID)

Description copied from interface: StatsStorage

List the times of all updates for the specified sessionID, typeID and workerID

Specified by:

getAllUpdateTimes in interface StatsStorage

Parameters:

sessionID - Session ID to get update times for

typeID - Type ID to get update times for

workerID - Worker ID to get update times for

Returns:

Times of all updates

getUpdates

public List<Persistable> getUpdates(String sessionID,
                                    String typeID,
                                    String workerID,
                                    long[] timestamps)

Description copied from interface: StatsStorage

Get updates for the specified times only

Specified by:

getUpdates in interface StatsStorage

Parameters:

sessionID - Session ID to get update times for

typeID - Type ID to get update times for

workerID - Worker ID to get update times for

timestamps - Timestamps to get the updates for. Note that if one of the specified times does not exist, it will be ommitted from the returned results list.

Returns:

List of updates at the specified times

putStaticInfo

public abstract void putStaticInfo(Persistable staticInfo)

Description copied from interface: StatsStorageRouter

Static info: reported once per session, upon initialization

Specified by:

putStaticInfo in interface StatsStorageRouter

Parameters:

staticInfo - Static info to store

putStaticInfo

public void putStaticInfo(Collection<? extends Persistable> staticInfo)

Description copied from interface: StatsStorageRouter

Static info: reported once per session, upon initialization

Specified by:

putStaticInfo in interface StatsStorageRouter

Parameters:

staticInfo - Static info to store

putUpdate

public abstract void putUpdate(Persistable update)

Description copied from interface: StatsStorageRouter

Updates: stored multiple times per session (periodically, for example)

Specified by:

putUpdate in interface StatsStorageRouter

Parameters:

update - Update info to store

putUpdate

public void putUpdate(Collection<? extends Persistable> updates)

Description copied from interface: StatsStorageRouter

Updates: stored multiple times per session (periodically, for example)

Specified by:

putUpdate in interface StatsStorageRouter

Parameters:

updates - Update info to store

putStorageMetaData

public abstract void putStorageMetaData(StorageMetaData storageMetaData)

Description copied from interface: StatsStorageRouter

Method to store some additional metadata for each session. Idea: record the classes used to serialize and deserialize the static info and updates (as a class name). This is mainly used for debugging and validation.

Specified by:

putStorageMetaData in interface StatsStorageRouter

Parameters:

storageMetaData - Storage metadata to store

putStorageMetaData

public void putStorageMetaData(Collection<? extends StorageMetaData> storageMetaData)

Specified by:

putStorageMetaData in interface StatsStorageRouter

registerStatsStorageListener

public void registerStatsStorageListener(StatsStorageListener listener)

Description copied from interface: StatsStorage

Add a new StatsStorageListener. The given listener will called whenever a state change occurs for the stats storage instance

Specified by:

registerStatsStorageListener in interface StatsStorage

Parameters:

listener - Listener to add

deregisterStatsStorageListener

public void deregisterStatsStorageListener(StatsStorageListener listener)

Description copied from interface: StatsStorage

Remove the specified listener, if it is present.

Specified by:

deregisterStatsStorageListener in interface StatsStorage

Parameters:

listener - Listener to remove

removeAllListeners

public void removeAllListeners()

Description copied from interface: StatsStorage

Remove all listeners from the StatsStorage instance

Specified by:

removeAllListeners in interface StatsStorage

getListeners

public List<StatsStorageListener> getListeners()

Description copied from interface: StatsStorage

Get a list (shallow copy) of all listeners currently present

Specified by:

getListeners in interface StatsStorage

Returns:

List of listeners