What Is DeepLearning4j?

DeepLearning4J is a java based toolkit for building, training and deploying Neural Networks.

DeepLearning4j Components

DeepLearning4J has the following sub-projects.

  • DataVec performs data ingestion, normalization and transformation into feature vectors
  • DeepLearning4j provides tools to configure neural networks and build computation graphs
  • DL4J-Examples contains working examples for classification and clustering of images, time series and text.
  • Keras Model Import helps import trained models from Python and Keras to DeepLearning4J and Java.
  • ND4J allows Java to access Native Libraries to quickly process Matrix Data on CPUs or GPUs.
  • ScalNet is a Scala wrapper for Deeplearning4j inspired by Keras. Runs on multi-GPUs with Spark.
  • RL4J implements Deep Q Learning, A3C and other reinforcement learning algorithms for the JVM.
  • Arbiter helps search the hyperparameter space to find the best neural net configuration.


Ingesting, cleaning, joining, scaling, normalizing and transforming data are jobs that must be done in any sort of analysis. This work may not be exciting, but it’s a precondition of deep learning. DataVec is our toolkit for that process. We give data scientists and developers tools to turn raw data such as images, video, audio, text and time series into feature vectors for neural nets.

Github Repo

The DataVec Github repo is here. Here is how the repo breaks down.

  • datavec-dataframe : DataVec’s built-in equivalent of Pandas Dataframe
  • datavec-api : rules for preprocessing data and defining data pipelines.
  • datavec-data : knows how to understand audio, video, images, text data types
  • datavec-spark : runs distributed data pipelines on Spark
  • datavec-local : runs Datavec standalone on desktop. For inference.
  • datavec-camel : connects to external Camel components. Camel allows you to define routes and integrates with many data sources. DataVec-camel sends data to datavec as a destination from whichever Camel source you specify.

DataVec Examples

There are DataVec examples in our examples repo on Github here.

A descriptive summary of many of the examples is here.


DataVec JavaDoc is here.

DataVec overview

Neural Networks process multi-dimensional arrays of numerical data. Getting your data from a CSV file, or a directory of images serialized into numeric arrays is the job of DataVec.

DataVec: Commonly used classes

Here’s a list of some important DataVec classes:

  • Input Split

Splitting data into Test and Train

  • InputSplit.sample to split data into Test and Train

Randomize Data

  • FileSplit.random to randomize data

Base class for reading and serializing data. RecordReaders ingest your data input and return a List of Serializable objects (Writables).

  • RecordReader

Implementations of particular RecordReaders

  • CSVRecordReader for CSV data
  • CSVNLinesSequenceRecordReader for Sequence Data
  • ImageRecordReader for images
  • JacksonRecordReader for JSON data
  • RegexLineRecordReader for parsing log files
  • WavFileRecordReader for audio files
  • LibSvmRecordReader for Support Vector Machine
  • VideoRecordReader for reading Video

For re-organizing, joining, Normalizing and transforming data.

  • Transform

Specific Transform implementations

  • CategoricalToIntegerTransform to convert category names to integers
  • CategoricalToOneHotTransform convert catagory name to onehot representation
  • ReorderColumnsTransform rearrange columns
  • RenameColumnsTransform rename columns
  • StringToTimeTransform convert timestring

The labels for data input may be based on the directory where the image is stored.

  • ParentPathLabelGenerator Label based on parent directory
  • PatternPathLabelGenerator Derives label based on a string within the file path


  • Normalizer Although part of ND4J seems like it should be mentioned here


DeepLearning4J is where you design your Neural Networks

Github Repo

The DeepLearning4J Github repo is here. Here’s how the repo breaks down.

  • deeplearning4j-core : datasetiterators and everything you need to run dl4j on the desktop.
  • deeplearning4j-cuda : cudnn and anything cuda specific.
  • deeplearning4j-graph : graph processing for deepwalk.
  • deeplearning4j-modelimport: this imports neural net models from Keras, which in turn can import models from major frameworks like Theano, Tensorflow, Caffe and CNTK
  • deeplearning4j-nlp-parent: text analytics for English, Japanese and Korean as well as external tokenizers and plugins to toolsets like like UIMA, which itself performs dependency parsing, semantic role labeling, relation extraction and QA systems. We integrate with toolsets like UIKMA to pass stuff to word2vec.
  • nlp: Word2vec, doc2vec and other tools.
  • deeplearning4j-nn : a pared-down neural net DSL with fewer dependencies. Configures multilayer nets with a builder pattern for setting hyperparameters.
  • deeplearning4j-scaleout : AWS provisioning, parallelwrapper desktop parameter averaging (single box 96 cores) so you don’t have to run spark if you don’t want to; one for parameter server and the other not; streaming is kafka and spark streaming; spark is spark training and nlp on spark: dist. word2vec
  • deeplearning4j-ui-parent : neural net training heuristics and visualization


The DeepLearning4J JavaDoc is available here.

DeepLearning4J Examples

There are DeepLearning4J examples in the Github repository here.

A descriptive summary of many of the examples is here.

DeepLearning4J frequently used classes

  • MultiLayerConfiguration Configure a network
  • MultiLayerConfiguration.Builder Builder interface to configure a network
  • MultiLayerNetwork Builds a Network from the configuration
  • ComputationGraph Builds a Computation Graph style Network
  • ComputationGraphConfiguration Configuration for Computation Graph
  • ComputationGraphConfiguration.GraphBuilder Builder interface for Computation Graph configuration
  • UiServer Adds a web based Gui to view training parameter progress and configuration of Network

Model Import

If you have worked with Keras the Python Library for Deeplearning and would like to import a trained model, or a model configuration into DeepLearning4J see our Model Import feature.

Github Repo

The Model Import is actually part of DeepLearning4J, but it is worth it’s own section. Github folder is here.

Model Import Examples

We will add examples here

Model Import Frequently Used Classes

  • KerasModel.Import saved Keras Model to DeepLearning4J MultiLayerNetwork or Computation Graph


Here’s a video showing how to import of a Keras model to DL4J:


ND4J is the numerical processing library for DeepLearning4J, a tensor library, the JVM’s answer to Numpy.

Github Repo

The ND4J Github repo is here. ND4J is a DSL for handling n-dimensional arrays (NDArrays).


The ND4J JavaDoc is available here.

ND4J Examples

There are ND4J examples here.

ND4J Frequently Used Classes

You may not use some of these classes directly, but when you configure a Neural Network. Behind the scenes the configurations you set for OptimizationAlgorithm, Updater, and LossFunction are all done in ND4J.

  • DataSetPreProcessor tools for Normalizing an image or numerical data
  • BaseTransformOp Activation functions, tanh, sigmoid, relu, Softmax …
  • GradientUpdater Stochastic Gradient Descent, AdaGrad, Adam, Nesterovs ..


ScalNet is Keras for Scala. It’s a Scala wrapper for Deeplearning4j that can run Spark on multi-GPUs.

Github Repo


RL4J is a library and environment for for Deep Q learning, A3C and other algorithms implemented in Java and integrated with DL4J and ND4J.

Github Repo


Arbiter helps you search the hyperparameter space to find the best tuning and architecture for a neural net. This is important because finding the right architecture and hyperparamters is a very large combinatorial problem. The winning ImageNet teams at corporate labs like Microsoft are searching through hyperparameters to surface 150-layer networks like ResNet.

Github Repo

The Github repository is here.

  • arbiter-core : Aribter-core searches the hyperparameter space with algorithms like grid search. Provides a GUI.
  • arbiter-deeplearning4j : Arbiter can talk to DL4J models. When you do model search, you need to be able to run the model. This pilots the model and finds the best model.
Chat with us on Gitter