What Is DeepLearning4j?

Deeplearning4j is a Java-based toolkit for building, training and deploying deep neural networks, the regressions and KNN.

Deeplearning4j Components

Deeplearning4j has the following sub-projects.

  • DataVec performs data ingestion, normalization and transformation into feature vectors
  • Deeplearning4j provides tools to configure neural networks and build computation graphs
  • DL4J-Examples contains working examples for classification and clustering of images, time series and text.
  • Keras Model Import helps import trained models from Python and Keras to DeepLearning4J and Java.
  • ND4J allows Java to access Native Libraries to quickly process Matrix Data on CPUs or GPUs.
  • ScalNet is a Scala wrapper for Deeplearning4j inspired by Keras. Runs on multi-GPUs with Spark.
  • RL4J implements Deep Q Learning, A3C and other reinforcement learning algorithms for the JVM.
  • Arbiter helps search the hyperparameter space to find the best neural net configuration.


Ingesting, cleaning, joining, scaling, normalizing and transforming data are jobs that must be done in any sort of data analysis. This work may not be exciting, but it’s a precondition of deep learning. DataVec is our toolkit for that process. We give data scientists and developers tools to turn raw data such as images, video, audio, text and time series into feature vectors for neural nets.

Github Repo

The DataVec Github repo is here. Here is how the repo is organized.

  • datavec-api : rules for preprocessing data and defining data pipelines.
  • datavec-data : knows how to understand audio, video, images, text data types
  • datavec-spark : runs distributed data pipelines on Spark
  • datavec-local : runs Datavec standalone on desktop. For inference.
  • datavec-camel : connects to external Camel components. Camel allows you to define routes and integrates with many data sources. DataVec-camel sends data to datavec as a destination from whichever Camel source you specify.

DataVec Examples

There are DataVec examples in our examples repo on Github.

A descriptive summary of many of the examples is here.


Here is the DataVec JavaDoc.

DataVec overview

Neural Networks process multi-dimensional arrays of numerical data. Getting your data from a CSV file, or a directory of images, to be serialized into numeric arrays is the job of DataVec. DataVec is an ETL tool specifically built for machine learning pipelines.

DataVec: Commonly used classes

Here’s a list of some important DataVec classes:

  • Input Split

Splitting data into Test and Train

  • InputSplit.sample to split data into Test and Train

Randomize Data

  • FileSplit.random to randomize data

Base class for reading and serializing data. RecordReaders ingest your data input and return a List of Serializable objects (Writables).

  • RecordReader

Implementations of particular RecordReaders:

  • CSVRecordReader for CSV data
  • CSVNLinesSequenceRecordReader for Sequence Data
  • ImageRecordReader for images
  • JacksonRecordReader for JSON data
  • RegexLineRecordReader for parsing log files
  • WavFileRecordReader for audio files
  • LibSvmRecordReader for Support Vector Machine
  • VideoRecordReader for reading Video

For re-organizing, joining, normalizing and transforming data.

  • Transform

Specific transform implementations

  • CategoricalToIntegerTransform to convert category names to integers
  • CategoricalToOneHotTransform convert catagory name to onehot representation
  • ReorderColumnsTransform rearrange columns
  • RenameColumnsTransform rename columns
  • StringToTimeTransform convert timestring

The labels for data input may be based on the directory where the image is stored.

  • ParentPathLabelGenerator Label based on parent directory
  • PatternPathLabelGenerator Derives label based on a string within the file path


  • Normalizer


Deeplearning4j is where you design your neural networks. It is a domain specific language (DSL) for configuring neural networks.

Github Repo

The Deeplearning4j Github repo is here. Here’s how the repo is organized.

  • deeplearning4j-core : datasetiterators and everything you need to run dl4j on the desktop.
  • deeplearning4j-cuda : cudnn and anything cuda specific.
  • deeplearning4j-graph : graph processing for deepwalk.
  • deeplearning4j-modelimport: this imports neural net models from Keras, which in turn can import models from major frameworks like Theano, Tensorflow, Caffe and CNTK
  • deeplearning4j-nlp-parent: text analytics for English, Japanese and Korean as well as external tokenizers and plugins to toolsets like like UIMA, which itself performs dependency parsing, semantic role labeling, relation extraction and QA systems. We integrate with toolsets like UIKMA to pass stuff to word2vec.
  • nlp: Word2vec, doc2vec and other tools.
  • deeplearning4j-nn : a pared-down neural net DSL with fewer dependencies. Configures multilayer nets with a builder pattern for setting hyperparameters.
  • deeplearning4j-scaleout : AWS provisioning, parallelwrapper desktop parameter averaging (single box 96 cores) so you don’t have to run spark if you don’t want to; one for parameter server and the other not; streaming is kafka and spark streaming; spark is spark training and nlp on spark: dist. word2vec
  • deeplearning4j-ui-parent : neural net training heuristics and visualization


Here is the Deeplearning4j JavaDoc here.

DeepLearning4J Examples

There are Deeplearning4j examples in the Github repository here.

A descriptive summary of many of the examples is here.

Deeplearning4j: Frequently used classes

  • MultiLayerConfiguration Configure a network
  • MultiLayerConfiguration.Builder Builder interface to configure a network
  • MultiLayerNetwork Builds a Network from the configuration
  • ComputationGraph Builds a Computation Graph style Network
  • ComputationGraphConfiguration Configuration for Computation Graph
  • ComputationGraphConfiguration.GraphBuilder Builder interface for Computation Graph configuration
  • UiServer Adds a web based Gui to view training parameter progress and configuration of Network

Keras Model Import

If you have worked with the Python Library Keras and would like to import a trained model, or a model configuration, into Deeplearning4j, please see our model import feature.

Github Repo

The Model Import is actually part of DeepLearning4J, but it is worth its own section. Github folder is here.

Model Import Examples

We will add examples here

Model Import Frequently Used Classes

  • KerasModel.Import saved Keras Model to DeepLearning4J MultiLayerNetwork or Computation Graph


Here’s a video showing how to import of a Keras model to DL4J:


ND4J is the numerical computing library that underpins Deeplearning4j. It is a tensor library, the JVM’s answer to Numpy.

Github Repo

Here is the ND4J Github repo. ND4J is a DSL for handling n-dimensional arrays (NDArrays), also known as tensors.


Here is the ND4J JavaDoc.

ND4J Examples

Here are ND4J examples.

ND4J: Frequently Used Classes

You may not use some of these classes directly, but when you configure a neural network. Behind the scenes, the configurations you set for OptimizationAlgorithm, Updater, and LossFunction are all done in ND4J.

  • DataSetPreProcessor tools for Normalizing an image or numerical data
  • BaseTransformOp Activation functions, tanh, sigmoid, relu, Softmax …
  • GradientUpdater Stochastic Gradient Descent, AdaGrad, Adam, Nesterovs ..


ScalNet is Keras for Scala. It’s a Scala wrapper for Deeplearning4j that can run Spark on multi-GPUs.

Github Repo


RL4J is a library and environment for reinforcement learning on the JVM. It includes Deep Q learning, A3C and other algorithms implemented in Java and integrated with DL4J and ND4J.

Github Repo


Arbiter helps you search the hyperparameter space to find the best tuning and architecture for a neural net. This is important because finding the right architecture and hyperparamters is a very large combinatorial problem. The winning ImageNet teams at corporate labs like Microsoft are searching through hyperparameters to surface 150-layer networks like ResNet. Arbiter includes grid search, random search, some Bayesian methods, as well as model evaluation tools.

Github Repo

Here is the Arbiter Github repository.

  • arbiter-core : Arbiter-core searches the hyperparameter space with algorithms like grid search. Provides a GUI.
  • arbiter-deeplearning4j : Arbiter can talk to DL4J models. When you do model search, you need to be able to run the model. This pilots the model and finds the best model.

More Machine Learning Tutorials

For people just getting started with deep learning, the following tutorials and videos provide an easy entrance to the fundamental ideas of deep neural networks:

Chat with us on Gitter