What is Eclipse Deeplearning4j?

Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Apache Spark, DL4J brings AI to business environments for use on distributed GPUs and CPUs.

Skymind is its commercial support arm, bundling Deeplearning4j and other libraries such as Tensorflow and Keras in the Skymind Intelligence Layer (Community Edition), a deep learning environment that gives developers an easy, fast way to train and deploy AI models. SKIL CE is free and downloadable here. SKIL acts as a bridge between Python data science environments and the JVM.

Welcome to Eclipse Deeplearning4j

Deeplearning4j aims to be cutting-edge plug and play, more convention than configuration, which allows for fast prototyping for data scientists, machine-learning practitioners and software engineers. DL4J is customizable at scale. Released under the Apache 2.0 license, all derivatives of DL4J belong to their authors. DL4J can import neural net models from most major frameworks via Keras, including TensorFlow, Caffe and Theano, bridging the gap between the Python ecosystem and the JVM with a cross-team toolkit for data scientists, data engineers and DevOps. Keras is Deeplearning4j's Python API. Skymind is the second-largest contributor to Keras after Google, and offers commercial support for Keras. Machine learning models are served in production with Skymind's machine learning server.

Distributed

DL4J takes advantage of the latest distributed computing frameworks including Apache Spark and Hadoop to accelerate training. On multi-GPUs, it is equal to Caffe in performance.

Open-Source

The libraries are completely open-source, Apache 2.0, and maintained by the developer community and Skymind team.

JVM/Python/C++

Deeplearning4j is written in Java and is compatible with any JVM language, such as Scala, Clojure or Kotlin. The underlying computations are written in C, C++ and Cuda. Keras will serve as the Python API.

What's included

Deep neural nets are capable of record-breaking accuracy. For a quick neural net introduction, please visit our overview page. In a nutshell, Deeplearning4j lets you compose deep neural nets from various shallow nets, each of which form a so-called `layer`. This flexibility lets you combine restricted Boltzmann machines, other autoencoders, convolutional nets or recurrent nets as needed in a distributed, production-grade framework that works with Spark and Hadoop on top of distributed CPUs or GPUs.

There are a lot of parameters to adjust when you're training a deep-learning network. We've done our best to explain them, so that Deeplearning4j can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers.


Features

  • Distributed CPUs and GPUs
  • Java, Scala and Python APIs
  • Adapted for micro-service architecture
  • Parallel training via iterative reduce
  • Scalable on Hadoop
  • GPU support for scaling on AWS

Libraries

  • Deeplearning4J: Neural Net Platform
  • ND4J: Numpy for the JVM
  • DataVec: Tool for Machine Learning ETL Operations
  • JavaCPP: The Bridge Between Java and Native C++
  • Arbiter: Evaluation Tool for Machine Learning Algorithms
  • RL4J: Deep Reinforcement Learning for the JVM

Learn More

If you have any questions, please join us on Gitter; for premium support, contact us at Skymind. ND4J is the Java-based scientific computing engine powering our matrix operations. On large matrices, our benchmarks show it runs roughly twice as fast as Numpy. DL4J's Programming Guide is available here.


  • What Are the Use Cases for AI and Machine Learning?
    AI tools like Deeplearning4j can be applied to robotic process automation (RPA), Fraud detection, network intrusion detection, Recommender Systems (CRM, adtech, churn prevention), Regression and predictive analytics, Face/image recognition, Voice search, Speech-to-text (transcription), and preventative hardware monitoring (anomaly detection).
  • Why Eclipse Deeplearning4j?
    With a versatile n-dimensional array class for Java and Scala, DL4J is Scalable on Hadoop, utlizes GPU support for scaling on AWS, includes a general vectorization tool for machine-learning libs, and most of all relies on ND4J: A matrix library much faster than Numpy and largely written in C++. We also built RL4J: Reinforcement Learning for Java with Deep Q learning and A3C.
  • How can I contribute?
    Developers who would like to contribute to Deeplearning4j can get started by reading our Developer's Guide.
  • Is DL4J parallelized?
    Deeplearning4j includes both a distributed, multi-threaded deep-learning framework and a normal single-threaded deep-learning framework. Training takes place in the cluster, which means it can process massive amounts of data quickly. Nets are trained in parallel via iterative reduce, and they are equally compatible with Java, Scala, Clojure and Kotlin. Deeplearning4j's role as a modular component in an open stack makes it the first deep-learning framework adapted for a micro-service architecture.
  • Can I program in Python with Deeplearning4j?
    Yes! Deeplearning4j's Python API employs Keras, a high-level, intuitive abstraction layer that also takes TensorFlow and Theano as a backend. Teams that have trained models on other Python frameworks can import them to the JVM and Deeplearning4j using Keras model import. And developers just starting to train configure and train a model can do so via our Python interface. Skymind is the second-largest contributor to Keras and offers commercial support for the framework.
  • What do I need to build AI at my company?
    Broadly speaking, CIOs and product managers building an AI solution require four things: talent, data, tooling and infrastructure.
  • Is another AI winter coming?
    Probably not, but some smart people are rightly tired of the AI hype, and they are trying to deflate it by talking about the next AI winter.

Continue reading

Chat with us on Gitter