public class JsonRemoteInference<I,O>
This class provides remote inference functionality via JSON-powered REST APIs.
Basically we assume that there's remote JSON server available (on bare metal or in k8s/swarm/whatever cluster), and with proper serializers/deserializers provided we can issue REST requests and get back responses.
So, in this way application logic can be separated from DL logic.
You just need to provide serializer/deserializer and address of the REST server, i.e. "http://model:8080/v1/serving"