org.datavec.python

Class PythonExecutioner

java.lang.Object

org.datavec.python.PythonExecutioner

public class PythonExecutioner
extends Object

Allows execution of python scripts managed by an internal interpreter. An end user may specify a python script to run via any of the execution methods available in this class. At static initialization time (when the class is first initialized) a number of components are setup: 1. The python path. A user may over ride this with the system property DEFAULT_PYTHON_PATH_PROPERTY 2. Since this executioner uses javacpp to manage and run python interpreters underneath the covers, a user may also over ride the system property JAVACPP_PYTHON_APPEND_TYPE with one of the PythonExecutioner.JavaCppPathType values. This will allow the user to determine whether the javacpp default python path is used at all, and if so whether it is appended, prepended, or not used. This behavior is useful when you need to use an external python distribution such as anaconda. 3. A proper numpy import for use with javacpp: We call numpy import ourselves to ensure proper loading of native libraries needed by numpy are allowed to load in the proper order. If we don't do this, it causes a variety of issues with running numpy. (User must still include "import numpy as np" in their scripts). 4. Various python scripts pre defined on the classpath included right with the java code. These are auxillary python scripts used for loading classes, pre defining certain kinds of behavior in order for us to manipulate values within the python memory, as well as pulling them out of memory for integration within the internal python executioner. For more information on how this works, please take a look at the init() method. Generally, a user defining a python script for use by the python executioner will have a set of defined target input values and output values. These values should not be present when actually running the script, but just referenced. In order to test your python script for execution outside the engine, we recommend commenting out a few default values as dummy input values. This will allow an end user to test their script before trying to use the server. In order to get output values out of a python script, all a user has to do is define the output variables they want being used in the final output in the actual pipeline. For example, if a user wants to return a dictionary, they just have to create a dictionary with that name and based on the configured PythonVariables passed as outputs to one of the execution methods, we can pull the values out automatically. For input definitions, it is similar. You just define the values you want used in PythonVariables and we will automatically generate code for defining those values as desired for running. This allows the user to customize values dynamically at runtime but reference them by name in a python script.

Author:

Fariz Rahman, Adam Gibson

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PythonExecutioner.JavaCppPathType One of a few desired values for how we should handle using javacpp's python path.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_APPEND_TYPE
static String	DEFAULT_PYTHON_PATH_PROPERTY
static String	JAVACPP_PYTHON_APPEND_TYPE

Constructor Summary

Constructors

Constructor and Description
PythonExecutioner()

Method Summary

Modifier and Type	Method and Description
static void	exec(String code)
static void	exec(String code, PythonVariables inputVariables, PythonVariables outputVariables)
static PythonVariables	execAndReturnAllVariables(String code)
static PythonVariables	execAndReturnAllVariables(String code, PythonVariables inputs)
static PythonVariables	getAllVariables()
static PythonObject	getVariable(String varName)
static <T> T	getVariable(String varName, PythonType<T> varType)
static void	getVariables(PythonVariables pyVars)
static void	initPythonPath() Set the python path.
static void	setVariable(String varName, PythonObject pythonObject) Sets a variable in the global scope of the current context (See @PythonContextManager).
static <T> void	setVariable(String varName, PythonType varType, Object value)
static void	setVariables(PythonVariables pyVars)
static boolean	validateVariableName(String s)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_PYTHON_PATH_PROPERTY

public static final String DEFAULT_PYTHON_PATH_PROPERTY

See Also:

Constant Field Values

JAVACPP_PYTHON_APPEND_TYPE

public static final String JAVACPP_PYTHON_APPEND_TYPE

See Also:

Constant Field Values

DEFAULT_APPEND_TYPE

public static final String DEFAULT_APPEND_TYPE

See Also:

Constant Field Values

Constructor Detail

PythonExecutioner

public PythonExecutioner()

Method Detail

validateVariableName

public static boolean validateVariableName(String s)

setVariable

public static void setVariable(String varName,
                               PythonObject pythonObject)
                        throws PythonException

Sets a variable in the global scope of the current context (See @PythonContextManager). This is equivalent to `exec("a = b");` where a is the variable name and b is the variable value.

Parameters:

varName - Name of the python variable being set. Should be a valid python identifier string

pythonObject - Value for the python variable

Throws:

Exception

PythonException

setVariable

public static <T> void setVariable(String varName,
                                   PythonType varType,
                                   Object value)
                            throws PythonException

Throws:

PythonException

setVariables

public static void setVariables(PythonVariables pyVars)
                         throws PythonException

Throws:

PythonException

getVariable

public static PythonObject getVariable(String varName)

getVariable

public static <T> T getVariable(String varName,
                                PythonType<T> varType)
                         throws PythonException

Throws:

PythonException

getVariables

public static void getVariables(PythonVariables pyVars)
                         throws PythonException

Throws:

PythonException

exec

public static void exec(String code)
                 throws PythonException

Throws:

PythonException

exec

public static void exec(String code,
                        PythonVariables inputVariables,
                        PythonVariables outputVariables)
                 throws PythonException

Throws:

PythonException

execAndReturnAllVariables

public static PythonVariables execAndReturnAllVariables(String code)
                                                 throws PythonException

Throws:

PythonException

getAllVariables

public static PythonVariables getAllVariables()
                                       throws PythonException

Throws:

PythonException

execAndReturnAllVariables

public static PythonVariables execAndReturnAllVariables(String code,
                                                        PythonVariables inputs)
                                                 throws Exception

Throws:

Exception

initPythonPath

public static void initPythonPath()

Set the python path. Generally you can just use the PYTHONPATH environment variable, but if you need to set it from code, this can work as well.