| Modifier and Type | Class and Description |
|---|---|
class |
Configuration
Provides access to configuration parameters.
|
| Modifier and Type | Interface and Description |
|---|---|
interface |
InputFormat
Create an input format
|
| Modifier and Type | Class and Description |
|---|---|
class |
BaseInputFormat |
| Modifier and Type | Class and Description |
|---|---|
class |
CSVInputFormat
Line input format creates an @link{LineRecordReader}
|
class |
LibSvmInputFormat
Lib svm input format
|
class |
LineInputFormat
Line input format creates an @link{LineRecordReader}
|
class |
ListStringInputFormat
Input format for the @link {ListStringRecordReader}
|
class |
MatlabInputFormat
Matlab input format
|
class |
SVMLightInputFormat
SVMLight input format
|
| Modifier and Type | Interface and Description |
|---|---|
interface |
WritableComparable<T>
A
Writable which is also Comparable. |
| Modifier and Type | Method and Description |
|---|---|
static <T extends Writable> |
WritableUtils.clone(T orig,
Configuration conf)
Make a copy of a writable object using serialization to a buffer.
|
| Modifier and Type | Method and Description |
|---|---|
Writable |
WritableConverter.convert(Writable writable)
Convert a writable to another kind of writable
|
| Modifier and Type | Method and Description |
|---|---|
Writable |
WritableConverter.convert(Writable writable)
Convert a writable to another kind of writable
|
static byte[] |
WritableUtils.toByteArray(Writable... writables)
Convert writables to a byte array
|
| Modifier and Type | Method and Description |
|---|---|
Writable |
DoubleWritableConverter.convert(Writable writable) |
Writable |
FloatWritableConverter.convert(Writable writable) |
Writable |
LabelWriterConverter.convert(Writable writable) |
Writable |
SelfWritableConverter.convert(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
DoubleWritableConverter.convert(Writable writable) |
Writable |
FloatWritableConverter.convert(Writable writable) |
Writable |
LabelWriterConverter.convert(Writable writable) |
Writable |
SelfWritableConverter.convert(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
ParentPathLabelGenerator.getLabelForPath(String path) |
Writable |
PathLabelGenerator.getLabelForPath(String path) |
Writable |
PatternPathLabelGenerator.getLabelForPath(String path) |
Writable |
ParentPathLabelGenerator.getLabelForPath(URI uri) |
Writable |
PathLabelGenerator.getLabelForPath(URI uri) |
Writable |
PatternPathLabelGenerator.getLabelForPath(URI uri) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
PathMultiLabelGenerator.getLabels(String uriPath) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
Record.getRecord()
Get the record values, as a
List<Writable> |
List<List<Writable>> |
SequenceRecord.getSequenceRecord()
Get the sequence record values
|
List<Writable> |
SequenceRecord.getTimeStep(int timeStep)
Get a single time step.
|
| Modifier and Type | Method and Description |
|---|---|
void |
Record.setRecord(List<Writable> record)
Get the record values for this Record
|
void |
SequenceRecord.setSequenceRecord(List<List<Writable>> sequenceRecord)
Set the sequence record values
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
SequenceRecord.getTimeStep(int timeStep) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
RecordReader.next()
Get the next record
|
List<List<Writable>> |
BaseRecordReader.next(int num) |
List<List<Writable>> |
RecordReader.next(int num)
This method will be used, if batchesSupported() returns true.
|
List<Writable> |
RecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
RecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<List<Writable>> |
SequenceRecordReader.sequenceRecord()
Returns a sequence record.
|
List<List<Writable>> |
SequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
RecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ComposableRecordReader.next() |
List<Writable> |
ConcatenatingRecordReader.next() |
List<Writable> |
FileRecordReader.next() |
List<Writable> |
LineRecordReader.next() |
List<List<Writable>> |
FileRecordReader.next(int num) |
List<Writable> |
ComposableRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
ConcatenatingRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
FileRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
LineRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
CollectionRecordReader.next() |
List<Writable> |
CollectionSequenceRecordReader.next() |
List<Writable> |
ListStringRecordReader.next()
Get the next record
|
List<Writable> |
CollectionRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
CollectionSequenceRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
ListStringRecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
ListStringRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<List<Writable>> |
CollectionSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
CollectionSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Constructor and Description |
|---|
CollectionRecordReader(Collection<? extends Collection<Writable>> records) |
CollectionSequenceRecordReader(Collection<? extends Collection<? extends Collection<Writable>>> records) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
CSVRecordReader.next() |
List<List<Writable>> |
CSVRecordReader.next(int num) |
protected List<Writable> |
CSVRecordReader.parseLine(String line) |
protected List<Writable> |
CSVRegexRecordReader.parseLine(String line) |
List<Writable> |
CSVRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
CSVLineSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
CSVMultiSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
CSVNLinesSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
CSVSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
CSVVariableSlidingWindowRecordReader.sequenceRecord() |
List<List<Writable>> |
CSVLineSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
CSVMultiSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
CSVNLinesSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
CSVSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
CSVVariableSlidingWindowRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Constructor and Description |
|---|
CSVMultiSequenceRecordReader(int skipNumLines,
char elementDelimiter,
char quote,
String sequenceSeparatorRegex,
CSVMultiSequenceRecordReader.Mode mode,
Writable padValue)
Create a sequence reader using the default value for skip lines (0), the default delimiter (',') and the default
quote character ('"')
|
CSVMultiSequenceRecordReader(String sequenceSeparatorRegex,
CSVMultiSequenceRecordReader.Mode mode,
Writable padValue)
Create a sequence reader using the default value for skip lines (0), the default delimiter (',') and the default
quote character ('"')
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
FileBatchRecordReader.next() |
List<Writable> |
FileBatchSequenceRecordReader.next() |
List<List<Writable>> |
FileBatchRecordReader.next(int num) |
List<List<Writable>> |
FileBatchSequenceRecordReader.next(int num) |
List<Writable> |
FileBatchRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
FileBatchSequenceRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
FileBatchSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
FileBatchSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
InMemoryRecordReader.next()
Get the next record
|
List<Writable> |
InMemorySequenceRecordReader.next()
Get the next record
|
List<List<Writable>> |
InMemoryRecordReader.next(int num) |
List<List<Writable>> |
InMemorySequenceRecordReader.next(int num) |
List<Writable> |
InMemoryRecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
InMemoryRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<Writable> |
InMemorySequenceRecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
InMemorySequenceRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<List<Writable>> |
InMemorySequenceRecordReader.sequenceRecord()
Returns a sequence record.
|
List<List<Writable>> |
InMemorySequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
InMemorySequenceRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
| Constructor and Description |
|---|
InMemoryRecordReader(List<List<Writable>> records) |
InMemorySequenceRecordReader(List<List<List<Writable>>> records) |
| Modifier and Type | Field and Description |
|---|---|
static Writable |
FieldSelection.DEFAULT_MISSING_VALUE |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
FieldSelection.getValueIfMissing() |
List<Writable> |
JacksonLineRecordReader.next() |
List<Writable> |
JacksonRecordReader.next() |
protected List<Writable> |
JacksonLineRecordReader.parseLine(String line) |
static List<Writable> |
JacksonReaderUtils.parseRecord(String line,
FieldSelection selection,
org.nd4j.shade.jackson.databind.ObjectMapper mapper) |
List<Writable> |
JacksonRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
JacksonLineSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
JacksonLineSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
FieldSelection.Builder |
FieldSelection.Builder.addField(Writable valueIfMissing,
String... fieldPath) |
| Modifier and Type | Field and Description |
|---|---|
static Writable |
SVMLightRecordReader.LABEL_ONE |
static Writable |
SVMLightRecordReader.LABEL_ZERO |
static Writable |
SVMLightRecordReader.ONE |
protected Writable |
SVMLightRecordReader.recordLookahead |
static Writable |
SVMLightRecordReader.ZERO |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
SVMLightRecordReader.getNextRecord()
Helper function to help detect lines that are
commented out.
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
MatlabRecordReader.next() |
List<Writable> |
SVMLightRecordReader.next()
Return next record as list of Writables.
|
List<Writable> |
MatlabRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
SVMLightRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
RegexLineRecordReader.next() |
List<Writable> |
RegexLineRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
RegexSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
RegexSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TransformProcessRecordReader.next()
Get the next record
|
List<Writable> |
TransformProcessSequenceRecordReader.next()
Get the next record
|
List<List<Writable>> |
TransformProcessRecordReader.next(int num) |
List<List<Writable>> |
TransformProcessSequenceRecordReader.next(int num) |
List<Writable> |
TransformProcessRecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
TransformProcessRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<Writable> |
TransformProcessSequenceRecordReader.record(URI uri,
DataInputStream dataInputStream)
Load the record from the given DataInputStream
Unlike
TransformProcessSequenceRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
List<List<Writable>> |
TransformProcessSequenceRecordReader.sequenceRecord()
Returns a sequence record.
|
List<List<Writable>> |
TransformProcessSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream)
Load a sequence record from the given DataInputStream
Unlike
TransformProcessSequenceRecordReader.next() the internal state of the RecordReader is not modified
Implementations of this method should not close the DataInputStream |
| Modifier and Type | Method and Description |
|---|---|
PartitionMetaData |
SequenceRecordWriter.write(List<List<Writable>> sequence)
Write a record
|
PartitionMetaData |
RecordWriter.write(List<Writable> record)
Write a record
|
PartitionMetaData |
RecordWriter.writeBatch(List<List<Writable>> batch)
Write a batch of records
|
| Modifier and Type | Method and Description |
|---|---|
PartitionMetaData |
LineRecordWriter.write(List<Writable> record) |
PartitionMetaData |
LineRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Method and Description |
|---|---|
PartitionMetaData |
CSVRecordWriter.write(List<Writable> record) |
PartitionMetaData |
CSVRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Method and Description |
|---|---|
PartitionMetaData |
MatlabRecordWriter.write(List<Writable> record) |
PartitionMetaData |
SVMLightRecordWriter.write(List<Writable> record)
Write next record.
|
PartitionMetaData |
MatlabRecordWriter.writeBatch(List<List<Writable>> batch) |
PartitionMetaData |
SVMLightRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Method and Description |
|---|---|
static Pair<INDArray,INDArray> |
TimeSeriesWritableUtils.convertWritablesSequence(List<List<List<Writable>>> timeSeriesRecord)
Convert the writables
to a sequence (3d) data set,
and also return the
mask array (if necessary)
|
static Pair<INDArray,INDArray> |
TimeSeriesWritableUtils.convertWritablesSequence(List<List<List<Writable>>> list,
TimeSeriesWritableUtils.RecordDetails details)
Convert the writables
to a sequence (3d) data set,
and also return the
mask array (if necessary)
|
static TimeSeriesWritableUtils.RecordDetails |
TimeSeriesWritableUtils.getDetails(List<List<List<Writable>>> record)
Get the
TimeSeriesWritableUtils.RecordDetails
detailing the length of the time series |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TransformProcess.execute(List<Writable> input)
Execute the full sequence of transformations for a single example.
|
List<List<Writable>> |
TransformProcess.executeSequence(List<List<Writable>> inputSequence)
Execute the full sequence of transformations for a single time series (sequence).
|
List<List<Writable>> |
TransformProcess.executeSequenceToSequence(List<List<Writable>> input) |
List<Writable> |
TransformProcess.executeSequenceToSingle(List<List<Writable>> inputSequence)
Execute a TransformProcess that starts with a sequence
record, and converts it to a single (non-sequence) record
|
List<List<Writable>> |
TransformProcess.executeToSequence(List<Writable> inputExample)
Execute a TransformProcess that starts with a single (non-sequence) record,
and converts it to a sequence record.
|
List<List<List<Writable>>> |
TransformProcess.executeToSequenceBatch(List<List<Writable>> inputExample)
Execute a TransformProcess that starts with a single (non-sequence) record,
and converts it to a sequence record.
|
List<Writable> |
Transform.map(List<Writable> writables)
Transform a writable
in to another writable
|
List<List<Writable>> |
Transform.mapSequence(List<List<Writable>> sequence)
Transform a sequence
|
List<Writable> |
TransformProcess.transformRawStringsToInput(String... values)
Based on the input schema,
map raw string values to the appropriate
writable
|
List<Writable> |
TransformProcess.transformRawStringsToInputList(List<String> values)
Based on the input schema,
map raw string values to the appropriate
writable
|
List<List<Writable>> |
TransformProcess.transformRawStringsToInputSequence(List<List<String>> sequence)
Transforms a sequence
of strings in to a sequence of writables
(very similar to
TransformProcess.transformRawStringsToInput(String...) |
| Modifier and Type | Method and Description |
|---|---|
TransformProcess.Builder |
TransformProcess.Builder.addConstantColumn(String newColumnName,
ColumnType newColumnType,
Writable fixedValue)
Add a new column, where all values in the column are identical and as specified.
|
TransformProcess.Builder |
TransformProcess.Builder.conditionalReplaceValueTransform(String column,
Writable newValue,
Condition condition)
Replace the values in a specified column with a specified new value, if some condition holds.
|
TransformProcess.Builder |
TransformProcess.Builder.conditionalReplaceValueTransformWithDefault(String column,
Writable yesVal,
Writable noVal,
Condition condition)
Replace the values in a specified column with a specified "yes" value, if some condition holds.
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TransformProcess.execute(List<Writable> input)
Execute the full sequence of transformations for a single example.
|
List<List<Writable>> |
TransformProcess.executeSequence(List<List<Writable>> inputSequence)
Execute the full sequence of transformations for a single time series (sequence).
|
List<List<Writable>> |
TransformProcess.executeSequenceToSequence(List<List<Writable>> input) |
List<Writable> |
TransformProcess.executeSequenceToSingle(List<List<Writable>> inputSequence)
Execute a TransformProcess that starts with a sequence
record, and converts it to a single (non-sequence) record
|
List<List<Writable>> |
TransformProcess.executeToSequence(List<Writable> inputExample)
Execute a TransformProcess that starts with a single (non-sequence) record,
and converts it to a sequence record.
|
List<List<List<Writable>>> |
TransformProcess.executeToSequenceBatch(List<List<Writable>> inputExample)
Execute a TransformProcess that starts with a single (non-sequence) record,
and converts it to a sequence record.
|
List<Writable> |
Transform.map(List<Writable> writables)
Transform a writable
in to another writable
|
List<List<Writable>> |
Transform.mapSequence(List<List<Writable>> sequence)
Transform a sequence
|
| Modifier and Type | Method and Description |
|---|---|
T |
AnalysisCounter.add(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
BytesAnalysisCounter |
BytesAnalysisCounter.add(Writable writable) |
CategoricalAnalysisCounter |
CategoricalAnalysisCounter.add(Writable writable) |
DoubleAnalysisCounter |
DoubleAnalysisCounter.add(Writable writable) |
IntegerAnalysisCounter |
IntegerAnalysisCounter.add(Writable writable) |
LongAnalysisCounter |
LongAnalysisCounter.add(Writable writable) |
NDArrayAnalysisCounter |
NDArrayAnalysisCounter.add(Writable writable) |
StringAnalysisCounter |
StringAnalysisCounter.add(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
HistogramCounter |
CategoricalHistogramCounter.add(Writable w) |
HistogramCounter |
DoubleHistogramCounter.add(Writable w) |
HistogramCounter |
HistogramCounter.add(Writable w) |
HistogramCounter |
NDArrayHistogramCounter.add(Writable w) |
HistogramCounter |
StringHistogramCounter.add(Writable w) |
| Modifier and Type | Method and Description |
|---|---|
T |
QualityAnalysisState.add(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
List<QualityAnalysisState> |
QualityAnalysisAddFunction.apply(List<QualityAnalysisState> analysisStates,
List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
BytesQualityAnalysisState |
BytesQualityAnalysisState.add(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
CategoricalQualityAnalysisState |
CategoricalQualityAnalysisState.add(Writable writable) |
CategoricalQuality |
CategoricalQualityAddFunction.apply(CategoricalQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
IntegerQualityAnalysisState |
IntegerQualityAnalysisState.add(Writable writable) |
IntegerQuality |
IntegerQualityAddFunction.apply(IntegerQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
LongQualityAnalysisState |
LongQualityAnalysisState.add(Writable writable) |
LongQuality |
LongQualityAddFunction.apply(LongQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
RealQualityAnalysisState |
RealQualityAnalysisState.add(Writable writable) |
DoubleQuality |
RealQualityAddFunction.apply(DoubleQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
StringQualityAnalysisState |
StringQualityAnalysisState.add(Writable writable) |
StringQuality |
StringQualityAddFunction.apply(StringQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
TimeQualityAnalysisState |
TimeQualityAnalysisState.add(Writable writable) |
TimeQuality |
TimeQualityAddFunction.apply(TimeQuality v1,
Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
BooleanCondition.condition(List<Writable> list) |
boolean |
Condition.condition(List<Writable> list)
Is the condition satisfied for the current input/example?
|
boolean |
BooleanCondition.conditionSequence(List<List<Writable>> sequence) |
boolean |
Condition.conditionSequence(List<List<Writable>> sequence)
Is the condition satisfied for the current input/sequence?
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
BooleanColumnCondition.columnCondition(Writable writable)
Returns whether the given element
meets the condition set by this operation
|
boolean |
CategoricalColumnCondition.columnCondition(Writable writable) |
boolean |
ColumnCondition.columnCondition(Writable writable)
Returns whether the given element
meets the condition set by this operation
|
boolean |
DoubleColumnCondition.columnCondition(Writable writable) |
boolean |
FloatColumnCondition.columnCondition(Writable writable) |
boolean |
InfiniteColumnCondition.columnCondition(Writable writable) |
boolean |
IntegerColumnCondition.columnCondition(Writable writable) |
boolean |
InvalidValueColumnCondition.columnCondition(Writable writable) |
boolean |
LongColumnCondition.columnCondition(Writable writable) |
boolean |
NaNColumnCondition.columnCondition(Writable writable) |
boolean |
NullWritableColumnCondition.columnCondition(Writable writable) |
boolean |
StringColumnCondition.columnCondition(Writable writable) |
boolean |
TimeColumnCondition.columnCondition(Writable writable) |
boolean |
TrivialColumnCondition.columnCondition(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
BaseColumnCondition.condition(List<Writable> list) |
boolean |
ColumnCondition.condition(List<Writable> list) |
boolean |
TrivialColumnCondition.condition(List<Writable> writables) |
boolean |
BaseColumnCondition.conditionSequence(List<List<Writable>> list) |
boolean |
ColumnCondition.conditionSequence(List<List<Writable>> list) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
SequenceLengthCondition.condition(List<Writable> list) |
boolean |
SequenceLengthCondition.conditionSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
StringRegexColumnCondition.columnCondition(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
abstract boolean |
BaseColumnFilter.removeExample(Writable writable)
Should the example or sequence be removed, based on the values from the specified column?
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
BaseColumnFilter.removeExample(List<Writable> writables) |
boolean |
ConditionFilter.removeExample(List<Writable> writables) |
boolean |
Filter.removeExample(List<Writable> writables) |
boolean |
FilterInvalidValues.removeExample(List<Writable> writables) |
boolean |
InvalidNumColumns.removeExample(List<Writable> writables) |
boolean |
BaseColumnFilter.removeSequence(List<List<Writable>> sequence) |
boolean |
ConditionFilter.removeSequence(List<List<Writable>> sequence) |
boolean |
Filter.removeSequence(List<List<Writable>> sequence) |
boolean |
FilterInvalidValues.removeSequence(List<List<Writable>> sequence) |
boolean |
InvalidNumColumns.removeSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
Join.joinExamples(List<Writable> leftExample,
List<Writable> rightExample)
Join the examples.
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
Join.joinExamples(List<Writable> leftExample,
List<Writable> rightExample)
Join the examples.
|
List<Writable> |
Join.joinExamples(List<Writable> leftExample,
List<Writable> rightExample)
Join the examples.
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
BinaryMetaData.isValid(Writable writable) |
boolean |
BooleanMetaData.isValid(Writable writable) |
boolean |
CategoricalMetaData.isValid(Writable writable) |
boolean |
ColumnMetaData.isValid(Writable writable)
Is the given Writable valid for this column, given the column type and any restrictions given by the
ColumnMetaData object?
|
boolean |
DoubleMetaData.isValid(Writable writable) |
boolean |
FloatMetaData.isValid(Writable writable) |
boolean |
IntegerMetaData.isValid(Writable writable) |
boolean |
LongMetaData.isValid(Writable writable) |
boolean |
NDArrayMetaData.isValid(Writable writable) |
boolean |
StringMetaData.isValid(Writable writable) |
boolean |
TimeMetaData.isValid(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
NDArrayColumnsMathOpTransform.doOp(Writable... input) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
NDArrayDistanceTransform.map(List<Writable> writables) |
List<List<Writable>> |
NDArrayDistanceTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
NDArrayColumnsMathOpTransform.doOp(Writable... input) |
NDArrayWritable |
NDArrayMathFunctionTransform.map(Writable w) |
NDArrayWritable |
NDArrayScalarOpTransform.map(Writable w) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
NDArrayDistanceTransform.map(List<Writable> writables) |
List<List<Writable>> |
NDArrayDistanceTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
<U extends IAggregableReduceOp<T,List<Writable>>> |
AggregableMultiOp.combine(U accu) |
<U extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableMean.combine(U acc) |
<U extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableStdDev.combine(U acc) |
<U extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableVariance.combine(U acc) |
<U extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableCountUnique.combine(U acc) |
<W extends IAggregableReduceOp<String,Writable>> |
StringAggregatorImpls.AggregableStringAppend.combine(W accu) |
<W extends IAggregableReduceOp<String,Writable>> |
StringAggregatorImpls.AggregableStringPrepend.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
AggregableCheckingOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
ByteWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
DoubleWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
FloatWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
IntWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
LongWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,T>> |
StringWritableOp.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableFirst.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableLast.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableSum.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableProd.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableMax.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableMin.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableRange.combine(W accu) |
<W extends IAggregableReduceOp<T,Writable>> |
AggregatorImpls.AggregableCount.combine(W accu) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
AggregatorImpls.AggregableFirst.get() |
Writable |
AggregatorImpls.AggregableLast.get() |
Writable |
AggregatorImpls.AggregableSum.get() |
Writable |
AggregatorImpls.AggregableProd.get() |
Writable |
AggregatorImpls.AggregableMax.get() |
Writable |
AggregatorImpls.AggregableMin.get() |
Writable |
AggregatorImpls.AggregableRange.get() |
Writable |
AggregatorImpls.AggregableCount.get() |
Writable |
AggregatorImpls.AggregableMean.get() |
Writable |
AggregatorImpls.AggregableStdDev.get() |
Writable |
AggregatorImpls.AggregableUncorrectedStdDev.get() |
Writable |
AggregatorImpls.AggregableVariance.get() |
Writable |
AggregatorImpls.AggregablePopulationVariance.get() |
Writable |
AggregatorImpls.AggregableCountUnique.get() |
Writable |
StringAggregatorImpls.AggregableStringAppend.get() |
Writable |
StringAggregatorImpls.AggregableStringPrepend.get() |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
AggregableMultiOp.get() |
| Modifier and Type | Method and Description |
|---|---|
void |
AggregableCheckingOp.accept(Writable writable) |
void |
ByteWritableOp.accept(Writable writable) |
void |
DoubleWritableOp.accept(Writable writable) |
void |
FloatWritableOp.accept(Writable writable) |
void |
IntWritableOp.accept(Writable writable) |
void |
LongWritableOp.accept(Writable writable) |
void |
StringWritableOp.accept(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
void |
DispatchWithConditionOp.accept(List<Writable> ts) |
| Constructor and Description |
|---|
DispatchWithConditionOp(List<IAggregableReduceOp<Writable,List<U>>> ops,
List<Condition> conds) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
ColumnReduction.reduceColumn(List<Writable> columnData)
Reduce a single column.
|
| Modifier and Type | Method and Description |
|---|---|
IAggregableReduceOp<List<Writable>,List<Writable>> |
IAssociativeReducer.aggregableReducer()
An aggregation that has the property that
reduce(List(reduce(List(l1, l2)), l3)) = reduce(List(l1, reduce(List(l2, l3)))
|
IAggregableReduceOp<List<Writable>,List<Writable>> |
IAssociativeReducer.aggregableReducer()
An aggregation that has the property that
reduce(List(reduce(List(l1, l2)), l3)) = reduce(List(l1, reduce(List(l2, l3)))
|
IAggregableReduceOp<List<Writable>,List<Writable>> |
Reducer.aggregableReducer() |
IAggregableReduceOp<List<Writable>,List<Writable>> |
Reducer.aggregableReducer() |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceBytesColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceBytesColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceColumn(List<ReduceOp> op,
ColumnType type,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceColumn(List<ReduceOp> op,
ColumnType type,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceDoubleColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceDoubleColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceFloatColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceFloatColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceIntColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceIntColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceLongColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceLongColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
IAggregableReduceOp<Writable,List<Writable>> |
AggregableColumnReduction.reduceOp()
Reduce a single column.
|
IAggregableReduceOp<Writable,List<Writable>> |
AggregableColumnReduction.reduceOp()
Reduce a single column.
|
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceStringOrCategoricalColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceStringOrCategoricalColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceTimeColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
static IAggregableReduceOp<Writable,List<Writable>> |
AggregableReductionUtils.reduceTimeColumn(List<ReduceOp> lop,
boolean ignoreInvalid,
ColumnMetaData metaData) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
ColumnReduction.reduceColumn(List<Writable> columnData)
Reduce a single column.
|
| Modifier and Type | Method and Description |
|---|---|
<W extends IAggregableReduceOp<Writable,List<Writable>>> |
CoordinatesReduction.CoordinateAggregableReduceOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,List<Writable>>> |
CoordinatesReduction.CoordinateAggregableReduceOp.combine(W accu) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
CoordinatesReduction.CoordinateAggregableReduceOp.get() |
IAggregableReduceOp<Writable,List<Writable>> |
CoordinatesReduction.reduceOp() |
IAggregableReduceOp<Writable,List<Writable>> |
CoordinatesReduction.reduceOp() |
| Modifier and Type | Method and Description |
|---|---|
void |
CoordinatesReduction.CoordinateAggregableReduceOp.accept(Writable writable) |
| Constructor and Description |
|---|
CoordinateAggregableReduceOp(int n,
Supplier<IAggregableReduceOp<Writable,List<Writable>>> initialOp,
String delim) |
CoordinateAggregableReduceOp(int n,
Supplier<IAggregableReduceOp<Writable,List<Writable>>> initialOp,
String delim) |
| Modifier and Type | Method and Description |
|---|---|
<W extends IAggregableReduceOp<Writable,List<Writable>>> |
GeographicMidpointReduction.AverageCoordinateReduceOp.combine(W accu) |
<W extends IAggregableReduceOp<Writable,List<Writable>>> |
GeographicMidpointReduction.AverageCoordinateReduceOp.combine(W accu) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
GeographicMidpointReduction.AverageCoordinateReduceOp.get() |
IAggregableReduceOp<Writable,List<Writable>> |
GeographicMidpointReduction.reduceOp() |
IAggregableReduceOp<Writable,List<Writable>> |
GeographicMidpointReduction.reduceOp() |
| Modifier and Type | Method and Description |
|---|---|
void |
GeographicMidpointReduction.AverageCoordinateReduceOp.accept(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
static Schema |
Schema.infer(List<Writable> record)
Infers a schema based on the record.
|
static Schema |
Schema.inferMultiple(List<List<Writable>> record)
Infers a schema based on the record.
|
static SequenceSchema |
SequenceSchema.inferSequence(List<List<Writable>> record)
Infers a sequence schema based
on the record
|
static SequenceSchema |
SequenceSchema.inferSequenceMulti(List<List<List<Writable>>> record)
Infers a sequence schema based
on the record
|
| Modifier and Type | Method and Description |
|---|---|
double |
TypeConversion.convertDouble(Writable writable) |
float |
TypeConversion.convertFloat(Writable writable) |
int |
TypeConversion.convertInt(Writable writable) |
long |
TypeConversion.convertLong(Writable writable) |
String |
TypeConversion.convertString(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ReduceSequenceTransform.map(List<Writable> writables) |
List<List<Writable>> |
ReduceSequenceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
SequenceSplit.split(List<List<Writable>> sequence)
Split a sequence in to multiple time steps
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ReduceSequenceTransform.map(List<Writable> writables) |
List<List<Writable>> |
ReduceSequenceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
SequenceSplit.split(List<List<Writable>> sequence)
Split a sequence in to multiple time steps
|
| Modifier and Type | Method and Description |
|---|---|
protected abstract int |
BaseColumnComparator.compare(Writable w1,
Writable w2) |
protected int |
NumericalColumnComparator.compare(Writable w1,
Writable w2) |
protected int |
StringComparator.compare(Writable w1,
Writable w2) |
| Modifier and Type | Method and Description |
|---|---|
int |
BaseColumnComparator.compare(List<Writable> o1,
List<Writable> o2) |
int |
BaseColumnComparator.compare(List<Writable> o1,
List<Writable> o2) |
| Modifier and Type | Method and Description |
|---|---|
protected abstract List<List<Writable>> |
BaseSequenceExpansionTransform.expandTimeStep(List<Writable> currentStepValues) |
List<Writable> |
BaseSequenceExpansionTransform.map(List<Writable> writables) |
List<List<Writable>> |
BaseSequenceExpansionTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
protected abstract List<List<Writable>> |
BaseSequenceExpansionTransform.expandTimeStep(List<Writable> currentStepValues) |
List<Writable> |
BaseSequenceExpansionTransform.map(List<Writable> writables) |
List<List<Writable>> |
BaseSequenceExpansionTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<List<Writable>> |
SequenceMerge.mergeSequences(List<List<List<Writable>>> multipleSequences) |
| Modifier and Type | Method and Description |
|---|---|
List<List<Writable>> |
SequenceMerge.mergeSequences(List<List<List<Writable>>> multipleSequences) |
| Modifier and Type | Method and Description |
|---|---|
List<List<List<Writable>>> |
SequenceSplitTimeSeparation.split(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
SplitMaxLengthSequence.split(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<List<List<Writable>>> |
SequenceSplitTimeSeparation.split(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
SplitMaxLengthSequence.split(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
SequenceTrimToLengthTransform.map(List<Writable> writables) |
List<Writable> |
SequenceTrimTransform.map(List<Writable> writables) |
List<List<Writable>> |
SequenceTrimToLengthTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceTrimTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
SequenceTrimToLengthTransform.map(List<Writable> writables) |
List<Writable> |
SequenceTrimTransform.map(List<Writable> writables) |
List<List<Writable>> |
SequenceTrimToLengthTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceTrimTransform.mapSequence(List<List<Writable>> sequence) |
| Constructor and Description |
|---|
SequenceTrimToLengthTransform(int maxLength,
SequenceTrimToLengthTransform.Mode mode,
List<Writable> pad) |
| Modifier and Type | Method and Description |
|---|---|
List<List<List<Writable>>> |
OverlappingTimeWindowFunction.applyToSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
TimeWindowFunction.applyToSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
WindowFunction.applyToSequence(List<List<Writable>> sequence)
Apply the windowing function to the given sequence
|
List<Writable> |
ReduceSequenceByWindowTransform.map(List<Writable> writables) |
List<List<Writable>> |
ReduceSequenceByWindowTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<List<List<Writable>>> |
OverlappingTimeWindowFunction.applyToSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
TimeWindowFunction.applyToSequence(List<List<Writable>> sequence) |
List<List<List<Writable>>> |
WindowFunction.applyToSequence(List<List<Writable>> sequence)
Apply the windowing function to the given sequence
|
List<Writable> |
ReduceSequenceByWindowTransform.map(List<Writable> writables) |
List<List<Writable>> |
ReduceSequenceByWindowTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
static Writable |
StringReducer.reduceStringOrCategoricalColumn(StringReduceOp op,
List<Writable> values) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
IStringReducer.reduce(List<List<Writable>> examplesList) |
List<Writable> |
StringReducer.reduce(List<List<Writable>> examplesList) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
IStringReducer.reduce(List<List<Writable>> examplesList) |
List<Writable> |
StringReducer.reduce(List<List<Writable>> examplesList) |
static Writable |
StringReducer.reduceStringOrCategoricalColumn(StringReduceOp op,
List<Writable> values) |
| Modifier and Type | Method and Description |
|---|---|
protected abstract Writable |
BaseColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseColumnTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
BaseColumnsMathOpTransform.map(List<Writable> writables) |
List<Writable> |
BaseColumnTransform.map(List<Writable> writables) |
List<List<Writable>> |
BaseColumnsMathOpTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
BaseTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
protected abstract Writable |
BaseColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseColumnTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
BaseColumnsMathOpTransform.map(List<Writable> writables) |
List<Writable> |
BaseColumnTransform.map(List<Writable> writables) |
List<List<Writable>> |
BaseColumnsMathOpTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
BaseTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
IntegerToCategoricalTransform.map(Writable columnWritable) |
Writable |
StringToCategoricalTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
CategoricalToIntegerTransform.map(List<Writable> writables) |
List<Writable> |
CategoricalToOneHotTransform.map(List<Writable> writables) |
List<Writable> |
FirstDigitTransform.map(List<Writable> writables) |
List<Writable> |
PivotTransform.map(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
IntegerToCategoricalTransform.map(Writable columnWritable) |
Writable |
StringToCategoricalTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
CategoricalToIntegerTransform.map(List<Writable> writables) |
List<Writable> |
CategoricalToOneHotTransform.map(List<Writable> writables) |
List<Writable> |
FirstDigitTransform.map(List<Writable> writables) |
List<Writable> |
PivotTransform.map(List<Writable> writables) |
| Constructor and Description |
|---|
PivotTransform(String keyColumnName,
String valueColumnName,
Writable defaultValue) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
AddConstantColumnTransform.map(List<Writable> writables) |
List<Writable> |
DuplicateColumnsTransform.map(List<Writable> writables) |
List<Writable> |
RemoveAllColumnsExceptForTransform.map(List<Writable> writables) |
List<Writable> |
RemoveColumnsTransform.map(List<Writable> writables) |
List<Writable> |
RenameColumnsTransform.map(List<Writable> writables) |
List<Writable> |
ReorderColumnsTransform.map(List<Writable> writables) |
List<List<Writable>> |
AddConstantColumnTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
DuplicateColumnsTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
RenameColumnsTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ReorderColumnsTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
AddConstantColumnTransform.map(List<Writable> writables) |
List<Writable> |
DuplicateColumnsTransform.map(List<Writable> writables) |
List<Writable> |
RemoveAllColumnsExceptForTransform.map(List<Writable> writables) |
List<Writable> |
RemoveColumnsTransform.map(List<Writable> writables) |
List<Writable> |
RenameColumnsTransform.map(List<Writable> writables) |
List<Writable> |
ReorderColumnsTransform.map(List<Writable> writables) |
List<List<Writable>> |
AddConstantColumnTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
DuplicateColumnsTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
RenameColumnsTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ReorderColumnsTransform.mapSequence(List<List<Writable>> sequence) |
| Constructor and Description |
|---|
AddConstantColumnTransform(String newColumnName,
ColumnType newColumnType,
Writable fixedValue) |
| Modifier and Type | Field and Description |
|---|---|
protected Writable |
ConditionalReplaceValueTransformWithDefault.noVal |
protected Writable |
ConditionalReplaceValueTransformWithDefault.yesVal |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ConditionalCopyValueTransform.map(List<Writable> writables) |
List<Writable> |
ConditionalReplaceValueTransform.map(List<Writable> writables) |
List<Writable> |
ConditionalReplaceValueTransformWithDefault.map(List<Writable> writables) |
List<List<Writable>> |
ConditionalCopyValueTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ConditionalReplaceValueTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ConditionalReplaceValueTransformWithDefault.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ConditionalCopyValueTransform.map(List<Writable> writables) |
List<Writable> |
ConditionalReplaceValueTransform.map(List<Writable> writables) |
List<Writable> |
ConditionalReplaceValueTransformWithDefault.map(List<Writable> writables) |
List<List<Writable>> |
ConditionalCopyValueTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ConditionalReplaceValueTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
ConditionalReplaceValueTransformWithDefault.mapSequence(List<List<Writable>> sequence) |
| Constructor and Description |
|---|
ConditionalReplaceValueTransform(String columnToReplace,
Writable newValue,
Condition condition) |
ConditionalReplaceValueTransformWithDefault(String columnToReplace,
Writable yesVal,
Writable noVal,
Condition condition) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
DoubleColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseDoubleTransform.map(Writable writable) |
Writable |
DoubleMathFunctionTransform.map(Writable w) |
Writable |
DoubleMathOpTransform.map(Writable columnWritable) |
Writable |
Log2Normalizer.map(Writable writable) |
Writable |
MinMaxNormalizer.map(Writable writable) |
Writable |
StandardizeNormalizer.map(Writable writable) |
Writable |
SubtractMeanNormalizer.map(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
DoubleColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseDoubleTransform.map(Writable writable) |
DoubleWritable |
ConvertToDouble.map(Writable writable) |
Writable |
DoubleMathFunctionTransform.map(Writable w) |
Writable |
DoubleMathOpTransform.map(Writable columnWritable) |
Writable |
Log2Normalizer.map(Writable writable) |
Writable |
MinMaxNormalizer.map(Writable writable) |
Writable |
StandardizeNormalizer.map(Writable writable) |
Writable |
SubtractMeanNormalizer.map(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
FloatColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseFloatTransform.map(Writable writable) |
Writable |
FloatMathFunctionTransform.map(Writable w) |
Writable |
FloatMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
FloatColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseFloatTransform.map(Writable writable) |
FloatWritable |
ConvertToFloat.map(Writable writable) |
Writable |
FloatMathFunctionTransform.map(Writable w) |
Writable |
FloatMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
CoordinatesDistanceTransform.doOp(Writable... input) |
Writable |
IPAddressToLocationTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
CoordinatesDistanceTransform.doOp(Writable... input) |
Writable |
IPAddressToLocationTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
IntegerColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseIntegerTransform.map(Writable writable) |
Writable |
IntegerMathOpTransform.map(Writable columnWritable) |
Writable |
ReplaceEmptyIntegerWithValueTransform.map(Writable writable) |
Writable |
ReplaceInvalidWithIntegerTransform.map(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
IntegerToOneHotTransform.map(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
IntegerColumnsMathOpTransform.doOp(Writable... input) |
abstract Writable |
BaseIntegerTransform.map(Writable writable) |
IntWritable |
ConvertToInteger.map(Writable writable) |
Writable |
IntegerMathOpTransform.map(Writable columnWritable) |
Writable |
ReplaceEmptyIntegerWithValueTransform.map(Writable writable) |
Writable |
ReplaceInvalidWithIntegerTransform.map(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
IntegerToOneHotTransform.map(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
LongColumnsMathOpTransform.doOp(Writable... input) |
Writable |
LongMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected Writable |
LongColumnsMathOpTransform.doOp(Writable... input) |
Writable |
LongMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
protected List<List<Writable>> |
TextToCharacterIndexTransform.expandTimeStep(List<Writable> currentStepValues) |
protected List<List<Writable>> |
TextToTermIndexSequenceTransform.expandTimeStep(List<Writable> currentStepValues) |
| Modifier and Type | Method and Description |
|---|---|
protected List<List<Writable>> |
TextToCharacterIndexTransform.expandTimeStep(List<Writable> currentStepValues) |
protected List<List<Writable>> |
TextToTermIndexSequenceTransform.expandTimeStep(List<Writable> currentStepValues) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ParseDoubleTransform.map(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ParseDoubleTransform.map(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
SequenceDifferenceTransform.map(List<Writable> writables) |
List<Writable> |
SequenceMovingWindowReduceTransform.map(List<Writable> writables) |
List<Writable> |
SequenceOffsetTransform.map(List<Writable> writables) |
List<List<Writable>> |
SequenceDifferenceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceMovingWindowReduceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceOffsetTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
SequenceDifferenceTransform.map(List<Writable> writables) |
List<Writable> |
SequenceMovingWindowReduceTransform.map(List<Writable> writables) |
List<Writable> |
SequenceOffsetTransform.map(List<Writable> writables) |
List<List<Writable>> |
SequenceDifferenceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceMovingWindowReduceTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
SequenceOffsetTransform.mapSequence(List<List<Writable>> sequence) |
| Constructor and Description |
|---|
SequenceDifferenceTransform(String columnName,
String newColumnName,
int lookback,
SequenceDifferenceTransform.FirstStepMode firstStepMode,
Writable specifiedValueWritable)
Create a SequenceDifferenceTransform with default lookback of 1, and using FirstStepMode.Default,
where the output column name is specified
|
SequenceMovingWindowReduceTransform(String columnName,
String newColumnName,
int lookback,
ReduceOp op,
SequenceMovingWindowReduceTransform.EdgeCaseHandling edgeCaseHandling,
Writable edgeCaseValue) |
SequenceOffsetTransform(List<String> columnsToOffset,
int offsetAmount,
SequenceOffsetTransform.OperationType operationType,
SequenceOffsetTransform.EdgeHandling edgeHandling,
Writable edgeCaseValue) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
AppendStringColumnTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ConcatenateStringColumns.map(List<Writable> writables) |
List<Writable> |
StringListToCategoricalSetTransform.map(List<Writable> writables) |
List<Writable> |
StringListToCountsNDArrayTransform.map(List<Writable> writables) |
List<List<Writable>> |
ConcatenateStringColumns.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
AppendStringColumnTransform.map(Writable columnWritable) |
abstract Text |
BaseStringTransform.map(Writable writable)
Transform the writable in to a
string
|
Text |
ChangeCaseStringTransform.map(Writable writable) |
Text |
ConvertToString.map(Writable writable)
Transform the writable in to a
string
|
Text |
MapAllStringsExceptListTransform.map(Writable writable) |
Text |
RemoveWhiteSpaceTransform.map(Writable writable) |
Text |
ReplaceEmptyStringTransform.map(Writable writable) |
Text |
ReplaceStringTransform.map(Writable writable) |
Text |
StringMapTransform.map(Writable writable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ConcatenateStringColumns.map(List<Writable> writables) |
List<Writable> |
StringListToCategoricalSetTransform.map(List<Writable> writables) |
List<Writable> |
StringListToCountsNDArrayTransform.map(List<Writable> writables) |
List<List<Writable>> |
ConcatenateStringColumns.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
StringToTimeTransform.map(Writable columnWritable) |
Writable |
TimeMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
DeriveColumnsFromTimeTransform.map(List<Writable> writables) |
List<List<Writable>> |
DeriveColumnsFromTimeTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
StringToTimeTransform.map(Writable columnWritable) |
Writable |
TimeMathOpTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
DeriveColumnsFromTimeTransform.map(List<Writable> writables) |
List<List<Writable>> |
DeriveColumnsFromTimeTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
static void |
HtmlSequencePlotting.createHtmlSequencePlotFile(String title,
Schema schema,
List<List<Writable>> sequence,
File output)
Create a HTML file with plots for the given sequence and write it to a file.
|
static String |
HtmlSequencePlotting.createHtmlSequencePlots(String title,
Schema schema,
List<List<Writable>> sequence)
Create a HTML file with plots for the given sequence.
|
| Modifier and Type | Method and Description |
|---|---|
static List<Writable> |
RecordUtils.toRecord(double[] record) |
static List<Writable> |
RecordUtils.toRecord(float[] record) |
| Modifier and Type | Method and Description |
|---|---|
static List<Writable> |
RecordConverter.toRecord(INDArray array)
Convert an ndarray to a record
|
static List<Writable> |
RecordConverter.toRecord(Schema schema,
List<Object> source)
Convert a collection into a `List
|
static List<List<Writable>> |
RecordConverter.toRecords(DataSet dataSet)
Convert a DataSet to a matrix
|
static List<List<Writable>> |
RecordConverter.toRecords(INDArray matrix)
Convert a set of records in to a matrix
|
| Modifier and Type | Method and Description |
|---|---|
static INDArray |
RecordConverter.toArray(Collection<? extends Writable> record)
Convert a record to an INDArray.
|
static INDArray |
RecordConverter.toArray(DataType dataType,
Collection<? extends Writable> record)
Convert a record to an INDArray.
|
static INDArray |
RecordConverter.toArray(DataType dataType,
Collection<Writable> record,
int size)
Convert a record to an ndarray
|
static INDArray |
RecordConverter.toMatrix(DataType dataType,
List<List<Writable>> records)
Convert a set of records in to a matrix
|
static INDArray |
RecordConverter.toMatrix(List<List<Writable>> records)
Convert a set of records in to a matrix
As per
RecordConverter.toMatrix(DataType, List) but hardcoded to Float datatype |
static INDArray |
RecordConverter.toTensor(List<List<List<Writable>>> records)
Convert a set of records in to a matrix
|
| Modifier and Type | Class and Description |
|---|---|
class |
ArrayWritable
A Writable to basically wrap an array of sorts.
|
class |
BooleanWritable
A WritableComparable for booleans.
|
class |
BytesWritable
Writable type for
a byte array. |
class |
ByteWritable
A WritableComparable for a single byte.
|
class |
DoubleWritable
Writable for Double values.
|
class |
FloatWritable
A WritableComparable for floats.
|
class |
IntWritable
A WritableComparable for ints.
|
class |
LongWritable
A WritableComparable for longs.
|
class |
NDArrayWritable
A Writable that basically wraps an INDArray.
|
class |
NullWritable
NullWritable.
|
class |
Text
This class stores text using standard UTF8 encoding.
|
| Modifier and Type | Method and Description |
|---|---|
static <T> Writable |
UnsafeWritableInjector.inject(T x) |
Writable |
WritableFactory.newWritable(short writableTypeKey)
Create a new writable instance (using reflection) given the specified key
|
Writable |
WritableFactory.readWithType(DataInput dataInput)
Read a Writable From the DataInput, where the Writable was previously written using
WritableFactory.writeWithType(Writable, DataOutput) |
| Modifier and Type | Method and Description |
|---|---|
Class<? extends Writable> |
WritableType.getWritableClass()
Return the class of the implementation corresponding to each WritableType.
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
ByteWritable.fuzzyEquals(Writable o,
double tolerance) |
boolean |
DoubleWritable.fuzzyEquals(Writable o,
double tolerance) |
boolean |
FloatWritable.fuzzyEquals(Writable o,
double tolerance) |
boolean |
IntWritable.fuzzyEquals(Writable o,
double tolerance) |
boolean |
LongWritable.fuzzyEquals(Writable o,
double tolerance) |
void |
WritableFactory.writeWithType(Writable w,
DataOutput dataOutput)
A convenience method for writing a given Writable object to a DataOutput.
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
NDArrayRecordBatch.get(int index) |
Iterator<List<List<Writable>>> |
AbstractTimeSeriesWritableRecordBatch.iterator() |
Iterator<List<Writable>> |
AbstractWritableRecordBatch.iterator() |
ListIterator<List<List<Writable>>> |
AbstractTimeSeriesWritableRecordBatch.listIterator() |
ListIterator<List<Writable>> |
AbstractWritableRecordBatch.listIterator() |
ListIterator<List<List<Writable>>> |
AbstractTimeSeriesWritableRecordBatch.listIterator(int i) |
ListIterator<List<Writable>> |
AbstractWritableRecordBatch.listIterator(int i) |
List<Writable> |
AbstractWritableRecordBatch.RecordBatchListIterator.next() |
List<Writable> |
AbstractWritableRecordBatch.RecordBatchListIterator.previous() |
List<List<Writable>> |
AbstractTimeSeriesWritableRecordBatch.remove(int i) |
List<Writable> |
AbstractWritableRecordBatch.remove(int i) |
List<List<Writable>> |
AbstractTimeSeriesWritableRecordBatch.set(int i,
List<List<Writable>> writable) |
List<Writable> |
AbstractWritableRecordBatch.set(int i,
List<Writable> writable) |
List<List<List<Writable>>> |
AbstractTimeSeriesWritableRecordBatch.subList(int i,
int i1) |
List<List<Writable>> |
AbstractWritableRecordBatch.subList(int i,
int i1) |
| Modifier and Type | Method and Description |
|---|---|
void |
AbstractTimeSeriesWritableRecordBatch.add(int i,
List<List<Writable>> writable) |
void |
AbstractWritableRecordBatch.add(int i,
List<Writable> writable) |
boolean |
AbstractTimeSeriesWritableRecordBatch.add(List<List<Writable>> writable) |
boolean |
AbstractWritableRecordBatch.add(List<Writable> writable) |
void |
AbstractWritableRecordBatch.RecordBatchListIterator.add(List<Writable> writables) |
boolean |
AbstractTimeSeriesWritableRecordBatch.addAll(Collection<? extends List<List<Writable>>> collection) |
boolean |
AbstractWritableRecordBatch.addAll(Collection<? extends List<Writable>> collection) |
boolean |
AbstractTimeSeriesWritableRecordBatch.addAll(int i,
Collection<? extends List<List<Writable>>> collection) |
boolean |
AbstractWritableRecordBatch.addAll(int i,
Collection<? extends List<Writable>> collection) |
List<List<Writable>> |
AbstractTimeSeriesWritableRecordBatch.set(int i,
List<List<Writable>> writable) |
List<Writable> |
AbstractWritableRecordBatch.set(int i,
List<Writable> writable) |
void |
AbstractWritableRecordBatch.RecordBatchListIterator.set(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
static Comparator<Writable> |
Comparators.forType(WritableType type) |
static Comparator<Writable> |
Comparators.forType(WritableType type,
boolean ascending) |
| Modifier and Type | Method and Description |
|---|---|
int |
DoubleWritableComparator.compare(Writable o1,
Writable o2) |
int |
FloatWritableComparator.compare(Writable o1,
Writable o2) |
int |
IntWritableComparator.compare(Writable o1,
Writable o2) |
int |
LongWritableComparator.compare(Writable o1,
Writable o2) |
int |
TextWritableComparator.compare(Writable o1,
Writable o2) |
| Modifier and Type | Method and Description |
|---|---|
static Writable |
ArrowConverter.fromEntry(int item,
org.apache.arrow.vector.FieldVector from,
ColumnType columnType)
Based on an input
ColumnType
get an entry from a FieldVector |
| Modifier and Type | Method and Description |
|---|---|
static List<Writable> |
ArrowConverter.toArrowWritablesSingle(List<org.apache.arrow.vector.FieldVector> fieldVectors,
Schema schema)
Return a singular record based on the converted
writables result.
|
static List<List<List<Writable>>> |
ArrowConverter.toArrowWritablesTimeSeries(List<org.apache.arrow.vector.FieldVector> fieldVectors,
Schema schema,
int timeSeriesLength)
Convert the input field vectors (the input data) and
the given schema to a proper list of writables.
|
| Modifier and Type | Method and Description |
|---|---|
static List<org.apache.arrow.vector.FieldVector> |
ArrowConverter.toArrowColumns(org.apache.arrow.memory.BufferAllocator bufferAllocator,
Schema schema,
List<List<Writable>> dataVecRecord)
Given a buffer allocator and datavec schema,
convert the passed in batch of records
to a set of arrow columns
|
static List<org.apache.arrow.vector.FieldVector> |
ArrowConverter.toArrowColumnsTimeSeries(org.apache.arrow.memory.BufferAllocator bufferAllocator,
Schema schema,
List<List<List<Writable>>> dataVecRecord)
Convert a set of input strings to arrow columns
for a time series.
|
static void |
ArrowConverter.writeRecordBatchTo(org.apache.arrow.memory.BufferAllocator bufferAllocator,
List<List<Writable>> recordBatch,
Schema inputSchema,
OutputStream outputStream)
Write the records to the given output stream
|
static void |
ArrowConverter.writeRecordBatchTo(List<List<Writable>> recordBatch,
Schema inputSchema,
OutputStream outputStream)
Write the records to the given output stream
|
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ArrowWritableRecordBatch.get(int i) |
List<List<Writable>> |
ArrowWritableRecordTimeSeriesBatch.get(int i) |
List<Writable> |
ArrowRecord.getRecord() |
Iterator<List<Writable>> |
ArrowWritableRecordBatch.iterator() |
Iterator<List<List<Writable>>> |
ArrowWritableRecordTimeSeriesBatch.iterator() |
ListIterator<List<Writable>> |
ArrowWritableRecordBatch.listIterator() |
ListIterator<List<List<Writable>>> |
ArrowWritableRecordTimeSeriesBatch.listIterator() |
ListIterator<List<Writable>> |
ArrowWritableRecordBatch.listIterator(int i) |
ListIterator<List<List<Writable>>> |
ArrowWritableRecordTimeSeriesBatch.listIterator(int i) |
List<Writable> |
ArrowRecordReader.next() |
List<List<Writable>> |
ArrowRecordReader.next(int num) |
List<Writable> |
ArrowRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
ArrowWritableRecordBatch.remove(int i) |
List<List<Writable>> |
ArrowWritableRecordTimeSeriesBatch.remove(int i) |
List<List<Writable>> |
ArrowWritableRecordTimeSeriesBatch.set(int i,
List<List<Writable>> writable) |
List<Writable> |
ArrowWritableRecordBatch.set(int i,
List<Writable> writable) |
List<List<Writable>> |
ArrowWritableRecordBatch.subList(int i,
int i1) |
List<List<List<Writable>>> |
ArrowWritableRecordTimeSeriesBatch.subList(int i,
int i1) |
List<List<Writable>> |
ArrowWritableRecordBatch.toArrayList() |
List<List<List<Writable>>> |
ArrowWritableRecordTimeSeriesBatch.toArrayList() |
| Modifier and Type | Method and Description |
|---|---|
void |
ArrowWritableRecordTimeSeriesBatch.add(int i,
List<List<Writable>> writable) |
void |
ArrowWritableRecordBatch.add(int i,
List<Writable> writable) |
boolean |
ArrowWritableRecordTimeSeriesBatch.add(List<List<Writable>> writable) |
boolean |
ArrowWritableRecordBatch.add(List<Writable> writable) |
boolean |
ArrowWritableRecordTimeSeriesBatch.addAll(Collection<? extends List<List<Writable>>> collection) |
boolean |
ArrowWritableRecordBatch.addAll(Collection<? extends List<Writable>> collection) |
boolean |
ArrowWritableRecordTimeSeriesBatch.addAll(int i,
Collection<? extends List<List<Writable>>> collection) |
boolean |
ArrowWritableRecordBatch.addAll(int i,
Collection<? extends List<Writable>> collection) |
List<List<Writable>> |
ArrowWritableRecordTimeSeriesBatch.set(int i,
List<List<Writable>> writable) |
List<Writable> |
ArrowWritableRecordBatch.set(int i,
List<Writable> writable) |
void |
ArrowRecord.setRecord(List<Writable> record) |
PartitionMetaData |
ArrowRecordWriter.write(List<Writable> record) |
PartitionMetaData |
ArrowRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Class and Description |
|---|---|
class |
WavInputFormat
Wave file input format
|
| Modifier and Type | Method and Description |
|---|---|
protected abstract List<Writable> |
BaseAudioRecordReader.loadData(File file,
InputStream inputStream) |
protected List<Writable> |
NativeAudioRecordReader.loadData(File file,
InputStream inputStream) |
protected List<Writable> |
WavFileRecordReader.loadData(File file,
InputStream inputStream) |
List<Writable> |
BaseAudioRecordReader.next() |
List<Writable> |
BaseAudioRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Class and Description |
|---|---|
class |
CodecInputFormat |
| Modifier and Type | Method and Description |
|---|---|
protected abstract List<List<Writable>> |
BaseCodecRecordReader.loadData(File file,
InputStream inputStream) |
protected List<List<Writable>> |
CodecRecordReader.loadData(File file,
InputStream inputStream) |
protected List<List<Writable>> |
NativeCodecRecordReader.loadData(File file,
InputStream inputStream) |
List<Writable> |
BaseCodecRecordReader.next() |
List<Writable> |
BaseCodecRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
BaseCodecRecordReader.sequenceRecord() |
List<List<Writable>> |
BaseCodecRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
MapFileRecordReader.next() |
List<Writable> |
MapFileSequenceRecordReader.next() |
List<List<Writable>> |
MapFileRecordReader.next(int num) |
List<List<Writable>> |
MapFileSequenceRecordReader.next(int num) |
List<Writable> |
MapFileRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<Writable> |
MapFileSequenceRecordReader.record(URI uri,
DataInputStream dataInputStream) |
List<List<Writable>> |
MapFileSequenceRecordReader.sequenceRecord() |
List<List<Writable>> |
MapFileSequenceRecordReader.sequenceRecord(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
protected List<Writable> |
AbstractMapFileWriter.convertTextWritables(List<Writable> record) |
| Modifier and Type | Method and Description |
|---|---|
protected List<Writable> |
AbstractMapFileWriter.convertTextWritables(List<Writable> record) |
protected org.apache.hadoop.io.Writable |
MapFileSequenceRecordWriter.getHadoopWritable(List<List<Writable>> input) |
protected org.apache.hadoop.io.Writable |
MapFileRecordWriter.getHadoopWritable(List<Writable> input) |
PartitionMetaData |
MapFileRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Class and Description |
|---|---|
class |
ImageWritable
Wraps a
Frame to allow serialization within this framework. |
| Modifier and Type | Class and Description |
|---|---|
class |
ImageInputFormat |
| Modifier and Type | Field and Description |
|---|---|
protected List<Writable> |
BaseImageRecordReader.record |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
BaseImageRecordReader.next() |
List<List<Writable>> |
BaseImageRecordReader.next(int num) |
List<Writable> |
BaseImageRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ObjectDetectionRecordReader.next() |
List<List<Writable>> |
ObjectDetectionRecordReader.next(int num) |
List<Writable> |
ObjectDetectionRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ImageTransformProcess.execute(List<Writable> image) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ImageTransformProcess.execute(List<Writable> image) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
JDBCRecordReader.next() |
List<Writable> |
JDBCRecordReader.record(URI uri,
DataInputStream dataInputStream) |
| Modifier and Type | Method and Description |
|---|---|
static Writable |
JdbcWritableConverter.convert(Object columnValue,
int columnType) |
| Modifier and Type | Method and Description |
|---|---|
static List<List<Writable>> |
LocalTransformExecutor.convertStringInput(List<List<String>> stringInput,
Schema schema)
Convert a string time series to
the proper writable set based on the schema.
|
static List<List<List<Writable>>> |
LocalTransformExecutor.convertStringInputTimeSeries(List<List<List<String>>> stringInput,
Schema schema)
Convert a string time series to
the proper writable set based on the schema.
|
static List<List<Writable>> |
LocalTransformExecutor.execute(List<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess returns non-sequence data. |
static List<List<Writable>> |
LocalTransformExecutor.executeJoin(Join join,
List<List<Writable>> left,
List<List<Writable>> right)
Execute a join on the specified data
|
static List<List<Writable>> |
LocalTransformExecutor.executeSequenceToSeparate(List<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, but returns non-sequential data (after reducing or converting sequential data to individual examples) |
static List<List<List<Writable>>> |
LocalTransformExecutor.executeSequenceToSequence(List<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, and also returns sequence data |
static List<List<List<Writable>>> |
LocalTransformExecutor.executeToSequence(List<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess starts with non-sequential data, but returns |
static Map<String,Set<Writable>> |
AnalyzeLocal.getUnique(List<String> columnNames,
Schema schema,
RecordReader data)
Get a list of unique values from the specified columns.
|
static Set<Writable> |
AnalyzeLocal.getUnique(String columnName,
Schema schema,
RecordReader data)
Get a list of unique values from the specified columns.
|
static Map<String,Set<Writable>> |
AnalyzeLocal.getUniqueSequence(List<String> columnNames,
Schema schema,
SequenceRecordReader sequenceData)
Get a list of unique values from the specified columns of a sequence
|
static Set<Writable> |
AnalyzeLocal.getUniqueSequence(String columnName,
Schema schema,
SequenceRecordReader sequenceData)
Get a list of unique values from the specified column of a sequence
|
List<Writable> |
LocalTransformProcessSequenceRecordReader.next() |
List<List<Writable>> |
LocalTransformProcessSequenceRecordReader.next(int num) |
List<List<Writable>> |
LocalTransformProcessSequenceRecordReader.sequenceRecord() |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
SequenceEmptyRecordFunction.apply(List<List<Writable>> v1) |
static List<List<String>> |
LocalTransformExecutor.convertWritableInputToString(List<List<Writable>> stringInput,
Schema schema)
Convert a string time series to
the proper writable set based on the schema.
|
static List<List<List<String>>> |
LocalTransformExecutor.convertWritableInputToStringTimeSeries(List<List<List<Writable>>> stringInput,
Schema schema)
Convert a string time series to
the proper writable set based on the schema.
|
static List<List<Writable>> |
LocalTransformExecutor.execute(List<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess returns non-sequence data. |
static List<List<Writable>> |
LocalTransformExecutor.executeJoin(Join join,
List<List<Writable>> left,
List<List<Writable>> right)
Execute a join on the specified data
|
static List<List<Writable>> |
LocalTransformExecutor.executeJoin(Join join,
List<List<Writable>> left,
List<List<Writable>> right)
Execute a join on the specified data
|
static List<List<Writable>> |
LocalTransformExecutor.executeSequenceToSeparate(List<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, but returns non-sequential data (after reducing or converting sequential data to individual examples) |
static List<List<List<Writable>>> |
LocalTransformExecutor.executeSequenceToSequence(List<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, and also returns sequence data |
static List<List<List<Writable>>> |
LocalTransformExecutor.executeToSequence(List<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess starts with non-sequential data, but returns |
| Modifier and Type | Method and Description |
|---|---|
List<AnalysisCounter> |
AnalysisAddFunction.apply(List<AnalysisCounter> analysisCounters,
List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<HistogramCounter> |
HistogramAddFunction.apply(List<HistogramCounter> histogramCounters,
List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
RecordReaderFunction.apply(Pair<String,InputStream> value) |
List<List<Writable>> |
SequenceRecordReaderFunction.apply(Pair<String,InputStream> value) |
List<Writable> |
LineRecordReaderFunction.apply(String s) |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
EmptyRecordFunction.apply(List<Writable> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
RecordReaderBytesFunction.apply(Pair<Text,BytesWritable> v1) |
List<List<Writable>> |
SequenceRecordReaderBytesFunction.apply(Pair<Text,BytesWritable> v1) |
| Modifier and Type | Method and Description |
|---|---|
Pair<List<Writable>,List<Writable>> |
ExtractKeysFunction.apply(List<Writable> writables) |
Pair<List<Writable>,List<Writable>> |
ExtractKeysFunction.apply(List<Writable> writables) |
List<List<Writable>> |
FilterAndFlattenJoinedValuesAdapter.call(JoinedValue joinedValue) |
List<List<Writable>> |
ExecuteJoinFromCoGroupFlatMapFunctionAdapter.call(Pair<List<Writable>,Pair<List<List<Writable>>,List<List<Writable>>>> t2) |
| Modifier and Type | Method and Description |
|---|---|
Pair<List<Writable>,List<Writable>> |
ExtractKeysFunction.apply(List<Writable> writables) |
List<List<Writable>> |
ExecuteJoinFromCoGroupFlatMapFunctionAdapter.call(Pair<List<Writable>,Pair<List<List<Writable>>,List<List<Writable>>>> t2) |
List<List<Writable>> |
ExecuteJoinFromCoGroupFlatMapFunctionAdapter.call(Pair<List<Writable>,Pair<List<List<Writable>>,List<List<Writable>>>> t2) |
List<List<Writable>> |
ExecuteJoinFromCoGroupFlatMapFunctionAdapter.call(Pair<List<Writable>,Pair<List<List<Writable>>,List<List<Writable>>>> t2) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
NDArrayToWritablesFunction.apply(INDArray arr) |
Pair<Writable,List<Writable>> |
ColumnAsKeyPairFunction.apply(List<Writable> writables) |
Pair<Writable,List<Writable>> |
ColumnAsKeyPairFunction.apply(List<Writable> writables) |
Pair<Writable,Long> |
ColumnToKeyPairTransform.apply(List<Writable> list) |
List<List<Writable>> |
SequenceMergeFunction.apply(Pair<T,Iterable<List<List<Writable>>>> t2) |
List<Writable> |
StringToWritablesFunction.apply(String s) |
| Modifier and Type | Method and Description |
|---|---|
static void |
WritablesToStringFunction.append(List<Writable> c,
StringBuilder sb,
String delim,
String quote) |
String |
SequenceWritablesToStringFunction.apply(List<List<Writable>> c) |
Pair<Writable,List<Writable>> |
ColumnAsKeyPairFunction.apply(List<Writable> writables) |
Pair<Writable,Long> |
ColumnToKeyPairTransform.apply(List<Writable> list) |
INDArray |
WritablesToNDArrayFunction.apply(List<Writable> c) |
String |
WritablesToStringFunction.apply(List<Writable> c) |
List<List<Writable>> |
SequenceMergeFunction.apply(Pair<T,Iterable<List<List<Writable>>>> t2) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
UnzipForCalculateSortedRankFunction.apply(Pair<Pair<Writable,List<Writable>>,Long> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
UnzipForCalculateSortedRankFunction.apply(Pair<Pair<Writable,List<Writable>>,Long> v1) |
List<Writable> |
UnzipForCalculateSortedRankFunction.apply(Pair<Pair<Writable,List<Writable>>,Long> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ReducerFunction.apply(Iterable<List<Writable>> lists) |
Pair<String,List<Writable>> |
MapToPairForReducerFunction.apply(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ReducerFunction.apply(Iterable<List<Writable>> lists) |
Pair<String,List<Writable>> |
MapToPairForReducerFunction.apply(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<List<Writable>> |
LocalGroupToSequenceFunction.apply(List<List<Writable>> lists) |
List<List<Writable>> |
LocalSequenceTransformFunction.apply(List<List<Writable>> v1) |
List<List<Writable>> |
ConvertToSequenceLengthOne.apply(List<Writable> writables) |
Pair<Writable,List<Writable>> |
LocalMapToPairByColumnFunction.apply(List<Writable> writables) |
Pair<Writable,List<Writable>> |
LocalMapToPairByColumnFunction.apply(List<Writable> writables) |
Pair<List<Writable>,List<Writable>> |
LocalMapToPairByMultipleColumnsFunction.apply(List<Writable> writables) |
Pair<List<Writable>,List<Writable>> |
LocalMapToPairByMultipleColumnsFunction.apply(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<List<Writable>> |
LocalGroupToSequenceFunction.apply(List<List<Writable>> lists) |
Boolean |
LocalSequenceFilterFunction.apply(List<List<Writable>> v1) |
List<List<Writable>> |
LocalSequenceTransformFunction.apply(List<List<Writable>> v1) |
List<List<Writable>> |
ConvertToSequenceLengthOne.apply(List<Writable> writables) |
Pair<Writable,List<Writable>> |
LocalMapToPairByColumnFunction.apply(List<Writable> writables) |
Pair<List<Writable>,List<Writable>> |
LocalMapToPairByMultipleColumnsFunction.apply(List<Writable> writables) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
LocalTransformFunction.apply(List<Writable> v1) |
List<List<List<Writable>>> |
SequenceSplitFunctionAdapter.call(List<List<Writable>> collections) |
List<List<Writable>> |
LocalTransformProcessFunctionAdapter.call(List<Writable> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
LocalTransformFunction.apply(List<Writable> v1) |
List<List<List<Writable>>> |
SequenceSplitFunctionAdapter.call(List<List<Writable>> collections) |
List<List<Writable>> |
LocalTransformProcessFunctionAdapter.call(List<Writable> v1) |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
FilterWritablesBySchemaFunction.apply(Writable v1) |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
LocalFilterFunction.apply(List<Writable> v1) |
| Modifier and Type | Class and Description |
|---|---|
class |
TextInputFormat |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TfidfRecordReader.next() |
| Modifier and Type | Method and Description |
|---|---|
Writable |
GazeteerTransform.map(Writable columnWritable) |
Writable |
MultiNlpTransform.map(Writable columnWritable) |
Writable |
TokenizerBagOfWordsTermSequenceIndexTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TokenizerBagOfWordsTermSequenceIndexTransform.map(List<Writable> writables) |
List<List<Writable>> |
GazeteerTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
MultiNlpTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Writable |
GazeteerTransform.map(Writable columnWritable) |
Writable |
MultiNlpTransform.map(Writable columnWritable) |
Writable |
TokenizerBagOfWordsTermSequenceIndexTransform.map(Writable columnWritable) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
TokenizerBagOfWordsTermSequenceIndexTransform.map(List<Writable> writables) |
List<List<Writable>> |
GazeteerTransform.mapSequence(List<List<Writable>> sequence) |
List<List<Writable>> |
MultiNlpTransform.mapSequence(List<List<Writable>> sequence) |
INDArray |
BagOfWordsTransform.transformFrom(List<List<Writable>> tokens)
|
INDArray |
GazeteerTransform.transformFrom(List<List<Writable>> tokens) |
INDArray |
MultiNlpTransform.transformFrom(List<List<Writable>> tokens) |
| Modifier and Type | Method and Description |
|---|---|
protected String |
TextVectorizer.toString(Collection<Writable> record) |
protected Counter<String> |
TextVectorizer.wordFrequenciesForRecord(Collection<Writable> record) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
ExcelRecordReader.next() |
| Modifier and Type | Method and Description |
|---|---|
PartitionMetaData |
ExcelRecordWriter.write(List<Writable> record) |
PartitionMetaData |
ExcelRecordWriter.writeBatch(List<List<Writable>> batch) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
PythonTransform.map(List<Writable> writables) |
List<List<Writable>> |
PythonTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
PythonCondition.condition(List<Writable> list) |
boolean |
PythonCondition.conditionSequence(List<List<Writable>> list) |
List<Writable> |
PythonTransform.map(List<Writable> writables) |
List<List<Writable>> |
PythonTransform.mapSequence(List<List<Writable>> sequence) |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
SequenceEmptyRecordFunction.call(List<List<Writable>> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
LineRecordReaderFunction.call(String s) |
List<Writable> |
RecordReaderFunction.call(scala.Tuple2<String,org.apache.spark.input.PortableDataStream> value) |
List<List<Writable>> |
SequenceRecordReaderFunction.call(scala.Tuple2<String,org.apache.spark.input.PortableDataStream> value) |
| Modifier and Type | Method and Description |
|---|---|
Boolean |
EmptyRecordFunction.call(List<Writable> v1) |
| Modifier and Type | Method and Description |
|---|---|
List<Writable> |
RecordReaderBytesFunction.call(scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable> v1) |
List<List<Writable>> |
SequenceRecordReaderBytesFunction.call(scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable> v1) |
| Modifier and Type | Method and Description |
|---|---|
scala.Tuple2<List<List<Writable>>,List<List<Writable>>> |
PairSequenceRecordReaderBytesFunction.call(scala.Tuple2<org.apache.hadoop.io.Text,BytesPairWritable> v1) |
scala.Tuple2<List<List<Writable>>,List<List<Writable>>> |
PairSequenceRecordReaderBytesFunction.call(scala.Tuple2<org.apache.hadoop.io.Text,BytesPairWritable> v1) |
| Modifier and Type | Method and Description |
|---|---|
static SequenceBatchCSVRecord |
SequenceBatchCSVRecord.fromWritables(List<List<List<Writable>>> input)
Convert a writables time series to a sequence batch
|
static BatchCSVRecord |
BatchCSVRecord.fromWritables(List<List<Writable>> batch)
Create a batch csv record
from a list of writables.
|
| Modifier and Type | Method and Description |
|---|---|
static org.apache.spark.api.java.JavaPairRDD<Long,List<Writable>> |
SparkStorageUtils.restoreMapFile(String path,
org.apache.spark.api.java.JavaSparkContext sc)
Restore a
JavaPairRDD<Long,List<Writable>> previously saved with SparkStorageUtils.saveMapFile(String, JavaRDD)}Note that if the keys are not required, simply use restoreMapFile(...).values() |
static org.apache.spark.api.java.JavaPairRDD<Long,List<List<Writable>>> |
SparkStorageUtils.restoreMapFileSequences(String path,
org.apache.spark.api.java.JavaSparkContext sc)
Restore a
JavaPairRDD<Long,List<List<Writable>>> previously saved with SparkStorageUtils.saveMapFile(String, JavaRDD)}Note that if the keys are not required, simply use restoreMapFileSequences(...).values() |
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkStorageUtils.restoreSequenceFile(String path,
org.apache.spark.api.java.JavaSparkContext sc)
Restore a
JavaRDD<List<Writable>> previously saved with SparkStorageUtils.saveSequenceFile(String, JavaRDD) |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
SparkStorageUtils.restoreSequenceFileSequences(String path,
org.apache.spark.api.java.JavaSparkContext sc)
Restore a
JavaRDD<List<List<Writable>> previously saved with SparkStorageUtils.saveSequenceFileSequences(String, JavaRDD) |
| Modifier and Type | Method and Description |
|---|---|
static void |
SparkStorageUtils.saveMapFile(String path,
org.apache.spark.api.java.JavaRDD<List<Writable>> rdd)
Save a
JavaRDD<List<Writable>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveMapFile(String path,
org.apache.spark.api.java.JavaRDD<List<Writable>> rdd,
org.apache.hadoop.conf.Configuration c,
Integer maxOutputFiles)
Save a
JavaRDD<List<Writable>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveMapFile(String path,
org.apache.spark.api.java.JavaRDD<List<Writable>> rdd,
int interval,
Integer maxOutputFiles)
Save a
JavaRDD<List<Writable>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveMapFileSequences(String path,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> rdd)
Save a
JavaRDD<List<List<Writable>>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveMapFileSequences(String path,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> rdd,
org.apache.hadoop.conf.Configuration c,
Integer maxOutputFiles)
Save a
JavaRDD<List<List<Writable>>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveMapFileSequences(String path,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> rdd,
int interval,
Integer maxOutputFiles)
Save a
JavaRDD<List<List<Writable>>> to a Hadoop MapFile. |
static void |
SparkStorageUtils.saveSequenceFile(String path,
org.apache.spark.api.java.JavaRDD<List<Writable>> rdd)
Save a
JavaRDD<List<Writable>> to a Hadoop SequenceFile. |
static void |
SparkStorageUtils.saveSequenceFile(String path,
org.apache.spark.api.java.JavaRDD<List<Writable>> rdd,
Integer maxOutputFiles)
Save a
JavaRDD<List<Writable>> to a Hadoop SequenceFile. |
static void |
SparkStorageUtils.saveSequenceFileSequences(String path,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> rdd)
Save a
JavaRDD<List<List<Writable>>> to a Hadoop SequenceFile. |
static void |
SparkStorageUtils.saveSequenceFileSequences(String path,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> rdd,
Integer maxOutputFiles)
Save a
JavaRDD<List<List<Writable>>> to a Hadoop SequenceFile. |
| Modifier and Type | Method and Description |
|---|---|
scala.Tuple2<Long,List<Writable>> |
RecordLoadPairFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,RecordWritable> t2) |
scala.Tuple2<Long,List<List<Writable>>> |
SequenceRecordLoadPairFunction.call(scala.Tuple2<org.apache.hadoop.io.LongWritable,SequenceRecordWritable> t2) |
| Modifier and Type | Method and Description |
|---|---|
scala.Tuple2<org.apache.hadoop.io.LongWritable,SequenceRecordWritable> |
SequenceRecordSavePrepPairFunction.call(scala.Tuple2<List<List<Writable>>,Long> t2) |
scala.Tuple2<org.apache.hadoop.io.LongWritable,RecordWritable> |
RecordSavePrepPairFunction.call(scala.Tuple2<List<Writable>,Long> t2) |
| Modifier and Type | Method and Description |
|---|---|
static Writable |
AnalyzeSpark.max(org.apache.spark.api.java.JavaRDD<List<Writable>> allData,
String columnName,
Schema schema)
Get the maximum value for the specified column
|
static Writable |
AnalyzeSpark.min(org.apache.spark.api.java.JavaRDD<List<Writable>> allData,
String columnName,
Schema schema)
Get the minimum value for the specified column
|
| Modifier and Type | Method and Description |
|---|---|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.execute(org.apache.spark.api.java.JavaRDD<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess returns non-sequence data. |
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.executeJoin(Join join,
org.apache.spark.api.java.JavaRDD<List<Writable>> left,
org.apache.spark.api.java.JavaRDD<List<Writable>> right)
Execute a join on the specified data
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.executeSequenceToSeparate(org.apache.spark.api.java.JavaRDD<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, but returns non-sequential data (after reducing or converting sequential data to individual examples) |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
SparkTransformExecutor.executeSequenceToSequence(org.apache.spark.api.java.JavaRDD<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, and also returns sequence data |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
SparkTransformExecutor.executeToSequence(org.apache.spark.api.java.JavaRDD<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess starts with non-sequential data, but returns |
static Map<String,List<Writable>> |
AnalyzeSpark.getUnique(List<String> columnNames,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Get a list of unique values from the specified columns.
|
static List<Writable> |
AnalyzeSpark.getUnique(String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Get a list of unique values from the specified columns.
|
static Map<String,List<Writable>> |
AnalyzeSpark.getUniqueSequence(List<String> columnNames,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Get a list of unique values from the specified columns of a sequence
|
static List<Writable> |
AnalyzeSpark.getUniqueSequence(String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Get a list of unique values from the specified column of a sequence
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Scale all data 0 to 1
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
double min,
double max)
Scale based on min,max
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
double min,
double max,
List<String> skipColumns)
Scale based on min,max
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
List<String> skipColumns)
Scale all data 0 to 1
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data) |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data,
double min,
double max)
Normalize each column of a sequence, based on min/max
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data,
double min,
double max,
List<String> excludeColumns)
Normalize each column of a sequence, based on min/max
|
static List<Writable> |
DataFrames.rowToWritables(Schema schema,
org.apache.spark.sql.Row row)
Convert a given Row to a list of writables, given the specified Schema
|
static List<List<Writable>> |
AnalyzeSpark.sample(int count,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample a set of examples
|
static List<Writable> |
AnalyzeSpark.sampleFromColumn(int count,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample values from a single column
|
static List<Writable> |
AnalyzeSpark.sampleFromColumnSequence(int count,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Randomly sample values from a single column, in all sequences.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumn(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample a set of invalid values from a specified column.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumn(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
boolean ignoreMissing)
Randomly sample a set of invalid values from a specified column.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumnSequence(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Randomly sample a set of invalid values from a specified column, for a sequence data set.
|
static Map<Writable,Long> |
AnalyzeSpark.sampleMostFrequentFromColumn(int nMostFrequent,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Sample the N most frequently occurring values in the specified column
|
static List<List<List<Writable>>> |
AnalyzeSpark.sampleSequence(int count,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Randomly sample a number of sequences from the data
|
static Pair<Schema,org.apache.spark.api.java.JavaRDD<List<Writable>>> |
DataFrames.toRecords(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Create a compatible schema
and rdd for datavec
|
static Pair<Schema,org.apache.spark.api.java.JavaRDD<List<List<Writable>>>> |
DataFrames.toRecordsSequence(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)
Convert the given DataFrame to a sequence
Note: It is assumed here that the DataFrame has been created by DataFrames.toDataFrameSequence(Schema, JavaRDD). |
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.zeromeanUnitVariance(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Normalize by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.zeromeanUnitVariance(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
List<String> skipColumns)
Normalize by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.zeroMeanUnitVarianceSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequence)
Normalize the sequence by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.zeroMeanUnitVarianceSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequence,
List<String> excludeColumns)
Normalize the sequence by zero mean unit variance
|
| Modifier and Type | Method and Description |
|---|---|
static DataAnalysis |
AnalyzeSpark.analyze(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Analyse the specified data - returns a DataAnalysis object with summary information about each column
|
static DataAnalysis |
AnalyzeSpark.analyze(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
int maxHistogramBuckets) |
static DataQualityAnalysis |
AnalyzeSpark.analyzeQuality(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Analyze the data quality of data - provides a report on missing values, values that don't comply with schema, etc
|
static DataQualityAnalysis |
AnalyzeSpark.analyzeQualitySequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Analyze the data quality of sequence data - provides a report on missing values, values that don't comply with schema, etc
|
static SequenceDataAnalysis |
AnalyzeSpark.analyzeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data) |
static SequenceDataAnalysis |
AnalyzeSpark.analyzeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data,
int maxHistogramBuckets) |
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.execute(org.apache.spark.api.java.JavaRDD<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess returns non-sequence data. |
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.executeJoin(Join join,
org.apache.spark.api.java.JavaRDD<List<Writable>> left,
org.apache.spark.api.java.JavaRDD<List<Writable>> right)
Execute a join on the specified data
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.executeJoin(Join join,
org.apache.spark.api.java.JavaRDD<List<Writable>> left,
org.apache.spark.api.java.JavaRDD<List<Writable>> right)
Execute a join on the specified data
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
SparkTransformExecutor.executeSequenceToSeparate(org.apache.spark.api.java.JavaRDD<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, but returns non-sequential data (after reducing or converting sequential data to individual examples) |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
SparkTransformExecutor.executeSequenceToSequence(org.apache.spark.api.java.JavaRDD<List<List<Writable>>> inputSequence,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given sequence input data
Note: this method can only be used if the TransformProcess starts with sequence data, and also returns sequence data |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
SparkTransformExecutor.executeToSequence(org.apache.spark.api.java.JavaRDD<List<Writable>> inputWritables,
TransformProcess transformProcess)
Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess starts with non-sequential data, but returns |
static Map<String,List<Writable>> |
AnalyzeSpark.getUnique(List<String> columnNames,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Get a list of unique values from the specified columns.
|
static List<Writable> |
AnalyzeSpark.getUnique(String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Get a list of unique values from the specified columns.
|
static Map<String,List<Writable>> |
AnalyzeSpark.getUniqueSequence(List<String> columnNames,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Get a list of unique values from the specified columns of a sequence
|
static List<Writable> |
AnalyzeSpark.getUniqueSequence(String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Get a list of unique values from the specified column of a sequence
|
static Writable |
AnalyzeSpark.max(org.apache.spark.api.java.JavaRDD<List<Writable>> allData,
String columnName,
Schema schema)
Get the maximum value for the specified column
|
static Writable |
AnalyzeSpark.min(org.apache.spark.api.java.JavaRDD<List<Writable>> allData,
String columnName,
Schema schema)
Get the minimum value for the specified column
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Scale all data 0 to 1
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
double min,
double max)
Scale based on min,max
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
double min,
double max,
List<String> skipColumns)
Scale based on min,max
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.normalize(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
List<String> skipColumns)
Scale all data 0 to 1
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data) |
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data,
double min,
double max)
Normalize each column of a sequence, based on min/max
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.normalizeSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data,
double min,
double max,
List<String> excludeColumns)
Normalize each column of a sequence, based on min/max
|
static List<List<Writable>> |
AnalyzeSpark.sample(int count,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample a set of examples
|
static List<Writable> |
AnalyzeSpark.sampleFromColumn(int count,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample values from a single column
|
static List<Writable> |
AnalyzeSpark.sampleFromColumnSequence(int count,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequenceData)
Randomly sample values from a single column, in all sequences.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumn(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Randomly sample a set of invalid values from a specified column.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumn(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
boolean ignoreMissing)
Randomly sample a set of invalid values from a specified column.
|
static List<Writable> |
AnalyzeSpark.sampleInvalidFromColumnSequence(int numToSample,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Randomly sample a set of invalid values from a specified column, for a sequence data set.
|
static Map<Writable,Long> |
AnalyzeSpark.sampleMostFrequentFromColumn(int nMostFrequent,
String columnName,
Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Sample the N most frequently occurring values in the specified column
|
static List<List<List<Writable>>> |
AnalyzeSpark.sampleSequence(int count,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Randomly sample a number of sequences from the data
|
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
DataFrames.toDataFrame(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Creates a data frame from a collection of writables
rdd given a schema
|
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> |
DataFrames.toDataFrameSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> data)
Convert the given sequence data set to a DataFrame.
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.zeromeanUnitVariance(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data)
Normalize by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<Writable>> |
Normalization.zeromeanUnitVariance(Schema schema,
org.apache.spark.api.java.JavaRDD<List<Writable>> data,
List<String> skipColumns)
Normalize by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.zeroMeanUnitVarianceSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequence)
Normalize the sequence by zero mean unit variance
|
static org.apache.spark.api.java.JavaRDD<List<List<Writable>>> |
Normalization.zeroMeanUnitVarianceSequence(Schema schema,
org.apache.spark.api.java.JavaRDD<List<List<Writable>>> sequence,
List<String> excludeColumns)
Normalize the sequence by zero mean unit variance
|