PathToKeyConverter: Used to match up files based on their file names, for PairSequenceRecordReaderBytesFunction For example, suppose we have files "/features_0.csv" and "/labels_0.csv", map both to same key: "0"
A Hadoop writable class for a pair of byte arrays, plus the original URIs (as Strings) of the files they came from
A function to read files (assuming exactly 2 per input) from a PortableDataStream and combine the contents into a BytesPairWritable
SequenceRecordReaderBytesFunction: Converts two sets of binary data (in the form of a BytesPairWritable) to DataVec format data (
Convert the path to a key by taking the full file name (excluding the file extension and directories)
A PathToKeyConverter that generates a key based on the file name.
Given a Tuple2
Copyright © 2020. All rights reserved.