Index (vortex-spark fa9d9ea API)

A B C D F G H I M N O P R S T V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form

A

abort() - Method in class dev.vortex.spark.write.VortexDataWriter: Aborts the write operation and cleans up resources.
abort(WriterCommitMessage[]) - Method in class dev.vortex.spark.write.VortexBatchWrite: Aborts the write job due to failures.
addAllColumns(Iterable<Column>) - Method in class dev.vortex.spark.read.VortexScanBuilder: Adds multiple columns to read.
addAllPaths(Iterable<String>) - Method in class dev.vortex.spark.read.VortexScanBuilder: Adds multiple file paths to scan.
addColumn(Column) - Method in class dev.vortex.spark.read.VortexScanBuilder: Adds a column to read.
addPath(String) - Method in class dev.vortex.spark.read.VortexScanBuilder: Adds a file path to scan.
ArrowUtils - Class in dev.vortex.spark: Utility class for converting Arrow types to Spark SQL data types.

B

build() - Method in class dev.vortex.spark.read.VortexScanBuilder: Builds a VortexScan with the configured paths and columns.
build() - Method in class dev.vortex.spark.write.VortexWriteBuilder: Builds a Write for executing the write operation.

C

capabilities() - Method in class dev.vortex.spark.VortexTable: Returns the capabilities supported by this table.
close() - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Closes this column vector and releases any associated resources.
close() - Method in class dev.vortex.spark.read.VortexColumnarBatch: Closes this columnar batch and releases all associated resources.
close() - Method in class dev.vortex.spark.read.VortexColumnarBatchIterator: Closes this iterator and releases all associated resources.
close() - Method in class dev.vortex.spark.write.VortexDataWriter: Closes the writer and releases resources.
closeIfFreeable() - Method in class dev.vortex.spark.read.VortexColumnarBatch: Closes this columnar batch if it is freeable and releases all associated resources.
columnarSupportMode() - Method in class dev.vortex.spark.read.VortexScan: Returns the columnar support mode for this scan.
commit() - Method in class dev.vortex.spark.write.VortexDataWriter: Commits the write operation and returns a commit message.
commit(WriterCommitMessage[]) - Method in class dev.vortex.spark.write.VortexBatchWrite: Commits the entire write job after all tasks complete successfully.
convert(StructType) - Static method in class dev.vortex.spark.write.SparkToArrowSchema: Converts a Spark StructType schema to an Arrow Schema.
createBatchWriterFactory(PhysicalWriteInfo) - Method in class dev.vortex.spark.write.VortexBatchWrite: Creates a DataWriterFactory for producing data writers on executors.
createReaderFactory() - Method in class dev.vortex.spark.read.VortexBatchExec: Creates a factory for creating partition readers.
createWriter(int, long) - Method in class dev.vortex.spark.write.VortexDataWriterFactory: Creates a new data writer for a specific partition and task.

D

description() - Method in class dev.vortex.spark.read.VortexScan: Logging-friendly readable description of the scan source.
dev.vortex.spark - package dev.vortex.spark
dev.vortex.spark.read - package dev.vortex.spark.read
dev.vortex.spark.write - package dev.vortex.spark.write

F

fromArrowField(Field) - Static method in class dev.vortex.spark.ArrowUtils: Converts an Arrow Field to a Spark SQL DataType.
fromArrowType(ArrowType) - Static method in class dev.vortex.spark.ArrowUtils: Converts an Arrow type to a Spark SQL DataType.

G

getArray(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the array value at the specified row.
getBinary(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the binary data (byte array) at the specified row.
getBoolean(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the boolean value at the specified row.
getByte(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the byte value at the specified row.
getBytesWritten() - Method in class dev.vortex.spark.write.VortexWriterCommitMessage: Gets the number of bytes written to the file.
getChild(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the child column at the specified ordinal.
getColumns() - Method in class dev.vortex.spark.VortexFilePartition: Returns the list of columns to be read from this partition.
getDecimal(int, int, int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the decimal value at the specified row with the given precision and scale.
getDouble(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the double value at the specified row.
getFilePath() - Method in class dev.vortex.spark.write.VortexWriterCommitMessage: Gets the path to the written Vortex file.
getFloat(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the float value at the specified row.
getInt(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the int value at the specified row.
getLong(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the long value at the specified row.
getMap(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the map value at the specified row.
getPath() - Method in class dev.vortex.spark.VortexFilePartition: Returns the file system path to the Vortex file for this partition.
getRecordCount() - Method in class dev.vortex.spark.write.VortexWriterCommitMessage: Gets the number of records written to the file.
getShort(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the short value at the specified row.
getTable(StructType, Transform[], Map<String, String>) - Method in class dev.vortex.spark.VortexDataSourceV2: Creates a Vortex table instance with the given schema and properties.
getUTF8String(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the UTF8String value at the specified row.
getValueVector() - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the underlying Apache Arrow ValueVector wrapped by this column vector.

H

hasNext() - Method in class dev.vortex.spark.read.VortexColumnarBatchIterator: Returns whether there are more columnar batches available.
hasNull() - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns whether this column contains any null values.

I

inferSchema(CaseInsensitiveStringMap) - Method in class dev.vortex.spark.VortexDataSourceV2: Infers the schema of the Vortex files specified in the options.
isNullAt(int) - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns whether the value at the specified row is null.

M

MAX_BUFFER_BYTES - Static variable in class dev.vortex.spark.read.VortexColumnarBatchIterator: Maximum buffer size in bytes for prefetching arrays.

N

name() - Method in class dev.vortex.spark.VortexTable: Returns the name of this table.
newScanBuilder(CaseInsensitiveStringMap) - Method in class dev.vortex.spark.VortexTable: Creates a new ScanBuilder for this table.
newWriteBuilder(LogicalWriteInfo) - Method in class dev.vortex.spark.VortexTable: Creates a new WriteBuilder for writing data to this table.
next() - Method in class dev.vortex.spark.read.VortexColumnarBatchIterator: Returns the next columnar batch from the iterator.
numNulls() - Method in class dev.vortex.spark.read.VortexArrowColumnVector: Returns the total number of null values in this column.

O

onDataWriterCommit(WriterCommitMessage) - Method in class dev.vortex.spark.write.VortexBatchWrite: Called when a single data writer task completes successfully.

P

planInputPartitions() - Method in class dev.vortex.spark.read.VortexBatchExec: Plans the input partitions for this batch scan.
pruneColumns(StructType) - Method in class dev.vortex.spark.read.VortexScanBuilder: Prunes the columns to only include those specified in the required schema.

R

readSchema() - Method in class dev.vortex.spark.read.VortexScan: Returns the schema for the data that will be read by this scan.

S

schema() - Method in class dev.vortex.spark.VortexTable: Returns the schema of this table.
shortName() - Method in class dev.vortex.spark.VortexDataSourceV2: Returns the short name identifier for this data source.
SparkToArrowSchema - Class in dev.vortex.spark.write: Utility class for converting Spark SQL schemas to Arrow schemas.
SparkTypes - Class in dev.vortex.spark: Helpers for converting between Spark and Vortex type systems.
supportsExternalMetadata() - Method in class dev.vortex.spark.VortexDataSourceV2: Indicates whether this data source supports external metadata (schemas).

T

toBatch() - Method in class dev.vortex.spark.read.VortexScan: Converts this scan to a Batch for execution.
toBatch() - Method in class dev.vortex.spark.write.VortexBatchWrite: Returns this object as a BatchWrite.
toColumns(DType) - Static method in class dev.vortex.spark.SparkTypes: Convert a STRUCT Vortex type to a Spark Column.
toDataType(DType) - Static method in class dev.vortex.spark.SparkTypes: Convert a STRUCT Vortex type to a Spark DataType.
toDType(StructType) - Static method in class dev.vortex.spark.SparkTypes
truncate() - Method in class dev.vortex.spark.write.VortexWriteBuilder: Configures the write operation to truncate existing data.

V

VortexArrowColumnVector - Class in dev.vortex.spark.read: Spark ColumnVector implementation that wraps Apache Arrow vectors from Vortex data.
VortexArrowColumnVector(ValueVector) - Constructor for class dev.vortex.spark.read.VortexArrowColumnVector: Creates a new VortexArrowColumnVector wrapping the specified Arrow ValueVector.
VortexBatchExec - Class in dev.vortex.spark.read: Execution source for batch scans of Vortex file tables.
VortexBatchExec(ImmutableList<String>, ImmutableList<Column>, Map<String, String>) - Constructor for class dev.vortex.spark.read.VortexBatchExec: Creates a new VortexBatchExec for scanning the specified Vortex files.
VortexBatchWrite - Class in dev.vortex.spark.write: Manages the batch write operation for creating Vortex files.
VortexBatchWrite(String, StructType, Map<String, String>, boolean) - Constructor for class dev.vortex.spark.write.VortexBatchWrite: Creates a new VortexBatchWrite.
VortexColumnarBatch - Class in dev.vortex.spark.read: A ColumnarBatch that returns Vortex-managed memory with Arrow format, shared over the C Data Interface.
VortexColumnarBatch(Array, ColumnVector[], int) - Constructor for class dev.vortex.spark.read.VortexColumnarBatch: Creates a new VortexColumnarBatch with the specified backing array and column vectors.
VortexColumnarBatchIterator - Class in dev.vortex.spark.read: Iterator that converts Vortex Arrays into Spark ColumnarBatch objects.
VortexColumnarBatchIterator(ArrayIterator) - Constructor for class dev.vortex.spark.read.VortexColumnarBatchIterator: Creates a new VortexColumnarBatchIterator that wraps the given ArrayIterator.
VortexDataSourceV2 - Class in dev.vortex.spark: Spark V2 data source for reading and writing Vortex files.
VortexDataSourceV2() - Constructor for class dev.vortex.spark.VortexDataSourceV2: Creates a new instance of the Vortex data source.
VortexDataWriter - Class in dev.vortex.spark.write: Writes Spark InternalRow data to a Vortex file.
VortexDataWriter(String, StructType, CaseInsensitiveStringMap) - Constructor for class dev.vortex.spark.write.VortexDataWriter: Creates a new VortexDataWriter.
VortexDataWriterFactory - Class in dev.vortex.spark.write: Factory for creating VortexDataWriter instances on Spark executors.
VortexDataWriterFactory(String, StructType, Map<String, String>) - Constructor for class dev.vortex.spark.write.VortexDataWriterFactory: Creates a new VortexDataWriterFactory.
VortexFilePartition - Class in dev.vortex.spark: An InputPartition for reading a whole Vortex file.
VortexFilePartition(String, ImmutableList<Column>) - Constructor for class dev.vortex.spark.VortexFilePartition: Creates a new Vortex file partition.
VortexScan - Class in dev.vortex.spark.read: Spark V2 Scan over a table of Vortex files.
VortexScan(ImmutableList<String>, ImmutableList<Column>, Map<String, String>) - Constructor for class dev.vortex.spark.read.VortexScan: Creates a new VortexScan for the specified file paths and columns.
VortexScanBuilder - Class in dev.vortex.spark.read: Spark V2 ScanBuilder for table scans over Vortex files.
VortexScanBuilder(Map<String, String>) - Constructor for class dev.vortex.spark.read.VortexScanBuilder: Creates a new VortexScanBuilder with empty paths and columns.
VortexTable - Class in dev.vortex.spark: Spark V2 Table of Vortex files that supports both reading and writing.
VortexTable(ImmutableList<String>, StructType, Map<String, String>) - Constructor for class dev.vortex.spark.VortexTable: Creates a new VortexTable with read/write support.
VortexWriteBuilder - Class in dev.vortex.spark.write: Builder for configuring Vortex write operations.
VortexWriteBuilder(String, LogicalWriteInfo, Map<String, String>) - Constructor for class dev.vortex.spark.write.VortexWriteBuilder: Creates a new VortexWriteBuilder.
VortexWriterCommitMessage - Class in dev.vortex.spark.write: Commit message containing information about a successfully written Vortex file.
VortexWriterCommitMessage(String, long, long) - Constructor for class dev.vortex.spark.write.VortexWriterCommitMessage: Creates a new commit message for a written Vortex file.

W

write(InternalRow) - Method in class dev.vortex.spark.write.VortexDataWriter: Writes a single row to the Vortex file.

A B C D F G H I M N O P R S T V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form