Package dev.vortex.spark.read

Class VortexBatchExec

java.lang.Object

dev.vortex.spark.read.VortexBatchExec

All Implemented Interfaces:

org.apache.spark.sql.connector.read.Batch

public final class VortexBatchExec
extends Object
implements org.apache.spark.sql.connector.read.Batch

Execution source for batch scans of Vortex file tables.

Constructor Summary

Constructors

Constructor	Description
VortexBatchExec(List<String> paths, List<org.apache.spark.sql.connector.catalog.Column> columns, Map<String,String> formatOptions, org.apache.spark.sql.connector.expressions.filter.Predicate[] pushedPredicates)	Creates a new VortexBatchExec for scanning the specified Vortex files.

Method Summary

Modifier and Type	Method	Description
org.apache.spark.sql.connector.read.PartitionReaderFactory	createReaderFactory()
org.apache.spark.sql.connector.read.InputPartition[]	planInputPartitions()	Plans the input partitions for this batch scan.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VortexBatchExec

public VortexBatchExec(List<String> paths,
 List<org.apache.spark.sql.connector.catalog.Column> columns,
 Map<String,String> formatOptions,
 org.apache.spark.sql.connector.expressions.filter.Predicate[] pushedPredicates)

Creates a new VortexBatchExec for scanning the specified Vortex files.

Parameters:

paths - the list of file paths to scan

columns - the list of columns to read from the files

pushedPredicates - predicates pushed down by Spark; converted to a single Vortex filter expression at read time

Method Details

planInputPartitions

public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()

Plans the input partitions for this batch scan.

Directory-like entries are expanded to concrete .vortex files. Each resolved file becomes its own VortexFilePartition; the partition carries the paths the reader should open, the requested schema, and any Hive-style partition values parsed out of the path.

Specified by:

planInputPartitions in interface org.apache.spark.sql.connector.read.Batch

createReaderFactory

public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()

Specified by:

createReaderFactory in interface org.apache.spark.sql.connector.read.Batch