Package dev.vortex.spark.read

Class VortexPartitionReaderFactory

java.lang.Object

dev.vortex.spark.read.VortexPartitionReaderFactory

All Implemented Interfaces:

Serializable, org.apache.spark.sql.connector.read.PartitionReaderFactory

public final class VortexPartitionReaderFactory
extends Object
implements org.apache.spark.sql.connector.read.PartitionReaderFactory, Serializable

Factory that produces columnar readers for Vortex files.

The set of paths belongs to each VortexFilePartition — the factory itself is stateless across partitions. For every input partition, VortexPartitionReader opens a single Session, DataSource and Scan spanning that partition's paths and consumes every Vortex partition produced by that scan before returning.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
VortexPartitionReaderFactory(List<String> dataColumnNames, Map<String,String> formatOptions, org.apache.spark.sql.connector.expressions.filter.Predicate[] pushedPredicates)

Method Summary

Modifier and Type	Method	Description
org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch>	createColumnarReader(org.apache.spark.sql.connector.read.InputPartition partition)
org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow>	createReader(org.apache.spark.sql.connector.read.InputPartition partition)
boolean	supportColumnarReads(org.apache.spark.sql.connector.read.InputPartition partition)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VortexPartitionReaderFactory

public VortexPartitionReaderFactory(List<String> dataColumnNames,
 Map<String,String> formatOptions,
 org.apache.spark.sql.connector.expressions.filter.Predicate[] pushedPredicates)

Method Details

createReader

public org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.catalyst.InternalRow> createReader(org.apache.spark.sql.connector.read.InputPartition partition)

Specified by:

createReader in interface org.apache.spark.sql.connector.read.PartitionReaderFactory

createColumnarReader

public org.apache.spark.sql.connector.read.PartitionReader<org.apache.spark.sql.vectorized.ColumnarBatch> createColumnarReader(org.apache.spark.sql.connector.read.InputPartition partition)

Specified by:

createColumnarReader in interface org.apache.spark.sql.connector.read.PartitionReaderFactory

supportColumnarReads

public boolean supportColumnarReads(org.apache.spark.sql.connector.read.InputPartition partition)

Specified by:

supportColumnarReads in interface org.apache.spark.sql.connector.read.PartitionReaderFactory