Package dev.vortex.spark.read

Class VortexBatchExec

java.lang.Object
dev.vortex.spark.read.VortexBatchExec
All Implemented Interfaces:
org.apache.spark.sql.connector.read.Batch
public final class VortexBatchExec extends Object implements org.apache.spark.sql.connector.read.Batch
Execution source for batch scans of Vortex file tables.

  • Constructor Summary

    Constructors
    Constructor
    Description
    VortexBatchExec(com.google.common.collect.ImmutableList<String> paths, com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column> columns, Map<String,String> formatOptions)
    Creates a new VortexBatchExec for scanning the specified Vortex files.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.connector.read.PartitionReaderFactory
    createReaderFactory()
    Creates a factory for creating partition readers.
    org.apache.spark.sql.connector.read.InputPartition[]
    planInputPartitions()
    Plans the input partitions for this batch scan.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • VortexBatchExec

      public VortexBatchExec(com.google.common.collect.ImmutableList<String> paths, com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column> columns, Map<String,String> formatOptions)
      Creates a new VortexBatchExec for scanning the specified Vortex files.
      Parameters:
      paths - the list of file paths to scan
      columns - the list of columns to read from the files

  • Method Details

    • planInputPartitions

      public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()
      Plans the input partitions for this batch scan.

      Creates one partition per file path, where each partition is responsible for reading data from a single Vortex file.

      Specified by:
      planInputPartitions in interface org.apache.spark.sql.connector.read.Batch
      Returns:
      an array of InputPartition objects, one per file path

    • createReaderFactory

      public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()
      Creates a factory for creating partition readers.

      Returns a singleton ReaderFactory instance that can create readers capable of reading Vortex file partitions.

      Specified by:
      createReaderFactory in interface org.apache.spark.sql.connector.read.Batch
      Returns:
      the PartitionReaderFactory for Vortex files