Package dev.vortex.spark

Class VortexFilePartition

java.lang.Object
dev.vortex.spark.VortexFilePartition
All Implemented Interfaces:
Serializable, org.apache.spark.sql.connector.read.InputPartition
public final class VortexFilePartition extends Object implements org.apache.spark.sql.connector.read.InputPartition, Serializable
An InputPartition for reading a whole Vortex file.

This class represents a partition that corresponds to a single Vortex file. It contains the file path and the columns to be read from that file. Each partition can be processed independently by Spark executors.

See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    VortexFilePartition(String path, com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column> columns)
    Creates a new Vortex file partition.

  • Method Summary

    Modifier and Type
    Method
    Description
    com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column>
    getColumns()
    Returns the list of columns to be read from this partition.
    String
    getPath()
    Returns the file system path to the Vortex file for this partition.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.read.InputPartition

    preferredLocations

  • Constructor Details

    • VortexFilePartition

      public VortexFilePartition(String path, com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column> columns)
      Creates a new Vortex file partition.
      Parameters:
      path - the file system path to the Vortex file
      columns - the list of columns to read from the file

  • Method Details

    • getPath

      public String getPath()
      Returns the file system path to the Vortex file for this partition.
      Returns:
      the file path

    • getColumns

      public com.google.common.collect.ImmutableList<org.apache.spark.sql.connector.catalog.Column> getColumns()
      Returns the list of columns to be read from this partition.
      Returns:
      the immutable list of columns