Package dev.vortex.spark.read

Class VortexColumnarBatch

java.lang.Object
org.apache.spark.sql.vectorized.ColumnarBatch
dev.vortex.spark.read.VortexColumnarBatch
All Implemented Interfaces:
AutoCloseable
public final class VortexColumnarBatch extends org.apache.spark.sql.vectorized.ColumnarBatch
A ColumnarBatch that returns Vortex-managed memory with Arrow format, shared over the C Data Interface.

  • Field Summary

    Fields inherited from class org.apache.spark.sql.vectorized.ColumnarBatch

    columns, numRows, row

  • Constructor Summary

    Constructors
    Constructor
    Description
    VortexColumnarBatch(dev.vortex.api.Array backingArray, org.apache.spark.sql.vectorized.ColumnVector[] columns, int numRows)
    Creates a new VortexColumnarBatch with the specified backing array and column vectors.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Closes this columnar batch and releases all associated resources.
    void
    closeIfFreeable()
    Closes this columnar batch if it is freeable and releases all associated resources.

    Methods inherited from class org.apache.spark.sql.vectorized.ColumnarBatch

    column, getRow, numCols, numRows, rowIterator, setNumRows

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • VortexColumnarBatch

      public VortexColumnarBatch(dev.vortex.api.Array backingArray, org.apache.spark.sql.vectorized.ColumnVector[] columns, int numRows)
      Creates a new VortexColumnarBatch with the specified backing array and column vectors.

      The backing array holds the native memory that contains the actual data, while the column vectors provide the Spark API for accessing that data.

      Parameters:
      backingArray - the Vortex Array that holds the native memory
      columns - the array of ColumnVector objects for data access
      numRows - the number of rows in this batch

  • Method Details

    • close

      public void close()
      Closes this columnar batch and releases all associated resources.

      This method frees the native memory held by the backing Vortex array and then delegates to the parent class to close the column vectors.

      Specified by:
      close in interface AutoCloseable
      Overrides:
      close in class org.apache.spark.sql.vectorized.ColumnarBatch

    • closeIfFreeable

      public void closeIfFreeable()
      Closes this columnar batch if it is freeable and releases all associated resources.

      This method frees the native memory held by the backing Vortex array and then delegates to the parent class to close the column vectors if freeable.

      Overrides:
      closeIfFreeable in class org.apache.spark.sql.vectorized.ColumnarBatch