Package dev.vortex.spark.write

Class VortexWriteBuilder

java.lang.Object
dev.vortex.spark.write.VortexWriteBuilder
All Implemented Interfaces:
org.apache.spark.sql.connector.write.SupportsTruncate, org.apache.spark.sql.connector.write.WriteBuilder
public final class VortexWriteBuilder extends Object implements org.apache.spark.sql.connector.write.WriteBuilder, org.apache.spark.sql.connector.write.SupportsTruncate
Builder for configuring Vortex write operations.

This class is responsible for creating BatchWrite instances that execute the actual write operations to create Vortex files from Spark DataFrames.

  • Constructor Summary

    Constructors
    Constructor
    Description
    VortexWriteBuilder(String paths, org.apache.spark.sql.connector.write.LogicalWriteInfo writeInfo, Map<String,String> options)
    Creates a new VortexWriteBuilder.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.connector.write.Write
    build()
    Builds a Write for executing the write operation.
    org.apache.spark.sql.connector.write.WriteBuilder
    truncate()
    Configures the write operation to truncate existing data.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.write.WriteBuilder

    buildForBatch, buildForStreaming

  • Constructor Details

    • VortexWriteBuilder

      public VortexWriteBuilder(String paths, org.apache.spark.sql.connector.write.LogicalWriteInfo writeInfo, Map<String,String> options)
      Creates a new VortexWriteBuilder.
      Parameters:
      paths - root path for write
      writeInfo - logical information about the write operation
      options - additional write options

  • Method Details

    • build

      public org.apache.spark.sql.connector.write.Write build()
      Builds a Write for executing the write operation.
      Specified by:
      build in interface org.apache.spark.sql.connector.write.WriteBuilder
      Returns:
      a new VortexBatchWrite configured with this builder's settings

    • truncate

      public org.apache.spark.sql.connector.write.WriteBuilder truncate()
      Configures the write operation to truncate existing data.

      When truncate is enabled, existing Vortex files at the output path will be removed before writing new data.

      Specified by:
      truncate in interface org.apache.spark.sql.connector.write.SupportsTruncate
      Returns:
      this builder for method chaining