duckdb::PartitionedColumnData Class Reference

#include <duckdb.hpp>

Inheritance diagram for duckdb::PartitionedColumnData:

Collaboration diagram for duckdb::PartitionedColumnData:

Public Member Functions
unique_ptr< PartitionedColumnData >	CreateShared ()
void	InitializeAppendState (PartitionedColumnDataAppendState &state) const
	Initializes a local state for parallel partitioning that can be merged into this PartitionedColumnData.
void	Append (PartitionedColumnDataAppendState &state, DataChunk &input)
	Appends a DataChunk to this PartitionedColumnData.
void	FlushAppendState (PartitionedColumnDataAppendState &state)
	Flushes any remaining data in the append state into this PartitionedColumnData.
void	Combine (PartitionedColumnData &other)
	Combine another PartitionedColumnData into this PartitionedColumnData.
vector< unique_ptr< ColumnDataCollection > > &	GetPartitions ()
	Get the partitions in this PartitionedColumnData.
template<class TARGET >
TARGET &	Cast ()
template<class TARGET >
const TARGET &	Cast () const

Protected Member Functions
virtual idx_t	BufferSize () const
	Size of the buffers in the append states for this type of partitioning (default 128)
virtual void	InitializeAppendStateInternal (PartitionedColumnDataAppendState &state) const
	Initialize a PartitionedColumnDataAppendState for this type of partitioning (optional)
virtual void	ComputePartitionIndices (PartitionedColumnDataAppendState &state, DataChunk &input)
virtual idx_t	MaxPartitionIndex () const
	Maximum partition index (optional)
	PartitionedColumnData (PartitionedColumnDataType type, ClientContext &context, vector< LogicalType > types)
	PartitionedColumnData can only be instantiated by derived classes.
	PartitionedColumnData (const PartitionedColumnData &other)
idx_t	HalfBufferSize () const
	If the buffer is half full, we append to the partition.
void	CreateAllocator ()
	Create a new shared allocator.
bool	UseFixedSizeMap () const
	Whether to use fixed size map or regular map.
void	BuildPartitionSel (PartitionedColumnDataAppendState &state, const idx_t append_count) const
template<bool fixed>
void	AppendInternal (PartitionedColumnDataAppendState &state, DataChunk &input)
	Appends a DataChunk to this PartitionedColumnData.
unique_ptr< ColumnDataCollection >	CreatePartitionCollection (idx_t partition_index) const
	Create a collection for a specific a partition.
unique_ptr< DataChunk >	CreatePartitionBuffer () const
	Create a DataChunk used for buffering appends to the partition.

Static Protected Member Functions
template<bool fixed>
static void	BuildPartitionSel (PartitionedColumnDataAppendState &state, const idx_t append_count)

Protected Attributes
PartitionedColumnDataType	type
ClientContext &	context
vector< LogicalType >	types
mutex	lock
shared_ptr< PartitionColumnDataAllocators >	allocators
vector< unique_ptr< ColumnDataCollection > >	partitions

Detailed Description

PartitionedColumnData represents partitioned columnar data, which serves as an interface for different types of partitioning, e.g., radix, hive

Member Function Documentation

BufferSize()

virtual idx_t duckdb::PartitionedColumnData::BufferSize ( ) const

inlineprotectedvirtual

Size of the buffers in the append states for this type of partitioning (default 128)

                                     {
        return MinValue<idx_t>(128, STANDARD_VECTOR_SIZE);
    }

Here is the caller graph for this function:

InitializeAppendStateInternal()

virtual void duckdb::PartitionedColumnData::InitializeAppendStateInternal ( PartitionedColumnDataAppendState &  state ) const

inlineprotectedvirtual

Initialize a PartitionedColumnDataAppendState for this type of partitioning (optional)

                                                                                              {
    }

ComputePartitionIndices()

virtual void duckdb::PartitionedColumnData::ComputePartitionIndices ( PartitionedColumnDataAppendState &  state, DataChunk &  input  )

inlineprotectedvirtual

Compute the partition indices for this type of partitioning for the input DataChunk and store them in the partition_data of the local state. If this type creates partitions on the fly (for, e.g., hive), this function is also in charge of creating new partitions and mapping the input data to a partition index

Reimplemented in duckdb::HivePartitionedColumnData.

                                                                                                    {
        throw NotImplementedException("ComputePartitionIndices for this type of PartitionedColumnData");
    }

MaxPartitionIndex()

virtual idx_t duckdb::PartitionedColumnData::MaxPartitionIndex ( ) const

inlineprotectedvirtual

Maximum partition index (optional)

                                            {
        return DConstants::INVALID_INDEX;
    }

HalfBufferSize()

idx_t duckdb::PartitionedColumnData::HalfBufferSize ( ) const

inlineprotected

If the buffer is half full, we append to the partition.

                                        {
        D_ASSERT(IsPowerOfTwo(BufferSize()));
        return BufferSize() / 2;
    }

Here is the call graph for this function:

BuildPartitionSel()

void duckdb::PartitionedColumnData::BuildPartitionSel ( PartitionedColumnDataAppendState &  state, const idx_t  append_count  ) const

protected

Builds a selection vector in the Append state for the partitions

• returns true if everything belongs to the same partition - stores partition index in single_partition_idx

CreatePartitionCollection()

unique_ptr< ColumnDataCollection > duckdb::PartitionedColumnData::CreatePartitionCollection ( idx_t  partition_index ) const

inlineprotected

Create a collection for a specific a partition.

                                                                                            {
        return make_uniq<ColumnDataCollection>(allocators->allocators[partition_index], types);
    }

Here is the caller graph for this function:

Cast() [1/2]

template<class TARGET >

TARGET & duckdb::PartitionedColumnData::Cast ( )

inline

                   {
        DynamicCastCheck<TARGET>(this);
        return reinterpret_cast<TARGET &>(*this);
    }

Cast() [2/2]

template<class TARGET >

const TARGET & duckdb::PartitionedColumnData::Cast ( ) const

inline

                               {
        DynamicCastCheck<TARGET>(this);
        return reinterpret_cast<const TARGET &>(*this);
    }

---

The documentation for this class was generated from the following file:

• external/duckdb/duckdb.hpp