duckdb::FixedSizeAllocator Class Reference

Collaboration diagram for duckdb::FixedSizeAllocator:

Public Member Functions
	FixedSizeAllocator (const idx_t segment_size, BlockManager &block_manager, MemoryTag memory_tag=MemoryTag::ART_INDEX)
	Construct a new fixed-size allocator.
IndexPointer	New ()
	Get a new IndexPointer to a segment, might cause a new buffer allocation.
void	Free (const IndexPointer ptr)
	Free the segment of the IndexPointer.
SegmentHandle	GetHandle (const IndexPointer ptr)
	Get a segment handle.
template<class T >
unsafe_optional_ptr< T >	Get (const IndexPointer ptr, const bool dirty=true)
data_ptr_t	Get (const IndexPointer ptr, const bool dirty=true)
bool	LoadedFromStorage (const IndexPointer ptr) const
	Has been loaded into a buffer-managed buffer from a persistent, file-backed block.
template<class T >
unsafe_optional_ptr< T >	GetIfLoaded (const IndexPointer ptr)
data_ptr_t	GetIfLoaded (const IndexPointer ptr)
void	Reset ()
	Resets the allocator, e.g., during 'DELETE FROM table'.
idx_t	GetInMemorySize () const
	Returns the in-memory size in bytes.
idx_t	GetSegmentSize () const
	Returns the segment size.
idx_t	GetSegmentCount () const
	Returns the total segment count.
idx_t	GetUpperBoundBufferId () const
	Returns the upper bound of the available buffer IDs, i.e., upper_bound > max_buffer_id.
void	Merge (FixedSizeAllocator &other)
	Merge another FixedSizeAllocator into this allocator. Both must have the same segment size.
bool	InitializeVacuum ()
	Initialize a vacuum operation, and return true, if the allocator needs a vacuum.
void	FinalizeVacuum ()
	Finalize a vacuum operation by freeing all vacuumed buffers.
bool	NeedsVacuum (const IndexPointer ptr) const
	Returns true, if an IndexPointer qualifies for a vacuum operation, and false otherwise.
IndexPointer	VacuumPointer (const IndexPointer ptr)
	Vacuums an IndexPointer.
FixedSizeAllocatorInfo	GetInfo () const
	Returns all FixedSizeAllocator information for serialization.
void	SerializeBuffers (PartialBlockManager &partial_block_manager)
	Serializes all in-memory buffers.
vector< IndexBufferInfo >	InitSerializationToWAL ()
	Sets the allocation sizes and returns data to serialize each buffer.
void	Init (const FixedSizeAllocatorInfo &info)
	Initialize a fixed-size allocator from allocator storage information.
void	Deserialize (MetadataManager &metadata_manager, const BlockPointer &block_pointer)
	Deserializes all metadata of older storage files.
bool	Empty ()
	Returns true, if the allocator does not contain any segments.
void	RemoveEmptyBuffers ()
	Removes empty buffers.
void	VerifyBuffers ()
	Verifies that the number of empty buffers does not exceed the empty buffer threshold.

Public Attributes
BlockManager &	block_manager
	Block manager of the database instance.
BufferManager &	buffer_manager
	Buffer manager of the database instance.

Static Public Attributes
static constexpr uint8_t	VACUUM_THRESHOLD = 10
	We can vacuum 10% or more of the total in-memory footprint.

Private Member Functions
idx_t	GetAvailableBufferId () const
	Returns an available buffer id.
void	NextBufferWithFreeSpace ()
	Caches the next buffer that we're going to fill.

Private Attributes
MemoryTag	memory_tag
	Memory tag of memory that is allocated through the allocator.
idx_t	segment_size
idx_t	bitmask_count
	Number of validity_t values in the bitmask.
idx_t	bitmask_offset
	First starting byte of the payload (segments)
idx_t	available_segments_per_buffer
	Number of possible segment allocations per buffer.
idx_t	total_segment_count
unordered_map< idx_t, unique_ptr< FixedSizeBuffer > >	buffers
	Buffers containing the segments.
unordered_set< idx_t >	buffers_with_free_space
	Buffers with free space.
optional_idx	buffer_with_free_space
unordered_set< idx_t >	vacuum_buffers
	Buffers qualifying for a vacuum (helper field to allow for fast NeedsVacuum checks)

Detailed Description

The FixedSizeAllocator provides pointers to fixed-size memory segments of pre-allocated memory buffers. The pointers are IndexPointers, and the leftmost byte (metadata) must always be zero. It is also possible to directly request a C++ pointer to the underlying segment of an index pointer.

Member Function Documentation

GetHandle()

SegmentHandle duckdb::FixedSizeAllocator::GetHandle ( const IndexPointer  ptr )

inline

Get a segment handle.

                                                           {
        D_ASSERT(ptr.GetOffset() < available_segments_per_buffer);
 
        auto buffer_it = buffers.find(ptr.GetBufferId());
        D_ASSERT(buffer_it != buffers.end());
 
        auto offset = ptr.GetOffset() * segment_size + bitmask_offset;
        auto &buffer = *buffer_it->second;
        return SegmentHandle(buffer, offset);
    }

Get() [1/2]

template<class T >

unsafe_optional_ptr< T > duckdb::FixedSizeAllocator::Get ( const IndexPointer  ptr, const bool  dirty = true  )

inline

Returns a pointer of type T to a segment. If dirty is false, then T must be a const class. DEPRECATED. Use segment handles.

                                                                                       {
        return (T *)Get(ptr, dirty);
    }

Here is the call graph for this function:

Here is the caller graph for this function:

Get() [2/2]

data_ptr_t duckdb::FixedSizeAllocator::Get ( const IndexPointer  ptr, const bool  dirty = true  )

inline

Returns the data_ptr_t to a segment, and sets the dirty flag of the buffer containing that segment. DEPRECATED. Use segment handles.

                                                                           {
        D_ASSERT(ptr.GetOffset() < available_segments_per_buffer);
 
        auto buffer_it = buffers.find(ptr.GetBufferId());
        D_ASSERT(buffer_it != buffers.end());
 
        auto offset = ptr.GetOffset() * segment_size + bitmask_offset;
        auto buffer_ptr = buffer_it->second->GetDeprecated(dirty);
        return buffer_ptr + offset;
    }

LoadedFromStorage()

bool duckdb::FixedSizeAllocator::LoadedFromStorage ( const IndexPointer  ptr ) const

inline

Has been loaded into a buffer-managed buffer from a persistent, file-backed block.

                                                                {
        D_ASSERT(ptr.GetOffset() < available_segments_per_buffer);
 
        auto buffer_it = buffers.find(ptr.GetBufferId());
        if (buffer_it == buffers.end()) {
            return false;
        }
 
        return buffer_it->second->InMemory();
    }

GetIfLoaded() [1/2]

template<class T >

unsafe_optional_ptr< T > duckdb::FixedSizeAllocator::GetIfLoaded ( const IndexPointer  ptr )

inline

Returns a pointer of type T to a segment, or nullptr, if the buffer is not in memory. DEPRECATED. Use segment handles.

                                                                      {
        return (T *)GetIfLoaded(ptr);
    }

Here is the call graph for this function:

Here is the caller graph for this function:

GetIfLoaded() [2/2]

data_ptr_t duckdb::FixedSizeAllocator::GetIfLoaded ( const IndexPointer  ptr )

inline

Returns the data_ptr_t to a segment, or nullptr, if the buffer is not in memory. DEPRECATED. Use segment handles.

                                                          {
        D_ASSERT(ptr.GetOffset() < available_segments_per_buffer);
        D_ASSERT(buffers.find(ptr.GetBufferId()) != buffers.end());
 
        auto &buffer = buffers.find(ptr.GetBufferId())->second;
        if (!buffer->InMemory()) {
            return nullptr;
        }
 
        auto buffer_ptr = buffer->GetDeprecated();
        auto raw_ptr = buffer_ptr + ptr.GetOffset() * segment_size + bitmask_offset;
        return raw_ptr;
    }

GetSegmentSize()

idx_t duckdb::FixedSizeAllocator::GetSegmentSize ( ) const

inline

Returns the segment size.

                                        {
        return segment_size;
    }

GetSegmentCount()

idx_t duckdb::FixedSizeAllocator::GetSegmentCount ( ) const

inline

Returns the total segment count.

                                         {
        return total_segment_count;
    }

NeedsVacuum()

bool duckdb::FixedSizeAllocator::NeedsVacuum ( const IndexPointer  ptr ) const

inline

Returns true, if an IndexPointer qualifies for a vacuum operation, and false otherwise.

                                                          {
        if (vacuum_buffers.find(ptr.GetBufferId()) != vacuum_buffers.end()) {
            return true;
        }
        return false;
    }

Empty()

bool duckdb::FixedSizeAllocator::Empty ( )

inline

Returns true, if the allocator does not contain any segments.

                        {
        return total_segment_count == 0;
    }

Member Data Documentation

segment_size

idx_t duckdb::FixedSizeAllocator::segment_size

private

Allocation size of one segment in a buffer We only need this value to calculate bitmask_count, bitmask_offset, and available_segments_per_buffer

total_segment_count

idx_t duckdb::FixedSizeAllocator::total_segment_count

private

Total number of allocated segments in all buffers We can recalculate this by iterating over all buffers

buffer_with_free_space

optional_idx duckdb::FixedSizeAllocator::buffer_with_free_space

private

Caches the next buffer to be filled up. Unordered sets make no guarantee that begin() returns the same element. By caching one of the buffers with free space, we get more consistency when filling buffers.

---

The documentation for this class was generated from the following file:

• external/duckdb/duckdb.cpp