duckdb::BlockManager Class Reference

Collaboration diagram for duckdb::BlockManager:

Public Member Functions
	BlockManager (BufferManager &buffer_manager, const optional_idx block_alloc_size_p, const optional_idx block_header_size_p)
BufferManager &	GetBufferManager () const
virtual unique_ptr< Block >	ConvertBlock (block_id_t block_id, FileBuffer &source_buffer)=0
	Creates a new block inside the block manager.
virtual unique_ptr< Block >	CreateBlock (block_id_t block_id, FileBuffer *source_buffer)=0
virtual block_id_t	GetFreeBlockId ()=0
	Return the next free block id.
virtual block_id_t	PeekFreeBlockId ()=0
virtual block_id_t	GetFreeBlockIdForCheckpoint ()=0
	Returns the next free block id and immediately include it in the checkpoint.
virtual bool	IsRootBlock (MetaBlockPointer root)=0
	Returns whether or not a specified block is the root block.
virtual void	MarkBlockAsCheckpointed (block_id_t block_id)=0
	Mark a block as included in the next checkpoint.
virtual void	MarkBlockAsUsed (block_id_t block_id)=0
	Mark a block as "used"; either the block is removed from the free list, or the reference count is incremented.
virtual void	MarkBlockAsModified (block_id_t block_id)=0
virtual void	IncreaseBlockReferenceCount (block_id_t block_id)=0
virtual idx_t	GetMetaBlock ()=0
	Get the first meta block id.
virtual void	Read (QueryContext context, Block &block)=0
	Read the content of the block from disk.
virtual void	ReadBlocks (FileBuffer &buffer, block_id_t start_block, idx_t block_count)=0
	Read the content of the block from disk.
virtual void	Write (FileBuffer &block, block_id_t block_id)=0
	Writes the block to disk.
virtual void	Write (QueryContext context, FileBuffer &block, block_id_t block_id)
void	Write (Block &block)
	Writes the block to disk.
virtual void	WriteHeader (QueryContext context, DatabaseHeader header)=0
	Write the header; should be the final step of a checkpoint.
virtual idx_t	TotalBlocks ()=0
	Returns the number of total blocks.
virtual idx_t	FreeBlocks ()=0
	Returns the number of free blocks.
virtual bool	IsRemote ()
	Whether or not the attached database is a remote file (e.g. attached over s3/https)
virtual bool	InMemory ()=0
	Whether or not the attached database is in-memory.
virtual bool	Prefetch ()
	Whether or not to prefetch.
virtual void	FileSync ()=0
	Sync changes made to the block manager.
virtual void	Truncate ()
	Truncate the underlying database file after a checkpoint.
shared_ptr< BlockHandle >	RegisterBlock (block_id_t block_id)
	Register a block with the given block id in the base file.
shared_ptr< BlockHandle >	ConvertToPersistent (QueryContext context, block_id_t block_id, shared_ptr< BlockHandle > old_block, BufferHandle old_handle, ConvertToPersistentMode mode=ConvertToPersistentMode::DESTRUCTIVE)
shared_ptr< BlockHandle >	ConvertToPersistent (QueryContext context, block_id_t block_id, shared_ptr< BlockHandle > old_block, ConvertToPersistentMode mode=ConvertToPersistentMode::DESTRUCTIVE)
void	UnregisterPersistentBlock (BlockHandle &block)
virtual void	UnregisterBlock (block_id_t id)
	UnregisterBlock, only accepts non-temporary block ids.
MetadataManager &	GetMetadataManager ()
	Returns a reference to the metadata manager of this block manager.
idx_t	GetBlockAllocSize () const
	Returns the block allocation size of this block manager.
optional_idx	GetOptionalBlockAllocSize () const
	Returns the possibly invalid block allocation size of this block manager.
optional_idx	GetOptionalBlockHeaderSize () const
	Returns the possibly invalid block header size of this block manager.
idx_t	GetBlockHeaderSize () const
	Returns the block header size including the 8-byte checksum of this block manager.
idx_t	GetBlockSize () const
	Returns the size of the block that is available for usage.
void	SetBlockAllocSize (const optional_idx block_alloc_size_p)
void	SetBlockHeaderSize (const optional_idx block_header_size_p)
virtual void	VerifyBlocks (const unordered_map< block_id_t, idx_t > &block_usage_count)
	Verify the block usage count.
template<class TARGET >
TARGET &	Cast ()
template<class TARGET >
const TARGET &	Cast () const

Public Attributes
BufferManager &	buffer_manager
	The buffer manager.

Protected Member Functions
bool	BlockIsRegistered (block_id_t block_id)
shared_ptr< BlockHandle >	TryGetBlock (block_id_t block_id)

Protected Attributes
bool	in_destruction = false

Private Attributes
mutex	blocks_lock
	The lock for the set of blocks.
unordered_map< block_id_t, weak_ptr< BlockHandle > >	blocks
	A mapping of block id -> BlockHandle.
unique_ptr< MetadataManager >	metadata_manager
	The metadata manager.
optional_idx	block_alloc_size
optional_idx	block_header_size

Detailed Description

BlockManager is an abstract representation to manage blocks. When writing or reading blocks, the BlockManager creates and accesses them. The concrete types implement specific block storage strategies.

Member Function Documentation

GetBufferManager()

BufferManager & duckdb::BlockManager::GetBufferManager ( ) const

inline

                                            {
        return buffer_manager;
    }

MarkBlockAsModified()

virtual void duckdb::BlockManager::MarkBlockAsModified ( block_id_t  block_id )

pure virtual

Mark a block as "modified"; modified blocks are added to the free list after a checkpoint (i.e. their data is assumed to be rewritten)

IncreaseBlockReferenceCount()

virtual void duckdb::BlockManager::IncreaseBlockReferenceCount ( block_id_t  block_id )

pure virtual

Increase the reference count of a block. The block should hold at least one reference before this method is called.

Write()

void duckdb::BlockManager::Write ( Block &  block )

inline

Writes the block to disk.

                             {
        Write(block, block.id);
    }

Here is the call graph for this function:

IsRemote()

virtual bool duckdb::BlockManager::IsRemote ( )

inlinevirtual

Whether or not the attached database is a remote file (e.g. attached over s3/https)

                            {
        return false;
    }

Prefetch()

virtual bool duckdb::BlockManager::Prefetch ( )

inlinevirtual

Whether or not to prefetch.

                            {
        return false;
    }

ConvertToPersistent()

shared_ptr< BlockHandle > duckdb::BlockManager::ConvertToPersistent	(	QueryContext	context,
		block_id_t	block_id,
		shared_ptr< BlockHandle >	old_block,
		BufferHandle	old_handle,
		ConvertToPersistentMode	mode = ConvertToPersistentMode::DESTRUCTIVE
	)

Convert an existing in-memory buffer into a persistent disk-backed block If mode is set to destructive (default) - the old_block will be destroyed as part of this method This can only be safely used when there is no other (lingering) usage of old_block If there is concurrent usage of the block elsewhere - use the THREAD_SAFE mode which creates an extra copy

GetBlockAllocSize()

idx_t duckdb::BlockManager::GetBlockAllocSize ( ) const

inline

Returns the block allocation size of this block manager.

                                    {
        return block_alloc_size.GetIndex();
    }

GetOptionalBlockAllocSize()

optional_idx duckdb::BlockManager::GetOptionalBlockAllocSize ( ) const

inline

Returns the possibly invalid block allocation size of this block manager.

                                                          {
        return block_alloc_size;
    }

GetOptionalBlockHeaderSize()

optional_idx duckdb::BlockManager::GetOptionalBlockHeaderSize ( ) const

inline

Returns the possibly invalid block header size of this block manager.

                                                           {
        return block_header_size;
    }

GetBlockHeaderSize()

idx_t duckdb::BlockManager::GetBlockHeaderSize ( ) const

inline

Returns the block header size including the 8-byte checksum of this block manager.

                                     {
        if (!block_header_size.IsValid()) {
            return Storage::DEFAULT_BLOCK_HEADER_SIZE;
        }
        return block_header_size.GetIndex();
    }

Here is the caller graph for this function:

GetBlockSize()

idx_t duckdb::BlockManager::GetBlockSize ( ) const

inline

Returns the size of the block that is available for usage.

                               {
        return block_alloc_size.GetIndex() - block_header_size.GetIndex();
    }

Here is the caller graph for this function:

SetBlockAllocSize()

void duckdb::BlockManager::SetBlockAllocSize ( const optional_idx  block_alloc_size_p )

inline

Sets the block allocation size. This should only happen when initializing an existing database. When initializing an existing database, we construct the block manager before reading the file header, which contains the file's actual block allocation size.

                                                                  {
        if (block_alloc_size.IsValid()) {
            throw InternalException("the block allocation size must be set once");
        }
        block_alloc_size = block_alloc_size_p.GetIndex();
    }

SetBlockHeaderSize()

void duckdb::BlockManager::SetBlockHeaderSize ( const optional_idx  block_header_size_p )

inline

Sets the block header size. Idem as above. This is only set once upon initialization of the database For now this method is unused

                                                                    {
        if (block_header_size.IsValid()) {
            throw InternalException("block header size already set, must be set once");
        }
        block_header_size = block_header_size_p.GetIndex();
    }

VerifyBlocks()

virtual void duckdb::BlockManager::VerifyBlocks ( const unordered_map< block_id_t, idx_t > &  block_usage_count )

inlinevirtual

Verify the block usage count.

                                                                                         {
    }

Cast() [1/2]

template<class TARGET >

TARGET & duckdb::BlockManager::Cast ( )

inline

                   {
        DynamicCastCheck<TARGET>(this);
        return reinterpret_cast<TARGET &>(*this);
    }

Cast() [2/2]

template<class TARGET >

const TARGET & duckdb::BlockManager::Cast ( ) const

inline

                               {
        DynamicCastCheck<TARGET>(this);
        return reinterpret_cast<const TARGET &>(*this);
    }

Member Data Documentation

in_destruction

bool duckdb::BlockManager::in_destruction = false

protected

A flag to be flipped in the destructor of the subclass, which is called first. Relevant for some Windows edge cases.

block_alloc_size

optional_idx duckdb::BlockManager::block_alloc_size

private

The allocation size of blocks managed by this block manager. Defaults to DEFAULT_BLOCK_ALLOC_SIZE for in-memory block managers. Default to default_block_alloc_size for file-backed block managers. This is NOT the actual memory available on a block (block_size).

block_header_size

optional_idx duckdb::BlockManager::block_header_size

private

The size of the block headers (including checksum) in this block manager. Defaults to DEFAULT_BLOCK_HEADER_SIZE for in-memory block managers. Defaults to default_block_header_size for file-backed block managers.

---

The documentation for this class was generated from the following file:

• external/duckdb/duckdb.hpp