duckdb::CSVBuffer Class Reference

Collaboration diagram for duckdb::CSVBuffer:

Public Member Functions
	CSVBuffer (ClientContext &context, idx_t buffer_size_p, CSVFileHandle &file_handle, const idx_t &global_csv_current_position)
	Constructor for Initial Buffer.
	CSVBuffer (CSVFileHandle &file_handle, ClientContext &context, idx_t buffer_size, idx_t global_csv_current_position, idx_t buffer_idx)
	Constructor for Next() Buffers.
shared_ptr< CSVBuffer >	Next (CSVFileHandle &file_handle, idx_t buffer_size, bool &has_seeked) const
	Creates a new buffer with the next part of the CSV File.
idx_t	GetBufferSize () const
	Gets the buffer actual size.
bool	IsCSVFileLastBuffer () const
	If this buffer is the last buffer of the CSV File.
void	AllocateBuffer (idx_t buffer_size)
	Allocates internal buffer, sets 'block' and 'handle' variables.
void	Reload (CSVFileHandle &file_handle)
shared_ptr< CSVBufferHandle >	Pin (CSVFileHandle &file_handle, bool &has_seeked)
void	Unpin ()
	Wrapper for unpin.
char *	Ptr ()
bool	IsUnloaded () const

Public Attributes
bool	last_buffer = false

Static Public Attributes
static constexpr idx_t	ROWS_PER_BUFFER = 16
	By default, we use CSV_BUFFER_SIZE to allocate each buffer.
static constexpr idx_t	MIN_ROWS_PER_BUFFER = 4

Private Attributes
ClientContext &	context
idx_t	actual_buffer_size
	Actual size can be smaller than the buffer size in case we allocate it too optimistically.
idx_t	requested_size
idx_t	global_csv_start = 0
	Global position from the CSV File where this buffer starts.
bool	can_seek
	If we can seek in the file or not.
bool	is_pipe
	If this file is being fed by a pipe.
idx_t	buffer_idx = 0
	Buffer Index, used as a batch index for insertion-order preservation.
shared_ptr< BlockHandle >	block
BufferHandle	handle

Detailed Description

CSV Buffers are parts of a decompressed CSV File. For a decompressed file of 100Mb. With our Buffer size set to 32Mb, we would generate 4 buffers. One for the first 32Mb, second and third for the other 32Mb, and the last one with 4 Mb These buffers are actually used for sniffing and parsing!

Member Function Documentation

Pin()

shared_ptr< CSVBufferHandle > duckdb::CSVBuffer::Pin	(	CSVFileHandle &	file_handle,
		bool &	has_seeked
	)

Wrapper for the Pin Function, if it can seek, it means that the buffer might have been destroyed, hence we must Scan it from the disk file again.

Ptr()

char * duckdb::CSVBuffer::Ptr ( )

inline

                {
        return char_ptr_cast(handle.Ptr());
    }

IsUnloaded()

bool duckdb::CSVBuffer::IsUnloaded ( ) const

inline

                            {
        return block->GetMemory().IsUnloaded();
    }

Member Data Documentation

block

shared_ptr<BlockHandle> duckdb::CSVBuffer::block

private

-----— Allocated Block ------—// Block created in allocation

---

The documentation for this class was generated from the following file:

• external/duckdb/duckdb.hpp