duckdb::ArrowSchemaMetadata Class Reference

Collaboration diagram for duckdb::ArrowSchemaMetadata:

Public Member Functions
	ArrowSchemaMetadata (const char *metadata)
	Constructor used to read a metadata schema, used when importing an arrow object.
	ArrowSchemaMetadata ()
	Constructor used to create a metadata schema, used when exporting an arrow object.
void	AddOption (const string &key, const string &value)
	Adds an option to the metadata.
string	GetOption (const string &key) const
	Gets an option from the metadata, returns an empty string if it does not exist.
unsafe_unique_array< char >	SerializeMetadata () const
	Transforms metadata to a char*, used when creating an arrow object.
bool	HasExtension () const
	If the arrow extension is set.
ArrowExtensionMetadata	GetExtensionInfo (string format)
string	GetExtensionName () const
	Get the extension name if set, otherwise returns empty.

Static Public Member Functions
static ArrowSchemaMetadata	ArrowCanonicalType (const string &extension_name)
	Creates the metadata based on an extension name.
static ArrowSchemaMetadata	NonCanonicalType (const string &type_name, const string &vendor_name)
	Creates the metadata based on an extension name.

Static Public Attributes
static constexpr const char *	ARROW_EXTENSION_NAME = "ARROW:extension:name"
	Key for encode of the extension type name.
static constexpr const char *	ARROW_METADATA_KEY = "ARROW:extension:metadata"
	Key for encode of the metadata key.

Private Attributes
unordered_map< string, string >	schema_metadata_map
	The unordered map that holds the metadata.
unique_ptr< ComplexJSON >	extension_metadata_map
	The extension metadata map, currently only used for internal types in arrow.opaque.

Constructor & Destructor Documentation

ArrowSchemaMetadata() [1/2]

duckdb::ArrowSchemaMetadata::ArrowSchemaMetadata ( const char *  metadata )

explicit

Constructor used to read a metadata schema, used when importing an arrow object.

                                                             {
    if (metadata) {
        // Read the number of key-value pairs (int32)
        int32_t num_pairs;
        memcpy(&num_pairs, metadata, sizeof(int32_t));
        metadata += sizeof(int32_t);
 
        // Loop through each key-value pair
        for (int32_t i = 0; i < num_pairs; ++i) {
            // Read the length of the key (int32)
            int32_t key_length;
            memcpy(&key_length, metadata, sizeof(int32_t));
            metadata += sizeof(int32_t);
 
            // Read the key
            std::string key(metadata, static_cast<idx_t>(key_length));
            metadata += key_length;
 
            // Read the length of the value (int32)
            int32_t value_length;
            memcpy(&value_length, metadata, sizeof(int32_t));
            metadata += sizeof(int32_t);
 
            // Read the value
            const std::string value(metadata, static_cast<idx_t>(value_length));
            metadata += value_length;
            schema_metadata_map[key] = value;
        }
    }
    // We ignore errors of the metadata parsing if the extension is different from arrow.opaque
    const bool ignore_errors =
        schema_metadata_map[ARROW_EXTENSION_NAME] != ArrowExtensionMetadata::ARROW_EXTENSION_NON_CANONICAL;
    extension_metadata_map = StringUtil::ParseJSONMap(schema_metadata_map[ARROW_METADATA_KEY], ignore_errors);
}

Here is the call graph for this function:

ArrowSchemaMetadata() [2/2]

duckdb::ArrowSchemaMetadata::ArrowSchemaMetadata

(

)

Constructor used to create a metadata schema, used when exporting an arrow object.

                                         {
    // Always initialize out metadata map
    extension_metadata_map = make_uniq<ComplexJSON>();
}

Member Function Documentation

AddOption()

void duckdb::ArrowSchemaMetadata::AddOption	(	const string &	key,
		const string &	value
	)

Adds an option to the metadata.

                                                                          {
    schema_metadata_map[key] = value;
}

Here is the caller graph for this function:

GetOption()

string duckdb::ArrowSchemaMetadata::GetOption

(

const string &

key

)

const

Gets an option from the metadata, returns an empty string if it does not exist.

                                                             {
    auto it = schema_metadata_map.find(key);
    if (it != schema_metadata_map.end()) {
        return it->second;
    } else {
        return "";
    }
}

Here is the caller graph for this function:

SerializeMetadata()

unsafe_unique_array< char > duckdb::ArrowSchemaMetadata::SerializeMetadata

(

)

const

Transforms metadata to a char*, used when creating an arrow object.

                                                                       {
    // First we have to figure out the total size:
    // 1. number of key-value pairs (int32)
    idx_t total_size = sizeof(int32_t);
    for (const auto &option : schema_metadata_map) {
        // 2. Length of the key and value (2 * int32)
        total_size += 2 * sizeof(int32_t);
        // 3. Length of key
        total_size += option.first.size();
        // 4. Length of value
        total_size += option.second.size();
    }
    auto metadata_array_ptr = make_unsafe_uniq_array<char>(total_size);
    auto metadata_ptr = metadata_array_ptr.get();
    // 1. number of key-value pairs (int32)
    const int32_t map_size = static_cast<int32_t>(schema_metadata_map.size());
    memcpy(metadata_ptr, &map_size, sizeof(int32_t));
    metadata_ptr += sizeof(int32_t);
    // Iterate through each key-value pair in the map
    for (const auto &pair : schema_metadata_map) {
        const std::string &key = pair.first;
        int32_t key_size = static_cast<int32_t>(key.size());
        // Length of the key (int32)
        memcpy(metadata_ptr, &key_size, sizeof(int32_t));
        metadata_ptr += sizeof(int32_t);
        // Key
        memcpy(metadata_ptr, key.c_str(), key.size());
        metadata_ptr += key_size;
        const std::string &value = pair.second;
        const int32_t value_size = static_cast<int32_t>(value.size());
        // Length of the value (int32)
        memcpy(metadata_ptr, &value_size, sizeof(int32_t));
        metadata_ptr += sizeof(int32_t);
        // Value
        memcpy(metadata_ptr, value.c_str(), value.size());
        metadata_ptr += value_size;
    }
    return metadata_array_ptr;
}

HasExtension()

bool duckdb::ArrowSchemaMetadata::HasExtension

(

)

const

If the arrow extension is set.

                                             {
    auto arrow_extension = GetOption(ArrowSchemaMetadata::ARROW_EXTENSION_NAME);
    return !arrow_extension.empty();
}

Here is the call graph for this function:

GetExtensionInfo()

ArrowExtensionMetadata duckdb::ArrowSchemaMetadata::GetExtensionInfo

(

string

format

)

                                                                          {
    return {schema_metadata_map[ARROW_EXTENSION_NAME], extension_metadata_map->GetValue("vendor_name"),
            extension_metadata_map->GetValue("type_name"), std::move(format)};
}

ArrowCanonicalType()

ArrowSchemaMetadata duckdb::ArrowSchemaMetadata::ArrowCanonicalType ( const string &  extension_name )

static

Creates the metadata based on an extension name.

                                                                                        {
    ArrowSchemaMetadata metadata;
    metadata.AddOption(ARROW_EXTENSION_NAME, extension_name);
    metadata.AddOption(ARROW_METADATA_KEY, "");
    return metadata;
}

Here is the call graph for this function:

NonCanonicalType()

ArrowSchemaMetadata duckdb::ArrowSchemaMetadata::NonCanonicalType ( const string &  type_name, const string &  vendor_name  )

static

Creates the metadata based on an extension name.

                                                                                                            {
    ArrowSchemaMetadata metadata;
    metadata.AddOption(ARROW_EXTENSION_NAME, ArrowExtensionMetadata::ARROW_EXTENSION_NON_CANONICAL);
    // We have to set the metadata key with type_name and vendor_name.
    metadata.extension_metadata_map->AddObjectEntry("vendor_name", make_uniq<ComplexJSON>(vendor_name));
    metadata.extension_metadata_map->AddObjectEntry("type_name", make_uniq<ComplexJSON>(type_name));
    metadata.AddOption(ARROW_METADATA_KEY, StringUtil::ToComplexJSONMap(*metadata.extension_metadata_map));
    return metadata;
}

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• external/duckdb/duckdb.cpp