ggma::GGMAConfig Struct Reference

#include <Config.h>

Collaboration diagram for ggma::GGMAConfig:

Data Fields
ModelConfig	model
int	cache_size = 32
int	ubatch = 32
KVCacheDataType	kv_cache_type = KVCacheDataType::FLOAT32

Detailed Description

Definition at line 65 of file Config.h.

Field Documentation

cache_size

int ggma::GGMAConfig::cache_size = 32

Definition at line 68 of file Config.h.

Referenced by ggma::Context::Context(), and ggma::Context::generate().

kv_cache_type

KVCacheDataType ggma::GGMAConfig::kv_cache_type = KVCacheDataType::FLOAT32

Definition at line 70 of file Config.h.

Referenced by ggma::KVCache::init().

model

ModelConfig ggma::GGMAConfig::model

Definition at line 67 of file Config.h.

Referenced by ggma::Context::generate(), ggma::KVCache::init(), ggma::Context::prefill(), and ggma::Context::sample().

ubatch

int ggma::GGMAConfig::ubatch = 32

Definition at line 69 of file Config.h.

Referenced by ggma::Context::unemb().

---

The documentation for this struct was generated from the following file:

• runtime/ggma/src/Config.h