onert::backend::basic::ExternalTensor Class Reference

Class that uses data from external memory that is not managed by a backend instead of allocating and copying the data. (ex. constant data from model file) ExternalTensor's data pointer points to an address of memory such as where memory is already allocated, or mmapped area. This is meaning that ExternalTensor can take all of types' ir::Data. To support this, assume below things no padding, always NHWC layout, constant tensor and not dynamic. More...

#include <Tensor.h>

Collaboration diagram for onert::backend::basic::ExternalTensor:

Public Member Functions
	ExternalTensor ()=delete
virtual	~ExternalTensor ()
	ExternalTensor (const ir::OperandInfo &info)
void	setData (const std::shared_ptr< ir::Data > data)
	set Data to be shared from external so that this ExternalTensor will not be allocated on CPU backend
uint8_t *	buffer () const override
void	set_dynamic () override
	set this tensor dynamic
void	setShape (const ir::Shape &) override
	Set the shape of tenser to new_shape.
void	increase_ref () override
void	decrease_ref () override
void	reset_ref () override
	Reset reference count to zero and release data.
int32_t	num_references () override
Public Member Functions inherited from onert::backend::basic::Tensor
	Tensor ()=delete
virtual	~Tensor ()
	Tensor (const ir::OperandInfo &info, DynamicMemoryManager *dynamic_mem_mgr)
void	setBuffer (uint8_t *buffer)
	Set the Buffer object. This method is called for static and non-const tensor.
void	setBuffer (const std::shared_ptr< Allocator > &alloc)
	Set the Buffer object. This method is called for dynamic or const tensor.
void	deallocBuffer () override
	Reset the buffer and deallocate the allocation if it is managed by itself.
uint8_t *	buffer () const override
void	set_dynamic () override
	set this tensor dynamic
bool	applyShape (const ir::Shape &new_shape) override
	Set the shape to shape and possibly re-allocate the buffer.
void	setShape (const ir::Shape &new_shape) override
	Set the shape of tenser to new_shape.
Public Member Functions inherited from onert::backend::IPortableTensor
	IPortableTensor (const ir::OperandInfo &info)
virtual	~IPortableTensor ()
const ir::OperandInfo &	get_info () const
const ir::Sparsity *	sparsity () const
size_t	total_size () const override final
size_t	calcOffset (const ir::Coordinates &coords) const override final
ir::DataType	data_type () const override final
float	data_scale () const override final
int32_t	data_zero_point () const override final
const std::vector< float > &	data_scales () const override final
const std::vector< int32_t > &	data_zero_points () const override
bool	is_constant () const override final
	Return true if the tensor is constant.
bool	is_dynamic () const override final
	Return true if the tensor needs dynamic allocation, meaning that during compile-time the outpus shape cannot be known and the output shape is calculated during kernel execution-time.
ir::Shape	getShape () const override final
	Get ir::Shape of tensor.
bool	has_padding () const final
void	access (const std::function< void(ITensor &tensor)> &fn) final
Public Member Functions inherited from onert::backend::ITensor
virtual	~ITensor ()
virtual bool	is_subtensor () const
virtual bool	needMemoryMap () const
virtual void	enqueueWriteBuffer (const void *, bool)
virtual void	enqueueReadBuffer (void *, bool)

Additional Inherited Members
Protected Attributes inherited from onert::backend::basic::Tensor
uint8_t *	_buffer
size_t	_size
int32_t	_num_references
DynamicMemoryManager *	_dynamic_mem_mgr
Protected Attributes inherited from onert::backend::IPortableTensor
ir::OperandInfo	_info

Detailed Description

Class that uses data from external memory that is not managed by a backend instead of allocating and copying the data. (ex. constant data from model file) ExternalTensor's data pointer points to an address of memory such as where memory is already allocated, or mmapped area. This is meaning that ExternalTensor can take all of types' ir::Data. To support this, assume below things no padding, always NHWC layout, constant tensor and not dynamic.

Definition at line 151 of file Tensor.h.

Constructor & Destructor Documentation

ExternalTensor() [1/2]

onert::backend::basic::ExternalTensor::ExternalTensor ( )

delete

~ExternalTensor()

onert::backend::basic::ExternalTensor::~ExternalTensor ( )

virtual

Definition at line 83 of file Tensor.cc.

{}

ExternalTensor() [2/2]

onert::backend::basic::ExternalTensor::ExternalTensor ( const ir::OperandInfo &  info )

inline

Definition at line 158 of file Tensor.h.

                                            : Tensor(info, nullptr)
  {
    assert(_info.isConstant());
    assert(_info.isDynamic() == false);
  }

References onert::backend::IPortableTensor::_info, onert::ir::OperandInfo::isConstant(), and onert::ir::OperandInfo::isDynamic().

Member Function Documentation

buffer()

uint8_t * onert::backend::basic::ExternalTensor::buffer ( ) const

inlineoverridevirtual

Implements onert::backend::ITensor.

Definition at line 180 of file Tensor.h.

{ return _buffer; }

References onert::backend::basic::Tensor::_buffer.

decrease_ref()

void onert::backend::basic::ExternalTensor::decrease_ref ( )

inlineoverridevirtual

Reimplemented from onert::backend::basic::Tensor.

Definition at line 194 of file Tensor.h.

  {
    assert(_data != nullptr);
    assert(_num_references > 0);
    --_num_references;
    if (_num_references == 0)
    {
      _data.reset();
      _buffer = nullptr;
    }
  }

References onert::backend::basic::Tensor::_buffer, and onert::backend::basic::Tensor::_num_references.

increase_ref()

void onert::backend::basic::ExternalTensor::increase_ref ( )

inlineoverridevirtual

Reimplemented from onert::backend::basic::Tensor.

Definition at line 192 of file Tensor.h.

{ ++_num_references; }

References onert::backend::basic::Tensor::_num_references.

num_references()

int32_t onert::backend::basic::ExternalTensor::num_references ( )

inlineoverridevirtual

Reimplemented from onert::backend::basic::Tensor.

Definition at line 219 of file Tensor.h.

{ return _num_references; }

References onert::backend::basic::Tensor::_num_references.

reset_ref()

void onert::backend::basic::ExternalTensor::reset_ref ( )

inlineoverridevirtual

Reset reference count to zero and release data.

Reimplemented from onert::backend::basic::Tensor.

Definition at line 209 of file Tensor.h.

  {
    assert(_data != nullptr);
    assert(_num_references > 0);
    _num_references = 0;
 
    _data.reset();
    _buffer = nullptr;
  }

References onert::backend::basic::Tensor::_buffer, and onert::backend::basic::Tensor::_num_references.

set_dynamic()

void onert::backend::basic::ExternalTensor::set_dynamic ( )

inlineoverridevirtual

set this tensor dynamic

Reimplemented from onert::backend::ITensor.

Definition at line 182 of file Tensor.h.

  {
    throw std::runtime_error("This tensor does not support changing dynamic");
  }

setData()

void onert::backend::basic::ExternalTensor::setData ( const std::shared_ptr< ir::Data >  data )

inline

set Data to be shared from external so that this ExternalTensor will not be allocated on CPU backend

Parameters

[in]

data

data of Operand to be set

Definition at line 170 of file Tensor.h.

  {
    assert(data != nullptr);
    _data = data;
    // Note. Some op such as cker::Conv could take buffer as nullptr.
    // That's why _buffer also would be used
    _buffer = const_cast<uint8_t *>(_data->base());
  }

References onert::backend::basic::Tensor::_buffer.

Referenced by onert::backend::basic::initConsts().

setShape()

void onert::backend::basic::ExternalTensor::setShape ( const ir::Shape &  )

inlineoverridevirtual

Set the shape of tenser to new_shape.

Note

Higer dimension will be placed on front.

Reimplemented from onert::backend::ITensor.

Definition at line 187 of file Tensor.h.

  {
    throw std::runtime_error("This tensor does not support changing shape");
  }

---

The documentation for this class was generated from the following files:

• runtime/onert/core/include/backend/basic/Tensor.h
• runtime/onert/core/src/backend/basic/Tensor.cc