locomotiv::Session Class Reference

Session for loco graph inference. More...

#include <Session.h>

Public Member Functions
	Session ()=delete
	Session (loco::Graph *g)
	Make Session for graph with graph outputs themselves.
	Session (loco::Graph *g, const std::vector< loco::Node * > &custom_outputs)
	Make Session for graph with selective custom outputs. Only subgraph to calculate given outputs would be executed.
template<typename InputIt >
	Session (loco::Graph *g, InputIt begin, InputIt end)
	Make Session by range.
	~Session ()
	Free all node annotations of the graph assigned by this Session.
uint32_t	input_size () const
	Get number of graph inputs held by this Session.
void	set_input (uint32_t index, std::unique_ptr< NodeData > &&data)
	Set graph input at specific index by NodeData.
void	infer ()
	Do inference for this session and graph.
uint32_t	output_size () const
	Get number of graph outputs held by this Session.
const NodeData *	get_output (uint32_t index)
	Get output of graph as NodeData.
const loco::Node *	get_output_node (uint32_t index)

Detailed Description

Session for loco graph inference.

Definition at line 33 of file Session.h.

Constructor & Destructor Documentation

Session() [1/4]

locomotiv::Session::Session ( )

delete

Session() [2/4]

locomotiv::Session::Session ( loco::Graph *  g )

inline

Make Session for graph with graph outputs themselves.

Definition at line 39 of file Session.h.

                        : Session(g, loco::output_nodes(g))
  {
    // DO NOTHING
  }

Session() [3/4]

locomotiv::Session::Session ( loco::Graph *  g, const std::vector< loco::Node * > &  custom_outputs  )

inline

Make Session for graph with selective custom outputs. Only subgraph to calculate given outputs would be executed.

Note

Set required inputs for given outputs, or inference may fail.

custom_outputs don't need to be graph output, but can be any nodes in the middle of the graph. @warn This approach may fail in case of graph with control flow

Definition at line 53 of file Session.h.

    : _graph(g), _outputs(custom_outputs)
  {
    // DO NOTHING
  }

Session() [4/4]

template<typename InputIt >

locomotiv::Session::Session ( loco::Graph *  g, InputIt  begin, InputIt  end  )

inline

Make Session by range.

Definition at line 61 of file Session.h.

                                                    : _graph(g), _outputs(begin, end)
  {
    // DO NOTHING
  }

~Session()

locomotiv::Session::~Session

(

)

Free all node annotations of the graph assigned by this Session.

Definition at line 30 of file Session.cpp.

{
  for (uint32_t i = 0; i < _graph->nodes()->size(); ++i)
  {
    auto node = _graph->nodes()->at(i);
    erase_user_data(node);
    erase_annot_data(node);
    erase_annot_domain(node);
  }
}

References loco::ObjectPool< T >::at(), locomotiv::erase_annot_domain(), loco::Graph::nodes(), and size.

Member Function Documentation

get_output()

const NodeData * locomotiv::Session::get_output

(

uint32_t

index

)

Get output of graph as NodeData.

Note

May return nullptr, for example, when graph output not yet calculated

Definition at line 85 of file Session.cpp.

{
  assert(index < output_size());
 
  auto output_node = _outputs.at(index);
  return annot_data(output_node);
}

References output_size().

Referenced by nnkit::support::moco::tf::OutputTensorContext::getConstFloatTensor().

get_output_node()

const loco::Node * locomotiv::Session::get_output_node ( uint32_t  index )

inline

Definition at line 99 of file Session.h.

{ return _outputs.at(index); }

infer()

void locomotiv::Session::infer

(

)

Do inference for this session and graph.

Note

Multiple run is possible. Abort program when inputs are not fully set or invalid calculation found in the middle.

Definition at line 75 of file Session.cpp.

{
  auto schedules = loco::postorder_traversal(_outputs);
 
  for (auto node : schedules)
  {
    NodeExecution::get().run(node);
  }
}

References locomotiv::NodeExecution::get(), loco::postorder_traversal(), and locomotiv::NodeExecution::run().

input_size()

uint32_t locomotiv::Session::input_size ( ) const

inline

Get number of graph inputs held by this Session.

Definition at line 70 of file Session.h.

{ return _graph->inputs()->size(); }

References loco::Graph::inputs(), and loco::ObjectPool< T >::size().

Referenced by set_input().

output_size()

uint32_t locomotiv::Session::output_size ( ) const

inline

Get number of graph outputs held by this Session.

Definition at line 90 of file Session.h.

{ return _outputs.size(); }

Referenced by get_output().

set_input()

void locomotiv::Session::set_input	(	uint32_t	index,
		std::unique_ptr< NodeData > &&	data
	)

Set graph input at specific index by NodeData.

Exceptions

runtime_error

In case when another NodeData already annotated for the input, and when given data type or shape are not congruent with loco node information.

Definition at line 41 of file Session.cpp.

{
  assert(index < input_size());
 
  // Check whether already annotated
  auto pull = loco::pull_node(_graph, index);
  if (user_data(pull))
  {
    throw std::runtime_error("Graph input already has NodeData");
  }
 
  // Check data type match
  if (pull->dtype() != data->dtype())
  {
    throw std::runtime_error("Data type mismatch");
  }
 
  // Check shape match
  auto shape = data->shape();
  if (pull->rank() != shape->rank())
  {
    throw std::runtime_error("Shape rank mismatch");
  }
  for (uint32_t i = 0; i < pull->rank(); ++i)
  {
    if (pull->dim(i).known() && pull->dim(i).value() != shape->dim(i))
    {
      throw std::runtime_error("Shape dimension mismatch");
    }
  }
 
  user_data(pull, std::move(data));
}

References input_size(), and loco::pull_node().

Referenced by package.infer.session::set_inputs().

---

The documentation for this class was generated from the following files:

• compiler/locomotiv/include/locomotiv/Session.h
• compiler/locomotiv/src/Session.cpp