5. Class¶

5.1. Builder¶

std::ostream &operator<<(std::ostream &out, Builder builder)¶: Feed the built IR to the specific output stream.

class topsgraph::Builder¶

#include <hlir_builder_common.h>

This class helps build IR, which consist of Ops.

Public Functions

explicit Builder()¶: Constructor of Builder.

bool IsDynamic() const¶

Judge if the build function is dynamic shape.

Param

void SetFuncAttribute(const char *name, const Attribute &value)¶

Set function attribute parameters.

Parameters

name -- attribute name
value -- attribute object

const Attribute GetFuncAttribute(const char *name) const¶

get function attribute parameters info.

Parameters: name -- attribute name
Returns: attribute object

void Print(std::ostream &out) const¶

Print the built IR to the specific output stream.

Parameters

out -- Output stream
flags -- Print flag

void Dump() const¶: Print the built IR to stdout.

std::shared_ptr<Impl> GetImpl() const¶: pimpl, only used for internal developers.

bool HaveFunc(const char *func_name) const¶

Does the function name already exist.

Parameters: func_name -- Name of the specific Function
Returns: Returns true if it exists, otherwise returns false

std::shared_ptr<hlir::Module> GetModule()¶

Get built IR as hlir::Module instance.

Returns: Built IR as hlir::Module instance

std::shared_ptr<char> GetModuleStr() const¶

Get the built IR string.

Parameters: flags -- Print flag. Default: PrintingFlags::NoFlag.
Returns: Built IR string.

std::size_t GetModuleHash() const¶

Get Built hlir::Module Hash.

Returns: Built hlir::Module Hash.

Attribute GetModuleAttribute(const char *name) const¶

get module attribute parameters info.

Parameters: name -- attribute name
Returns: attribute object

Attribute GetFuncAttribute(const char *func_name, const char *name) const¶

get function attribute parameters info.

Parameters

func_name -- Name of the specific Function
name -- attribute name

Returns

attribute object

void SetModuleAttribute(const char *name, const Attribute &value)¶

Set module attribute parameters.

Parameters

name -- attribute name
value -- attribute object

void SetModuleAttribute(const char *name, const std::vector<Attribute> &value)¶

Set module array attribute parameters.

Parameters

name -- attribute name
value -- vector of attribute object

void SetFuncAttribute(const char *func_name, const char *name, const Attribute &value)¶

Set function attribute parameters.

Parameters

func_name -- Name of the specific Function
name -- attribute name
value -- attribute object

void SetFuncAttribute(const char *func_name, const char *name, const std::vector<Attribute> &value)¶

Set function array attribute parameters.

Parameters

func_name -- Name of the specific Function
name -- attribute name
value -- vector of attribute object

void SetMaxMinShape(const std::vector<std::vector<int64_t>> &max_shapes, const std::vector<std::vector<int64_t>> &min_shapes)¶

Set Dynamic input's max_shape_dim, min_shape_dim attributes. If the input dim is not unknown dim, the value in max_shape_dim and min_shape_dim is same, and equal to the real dim.

Parameters

max_shapes -- input's max_shape_dims. max_shapes.size equal to the count of input op.
min_shapes -- input's min_shape_dims. min_shapes.size equal to the count of input op.

Op CreateInput(const Type &type, const char *func_name = ~~"main"~~)¶

Create an input Op for a Function. You can smiply consider a Function as a C++ function which consist of Ops.

Parameters

type -- Type of the input
func_name -- Name of the specific Function, default is main

Returns

The created input Op

Op CreateInput(const Type &type, const std::vector<const char*> &symbolic_dims)¶

create an input op for 'main' function with symbolic dims.

ex1. input type: -1x1x3xf32, symbolic_dims is {"s0", "1", "3"} ex2. input type: f32, symbolic_dims is {} ex3. input type: 1x2xf32, symbolic_dims is {} or {"1", "2"}

Parameters

type -- Type of the input
symbolic_dims -- symbolic dims for the input.

Returns

The created input Op

void AddFunc(const char *name)¶: Add a Function with a specific name to the Builder.

void SetOutput(const std::vector<Op> &outputs = {}, const char *func_name = ~~"main"~~)¶

Set output Ops of a Function.

Parameters

outputs -- A vector of output Ops
func_name -- Name of the specific Function, default is main

5.2. Type¶

std::ostream &operator<<(std::ostream &out, Type type)¶: Feed the Type information to the specific output stream.

class topsgraph::Type¶

#include <hlir_builder_common.h>

This represents the output type of an Op. It contains the information of both the shape and the primitive type.

Public Functions

explicit Type(const PrimitiveType &primitive_type)¶

Constructor of an scalar Type.

Parameters: primitive_type -- Data type of the scalar

bool IsDynamic() const¶

Judge if it's dynamic, i.e. with unknown dim shape or unrank.

Returns

Judge result

Returns

true -- Dynamic shape
false -- Static shape

bool IsTuple() const¶

Judge if it's a tuple type.

Returns

Judge result

Returns

true -- tuple
false -- others

const Shape GetShape() const¶

Get the shape.

Returns: the shape

const PrimitiveType GetPrimitiveType() const¶

Get the data type.

Returns: the data type

void Print(std::ostream &out) const¶

Print the Type information to the specific output stream.

Parameters: out -- Output stream

void Dump() const¶: Print the Type information to stdout.

bool operator==(const Type &other) const¶

Overloaded operator "==" to judge if 2 Type are the same.

Parameters

type -- another Type instance

Returns

Judge result

Returns

true -- shape and primitive type are the same
false -- otherwise

bool operator!=(const Type &other) const¶

Overloaded operator "!=" to judge if 2 Type are different.

Parameters

type -- another Type instance

Returns

Judge result

Returns

true -- shape or primitive type is different
false -- otherwise

std::shared_ptr<Impl> GetImpl() const¶: pimpl, only used for internal developers.

Type(const std::vector<int64_t> &shape, const PrimitiveType &primitive_type)¶

Constructor of an tensor Type.

Parameters

shape -- Shape of the tensor
primitive_type -- Data type of the tensor

Type(const std::vector<std::vector<int64_t>> &shape, const std::vector<PrimitiveType> &primitive_type)¶

Constructor of an tuple Type.

Parameters

shape -- Shape of the tuple
primitiveType -- Data type of the tuple

bool IsEmpty() const¶

Judge if it's empty, i.e. with empty shape and NONE data type.

Returns

Judge result

Returns

true -- Empty Type
false -- Scalar, Tensor or Tuple Type

void SetShape(const std::vector<int64_t> &shape)¶

Set the shape.

Parameters: shape -- the shape

void AddDimSize(int64_t size)¶

Add a dimension at the back of the shape.

Parameters: size -- the size to be added

void SetDimSize(int index, int64_t size)¶

Set the size of the specified dimension.

Parameters

index -- the index of the specified dimension
size -- the size of the specified dimension

void SetPrimitiveType(const PrimitiveType &prim_type)¶

Set the data type.

Parameters: prim_type -- the data type

int64_t GetTupleSize() const¶

Get the element number for the TUPLE type.

Returns: 0 if it is not a TUPLE, else the element number for the TUPLE

std::vector<std::vector<int64_t>> GetTupleShapes() const¶

Get the shapes or the TUPLE type.

Returns: the shapes

std::vector<PrimitiveType> GetTuplePrimitiveTypes() const¶

Get the data types or the TUPLE type.

Returns: the data types

int64_t GetDimSize(int index) const¶

Get the size of the specified dimension.

Parameters: index -- the index of the specified dimension
Returns: the size of the specified dimension

int64_t GetRank() const¶

Get the rank.

Returns: the rank

int64_t GetSize() const¶

Get the total elemnent number.

Returns: the total elemnent number.

5.3. Attribute¶

std::ostream &operator<<(std::ostream &out, Attribute attr)¶: Feed the Attribute information to the specific output stream.

class topsgraph::Attribute¶

Public Functions

explicit Attribute(const char *value)¶

Constructor of an attribute of type string.

Parameters: value -- Incoming information data

explicit Attribute(bool value)¶

Constructor of an attribute of type bool.

Parameters: value -- Incoming information data

explicit Attribute(int32_t value)¶

Constructor of an attribute of type int32_t.

Parameters: value -- Incoming information data

explicit Attribute(int64_t value)¶

Constructor of an attribute of type int64_t.

Parameters: value -- Incoming information data

explicit Attribute(float value)¶

Constructor of an attribute of type float.

Parameters: value -- Incoming information data

explicit Attribute(double value)¶

Constructor of an attribute of type double.

Parameters: value -- Incoming information data

explicit Attribute(Type type, const void *data)¶

Constructor of an attribute of type Dense.

Parameters

type -- tensor type
data -- dense data pointer

const Type GetType() const¶

Get the attribute Type.

Returns: The attribute Type

void Print(std::ostream &out) const¶

Print the Attribute information to the specific output stream.

Parameters: out -- Output stream

void Dump() const¶: Print the Attribute information to stdout.

double GetValueAsDouble() const¶

Get the attribute value.

Returns: If it is float and double type, return value, otherwise return 0.0

int64_t GetValueAsInt() const¶

Get the attribute value.

Returns: If it is integer type, return value, otherwise return 0

bool GetValueAsBool() const¶

Get the attribute value.

Returns: If it is bool type, return value, otherwise return 0

const char *GetValueAsString() const¶

Get the attribute value.

Returns: If it is string type, return value, otherwise return nullptr.

std::shared_ptr<void> GetValueAsDataPtr() const¶

Get the attribute value.

Returns: If it is dense type, return value, otherwise return 0

5.4. Op¶

std::ostream &operator<<(std::ostream &out, Op op)¶: Feed the Attribute information to the specific output stream.

class topsgraph::Op¶

#include <hlir_builder_common.h>

This represents an instruction that has been enqueued using the Builder.

Public Functions

bool IsInited() const¶

Judge if this op has been initialized.

Returns

Judge result

Returns

true -- a valid op which has been inited.
false -- otherwise

bool IsArgument() const¶

Judge if it's an argument.

Returns

Judge result

Returns

true -- an argument which corresponding to a mlir::BlockArgument.
false -- otherwise

bool IsConstant() const¶

Judge if it's a constant op.

Returns

Judge result

Returns

true -- constant op
false -- otherwise

bool IsDynamic() const¶

Judge if it's dynamic shape.

Returns

Judge result

Returns

true -- dynamic shape
false -- static shape

const char *GetName() const¶

Get the op name.

Returns: The op name, which is a constant char literal.

Builder GetBuilder() const¶

Get the used Builder object.

Returns: The Builder object.

const Type GetType() const¶

Get the op Type.

Returns: The op Type

const Attribute GetAttribute(const char *name) const¶

get op attribute parameters info.

Parameters: name -- attribute name
Returns: attribute object

template<typename T> inline T GetValue() const¶

get constant op's value.

basic types: bool, int8, int16, ...
dense elements: std::vector<basic types>

Parameters: T -- const value type. support below types:

void Print(std::ostream &out) const¶

Print the Op IR to the specific output stream.

Parameters

out -- Output stream
flags -- Print flag

void Dump() const¶: Dump op info.

std::shared_ptr<Impl> GetImpl() const¶: pimpl, only used for internal developers.

void SetAttribute(const char *name, const Attribute &value, const char *mode)¶

Set op attribute parameters.

Parameters

name -- attribute name
value -- attribute object
mode -- mode of setting attribute of accessories. Options: "": do not set the attribute for accessories. "copy": copy the attribute for accessories. "derive": derive the attribute for accessories with postfix "_index", where index starts with 0. Only work for string.

void SetAttribute(const char *name, const std::vector<Attribute> &value)¶

Set op array attribute parameters.

Parameters

name -- attribute name
value -- vector of attribute object

bool IsUnknownRank() const¶

Judge if its shape is unknown rank.

Returns

Judge result

Returns

true -- unknown rank
false -- otherwise

void SetMaxMinShape(const std::vector<int64_t> &max_shape, const std::vector<int64_t> &min_shape)¶

Set Dynamic op's max_shape_dim, min_shape_dim attributes. If the input dim is not unknown dim, the value in max_shape_dim and min_shape_dim is same, and equal to the real dim.

For tuple op, max_shape/min_shape including all outputs.

Parameters

max_shape -- input's max_shape_dims.
min_shape -- input's min_shape_dims.

std::shared_ptr<void> GetConstDataPtr() const¶

Get data pointer of ConstOp.

Returns: The data pointer of ConstOp, nullptr for other Ops

template<typename T> inline std::vector<T> GetConstData() const¶

Get vector contains data of ConstOp. Error occurs for other Ops.

Returns: A vector contains data of ConstOp

5.5. Structs¶

class ConvDimensionNumbers¶: #include <hlir_builder_structs.h>

Structure of dimension information for conv op.

class DimensionsLayout¶: #include <hlir_builder_structs.h>

Structure of dimension information for layout.

class DotDimensionNumbers¶: #include <hlir_builder_structs.h>

Structure of dimension information for dot product.

class GatherDimensionNumbers¶: #include <hlir_builder_structs.h>

Structure of dimension information for gather.

class ScatterDimensionNumbers¶: #include <hlir_builder_structs.h>

Structure of dimension information for scatter.

5.6. TopsGraphExecutor¶

class topsgraph::execute::TopsGraphExecutor¶

#include <topsgraph_executor.h>

A class for executing graph operations on a specific device.

This class provides a set of methods to manage device resources, load graph modules, allocate tensors, set inputs and outputs, compile and execute graphs, and register memory allocation and deallocation functions.

Public Functions

TopsGraphExecutor()¶

Default constructor.

Initializes a new instance of the TopsGraphExecutor class.

~TopsGraphExecutor()¶

Destructor.

Cleans up the resources associated with the TopsGraphExecutor instance.

topsgraphError_t SetDeviceMallocFunc(topsgraph::runtime::TopsGraphMallocFunc func)¶

Set a custom device memory allocation function, defaultly cal tops runtime.

Parameters: func -- The memory allocation function to register.
Returns: #Success

topsgraphError_t SetDeviceMallocAsyncFunc(topsgraph::runtime::TopsGraphMallocAsyncFunc func)¶

Set a custom device asynchronous memory allocation function, defaultly cal tops runtime.

Parameters: func -- The asynchronous memory allocation function to register.
Returns: #Success

topsgraphError_t SetDeviceFreeFunc(topsgraph::runtime::TopsGraphFreeFunc func)¶

Set a custom device memory deallocation function, defaultly cal tops runtime.

Parameters: func -- The memory deallocation function to register.
Returns: #Success

topsgraphError_t SetDeviceFreeAsyncFunc(topsgraph::runtime::TopsGraphFreeAsyncFunc func)¶

Set a custom asynchronous device memory deallocation function, defaultly cal tops runtime.

Parameters: func -- The asynchronous memory deallocation function to register.
Returns: #Success

topsgraphError_t GetDeviceID(size_t &id) const¶

Get the ID of the device on which the executor operates.

Returns: #Success

topsgraphError_t SetDeviceID(size_t id) const¶

Set the ID of the device on which the executor will operate.

Returns: #Success

topsgraph::runtime::GraphStream CreateStream() const¶

Create a new graph stream.

Returns: A new topsgraph::runtime::GraphStream object.

topsgraph::runtime::GraphStream CreateStream(const topsStream_t stream) const¶

Create a new graph stream using an existing topsStream_t object.

Parameters: stream -- The existing topsStream_t object to use.
Returns: A new topsgraph::runtime::GraphStream object.

topsgraphError_t DeallocateStream(topsgraph::runtime::GraphStream &stream) const¶

Deallocate a graph stream.

Parameters: stream -- The topsgraph::runtime::GraphStream object to deallocate.
Returns: #Success, #RuntimeError

topsgraphError_t LoadModuleStr(const char *module_str)¶

Load a graph module from a string.

Parameters: module_str -- The string containing the graph module.
Returns: #Success, #ErrorUnknown

int GetInputNums() const¶

Get the number of input tensors of the graph.

Returns: The number of input tensors as an integer.

int GetOutputNums() const¶

Get the number of output tensors of the graph.

Returns: The number of output tensors as an integer.

Type GetInputType(size_t index) const¶

Get the data type of an input tensor at a specific index.

Parameters: index -- The index of the input tensor.
Returns: The data type of the input tensor.

std::vector<Type> GetInputTypes() const¶

Get the data types of all input tensors.

Returns: A vector containing the data types of all input tensors.

Type GetOutputType(size_t index) const¶

Get the data type of an output tensor at a specific index.

Parameters: index -- The index of the output tensor.
Returns: The data type of the output tensor.

std::vector<Type> GetOutputTypes() const¶

Get the data types of all output tensors.

Returns: A vector containing the data types of all output tensors.

Tensor AllocTensor(const std::vector<std::size_t> &shape, const Type type = UNKNOWN) const¶

Allocate a tensor with the specified shape and data type.

Parameters

shape -- The shape of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).

Returns

A newly allocated Tensor object.

Tensor AllocTensor(const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const Type type = UNKNOWN) const¶

Allocate a tensor with the specified shape, stride, and data type.

Parameters

shape -- The shape of the tensor as a vector of sizes.
stride -- The stride of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).

Returns

A newly allocated Tensor object.

Tensor AllocTensorAsync(const std::vector<std::size_t> &shape, const Type type, topsgraph::runtime::GraphStream &stream, const uint64_t flag = 0) const¶

Asynchronously allocate a tensor with the specified shape, data type, and stream.

Parameters

shape -- The shape of the tensor as a vector of sizes.
type -- The data type of the tensor.
stream -- The graph stream to use for the asynchronous operation.
flag -- A flag for the allocation operation (default is 0).

Returns

A newly allocated Tensor object.

Tensor AllocTensorAsync(const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const Type type, topsgraph::runtime::GraphStream &stream, const uint64_t flag = 0) const¶

Asynchronously allocate a tensor with the specified shape, stride, data type, and stream.

Parameters

shape -- The shape of the tensor as a vector of sizes.
stride -- The stride of the tensor as a vector of sizes.
type -- The data type of the tensor.
stream -- The graph stream to use for the asynchronous operation.
flag -- A flag for the allocation operation (default is 0).

Returns

A newly allocated Tensor object.

Tensor AllocHostTensor(const std::vector<std::size_t> &shape, const Type type = UNKNOWN) const¶

Allocate a host tensor with the specified shape and data type.

Parameters

shape -- The shape of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).

Returns

A newly allocated host Tensor object.

Tensor AllocHostTensor(const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const Type type = UNKNOWN) const¶

Allocate a host tensor with the specified shape, stride, and data type.

Parameters

shape -- The shape of the tensor as a vector of sizes.
stride -- The stride of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).

Returns

A newly allocated host Tensor object.

topsgraphError_t CopyTensor(Tensor &from, Tensor &to) const¶

Copy data from one tensor to another.

Parameters

from -- The source tensor.
to -- The destination tensor.

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t CopyTensorAsync(Tensor &from, Tensor &to, topsgraph::runtime::GraphStream &stream) const¶

Asynchronously copy data from one tensor to another using a graph stream.

Parameters

from -- The source tensor.
to -- The destination tensor.
stream -- The graph stream to use for the asynchronous operation.

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

Tensor CreateTensor(void *data, const std::vector<std::size_t> &shape, const Type type = UNKNOWN, const bool is_host = false) const¶

Create a tensor from existing data with the specified shape and data type.

Parameters

data -- A pointer to the existing data.
shape -- The shape of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).
is_host -- A boolean indicating whether the data is on the host (default is false).

Returns

A newly created Tensor object.

Tensor CreateTensor(void *data, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const Type type = UNKNOWN, const bool is_host = false) const¶

Create a tensor from existing data with the specified shape, stride, and data type.

Parameters

data -- A pointer to the existing data.
shape -- The shape of the tensor as a vector of sizes.
stride -- The stride of the tensor as a vector of sizes.
type -- The data type of the tensor (default is UNKNOWN).
is_host -- A boolean indicating whether the data is on the host (default is false).

Returns

A newly created Tensor object.

topsgraphError_t DeallocTensor(const Tensor &tensor) const¶

Deallocate a tensor.

Parameters: tensor -- The tensor to deallocate.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t DeallocTensorAsync(const Tensor &tensor, topsgraph::runtime::GraphStream &stream) const¶

Asynchronously deallocate a tensor using a graph stream.

Parameters

tensor -- The tensor to deallocate.
stream -- The graph stream to use for the asynchronous operation.

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetInput(const size_t index, const Tensor &tensor)¶

Set an input tensor at a specific index.

Parameters

index -- The index of the input tensor.
tensor -- The tensor to set as the input.

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutput(const size_t index, const Tensor &tensor)¶

Set an output tensor at a specific index.

Parameters

index -- The index of the output tensor.
tensor -- The tensor to set as the output.

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutput(const size_t index, const std::vector<std::size_t> &shape, const bool is_host = false)¶

Set an output tensor at a specific index with the specified shape.

Parameters

index -- The index of the output tensor.
shape -- The shape of the output tensor as a vector of sizes.
is_host -- A boolean indicating whether the tensor is on the host (default is false).

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutput(const size_t index, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const bool is_host = false)¶

Set an output tensor at a specific index with the specified shape and stride.

Parameters

index -- The index of the output tensor.
shape -- The shape of the output tensor as a vector of sizes.
stride -- The stride of the output tensor as a vector of sizes.
is_host -- A boolean indicating whether the tensor is on the host (default is false).

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetInputs(const std::vector<Tensor> &tensors)¶

Set multiple input tensors.

Parameters: tensors -- A vector of tensors to set as inputs.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutputs(const std::vector<Tensor> &tensors)¶

Set multiple output tensors.

Parameters: tensors -- A vector of tensors to set as outputs.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutputs(const std::vector<std::vector<std::size_t>> &shapes, const bool is_host = false)¶

Set multiple output tensors with the specified shapes.

Parameters

shapes -- A vector of shapes for the output tensors.
is_host -- A boolean indicating whether the tensors are on the host (default is false).

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t SetOutputs(const std::vector<std::vector<std::size_t>> &shapes, const std::vector<std::vector<std::size_t>> &strides, const bool is_host = false)¶

Set multiple output tensors with the specified shapes and strides.

Parameters

shapes -- A vector of shapes for the output tensors.
strides -- A vector of strides for the output tensors.
is_host -- A boolean indicating whether the tensors are on the host (default is false).

Returns

#Success, #ErrorUnknown, #RuntimeError, #InvalidValue

bool IsDynamic() const¶

Check if the graph is dynamic.

Returns: true if the graph is dynamic, false otherwise.

bool IsReady() const¶

Check if the graph is ready for execution.

Returns: true if the graph is ready, false otherwise.

topsgraphError_t Compile(const std::vector<const char*> &options)¶

Compile the graph with the specified options.

Parameters: options -- A vector of compilation options as strings.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t Execute(topsgraph::runtime::GraphStream &stream)¶

Execute the graph using a specified graph stream.

Parameters: stream -- The graph stream to use for the execution.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

bool Execute()¶: Execute the graph not use stream. this api is for torch_gcu and will remove.

topsgraphError_t SaveExecute(const std::string &save_file)¶

Save execute production.

Parameters: save_dir -- Save path.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

topsgraphError_t LoadExecute(const std::string &load_file)¶

Load execute production.

Parameters: save_dir -- Load path.
Returns: #Success, #ErrorUnknown, #RuntimeError, #InvalidValue

Tensor GetInput(const size_t index) const¶

Get an input tensor at a specific index.

Parameters: index -- The index of the input tensor.
Returns: The input tensor at the specified index.

Tensor GetResult(const size_t index) const¶

Get the result tensor at a specific index.

Parameters: index -- The index of the result tensor.
Returns: The result tensor at the specified index.

std::vector<Tensor> GetInputs() const¶

Get all input tensors.

Returns: A vector containing all input tensors.

std::vector<Tensor> GetResults() const¶

Get all result tensors.

Returns: A vector containing all result tensors.

std::vector<std::vector<std::size_t>> InferShape(topsgraph::runtime::GraphStream &stream) const¶

Infer the shapes of the tensors in the graph.

Returns: A vector of vectors containing the inferred shapes of the tensors.

Private Functions

topsgraphError_t SetInputs(const std::vector<std::vector<std::size_t>> &shapes, const bool is_host = false)¶

Set multiple input tensors with the specified shapes.

Note

This API is currently unavailable.

Parameters

shapes -- A vector of shapes for the input tensors.
is_host -- A boolean indicating whether the tensors are on the host (default is false).

Returns

true if all inputs are set successfully, false otherwise.

topsgraphError_t SetInputs(const std::vector<std::vector<std::size_t>> &shapes, const std::vector<std::vector<std::size_t>> &strides, const bool is_host = false)¶

Set multiple input tensors with the specified shapes and strides.

Note

This API is currently unavailable.

Parameters

shapes -- A vector of shapes for the input tensors.
strides -- A vector of strides for the input tensors.
is_host -- A boolean indicating whether the tensors are on the host (default is false).

Returns

true if all inputs are set successfully, false otherwise.

topsgraphError_t SetInput(const size_t index, const std::vector<std::size_t> &shape, const bool is_host = false)¶

Set an input tensor at a specific index with the specified shape.

Note

This API is currently unavailable.

Parameters

index -- The index of the input tensor.
shape -- The shape of the input tensor as a vector of sizes.
is_host -- A boolean indicating whether the tensor is on the host (default is false).

Returns

true if the input is set successfully, false otherwise.

topsgraphError_t SetInput(const size_t index, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &stride, const bool is_host = false)¶

Set an input tensor at a specific index with the specified shape and stride.

Note

This API is currently unavailable.

Parameters

index -- The index of the input tensor.
shape -- The shape of the input tensor as a vector of sizes.
stride -- The stride of the input tensor as a vector of sizes.
is_host -- A boolean indicating whether the tensor is on the host (default is false).

Returns

true if the input is set successfully, false otherwise.

topsgraphError_t LoadModuleFile(const char *module_path)¶

Load a graph module from a file.

Note

This API is currently unavailable.

Parameters: module_path -- The file containing the graph module.
Returns: #Success, #ErrorUnknown

std::string GetDeviceArch() const¶

Get the Device Arch info.

Note

This API is currently unavailable.

Returns: Device Arich info.

5.7. GraphStream¶

class topsgraph::runtime::GraphStream¶

#include <topsgraph_stream.h>

A class for managing graph streams.

Public Functions

GraphStream()¶: Default constructor. Initializes the GraphStream with the default topsStream.

GraphStream(const topsStream_t stream)¶

Constructor that takes a specific topsStream_t object.

Parameters: stream -- The topsStream_t object to initialize the GraphStream with.

GraphStream(const GraphStream&) = delete¶: Deleted copy constructor to prevent copying of GraphStream objects.

GraphStream &operator=(const GraphStream&) = delete¶: Deleted assignment operator to prevent assignment between GraphStream objects.

~GraphStream()¶: Destructor. Cleans up the resources associated with the GraphStream.

topsgraphError_t SwitchToDefaultStream()¶

Switches the current stream to the default stream. Current stream will be destroy.

Returns: #Success

topsgraphError_t SwitchToNewStream(const topsStream_t new_stream)¶

Switches the current stream to a new specified stream.

Parameters: new_stream -- The new topsStream_t object to switch to.
Returns: #Success

topsgraphError_t SwitchToNewStream()¶

Switches the current stream to a new specified stream.

Parameters: new_stream -- The new topsStream_t object to switch to.
Returns: #Success

topsgraphError_t Synchronize()¶

synchronize the stream.

Returns: #Success

topsStream_t *ReleaseStream()¶

Releases the managed stream and returns a pointer to it. After this call, the GraphStream no longer manages the stream.

Returns: A pointer to the released topsStream_t object.

topsStream_t *Stream()¶

Returns a pointer to the currently managed stream.

Returns: A pointer to the topsStream_t object.

5.8. GraphTensor¶

enum Type¶

An enumeration class that represents a data type.

Values:

enumerator UNKNOWN¶

enumerator INT1¶

enumerator INT8¶

enumerator INT16¶

enumerator INT32¶

enumerator INT64¶

enumerator FLOAT16¶

enumerator FLOAT32¶

enumerator DOUBLE64¶

enumerator BFLOAT16¶

enumerator UINT8¶

enumerator UINT16¶

enumerator UINT32¶

enumerator UINT64¶

class topsgraph::execute::Tensor¶

#include <topsgraph_tensor.h>

Tensor class representing multidimensional array data with host/device memory management.

Note

Friend classes TopsGraphExecutorImpl and Engine can directly access private members for graph execution optimization

Public Functions

Tensor(void *data, std::vector<size_t> &sizes, std::vector<size_t> &strides, size_t offset = 0, Type type = Type::UNKNOWN)¶

Constructs tensor from existing memory.

Parameters

data -- [in] Pointer to data (host or device memory)
sizes -- [in] Dimension sizes vector
strides -- [in] Dimension strides vector
offset -- [in] Data offset in bytes (default=0)
type -- [in] Data type identifier (default=UNKNOWN)

Tensor(size_t base, size_t data, size_t type, size_t offset, std::vector<size_t> &sizes, std::vector<size_t> &strides)¶

Torch-specific API constructor.

Parameters

base -- [in] Base memory address
data -- [in] Data pointer
type -- [in] Data type identifier
offset -- [in] Data offset in bytes
sizes -- [in] Dimension sizes vector
strides -- [in] Dimension strides vector

void DebugPrint(size_t num = 0) const¶

Print tensor information for Debug.

Parameters: num -- [in] Number of elements to print (0 means all)

size_t GetRank() const¶

Gets the number of dimensions (rank) of the tensor.

Returns: The rank as a size_t value (0 for scalar-like tensors)

Type GetType() const¶

Retrieves the data type identifier of the tensor.

Returns: Current data type from the Type enumeration

topsgraphError_t SetType(Type type)¶

Sets the data type identifier for the tensor.

Parameters: type -- [in] The new data type to set (from Type enum)
Returns: #Success

size_t GetOffSet() const¶

Gets the byte offset from base memory address.

Returns: Current offset value in bytes

topsgraphError_t SetOffSet(size_t offset)¶

Sets the byte offset from base memory address.

Warning

Incorrect offset may cause data access violations

Parameters: offset -- [in] New offset value in bytes (must be within allocated memory range)
Returns: #Success

void *GetBase() const¶

Gets the base memory address of the underlying storage.

Note

This may include padding areas before the actual data

Returns: Raw pointer to the base memory block

topsgraphError_t SetBase(void *date_ptr)¶

Sets the base memory address for the tensor storage.

Warning

Changing base address invalidates existing data pointers

Parameters: data_ptr -- [in] Pointer to the new base memory block
Returns: #Success

void *GetData() const¶

Gets the effective data starting address (base + offset)

Returns: Pointer to the first valid data element

topsgraphError_t SetData(void *date_ptr)¶

Directly sets the effective data pointer (base + offset)

Parameters: data_ptr -- [in] New data pointer address
Returns: #Success

std::vector<size_t> GetShape() const¶

Gets a copy of the dimension sizes (shape)

Returns: std::vector<size_t> containing all dimension sizes

topsgraphError_t SetShape(const std::vector<size_t> &shape)¶

Sets the tensor shape via move semantics.

Parameters: shape -- [in] New dimension sizes vector (will be moved)
Returns: #Success

topsgraphError_t SetShape(std::vector<size_t> &&shape)¶

Sets the tensor shape via move semantics.

Parameters: shape -- [in] New dimension sizes vector (will be moved)
Returns: #Success

std::vector<size_t> GetStrides() const¶

Gets a copy of the stride values (bytes per dimension step)

Returns: std::vector<size_t> containing stride values

topsgraphError_t SetStrides(std::vector<size_t> &&stride)¶

Sets strides via move semantics.

Warning

Don't check if the stride is correct

Parameters: stride -- [in] New stride vector (will be moved)
Returns: #Success

topsgraphError_t SetStrides(const std::vector<size_t> &stride)¶

Sets strides via copy semantics.

Parameters: stride -- [in] New stride vector (copied)
Returns: #Success

topsgraphError_t SetIsHost(const bool is_host)¶

Sets the memory location flag (host/device)

Note

Actual memory movement may be deferred

Parameters: is_host -- [in] true for host memory, false for device
Returns: #Success

bool IsHost() const¶

Checks if tensor resides in host memory.

5.9. GraphMemory¶

typedef topsError_t (*TopsGraphMallocFunc)(void**, size_t)¶

Defines a type alias for a function that is used to allocate memory.

This function takes a pointer to a pointer (void**) to store the allocated memory address, and a size_t indicating the number of bytes to allocate.

typedef topsError_t (*TopsGraphMallocHostFunc)(void**, size_t, unsigned int)¶

Defines a type alias for a function that is used to allocate memory.

This function takes a pointer to a pointer (void**) to store the allocated memory address, and a size_t indicating the number of bytes to allocate.

typedef topsError_t (*TopsGraphMallocAsyncFunc)(void**, size_t, topsStream_t, uint64_t)¶

Defines a type alias for an asynchronous memory allocation function.

This asynchronous function takes a pointer to a pointer (void**) to store the allocated memory address, a size_t representing the number of bytes to allocate, a reference to a topsgraph::runtime::GraphStream object which is likely related to the execution context of the graph, and a uint64_t flag which can be used to specify additional allocation options.

typedef topsError_t (*TopsGraphFreeFunc)(void*)¶

Defines a type alias for a function that is used to free previously allocated memory.

This function takes a pointer (void*) to the memory that needs to be freed.

typedef topsError_t (*TopsGraphFreeHostFunc)(void*)¶

Defines a type alias for a function that is used to free previously allocated memory.

This function takes a pointer (void*) to the memory that needs to be freed.

typedef topsError_t (*TopsGraphFreeAsyncFunc)(void*, topsStream_t)¶

Defines a type alias for an asynchronous memory freeing function.

This asynchronous function takes a pointer (void*) to the memory that needs to be freed, and a reference to a topsgraph::runtime::GraphStream object which is likely related to the execution context of the graph.

topsgraphError_t RegisterMallocFunc(TopsGraphMallocFunc func)¶

Register a custom memory allocation function.

Parameters: func -- The memory allocation function to register.
Returns: #Success

topsgraphError_t RegisterMallocAsyncFunc(TopsGraphMallocAsyncFunc func)¶

Register a custom asynchronous memory allocation function.

Parameters: func -- The asynchronous memory allocation function to register.
Returns: #Success

topsgraphError_t RegisterFreeFunc(TopsGraphFreeFunc func)¶

Register a custom memory deallocation function.

Parameters: func -- The memory deallocation function to register.
Returns: #Success

topsgraphError_t RegisterFreeAsyncFunc(TopsGraphFreeAsyncFunc func)¶

Register a custom asynchronous memory deallocation function.

Parameters: func -- The asynchronous memory deallocation function to register.
Returns: #Success

topsError_t TopsGraphHostMalloc(void **p, size_t bytes, unsigned int flag = 0)¶

Allocates memory on the host.

This function attempts to allocate the specified number of bytes on the host. The address of the allocated memory is stored in the location pointed to by 'p'.

Parameters

p -- A pointer to a pointer where the address of the allocated memory will be stored.
bytes -- The number of bytes to allocate.
flag -- flag about how to allocate.

Returns

#topsSuccess

topsError_t TopsGraphHostFree(void *p)¶

Frees memory allocated on the host.

This function frees the memory pointed to by 'p' that was previously allocated on the host.

Parameters: p -- A pointer to the memory to be freed.
Returns: #topsSuccess

topsError_t TopsGraphMalloc(void **p, size_t bytes)¶

Allocates memory.

This function attempts to allocate the specified number of bytes. The address of the allocated memory is stored in the location pointed to by 'p'.

Parameters

p -- A pointer to a pointer where the address of the allocated memory will be stored.
bytes -- The number of bytes to allocate.

Returns

#topsSuccess

topsError_t TopsGraphFree(void *p)¶

Frees previously allocated memory.

This function frees the memory pointed to by 'p' that was previously allocated.

Parameters: p -- A pointer to the memory to be freed.
Returns: #topsSuccess

topsError_t TopsGraphMallocAsync(void **p, size_t bytes, topsgraph::runtime::GraphStream &stream, uint64_t flag)¶

Asynchronously allocates memory.

This function attempts to asynchronously allocate the specified number of bytes. The address of the allocated memory is stored in the location pointed to by 'p'.

Parameters

p -- A pointer to a pointer where the address of the allocated memory will be stored.
bytes -- The number of bytes to allocate.
stream -- A reference to a topsgraph::runtime::GraphStream object representing the execution context.
flag -- A uint64_t flag that can be used to specify additional allocation options.

Returns

#topsSuccess

topsError_t TopsGraphFreeAsync(void *p, topsgraph::runtime::GraphStream &stream)¶

Asynchronously frees previously allocated memory.

This function asynchronously frees the memory pointed to by 'p' that was previously allocated.

Parameters

p -- A pointer to the memory to be freed.
stream -- A reference to a topsgraph::runtime::GraphStream object representing the execution context.

Returns

#topsSuccess