Formation of Network

One operator instance can take inputs to outputs. If the output tensor continues to be used as the input tensor of another operator instance, the two operator instances are connected together, forming a computational graph. In deep learning, the computational graph is called a network.

A typical network is like a DAG (Directed Acyclic Graph), which means the graph has direction and no cycle. The input tensors (generated by manual or helper functions) are the source nodes of the network, and the output tensors (generated by operators, but never used as input tensors of other operators) are the sink nodes of the network.