Saw it in
void TF_SetAttrTensor(TF_OperationDescription* desc, const char* attr_name,
TF_Tensor* value, TF_Status* status) {
Tensor t;
status->status = TF_TensorToTensor(value, &t);
if (status->status.ok()) desc->node_builder.Attr(attr_name, t);
}
So what it means to build a node.
TF_Operation * Const(TF_Graph * graph, TF_Status * status, TF_Tensor * tensor, const char * name) {
TF_OperationDescription * desc = TF_NewOperation(graph, "Const", name);
TF_SetAttrTensor(desc, "value", tensor, status);
TF_SetAttrType(desc, "dtype", TF_TensorType(tensor));
return TF_FinishOperation(desc, status);
}
That's how to build a node use C interface.
C interface is easier to understand, plus c++ interface code is generated when compile, so it's difficult to change any of it.
c_api_h:
// Operation being built. The underlying graph must outlive this.
typedef struct TF_OperationDescription TF_OperationDescription;
// Operation that has been added to the graph. Valid until the graph is
// deleted -- in particular adding a new operation to the graph does not
// invalidate old TF_Operation* pointers.
typedef struct TF_Operation TF_Operation;
// Represents a specific input of an operation.
typedef struct TF_Input {
TF_Operation* oper;
int index; // The index of the input within oper.
} TF_Input;
// Represents a specific output of an operation.
typedef struct TF_Output {
TF_Operation* oper;
int index; // The index of the output within oper.
} TF_Output;
TF_Input is equivalent to TF_Output, they do use it interchangeably.
c_api_internal.h:
struct TF_Operation {
tensorflow::Node node;
};
So the TF_Operation pointer just a pointer that points to the new node just created.
struct TF_OperationDescription {
TF_OperationDescription(TF_Graph* g, const char* op_type,
const char* node_name)
: node_builder(node_name, op_type, g->graph.op_registry()), graph(g) {}
tensorflow::NodeBuilder node_builder;
TF_Graph* graph;
std::set<tensorflow::string> colocation_constraints;
};
TF_OperationDescription more like a class, to help building the node.
tensorflow::NodeBuilder is a class, defined in tensorflow/core/graph/node_builder.h
// This is a helper for creating a Node and adding it to a Graph.
// Internally, it uses a NodeDefBuilder to automatically set attrs
// that can be inferred from the inputs, and use default values
// (where they exist) for unspecified attrs. Example usage:
//
// Node* node;
// Status status = NodeBuilder(node_name, op_name)
// .Input(...)
// .Attr(...)
// .Finalize(&graph, &node);
// if (!status.ok()) return status;
// // Use node here.
class NodeBuilder {
public:
// For specifying the output of a Node to provide to one of the Input()
// functions below. It supports both regular inputs (where you are
// connecting to an existing Node*), and inputs from outside the graph
// (or haven't been added to the graph yet, like back edges, where
// you don't have a Node*). Both types can be mixed, e.g. in an
// ArraySlice.
struct NodeOut {
// For referencing an existing Node.
NodeOut(Node* n, int32 i = 0);
// For referencing Nodes not in the graph being built. It is
// useful when preparing a graph for ExtendSession or creating a
// back edge to a node that hasn't been added to the graph yet,
// but will be.
NodeOut(StringPiece name, int32 i, DataType t);
// Default constructor for std::vector<NodeOut>.
NodeOut();
Node* node;
// error is set to true if:
// * the NodeOut was default constructed and never overwritten,
// * a nullptr Node* was passed to the NodeOut constructor, or
// * an out-of-range index was passed to the NodeOut constructor.
bool error;
string name;
int32 index;
DataType dt;
};
// Specify the name and the Op (either via an OpDef or the name of
// the Op plus a registry) for the Node. Other fields are
// specified by calling the methods below.
// REQUIRES: The OpDef must satisfy ValidateOpDef().
NodeBuilder(StringPiece name, StringPiece op_name,
const OpRegistryInterface* op_registry = OpRegistry::Global());
NodeBuilder(StringPiece name, const OpDef* op_def);
// Create a NodeBuilder from an existing NodeDefBuilder.
NodeBuilder(const NodeDefBuilder& def_builder);
// You must call one Input() function per input_arg in the Op,
// *and in the same order as the input_args appear in the OpDef.*
// For inputs that take a single tensor.
NodeBuilder& Input(Node* src_node, int src_index = 0);
NodeBuilder& Input(NodeOut src);
// For inputs that take a list of tensors.
NodeBuilder& Input(gtl::ArraySlice<NodeOut> src_list);
// Require that this node run after src_node(s).
NodeBuilder& ControlInput(Node* src_node);
NodeBuilder& ControlInputs(gtl::ArraySlice<Node*> src_nodes);
// Sets the "requested device spec" in the NodeDef (not the
// "assigned device" in the Node).
NodeBuilder& Device(StringPiece device_spec);
// Set the value of an attr. attr_name must match the name of one of
// attrs defined by the Op, and value must have the corresponding type
// (see SetAttrValue() in ../framework/attr_value_util.h for legal
// types for value). Note that attrs will be set automatically if
// they can be determined by the inputs.
template <class T>
NodeBuilder& Attr(StringPiece attr_name, T&& value);
template <class T>
NodeBuilder& Attr(StringPiece attr_name, std::initializer_list<T> value);
// Validates the described node and adds it to *graph, adding edges
// for all (non-back) inputs. If created_node is not nullptr,
// *created_node will be set to the new node (or nullptr on error).
Status Finalize(Graph* graph, Node** created_node) const;
// Accessors for the values set in the constructor.
const string& node_name() const { return def_builder_.node_name(); }
const OpDef& op_def() const { return def_builder_.op_def(); }
private:
static DataType SafeGetOutput(const Node* node, int i, bool* error) {
if (node != nullptr && i >= 0 && i < node->num_outputs()) {
*error = false;
return node->output_type(i);
} else {
*error = true;
return DT_FLOAT;
}
}
// If SafeGetOutput indicates a range error, add it to errors_.
void AddIndexError(const Node* node, int i);
// Set *dt and returns true if i is in range. Combines
// SafeGetOutput() and AddIndexError().
bool GetOutputType(const Node* node, int i, DataType* dt);
NodeDefBuilder def_builder_;
std::vector<NodeOut> inputs_;
std::vector<Node*> control_inputs_;
std::vector<string> errors_;
};
Where NodeDefBuilder is defined in tensorflow/core/framework/node_def_builder.h
// This is a helper for creating a NodeDef. Automatically sets attrs
// that can be inferred from the inputs, and uses default values
// (where they exist) for unspecified attrs. Example usage:
//
// NodeDef node_def;
// Status status = NodeDefBuilder(node_name, op_name)
// .Input(...)
// .Attr(...)
// .Finalize(&node_def);
// if (!status.ok()) return status;
// // Use node_def here.
class NodeDefBuilder {
public:
// To specify an output to be consumed by one of the Input() methods below.
struct NodeOut {
NodeOut(StringPiece n, int i, DataType dt);
NodeOut(); // uninitialized, call Reset() before use.
void Reset(StringPiece n, int i, DataType dt);
string node;
int index;
DataType data_type;
};
// Specify the name and the Op (either via an OpDef or the name of
// the Op plus a registry) for the NodeDef. Other fields are
// specified by calling the methods below.
// REQUIRES: The OpDef must satisfy ValidateOpDef().
NodeDefBuilder(StringPiece name, StringPiece op_name,
const OpRegistryInterface* op_registry = OpRegistry::Global());
// REQUIRES: in addition, *op_def must outlive *this.
NodeDefBuilder(StringPiece name, const OpDef* op_def);
...
const OpDef* op_def_;
NodeDef node_def_;
int inputs_specified_;
std::vector<string> control_inputs_;
std::vector<string> errors_;
};
We can tell that to build a node, we need to file a NodeDef structure, and probably link it to the graph.
Then what's the differences be Node and NodeDef?
First of all, the definition of class Node is easy to find. It's located in tensorflow/core/graph/graph.h
class Node {
public:
string DebugString() const;
int id() const { return id_; }
int cost_id() const { return cost_id_; }
const string& name() const;
const string& type_string() const;
// def() provides the NodeDef the user supplied, but the specifics
// of this Node may have changed due to placement, optimization, etc.
// In particular:
// * def().name() will match name();
// * def().op() will match type_string() and op_def().name();
// * def().input() is not reliable, use "in_edges()" below instead;
// * def().device() is the "user's requested device" and may not match
// the actual assigned device, see assigned_device_name() below;
// * def().attr() is authoritative.
// TODO(irving): Replace with NodeInfo.
const NodeDef& def() const;
const OpDef& op_def() const;
// input and output types
int32 num_inputs() const;
DataType input_type(int32 i) const;
const DataTypeVector& input_types() const;
int32 num_outputs() const;
DataType output_type(int32 o) const;
const DataTypeVector& output_types() const;
// The device requested by the user. For the actual assigned device,
// use assigned_device_name() below.
....
int id_; // -1 until Initialize() is called
int cost_id_; // -1 if there is no corresponding cost accounting node
NodeClass class_;
EdgeSet in_edges_;
EdgeSet out_edges_;
// NOTE(skyewm): inheriting from core::RefCounted may have a slight
// performance benefit over using shared_ptr, at the cost of manual ref
// counting
std::shared_ptr<NodeProperties> props_;
// Index within Graph::device_names_ of the name of device assigned
// to perform this computation.
int assigned_device_name_index_;
// A back-pointer to the Graph that owns this node. Currently, this exists
// solely to allow Node::[set_]assigned_device_name() to work. However, if all
// callers of Node::[set_]assigned_device_name() are modified to use the
// equivalent methods defined directly on Graph, then we can remove this
// field and reclaim that memory.
Graph* graph_;
// Set if this is an exit node of a while loop with an associated
// WhileContext. Otherwise null. (This is only set for exit nodes because
// they're the first nodes of a loop encountered while creating the gradient
// graph. Exit nodes that are part of while loop gradient graphs will not have
// this set.)
WhileContext* while_ctx_;
TF_DISALLOW_COPY_AND_ASSIGN(Node);
};
But NodeDef is not so easy to find.
It's defined by a non c/c++ code at tensorflow/core/framework/node_def.proto
message NodeDef {
// The name given to this operator. Used for naming inputs,
// logging, visualization, etc. Unique within a single GraphDef.
// Must match the regexp "[A-Za-z0-9.][A-Za-z0-9_./]*".
string name = 1;
// The operation name. There may be custom parameters in attrs.
// Op names starting with an underscore are reserved for internal use.
string op = 2;
// Each input is "node:src_output" with "node" being a string name and
// "src_output" indicating which output tensor to use from "node". If
// "src_output" is 0 the ":0" suffix can be omitted. Regular inputs
// may optionally be followed by control inputs that have the format
// "^node".
repeated string input = 3;
// A (possibly partial) specification for the device on which this
// node should be placed.
// The expected syntax for this string is as follows:
//
// DEVICE_SPEC ::= PARTIAL_SPEC
//
// PARTIAL_SPEC ::= ("/" CONSTRAINT) *
// CONSTRAINT ::= ("job:" JOB_NAME)
// | ("replica:" [1-9][0-9]*)
// | ("task:" [1-9][0-9]*)
// | ("device:" [A-Za-z]* ":" ([1-9][0-9]* | "*") )
//
// Valid values for this string include:
// * "/job:worker/replica:0/task:1/device:GPU:3" (full specification)
// * "/job:worker/device:GPU:3" (partial specification)
// * "" (no specification)
//
// If the constraints do not resolve to a single device (or if this
// field is empty or not present), the runtime will attempt to
// choose a device automatically.
string device = 4;
// Operation-specific graph-construction-time configuration.
// Note that this should include all attrs defined in the
// corresponding OpDef, including those with a value matching
// the default -- this allows the default to change and makes
// NodeDefs easier to interpret on their own. However, if
// an attr with a default is not specified in this list, the
// default will be used.
// The "names" (keys) must match the regexp "[a-z][a-z0-9_]+" (and
// one of the names from the corresponding OpDef's attr field).
// The values must have a type matching the corresponding OpDef
// attr's type field.
// TODO(josh11b): Add some examples here showing best practices.
map<string, AttrValue> attr = 5;
};
The comment in the header tells the differences.
// def() provides the NodeDef the user supplied, but the specifics
// of this Node may have changed due to placement, optimization, etc.
// In particular:
// * def().name() will match name();
// * def().op() will match type_string() and op_def().name();
// * def().input() is not reliable, use "in_edges()" below instead;
// * def().device() is the "user's requested device" and may not match
// the actual assigned device, see assigned_device_name() below;
// * def().attr() is authoritative.
NodeDef is only for user to establish a node, but after going through the framework, NodeDef structure is regulated as a Node.
Also, we see map<string, AttrValue> attr = 5;
Each node has several Attr, the node builder need to fill them. Every Attr has a name, and the name is associated with other object. For example, "value" with tensor, "shape" with tensor shape.
FIX: It's AttrValue, not Tensor but TensorProto
tensorflow/core/framework/node_def_util.cc
void AddNodeAttr(StringPiece name, const AttrValue& value, NodeDef* node_def) {
node_def->mutable_attr()->insert(
AttrValueMap::value_type(std::string(name), value));
}
