Login
- Released At: 12-05-2025
- Page Views:
- Downloads:
- Table of Contents
- Related Documents
-
H3C Switches
Comware 7 Telemetry Developers Guide
Copyright © 2023 New H3C Technologies Co., Ltd. All rights reserved.
No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of New H3C Technologies Co., Ltd.
Except for the trademarks of New H3C Technologies Co., Ltd., any trademarks that may be mentioned in this document are the property of their respective owners.
This document provides generic technical information, some of which might not be applicable to your products.
Preface
This guide is intended to guide developers to develop applications for collection of telemetry data from H3C switches for analytics and management purposes.
This preface includes the following topics about the documentation:
· Audience
Audience
This documentation is intended for:
· Developers familiar with gRPC and GPB encoding.
· Developers that use C++, Java, Python, or Go.
· Network administrators or maintainer familiar with XML and NETCONF.
Conventions
Command conventions
|
Convention |
Description |
|
Boldface |
Bold text represents commands and keywords that you enter literally as shown. |
|
Italic |
Italic text represents arguments that you replace with actual values. |
|
[ ] |
Square brackets enclose syntax choices (keywords or arguments) that are optional. |
|
{ x | y | ... } |
Braces enclose a set of required syntax choices separated by vertical bars, from which you select one. |
|
[ x | y | ... ] |
Square brackets enclose a set of optional syntax choices separated by vertical bars, from which you select one or none. |
|
{ x | y | ... } * |
Asterisk marked braces enclose a set of required syntax choices separated by vertical bars, from which you select a minimum of one. |
|
[ x | y | ... ] * |
Asterisk marked square brackets enclose optional syntax choices separated by vertical bars, from which you select one choice, multiple choices, or none. |
|
&<1-n> |
The argument or keyword and argument combination before the ampersand (&) sign can be entered 1 to n times. |
|
# |
A line that starts with a pound (#) sign is comments. |
Symbols
|
Convention |
Description |
|
|
An alert that calls attention to important information that if not understood or followed can result in personal injury. |
|
|
An alert that calls attention to important information that if not understood or followed can result in data loss, data corruption, or damage to hardware or software. |
|
|
An alert that calls attention to essential information. |
|
NOTE: |
An alert that contains additional or supplementary information. |
|
|
An alert that provides helpful information. |
Network topology icons
|
|
Represents a generic network device, such as a router, switch, or firewall. |
|
|
Represents a routing-capable device, such as a router or Layer 3 switch. |
|
|
Represents a generic switch, such as a Layer 2 or Layer 3 switch, or a router that supports Layer 2 forwarding and other Layer 2 features. |
|
|
Represents an access controller, a unified wired-WLAN module, or the access controller engine on a unified wired-WLAN switch. |
|
|
Represents an access point. |
|
|
Represents a wireless terminator unit. |
|
|
Represents a wireless terminator. |
|
|
Represents a mesh access point. |
|
|
Represents omnidirectional signals. |
|
|
Represents directional signals. |
|
|
Represents a security product, such as a firewall, UTM, multiservice security gateway, or load balancing device. |
|
|
Represents a security module, such as a firewall, load balancing, NetStream, SSL VPN, IPS, or ACG module. |
Examples provided in this document
Screenshots and examples provided in this documentation are for illustration only. They might differ depending on the hardware model, software version, and configuration. Examples in this document might use devices that differ from your device in hardware model, configuration, or software version. It is normal that the port numbers, sample output, screenshots, and other information in the examples differ from what you have on your device.
Documentation feedback
You can e-mail your comments about product documentation to [email protected].
We appreciate your comments.
Contents
Proto files used in different telemetry modes
Telemetry restrictions and guidelines
Configuring telemetry subscription settings on the device
Configuring a telemetry subscription in dial-in mode
Example: Configuring gRPC dial-in mode
Configuring a telemetry subscription in dial-out mode
Example: Configuring gRPC dial-out mode
Example: Developing a collector-side application (dial-in mode)
Expertise and skill requirements for developers
Setting up the development environment
Configuring telemetry subscription settings on the device
Developing the collector-side application in C++
Developing the collector-side application in Go
Developing the collector-side application in Python
Developing the collector-side application in Java
Example: Developing a collector-side application (dial-out mode)
Expertise and skill requirements
Setting up the development environment
Configuring telemetry subscription settings on the device
Developing the collector-side application in C++
Developing the collector-side application in Go
Developing the collector-side application in Python
Developing the collector-side application in Java
Why does the actual data sampling interval sometimes differ from the configured interval?
Overview
About Telemetry
Telemetry is an automated communications process by which performance and other data are collected on network devices and sent to the managed devices for monitoring and analysis. Today, this term typically refers to model-driven telemetry, which provides a high-speed, large-scale remote data collection mechanism for live, network-wide network device performance monitoring.
Today’s network has grown dramatically in size and complexity. Network operations are facing the following challenges:
· Growing network size—The number of managed devices has grown dramatically, so is the number of metrics that must be monitored.
· Quick fault isolation—The administrators must identify points of failure in seconds or subseconds to minimize service interruption.
· Granular monitoring—To help administrators make data-driven fault prediction and network optimization decisions, a network management and operations system must monitor diverse, granular metrics to present an accurate, holistic view of the network. For example, a network management and operations system might need to monitor not only interface traffic statistics, packet drops per flow, and CPU and memory consumed per flow. It might also need to monitor the delay jitter of every flow, delay of every packet on their transmission paths, and buffer usage of every device.
Traditional network monitoring through SNMP, CLI, and log collection can hardly address these challenges.
· Traditional management systems typically use SNMP or access the CLI of a device to request data from it. This pull-based data retrieval mechanism is slow and can become troublesome as the network size grows.
· Even though SNMP and NetStream enable network devices to push notifications and logs, respectively, this information is quite limited and can hardly reflect the network health status.
To collect massive granular performance metrics data at high speed from a large number of devices for intelligent network operations, modern networks must deploy model-driven telemetry.
· With modern telemetry techniques, network devices can push various telemetry data periodically or instantly to data collectors.
· The collectors can then deliver massive telemetry data pushed by network devices to an analyzer for analysis, which in turn reports the analytic data to the network controller. Then, the controller reconfigures the network devices as needed to deliver optimal network performance and service experience.
Telemetry network model
As shown in Figure 1, a typical telemetry network model contains the following elements:
· Network device—Monitored node, which acts as a sensor to provide data for different types of management systems. A network device periodically or instantly delivers telemetry data to data collectors in Google remote procedure call (gRPC) messages.
· Collector—Ingests and stores telemetry data reported by the monitored network devices.
· Analyzer—Analyzes telemetry data received from the collector and presents visualized analytic data.
· Controller—Operates and configures the network devices through NETCONF or other methods. A controller reconfigures the network devices based on the received analytic data to control and optimize the forwarding behavior of network devices for delivery of optimal service experience. A controller can also specify the telemetry data to be collected and reported by its managed network devices.
Figure 1 Telemetry network model
gRPC telemetry
H3C devices push telemetry data to collectors through gRPC. After a device establishes a gRPC connection with the management system, the device automatically streams the subscribed CPU, memory, interface, or other telemetry data to the target collectors as configured.
About gRPC
gRPC is an open source remote procedure call (RPC) system initially developed at Google. It uses HTTP 2.0 and provides methods neutral to programming languages for network device configuration and management. The server and client developers can develop server-side and client-side custom capabilities in the gRPC framework.
gRPC protocol stack
Figure 2 shows the gRPC protocol stack.
Table 1 describes each layer of the gRPC protocol stack, from the top down.
Table 1 gRPC protocol stack layers
|
Layer |
Description |
|
Content |
Conveys encoded service data. The following are encodings supported at the time of this writing: · Google Protocol Buffer (GPB)—An efficient binary encoding format that use .proto files to describe the data structures used for encoding. ·This encoding can deliver more data than encodings such as JavaScript Object Notation (JSON) encoding. If GPB encoding is used, the client side must use .proto service module files for decoding. · JavaScript Object Notation (JSON)—A lightweight data-interchange text format that is language independent. JSON is easy for humans to read and write and easy for devices to parse and generate. A data collector only needs to have common .proto files to decode the JSON-encoded service data sent by the device. For a successful decoding, the data collector must use the same .proto files as the device. |
|
gRPC |
Defines the protocol interaction format for remote procedure calls. The definitions of the public RPC methods are provided in public .proto files, for example, grpc_dialout.proto. |
|
HTTP 2.0 |
Carries gRPC when the transport protocol is TCP. This layer takes advantage of the performance enhancements provided by HTTP 2.0, for example, HTTP header compression, full request and response multiplexing, and flow control. |
|
Transport Layer Security (TLS) |
(Optional.) Uses TLS to do channel encryption and bidirectional certificate authentication for communication security. |
|
Transport |
Underlying transport protocol. Comware 7 supports only TCP to provide connection-oriented reliable data links. |
gRPC network architecture
As shown in Figure 3, a gRPC network uses the client/server model and transport messages between the client and the server over HTTP 2.0.
Figure 3 gRPC network architecture
The gRPC network uses the following mechanism:
1. The gRPC server listens to connection requests from clients at the gRPC service port.
2. A user runs a gRPC client application to log in to the gRPC server.
3. The gRPC client calls methods provided in the .proto file to send requests. For more information about .proto files, see "Proto files."
4. The gRPC server responds to the requests from the gRPC client.
|
|
NOTE: H3C devices can act as gRPC servers or clients. |
gRPC telemetry modes
gRPC telemetry operates in dial-in mode or dial-out mode, depending on the roles of network devices and collectors in a gRPC session.
Dial-in mode
In dial-in mode, the device acts as a gRPC server and the collectors act as gRPC clients. A collector initiates a gRPC connection to the device to subscribe to telemetry data. Dial-in mode typically applies to small networks and scenarios where collectors deploy configurations to devices.
Dial-in mode supports the following types of subscriptions:
· Non-gNMI subscription—Subscribes to event-triggered data. A non-gNMI subscription in dial-in mode requires the public RPC methods defined in the grpc_server.proto file and the RPC methods defined in the proto files for the target service modules.
· gNMI subscription—Subscribes to data push services provided by the target device. The data might be generated by periodical data collection or event-triggered data collection. To do a gNMI subscription, you need to use the grpc_server.proto, gnmi.proto, and gnmi_ext.proto files. The required public RPC methods are defined in the grpc_server.proto file.
|
|
NOTE: gNMI is a unified management protocol developed based on the open source gRPC framework for streaming telemetry and configuration management. It defines a set of RPC methods to obtain operational state data from devices or to configure devices. gNMI supports generic data models. It does not require additional service module specific proto files at the content layer. |
Dial-out mode
In dial-out mode, the device acts as a gRPC client and collectors act as gRPC servers. The device initiates gRPC connections and pushes subscribed telemetry data to the collectors. Dial-out mode typically applies to large networks where the collectors must collect data from a large number of devices.
The device uses a two-layer data model for dial-out telemetry. In this model, data is processed in two layers. Depending on the use of gNMI, this model is implemented in non-gNMI mode and gNMI mode.
· Non-gNMI mode:
¡ RPC layer—Defined in grpc_dialout.proto, a public proto file. This file defines the public RPC methods such as the message format. The jsonData message in the grpc_dialout.proto file contains JSON-encoded data.
¡ Content layer—Conveys JSON-encoded service data. The recipient collectors only need to use the grpc_dialout.proto file for decoding. No service proto files are required.
· gNMI mode:
¡ RPC layer—Defined in the gnmi.proto and gnmi_ext.proto files. These files define the public RPC methods such as the message format. The gnmi.proto file defines the Notification message to transmit JSON-encoded data.
¡ Content layer—Conveys JSON-encoded service data. The recipient collectors only need to use the gnmi.proto file for decoding. No service proto files are required.
gRPC service methods
The proto file for a gRPC service defines the RPC methods for the service, in the following format:
service service_name{
rpc method_name(parameter_name) returns(return_value) {}
}
An RPC method is identified by its parameter name and return type. The following is the sample definition of RPC methods for a service named gRPCService:
service gRPCService{
rpc gRPCMethod(para) returns(response) {}
rpc gRPCMethod(para) returns(stream response) {}
rpc gRPCMethod(stream para) returns(response) {}
rpc gRPCMethod(stream para) returns(stream response) {}
}
Table 2 gRPC service methods
|
Service method |
Description and example |
|
Unary |
The unary method uses a regular request/reply communication pattern in which the client sends a single request and gets a single response back. Example: rpc Login (LoginRequest) returns (LoginReply) |
|
Server-side streaming |
The server-side streaming method uses a communication pattern in which the client sends a request to the server and gets a stream to read a sequence of messages back. The client reads from the returned stream until there are no more messages. Example: rpc Subscribe(SubsPara) returns(stream SubsRply) |
|
Client-side streaming |
The client-side streaming method uses a communication pattern in which the client sends a sequence of messages to the server, and then waits for the server to read them and return its response. Example: rpc Dialout(stream DialoutMsg) returns (DialoutResponse) |
|
Bidirectional streaming |
The bidirectional streaming method uses a communication pattern in which the server and client each send a sequence of messages by using a read-write stream. The two streams operate independently, so the client and the server can read and write in whatever order they like. Example: rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse) |
Telemetry data
Telemetry data includes information about the device, including its interface traffic statistics, CPU usage statistics, and memory usage statistics. To obtain information about available telemetry data, see Telemetry Performance Metrics Reference.
Telemetry data is organized in YANG models.
A sensor path represents a subset of data definitions in a YANG model. You can access sensor paths to collect telemetry data of interest.
|
|
NOTE: · Yet Another Next Generation (YANG) is a data modeling language for the definition of data sent over transport protocols such as NETCONF and RESTCONF. You can use YANG to define models including configuration data models, state data models, remote procedure call (RPC) models, and notification mechanisms. · To retrieve only a subset of data from a sensor path, you can set filtering conditions when you specify a sensor path. For more information about specifying sensor paths, see "Sensor paths." · The device collects data from a sensor path periodically at the specified intervals. If the data in a sensor path is too much to be fully collected in one interval, the device will continue the operation to the next interval. If the CPU is rate limited, the telemetry intervals might be prolonged to accommodate the latency. |
Sensor paths
A sensor path represents a path in the hierarchy of a YANG model. You subscribe to data of interest by subscribing to sensor paths.
Sensor path format
Basic format
The basic sensor path format is node-name/child-node-name in the YANG model, for example, Device/CPUs.
In this example, Device represents a node name in a YANG model and CPUs is a child node under the Device node.
Organization-prefixed format
A sensor path might be present in yang_module_name:basic_sensor_path_format format. The yang_module_name argument represents a YANG module name and the basic_sensor_path_format argument represents a sensor path in this module. The YANG module name is prefixed with the name of the organization that defined the module.
For example, the openconfig-lldp:lldp/state sensor path represents the lldp/state node in the openconfig-lldp YANG module. The openconfig segment represents OpenConfig, the organization that defined this module.
At the time of this writing, the organization names for IETF, OpenConfig, and GSTA are supported.
Data filtering
To collect only data of interest, you can set data filters on sensor paths.
You can set filters only on the leaf nodes in YANG models.
Data filtering on periodic sensor paths
With periodic data collection, the device pushes data from periodic sensor paths to the collector at intervals. Sensor paths specified for periodic data collection are called periodic sensor paths. The following information describes the types of filters supported for them.
Value-based predicate filtering
With value-based predicate filtering, you set a filter in [column="value"] format to match data with the specified attribute value.
For example, to retrieve data for all GigabitEthernet interfaces with an interface number that starts with 1/0/, set the sensor path to ifmgr/statistics[ifindex="GigabitEthernet1/0/*"]. To retrieve data for the interface with an interface index of 12, set the sensor path to ifmgr/statistics[ifindex="12"].
When you use a value-based predicate filter, use the following guidelines:
· You cannot use predicate filters together with any other types of filters, and vice versa.
· You can configure a maximum of 64 predicate filters on one sensor path. The device will push all data that matches any one of the filters to the collector.
· You can use the asterisk (*) wildcard in a predicate filter. When you use this wildcard, follow these guidelines:
¡ You can use this wildcard only at the end of an index node value.
¡ A filter can contain only one asterisk.
|
|
NOTE: At the time of this writing, predicate filtering is available only for the ifmgr/statistics sensor path. |
Column-based filtering
You do column-based filtering by using the selection-nodes keyword. A column filter enables you to collect a subset of data from the specified nodes under a sensor path. You can specify a maximum of 24 space-separated nodes in a column filter.
For example, to retrieve only inrate and outrate data from the ifmgr/statistics node, set the sensor path to ifmgr/statistics selection-nodes inrate outrate.
Encodings
gRPC supports the following encodings:
· JavaScript Object Notation (JSON).
· Google Protocol Buffer (GPB).
· JSON IETF (JSON_IETF).
GPB encoding
GPB encoding is a language-neutral, platform-agnostic, extensible mechanism for serializing structured data. Encoding data in binary format, GPB provides higher performance than XML and JSON.
At the time of this writing, GPB is available in proto2 and proto3 versions. The device supports the proto3 version.
Table 3 shows a sample data section in its GPB-encoded and decoded formats.
Table 3 Sample data in GPB-encoded and decoded formats
|
GPB encoded |
GPB decoded |
|
1 : 1 3: EOK 4: 0 1: H3C 2: m16287 3: H3C CR16006-F 15: 2 16: Device/Base 17: 188 18: 1617276638160 19: 1617276638208 20: 1617276638208 21: 5000 22: OK 23: 1 1{ 1: 1617276638207 11: 3 { 1: 147699 2: "m16287" 3 "1.3.6.1.4.1.25506.1.1101" 4: 22 5: 1 10: "H3C Comware Platform Software, Software Version 7.1.075, ESS 8305\r\nH3C CR16006-F\r\nCopyright (c) 2004-2021 New H3C Technologies Co., Ltd. All rights reserved." 11: "2021-04-01T11:30:38" 12 { 1: "Z" } 13 { 1: 3 2: 1 } } }
|
ReqId : 1 errors: EOK totalSize: 0 producer_name: H3C Node_Id_str: m16287 ProductName: H3C CR16006-F Sub_Id_str: 2 Sensor_path: Device/Base Collection_Id: 188 Collection_start_time: 1617276638160 msg_timestamp: 1617276638208 Collection_end_time: 1617276638208 Current_period: 5000 except_desc: OK Encoding: Encoding_GPB row { timestamp: 1617276638207 content: Base { Uptime: 147699 HostName: "m16287" HostOid: "1.3.6.1.4.1.25506.1.1101" MaxSlotNum: 22 MaxCPUIDNum: 1 HostDescription: "H3C Comware Platform Software, Software Version 7.1.075, ESS 8305\r\nH3C CR16006-F\r\nCopyright (c) 2004-2021 New H3C Technologies Co., Ltd. All rights reserved." LocalTime: "2021-04-01T11:30:38" TimeZone { Zone: "Z" } ClockProtocol { Protocol: 3 MDCID: 1 } } }
|
JSON encoding
JSON encoding is a lightweight language-independent data interchange text format. It uses a syntactic subset of ECMAScript and stores and represents data in language-neutral text format. JSON is ideal for data interchange, because it uses a concise, clear-cut layered structure not only easy for developers to read and code but also for machines to parse and generate code.
Table 4 shows a sample data section in its JSON-encoded and decoded formats.
Table 4 Sample data in JSON-encoded and decoded formats
|
JSON encoded |
JSON decoded |
|
{ 1:"H3C" 2:"H3C" 3:"H3C device_test" 4:"not-config" 5:"sample" 2:"Syslog/LogBuffer" 3:"notification": { "Syslog": { "LogBuffer": { "BufferSize": 512, "BufferSizeLimit": 1024, "DroppedLogsCount": 0, "LogsCount": 100, "LogsCountPerSeverity": { "Alert": 0, "Critical": 1, "Debug": 0, "Emergency": 0, "Error": 3, "Informational": 80, "Notice": 15, "Warning": 1 }, "OverwrittenLogsCount": 0, "State": "enable" } }, "Timestamp": "1527206160022" } } |
{ "producerName": "H3C", "deviceName": "H3C", "deviceModel": "H3C device_test", “deviceIpAddr” : "not-config", “eventType” : "sample", "sensorPath": "Syslog/LogBuffer", "jsonData": { "notification": { "Syslog": { "LogBuffer": { "BufferSize": 512, "BufferSizeLimit": 1024, "DroppedLogsCount": 0, "LogsCount": 100, "LogsCountPerSeverity": { "Alert": 0, "Critical": 1, "Debug": 0, "Emergency": 0, "Error": 3, "Informational": 80, "Notice": 15, "Warning": 1 }, "OverwrittenLogsCount": 0, "State": "enable" } }, "Timestamp": "1527206160022" } } } |
JSON_IETF encoding
JSON_IETF encoding is a JSON encoding defined based on YANG 1.1 for YANG data trees and their subtrees. As in JSON, JSON_IETF encodes YANG data nodes (leaf, container, leaf list, list, anydata, and anyxml nodes) as JSON objects. With JSON_IETF encoding, JSON object members use the nsPrefix:memberID naming format, where:
· The nsPrefix argument represents the module name for the data node.
· The memberID argument represents the identifier of the YANG data node.
The nsPrefix and memberID arguments are colon (:) separated. For example, H3C-device-data:Uptime represents the Uptime node in the namespace for the H3C-device-data YANG module.
The submodules of a module must use the parent module's NS prefix in the JSON object member names for their nodes.
Table 5 shows a sample data section in its JSON_IETF encoded and decoded formats.
Table 5 Sample data in JSON_IETF-encoded and decoded formats
|
JSON_IETF encoded |
JSON_IETF decoded |
|
{ 1:"H3C" 2:"H3C 3:"H3C device_test" 4:"not-config" 5:"sample" 2:"Syslog/LogBuffer" 3:"notification": { "Syslog": { "LogBuffer": { "BufferSize": 512, "BufferSizeLimit": 1024, "DroppedLogsCount": 0, "LogsCount": 100, "LogsCountPerSeverity": { "Alert": 0, "Critical": 1, "Debug": 0, "Emergency": 0, "Error": 3, "Informational": 80, "Notice": 15, "Warning": 1 }, "OverwrittenLogsCount": 0, "State": "enable" } }, "Timestamp": "1527206160022" } } |
{ "producerName": "H3C", "deviceName": "H3C", "deviceModel": "H3C device_test", “deviceIpAddr” : "not-config", “eventType” : "sample", "sensorPath": "Syslog/LogBuffer", "jsonData": { "notification": { "Syslog": { "LogBuffer": { "H3C-syslog-data:BufferSize": 512, "H3C-syslog-data:BufferSizeLimit": 1024, "H3C-syslog-data:DroppedLogsCount": 0, "H3C-syslog-data:LogsCount": 100, "H3C-syslog-data:LogsCountPerSeverity": { "H3C-syslog-data:Alert": 0, "H3C-syslog-data:Critical": 1, "H3C-syslog-data:Debug": 0, "H3C-syslog-data:Emergency": 0, "H3C-syslog-data:Error": 3, "H3C-syslog-data:Informational": 80, "H3C-syslog-data:Notice": 15, "H3C-syslog-data:Warning": 1 }, "OverwrittenLogsCount": 0, "State": "enable" } }, "Timestamp": "1527206160022" } } } |
Proto files
About .proto files
GPB encoding requires metadata in the form of compiled .proto files to convert data to the GPB format. GPB provides an extensible, language-neutral, cross-platform mechanism for serializing structured data. Protocol buffer code is binary and provides higher performance than XML code and JSON code. A .proto file contains a set of service and message definitions.
You can install the protoc compiler on a collector to compile .proto files and generate code in a programing language such as C++, Java, Go, or Python. Based on the generated code, you can develop an application for the collector to interoperate with the device.
Comware provides public .proto files and service module .proto files.
Public .proto files
Comware provides the following public .proto files:
· grpc_dialout.proto
· grpc_dialout_v3.proto
· telemetry.proto
· grpc_service.proto
· gnmi.proto and gnmi_ext.proto
· dialout.proto
grpc_dialout.proto
The grpc_dialout.proto file applies when the device acts as a gRPC client to push data. This file defines the RPC methods and describes the conveyed data messages for the device to push data as a client, as shown below. The following file content is for illustration only. It might differ depending on the device.
syntax = "proto2";//The version of Protobuf is proto2
package grpc_dialout;//The package name is grpc_dialout
message DeviceInfo{//Device information to push
required string producerName = 1;//Vendor name
required string deviceName = 2;//Device name
required string deviceModel = 3;//Device model
optional string deviceIpAddr = 4;//Source IP
optional string eventType = 5;//Data sampling type (sensor path type)
}
message ChunkInfo{//Definition of the message format for ChunkInfo
required int64 totalSize = 1;//Packet size
required uint64 totalFragments = 2;//Total fragments of the packet
required uint64 nodeId = 3;//Packet ID
}
message DialoutMsg{//Definition of the message format for DialoutMsg
required DeviceInfo deviceMsg = 1;//Device information
required string sensorPath = 2;//Sensor path
required string jsonData = 3;//JSON data
optional ChunkInfo chunkMsg = 4;//Packet fragment information
}
message DialoutResponse{//Response from the collector (gRPC server). This message is reserved for future use and can contain an arbitrary value
required string response = 1;//Response sent by the device
}
service GRPCDialout {//The service name is GRPCDialout
rpc Dialout(stream DialoutMsg) returns (DialoutResponse);//The RPC method is dial-out mode (client-side streaming), and the input parameter is the DialoutMsg message
}
grpc_dialout_v3.proto
The grpc_dialout_v3.proto file applies when the three-layer data model is used in dial-out telemetry mode. This file defines the RPC methods and describes the conveyed data messages for the device to push data as a client. The three-layer data model supports both GPB and JSON for encoding. The following information provides a sample of this file. The file content is for illustration only. It might differ depending on the device.
syntax = "proto3";//The version of Protobuf is proto3
package grpc_dialout_v3;//The package name is grpc_dialout_v3
message DialoutV3Args{//Format of the three-layer encoding description message
int64 ReqId = 1;//Request ID
bytes data = 2;//Conveyed data
string errors = 3;//Error description
int32 totalSize = 4;//Total size of the message when fragmented. A value of 0 indicates that the message is not fragmented
}
service gRPCDialoutV3{//The service name is gRPCDialoutV3
rpc DialoutV3(stream DialoutV3Args) returns (stream DialoutV3Args) {};
//The RPC method is DialoutV3, which does bidirectional streaming and the input parameter is the DialoutV3Args message
}
telemetry.proto
The telemetry.proto file applies when the device acts as a client to push data to collectors. This file defines the messages for the telemetry layer in the three-layer telemetry data model. It describes important information such as sensor paths and timestamps. The following information provides a sample of this file. The file content is for illustration only. It might differ depending on the device.
syntax = "proto3";//The version of Protobuf is proto3
package telemetry; //The package name is telemetry
message Telemetry {//Definition of the message format for telemetry
string producer_name = 1;//Vendor name
string node_id_str = 2;//Device name
string product_name = 3;//Product name
string subscription_id_str = 15;//Subscription name
string sensor_path = 16;//Sensor path
uint64 collection_id = 17;//Number that identifies the round of data sampling
uint64 collection_start_time = 18;//Start time of data sampling
uint64 msg_timestamp = 19;//Timestamp of the message
uint64 collection_end_time = 20; //End time of data sampling
uint32 current_period = 21;//Sampling precision
string except_desc = 22;//Exception description
enum Encoding {//Supported encoding formats
Encoding_JSON = 0;//JSON encoding
Encoding_GPB = 1;//GPB encoding
};
Encoding encoding = 23;//Applied data encoding format
string data_str = 24;//This field is valid only when a non-GPB encoding is used. If GPB encoding is used, this field is null
TelemetryGPBTable data_gpb = 25; //Conveyed data is specified by using the TelemetryGPBTable message
}
message TelemetryGPBTable {//Definition of the message format for TelemetryGPBTable
repeated TelemetryRowGPB row = 1; //Array definition, which indicates that the data contains multiple TelemetryRowGPB structures
}
message TelemetryRowGPB {//Definition of the message format for TelemetryRowGPB
uint64 timestamp = 1; //Timestamp that indicates when the instance was sampled
bytes keys = 10; //Reserved field
bytes content = 11; //Instances of telemetry data
}
grpc_service.proto
The grpc_service.proto file defines the RPC methods and describes the conveyed data messages for the device to push data as a server. The following example shows a sample of this file. The file content is for illustration only. It might differ depending on the device.
syntax = "proto2";//The version of Protobuf is proto2
package grpc_service;//The package name is grpc_service
message GetJsonReply { //Reply to Get methods
required string result = 1;
}
message SubscribeReply { //Subscription result
required string result = 1;
}
message ConfigReply { //Configuration result
required string result = 1;
}
message ReportEvent { //Subscription event result definition
required string token_id = 1; //Login token ID
required string stream_name = 2; //Name of the subscribed event stream
required string event_name = 3; //Name of the subscribed event
required string json_text = 4; //Subscription result, a JSON string
}
message GetReportRequest{ //Request to obtain the event subscription result
required string NETCONF client successfully token_id = 1; //Token ID after a successful login
}
message LoginRequest {//Definitions of the login request parameters
required string user_name = 1; //Login username
required string password = 2; //Login password
}
message LoginReply {//Definition of the response to a login request
required string token_id = 1; //Token ID returned after a successful login
}
message LogoutRequest {//Definitions of the logout request parameters
required string token_id = 1; //token_id
}
message LogoutReply {//Definition of the reply to a logout request
required string result = 1; //Logout result
}
message SubscribeRequest {//Event stream definition
required string stream_name = 1; //Event stream name
}
message CliConfigArgs {//Issues a configuration command and its parameters to the device
required int64 ReqId = 1; //Configuration command request ID
required string cli = 2; //Configuration command
}
message CliConfigReply { //Reply to a configuration command execution request
required int64 ResReqId = 1; //Configuration command request ID, which corresponds to that in CliConfigArgs
required string output = 2; //Output from the configuration command
required string errors = 3; //Command execution result
}
message DisplayCmdArgs {//Issues a display command and its parameters to the device
required int64 ReqId = 1; //Display command request ID
required string cli = 2; //The display command
}
message DisplayCmdReply {//Returned display command execution result
required int64 ResReqId = 1; //Display command request ID, which corresponds to that in DisplayCmdArgs
required string output = 2; //Output from the display command
required string errors = 3; //Command execution result
}
service GrpcService {//RPC methods in gRPC dial-in mode
rpc Login (LoginRequest) returns (LoginReply) {}//Login method
rpc Logout (LogoutRequest) returns (LogoutReply) {}//Logout method
rpc SubscribeByStreamName (SubscribeRequest) returns (SubscribeReply) {}//Event stream subscription method
rpc GetEventReport (GetReportRequest) returns (stream ReportEvent) {}//Method for retrieving subscribed events
rpc CliConfig (CliConfigArgs) returns (stream CliConfigReply) {}//Method for executing a configuration command in CLI and returning the execution result
rpc DisplayCmdTextOutput(DisplayCmdArgs) returns(stream DisplayCmdReply) {}//Method for executing a display command in CLI and returning the retrieval result
}
gnmi.proto and gnmi_ext.proto
The gnmi.proto and gnmi_ext.proto files define the common methods for gNMI RPC operations. They are open source files contributed by Google. To download or obtain information about these two files, go to:
· https://github.com/openconfig/gnmi/tree/master/proto/gnmi/gnmi.proto
· https://github.com/openconfig/gnmi/tree/master/proto/gnmi_ext/gnmi_ext.proto
dialout.proto
The dial_out.proto file defines the RPC methods for gNMI RPC operations. This file is from the sonic-telemetry repository and is available at https://github.com/Azure/sonic-telemetry/blob/master/proto/dial_out.proto.
Service .proto files
Some of the service modules have service-specific .proto files called service .proto files. A service .proto file defines the service-specific RPC methods and messages for data subscription in dial-in mode or service-specific messages for GPB encoding in dial-out mode. When you develop a collector to monitor a service, obtain its service .proto file or files.
To obtain the .proto files of interest, contact H3C Support.
Proto files used in different telemetry modes
|
Telemetry mode |
Proto files |
|
Dial-in mode |
Non-gNMI subscription: · grpc_service.proto · Service .proto files gNMI subscription: · grpc_service.proto · gnmi.proto and gnmi_ext.proto |
|
Dial-out mode |
Two-layer data model: · Non-gNMI mode: · gNMI ¡ dialout.proto ¡ gnmi.proto and gnmi_ext.proto Three-layer data model: · grpc_dialout_v3.proto · telemetry.proto · Service proto files |
Telemetry restrictions and guidelines
To configure gRPC on the device, you must first execute the grpc enable command to enable gRPC.
If you execute the undo grpc enable command, the device will delete all gRPC settings in addition to disabling gRPC.
By default, the gRPC connections established by the device are unencrypted. If you specify a PKI domain, the device and the collector will use the Transport Layer Security (TLS) protocol for data encryption and bidirectional certificate-based authentication to improve communication security.
When you specify a PKI domain for establishment of secure gRPC connections, follow these restrictions and guidelines:
· The specified PKI domain must already exist and has correct and complete certificate and key configuration.
· After you specify a PKI domain, the gRPC function will reboot, closing the connections to collectors. In dial-in mode, the collectors must re-initiate the connections. In dial-out mode, the device will automatically re-initiate the connections.
Configuring telemetry subscription settings on the device
The following information describes the procedure to configure telemetry subscription settings on the device. For information about developing a collector to work with the device for gRPC telemetry, see "Example: Developing a collector-side application (dial-in mode)" or "Example: Developing a collector-side application (dial-out mode)."
Configuring a telemetry subscription in dial-in mode
About this task
In dial-in mode, the device acts as a gRPC server and collectors act as gRPC clients. A collector initiates a gRPC connection to the device to subscribe to telemetry data. This task enables collectors to retrieve or subscribe to data and events on the device in dial-in mode.
Restrictions and guidelines
The device closes the subscription session with a collector and stops collecting telemetry data for that collector when their gRPC connection disconnects. To collect data from the device again, the controller must re-establish a gRPC connection with the device.
Prerequisites
Before you configure dial-in mode settings, perform the following tasks:
· Make sure the gRPC server and client can reach each other. (Details not shown.)
· To encrypt gRPC connections, you must configure a PKI domain and make sure the domain contains complete certificate and key settings. For more information about configuring PKI, see PKI configuration in Security Configuration Guide.
Procedure
Configuring gRPC settings
1. Enter system view.
system-view
2. Enable gRPC.
grpc enable
By default, gRPC is disabled.
3. (Optional.) Set the gRPC service port number.
grpc port port-number
By default, the gRPC service port number is 50051.
Changing the gRPC service port number will cause the gRPC service to restart, disconnecting the ongoing sessions. The clients must re-establish gRPC sessions to collect telemetry data from the device.
4. (Optional.) Set the gRPC session idle timeout timer.
grpc idle-timeout minutes
By default, the gRPC session idle timeout timer is 5 minutes.
(Optional.) Specifying a PKI domain for secure communication with collectors
1. Enter system view.
system-view
2. Specify the PKI domain for secure communication with collectors.
grpc pki domain domain-name
By default, no PKI domain is specified for secure communication with collectors.
Configuring a gRPC user account
1. Enter system view.
system-view
2. Create a local device management user account.
local-user user-name [ class manage ]
To establish a gRPC session between a client and the device, you must create a gRPC user account for that client.
3. Set a password for the user.
password [ { hash | simple } password ]
By default, no password is configured for a local user. A passwordless user can pass authentication after providing the correct username and passing attribute checks.
4. Assign the network-admin user role to the user.
authorization-attribute user-role network-admin
By default, a local user is assigned the network-operator role.
5. Authorize the user to use the HTTPS service.
service-type https
By default, no service types are authorized to a local user.
For more information about the local-user, password, authorization-attribute, and service-type commands, see AAA commands in the security command reference for the device.
(Optional) Enabling gRPC logging in dial-in mode
About this task
To identify issues with gRPC in dial-in mode, enable gRPC logging in dial-in mode.
gRPC operation logging might degrade device performance if gRPC operations are frequent. As a best practice, use gRPC operation logging only when necessary and log only gRPC operations of interest if gRPC operation logging is enabled.
Procedure
1. Enter system view.
system-view
2. Enable gRPC logging in dial-in mode. Choose the options to configure as needed:
¡ Enable gRPC logging for RPC operations in dial-in mode.
grpc log dial-in rpc { all | { cli | get }* }
By default, gRPC logging is disabled for RPC operations in dial-in mode.
¡ Enable gRPC logging for gNMI operations in dial-in mode.
grpc log dial-in gnmi { all | { capabilities | get | set | subscribe }* }
By default, gRPC logging is enabled for gNMI Set operations and disabled for other gNMI operations in dial-in mode.
Verifying the configuration
Execute the display grpc command in any view to verify that gRPC dial-in mode operates as expected.
Example: Configuring gRPC dial-in mode
Network configuration
As shown in Figure 4, the device acts as a gRPC server and connects to a telemetry data collector (a gRPC client).
Configure the device to operate in gRPC dial-in mode. Enable the gRPC client to subscribe to events of interest on the device.
Configuring the device (gRPC server)
# Enable gRPC.
<Device> system-view
[Device] grpc enable
# Create a local user named test. Set the password of the user and assign the network-admin user role and the HTTPS service to the user.
[Device] local-user test
[Device-luser-manage-test] password simple 123456TESTplat&!
[Device-luser-manage-test] authorization-attribute user-role network-admin
[Device-luser-manage-test] service-type https
[Device-luser-manage-test] quit
Configuring a telemetry subscription in dial-out mode
About this task
In dial-out mode, the device acts as a gRPC client and collectors act as gRPC servers. The device initiates gRPC connections to push data to the collectors. Telemetry subscription in dial-out mode enables the device to push data and events subscribed by collectors without solicitation.
The following are the major elements for configuring subscriptions and data push in dial-out mode:
· Sensors—A sensor represents a data source identified by its data path from which the device collects data. A sensor group is a collection of one or multiple sensor paths. Sensor groups include non-gNMI sensor groups and gNMI sensor groups.
The following are types of sensor paths:
¡ Periodic—The device pushes data from the sensor path to collectors at intervals. For information about the periodic sensor paths for a module, see the telemetry performance metrics reference for that module.
¡ Event-triggered—The device pushes data from the sensor path to collectors when certain events occur. For information about the event-triggered sensor paths for a module, see the telemetry performance metrics reference for that module.
¡ Condition-triggered—The device periodically checks the sensor path and pushes data from the sensor path to collectors if the push conditions are met. For information about configuring condition-triggered sensor paths and the sensor path check interval and data push conditions for them, contact H3C Support.
|
|
IMPORTANT: Condition-triggered sensor paths are valid only when they are configured in gNMI sensor groups. To ensure successful data push, make sure a sensor group has only one type of sensor path. |
· Collectors—A collector ingests and stores telemetry data pushed by the monitored network devices. For the device to communicate with collectors, you must create a destination group and add collectors to the destination group.
To push data from the sensor paths in a sensor group to the collectors in a destination group, you must create a subscription, and then bind the sensor group and the destination group in the subscription.
Subscriptions include non-gNMI subscriptions and gNMI subscriptions. For a non-gNMI subscription, specify non-gNMI sensor groups. For a gNMI subscription, specify gNMI sensor groups.
Restrictions and guidelines
Data loss occurs in the following situations:
· When the gRPC connection to a collector disconnects, the device automatically reconnects to that collector and continues to push data. Telemetry data loss occurs during reconnection.
· gRPC service automatically restores after an active/standby switchover of the system or after you save the gRPC service configuration and reboot the device. Telemetry data loss occurs during restart or switchover.
Prerequisites
Before you configure dial-out mode, perform the following tasks:
· Make sure the gRPC server and client can reach each other. (Details not shown.)
· To encrypt gRPC connections, configure a PKI domain and make sure the domain contains complete certificate and key settings. For more information about configuring PKI, see PKI configuration in the security configuration guide for the device.
Procedure
Enabling the gRPC service
1. Enter system view.
system-view
2. Enable gRPC.
grpc enable
By default, gRPC is disabled.
3. (Optional.) Specify the architecture of telemetry data models.
grpc data-model { 2-layer | 3-layer }
By default, the device uses the two-layer telemetry data model architecture to push data.
With the two-layer telemetry data model architecture, the device cannot encode pushed data in GPB format.
(Optional.) Specifying a PKI domain for secure communication with collectors
1. Enter system view.
system-view
Specify the PKI domain for secure communication with collectors.
2. grpc pki domain domain-name
By default, no PKI domain is specified for secure communication with collectors.
Configuring non-gNMI sensors
1. Enter system view.
system-view
2. Enter telemetry view.
telemetry
3. Create a non-gNMI sensor group and enter non-gNMI sensor group view.
sensor-group group-name
4. Specify a periodic sensor path.
sensor path path [ selection-nodes node-list | depth depth ]
To configure multiple sensor paths, repeat this command. If you execute this command multiple times for the same sensor path, the most recent configuration takes effect.
|
Parameter |
Description |
|
selection-nodes |
Pushes data only from the specified nodes in the specified data path. |
|
depth |
Retrieval depth. The default depth is 1. Available depth options: · 1—Retrieves data from all columns under the specified path. · 2—Retrieves data from both columns and columns of the subtables under the specified path. · 3—Retrieves data from columns of the subtables (if any) attached to the subtables under the specified path, in addition to the columns and columns of the subtables under the specified path. |
|
|
IMPORTANT: If you include a predicate filter in the specified path, you cannot use the selection-nodes keyword to specify a data push condition, and vice versa. |
Configuring a gNMI sensor group
1. Enter system view.
system-view
2. Enter telemetry view.
telemetry
3. Create a gNMI sensor group and enter sensor group view.
sensor-group group-name gnmi
To enter the view of an existing gNMI sensor group, you do not need to specify the gnmi keyword.
4. Specify a sensor path.
sensor path path
To configure multiple sensor paths, repeat this command.
Configuring collectors
1. Enter system view.
system-view
2. Enter telemetry view.
telemetry
3. Create a destination group and enter destination group view.
destination-group group-name
As a best practice to avoid impact on system performance, limit the number of destination groups to five.
4. Specify a collector.
IPv4:
ipv4-address ipv4-address [ port port-number ] [ vpn-instance vpn-instance-name ]
IPv6:
ipv6-address ipv6-address [ port port-number ] [ vpn-instance vpn-instance-name ]
The IPv6 address of a collector cannot be a link-local address.
To add multiple collectors, repeat this command. A collector is uniquely identified by a three-tuple of IP address, port, and VPN instance. One collector must have a different IP address, port, or VPN instance than the other collectors.
Configuring a subscription
1. Enter system view.
system-view
2. Enter telemetry view.
telemetry
3. Create a subscription and enter subscription view. Choose one of the following options as needed:
¡ Create a non-gNMI subscription and enter subscription view.
subscription subscription-name
¡ Create a gNMI subscription and enter subscription view.
subscription subscription-name gnmi
To enter the view of an existing gNMI subscription, you do not need to specify the gnmi keyword.
4. (Optional.) Enable condition-triggered push mode.
push-mode condition-triggered
Perform this step only if the sensor group specified for the subscription contains condition-triggered sensor paths.
This command is available only for gNMI subscriptions.
By default, the device pushes only data from periodical and event-triggered sensor paths.
5. (Optional.) Set the DSCP value for packets sent to collectors.
dscp dscp-value
By default, the DSCP value for packets sent to collectors is 0.
A greater DSCP value represents a higher priority.
6. (Optional.) Specify the source IP address for packets sent to collectors.
source-address { ipv4-address | interface interface-type interface-number | ipv6 ipv6-address } [ port port-number ]
By default, the device uses the primary IPv4 address of the routed outgoing interface towards the collectors as the source address.
The device reconnects to collectors (gRPC servers) when the source IP address of the packets sent to them changes.
7. Specify a sensor group.
sensor-group group-name [ sample-interval [ msec ] interval | suppress-time suppress-time ]
Specify the sample-interval option only for periodic sensor paths.
Specify the suppress-time option only for condition-triggered sensor paths.
8. Specify a destination group.
destination-group group-name
You cannot use a destination group in both a gRPC subscription and a gNMI subscription.
Enabling gRPC logging in dial-out mode
1. Enter system view.
system-view
2. Enable gRPC logging in dial-out mode.
grpc log dial-out { all | { event | sample }* }
By default, gRPC logging is disabled for RPC operations in dial-out mode.
To identify issues with gRPC in dial-out mode, enable gRPC logging in dial-out mode.
gRPC logging in dial-out mode is not available for gNMI subscriptions.
Verifying the configuration
Execute the display system internal telemetry command in probe view to verify that gRPC dial-out mode operates as expected.
Example: Configuring gRPC dial-out mode
Network configuration
As shown in Figure 5, the device acts as a gRPC client and connects to a telemetry data collector (a gRPC server). The collector receives data at port 50051.
Configure the device to operate in gRPC dial-out mode to push the device capability information for its interface module to the collector at 10-second intervals.
Configuring the device (the gRPC client)
# Configure IP addresses as required so the device and the collector can reach each other. (Details not shown.)
# Enable gRPC.
<Device> system-view
[Device] grpc enable
# Create a sensor group named test and add sensor path ifmgr/devicecapabilities to it.
[Device] telemetry
[Device-telemetry] sensor-group test
[Device-telemetry-sensor-group-test] sensor path ifmgr/devicecapabilities
[Device-telemetry-sensor-group-test] quit
# Create a destination group named collector1 and add the collector to the group. In this example, the IP address and port number of the collector are 2.2.2.2 and 50051, respectively.
[Device-telemetry] destination-group collector1
[Device-telemetry-destination-group-collector1] ipv4-address 2.2.2.2 port 50051
[Device-telemetry-destination-group-collector1] quit
# Configure a subscription named A to bind sensor group test with destination group collector1. Set the data push interval to 10 seconds.
[Device-telemetry] subscription A
[Device-telemetry-subscription-A] sensor-group test sample-interval 10
[Device-telemetry-subscription-A] destination-group collector1
[Device-telemetry-subscription-A] quit
Example: Developing a collector-side application (dial-in mode)
|
|
IMPORTANT: The code samples in this document are for illustration only. They provide only the core business logic. You cannot use them without adaption to the code framework in use. |
To use gRPC dial-in mode, the developer must develop gRPC client code for the collector to retrieve and parse data from network devices. The client code primarily implements the following functions:
· Log in to the gRPC server and obtain the token ID (token_id).
· Prepare parameters for the RPC methods to call, use the service classes generated from the .proto files to initiate RPC calls, and then parse the returned results.
· Log out.
The following information describes the generic procedure to develop client software code for the following operations in gRPC dial-in mode:
· Non-gNMI Subscribe.
· gNMI Subscribe.
Prerequisites
Expertise and skill requirements for developers
· Be familiar with gRPC development.
· Be familiar with GPB coding.
· Be familiar with the required programming language, such as C++, Java, Python, or Go.
Setting up the development environment
Obtaining .proto files
To obtain the .proto files of interest, contact H3C Support.
Obtaining the protoc tool for processing proto files
The protoc tool is available for download at https://github.com/google/protobuf/releases.
Obtaining the protobuf plug-in for the used programming language
The protobuf plug-ins are available for download at https://github.com/google/protobuf/releases.
Prepare the development environment for the target language. For example, obtain protobuf plug-in protobuf-cpp for C++. This guide provides examples of coding in mainstream languages, including C++, Go, Python, and Java.
Storing the .proto files and tools
As a best practice, store the .proto files and protoc tool in the project directory for code development.
Network configuration
As shown in Figure 6, the device acts as a gRPC server and connects to a telemetry data collector (a gRPC client). Create a user account for the gRPC client to log in to the device. Set the username and password to admin and 123456, respectively.
Configure the device to operate in gRPC dial-in mode. Enable the gRPC client to subscribe to events of interest on the device.
Configuring telemetry subscription settings on the device
# Configure IP addresses as required so the gRPC server and client can reach each other. (Details not shown.)
# Enable gRPC.
<Device> system-view
[Device] grpc enable
# Create a local user named admin for the gRPC client and set the password to 123456. Assign the network-admin user role and the HTTPS service to the user.
[Device] local-user admin
[Device-luser-manage-admin] password simple 123456
[Device-luser-manage-admin] authorization-attribute user-role network-admin
[Device-luser-manage-admin] service-type https
[Device-luser-manage-admin] quit
Developing the collector-side application in C++
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into C++ code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Non-gNMI Subscribe |
grpc_service.proto. Service .proto files. This example also uses the BufferMonitor.proto file for buffer monitoring. |
|
gNMI Subscribe |
grpc_service.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate the C++ code for the proto files used in this example, as shown below:
$protoc --plugin=./grpc_cpp_plugin --grpc_out=. --cpp_out=. *.proto
Developing code
Non-gNMI Subscribe
The following information describes coding for a non-gNMI subscription by calling GrpcService and BufferMonitorService classes for example.
1. Create a GrpcServiceTest class.
# In the GrpcServiceTest class, use the GrpcService::Stub class generated from grpc_service.proto. Implement login and logout by using the Login and Logout methods generated from grpc_service.proto.
class GrpcServiceTest
{
public:
/* Construct function */
GrpcServiceTest(std::shared_ptr<Channel> channel): GrpcServiceStub(GrpcService::NewStub(channel)) {}
/* Member functions */
int Login(const std::string& username, const std::string& password);
void Logout();
void listen();
Status listen(const std::string& command);
/* Member variables */
std::string token;
private:
std::unique_ptr<GrpcService::Stub> GrpcServiceStub; //Use the GrpcService::Stub class generated from grpc_service.proto
};
2. Customize the Login method.
# Call the Login method in the GrpcService::Stub class to allow login with the correct username and password.
int GrpcServiceTest::Login(const std::string& username, const std::string& password)
{
LoginRequest request; //Set the username and password
request.set_user_name(username);
request.set_password(password);
LoginReply reply;
ClientContext context;
//Call the login method
Status status = GrpcServiceStub->Login(&context, request, &reply);
if (status.ok())
{
std::cout << "login ok!" << std::endl;
std::cout <<"token id is :" << reply.token_id() << std::endl;
token = reply.token_id(); //Login succeeded and a token was obtained
return 0;
}
else{
std::cout << status.error_code() << ": " << status.error_message()
<< ". Login failed!" << std::endl;
return -1;
}
}
3. Initiate an RPC request to the device.
In this example, the application subscribes to packet drop events on interfaces.
rpc SubscribePortQueDropEvent(PortQueDropEvent) returns (grpc_service.SubscribeReply) {}
4. Create the BufMon_GrpcClient class to encapsulate the RPC method.
# Use the BufferMonitorService::Stub class generated from BufferMonitor.proto to call the RPC method.
class BufMon_GrpcClient
{
public:
BufMon_GrpcClient(std::shared_ptr<Channel> channel): mStub(BufferMonitorService::NewStub(channel))
{}
std::string BufMon_Sub_AllEvent(std::string token);
std::string BufMon_Sub_BoardEvent(std::string token);
std::string BufMon_Sub_PortOverrunEvent(std::string token);
std::string BufMon_Sub_PortDropEvent(std::string token);
/* get data entries */
std::string BufMon_Sub_GetStatistics(std::string token);
std::string BufMon_Sub_GetGlobalCfg(std::string token);
std::string BufMon_Sub_GetBoardCfg(std::string token);
std::string BufMon_Sub_GetNodeQueCfg(std::string token);
std::string BufMon_Sub_GetPortQueCfg(std::string token);
private:
std::unique_ptr<BufferMonitorService::Stub> mStub; //class automatically generated by using BufferMonitor.proto
};
5. Use the self-defined std::string BufMon_Sub_PortDropEvent(std::string token) method to subscribe to packet drop events on interfaces.
std::string BufMon_GrpcClient::BufMon_Sub_PortDropEvent(std::string token)
{
std::cout << "-------BufMon_Sub_PortDropEvent-------- " << std::endl;
PortQueDropEvent stNodeEvent;
PortQueDropEvent_PortQueDrop* pstParam = stNodeEvent.add_portquedrop();
UINT uiIfIndex = 0;
UINT uiQueIdx = 0;
UINT uiAlarmType = 0;
std::cout<<"Please input interface queue info : ifIndex queIdx alarmtype " << std::endl;
cout<<"alarmtype : 1 for ingress; 2 for egress; 3 for port headroom"<<endl;
std::cin>>uiIfIndex>>uiQueIdx>>uiAlarmType; //Set subscription parameters, interface indexes, and other parameters as needed
pstParam->set_ifindex(uiIfIndex);
pstParam->set_queindex(uiQueIdx);
pstParam->set_alarmtype(uiAlarmType);
ClientContext context;
/* token need add to context */ //Set the token ID to be returned after a successful login
std::string key = "token_id";
std::string value = token;
context.AddMetadata(key, value);
SubscribeReply reply;
Status status = mStub->SubscribePortQueDropEvent(&context,stNodeEvent,&reply); //Call the RPC method
return reply.result();
}
6. Use a loop to listen for event reports.
# Implement this method in the GrpcServiceTest class.
void GrpcServiceTest::listen()
{
GetReportRequest reportRequest;
ClientContext context;
ReportEvent reportedEvent;
/* add token to request */
reportRequest.set_token_id(token);
std::unique_ptr< ClientReader< ReportEvent>> reader(GrpcServiceStub->GetEventReport(&context, reportRequest)); //Use GetEventReport (generated from grpc_service.proto) to obtain event information
std::string streamName;
std::string eventName;
std::string jsonText;
std::string token;
JsonFormatTool jsonTool;
std::cout << "Listen to server for Event" << std::endl;
while(reader->Read(&reportedEvent) ) //Read the received event report
{
streamName = reportedEvent.stream_name();
eventName = reportedEvent.event_name();
jsonText = reportedEvent.json_text();
token = reportedEvent.token_id();
std::cout << "/***********EVENT COME**************/" << std::endl;
std::cout << "TOKEN: " << token << std::endl;
std::cout << "StreamName: "<< streamName << std::endl;
std::cout << "EventName: " << eventName << std::endl;
std::cout << "JsonText without format: " << std::endl << jsonText << std::endl;
std::cout << std::endl;
std::cout << "JsonText Formated: " << jsonTool.formatJson(jsonText) << std::endl;
std::cout << std::endl;
}
Status status = reader->Finish();
std::cout << "Status Message:" << status.error_message() << "ERROR code :" << status.error_code();
}
7. Send a request to subscribe to packet drop events on the device.
void bufmon_test_PortDropStream()
{
auto channel = grpc::CreateChannel(g_server_address, grpc::InsecureChannelCredentials());
/* 1. login */
GrpcServiceTest reporter(channel);
if(0 != reporter.Login(g_username, g_password))
{
return;
}
/* 2. subscribe */
BufMon_GrpcClient cSubscriber(channel);
std::string replyForSysLog;
/* subscribe bufmon port overrun event */
replyForSysLog = cSubscriber.BufMon_Sub_PortDropEvent(reporter.token);
std::cout<<"BufMon board event: "<<replyForSysLog<<std::endl;
/* 3. listen to the server and get event */
reporter.listen();
std::cout<<"End of main."<<std::endl;
return;
}
8. Call the Logout method to log out.
void GrpcServiceTest:: Logout ()
{
LogoutRequest request;
request.set_token_id(token);
LogoutReply reply;
ClientContext context;
Status status = mStub->Logout(&context, request, &reply);
std::cout << "Logout! :" << reply.result() << std::endl;
}
gNMI Subscribe
1. Create a GrpcServiceTest class.
For more information about the procedure, see "Non-gNMI Subscribe."
2. Customize the Login method.
For more information about the procedure, see "Non-gNMI Subscribe."
3. Initiate an RPC request to the device.
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
4. Create the gnmi_client class to encapsulate the RPC method.
class gnmi_client
{
public:
explicit gnmi_client(const std::string &address,const std::string &tokenId);
bool TestCapabilities();
bool TestGet();
bool TestSet();
bool TestSubscribePool();
bool TestSubscribeOnce();
bool TestSubscribeStream();
bool TestSubscribeStreamWithAlias();
private:
void PrintCapabilityResponse(const gnmi::CapabilityResponse &response);
void PrintGetResponse(const gnmi::GetResponse &response);
void PrintSubscribeResponse(const gnmi::SubscribeResponse &response);
void PrintSubscribeRequest(const gnmi::SubscribeRequest &request);
void PrintGetRequest(const gnmi::GetRequest &request);
void PrintSetRequest(const gnmi::SetRequest &request);
void FillGetRequest(gnmi::GetRequest &request);
void FillSetRequest(gnmi::SetRequest &request);
void FillSubscribeRequestByOnce(gnmi::SubscribeRequest &request);
void FillSubscribeRequestByPool(gnmi::SubscribeRequest &request);
void FillSubscribeRequestByStream(gnmi::SubscribeRequest &request);
void FillSubscribePool(gnmi::SubscribeRequest &request);
void FillSubscribeAlias(gnmi::SubscribeRequest &request);
private:
std::unique_ptr<gnmi::gNMI::Stub> mStubGnmiService;
std::string mTokenID;
};
5. Create a self-defined Subscribe method.
void gnmi_client::FillSubscribeRequestByOnce(gnmi::SubscribeRequest &request)
{
auto subscribeList = request.mutable_subscribe();
auto prefix = subscribeList->mutable_prefix();
auto pathelem01 = prefix->add_elem();
pathelem01->set_name("LLDP");
auto subscribe = subscribeList->add_subscription();
auto path = subscribe->mutable_path();
auto pathelem02 = path->add_elem();
pathelem02->set_name("NeighborEvent");
auto pathelem03 = path->add_elem();
pathelem03->set_name("Neighbor");
(*pathelem03->mutable_key())["IfName"] = "xxx";
subscribeList->set_mode(::gnmi::SubscriptionList_Mode_ONCE);
subscribeList->set_encoding(::gnmi::JSON);
}
void gnmi_client::FillSubscribeRequestByPool(gnmi::SubscribeRequest &request)
{
auto subscribeList = request.mutable_subscribe();
auto prefix = subscribeList->mutable_prefix();
auto pathelem01 = prefix->add_elem();
pathelem01->set_name("Device");
auto subscribe = subscribeList->add_subscription();
auto path = subscribe->mutable_path();
auto pathelem02 = path->add_elem();
pathelem02->set_name("CPUs");
auto pathelem03 = path->add_elem();
pathelem03->set_name("CPU");
auto pathelem04 = path->add_elem();
pathelem04->set_name("CPUUsage");
subscribeList->set_mode(::gnmi::SubscriptionList_Mode_POLL);
subscribeList->set_encoding(::gnmi::JSON);
}
void gnmi_client::FillSubscribeRequestByStream(gnmi::SubscribeRequest &request)
{
auto subscribeList = request.mutable_subscribe();
auto prefix = subscribeList->mutable_prefix();
auto pathelem01 = prefix->add_elem();
pathelem01->set_name("Diagnostic");
auto subscribe = subscribeList->add_subscription();
auto path = subscribe->mutable_path();
auto pathelem02 = path->add_elem();
pathelem02->set_name("CPUEvent");
auto pathelem03 = path->add_elem();
pathelem03->set_name("CPU");
(*pathelem03->mutable_key())["Chassis#condition"] = "equal:1";
subscribe->set_mode(::gnmi::ON_CHANGE);
subscribe->set_sample_interval(1000);
subscribe->set_suppress_redundant(false);
subscribe->set_heartbeat_interval(1000);
subscribeList->set_mode(::gnmi::SubscriptionList_Mode_STREAM);
subscribeList->set_encoding(::gnmi::JSON);
}
void gnmi_client::FillSubscribeAlias(gnmi::SubscribeRequest &request)
{
auto aliases = request.mutable_aliases();
auto alias = aliases->add_alias();
auto path = alias->mutable_path();
auto pathelem01 = path->add_elem();
pathelem01->set_name("Device");
auto pathelem02 = path->add_elem();
pathelem02->set_name("CPUs");
auto pathelem03 = path->add_elem();
pathelem03->set_name("CPU");
auto pathelem04 = path->add_elem();
pathelem04->set_name("CPUUsage");
alias->set_alias("#cpu_usage");
}
6. Call the Logout method to log out.
void GrpcServiceTest:: Logout ()
{
LogoutRequest request;
request.set_token_id(token);
LogoutReply reply;
ClientContext context;
Status status = mStub->Logout(&context, request, &reply);
std::cout << "Logout! :" << reply.result() << std::endl;
}
Developing the collector-side application in Go
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into Go code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Non-gNMI Subscribe |
grpc_service.proto. Service .proto files. This example also uses the Syslog.proto file for Syslog. |
|
gNMI Subscribe |
grpc_service.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate the Go code for the proto files used in this example, as shown below:
[root@ grpc]# cd protobuf
[root@ protobuf]# protoc --go_out=plugins=grpc:. grpc_service.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. Syslog.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. gnmi.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. gnmi_ext.proto
Developing code
Non-gNMI Subscribe
The following information describes coding for non-gNMI Subscribe operations by calling GrpcService and SyslogService classes for example.
1. Create the grpcConnect method for logging in to and logging out of the device.
import(
"flag"
"io"
"log"
"fmt"
"time"
"github.com/gnmiTest/comwaresdk/sdk"
"github.com/gnmiTest/comwaresdk/cmwproto/gnmi"
"github.com/gnmiTest/comwaresdk/cmwproto/device"
syslog "github.com/gnmiTest/comwaresdk/cmwproto/syslog"
h3c "github.com/gnmiTest/comwaresdk/cmwproto/grpc_service"
)
var(
address string
port uint
username string
password string
grpcSession* sdk.GrpcSession
isSyslog bool
)
type grpcCon struct {
grpcSession *sdk.GrpcSession
sysClient syslog.SyslogServiceClient
h3cClient h3c.GrpcServiceClient
}
func grpcConnect() (*grpcCon, error) {
grpcSession, err := sdk.NewClient(address, port, username , password)
if err != nil {
log.Println("Failed to open session.")
return nil, err
}
sysClient := syslog.NewSyslogServiceClient(grpcSession.Conn)
gt := grpcCon{
grpcSession: grpcSession,
sysClient: sysClient,
h3cClient: grpcSession.Client,
}
return >, err
}
2. Initiate an RPC request to the device.
In this example, the application subscribes to events in the Syslog/LOGEvent sensor path.
rpc SubscribeLOGEvent(LOGEvent) returns (grpc_service.SubscribeReply) {}
3. Create the syslogSubscribeEvent method for event subscription and data receiving.
func (gt *grpcCon) syslogSubscribeEvent() error {
streamName := "Syslog"
mSubscribeRequest := syslog.EventStream{StreamName: &streamName}
ctxWithToken, cancel := sdk.CtxWithToken(gt.grpcSession.Token, time.Second*100)
defer cancel()
sysReply, err := gt.sysClient.SubscribeEventStream(ctxWithToken, &mSubscribeRequest)
if err != nil {
log.Println("syslog subscribe error: ", err)
return err
}
// print subscribe result
log.Printf("SyslogResult: \n %v", sysReply.GetResult())
// receive subscribe result
mGetReportquest := h3c.GetReportRequest{TokenId: >.grpcSession.Token}
stream, err := gt.h3cClient.GetEventReport(ctxWithToken, &mGetReportquest)
for {
mSubscribeResponse, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Println("Recv error: ", err)
return err
}else{
fmt.Println("success:", mSubscribeResponse.GetJsonText())
}
}
return nil
}
4. Create a main function to initiate an RPC request to the device.
func test() error {
con, err := grpcConnect()
defer con.grpcSession.Close()
if err != nil {
return err
}
if isSyslog{
err := con.syslogSubscribeEvent()
if err != nil {
return err
}
}
//Parameter parsing
func init() {
flag.StringVar(&address, "a", "192.168.2.1", "Address to comware")
flag.UintVar(&port, "gp", 50051, "Grpc port of comware")
flag.StringVar(&username, "u", "admin", "Username to comware")
flag.StringVar(&password, "up", "123456", "Password to comware")
flag.BoolVar(&isSyslog, "syslog", false, "dial in syslog event example")
}
//Main function
func main() {
flag.Parse()
test()
}
gNMI Subscribe
1. Create the grpcConnect method for logging in to and logging out of the device.
type grpcCon struct {
gnmiClient gnmi.GNMIClient
grpcSession *sdk.GrpcSession
}
func grpcConnect() (*grpcCon, error) {
grpcSession, err := sdk.NewClient(address, port, username , password)
if err != nil {
log.Println("Failed to open session.")
return nil, err
}
gnmiClient := gnmi.NewGNMIClient(grpcSession.Conn)
gt := grpcCon{
gnmiClient: gnmiClient,
grpcSession: grpcSession,
}
return >, err
}
2. Initiate an RPC request to the device.
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
3. Create a self-defined Subscribe method.
//Once-mode subscription
func (gt *grpcCon) gnmiSubscribeOnceTest() error {
mSubscribeRequest := gnmi.SubscribeRequest{
Request: &gnmi.SubscribeRequest_Subscribe{
Subscribe: &gnmi.SubscriptionList{
Prefix: &gnmi.Path{
Elem: []*gnmi.PathElem{
{Name: "Device"},
},
},
Subscription: []*gnmi.Subscription{
{Path: &gnmi.Path{
Elem: []*gnmi.PathElem{{Name: "Base"}},
},},
},
Mode: gnmi.SubscriptionList_ONCE,
Encoding: gnmi.Encoding_JSON,
},
},
}
ctxWithToken, cancel := sdk.CtxWithToken(gt.grpcSession.Token, time.Second*10)
defer cancel()
stream, err := gt.gnmiClient.Subscribe(ctxWithToken)
if err != nil {
log.Println("Subscribe error: ", err)
return err
}
stream.Send(&mSubscribeRequest)
for {
mSubscribeResponse, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Println("Recv error: ", err)
return err
}else{
fmt.Println("success:", mSubscribeResponse)
}
}
return nil
}
//Stream-mode subscription
func (gt *grpcCon) gnmiSubscribeStreamTest() error {
mSubscribeRequest := gnmi.SubscribeRequest{
Request: &gnmi.SubscribeRequest_Subscribe{
Subscribe: &gnmi.SubscriptionList{
Prefix: &gnmi.Path{
Elem: []*gnmi.PathElem{
// {Name: "Ifmgr"}, {Name: "Interfaces"},
{Name: "Device"},
},
},
Subscription: []*gnmi.Subscription{
{Path: &gnmi.Path{
// Elem: []*gnmi.PathElem{{Name: "Interface", Key: map[string]string{"IfIndex": "2",}},{Name: "ConfigDuplex"}},
Elem: []*gnmi.PathElem{{Name: "Base"}},
},
Mode: gnmi.SubscriptionMode_TARGET_DEFINED,
SampleInterval: uint64(1000000000),
SuppressRedundant: false,
HeartbeatInterval: uint64(1000),},
},
Mode: gnmi.SubscriptionList_STREAM,
Encoding: gnmi.Encoding_JSON,
},
},
}
ctxWithToken, cancel := sdk.CtxWithToken(gt.grpcSession.Token, time.Second*10)
defer cancel()
stream, err := gt.gnmiClient.Subscribe(ctxWithToken)
if err != nil {
log.Println("Subscribe error: ", err)
return err
}
stream.Send(&mSubscribeRequest)
for {
mSubscribeResponse, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Println("Recv error: ", err)
return err
}else{
fmt.Println("success:", mSubscribeResponse)
}
}
return nil
}
//Poll-mode subscription
func (gt *grpcCon) gnmiSubscribePollTest() error {
mSubscribeRequest := gnmi.SubscribeRequest{
Request: &gnmi.SubscribeRequest_Subscribe{
Subscribe: &gnmi.SubscriptionList{
Prefix: &gnmi.Path{
Elem: []*gnmi.PathElem{
{Name: "Device"},
},
},
Subscription: []*gnmi.Subscription{
{Path: &gnmi.Path{
Elem: []*gnmi.PathElem{{Name: "Base"}},
},},
},
Mode: gnmi.SubscriptionList_POLL,
Encoding: gnmi.Encoding_JSON,
},
},
}
mPoll := gnmi.SubscribeRequest{
Request: &gnmi.SubscribeRequest_Poll{
Poll: &gnmi.Poll{
},
},
}
ctxWithToken, cancel := sdk.CtxWithToken(gt.grpcSession.Token, time.Second*10)
defer cancel()
stream, err := gt.gnmiClient.Subscribe(ctxWithToken)
if err != nil {
log.Println("Subscribe error: ", err)
return err
}
stream.Send(&mSubscribeRequest)
for {
time.Sleep(time.Second)
stream.Send(&mPoll)
mSubscribeResponse, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Println("Recv error: ", err)
return err
}else{
fmt.Println("success:", mSubscribeResponse)
}
}
return nil
}
4. Create a main function to initiate an RPC request to the device.
func test() error {
con, err := grpcConnect()
defer con.grpcSession.Close()
if err != nil {
return err
}
if isSub{
err := con.gnmiSubscribeOnceTest()
if err != nil {
return err
}
}
return nil
}
//Parameter parsing
func init() {
flag.StringVar(&address, "a", "192.168.2.1", "Address to comware")
flag.UintVar(&port, "gp", 50051, "Grpc port of comware")
flag.StringVar(&username, "u", "admin", "Username to comware")
flag.StringVar(&password, "up", "123456", "Password to comware")
flag.BoolVar(&isSub, "sub", false, "gnmi subscribe examlpe")
}
//Main function
func main() {
flag.Parse()
test()
}
Developing the collector-side application in Python
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into Python code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Non-gNMI Subscribe |
grpc_service.proto. Service .proto files. This example uses the Syslog.proto file for Syslog. |
|
gNMI Subscribe |
grpc_service.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate Python code for the proto files used in this example, as shown below:
[root@ grpc]# cd protobuf
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. grpc_service.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. Syslog.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. gnmi.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. gnmi_ext.proto
Developing code
Non-gNMI Subscribe
The following information describes coding for non-gNMI Subscribe operations by calling GrpcService and SyslogService classes for example.
1. Initiate an RPC request to the device.
In this example, the application subscribes to log events.
rpc SubscribeLOGEvent(LOGEvent) returns (grpc_service.SubscribeReply) {}
2. Create a Client class for logging in to and logging out of the device.
import grpc
import json
from collections import OrderedDict
import grpc_service_pb2, grpc_service_pb2_grpc
import Syslog_pb2, Syslog_pb2_grpc
//Client programming class
class Client:
def __init__(self, username, password, channel):
self.username = username
self.password = password
self.channel = channel
self.__stub = grpc_service_pb2_grpc.GrpcServiceStub(channel)
self.tokenid = ""
def __enter__(self):
return
def __exit__(self, exc_type, exc_value, traceback):
if self.tokenid:
self.Logout()
def __str__(self):
return "{username=%s, password=%s, tokenid=%s}" % (self.username, self.password, self.tokenid)
def metadata(self):
return (("token_id", self.tokenid), )
//Login function, which calls the grpc_service_pb2.LoginRequest function for logging in to the device
def Login(self):
if self.tokenid:
return self
request = grpc_service_pb2.LoginRequest(user_name=self.username, password=self.password)
reply = self.__stub.Login(request)
self.tokenid = reply.token_id
return self
//Logout function, which calls the grpc_service_pb2.LogoutRequest function for logging out of the device
def Logout(self):
if not self.tokenid:
return
request = grpc_service_pb2.LogoutRequest(token_id=self.tokenid)
try:
self.__stub.Logout(request)
except Exception as e:
logging.warning("Logout:" + e)
self.tokenid = ""
return
//Subscribe to an event stream
def SubscribeByStreamName(self, stream):
request = grpc_service_pb2.SubscribeRequest(stream_name=stream)
reply = self.__stub.SubscribeByStreamName(request, metadata=self.metadata())
return reply.result
//Retrieve event report results
def GetEventReport(self):
request = grpc_service_pb2.GetReportRequest(token_id=self.tokenid)
yield from self.__stub.GetEventReport(request)
//Subscribe to Syslog log events
def sub(self, path):
if path == "SubscribeLOGEvent":
request = Syslog_pb2.LOGEvent()
RpcMethod = Syslog_pb2_grpc.SyslogServiceStub(self.channel)
reply = RpcMethod.SubscribeLOGEvent(request, metadata=self.metadata())
return reply.result
3. Create a main function to issue subscription requests and receive subscribed data.
//Format JSON strings
def format_json(jsonstr):
obj = json.loads(jsonstr, object_hook=OrderedDict)
return json.dumps(obj, ensure_ascii=False, indent=4)
//The client subscribes to events and receives events
def test():
channel = grpc.insecure_channel("192.168.2.1:50051")
client = Client("admin", "123456", channel)
with client.Login():
print(client)
print(client.sub("SubscribeLOGEvent"))
for e in client.GetEventReport():
print(e)
print(format_json(e.json_text))
//Main function
if __name__ == "__main__":
test()
gNMI Subscribe
1. Inherit the gRPC Client class you have created as described in "Non-gNMI Subscribe", and define the gNMI RPC methods to be used.
from dialin import Client
import gnmi_pb2, gnmi_pb2_grpc
import grpc
import time
class GnmiClient(Client):
@staticmethod
def get_mode(s:str):
mapping = {
"stream": gnmi_pb2.SubscriptionList.Mode.STREAM,
"once": gnmi_pb2.SubscriptionList.Mode.ONCE,
"poll": gnmi_pb2.SubscriptionList.Mode.POLL,
"target_defined": gnmi_pb2.TARGET_DEFINED,
"on_change": gnmi_pb2.ON_CHANGE,
"sample": gnmi_pb2.SAMPLE,
}
return mapping[s.lower()]
def __init__(self, username, password, channel):
super().__init__(username, password, channel)
self.__stub = gnmi_pb2_grpc.gNMIStub(channel)
return
def make_poll_req(self):
return gnmi_pb2.SubscribeRequest(poll=gnmi_pb2.Poll())
def Subscribe(self, req):
return self.__stub.Subscribe(req, metadata=self.metadata())
def make_sub_obj(self,interval):
path_obj = gnmi_pb2.Path()
ele = path_obj.elem.add()
ele.name = "Device"
ele1 = path_obj.elem.add()
ele1.name = "Base"
mode = self.get_mode("sample")
sample_interval = interval
return gnmi_pb2.Subscription(path=path_obj, mode=mode, sample_interval=sample_interval)
def make_sub_eventobj(self,interval,mode= "on_change"):
path_obj = gnmi_pb2.Path()
ele = path_obj.elem.add()
ele.name = "Ifmgr"
ele1 = path_obj.elem.add()
ele1.name = "InterfaceEvent"
mode = self.get_mode("on_change")
sample_interval = interval
return gnmi_pb2.Subscription(path=path_obj, mode=mode, sample_interval=sample_interval)
def make_sub_req(self, sample_interval , mode, type ="sample",qos=0, updates_only=False):
kwargs = {}
kwargs["prefix"] = gnmi_pb2.Path()
kwargs["qos"] = gnmi_pb2.QOSMarking(marking=qos)
kwargs["mode"] = self.get_mode(mode)
kwargs["encoding"] = gnmi_pb2.JSON
kwargs["subscription"] = []
if type == "sample":
kwargs["subscription"].append(self.make_sub_obj(sample_interval))
else:
kwargs["subscription"].append(self.make_sub_eventobj(sample_interval))
kwargs["updates_only"] = updates_only
subscribeList = gnmi_pb2.SubscriptionList(**kwargs)
return gnmi_pb2.SubscribeRequest(subscribe=subscribeList)
2. Initiate an RPC request to the device.
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
3. Create a self-defined Subscribe method.
def test_sub(client:GnmiClient):
def poll_generator(n):
req = client.make_sub_req(2000000000, mode="poll")
yield req
for _ in range(n):
time.sleep(2)
yield client.make_poll_req()
return
def sample_generator(t, sample_interval=2000000000, mode="stream"):
req = client.make_sub_req(sample_interval,mode)
yield req
time.sleep(t)
return
def event_generator(t):
req = client.make_sub_req(type ="event", sample_interval=10000000, mode="stream")
yield req
time.sleep(t)
return
4. Create a main function to issue subscription requests to and receive subscribed data from the device.
if __name__ == "__main__":
channel = grpc.insecure_channel("192.168.2.1:50051")
client = GnmiClient("admin", "123456", channel)
with client.Login():
test_sub(client)
Developing the collector-side application in Java
Generating code
Before you develop code in Java, install Maven, create a Maven project, and then edit the pom.xml for it to download the protoc tool, as follows:
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.6.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.14.0:exe:${os.detected.classifier}</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
Execute the mvn install command in Maven to convert the .proto files into Java code.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Non-gNMI Subscribe |
grpc_service.proto. Service .proto files. This example uses the Syslog.proto file for Syslog. |
|
gNMI Subscribe |
grpc_service.proto gnmi.proto and gnmi_ext.proto |
Developing code
Non-gNMI Subscribe
The following information describes coding for non-gNMI Subscribe operations by calling GrpcService and SyslogService classes for example.
1. Create a DialinClient class for logging in to and logging out of the device.
public class DialinClient {
private static final Logger logger = Logger.getLogger(DialinClient.class.getName());
private final GrpcServiceGrpc.GrpcServiceBlockingStub blockingStub;
private String tokenid;
public DialinClient(Channel channel) {
blockingStub = GrpcServiceGrpc.newBlockingStub(channel);
tokenid = "";
}
//Log in to the device
public void login(String username, String password) throws Exception {
logger.info("try to login as " + username + "...");
LoginRequest request = LoginRequest.newBuilder().setUserName(username).setPassword(password).build();
LoginReply loginReply;
try {
loginReply = blockingStub.login(request);
tokenid = loginReply.getTokenId();
} catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}",e.getStatus());
throw e;
}
System.out.println("Login : " + getTokenid());
}
public String getTokenid() {
return tokenid;
}
//Retrieve reports of subscribed events
public Iterator getEventReport(String tokenid) {
GrpcServiceOuterClass.GetReportRequest request = GrpcServiceOuterClass.GetReportRequest.newBuilder().setTokenId(tokenid).build();
Iterator< GrpcServiceOuterClass.ReportEvent> eventIterator = null;
try {
eventIterator = blockingStub.getEventReport(request);
} catch (Exception ignored) {
}
return eventIterator;
}
//Log out of the device
public void logout() {
LogoutRequest request = LogoutRequest.newBuilder().setTokenId(tokenid).build();
LogoutReply logoutReply = null;
try {
logoutReply = blockingStub.logout(request);
tokenid = "";
} catch (Exception ignored) {
}
if(logoutReply != null) {
System.out.println("Logout result: " + logoutReply.getResult());
}
return;
}
}
2. Initiate an RPC request to the device.
In this example, the application subscribes to log events.
rpc SubscribeLOGEvent(LOGEvent) returns (grpc_service.SubscribeReply) {}
3. Create a SyslogClient class to encapsulate the RPC methods.
public class SyslogClient {
private static final Logger logger = Logger.getLogger(SyslogClient.class.getName());
private SyslogServiceGrpc.SyslogServiceBlockingStub blockingStub;
private String tokenid;
public SyslogClient(Channel channel, String tokenid) {
blockingStub = SyslogServiceGrpc.newBlockingStub(channel);
this.tokenid = tokenid;
}
public String getTokenid() {
return tokenid;
}
public void setTokenid(String tokenid) {
this.tokenid = tokenid;
}
public String subLogEvent() throws Exception {
Syslog.LOGEvent requset = Syslog.LOGEvent.newBuilder().addLog(Syslog.LOGEvent.LOG.newBuilder()).build();
Metadata header = new Metadata();
Metadata.Key<String> key = Metadata.Key.of("token_id", Metadata.ASCII_STRING_MARSHALLER);
header.put(key, getTokenid());
SyslogServiceGrpc.SyslogServiceBlockingStub blockingStub_tmp = MetadataUtils.attachHeaders(blockingStub, header);
GrpcServiceOuterClass.SubscribeReply subscribeReply;
try {
subscribeReply = blockingStub_tmp.subscribeLOGEvent(requset);
}catch (StatusRuntimeException e) {
logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
throw e;
}
return subscribeReply.getResult();
}
}
4. Create a main function for event subscription.
public class Main {
private static String ipPort = "192.168.2.1:50051";
private static String usrname = "admin";
private static String password = "123456";
//User enters the IP address, username, and password
public static ArrayList UserInput( ){
System.out.println("Input ipaddress:port :");
Scanner scanner = new Scanner(System.in);
String str = "";
ArrayList<String> stringArrayList = new ArrayList<String>();
for (int i = 0;i < 3; i++){
str = scanner.nextLine();
stringArrayList.add(str);
if(i == 0) {
System.out.println("Input UserName :");
}else if (i == 1) {
System.out.println("Input PassWord :");
}
}
scanner.close();
return stringArrayList;
}
//Call the SyslogClient class to subscribe to events in syslog/logevent
public static void testSub (String tokenid) {
long begin=0, end=0;
int i;
String s;
ManagedChannel channel = null;
try {
channel = ManagedChannelBuilder.forTarget(ipPort).usePlaintext().build();
SyslogClient syslogClient = new SyslogClient(channel, tokenid);
begin = System.currentTimeMillis();
s = syslogClient.subLogEvent();
System.out.println(s);
end = System.currentTimeMillis();
System.out.printf("cost %dms\n", end-begin);
} catch (Exception e) {
}
}
//Call the DialinClient class for login, subscription, and logout
public static void testDialin() throws Exception {
DialinClient client = null;
ArrayList<String> arrayList = UserInput();
ipPort = arrayList.get(0);
usrname = arrayList.get(1);
password = arrayList.get(2);
ManagedChannel channel = ManagedChannelBuilder.forTarget(ipPort).usePlaintext().build();
String tokenID = "";
try {
client = new DialinClient(channel);
client.login(usrname, password);
tokenID = client.getTokenid();
testSub(tokenID);
Iterator<GrpcServiceOuterClass.ReportEvent> eventIterator = client.getEventReport(tokenID);
while (eventIterator.hasNext()) {
System.out.println(eventIterator.next());
}
} catch (Exception e) {
e.printStackTrace();
}
finally {
System.out.println("created ");
if (client != null) {
client.logout();
}
}
}
//Main function
public static void main (String[] agrs) throws Exception {
testDialin();
}
}
gNMI Subscribe
To program a gNMI Subscribe operation:
1. Create a DialinClient class for logging in to and logging out of the device.
For more information about the procedure, see "Non-gNMI Subscribe."
2. Initiate an RPC request to the device.
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
3. Create the gNMIClient class to implement self-defined Subscribe methods.
public class gNMIClient {
private static final Logger logger = Logger.getLogger(gNMIClient.class.getName());
private gNMIGrpc.gNMIStub gNMIStub;
private final String tokenId;
public gNMIClient(Channel channel, String tokenId) {
gNMIStub = gNMIGrpc.newStub(channel);
this.tokenId = tokenId;
}
//gNMI subscription request message assembly
public Gnmi.SubscribeRequest gnmiSubRequest(Gnmi.Path prefix, Gnmi.Path paths,
Gnmi.SubscriptionList.Mode mode, Gnmi.SubscriptionMode subscriptionMode,
int sample_interval) {
Gnmi.Subscription.Builder subBuilder = Gnmi.Subscription.newBuilder();
subBuilder.setPath(paths);
if (subscriptionMode != null) {
subBuilder.setMode(subscriptionMode);
}
if (sample_interval != 0)
{
subBuilder.setSampleInterval(sample_interval);
}
Gnmi.Subscription subscription = subBuilder.build();
Gnmi.SubscriptionList.Builder subList = Gnmi.SubscriptionList.newBuilder();
if (prefix != null) {
subList.setPrefix(prefix);
}
subList.addSubscription(subscription);
if (mode != null) {
subList.setMode(mode);
}
subList.setEncoding(Gnmi.Encoding.forNumber(0));
Gnmi.SubscriptionList subListReq = subList.build();
Gnmi.SubscribeRequest.Builder requestBuilder = Gnmi.SubscribeRequest.newBuilder();
requestBuilder.setSubscribe(subListReq);
Gnmi.SubscribeRequest request = requestBuilder.build();
return request;
}
//Poll-mode subscription request
public Gnmi.SubscribeRequest gnmiSubRequestByPoll (Gnmi.Path prefix, Gnmi.Path paths,Gnmi.SubscriptionList.Mode mode) {
return gnmiSubRequest(prefix, paths,mode, null, 0);
}
//Once-mode subscription request
public Gnmi.SubscribeRequest gnmiSubRequestByOnce(Gnmi.Path prefix, Gnmi.Path paths,Gnmi.SubscriptionList.Mode mode) {
return gnmiSubRequest(prefix, paths,mode, null, 0);
}
//Stream-mode subscription request
public Gnmi.SubscribeRequest gnmiSubRequestByStream(Gnmi.Path prefix, Gnmi.Path paths,Gnmi.SubscriptionList.Mode mode) {
return gnmiSubRequest(prefix, paths,mode, Gnmi.SubscriptionMode.forNumber(2), 2000000000);
}
//Once-mode subscription
public void testOnce(StreamObserver<Gnmi.SubscribeRequest> requestStreamObserver, Gnmi.Path prefix, Gnmi.Path paths) {
Gnmi.SubscribeRequest request = gnmiSubRequestByOnce(prefix, paths, Gnmi.SubscriptionList.Mode.forNumber(1));
requestStreamObserver.onNext(request);
}
//Poll-mode subscription
public void testPoll(StreamObserver<Gnmi.SubscribeRequest> requestStreamObserver, Gnmi.Path prefix, Gnmi.Path paths) {
Gnmi.SubscribeRequest request = gnmiSubRequestByPoll(prefix, paths, Gnmi.SubscriptionList.Mode.forNumber(2));
requestStreamObserver.onNext(request);
for (int i = 0; i < 5; i++) {
Gnmi.SubscribeRequest poll = Gnmi.SubscribeRequest.newBuilder().setPoll(Gnmi.Poll.newBuilder()).build();
requestStreamObserver.onNext(poll);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
//Stream-mode subscription
public void testStream(StreamObserver<Gnmi.SubscribeRequest> requestStreamObserver, Gnmi.Path prefix, Gnmi.Path paths) {
Gnmi.SubscribeRequest request = gnmiSubRequestByStream(prefix, paths, Gnmi.SubscriptionList.Mode.forNumber(0));
requestStreamObserver.onNext(request);
}
//Issue subscription request and receive subscribed data
public void testSubscribe(String name) {
Metadata header = new Metadata();
Metadata.Key<String> key = Metadata.Key.of("token_id", Metadata.ASCII_STRING_MARSHALLER);
header.put(key,tokenId);
gNMIGrpc.gNMIStub stub = MetadataUtils.attachHeaders(gNMIStub, header);
Gnmi.Path.Builder pathBuilder = Gnmi.Path.newBuilder();
String[] names = name.split("/");
for (String na:names) {
pathBuilder.addElem(Gnmi.PathElem.newBuilder().setName(na));
}
Gnmi.Path path = pathBuilder.build();
StreamObserver<Gnmi.SubscribeResponse> observer = new StreamObserver<Gnmi.SubscribeResponse>() {
@Override
public void onNext(Gnmi.SubscribeResponse subscribeResponse) {
if (subscribeResponse != null) {
System.out.println(subscribeResponse.getUpdate());
}
}
@Override
public void onError(Throwable throwable) {
logger.log(Level.WARNING, "Subscribe is failed");
throwable.printStackTrace();
System.out.println(throwable);
}
@Override
public void onCompleted() {
System.out.println("onCompleted");
}
};
StreamObserver<Gnmi.SubscribeRequest> requestStreamObserver = stub.subscribe(observer);
//Three modes for subscription
//testOnce(requestStreamObserver,null,path);
//testPoll(requestStreamObserver,null,path);
//testStream(requestStreamObserver,null,path);
try {
Thread.sleep(50000);
} catch (InterruptedException e) {
e.printStackTrace();
}
requestStreamObserver.onCompleted();
}
}
4. Create a main function for event subscription.
public class Main {
private static String ipPort = "192.168.2.1:50051";
private static String usrname = "admin";
private static String password = "123456";
public static ArrayList UserInput( ){
System.out.println("Input ipaddress:port :");
Scanner scanner = new Scanner(System.in);
String str = "";
ArrayList<String> stringArrayList = new ArrayList<String>();
for (int i = 0;i < 3; i++){
str = scanner.nextLine();
stringArrayList.add(str);
if(i == 0) {
System.out.println("Input UserName :");
}else if (i == 1) {
System.out.println("Input PassWord :");
}
}
scanner.close();
return stringArrayList;
}
//Subscribe to data, for example, subscribe to data in Device/Base
public static void testGnmi() throws Exception{
DialinClient dialinClient = null;
ArrayList<String> arrayList = UserInput();
ipPort = arrayList.get(0);
usrname = arrayList.get(1);
password = arrayList.get(2);
ManagedChannel channel = ManagedChannelBuilder.forTarget(ipPort).usePlaintext().build();
try {
dialinClient = new DialinClient(channel);
dialinClient.login(usrname, password);
gNMIClient client = new gNMIClient(channel, dialinClient.getTokenid());
client.testSubscribe("Device/Base");
} finally {
dialinClient.logout();
}
}
public static void main (String[] agrs) throws Exception {
testGnmi();
}
}
Example: Developing a collector-side application (dial-out mode)
|
|
IMPORTANT: The code samples in this document are for illustration only. They provide only the core business logic. You cannot use them without adaption to the code framework in use. |
To use gRPC dial-out mode, the developer must develop gRPC server code for the collector to receive and parse data from network devices. The server code primarily implements the following functions:
· Inherits the automatically generated GRPCDialout::Service class, reloads the automatically generated Dialout RPC service, and parse the fields.
· Register the RPC service and the service port specified for it.
The following information describes the generic procedure to develop server code for the following gRPC dial-out modes:
· Two-layer non-gNMI dial-out mode.
· Three-layer dial-out mode.
· Two-layer gNMI dial-out mode.
Prerequisites
Expertise and skill requirements
· Be familiar with gRPC development.
· Be familiar with GPB coding.
· Be familiar with the required programming language, such as C++, Java, Python, or Go.
Setting up the development environment
Obtaining .proto files
To obtain the .proto files of interest, contact H3C Support.
Obtaining the protoc tool for processing proto files
The protoc tool is available for download at https://github.com/google/protobuf/releases.
Obtaining the protobuf plug-in for the used programming language
The protobuf plug-ins are available for download at https://github.com/google/protobuf/releases.
Prepare the development environment for the target language. For example, obtain protobuf plug-in protobuf-cpp for C++. This guide provides examples of coding in mainstream languages, including C++, Go, Python, and Java.
Storing the .proto files and tools
As a best practice, store the .proto files and protoc tool in the project directory for code development.
Network configuration
As shown in Figure 7, the device acts as a gRPC client and connects to the telemetry data collector (a gRPC server). The collector receives data on port 50051.
Configure the device to operate in gRPC dial-out mode to push the device capability information for its interface module to the collector at 5-second intervals.
Configuring telemetry subscription settings on the device
# Configure IP addresses as required so the gRPC server and client can reach each other. (Details not shown.)
# Enable gRPC.
<Device> system-view
[Device] grpc enable
# Create a sensor group named test, and add sensor path ifmgr/devicecapabilities to it.
[Device] telemetry
[Device-telemetry] sensor-group test
[Device-telemetry-sensor-group-test] sensor path ifmgr/devicecapabilities
[Device-telemetry-sensor-group-test] quit
# Create a destination group named collector1 and add the collector to the group. In this example, the IP address and port number of the collector are 2.2.2.2 and 50051, respectively.
[Device-telemetry] destination-group collector1
[Device-telemetry-destination-group-collector1] ipv4-address 2.2.2.2 port 50051
[Device-telemetry-destination-group-collector1] quit
# Configure a subscription named A to bind sensor group test with destination group collector1. Set the data sampling interval to 5 seconds.
[Device-telemetry] subscription A
[Device-telemetry-subscription-A] sensor-group test sample-interval 5
[Device-telemetry-subscription-A] destination-group collector1
[Device-telemetry-subscription-A] quit
Developing the collector-side application in C++
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into C++ code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Two-layer non-gNMI dial-out mode |
grpc_dialout.proto |
|
Three-layer dial-out mode |
grpc_dialout_v3.proto telemetry.proto Service proto files (Ifmgr_v3.proto in this example). |
|
Two-layer gNMI dial-out mode |
dialout.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate C++ code for the proto files used in this example, as shown below:
$ protoc --plugin=./grpc_cpp_plugin --grpc_out=. --cpp_out=. *.proto
Developing code
Two-layer non-gNMI dial-out mode
1. Inherit and reload RPC service Dialout.
# Create class DialoutTest and inherit the attributes and methods from GRPCDialout::Service.
class DialoutTest final : public GRPCDialout::Service { //Reload the automatically generated abstract class
Status Dialout(ServerContext* context, ServerReader< DialoutMsg>* reader, DialoutResponse* response) override; //Implement RPC method Dialout
};
2. Register the DialoutTest service as a gRPC service and specify its service port.
using grpc::Server;
using grpc::ServerBuilder;
std::string server_address("0.0.0.0:50051"); //Specify the server address and its service port.
DialoutTest dialout_test; //Define the object declared in step 1
ServerBuilder builder;
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());//Add the listening port
builder.RegisterService(&dialout_test); //Register the service
std::unique_ptr<Server> server(builder.BuildAndStart()); //Start the service
server->Wait();
3. Implement the Dialout method and data parsing.
Status DialoutTest::Dialout(ServerContext* context, ServerReader< DialoutMsg>* reader, DialoutResponse* response)
{
DialoutMsg msg;
while( reader->Read(&msg))
{
const DeviceInfo &device_msg = msg.devicemsg();
std::cout<< "Producer-Name: " << device_msg.producername() << std::endl;
std::cout<< "Device-Name: " << device_msg.devicename() << std::endl;
std::cout<< "Device-Model: " << device_msg.devicemodel() << std::endl;
std::cout<<"Sensor-Path: " << msg.sensorpath()<<std::endl;
std::cout<<"Json-Data: " << msg.jsondata()<<std::endl;
std::cout<<std::endl;
}
response->set_response("test");
return Status::OK;
}
4. Use the Read method to obtain the DialoutMsg objects created for the proto files, and then call the correct method to read field values.
Three-layer dial-out mode
1. Inherit and reload RPC service Dialout.
# Create class DialoutV3Test and inherit the attributes and methods from DialoutV3::Service.
class DialoutV3Testfinal : public DialoutV3::Service { //Reload the automatically generated abstract class
Status DialoutV3Test::DialoutV3(::grpc::ServerContext* context, ::grpc::ServerReaderWriter< ::grpc_dialout_v3::DialoutV3Args, ::grpc_dialout_v3::DialoutV3Args>* stream)override; //Implement the DialoutV3 RPC method
};
2. Register DialoutV3Test service as a gRPC service and specify its service port.
string server_address("0.0.0.0.101:50051");
DialoutV3Test dialout_test;
ServerBuilder builder;
cout << "runing on " << server_address << endl;
builder.AddListeningPort(server_address, InsecureServerCredentials());
builder.RegisterService(&dialout_test);
unique_ptr<Server> server(builder.BuildAndStart());
server->Wait();
3. Implement the DialoutV3 method and data parsing.
Status DialoutV3Test::DialoutV3(::grpc::ServerContext* context, ::grpc::ServerReaderWriter< ::grpc_dialout_v3::DialoutV3Args, ::grpc_dialout_v3::DialoutV3Args>* stream)
{
DialoutV3Args msg;
Telemetry msgTele;
TelemetryGPBTable msgTable;
TelemetryRowGPB msgRow;
string buffdata = "";
int buffsize = 0;
string content;
std::cout << "peer info : " << context->peer() << std::endl;
while(stream->Read(&msg))
{
int row_size = 0;
int64_t ReqId = msg.reqid();
string data = msg.data();
string Err = msg.errors();
int32_t TotalSize = msg.totalsize();
buffdata = buffdata + data;
buffsize = buffsize + data.size();
if(buffsize >= TotalSize)
{
std::cout << "ReqId : " << ReqId << std::endl;
std::cout << "errors: " << Err << std::endl;
std::cout << "totalSize: " << TotalSize << std::endl;
msgTele.ParseFromString(buffdata);
std::cout << "data_size:" << buffdata.size() << std::endl;
std::cout << "producer_name: " << msgTele.producer_name() << std::endl;
std::cout << "Node_Id_str: " << msgTele.node_id_str() << std::endl;
std::cout << "ProductName: " << msgTele.product_name() << std::endl;
std::cout << "Sub_Id_str: " << msgTele.subscription_id_str() << std::endl;
std::cout << "Sensor_path: " << msgTele.sensor_path() << std::endl;
std::cout << "Collection_Id: " << msgTele.collection_id() << std::endl;
std::cout << "Collection_start_time: " << msgTele.collection_start_time() << std::endl;
std::cout << "msg_timestamp: " << msgTele.msg_timestamp() << std::endl;
std::cout << "Collection_end_time: " << msgTele.collection_end_time() << std::endl;
std::cout << "Current_period: " << msgTele.current_period() << std::endl;
std::cout << "except_desc: " << msgTele.except_desc() << std::endl;
std::cout << "Encoding: " << msgTele.Encoding_Name(msgTele.encoding()) << std::endl;
if(msgTele.encoding() == 1)
{
std::cout << "Start----------------------------------" << std::endl;
string path = msgTele.sensor_path();
ConverXpath2MessageName(path);//Convert Xpaths to message names
Message *pGpbMsg = createMessage(path);//Create a message for three-layer data parsing
if (pGpbMsg == NULL)
{
std::cout << "break------------------------ " << std::endl;
return Status::OK;
}
msgTable = msgTele.data_gpb();
row_size = msgTable.row_size();
std::cout << msgTable.DebugString() << std::endl;
std::cout << "row_size: " << row_size << std::endl;
for (int i = 0; i < row_size; i++)
{
msgRow = msgTable.row(i);
content = msgRow.content();
pGpbMsg->ParseFromString(content);
std::cout << pGpbMsg->DebugString() << std::endl;
}
}
if(msgTele.encoding() == 0)
{
std::cout << "JSON-Data: " << msgTele.data_str() << std::endl;
}
std::cout << "------------------------------------------" << std::endl;
buffdata = "";
buffsize = 0;
}
}
return Status::OK;
}
Message *createMessage(const std::string &type_name)
{
Message* pMessage = NULL;
const google::protobuf::Descriptor* descriptor =
google::protobuf::DescriptorPool::generated_pool()->FindMessageTypeByName(type_name);
if (descriptor)
{
const Message* prototype =
google::protobuf::MessageFactory::generated_factory()->GetPrototype(descriptor);
if (prototype)
{
pMessage = prototype->New();
}
}
return pMessage;
}
void ConverXpath2MessageName(string &path)
{
int pos = path.find("/", 0);
if (0 < pos)
{
string module = path.substr(0, pos);
string msgname = path.substr(0, pos);
for (int i = 0; i < pos; i++)
{
if ('-' == msgname[i])
{
msgname[i] = '_';
}
msgname[i] = tolower(msgname[i]);
}
for (int i = 0; i < pos; i++)
{
if ('-' == module[i])
{
module[i] = '_';
}
}
msgname.append("_v3.");
msgname.append(module);
path.replace(0, path.length(), msgname);
}
}
4. Use the Read method to obtain the DialoutV3Args objects created for the .proto files, and then call the correct method to read field values.
Two-layer gNMI dial-out mode
1. Inherit and reload RPC service gNMIDialOut.
# Create class gNMI_Dialout_Sever and inherit the attributes and methods from gNMIDialOut::Service.
class gNMI_Dialout_Sever final : public gNMIDialOut::Service{
Status Publish(ServerContext* context, ServerReaderWriter< PublishResponse, SubscribeResponse>* stream) override;
};
2. Register the gNMI_Dialout_Sever service as a gRPC service and specify its service port.
using grpc::Server;
using grpc::ServerBuilder;
std::string server_address("0.0.0.0:50051");//Specify the server address and its service port
gNMI_Dialout_Sever dialout_test; //Define the object declared in step 1
ServerBuilder builder;
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());// Add the service port
builder.RegisterService(&dialout_test); //Register the service
std::unique_ptr<Server> server(builder.BuildAndStart());//Start the service
std::cout << "Server listening on " << server_address << std::endl;
server->Wait();
3. Implement the gNMI Dialout method and data parsing.
Status gNMI_Dialout_Sever::Publish(ServerContext* context, ServerReaderWriter< PublishResponse, SubscribeResponse>* stream)
{
long long timestamp;
SubscribeResponse response;
std::vector<PublishResponse> publishes;
while(stream->Read(&response)){
std::cout << std::endl << "SubscribeResponse:" << response.DebugString() << std::endl;
if(response.response_case() == ::gnmi::SubscribeResponse::kUpdate)
{
auto& notification = response.update();
timestamp = notification.timestamp();
for(int i = 0; i < notification.update_size(); ++i)
{
auto& update = notification.update(i);
if(!update.has_val())
continue;
auto& val = update.val();
try
{
if(val.value_case() == ::gnmi::TypedValue::kJsonVal)
{
std::cout << json::JSONPrettifyWithRapid(val.json_val()) << std::endl;
}
}
catch(std::exception& e)
{
std::cout << "Json Format Error!" << std::endl;
}
}
PublishResponse publish;
publish.set_timestamp(timestamp);
auto prefix = publish.mutable_prefix();
prefix->add_elem()->set_name("test_prefix");
auto path_add = publish.add_path();
path_add->add_elem()->set_name("test_path");
publishes.push_back(publish);
}
return Status::OK;
}
4. Use the Read method to obtain the DialoutMsg objects created for the .proto files, and then call the correct method to read field values.
Developing the collector-side application in Go
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into C++ code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Two-layer non-gNMI dial-out mode |
grpc_dialout.proto |
|
Three-layer dial-out mode |
grpc_dialout_v3.proto telemetry.proto Service proto files (Ifmgr_v3.proto in this example). |
|
Two-layer gNMI dial-out mode |
dialout.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate the Go code for the .proto files used in this example, as shown below:
[root@ grpc]# cd protobuf
[root@ protobuf]# protoc --go_out=plugins=grpc:. grpc_dialout.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. telemetry.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. dialout.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. gnmi.proto
[root@ protobuf]# protoc --go_out=plugins=grpc:. gnmi_ext.proto
Developing code
Two-layer non-gNMI dial-out mode
1. Create the container for service implementation methods.
type Server struct {
grpc_dialout.GRPCDialoutServer1.
}
//Telemetry data struct
type TelemetryData struct {
DeviceMsg *grpc_dialout.DeviceInfo
SensorPath string
JsonData string
}
2. Register the DialoutTest service as a gRPC service and specify its service port.
lis, err := net.Listen("tcp", 50051)
if err != nil {
fmt.Printf("failed to listen: %v\n", err)
}
s := grpc.NewServer([]grpc.ServerOption{grpc.NumStreamWorkers(10)}...)
grpc_dialout.RegisterGRPCDialoutServer(s, &Server{})
if err := s.Serve(lis); err != nil {
fmt.Printf("failed to serve: %v\n", err)
}
3. Implement the Dialout method and data parsing.
func (s *Server) Dialout(gs grpc_dialout.GRPCDialout_DialoutServer) error {
for{
recvData, err := gs.Recv()
if err == io.EOF {
return gs.SendAndClose(&grpc_dialout.DialoutResponse{
Response: &grpc_dialout.DialoutMsg{
DeviceMsg: &grpc_dialout.DeviceInfo{
DeviceName: "H3C",
},
},
})
}
if err != nil {
return err
}
var data TelemetryData
data.DeviceMsg = recvData.GetDeviceMsg()
data.SensorPath = recvData.GetSensorPath()
data.JsonData = recvData.GetJsonData()
}
return nil
}
4. Obtain the DialoutMsg objects created for the .proto files, and then call the correct method to retrieve field values.
Three-layer dial-out mode
1. Create the container for service implementation methods.
//Container for service implementation methods
type Server struct {
grpc_dialout_v3.GRPCDialoutV3Server
}
type TelemetryV3Data struct {
ReqId int64
data []byte
errors string
totalSize int32
}
type telemetryGpb struct {
Row []*telemetry.TelemetryRowGPB
}
type devicegbp struct {
Base *device_v3.Device_MsgBase
}
2. Register GRPCDialoutV3 service as a gRPC service and specify the listening port.
func RunS() {
lis, err := net.Listen("tcp", port)
if err != nil {
fmt.Printf("failed to listen: %v\n", err)
}
s := grpc.NewServer([]grpc.ServerOption{grpc.NumStreamWorkers(10)}...)
grpc_dialout_v3.RegisterGRPCDialoutV3Server(s, &Server{})
if err := s.Serve(lis); err != nil {
fmt.Printf("failed to serve: %v\n", err)
}
}
func main() {
fmt.Println("start grpc server for h3c roce telemetry")
fmt.Printf("binding port:%v\n", port)
RunS()
}
3. Implement the GRPCDialoutV3 method and parse data.
func (s *Server) DialoutV3(stream grpc_dialout_v3.GRPCDialoutV3_DialoutV3Server) error {
for{
recvData, err := stream.Recv()
if err == io.EOF {
return grpc.Errorf(codes.Aborted, "stream EOF received")
}
if err != nil {
return grpc.Errorf(grpc.Code(err), "received error from client")
}
var dataRecv TelemetryV3Data
dataRecv.ReqId = recvData.GetReqId()
dataRecv.data = recvData.GetData()
dataRecv.totalSize = recvData.GetTotalSize()
dataRecv.errors = recvData.GetErrors()
//Parse data at the telemetry layer
printGpb(dataRecv.data)
}
return nil
}
4. Parse data at the telemetry layer.
Take the Ifmgr/devicecapabilities sensor path for example.
func typeForName(name string) (reflect.Type, error) {
pt := proto.MessageType(name)
if pt == nil {
return nil, fmt.Errorf("unknown type: %q", name)
}
st := pt.Elem()
return st, nil
}
//Obtain the service proto file for the service of interest to decode messages.
func getProto(messageType string, messageBytes []byte) proto.Message {
pt, _:= typeForName(messageType)
msg := reflect.New(pt).Interface().(proto.Message)
proto.Unmarshal(messageBytes, msg)
return msg
}
func printGpb(bytes []byte) {
msg := new(telemetry.Telemetry)
//Decode data at the telemetry layer
proto.Unmarshal(bytes, msg)
if msg.Encoding == telemetry.Telemetry_Encoding_JSON {
fmt.Println("Json")
fmt.Println(msg.DataStr)
}else {
fmt.Println("GPB")
var gpbRow telemetryGpb
gpbRow.Row = msg.GetDataGpb().GetRow()
rowcontet := gpbRow.Row[0].GetContent()
str := strings.Split(msg.SensorPath,"/")
messageName := strings.ToLower(str[0]) + "_v3."+str[0]
fmt.Println(messageName)
//Decode data at the service layer
context := getProto(messageName, rowcontet)
fmt.Println(context)
}
}
Two-layer gNMI dial-out mode.
1. Create the container for service implementation methods.
// server is used to implement
type Server struct {
dataStore interface{} //For storing the data received
}
2. Register the GNMIDialout service as a gRPC service and specify the service port.
func main() {
lis, err := net.Listen("tcp", 50051)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
gs.RegisterGNMIDialOutServer(s, &Server{})
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
3. Implement the GNMIDialout method and data parsing.
// Publish implements
func (srv *Server) Publish(stream gs.GNMIDialOut_PublishServer) error {
for {
subscribeResponse, err := stream.Recv()
if err != nil {
if err == io.EOF {
return grpc.Errorf(codes.Aborted, "stream EOF received")
}
return grpc.Errorf(grpc.Code(err), "received error from client")
}else {
fmt.Println("success:", subscribeResponse)
}
}
}
4. Call the methods in the Ifmgr_v3.proto files to retrieve data.
Developing the collector-side application in Python
Generating code
Before you develop code, use the protoc tool to convert the collected .proto files into C++ code, and then add the generated code to the development project.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Two-layer non-gNMI dial-out mode |
grpc_dialout.proto |
|
Three-layer dial-out mode |
grpc_dialout_v3.proto telemetry.proto Service proto files (Ifmgr_v3.proto in this example). |
|
Two-layer gNMI dial-out mode |
dialout.proto gnmi.proto and gnmi_ext.proto |
Use the protoc tool to generate Python code for the .proto files used in this example, as shown below:
[root@ grpc]# cd protobuf
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. grpc_dialout.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. grpc_dialout_v3.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. telemetry.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. Ifmgr_v3.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. dialout.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. gnmi.proto.proto
[root@ protobuf]# python -m grpc_tools.protoc -I . --python_out=. --grpc_python_out=. gnmi_ext.proto.proto
Developing code
Two-layer non-gNMI dial-out mode
1. Create the DialoutServicer class for receiving data and sending replies.
class DialoutServicer(grpc_dialout_pb2_grpc.GRPCDialoutServicer):
def Dialout(self, request_iterator, context):
for i, req in enumerate(request_iterator):
print("thread: %d, message index: %d" % (threading.get_ident(), i))
print(req)
print()
print("a client is disconnected")
return grpc_dialout_pb2.DialoutResponse(response="anything")
2. Register the DialoutServicer service as a gRPC service and specify its service port.
def serve():
addr = "[::]:50051"
server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
grpc_dialout_pb2_grpc.add_GRPCDialoutServicer_to_server(DialoutServicer(), server)
server.add_insecure_port(addr)
server.start()
print(f"grpc dialout server is running on {addr}")
try:
server.wait_for_termination()
except KeyboardInterrupt:
print("Interrupted, now quiting")
return
3. Use the proto files to decode telemetry data.
Three-layer dial-out mode
1. Create the DialoutV3Servicer class for receiving data and sending replies.
class DialoutV3Servicer(grpc_dialout_v3_pb2_grpc.gRPCDialoutV3Servicer):
def DialoutV3(self, request_iterator, context):
for i, req in enumerate(request_iterator):
print("thread: %d, message index: %d" % (threading.get_ident(), i))
Self.print_gpb(req)//use the proto files to decode received data
print()
print("a client is disconnected\n")
return
2. Register the DialoutServicer service as a gRPC service and specify its service port.
def serve():
addr = "[::]:50051"
server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
grpc_dialout_v3_pb2_grpc.add_gRPCDialoutV3Servicer_to_server(DialoutV3Servicer(), server)
server.add_insecure_port(addr)
server.start()
print(f"grpc dialout server is running on {addr}")
try:
server.wait_for_termination()
except KeyboardInterrupt:
print("Interrupted, now quiting")
return
3. Add a method in the DialoutV3Servicer class for data decoding.
def find_msg_cls(self, sensor_path:str):
module_name = sensor_path.split("/")[0]
proto_name = f"{module_name}_v3"
pool = descriptor_pool.Default()
file_desc = pool.FindFileByName(f"{proto_name}.proto")//Service proto file for GPB decoding
module_desc = file_desc.message_types_by_name[module_name]
msg_fact = message_factory.MessageFactory(pool=pool)
return msg_fact.GetPrototype(module_desc)
def print_gpb(self, msg:grpc_dialout_v3_pb2.DialoutV3Args):
telemetry = telemetry_pb2.Telemetry.FromString(msg.data)
sensor_path = telemetry.sensor_path
msg.ClearField("data")
s = str(msg)
if telemetry.encoding == telemetry_pb2.Telemetry.Encoding.Encoding_JSON:
s += "\n" + str(telemetry)
print(s)
return
tmp = ""
try:
msg_cls = self.find_msg_cls(sensor_path)
for row in telemetry.data_gpb.row:
gpb = row.content
msec = "%03d" % (row.timestamp % 1000)
timestr = "({})".format(datetime.fromtimestamp(row.timestamp/1000).strftime(f"%Y-%m-%d %H:%M:%S.{msec}"))
tmp += "\ntimestamp: " + str(row.timestamp) + timestr
tmp += "\nkeys: " + str(row.keys)
tmp += f"\ncontent({sensor_path}):\n" + str(msg_cls.FromString(gpb))
telemetry.ClearField("data_gpb")
s += "\n" + str(telemetry) + tmp
print(s)
except Exception as e:
print(e)
return
Two-layer gNMI dial-out mode
1. Create the GnmiDialoutServicer class for receiving data and sending replies.
class GnmiDialoutServicer(dial_out_pb2_grpc.gNMIDialOutServicer):
def Publish(self, request_iterator, context):
for i, req in enumerate(request_iterator):
print("thread: %d, message index: %d" % (threading.get_ident(), i))
if req.HasField("update"):
tm = req.update.timestamp
msec = "%03d" % ((tm % 1000000000) // 1000000)
print(datetime.fromtimestamp(tm/1000000000).strftime(f"%Y-%m-%d %H:%M:%S.{msec}"))
print(req)
print()
print("a client is disconnected")
return
2. Register the GnmiDialoutServicer service as a gRPC service and specify its service port.
def serve():
addr = "[::]:50051"
server = grpc.server(futures.ThreadPoolExecutor(max_workers=4))
dial_out_pb2_grpc.add_gNMIDialOutServicer_to_server(GnmiDialoutServicer(), server)
server.add_insecure_port(addr)
server.start()
print(f"grpc dialout server is running on {addr}")
try:
server.wait_for_termination()
except KeyboardInterrupt:
print("Interrupted, now quiting")
return
Developing the collector-side application in Java
Generating code
Before you develop code in Java, install Maven, create a Maven project, and then edit the pom.xml for it to download the protoc tool, as follows:
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.6.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:3.6.1:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.14.0:exe:${os.detected.classifier}</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
Execute the mvn install command in Maven to convert the .proto files into Java code.
This example uses the following .proto files to develop code for a subscription:
|
Subscribe operation type |
Files |
|
Two-layer non-gNMI dial-out mode |
grpc_dialout.proto |
|
Three-layer dial-out mode |
grpc_dialout_v3.proto telemetry.proto Service proto files (Ifmgr_v3.proto in this example). |
|
Two-layer gNMI dial-out mode |
dialout.proto gnmi.proto and gnmi_ext.proto |
Developing code
Two-layer non-gNMI dial-out mode
1. Create the DialoutServer class and register the service and its service port.
public DialoutServer(int port) {
this.port = port;
ServerBuilder<?> serverBuilder = ServerBuilder.forPort(port);
server = serverBuilder.addService(new DialoutService()).build();
}
2. In the DialoutServer class, create the DialoutService class and inherit the attributes and methods from PCDialoutGrpc.GRPCDialoutImplBase for receiving and decoding data.
private static class DialoutService extends GRPCDialoutGrpc.GRPCDialoutImplBase {
DialoutService() {}
/**
* Gets a stream of DialoutMsg
*
* @param responseObserver an observer to receive the dialout response.
* @return an observer to receive the requested dialout messages.
*/
@Override
public StreamObserver<GrpcDialout.DialoutMsg> dialout(final StreamObserver<GrpcDialout.DialoutResponse> responseObserver) {
return new StreamObserver<GrpcDialout.DialoutMsg>() {
@Override
public void onNext(GrpcDialout.DialoutMsg msg) {
//Data parsing
System.err.println("--- received a msg ---");
GrpcDialout.DeviceInfo deviceInfo = msg.getDeviceMsg();
System.out.printf("producerName: %s\n", deviceInfo.getProducerName());
System.out.printf("deviceName: %s\n", deviceInfo.getDeviceName());
System.out.printf("deviceModel: %s\n", deviceInfo.getDeviceModel());
if (deviceInfo.hasDeviceModel()) {
System.out.printf("deviceIpAddr: %s\n", deviceInfo.getDeviceIpAddr());
}
if (deviceInfo.hasEventType()) {
System.out.printf("eventType: %s\n", deviceInfo.getEventType());
}
System.out.printf("sensorPath: %s\n", msg.getSensorPath());
System.out.printf("jsonData len: %s\n", msg.getJsonData().length());
System.out.printf("jsonData: %s\n", msg.getJsonData());
if (msg.hasChunkMsg()) {
GrpcDialout.ChunkInfo chunk = msg.getChunkMsg();
System.out.printf("totalSize: %d\n", chunk.getTotalSize());
System.out.printf("totalFragments: %d\n", chunk.getTotalFragments());
System.out.printf("nodeId: %d\n", chunk.getNodeId());
}
System.out.println();
}
@Override
public void onError(Throwable throwable) {
logger.log(Level.WARNING, "Dialout cancelled");
}
@Override
public void onCompleted() {
System.out.println("a client disconnectd.");
responseObserver.onCompleted();
}
};
}
}
3. Implement the main function in the DialoutServer class and start listening for the service.
public void start() throws IOException {
server.start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
System.err.println("*** shutting down gRPC server since JVM is shutting down");
try {
DialoutServer.this.stop();
}catch (InterruptedException e) {
e.printStackTrace(System.err);
}
System.err.println("*** server shut down");
}
});
}
public void stop() throws InterruptedException {
if (server != null) {
server.shutdown().awaitTermination(30, TimeUnit.SECONDS);
}
}
public void blockUnitilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
//Main function
public static void main (String[] args) throws Exception {
//Argument parser
ArgParser argParser = new ArgParser();
argParser.Parse(args);
DialoutServer server = new DialoutServer(argParser.port);
server.start();
server.blockUnitilShutdown();
}
Three-layer dial-out mode
1. Create the gRPCgpb class and register the service and its service port.
public gRPCgpb(int port) {
this.port = port;
ServerBuilder<?> serverBuilder = ServerBuilder.forPort(port);
server = serverBuilder.addService(new Dialoutv3Service()).build();
}
2. In the gRPCgpb class, create the Diaoutv3Service class and inherit the attributes and methods from gRPCDialoutV3Grpc.gRPCDialoutV3ImplBase.
private static class Dialoutv3Service extends gRPCDialoutV3Grpc.gRPCDialoutV3ImplBase {
Dialoutv3Service() {}
@Override
public StreamObserver<GrpcDialoutV3.DialoutV3Args> dialoutV3(final StreamObserver<GrpcDialoutV3.DialoutV3Args> responseObserver) {
return new StreamObserver<GrpcDialoutV3.DialoutV3Args>() {
@Override
public void onNext(GrpcDialoutV3.DialoutV3Args dialoutV3Args) {
System.err.println("--- received a msg ---");
System.out.printf("reqId: %d\n", dialoutV3Args.getReqId());
System.out.printf("errors: %s\n", dialoutV3Args.getErrors());
System.out.printf("reqId: %d\n", dialoutV3Args.getTotalSize());
paraseData(dialoutV3Args.getData());
}
@Override
public void onError(Throwable throwable) {
logger.log(Level.WARNING, "Dialout cancelled");
}
@Override
public void onCompleted() {
System.out.println("a client disconnectd.");
responseObserver.onCompleted();
}
};
}
}
}
3. Implement three-layer data decoding in the Dialoutv3Service class.
Take the Ifmgr/devicecapabilities module for example.
//Use the service .proto file to parse service data
private void printObject(String sensorPath, TelemetryOuterClass.TelemetryRowGPB rowGPB){
try {
if (true ==sensorPath.equals("Ifmgr/Devicecapabilities")) {
Object object =IfmgrV3.Ifmgr.parseFrom(rowGPB.getContent());
System.out.println(object.toString());
}
}catch (InvalidProtocolBufferException e)
{
logger.log(Level.WARNING, "sensorPath:" + sensorPath + ",TelemetryRowGPB:" + rowGPB);
}
}
private void copyTelemetryRowGPB(TelemetryOuterClass.TelemetryRowGPB rowGPB,String sensorPath){
if (rowGPB != null) {
System.out.printf("timestamp: %d\n", rowGPB.getTimestamp());
System.out.printf("key: %s\n", rowGPB.getKeys().toString());
printObject(sensorPath, rowGPB);
}
}
//GPB decoding
public void paraseData (ByteString byteString) {
try {
TelemetryOuterClass.Telemetry telemetry TelemetryOuterClass.Telemetry.parseFrom(byteString);
if (telemetry == null) {
return;
}
System.out.printf("producer_name: %s\n", telemetry.getProducerName());
System.out.printf("node_id_str: %s\n", telemetry.getNodeIdStr());
System.out.printf("product_name: %s\n", telemetry.getProductName());
System.out.printf("sensorPath: %s\n", telemetry.getSensorPath());
System.out.printf("collection_id: %d\n", telemetry.getCollectionId());
System.out.printf("collection_start_time: %d\n", telemetry.getCollectionStartTime());
System.out.printf("msg_timestamp: %d\n", telemetry.getMsgTimestamp());
System.out.printf("collection_end_time: %d\n", telemetry.getCollectionEndTime());
System.out.printf("current_period: %d\n", telemetry.getCurrentPeriod());
System.out.printf("execpt_desc: %s\n", telemetry.getExceptDesc());
System.out.printf("encoding: %s\n", telemetry.getEncoding().toString());
if (telemetry.getEncodingValue() == 0) {
System.out.println(telemetry.getDataStr());
}else {
TelemetryOuterClass.TelemetryGPBTable telemetryGPBTable = telemetry.getDataGpb();
if (telemetryGPBTable != null) {
int rowCount = telemetryGPBTable.getRowCount();
for (int i = 0; i < rowCount; i++) {
//Decode telemetry row data
copyTelemetryRowGPB(telemetryGPBTable.getRow(i),telemetry.getSensorPath());
}
}
}
}catch (Exception e) {
System.err.printf("TelemetryRowGPB parased false");
}
}
4. Implement the main function in the gRPCgpb class and start listening for the service.
public void start() throws IOException {
server.start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
System.err.println("*** shutting down gRPC server since JVM is shutting down");
try {
gRPCgpb.this.stop();
}catch (InterruptedException e) {
e.printStackTrace(System.err);
}
System.err.println("*** server shut down");
}
});
}
public void stop() throws InterruptedException {
if (server != null) {
server.shutdown().awaitTermination(30, TimeUnit.SECONDS);
}
}
public void blockUnitilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
//Main function
public static void main (String[] args) throws Exception {
//Argument parser
ArgParser argParser = new ArgParser();
argParser.Parse(args);
gRPCgpb server = new gRPCgpb(argParser.port);
server.start();
server.blockUnitilShutdown();
}
Two-layer gNMI dial-out mode
1. Create the gnmiDialout class and register the service and its service port.
public gnmiDailout(int port) {
this.port = port;
ServerBuilder<?> serverBuilder = ServerBuilder.forPort(port);
server = serverBuilder.addService(new gnmiDailoutService()).build();
}
2. In the gnmiDialout class, create the gnmiDialoutService class and inherit the attributes and methods from gNMIDialOutGrpc.gNMIDialOutImplBase for data parsing.
private static class gnmiDailoutService extends gNMIDialOutGrpc.gNMIDialOutImplBase {
gnmiDailoutService() {}
@Override
public StreamObserver<Gnmi.SubscribeResponse> publish(final StreamObserver<DialOut.PublishResponse> responseObserver) {
return new StreamObserver<Gnmi.SubscribeResponse>() {
@Override
public void onNext(Gnmi.SubscribeResponse subscribeResponse) {
System.err.println("--- received a msg ---");
//Parse data
Object object = subscribeResponse.getUpdate();
System.out.println(object.toString());
}
@Override
public void onError(Throwable throwable) {
logger.log(Level.WARNING, "Dialout cancelled");
}
@Override
public void onCompleted() {
System.out.println("a client disconnectd.");
responseObserver.onCompleted();
}
};
}
}
3. Implement the main function in the gnmiDialout class and start listening for the service.
public void start() throws IOException {
server.start();
logger.info("Server started, listening on " + port);
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
System.err.println("*** shuting down gRPC server since JVM is shutting down");
try {
gnmiDailout.this.stop();
}catch (InterruptedException e) {
e.printStackTrace(System.err);
}
System.err.println("*** server shut down");
}
});
}
public void stop() throws InterruptedException {
if (server != null) {
server.shutdown().awaitTermination(30, TimeUnit.SECONDS);
}
}
public void blockUnitilShutdown() throws InterruptedException {
if (server != null) {
server.awaitTermination();
}
}
public static void main (String[] args) throws Exception {
ArgParser argParser = new ArgParser();
argParser.Parse(args);
gnmiDailout server = new gnmiDailout(argParser.port);
server.start();
server.blockUnitilShutdown();
}
FAQ
Why does the actual data sampling interval sometimes differ from the configured interval?
The device samples data at configured intervals as long as possible. Depending on the complexity in network and services, the actual sampling interval might differ from the configured one in the following situations:
· Outsized amount of data—Data in some sensor paths might grow to be too much for the gRPC module to sample all data in a sampling interval. For example, when the number of IPv4 route table entries reaches 100000, the gRPC module will be unable to collect all data from the route/ipv4routes at a short interval.
· Data granularity limit—Some data sources have a minimum data sampling interval. This minimum interval applies if the configured data sampling interval is shorter than it. For example, the minimum data sampling interval for the device/base sensor path is 5 seconds. Even if you set the sampling interval to 100 milliseconds, the gRPC module will collect and push data from it at intervals of 5 seconds.
· Busy CPU—The gRPC module might be unable to sample complete telemetry data as expected for a sampling interval while the CPU is busy.
