1. VPP-Agent架构调研

VPPAgent主程序实现：vpp-agent/cmd/vpp-agent；

1.1. VPPAgent主程序分析

vpp-agent/cmd/vpp-agent/main.go

抽象Agent基类的定义在 go.ligato.io/cn-infra/v2/agent

vppAgent := app.New()
    a := agent.NewAgent(
        agent.AllPlugins(vppAgent),
        agent.StartTimeout(startTimeout),
        agent.StopTimeout(stopTimeout),
    )

各个具体实现模块以Plugin方式嵌入;

同时ETCD和GRPC等模块都以插件模式植入Agent总代理,这里返回的VPP和Linux里面，定义了多种类型插件，比如L3，L2, Linux

etcdDataSync := kvdbsync.NewPlugin(kvdbsync.UseKV(&etcd.DefaultPlugin))
consulDataSync := kvdbsync.NewPlugin(kvdbsync.UseKV(&consul.DefaultPlugin))
redisDataSync := kvdbsync.NewPlugin(kvdbsync.UseKV(&redis.DefaultPlugin))

return &VPPAgent{
        PluginName:     "VPPAgent",
        LogManager:     &logmanager.DefaultPlugin,
        Orchestrator:   &orchestrator.DefaultPlugin,
        ETCDDataSync:   etcdDataSync,
        ConsulDataSync: consulDataSync,
        RedisDataSync:  redisDataSync,
        VPP:            vpp,
        Linux:          linux,
        Netalloc:       &netalloc.DefaultPlugin,
        Configurator:   &configurator.DefaultPlugin,
        RESTAPI:        &restapi.DefaultPlugin,
        Probe:          &probe.DefaultPlugin,
        StatusCheck:    &statuscheck.DefaultPlugin,
        Telemetry:      &telemetry.DefaultPlugin,
    }

1.2. 南向通信

GoVPP是核心模块

1.2.1. 测试示范用例

vpp-agent/examples/tutorials/07_vpp-connection/main.go

1.2.2. CHANNEL通道，南向传递消息

传递方式可以同步，可以异步；传递消息为Message，其中消息定义位置在下一节；通过govpp/api/api.go中的Channel同南向底层交互；govpp项目主要是同底层核心VPP进行交互的同一接口，git地址https://git.fd.io/govpp.git

文件路径govpp/api/api.go为接口文件，具体实现在govpp/core/channel.go文件中；

// Channel provides methods for direct communication with VPP channel.
type Channel interface {
    // SendRequest asynchronously sends a request to VPP. Returns a request context, that can be used to call ReceiveReply.
    // In case of any errors by sending, the error will be delivered to ReplyChan (and returned by ReceiveReply).
    SendRequest(msg Message) RequestCtx
    ...
}

该Channel实例化类为Channel,定义在govpp项目中，govpp/core/channel.go

type Channel struct {
    id   uint16
    conn *Connection        //这个是链接VPP数据面的基础结构体；

    reqChan   chan *vppRequest // channel for sending the requests to VPP，发送VPP请求的chan
    replyChan chan *vppReply   // channel where VPP replies are delivered to，接收VPP请求的chan

    msgCodec      MessageCodec      // used to decode binary data to generated API messages     //解码二进制数据为API message的接口
    msgIdentifier MessageIdentifier // used to retrieve message ID of a message

    lastSeqNum uint16 // sequence number of the last sent request

    delayedReply        *vppReply     // reply already taken from ReplyChan, buffered for later delivery
    replyTimeout        time.Duration // maximum time that the API waits for a reply from VPP before returning an error, can be set with SetReplyTimeout
    receiveReplyTimeout time.Duration // maximum time that we wait for receiver to consume reply
}

type requestCtx struct {
    ch     *Channel
    seqNum uint16
}

该Channel中，重点是Connection，用于管理和VPP链接的客户端句柄；

// Connection represents a shared memory connection to VPP via vppAdapter.
type Connection struct {
    vppClient adapter.VppAPI // VPP binary API client

    maxAttempts int           // interval for reconnect attempts
    recInterval time.Duration // maximum number of reconnect attempts

    vppConnected uint32 // non-zero if the adapter is connected to VPP

    connChan chan ConnectionEvent // connection status events are sent to this channel

    codec        MessageCodec                      // message codec
    msgIDs       map[string]uint16                 // map of message IDs indexed by message name + CRC
    msgMapByPath map[string]map[uint16]api.Message // map of messages indexed by message ID which are indexed by path

    maxChannelID uint32              // maximum used channel ID (the real limit is 2^15, 32-bit is used for atomic operations)
    channelsLock sync.RWMutex        // lock for the channels map
    channels     map[uint16]*Channel // map of all API channels indexed by the channel ID

    subscriptionsLock sync.RWMutex                  // lock for the subscriptions map
    subscriptions     map[uint16][]*subscriptionCtx // map od all notification subscriptions indexed by message ID

    pingReqID   uint16 // ID if the ControlPing message
    pingReplyID uint16 // ID of the ControlPingReply message

    lastReplyLock sync.Mutex // lock for the last reply
    lastReply     time.Time  // time of the last received reply from VPP

    msgControlPing      api.Message
    msgControlPingReply api.Message
}

在Connection中，实现同VPP交互的基础类为adapter.VppAPI。该类定义在govpp/adaptor中，主要用于具体VPP交互的实现基础库；

// Package adapter provides an interface between govpp core and the VPP. It is responsible for sending
// and receiving binary-encoded data to/from VPP via shared memory.
适配器包提供了govpp和VPP的接口，用于从VPP共享存储中，发送和接收二进制编码的数据；

// The default adapter being used for connection with real VPP is called vppapiclient. It is based on the
// communication with the vppapiclient VPP library written in C via CGO.
默认用于链接VPP的适配器是vppapiclient，它和基于C语言CGO编写的vppapiclient VPP 库进行通信；

// Apart from the vppapiclient adapter, mock adapter is provided for unit/integration testing where the actual
// communication with VPP is not demanded.
 mock adapter提供单元或集成测试，因此不需要跟VPP进行实际交互。

1.3. 南向接口位置

定义protobuf文件位置位于vpp-agent/plugins/vpp/binapi

vpp-agent/plugins/vpp/binapi/vpp2101/interface/interface.ba.go