atx-agent minicap、minitouch源码分析

项目描述:

　　因为公司需要,特别研究了一下openatx系列手机群控源码

　　该项目主要以go语言来编写服务端、集成 OpenSTF中核心组件 minicap和minitouch来完成

今天主要来分析一下atx-agent服务源码中 minicap和minitouch 相关接口源码

1.minicap

　　简介:　　　

　　minicap工具是用NDK开发的，属于Android的底层开发，该工具分为两个部分，一个是动态连接库.so文件，一个是minicap可执行文件。但不是通用的，
因为CPU架构的不同分为不同的版本文件，STF提供的minicap文件根据CPU 的ABI分为如下4种：从上面可以看出，minicap可执行文件分为4种，
分别针对arm64-v8a、armeabi-v7a,x86,x86_64 架构。而minicap.so文件在这个基础上还要分为不同的sdk版本。
　　minicap采集屏幕的原理很简单：通过ndk的截屏接口不停的截屏并通过socket接口实时发送，这样客户端便可以得到一序列的图片流，图片流合成后就成为视频。
　　使用原生screencap工具截屏并输出到图像需要4s多，对比minicap则只需要190ms，差距明显。minicap使用了libjpeg-turbo作为编码压缩工具，
压缩后的图片体积更小1080P分辨率的手机截图根据色彩丰富度不同一般只需要100k，sceencap则需要2M。

atx-agent minicap部分

 1 m.HandleFunc(\"/minicap\", singleFightNewerWebsocket(func(w http.ResponseWriter, r *http.Request, ws *websocket.Conn) {
 2         defer ws.Close()
 3 
 4         const wsWriteWait = 10 * time.Second
 5         wsWrite := func(messageType int, data []byte) error {
 6             //设置websocket写入最长超时间
 7             ws.SetWriteDeadline(time.Now().Add(wsWriteWait))
 8             return ws.WriteMessage(messageType, data)
 9         }
10         wsWrite(websocket.TextMessage, []byte(\"restart @minicap service\"))
11         //重启minicap
12         if err := service.Restart(\"minicap\"); err != nil && err != cmdctrl.ErrAlreadyRunning {
13             wsWrite(websocket.TextMessage, []byte(\"@minicap service start failed: \"+err.Error()))
14             return
15         }
16 
17         wsWrite(websocket.TextMessage, []byte(\"dial unix:@minicap\"))
18         log.Printf(\"minicap connection: %v\", r.RemoteAddr)
19         dataC := make(chan []byte, 10)
20         quitC := make(chan bool, 2)
21 
22         go func() {
23             defer close(dataC)
24             retries := 0
25             for {
26                 if retries > 10 {
27                     log.Println(\"unix @minicap connect failed\")
28                     dataC <- []byte(\"@minicap listen timeout, possibly minicap not installed\")
29                     break
30                 }
31                 conn, err := net.Dial(\"unix\", \"@minicap\")
32                 if err != nil {
33                     retries++
34                     log.Printf(\"dial @minicap err: %v, wait 0.5s\", err)
35                     select {
36                     case <-quitC:
37                         return
38                     case <-time.After(500 * time.Millisecond):
39                     }
40                     continue
41                 }
42                 dataC <- []byte(\"rotation \" + strconv.Itoa(deviceRotation))
43                 retries = 0 // connected, reset retries
44                 if er := translateMinicap(conn, dataC, quitC); er == nil {
45                     conn.Close()
46                     log.Println(\"transfer closed\")
47                     break
48                 } else {
49                     conn.Close()
50                     log.Println(\"minicap read error, try to read again\")
51                 }
52             }
53         }()
54         go func() {
55             for {
56                 if _, _, err := ws.ReadMessage(); err != nil {
57                     quitC <- true
58                     break
59                 }
60             }
61         }()
62         var num int = 0
63         //遍历管道循环发送数据
64         for data := range dataC {
65             //丢弃一半的数据包降低帧率
66             if string(data[:2]) == \"\\xff\\xd8\" { // jpeg data
67                 if num %2 == 0{
68                     num ++
69                     continue
70                 }
71                 if err := wsWrite(websocket.BinaryMessage, data); err != nil {
72                     break
73                 }
74                 if err := wsWrite(websocket.TextMessage, []byte(\"data size: \"+strconv.Itoa(len(data)))); err != nil {
75                     break
76                 }
77             } else {
78                 if err := wsWrite(websocket.TextMessage, data); err != nil {
79                     break
80                 }
81             }
82         }
83         quitC <- true
84         log.Println(\"stream finished\")
85     })).Methods(\"GET\")

　　大致逻辑是当有客户端和agent的minicap接口建立websocket连接后会先开启一个goroutine来和minicap进行通信，并将minicap返回的数据存放到dataC中，然后for循环遍历该管道取出

所有数据，如果是图片格式直接通过websocket传输到客户端进行展示

流量优化:

　　1.帧率优化

1                 if num %2 == 0{
2                     num ++
3                     continue
4                 }

　　考虑到网络流量造成的带宽问题在这里做了一些小小优化对minicap返回的图片丢弃一半来达到优化效果

　　2.图片质量优化

1     //降低图片画质
2     service.Add(\"minicap\", cmdctrl.CommandInfo{
3         Environ: []string{\"LD_LIBRARY_PATH=/data/local/tmp\"},
4         Args: []string{\"/data/local/tmp/minicap\", \"-S\", \"-P\",
5             fmt.Sprintf(\"%dx%d@%dx%d/0\", width, height, displayMaxWidthHeight, displayMaxWidthHeight),
6         \"-Q\", \"50\"},
7     })

　　在atx-agent项目源码main.go中有启动minicap的脚本命令其中-Q 为图片质量范围在(0-100)之间详细解释在minicap源码中我在这里设置为50 大概传输200张图片在4M，从而缓解网络占用问题参考文章(https://www.jianshu.com/p/5b5fef0241af)

2.minitouch

　　简介:

　　跟minicap一样，minitouch也是用NDK开发的，跟minicap使用方法类似，不过它只要上传一个minitouch文件就可以了。对应的文件路径树跟minicap一样就不重复
介绍(不过它只需要对应不同的CPU的ABI，而不需要对应SDK版本)。实际测试这个触摸操作和minicap一样，实时性很高没什么卡顿。

atx-agent minitouch部分

 1 m.HandleFunc(\"/minitouch\", singleFightNewerWebsocket(func(w http.ResponseWriter, r *http.Request, ws *websocket.Conn) {
 2         defer ws.Close()
 3         const wsWriteWait = 10 * time.Second
 4         wsWrite := func(messageType int, data []byte) error {
 5             ws.SetWriteDeadline(time.Now().Add(wsWriteWait))
 6             return ws.WriteMessage(messageType, data)
 7         }
 8         wsWrite(websocket.TextMessage, []byte(\"start @minitouch service\"))
 9         if err := service.Start(\"minitouch\"); err != nil && err != cmdctrl.ErrAlreadyRunning {
10             wsWrite(websocket.TextMessage, []byte(\"@minitouch service start failed: \"+err.Error()))
11             return
12         }
13         wsWrite(websocket.TextMessage, []byte(\"dial unix:@minitouch\"))
14         log.Printf(\"minitouch connection: %v\", r.RemoteAddr)
15         retries := 0
16         quitC := make(chan bool, 2)
17         operC := make(chan TouchRequest, 10)
18         defer func() {
19             wsWrite(websocket.TextMessage, []byte(\"unix:@minitouch websocket closed\"))
20             close(operC)
21         }()
22         go func() {
23             for {
24                 if retries > 10 {
25                     log.Println(\"unix @minitouch connect failed\")
26                     wsWrite(websocket.TextMessage, []byte(\"@minitouch listen timeout, possibly minitouch not installed\"))
27                     ws.Close()
28                     break
29                 }
30                 conn, err := net.Dial(\"unix\", \"@minitouch\")
31                 if err != nil {
32                     retries++
33                     log.Printf(\"dial @minitouch error: %v, wait 0.5s\", err)
34                     select {
35                     case <-quitC:
36                         return
37                     case <-time.After(500 * time.Millisecond):
38                     }
39                     continue
40                 }
41                 log.Println(\"unix @minitouch connected, accepting requests\")
42                 retries = 0 // connected, reset retries
43                 err = drainTouchRequests(conn, operC)
44                 conn.Close()
45                 if err != nil {
46                     log.Println(\"drain touch requests err:\", err)
47                 } else {
48                     log.Println(\"unix @minitouch disconnected\")
49                     break // operC closed
50                 }
51             }
52         }()
53         var touchRequest TouchRequest
54         //轮询
55         for {
56             err := ws.ReadJSON(&touchRequest)
57             if err != nil {
58                 log.Println(\"readJson err:\", err)
59                 quitC <- true
60                 break
61             }
62             select {
63             case operC <- touchRequest:
64             //两秒钟
65             case <-time.After(2 * time.Second):
66                 wsWrite(websocket.TextMessage, []byte(\"touch request buffer full\"))
67             }
68         }
69     })).Methods(\"GET\")

 1 func drainTouchRequests(conn net.Conn, reqC chan TouchRequest) error {
 2     var maxX, maxY int
 3     var flag string
 4     var ver int
 5     var maxContacts, maxPressure int
 6     var pid int
 7 
 8     lineRd := lineFormatReader{bufrd: bufio.NewReader(conn)}
 9     lineRd.Scanf(\"%s %d\", &flag, &ver)
10     lineRd.Scanf(\"%s %d %d %d %d\", &flag, &maxContacts, &maxX, &maxY, &maxPressure)
11     if err := lineRd.Scanf(\"%s %d\", &flag, &pid); err != nil {
12         return err
13     }
14 
15     log.Debugf(\"handle touch requests maxX:%d maxY:%d maxPressure:%d maxContacts:%d\", maxX, maxY, maxPressure, maxContacts)
16     go io.Copy(ioutil.Discard, conn) // ignore the rest output
17     var posX, posY int
18     for req := range reqC {
19         var err error
20         switch req.Operation {
21         case \"r\": // reset
22             _, err = conn.Write([]byte(\"r\\n\"))
23         case \"d\":
24             fallthrough
25         case \"m\":
26             //计算点击位置   req.PercentX 前端传过来的值 乘 最大x值
27             posX = int(req.PercentX * float64(maxX))
28             posY = int(req.PercentY * float64(maxY))
29             pressure := int(req.Pressure * float64(maxPressure))
30             if pressure == 0 {
31                 pressure = maxPressure - 1
32             }
33             line := fmt.Sprintf(\"%s %d %d %d %d\\n\", req.Operation, req.Index, posX, posY, pressure)
34             log.Debugf(\"write to @minitouch %v\", line)
35             _, err = conn.Write([]byte(line))
36         case \"u\":
37             _, err = conn.Write([]byte(fmt.Sprintf(\"u %d\\n\", req.Index)))
38         case \"c\":
39             _, err = conn.Write([]byte(\"c\\n\"))
40         case \"w\":
41             _, err = conn.Write([]byte(fmt.Sprintf(\"w %d\\n\", req.Milliseconds)))
42         default:
43             err = errors.New(\"unsupported operation: \" + req.Operation)
44         }
45         if err != nil {
46             return err
47         }
48     }
49     return nil
50 }

大致逻辑为接收客户端发送过来的json数据并将数据存储到operC管道中, 开启一个goroutine来和minitouch建立连接并根据不同的类型来执行不同的操作

atx-agent minicap、minitouch源码分析

Redis集群管理

vue路由对不同界面进行传参及跳转的总结