Tensorflow serving启动方式说明

1. 介绍地址：https://github.com/tensorflow/serving

   docker安装（推荐）	https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/docker.md
   非docker	https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md
   docker中源码编译	https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/building_with_docker.md
   k8s部署	https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/serving_kubernetes.md

    

    

2. docker基础镜像地址：https://hub.docker.com/r/tensorflow/serving/tags/?page=1&ordering=last_updated

   1. cpu安装镜像，对应Dockerfile

   2. cpu开发镜像devel，对应Dockerfile.devel
   3. gpu安装镜像，对应Dockerfile.gpu，使用nvidia-docker支持GPU
   4. gpu开发镜像，对应Dockerfile.devel-gpu

3. 以cpu为例，演示快速部署

   # Download the TensorFlow Serving Docker image and repo

   docker pull tensorflow/serving

    

   git clone https://github.com/tensorflow/serving

   # Location of demo models

   TESTDATA="$(pwd)/serving/tensorflow_serving/servables/tensorflow/testdata"

    

   # Start TensorFlow Serving container and open the REST API port

   docker run -t --rm -p 8501:8501 \

       -v "$TESTDATA/saved_model_half_plus_two_cpu:/models/half_plus_two" \

       -e MODEL_NAME=half_plus_two \

       tensorflow/serving &

    

   # Query the model using the predict API

   curl -d '{"instances": [1.0, 2.0, 5.0]}' \

       -X POST http://localhost:8501/v1/models/half_plus_two:predict

    

   # Returns => { "predictions": [2.5, 3.0, 4.5] }

   其中8500默认为rpc端口，8501默认为REST端口，可以同时开启。环境变量MODEL_NAME指向模型目录（默认model），MODEL_BASE_PATH指向模型目录所在根目录（默认/models）

   同时还可以传递额外参数，如指定配置文件：

   docker run -p 8500:8500 -p 8501:8501 \

     --mount type=bind,source=/path/to/my_model/,target=/models/my_model \

     --mount type=bind,source=/path/to/my/models.config,target=/models/models.config \

     -t tensorflow/serving --model_config_file=/models/models.config

4. cpu镜像启动过程详解：

   1. ENTRYPOINT：/usr/bin/tf_serving_entrypoint.sh

      #!/bin/bash

       

      tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"

   2. bin文件路径：/usr/bin/tensorflow_model_server

   3. 模型目录通过环境变量MODEL_BASE_PATH和MODEL_NAME传递
5. 开发调试方法：
   1. 在devel镜像中，支持通过bin文件启动server，并通过python请求调试
6. 支持通过给定的Dockerfile编译自己的镜像版本：https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/building_with_docker.md