建置模型部署所需之映像檔 (Python)

此文件說明如何建置映像檔 (Docker Image) ，以利在 PrimeHub 模型部署中使用。

PrimeHub 模型部署功能是基於 Seldon 的開源套件。此文件參考 Seldon 和其他相關的文件，我們皆列在最後一個部分。

軟體需求

請先安裝好以下軟體

docker: https://www.docker.com/products/docker-desktop

撰寫模型部署程式與設定

產生 requirements.txt 檔，並寫入以下套件
```
seldon-core
keras
tensorflow
numpy
...
```

產生 Dockerfile 檔，並寫入以下內容

FROM python:3.7-slim
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 9000

# Define environment variable
ENV MODEL_NAME MyModel
ENV SERVICE_TYPE MODEL
ENV PERSISTENCE 0

CMD exec seldon-core-microservice $MODEL_NAME --service-type $SERVICE_TYPE --persistence $PERSISTENCE --access-log

建立 MyModel.py 檔，內容可以參考以下的格式內容

class MyModel(object):
    """
    Model template. You can load your model parameters in __init__ from a location accessible at runtime
    """

    def __init__(self):
        """
        Add any initialization parameters. These will be passed at runtime from the graph definition parameters defined in your seldondeployment kubernetes resource manifest.
        """
        print("Initializing")

    def predict(self, X, features_names=None):
        """
        Return a prediction.

        Parameters
        ----------
        X : array-like
        feature_names : array of feature names (optional)
        """
        print("Predict called - will run identity function")
        return X

檔名和類別名稱 MyModel 需與在 Dockerfile 下的 MODEL_NAME 一致
你可以在 __init__ 初始化或載入你的模型
predict 會接受 numpy-array X 及 feature_names (非必須)，並回傳預測結果 (至少需為二維陣列)

更多如何撰寫 MyModel.py 的範例和細節，可以參考 相關 Frameworks 的範例程式 。

建置映像檔

進入到包含 requirements.txt 、 Dockerfile 、 python file for model deployment 、 model file 的資料夾。

建置模型部署映像檔 my-model-image：

docker build . -t my-model-image

透過 docker images 檢查建置完成的映像檔，範例結果如下所示：

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
my-model-image      latest              f373fdcc10c5        3 minutes ago       2.46GB
python              3.7-slim            ea12296513d7        2 weeks ago         112MB

測試映像檔

為了確保模型映像檔可於後續的模型部署中使用，你可先在本機上透過 Docker 運行 container:

docker run -p 9000:9000 --rm my-model-image

透過 curl 進行測試：

    curl -X POST localhost:9000/api/v1.0/predictions \
        -H 'Content-Type: application/json' \
        -d '{ "data": { "ndarray": [[5.964,4.006,2.081,1.031]]}}'

到此我們已經成功建置出可以給 PrimeHub 模型部署功能使用的映像檔。

推送映像檔

接下來請將其推送到 docker hub (或其他 docker registry) ，並參考 PrimeHub 的文件繼續將模型部署到 PrimeHub 上

標記模型映像檔:

docker tag my-model-image test-repo/my-model-image

推送至 docker registry:

docker push test-repo/my-model-image

(Optional) 相關 Frameworks 的範例程式

參考以下的範例， PrimeHub 可以部署各種 Frameworks 的模型。

Tensorflow 1

Output model example code

    saver = tf.train.Saver()
    saver.save(sess, "model/deep_mnist_model")

MyModel.py example code

    import tensorflow as tf
    import numpy as np
    import os
    
    class DeepMnist(object):
        def __init__(self):
            self.loaded = False
            
        def load(self):
            print("Loading model",os.getpid())
            self.sess = tf.Session()
            saver = tf.train.import_meta_graph("model/deep_mnist_model.meta")
            saver.restore(self.sess,tf.train.latest_checkpoint("./model/"))
            graph = tf.get_default_graph()
            self.x = graph.get_tensor_by_name("x:0")
            self.y = graph.get_tensor_by_name("y:0")
            self.loaded = True
            print("Loaded model")
            
        def predict(self,X,feature_names):
            if not self.loaded:
                self.load()
            predictions = self.sess.run(self.y,feed_dict={self.x:X})
            return predictions.astype(np.float64)

Tensorflow 2

Output model example code
```
    model.save("1")
```

MyModel.py example code

    import tensorflow as tf
    
    class MNISTModel:
       def __init__(self):
            self.loaded = False

        def load(self):
            self._model = tf.keras.models.load_model('1')
            self.loaded = True
    
        def predict(self, X, feature_names=None, meta=None):
            if not self.loaded:
                self.load()
            output = self._model.predict(X)
            probability = output[0]
            predicted_number = tf.math.argmax(probability)
            return {"predicted_number": predicted_number.numpy().tolist(), "probability": probability.tolist()}

Keras

Output model example code
```
    model.save('keras-mnist.h5')
```

MyModel.py example code

    from keras.models import load_model
    from PIL import Image
    from io import BytesIO
    import numpy as np
    
    class MyModel(object):
        def __init__(self):
            self.loaded = False

        def load(self):
            self.model = load_model('keras-mnist.h5')
            self.loaded = True
            
        def predict(self,X,features_names):
            if not self.loaded:
                self.load()
            imageStream = BytesIO(X)
            image = Image.open(imageStream).resize((28, 28)).convert('L')
            data = np.asarray(image)
            data = np.expand_dims(data, axis=0)
            data = np.expand_dims(data, axis=-1)
            return self.model.predict(data)

Scikit-learn

Output model example code
```
    joblib.dump(p, filename_p)
```

MyModel.py example code

    from sklearn.externals import joblib
    
    class IrisClassifier(object):
    
        def __init__(self):
            self.model = joblib.load('IrisClassifier.sav')
    
        def predict(self,X,features_names):
            return self.model.predict_proba(X)

Pytorch

Output model example code

    torch.save(model.state_dict(), "mnist_cnn.pt")

MyModel.py example code

    import torch
    import torch.nn as nn
    import torch.nn.functional as F
    
    class Net(nn.Module):
        def __init__(self):
            super(Net, self).__init__()
            self.conv1 = nn.Conv2d(1, 32, 3, 1)
            self.conv2 = nn.Conv2d(32, 64, 3, 1)
            self.dropout1 = nn.Dropout2d(0.25)
            self.dropout2 = nn.Dropout2d(0.5)
            self.fc1 = nn.Linear(9216, 128)
            self.fc2 = nn.Linear(128, 10)
    
        def forward(self, x):
            x = self.conv1(x)
            x = F.relu(x)
            x = self.conv2(x)
            x = F.relu(x)
            x = F.max_pool2d(x, 2)
            x = self.dropout1(x)
            x = torch.flatten(x, 1)
            x = self.fc1(x)
            x = F.relu(x)
            x = self.dropout2(x)
            x = self.fc2(x)
            output = F.softmax(x, dim=1)
            return output
    
    class MNISTModel:
        def __init__(self):
            self._model = Net()
            self._model.load_state_dict(torch.load("mnist_cnn.pt"))
            self._model.eval()
    
        def predict(self, x, names):
            output = self._model(torch.from_numpy(x).float())
            return {"probability": output.tolist()}

XGBoost

Output model example code

    bst = xgb.train(...)
    bst.save_model('xgboost.model')

MyModel.py example code

    import xgboost as xgb
    
    class MyModel(object):
        def __init__(self):
            self.bst = xgb.Booster({'nthread':4})
            self.bst.load_model("xgboost.model") 
            
        def predict(self,X,features_names):
            dtest = xgb.DMatrix(X)
            return self.bst.predict(dtest)

MXNet

Output model example code

    model_prefix = 'mx-model'
    checkpoint = mx.callback.do_checkpoint(model_prefix)
    mod.fit(..., epoch_end_callback=checkpoint)

MyModel.py example code

    import mxnet as mx
    from PIL import Image
    import numpy as np
    from io import BytesIO
    
    class MyModel(object):
        def __init__(self):
            model_prefix = 'mx-model'
            epoch_num = 2
            self.model = mx.mod.Module.load(model_prefix, epoch_num)
            
            data_shape = [("data", (1, 28, 28, 1))]
            label_shape = [("softmax_label", (1,))]
            self.model.bind(data_shape, label_shape)
    
        def predict(self,X,features_names):
            imageStream = BytesIO(X)
            image = Image.open(imageStream).resize((28, 28)).convert('L')
    
            data = np.asarray(image)
            data = np.expand_dims(data, axis=0)
            data = np.expand_dims(data, axis=-1)
            return self.model.predict(data).asnumpy()

LightGBM

Output model example code

    gbm = lgb.train(...)
    with open('model.pkl', 'wb') as fout:
        pickle.dump(gbm, fout)

MyModel.py example code

    import pickle
    
    class MyModel(object):
        def __init__(self):
            with open('model.pkl', 'rb') as fin:
                self.pkl_bst = pickle.load(fin)
    
        def predict(self,X,features_names):
            return self.pkl_bst.predict(X)

PrimeHub

3.8

Notebook Extension

教學

Tutorials

Pre-packaged servers

Package from Language Wrapper 教學

Advanced

更多資訊

使用者及群組管理

Instance Type 管理

Image 創建及管理

Dataset 管理及上傳

Secret 管理及取得

Notebook Instance 管理

使用量月報表

系統

維運工具

快速上手

建置模型部署所需之映像檔 (Python)

軟體需求

撰寫模型部署程式與設定

建置映像檔

測試映像檔

推送映像檔

(Optional) 相關 Frameworks 的範例程式

Tensorflow 1

Tensorflow 2

Keras

Scikit-learn

Pytorch

XGBoost

MXNet

LightGBM

Reference