Created by: Fangkey

基于CTR模型的Demo修改建立DNN网络进行Regression预测，训练结果不收敛，infer的结果也不正确。帮忙排查原因。

训练输出如下：

WARNING:paddle:Pass 0, Samples 0, Cost 11620.388000, {}
WARNING:paddle:Test 0-0, Cost 5308.589006, {}
WARNING:paddle:Pass 1, Samples 0, Cost 12167.511000, {}
WARNING:paddle:Test 1-0, Cost 5269.448265, {}
WARNING:paddle:Pass 2, Samples 0, Cost 12159.679000, {}
WARNING:paddle:Test 2-0, Cost 5264.473139, {}
WARNING:paddle:Pass 3, Samples 0, Cost 12154.542000, {}
WARNING:paddle:Test 3-0, Cost 5261.147334, {}
WARNING:paddle:Pass 4, Samples 0, Cost 12151.005000, {}
WARNING:paddle:Test 4-0, Cost 5258.846167, {}
WARNING:paddle:Pass 5, Samples 0, Cost 12148.595000, {}
WARNING:paddle:Test 5-0, Cost 5257.272729, {}
WARNING:paddle:Pass 6, Samples 0, Cost 12146.910000, {}
WARNING:paddle:Test 6-0, Cost 5256.165883, {}
WARNING:paddle:Pass 7, Samples 0, Cost 12145.686000, {}
WARNING:paddle:Test 7-0, Cost 5255.357524, {}
WARNING:paddle:Pass 8, Samples 0, Cost 12144.765000, {}
WARNING:paddle:Test 8-0, Cost 5254.747666, {}
WARNING:paddle:Pass 9, Samples 0, Cost 12144.054000, {}
WARNING:paddle:Test 9-0, Cost 5254.274953, {}

模型相关代码：

# 模型定义
class DNNmodel(object):
    def __init__(self,
                 dnn_layer_dims,
                 dnn_input_dim,
                 is_infer=False):
        self.dnn_layer_dims = dnn_layer_dims
        self.dnn_input_dim = dnn_input_dim
        self.is_infer = is_infer

        self._declare_input_layers()
        self.dnn = self._build_dnn_submodel_(self.dnn_layer_dims)
        self.model = self._build_regression_model(self.dnn)

    def _declare_input_layers(self):
        self.dnn_merged_input = layer.data(
            name='dnn_input',
            type=dtype.sparse_vector(self.dnn_input_dim))

        if not self.is_infer:
            self.target = paddle.layer.data(
                name='target', type=dtype.dense_vector(1))

    def _build_dnn_submodel_(self, dnn_layer_dims):
        dnn_embedding = layer.fc(
            input=self.dnn_merged_input, size=dnn_layer_dims[0])
        _input_layer = dnn_embedding
        for i, dim in enumerate(dnn_layer_dims[1:]):
            fc = layer.fc(
                input=_input_layer,
                size=dim,
                act=paddle.activation.Relu(),
                name='dnn-fc-%d' % i)
            _input_layer = fc
        return _input_layer

    def _build_regression_model(self, dnn):
        self.output = layer.fc(
            input=dnn, size=1, act=paddle.activation.Sigmoid())
        if not self.is_infer:
            self.train_cost = paddle.layer.square_error_cost(
                input=self.output, label=self.target)
        return self.output

模型训练

dnn_layer_dims = [128, 64, 32, 1]
def train():
    args = parse_args()
    paddle.init(use_gpu=False, trainer_count=1)
    dnn_input_dim = reader.load_data_meta(args.data_meta_file)

    # create ctr model.
    model = DNNmodel(
        dnn_layer_dims,
        dnn_input_dim,
        is_infer=False)

    params = paddle.parameters.create(model.train_cost)
    optimizer = paddle.optimizer.AdaGrad()

    trainer = paddle.trainer.SGD(
        cost=model.train_cost, parameters=params, update_equation=optimizer)

    dataset = reader.Dataset()

    def __event_handler__(event):
        if isinstance(event, paddle.event.EndIteration):
            num_samples = event.batch_id * args.batch_size
            if event.batch_id % 100 == 0:
                logger.warning("Pass %d, Samples %d, Cost %f, %s" % (
                    event.pass_id, num_samples, event.cost, event.metrics))

            if event.batch_id % 1000 == 0:
                if args.test_data_path:
                    result = trainer.test(
                        reader=paddle.batch(
                            dataset.test(args.test_data_path),
                            batch_size=args.batch_size),
                        feeding=reader.feeding_index)
                    logger.warning("Test %d-%d, Cost %f, %s" %
                                   (event.pass_id, event.batch_id, result.cost,
                                    result.metrics))

                path = "{}-pass-{}-batch-{}-test-{}.tar.gz".format(
                    args.model_output_prefix, event.pass_id, event.batch_id,
                    result.cost)
                with gzip.open(path, 'w') as f:
                    params.to_tar(f)

    trainer.train(
        reader=paddle.batch(
            paddle.reader.shuffle(
                dataset.train(args.train_data_path), buf_size=500),
            batch_size=args.batch_size),
        feeding=reader.feeding_index,
        event_handler=__event_handler__,
        num_passes=args.num_passes)

Created by: Fangkey

特征说明： 训练任务是根据超市以往的销售情况，预测未来三天的销量。 特征有两大类构成，一种是数值类型，例如今天的总销量，取值是几十到几万的整数；一种是类别类型，例如门店的加盟类型，今天是星期几，今天的天气状况是阴晴雨雪。每种特征的类别不超过30种。 两类特征统一进行ID编址。 目标值是某一大类商品的销量，取值为几十到几百的整数。

预处理方法： 对于数值型特征，针对该特征自身的均值和标准差，进行z-score标准化。对于类别类型的特征变成one-hot，只提取取值为1的特征ID。 对目标值不进行处理 以下为输入特征的示例

38	0:2.28118198733,1:0.202314159427,2:0.0886267024849,3:-0.550001470584,4:-0.121215078255,5:0.298959864421,6:0.603720412001,7:1.65992918385,8:1.78968294362,9:1.87105126401,10:0.327171927795,11:0.675113464698,12:1.50642678128,13:1.50290365956,15:1,22:1,23:1.49611192991,24:1.68031073732,25:1,36:1,51:1,55:1,63:1.38721049503,64:1.68149695891,65:1,76:1,87:1,95:1,103:1.49528794109,104:1.68205286916,105:1,116:1,127:1,135:1,143:1.38789944247,144:1.5752212519,145:1,156:1,167:1,175:1,183:1.38782201419,184:1.46596448655,185:1,196:1,207:1,215:1,223:1.49349738674,224:1.35509928631,225:1,236:1,247:1,255:1,263:1.38733934703,264:1.24545908959,265:1,279:1,287:1,295:1,308:1,314:1,320:1,326:1,332:1,338:1,351:1,357:1,363:1,369:1,373:1,378:1,383:1,388:1,393:1,398:1,403:1,408:1,413:1,418:1
94	0:0.201020560678,1:0.0841369763735,2:-0.550540789964,3:-0.123659895013,4:0.307055875027,5:0.606352484813,6:1.76129738569,7:1.78609175281,8:1.86803227438,9:0.330408421117,10:0.676416749734,11:1.50984827411,12:1.50165875143,13:1.86194890906,15:1,22:1,23:1.38831845595,24:1.68031073732,25:1,36:1,47:1,55:1,63:1.49489863846,64:1.68149695891,65:1,76:1,87:1,95:1,103:1.38758674996,104:1.57324976392,105:1,116:1,127:1,135:1,143:1.38789944247,144:1.46634910313,145:1,156:1,167:1,175:1,183:1.495520816,184:1.35700208568,185:1,196:1,207:1,215:1,223:1.38595970358,224:1.24629357039,225:1,239:1,247:1,255:1,263:1.17263014489,264:1.24545908959,265:1,276:1,287:1,295:1,307:1,313:1,319:1,325:1,331:1,344:1,350:1,356:1,362:1,368:1,373:1,378:1,383:1,388:1,393:1,398:1,403:1,408:1,413:1,418:1

Created by: wanghaoshuang

@Fangkey 训练和测试数据都是什么量级的？

Created by: Fangkey

@wanghaoshuang 训练集32000条数据，测试集8000条数据，特征维度400+维。

Created by: wanghaoshuang

@Fangkey 试试更浅层的网络，调整下learning rate？还有dropout rate.

Created by: shanyi15

您好，此issue在近一个月内暂无更新，我们将于今天内关闭。若在关闭后您仍需跟进提问，可重新开启此问题，我们将在24小时内回复您。因关闭带来的不便我们深表歉意，请您谅解~感谢您对PaddlePaddle的支持! Hello, this issue has not been updated in the past month. We will close it today for the sake of other user‘s experience. If you still need to follow up on this question after closing, please feel free to reopen it. In that case, we will get back to you within 24 hours. We apologize for the inconvenience caused by the closure and thank you so much for your support of PaddlePaddle Group!