Created by: linrongyi

我split了1000条样本作为dev set, 可是在训练过程中, 打印出来的tester信息显示有100多W条. 在test过程中, 停顿了许久, 所以应该是eval了这么多条数据.

我训练的命令是

${TRAINER_BIN} \
	--job=train \
	--config=trainer_config.conf \
	--save_dir=output \
	--trainer_count=11 \
	--use_gpu=0 \
	--save_dir=./output.${TAG} \
	--dot_period=100 \
	--log_period=1000 \
	--test_period=10000 \
	--num_passes=15 \
	--init_model_path=./model.init \
	--load_missing_parameter_strategy=rand \
	--test_wait=1  --show_parameter_stats_period=1000

打印出来的log

I1112 16:08:26.652833  8142 TrainerInternal.cpp:182]  Pass=0 Batch=15091 samples=1931613 AvgCost=0.481067 Eval: err_rate_out1=0.0968464 
I1112 16:09:07.795814  8142 Tester.cpp:127]  Test samples=1248750 cost=0.488171 Eval: err_rate_out1=0.0605288 

我在dataprovider里面打了debug, 确认data_provider只load了1000条数据

0 insts loaded
0 insts loaded
[999] instances loaded from /home/aladdin/paddle_tasks/tagging/data/caipu.refined.test [load结束打出的日志]
100000 insts loaded
200000 insts loaded
300000 insts loaded
400000 insts loaded
500000 insts loaded
I1112 16:04:13.903460  8165 ThreadLocal.cpp:37] thread use undeterministic rand seed:8166
...600000 insts loaded
....700000 insts loaded