Created by: reyoung

Problem

A global step counter to record how many times a program has executed is needed. It is needed not only by optimizers with learning rate schedulers but also by printing metrics or debugging(such as log_every_n layer).

Solution

We can add a layer function, get_or_create_global_step_counter() in layers.control_flow module. It will return an int64 variable in CPU with an increment operator. If the counter is created before, it will not be created again.