Created by: willthefrog

some code could be pretty hacky, due to time and other limitations 🤷‍♂️ current reader does not support partition (status may change very soon), so each worker will loop through the whole dataset, make sure there are enough iterations.