We shouldn't mark the file accessed in the case that it
wasn't accessed.
Signed-off-by: NSteven Whitehouse <swhiteho@redhat.com>