Commit dc98a5bc refactored the code a lot and forget about checking if
listen attribute is specified.  This ensures that listen attribute and
first listen element are compared only if both exist.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>