serial: use tty_port pointers in the core code

Extract out a lot of the x.port. uses and also show up where there are
things left to be isolated that prevent use using the port helpers in the
serial layer at this point
Signed-off-by: NAlan Cox <alan@linux.intel.com>