Just four of the nine Supreme Court Justices attend SOTU