You are expected to create package pin constraints for each instantiated transceiver differential reference clock buffer primitive as well as each instantiated differential recovered clock output buffer primitive if used. The constraints reflect the transceiver primitive site locations.
The MGT reference clock frequency must be constrained at the Vivado Project top level XDC file at specified frequency, that is, for GTYE5 and GTYP:
create_clock -period 3.367 [get_ports HDMI_RX_CLK_IN_clk_p]
- HDMI 2.1
create_clock -period 2.500 [get_ports HDMI_RX_CLK_P_IN_V_clk_p]