Vivado Hardware Manager supports debugging on DFX designs. To debug such a design successfully, it is necessary to program the full design bitstream before programming the partial bitstream to replace specific reconfigurable modules.
For an example of instantiating debug cores in a DFX design, as well as functionality within the Vivado Hardware Manager, see this link in Vivado Design Suite Tutorial: Dynamic Function eXchange (UG947).