The recently proposed phase-matching quantum key distribution offers means to overcome the linear rate-transmittance bound. Since information is encoded onto phases of coherent states, misalignment between two remote reference frames would yield errors and significantly degrade generation rate from ideal case. In this work, we propose a reference-frame-independent design by introducing high-dim...