Can I set a hot key for a certain slot permanently so that even when I move the slot the hot key remains the same?

Yes. Right click on the hot key button left to the audio file´s name. A context menu appears. Click on "Lock Hotkey". Now the hot key is permanently associated to this slot.