Turn off power to the circuit and remove the fuse
Set the multimeter to continuity mode or the lowest resistance setting
Touch the multimeter probes together to confirm it works
Place one probe on each end of the fuse
Read the meter display
A beep or near-zero resistance means the fuse is good
No beep or infinite resistance means the fuse is blown
Replace the fuse with one of the same rating if it is blown
Reinstall the fuse after testing if it is good