Turn off power to the circuit and remove the fuse
Set the multimeter to continuity mode or the lowest resistance setting
Touch one probe to each end of the fuse
Check the reading
A beep or near-zero resistance means the fuse is good
No beep or infinite resistance means the fuse is blown
If testing in-circuit, make sure the circuit is de-energized first
Replace the fuse with one of the same type and rating if it is blown