Open the Phone app
Tap Voicemail
Listen to or open the voicemail notification
Delete the voicemail message
Clear the voicemail notification from the notification shade
Restart the phone if the notification remains
Contact your carrier if the voicemail icon still does not disappear