Open the Phone app
Check and listen to any new voicemail messages
Delete the voicemail messages
Clear the Phone app notification
Restart your phone
Turn voicemail notifications off in Phone app settings
Disable call forwarding if it is sending calls to voicemail
Reset network settings if the notification still stays
Contact your carrier if the voicemail icon will not disappear