Go to your Google Account
Select Personal info
Under Contact info, click Phone
Sign in if prompted
Select the phone number you want to remove
Click Delete or Remove
Confirm the removal
If the number is used for security, go to Security
Open 2-Step Verification or Recovery options
Replace the phone number with another number if required
Save changes