Gather 8 paper and 1 compass
Open a crafting table
Place the compass in the center
Place the 8 paper around the compass
Take the empty map
Open a cartography table
Place the map in the first slot
Add paper to the second slot to zoom out the map
Repeat adding paper to make the map bigger
Use the map in the area you want to explore
Keep exploring to fill in more of the map
Use additional maps for neighboring areas if needed