Yes, some babies are born with teeth, known as natal teeth.
According to the provided reference, natal teeth are teeth that are already present at birth. These are distinct from neonatal teeth, which emerge within the first 30 days of life.
Therefore, while most babies do not have teeth at birth, the occurrence of natal teeth means that the answer to the question is yes, some births do have teeth. The reference information also says that baby teeth (deciduous or milk teeth) and permanent teeth have fairly well-defined times of eruption.