Babies have developing immune systems, making them highly vulnerable to infections. Even the smallest traces of bacteria on feeding bottles or nipples can cause stomach infections, diarrhea, or other illnesses.