Babies have delicate immune systems that are still developing, which makes them vulnerable to germs and infections. Feeding bottles, nipples, and breast pump parts can harbor harmful bacteria if not cleaned properly.