Email responses are going to the junk folder

We are having an issue where customers will respond to a quote or a general email and their response is going to our junk folder. This started happening with many clients across multiple email service providers, but it only happens if they are responding to emails that were sent through Infusionsoft, not directly from our email system.
Any thoughts?