Working API app starting to time out

API Q&A
1

We have been running this API code for over a year now with little problem. However, starting a few weeks ago, we have been getting increasing errors cURL 28 timeout errors.

Essentially what we are doing is a query to retrieve a list of contacts (1000 at a time) that want particular email information and send it to them. There can be thousands who want this email. When that HTTP post completes (either succeeding or failing) we move on to the next call which does the same thing to a different set of contacts and a different requested email.

They have been adding about 2500 contacts per month and are not at 97,000. I was wondering if the growing number of contacts could be the cause or at least a contributing factor.

We tried doubling the timeout and reducing the number of contacts in the query to 200 at a time and this seemed to make the problem worse.

Any ideas? TIA!

2

Dom,

Which API endpoint are you using to make your call? What criteria are you specifying with it?

3

Thanks for the reply!

Essentially we are looking for contacts with a specific tag:

 $results = $api->data()->findByField('ContactGroupAssign', 1000, $page++, 'GroupId', (int)$tag_id, array('ContactId'));

Then we send a specific email to every contact with that tag.

4

I assume you meant how we get the contacts that we send the email to…if that is not the case, please let me know.

5

It looks like you are using the legacy XMLRPC DataService to look up the table that stores that information directly and paging through the results, which is probably why you’re having performance issues. As the number of records in that table increases, the legacy DataService (which has no abstraction layer) has to retrieve an ever-increasing set of results to provide the requested page.

We do have a much newer Rest endpoint that allows you to obtain those results directly via a more optimized query: Keap REST API

6

@Dom_Cassone, I can confirm with you that the querying the Contact Group Assign table can either be slow or non-responsive if it has a large number of Tag Assignments. It also depends on what data is being stored in the fields.

One reason that the REST API is faster is that is returns very few number of fields in comparison to the XML-RPC version. But if you need to get additional Contact data you will need to do additional queries via the API.

I recommend that you pull the data from Infusionsoft and Cache it into a Local Database. From there you can perform faster lookups and complex queries. You can create a Cron Job that can pull the Contact Records at regular intervals. The Contact Records do have the Tag Id references.