Hi all,

I am trying to use the Assistant through ChatGPT API. However, sometimes Heroku breaks down and shows H12 Request Timeout error as Heroku automatically terminates any request that takes more than 30 seconds. I am not sure if this is due to my code or ChatGPT. From my side, because the Assistant may generate multiple responses to one user input, so I need to regularly check if there is any new message from the API. I wonder if putting this task to background would solve the problem, but looks like it will use the worker dyno? Any thoughts would be appreciated!

Best,
Mashu

I will say I have not had this issue. Can I ask what model you are using and where you are calling it in oTree? Does the error occur if you are just sending one response per user input?

Hi clintmckenna, thanks for replying. I used 4o. I guess the errors may due to I asked the Assistant to do some calculation which takes time. However I am not sure if this is the issue. In my code, sending messages and checking respnses are independent functions. After a message is sent, the run id is recorded, and then message checking will be regularly called to see if there is any new message from this run id. Looks like it shouldn't cause the timeout error.

Hmm, I would agree, I don't think it should cause the error so it might just be a heroku thing. If your requests need to take longer than 30 seconds, could you maybe try moving the api call to the client page with javascript? Maybe you could run the request and then send it to the server via live pages when it is done? Might be less reliable, but then you could know exactly when new messages come in and could trigger some action when it happens. I am not sure that running background processes the way you describe in your first message is supported in oTree, but someone could correct me.

Hi clintmckenna, calling the api at the client page sounds risky as users may refresh pages any time. I couldn't figure out why but decided to tell users that they may refresh page and send messages again if chatgpt is not responding. If anyone has better way to do it, let me know!