Hi everyone,

I am trying to scale a double-auction market using a live interaction and run into (server) performance issues at the latest with more than 100 participants. Hoping for the best, I am looking for advice on how to overcome my performance issue and make the market capable to accommodate 500-1000 traders who act simultaneously. Planning for the worst, I would at least like to share my experiences and shed some light on the performance impact of many people interacting in an oTree based experiment.

The setup: The market at hand is a based on the "Double auction market" featured on https://otree-more-demos.herokuapp.com/demo, code can be found here https://github.com/da-lbrecht/otree_EconomyGame2.0 . In essence, requests to the server are sent when participants take an action, i.e. placing or removing an offer. The server sends information to the participants, whenever something changed: offers were placed or deleted, trades took place, the market parameters changed. Additionally, an "admin participant" sends a request to the server every second, to send updated information to every player/trader (continuous time updates of marginal costs and marginal utilities to all buyers and sellers). All in all, more than 1 request/second is sent to the server and the server needs to send information to every participant at least once per second.

Experienced performance: around 50 participants work fine when hosted on Heroku (Performance L Dyno web worker, Standard-0 Heroku postgres database), 100 participants experience a small but noticeable lag, 250+ participants paralyse the server, i.e. server response times > 30 seconds and at least some web-sockets connections break down due to server timeouts.

Best wishes
David

That is quite a large live_method, and it looks like you are sending large amounts of data to each participant every second. A few suggestions:

- Do you need to update everyone once per second? If you could batch it to once every 5 seconds, that reduces the amount of traffic way down. You can still process every message that comes in, but just wait until 5 seconds have elapsed to send any big update.
- Do you need to send such large amounts of data? I see your variable live_data contains a lot of things, I wonder if all of these need to be re-sent on each message.
- put calls to time.time() inside your live_method, like every 10 lines. Then you can narrow down to understand what lines of code are taking the most time to execute. In particular lines like group.get_players().