Even in quite basic experiments (50 participants, 20 rounds, 10 pages) the response times are quite high: participants have to wait up to 4s for the page to load. I know it's very hard to diagnose where the bottleneck is, so I was wondering what other's experiences with this are, and if can be that oTree is not really capable of managing multiple requests at the same time

I am running my oTree server on a really basic system (1 core w/ 2300 MHz, 2 GB RAM, Postgres) and I can confirm that at least 70 parallel subjects pose no issue whatsoever, response times and CPU/RAM usage are minimal.

It is impossible to answer this question in general. Just like with any software, oTree’s performance depends on the complexity of the calculations that it needs to perform. It also depends on the complexity of the HTML code that it is asked to generate.

Let me illustrate this with an example:

It is good practice to provide participants with an overview/feedback screen at the end of an experiment so that it is transparent to the participants how their remuneration is calculated.

In one project of mine this overview screen involved the generation of 9 tables with more than 120 rows each for each participant. In each table row, one of two options was highlighted, depending on the respective participant’s decisions. Hence, oTree had to generate around 1,100 table rows per participant, it had to retrieve around 2,200 values from the database per participant, and it had to apply conditional formatting to all the associated HTML elements.

Initially, all participants proceeded to this overview screen at the same time. Even with a mere 30 participants, this meant that participants had to wait around 30 seconds until their overview page was displayed.

(Everything worked fine, but it was slow. To avoid this sluggishness, we decided to move a questionnaire from after the overview page to right before the overview page. Since it took each participant a different time to complete the questionnaire, the excessive CPU load caused by simultaneously generating the overview pages for 30 participants could be avoided, and the waiting time was reduced to a couple of seconds per participant.)

That is, if you have some rather involved calculations that oTree needs to perform before a page is being displayed, a delay is not surprising — in particular, if you combine such calculations with wait pages. Similarly, large image or video files that all clients attempt to access simultaneously may create bottlenecks.

Then I don't understand: with fairly simple experiments a Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz with 16 cores, 64 GB of RAM struggles to serve to 100 concurrent participants in even simple experiments. I see the otree process spikes to 99% CPU usage (of a single core, of course), that's why I was wondering if otree was the culprit