oTree Forum >

Heroku - listen to multiple ports - reverse proxy

#1 by fabsi (edited )

Hi there,

My goal is to provide and use data from different sensors in oTree experiments.

To enable this functionality I implemented a WebSocket server. This runs as in a separate process (with the help of the multiprocessing package) and is started from within my oTree experiment.

In the local network this works without problems. oTree runs on localhost:8000 and the WebSocket server listens on localhost:8001.

Now I would like to deploy the application on Heroku. Unfortunately, this does not work without adjusting the routing. At least I think so ...

I thought about using a reverse proxy (e.g. with nginx) to connect the clients of the sensors to the WebSocket server.
Have any of you had success publishing nginx as a reverse proxy with/on top of oTree?

Is there another, maybe better, way to implement my desired goal?

Glad to get new input.

Thank you in advance.

#2 by Chris_oTree

Can you describe your use case more? What kind of sensors? And how does this data get used by oTree?

#3 by fabsi (edited )

Sensors can be heart rate monitors or photoelectric sensor, for example. A concrete example would be that player B performs a certain work step in a production line. Now player B is shown on a monitor at what speed player A is performing the same work step. This is automatically counted via a photoelectric sensor. The question now is how the performance of player A influences the performance of player B.

This is only a simple example. I would like to develop a "framework"/infrastructure that makes it as easy as possible for experimenters to integrate such sensor data into their experiments.

After the client software of a player connects to the WebSocket server, the client waits for instructions to start or stop recording when a certain interaction has been made in the oTree experiment. But other data can also be sent back and forth.

My problem is that when I deploy the application on Heroku, the oTree application works, but I have not yet found a working solution to connect to my WebSocket server which is running on a different port.

I hope that my explanation makes sense. Thanks :)

#4 by BonnEconLab

Hi fabsi,

You asked,

> Have any of you had success publishing nginx as a reverse proxy with/on top of oTree?

Yes. At the BonnEconLab, we use nginx as a reverse proxy for oTree to make all traffic between the participants’ devices and the oTree server use HTTPS. Here are the relevant parts from my internal manual to set up nginx with oTree:

Edit the nginx settings:

sudo nano /etc/nginx/sites-available/default

Change

server {
    listen 80 default_server;
    listen [::]:80 default_server;
}

to

server {
    listen 80;
    listen [::]:80;
    return 301 https://$host$request_uri;
}

and add the following lines:

map $http_upgrade $connection_upgrade { # Some variables for configuring the protocol switch
    default upgrade;
    ''      close;
}

as well as:

server {
    listen 443 ssl default_server;
    listen [::]:443 default_server;
    add_header Strict-Transport-Security max-age=31536000;
    ssl_session_cache shared:ssl_session_cache:10m;
    ssl_certificate /etc/letsencrypt/live/web51.bonneconlab.uni-bonn.de/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/web51.bonneconlab.uni-bonn.de/privkey.pem;
    root /home/administrator/oTree;
    ...
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1; # protocol switch needed
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade; # End of the configuration for oTree
        # add ssl port to all server addresses
        proxy_set_header HOST $host;
        proxy_set_header X-Real-Ip $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
    }
}

The central line is

    proxy_pass http://127.0.0.1:8000;

Specifying root via the line

    root /home/administrator/oTree;

is actually not necessary, I have realized in the meantime.

See also Joachim Gassen’s GitHub repository, in particular, https://github.com/trr266/otree_docker/blob/main/otree5_nginx_site.conf.

Best,

Holger

#5 by fabsi

Hi Holger, 

thank you very much for the detailed answer. I got Nginx working with oTree and my WebSocket server on my local machine. 

Do you deploy Nginx with oTree on your own infrastructure or have you ever deployed the whole thing to Heroku as well? 
If so, how?

I tried to get it running with a buildpack for Nginx. But without success so far. One challenge is that the official one from Heroku (https://elements.heroku.com/buildpacks/heroku/heroku-buildpack-nginx) requires your webserver to listen to the socket at '/tmp/nginx.socket'. How do I take that into account?

Any idea to make it work? That would be awesome. Thanks.

#6 by BonnEconLab

A quick follow-up on my previous post: You can also use nginx to allow participants to connect to multiple oTree instances on the same server.

A requirement is that the server can be reached via multiple domain names. (You will most likely have to ask your IT department to add the respective entries to the domain name server.)

Say, your server’s IP address is associated with two difference domains: firstsubdomain.example.org and secondsubdomain.example.org. Then simply add multiple server entries to the nginx config file so that each is associated with a different oTree port. For instance,

server {
    listen 443 ssl;
    listen [::]:443;
    server_name firstsubdomain.example.org;
    ...
    ssl_certificate /etc/letsencrypt/live/firstsubdomain.example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/firstsubdomain.example.org/privkey.pem;
    ...
    location / {
        proxy_pass http://127.0.0.1:8000;
        ...
    }
}

and

server {
    listen 443 ssl;
    listen [::]:443;
    server_name secondsubdomain.example.org;
    ...
    ssl_certificate /etc/letsencrypt/live/secondsubdomain.example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/secondsubdomain.example.org/privkey.pem;
    ...
    location / {
        proxy_pass http://127.0.0.1:8001;
    ...
    }
}

If you then execute both otree prodserver 8000 and otree prodserver 8001, both oTree instances can be accessed independently and simultaneously.

#7 by BonnEconLab

fabsi asked:

> Do you deploy Nginx with oTree on your own infrastructure or have you ever deployed the whole thing to Heroku as well?

I am only in charge of our own infrastructure. I have never used Heroku. Hence, I am unfortunately unable to help.

Write a reply

Set forum username