The documentation says 

"
Browser bots: misc notes

You can use a browser other than Chrome by setting BROWSER_COMMAND in settings.py. Then, oTree will open the browser by doing something like subprocess.Popen(settings.BROWSER_COMMAND)."

But what does Otree expect?

BROWSER_COMMAND = Firefox 

does not work.

First, figure out the command you need to use to launch Firefox from your command line (terminal/powershell), then insert that as a string in your settings, e.g.:

BROWSER_COMMAND = '/path/to/firefox/'

also read about how to launch a browser using subprocess.Popen().