My problem is that the payoff in otree studio is always rounded. If the payoff should be 0.5, otreestudio indicates a payoff of 1.

Probably, it says, USE_POINTS = True in your settings.py file.

In that case, oTree stores the payoff as an integer:

> Decimal places
>
> Money amounts are displayed with 2 decimal places.
>
> On the other hand, points are integers. This means amounts will get rounded to whole numbers, like 10 divided by 3 is 3. 

See https://otree.readthedocs.io/en/latest/currency.html#decimal-places.