oTree Forum >

MTurk participants stuck on wait page

#1 by lindaaldehoff

Hello everyone,

I finally finished my code and uploaded the experiment on MTurk using heroku. In the MTurk sandbox everything worked fine and when I play on my devserver it all works great as well.
But when I uploded my session to MTurk participants were stuck on the Grouping page, even though they should be playing with a bot after 3 minutes of waiting time. Even when I filled in for other players and we had enough people to create a group, participants were not advanced from the wait page. (I watched with the monitor function)

Does anyone know what happened here? 

Thank you for your help! 

Best,

Linda

#2 by Daniel_Frey

I think I know what happened here: When the participants close the wait-page, they are no longer recognized as waiting-players and therefore will not get grouped.

I suggest you test your code once again and close some pages when you reach the wait-page.

If this is indeed the problem, I can send you the code how I managed to group all the players on the wait-page, even those who closed the page. :)

#3 by lindaaldehoff

Hi Daniel,

thank you for your help! I played and closed the tab on the wait page and the error occured. Thanks for the tip I wouldn't have thought of it :)

Could you also help e fix this?

Best,

Linda

#4 by Daniel_Frey

Hi Linda

Sure! I had the exact same problem with my experiment and figured a way around it.

I used the group_by_arrival_time_method, you can find a detailed explanation how that function works here: https://otree.readthedocs.io/en/latest/multiplayer/waitpages.html#group-by-arrival-time-method

In my experiment, I wanted to group the participants in groups of three. Furthermore, all participants in the same group needed to have the same result in a lottery (stored in a PARTICIPANT_FIELD as 0 or 1).

In summary, I store the ID's of the participants who reach the wait-page in a predefined list and as soon this list has enough elements (i.e. enough participants who enterd the wait-page), a group gets created. In this way, every participant who entered the wait-page got grouped, even if they were no longer recognized as 'waiting_players'.

First, I need an empty dictionary of two sets, this dict is also in SESSION_FIELDS

def creating_session(subsession: Subsession):
    session = subsession.session
    # initiate the dict ids_finished_no_del_treatment, so they can be filled in group_by_arrival_time_method with the IDs of
    # the players that finished the experiment.
    session.ids_finished_no_del_treatment = {outcome: set() for outcome in ['Win', 'Lose']}

Here is my code of the gbat-method:

def group_by_arrival_time_method(subsession, waiting_players):
    session = subsession.session
    # create a set with the ID's of all waiting players, initiated in settings.py using SESSION_FIELDS
    for p in waiting_players:
        pp = p.participant
        player_id = int(p.id_in_subsession)
        lottery_win = pp.lottery_win

        # The IDs of all players get stored in the correct list at the correct place. 
        if lottery_win == 1:
            session.ids_finished_no_del_treatment['Win'].add(player_id)
        if lottery_win == 0:
            session.ids_finished_no_del_treatment['Lose'].add(player_id)

    if len(session.ids_finished_no_del_treatment['Win']) >= 3:
        all_players = subsession.get_players() #list of all players
        players_finished_win = [p for p in all_players if p.id_in_subsession in session.ids_finished_no_del_treatment['Win']]
        
        print('IDs that finished:', session.ids_finished_no_del_treatment)

        # the participants with the lowest 3 IDs will get grouped, so they have to be removed from the set ids_finished
        ids_grouped = sorted(session.ids_finished_no_del_treatment['Win'])[:3]
        print('ids_grouped:', ids_grouped)
        session.ids_finished_no_del_treatment['Win'].difference_update(ids_grouped)

        return[players_finished_win[0], players_finished_win[1], players_finished_win[2]]

    if len(session.ids_finished_no_del_treatment['Lose']) >= 3:
        all_players = subsession.get_players()
        players_finished_lose = [p for p in all_players if p.id_in_subsession in session.ids_finished_no_del_treatment['Lose']]

        # the participants with the lowest 3 IDs will get grouped, so they have to be removed from the set ids_finished
        ids_grouped = sorted(session.ids_finished_no_del_treatment['Lose'])[:3]
        print('ids_grouped:', ids_grouped)
        session.ids_finished_no_del_treatment['Lose'].difference_update(ids_grouped)

        return[players_finished_lose[0], players_finished_lose[1], players_finished_lose[2]]
        
# End of code
        
If you don't have a distinction of cases and just want to group the first three participants that enter the wait-page, the code can be fairly simplified: You only need one set and one condition.      
        
 I hope this helps you (and others who might be interested)! 
 
 Best,
 Daniel

#5 by lindaaldehoff

Thank you Daniel!

I don't understand this variable: session.ids_finished_no_del_treatment = {outcome: set() for outcome in ['Win', 'Lose']}. Do I need it if I don't need to distinguish between winners and losers?

Thanks again :)

#6 by Daniel_Frey

No, you don't need that, if you don't need to distingiush between different cases.

You can define a variable like for example session.ids_finished = set() and use this in your code. 

Just remember that in SESSION_FIELDS there has to be a variable named 'ids_finished' as well.

#7 by lindaaldehoff

Hey Daniel,

I've been trying and trying, but I can't get the code to work.

My current grouping page looks like this:

Class Subsession(BaseSubsession):

    def group_by_arrival_time_method(subsession, waiting_players):
        for p in waiting_players:
            p.category = p.participant.vars['category']
            p.treatment = p.session.vars['treatment']

        if p.treatment == 1:
            print('in group_by_arrival_time_method')
            a_players = [p for p in waiting_players if p.category == 'A']
            b_players = [p for p in waiting_players if p.category == 'B']
            if len(a_players) >= 2 and len(b_players) >= 1:
                print('about to create a group')
                return [a_players[0], a_players[1], b_players[0]]
            print('not enough players yet to create a group')
            for p in waiting_players:
                if p.waiting_too_long():
                    p.alone = 1
                    return [p]
        else:
            if len(waiting_players) >= 3:
                return waiting_players[:3]
            for p in waiting_players:
                if p.waiting_too_long():
                    p.alone = 1
                    return [p]
                    
 
I have no clue how to integrate your approach, I'm sorry. I#ve tried but I think my approaches were ridiculous :D 
Do you think you could help me?

Thank you either way!

Best,

Linda

#8 by Daniel_Frey

Hi Linda

I think the following code should work, though I have not tested it.

Am I right in the assumption that you have different treatments, and all players who have treatment 1 should get grouped together? Thats why I suggest you create two different lists, one for treatment 1 (ids_finished_treatment1) and one for the other treatments (ids_finished). 

I'm not sure if the "waiting_too_long" function works with this approach, I have never used it myself.

I hope this helps and you can understand my code! I added some print-commands so that you can directly see, if the IDs get stored in the correct places. :)

Now the code:

in settings.py:

SESSION_FIELDS = ['ids_finished', 'ids_finished_treatment1'] # just add the two new variables

in __init.py__:

def creating_session(subsession: Subsession):
	session = subsession.session
	# initiate the set ids_finished and the dict ids_finished_category, so they can be filled with the IDs 
	# of the participants who enter the WaitPage, see group_by_arriva_time_method
	session.ids_finished = set()
	session.ids_finished_treatment1 = {category: set() for category in ['A', 'B']}

class Subsession(BaseSubsession):

    def group_by_arrival_time_method(subsession, waiting_players):
    	print('in group_by_arrival_time_method')
    	session = subsession.session
    	all_players = subsession.get_players()
        for p in waiting_players:
            p.category = p.participant.vars['category']
            p.treatment = p.session.vars['treatment']
            player_id = int(p.id_in_subsession)
            
            if p.treatment != 1:
            	session.ids_finished.add(player_id) #add all players with no distinction, will get grouped in second part
            if p.treatment == 1:
            	session.ids_finished_treatment1[p.category].add(player_id) #add all the players who had treatment 1 in their correct category
		print(session.ids_finished)
        print('IDs with treatment 1:', session.ids_finished_treatment1)
        
        if len(session.ids_finished_treatment1['A']) >= 2 and len(session.ids_finished_treatment1['B']) >= 1:
        	print('about to create a group')
            a_players = [p for p in all_players if p.id_in_subsession in session.ids_finished_treatment1['A']]
            b_players = [p for p in all_players if p.id_in_subsession in session.ids_finished_treatment1['B']]
            
            # the grouped participants need to be removed from the dict
            ids_grouped_a = sorted(session.ids_finished_treatment1['A'])[:2]
            session.ids_finished_treatment1['A'].difference_update(ids_grouped_a)
            session.ids_finished_treatment1['B'].remove(min(session.ids_finished_treatment1['B']))
            
            return [a_players[0], a_players[1], b_players[0]]
        
        # grouping of all players who did not play treatment 1
        if len(session.ids_finished) >= 3:
        	print('about to create a group')
        	players_finished = [p for p in all_players if p.id_in_subsession in session.ids_finished]
        	
        	# the participants with the lowest 3 IDs will get grouped, so they have to be removed from the set ids_finished
    		ids_grouped = sorted(session.ids_finished)[:3]
    		print('ids_grouped:', ids_grouped)
    		session.ids_finished.difference_update(ids_grouped)
    
            return [players_finished[0], players_finished[1], players_finished[2]]
        
        # participants who waited too long continue alone
        for p in all_players:
            if p.waiting_too_long():
                p.alone = 1
                return [p]

#9 by lindaaldehoff

It worked!! I can't thank you enough Daniel!

For anyone wondering something similar, I made some minor changes (my code is structured in models instead of _int.py:

class Subsession(BaseSubsession):

    ids_finished = set()
    ids_finished_treatment1 = {category: set() for category in ['A', 'B']}

    def group_by_arrival_time_method(subsession, waiting_players):
        print('about to create a group')
        #session = subsession.session
        all_players = subsession.get_players()
        for p in waiting_players:
            p.category = p.participant.vars['category']
            p.treatment = p.session.vars['treatment']
            player_id = int(p.id_in_subsession)
            if p.waiting_too_long():
                p.alone = 1
                return [p]

            if p.treatment != 1:
                subsession.ids_finished.add(
                    player_id)  # add all players with no distinction, will get grouped in second part
            if p.treatment == 1:
                subsession.ids_finished_treatment1[p.category].add(
                    player_id)  # add all the players who had treatment 1 in their correct category
                print(subsession.ids_finished)
        print('IDs with treatment 1:', subsession.ids_finished_treatment1)

        if len(subsession.ids_finished_treatment1['A']) >= 2 and len(subsession.ids_finished_treatment1['B']) >= 1:
            print('about to create a group')
            a_players = [p for p in all_players if p.id_in_subsession in subsession.ids_finished_treatment1['A']]
            b_players = [p for p in all_players if p.id_in_subsession in subsession.ids_finished_treatment1['B']]
            return [a_players[0], a_players[1], b_players[0]]
        # the grouped participants need to be removed from the dict
        ids_grouped_a = sorted(subsession.ids_finished_treatment1['A'])[:2]
        ids_grouped_b = sorted(subsession.ids_finished_treatment1['B'])[:1]
        subsession.ids_finished_treatment1['A'].difference_update(ids_grouped_a)
        subsession.ids_finished_treatment1['B'].difference_update(ids_grouped_b)
        #subsession.ids_finished_treatment1['B'].remove(min(subsession.ids_finished_treatment1['B']))


        #print('not enough players yet to create a group')
        if len(subsession.ids_finished) >= 3:
            players_finished = [p for p in all_players if p.id_in_subsession in subsession.ids_finished]

            #the participants with the lowest 3 IDs will get grouped, so they have to be removed from the set ids_finished
            ids_grouped = sorted(subsession.ids_finished)[:3]
            print('ids_grouped:', ids_grouped)
            subsession.ids_finished.difference_update(ids_grouped)

            return [players_finished[0], players_finished[1], players_finished[2]]

#10 by Chris_oTree

I just wonder why you would want someone to be grouped even if they closed the page? That means that active players will be grouped with dropouts, which would make multiplayer games get stuck.

oTree 5.8 is stricter than before about dropouts. Now, a player is only considered active if they keep the tab open AND focused, meaning if they switch to another tab, they will not be grouped until they switch back: https://groups.google.com/g/otree/c/RZObx8UzrgM/m/ZOx_8Ue7CAAJ

#11 by Chris_oTree

Maybe those participants who got stuck on the wait page were on a different browser tab? If so, the solution could be to tell them not to switch to another tab.

#12 by lindaaldehoff

Hey Chris, 

I did tell them not to switch tabs, but most MTurkers don't really read the instructions. For me its not a problem if they switch pages while waiting to be grouped. The main problem is that they can keep playing until they are supposed to get the result, only then they get an error message and have to stop playing, so I'd basically be wasting their time even though they did nothing but open a different tab during the experiment. This also causes trouble on later wait pages. So sometimes it's just better to have them grouped even though they were inactive for a second :)

#13 by Daniel_Frey

In my case, the participants get grouped right at the end of the experiment. I even tell the participants that the experiment is finished, they can close the window and give the completion-code on the WaitPage. 

But all those who close the window still have to get grouped, so that their payoff gets calculated.

Write a reply

Set forum username