Dear Community,

I have a fairly big app that needs to be repeated multiple times in an experiment, with multiple other apps in-between. Rolling everything into one app does not seem like a feasible solution given the huge number of rounds / pages that would create.

I now try to implement the solution proposed by Chris here: 
https://groups.google.com/g/otree/c/2Z3d_SStb5o

That is, I create an app (app1) and create a second app (app1_copy), which imports everything from app (from app1 import *). I know I need to copy over the models - so far so good. This appears to work well enough.

However, when I try to store data in an ExtraModel (Bid), I receive the following error message:
sqlalchemy.orm.exc.FlushError: Attempting to flush an item of type <class 'app1_copy.Player'> as a member of collection "Bid.player". Expected an object of type <class 'app1.Player'> or a polymorphic subclass of this type.

I understand the error in that I am trying to link the Bid to an app1_copy.Player while it expects me to provide a app1.Player.

Do you have any ideas on how I can solve this?

I've tried:
- Importing the ExtraModel
- having copies of the ExtraModel in both apps (I don't understand the error in this case, here the Link to Player seems to use the app1.Player, although the Player class definition in the same file should override it).
- selectively importing only pages from app1, such that the models should not be overwritten. (-> same problem apparently the link to Player on the ExtraModel still refers to the Player class from app1 that is not even imported...)
- defining a CustomExtraModel separately, then inheriting from it in both apps

Minimal example that demonstrates the issue: https://github.com/chkgk/repeated_app
Start app 2, klick the live send button on the page and check the error in the console.

Your ideas are much appreciated. I'd really like to avoid copying over everything.

Best
Christian

Update:
It seems one could get this working if the ExtraModel did not have a Link field. In this case one needs to store session/subsession/group/player ids and track relationships manually. 

Unfortunately, ExtraModel.filter() appears to require a linked model to work at all, which is a very unfortunate design choice in my opinion, because it means we would always have to request every single row from the database and would have to filter manually in Python. Typically, one would offload selecting and joining to the database as much as possible, because that’s what they are optimized for (and much faster at).

Has anyone tried to access the underlying sqlalchemy ORM directly to create additional models without the link restriction? I am unfortunately not too familiar with sqlalchemy.

Again, this is not solved, any ideas are appreciated. Thank you in advance!

Best
Christian

Hi,

To select data using Python, you can structure your `Bid` model like this:

```python
class Bid(ExtraModel):
    player_id = models.IntegerField()
    app_label = models.StringField()
    bid = models.IntegerField()
```

This model needs to be defined only once, in `app1`; there's no need to redefine it in `app1_copy`. I have a question for you, though: Are you looking to have two separate tables, one for `app1_bid` and another for `app1_copy_bid`, or would a single `app1_bid` table suffice?

To retrieve data, you can use the following method:

```python
def filter_bids_by_player(app_label, player_id):
    # Fetch all Bid instances
    all_bids = Bid.objects.all()
    # Manually filter the bids based on app_label and player_id
    filtered_bids = [bid for bid in all_bids if bid.app_label == app_label and bid.player_id == player_id]
    return filtered_bids
```

However, this approach's effectiveness depends on the scale of your data. If you have a large dataset, you might encounter performance issues. But if your dataset is relatively small, this method should work well for filtering your data.

Best regards,
Tina Zheng
Visit our website https://otree.xindamate.xyz/en for more information

Hi Tina,

Thank you for your reply! It is very much appreciated! A single table for all bids suffices. As I said, I could track everything manually similar to how you describe it, but it feels wrong. I'd really like to avoid filtering in python, given that this is such a typical task for the database.

To give a sense of scale:
I am looking at more almost two thousand participants, double-digit rounds, and multiple repetitions. Of course, I am not only storing bids, but many more data points. Participants interact in real-time via live pages. I need to query the database for at least three different models for every request that comes in. I am very worried about the performance. It is very conceivable, that towards the end of data collection the Bid table will contain more than 150,000 rows.

I believe there must be way to filter in oTree without providing a Foreign Key (model.Link) or to solve the polymorphism issue arising form copying the app in a smart way.

Maybe Chris could chime in and give me a hint how I might get access to the underlying sqlalchemy layer?

Hi Christian,

I'm really interested in this and am eager to hear Chris's feedback.