Performance of SafeDrug and Molerec

[flick-ai]

Hello, esteemed author. I noticed that when running the safedrug and molerec algorithms from your library, their performance falls far short of what is claimed in your papers and benchmarks. I would like to inquire whether you used any specific parameters or techniques during testing. Thank you for your response.

[ycq091044]

Hello, could you please provide your training code? There are also some earlier issues related to this question which you could take a look #220 #209.

[flick-ai]

Hello, I have fixed the issue based on the suggestions from the previous issue. The 'safedrug' table is now back to normal, but there are still issues with the 'molerec' table. Additionally, I noticed that you haven't implemented the 'multimarginloss' as described in the original paper for the 'safedrug' model, and the 'ddiloss' for the 'gamenet' network. Will these omissions affect the performance of these models?
Thanks.

[ycq091044]

It is a good question. Based on our empirical evaluation, we find the multimarginloss and ddiloss will marginally affect the model. However, the dataset processing steps in pyhealth is also slightly different from the original paper (the impact from data processing difference can be larger than adding the extra loss or not). For reproducing the exact results from the original paper, we refer you to the original paper github. We will also spent time to add the extra loss as well (which could be easy to add by your own as well).

[yangnianzu0515]

Kindly consult our response regarding this matter.

We intend to review the specific issue at our earliest convenience.