Comparing simulation results between simulators will ultimately demonstrate differences in the interpretation of the SBML between the simulators. But what to do in cases where all simulators seem to yield different results? This case occurred for the longest time with Biomodel #104:
Here we see for five different simulators five different outcomes of the simulation. Looking at the Biomodels database, or the original paper made us assume that our simulator (‘roadRunner’) was interpreting the results as intended by the author. And thus we did not analyze the issue further.
New simulation results were then submitted from Henning Schmidt, using SBToolBox2. Those results interpreted the event in a similar fashion as MathSBML, which made us analyze step for step what should happen in this case. As it turned out this made us realize an updated wording in the SBML specification:
The order of the EventAssignment objects is not significant; the effect of one assignment cannot affect the result of another assignment. The identifiers occurring in the MathML ci attributes of the EventAssignment object represent the value of the identifier at the point when the Event is fired.
This was clearly not the case for us. There were two event assignments in Biomodel 104, and if the order of the assignments was reversed, we found different results. The results here helped us to find this error in interpretation. And now the correct results are reported:
