Skip to content

ExperimentsToSbmlConverter nests initial-assignment Piecewise once per experiment, exceeding lxml's depth limit #540

Closed

Description

@FFroehlich

ExperimentsToSbmlConverter nests the initial assignment of a target one Piecewise level deeper for every experiment that sets that target at the start. With a few hundred such experiments, the next read of the assignment (sbml_math_to_sympy(ia.getMath()), converters.py:296) fails in lxml:

lxml.etree.XMLSyntaxError: Excessive depth in document: 256, use XML_PARSE_HUGE option, line 1267, column 516

This breaks Raimundez_PCB2020 from the benchmark collection (174 experiments) once upgraded to v2.

Cause

convert() calls _convert_experiment once per experiment. Each call reads the existing initial assignment back as default and writes

expr = sp.Piecewise(*expr_cond_pairs, (default, True))

so the previous experiments' Piecewise becomes the "otherwise" branch of the new one. The depth grows linearly with the number of experiments.

Reproducer

One species whose initial value is set by N conditions (PEtab v1, upgraded on load):

import sys
import tempfile
from pathlib import Path

import libsbml
from petab.v2 import Problem
from petab.v2.converters import ExperimentsToSbmlConverter

N = int(sys.argv[1])  # number of conditions setting the initial value of A
d = Path(tempfile.mkdtemp())
doc = libsbml.readSBMLFromString(
    """<?xml version="1.0" encoding="UTF-8"?>
<sbml xmlns="http://www.sbml.org/sbml/level3/version2/core" level="3" version="2">
  <model id="m">
    <listOfCompartments><compartment id="C" size="1" constant="true"/></listOfCompartments>
    <listOfSpecies>
      <species id="A" compartment="C" initialAmount="1" hasOnlySubstanceUnits="false"
               boundaryCondition="false" constant="false"/>
    </listOfSpecies>
    <listOfParameters><parameter id="k" value="1" constant="true"/></listOfParameters>
  </model>
</sbml>"""
)
libsbml.writeSBMLToFile(doc, str(d / "model.xml"))
(d / "conditions.tsv").write_text(
    "conditionId\tA\n" + "".join(f"c{i}\t{i + 1}\n" for i in range(N))
)
(d / "observables.tsv").write_text(
    "observableId\tobservableFormula\tnoiseFormula\nobs_A\tA\t1\n"
)
(d / "measurements.tsv").write_text(
    "observableId\tsimulationConditionId\ttime\tmeasurement\n"
    + "".join(f"obs_A\tc{i}\t1\t1\n" for i in range(N))
)
(d / "parameters.tsv").write_text(
    "parameterId\tparameterScale\tlowerBound\tupperBound\tnominalValue\testimate\n"
    "k\tlin\t0\t10\t1\t1\n"
)
(d / "problem.yaml").write_text(
    "format_version: 1\nparameter_file: parameters.tsv\nproblems:\n"
    "- sbml_files: [model.xml]\n  condition_files: [conditions.tsv]\n"
    "  measurement_files: [measurements.tsv]\n  observable_files: [observables.tsv]\n"
)

problem = Problem.from_yaml(d / "problem.yaml")  # upgraded from PEtab v1
converted = ExperimentsToSbmlConverter(problem).convert()
model = converted.model.sbml_model
math = model.getInitialAssignmentBySymbol("A").getMath()


def depth(node):
    return 1 + max((depth(node.getChild(i)) for i in range(node.getNumChildren())), default=0)


print(f"{N} conditions: initial assignment of A nests {depth(math)} levels deep")
5 conditions: initial assignment of A nests 7 levels deep
50 conditions: initial assignment of A nests 52 levels deep
300 conditions: lxml.etree.XMLSyntaxError: Excessive depth in document: 256, ...

Possible fixes

  • Flatten when wrapping: if default is already a Piecewise, extend its pairs instead of nesting it, e.g. sp.Piecewise(*expr_cond_pairs, *default.args). The conditions are mutually exclusive experiment indicators, so the order of the pairs does not matter.
  • Or collect the period-0 changes of all experiments first and write one initial assignment per target.
  • Independently, sbmlmath could parse with etree.XMLParser(huge_tree=True), but that only hides the growing expression.

Versions: libpetab-python 402393d (petab.__version__ 0.9.0); the code is the same on main (1c3d7b9). sbmlmath 0.4.1, lxml 6.1.3, sympy 1.14.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions