Skip to content

Format for logs of the execution of simulation experiments captured in COMBINE/OMEX archives

Overview

This format enables simulation tools to communicate information about the status and outcome of the execution of SED-ML files in COMBINE/OMEX archives. The format can capture the following information:

  • The status and outcome of the COMBINE/OMEX archive and each SED-ML document, task, report, plot, data set, curve and surface (e.g., queued, running, succeeded, failed).
  • Information about simulation functions that were executed and the arguments that were used.
  • The standard output/error produced from executing the COMBINE/OMEX archive and each SED-ML document, task, and output.
  • The duration of the execution of the COMBINE/OMEX archive and each SED-ML document, task, and output.
  • The reason for each skipped or failed SED-ML document, task or output.

Specifications

The schema for the format is available in JSON schema and Open API formats. Documentation for the schema is available at https://api.biosimulations.org/.

Simulators are encouraged to log the execution of each individual SED-ML element, including each task, report, plot, data set, curve and surface.

At the same time, the format provides simulation tools flexibility to log their execution at whatever level of granularity is possible. Below are several possible levels of granularity.

  • Individual task and output elements (e.g., data sets, curves, surfaces)
  • Individual tasks
  • Individual SED-ML documents
  • Entire COMBINE/OMEX archives

Note

The output attribute for the log of each COMBINE/OMEX archive, SED-ML document, task, and output can include ANSI escape codes for color and other terminal formatting.

Example logs

Below is an example of an element-level log of a COMBINE/OMEX archive that involves a single SED-ML document with two tasks at the beginning of its execution.

{
  "status": "QUEUED",
  "exception": null,
  "skipReason": null,
  "output": null,
  "duration": null,
  "sedDocuments": [
    {
      "location": "doc_1.sedml",
      "status": "QUEUED",
      "exception": null,
      "skipReason": null,
      "output": null,
      "duration": null,
      "tasks": [
        {
          "id": "task_1_ss",
          "status": "QUEUED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": null,
          "algorithm": null,
          "simulatorDetails": null
        },
        {
          "id": "task_2_time_course",
          "status": "QUEUED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": null,
          "algorithm": null,
          "simulatorDetails": null
        }
      ],
      "outputs": [
        {
          "id": "report_1",
          "status": "QUEUED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": null,
          "dataSets": [
            {
              "id": "dataset_1",
              "status": "QUEUED"
            },
            {
              "id": "dataset_2",
              "status": "QUEUED"
            }
          ]
        },
        {
          "id": "plot_1",
          "status": "QUEUED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": null,
          "curves": [
            {
              "id": "curve_1",
              "status": "QUEUED"
            }
          ]
        }
      ]
    }
  ]
}

Below is an example of an element-level log of the final state of the successful execution of the same COMBINE/OMEX archive.

{
  "status": "SUCCEEDED",
  "exception": null,
  "skipReason": null,
  "output": null,
  "duration": 6,
  "sedDocuments": [
    {
      "location": "doc_1.sedml",
      "status": "SUCCEEDED",
      "exception": null,
      "skipReason": null,
      "output": null,
      "duration": 5,
      "tasks": [
        {
          "id": "task_1_ss",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": "Reading model ... done\nInitializing simulation ... done\nExecuting simulation ... done\n",
          "duration": 2,
          "algorithm": null,
          "simulatorDetails": null
        },
        {
          "id": "task_2_time_course",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": "Reading model ... done\nInitializing simulation ... done\nExecuting simulation ... done\n",
          "duration": 1,
          "algorithm": null,
          "simulatorDetails": null
        }
      ],
      "outputs": [
        {
          "id": "report_1",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": 0.1,
          "dataSets": [
            {
              "id": "dataset_1",
              "status": "SUCCEEDED"
            },
            {
              "id": "dataset_2",
              "status": "SUCCEEDED"
            }
          ]
        },
        {
          "id": "plot_1",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": 0.01,
          "curves": [
            {
              "id": "curve_1",
              "status": "SUCCEEDED"
            }
          ]
        }
      ]
    }
  ]
}

Below is an example of an element-level log of the final state of the failed execution of the same COMBINE/OMEX archive.

{
  "status": "FAILED",
  "exception": null,
  "skipReason": null,
  "output": null,
  "duration": 6,
  "sedDocuments": [
    {
      "location": "doc_1.sedml",
      "status": "FAILED",
      "exception": null,
      "skipReason": null,
      "output": null,
      "duration": 5,
      "tasks": [
        {
          "id": "task_1_ss",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": "Reading model ... done\nInitializing simulation ... done\nExecuting simulation ... done\n",
          "duration": 2,
          "algorithm": null,
          "simulatorDetails": null
        },
        {
          "id": "task_2_time_course",
          "status": "FAILED",
          "exception": {
            "type": "FileNotFoundError",
            "message": "Model `model2.xml` does not exist."
          },
          "skipReason": null,
          "output": null,
          "duration": 1,
          "algorithm": null,
          "simulatorDetails": null
        }
      ],
      "outputs": [
        {
          "id": "report_1",
          "status": "SUCCEEDED",
          "exception": null,
          "skipReason": null,
          "output": null,
          "duration": 0.1,
          "dataSets": [
            {
              "id": "dataset_1",
              "status": "SUCCEEDED"
            },
            {
              "id": "dataset_2",
              "status": "SUCCEEDED"
            }
          ]
        },
        {
          "id": "plot_1",
          "status": "SKIPPED",
          "exception": null,
          "skipReason": {
            "type": "2DPlotNotImplemented",
            "message": "Output skipped because the simulator cannot generate plots."
          },
          "output": null,
          "duration": 0.01,
          "curves": [
            {
              "id": "curve_1",
              "status": "SKIPPED"
            }
          ]
        }
      ]
    }
  ]
}

Below is an example of a SED-ML document-level log of the final state of the failed execution of the same COMBINE/OMEX archive.

{
  "status": "FAILED",
  "exception": null,
  "skipReason": null,
  "output": null,
  "duration": 6,
  "sedDocuments": [
    {
      "location": "doc_1.sedml",
      "status": "FAILED",
      "exception": {
        "type": "FileNotFoundError",
        "message": "Model `model2.xml` does not exist."
      },
      "skipReason": null,
      "output": "Reading model ... done\nInitializing simulation ... done\nExecuting simulation ... done\n",
      "duration": 5,
      "tasks": null,
      "outputs": null
    }
  ]
}

Events which should trigger simulators to update the status of COMBINE/OMEX archives

Simulation tools are encouraged to flush logs of the execution of COMBINE/OMEX archives upon each of the following events:

  • Start of the execution of the COMBINE/OMEX archive. After this event, the status of the archive should be RUNNING, the status of each SED-ML document and SED-ML element that the simulation tool is capable of executing should be QUEUED, and the status of each SED-ML document and SED-ML element that the simulation tool is not capable fo executing should be SKIPPED.
  • Start of the execution of each SED-ML document. This event should change the status of the document to RUNNING.
  • Start of the execution of each SED-ML task. This event should change the status of the task to RUNNING.
  • End of the execution of each SED-ML task. If the task succeeded, its status should be changed to SUCCEEDED. In addition, all data sets, curves, and surfaces which can now be generated should be generated, and their status should be changed to SUCCEEDED. The status of the parent reports and plots should be set to SUCCEEDED if all of their data sets, curves, and surfaces have been generated, or RUNNING if some of their data sets, curves, and surfaces cannot yet be generated because they depend on tasks which have not yet been executed. If the task failed, its status should be changed to FAILED, and the status of all data sets, curves, surfaces, reports, and plots which depend on the task should be changed to FAILED.
  • End of the execution of each SED-ML document. If all of the task and outputs in the document succeeded, the document's status should be changed to SUCCEEDED. Otherwise, the document's status should be changed to FAILED.

By the end of the execution of a COMBINE/OMEX archive, the status of each SED-ML document, task, report, plot, data set, curve, and surface should be one of SUCCEEDED, SKIPPED, or FAILED.

Below are helpful tools for building logs of the execution of COMBINE/OMEX archives:

  • BioSimulators utils is a Python library which provides functions for generating reports to the above specifications.
  • capturer is a Python library for capturing standard output and standard error streams.

Last update: 2022-07-03