Infrastructure Coordination for new developments#

Infrastructure in the following is specifically used for technical infrastructure. Technical infrastructure concerns every code modification that does not primarily aim at improving science or forecasts. Examples include:

  • Communication/output schemes

  • Testing infrastructure

  • Interfaces between code parts (Coupler, Plugin-Infrastructure, modularization/librarization, glue code to other libraries)

  • Programming paradigms

TL;DR#

  • Discuss in the gitlab issue/merge request proposing a development

  • Keep the meeting concise — ideally just to clarify the few remaining points. Major points must be discussed beforehand, possibly in a separate meeting.

  • Aim for a commitment from all sides (developing group and rest of the ICON partnership) to develop and include a feature in a timely manner so that others can rely on it.

Infrastructure coordination workflow

Infrastructure Coordination Process#

  • Every major development of infrastructure components that can be expected to influence the work of more than one partner institution needs to go through the Infrastructure Coordination Workflow. The workflow is also open for major scientific developments on a voluntary basis - which would further promote openness of the development process.

  • During the design phase:

    • Open an issue in icon/icon, tag it with infrastructure-coordination and describe what you are planing to do and the reasoning behind it. See below for a template.

    • Try to raise awareness using all reasonables channel with everybody who has stakes in this and ask for review.

    • Discuss in the issue; ideally consensus is reached.

    • The issue will be covered in the next Infrastructure Coordination meeting.

    • A few days before the IC meeting, the author of the issue is asked to sum up the discussion up to this point.

    • In the IC meeting a decision will be taken to pursue or not to pursue the development. The decision may contain objectives like the following of which some can be deferred until the development is complete. It is mandatory that all requirements are written down at this stage.

      • “We are willing to sacrifice up to 3% of performance since this development will be helpful for X”

      • “The new development must eventually replace the existing completely.” (A good candidate to defer until completion to allow for incremental development)

    • In the issue and in the IC meeting, stake holders of the affected code parts are collected. If the developing group sees a benefit, a mentor can be chosen who will support the development but not slow it down.

    • After the decision in the IC meeting, add an estimated timeline to the issue. The (discussion) issue will afterwards be closed formally.

  • Infrastructure Coordination meetings

    • are held every three months, usually at the end of the second month in the quarter.

    • An invitation containing all developments covered in the meeting is sent two weeks before that.

    • Cut-off for topics covered is two days before that.

    • All dates for the next meeting will be circulated shortly after each meeting on developer@lists.icon-model.org

    • Attendance is mandatory for all persons proposing a development and one representative of each ICON partner. In addition, maintainers are recommended to attend the meeting.

    • Decisions are taken based on consensus. If a consensus cannot be reached, the decision is forwarded to the C5.

    • Decisions are documented in the respective issues/merge requests

    • A protocol in the ICON Wiki details which issues and merge requests were covered.

    • The final decision for each development is taken in one meeting. Only if the proposal can be improved substantially until the next meeting, it is considered twice.

  • During the development:

    • If major changes to the original envisioned design are needed, reopen the issue.

    • Before each IC meeting, update the estimated timeline in the issue.

    • If a development fails to update the issue before the IC meeting twice, the decision on the development will be revoked

    • If a development continuously fails to meet its timelines, it will be reconsidered.

  • After major parts of the development are done:

    • If major parts of the development are done and the development is in a shape so that it and its impact on overall ICON can be asessed, continue here.

    • Open a draft merge request, tag it with infrastructure-coordination, reference the issue from the design phase, and tag all stake holders.

    • Describe in the merge request

      • what has changed compared to the design phase

      • which design decisions were taken

      • how the IC objectives are met (deferred objectives can be considered later)

    • The IC meeting will decide whether the development can be merged in general and whether it needs mandatory changes.

    • The final decision for the merge will be taken by the maintainers as usual, but additionally checking for the changes decided on in the meeting. Maintainers are kindly requested not to bring up major topics after the IC meeting, but to raise these before or in the meeting. For this, their attendance is recommended.

  • Review of the final result:

    • After the decision to merge a development was taken, the development is overseen in the following meetings until it is completely done. For this, the proponent is asked to report.

    • If a development fails to meet its deferred objectives (for example it does not completely replace its predecessor), the meeting can decide to remove the development again.

Coordination for small efforts#

Smaller development effort below the threshold of Infrastructure Coordination can and should still be documented in a similar but leaner process. This helps to avoid double and conflicting developments. The main difference to the full process is that these development do not need decisions from the Infrastructure Coordination meeetings and they are tagged with infrastructure-coordination-lean. Proposing decide on their own how much feedback they want and in simple cases (e.g. “We want to port Kinne Aerosols to GPU”) directly go ahead. If proposed developments turn out to be controversial, they can be upgraded to a full Infrastructure Coordination case.

From Brainstorming to an Infrastructure Coordination Issue#

If there is significant interest in implementing a feature, but the best approach is not yet clear, you can use the infrastructure-coordination-brainstorming tag to discuss possible solutions and prepare for a future infrastructure coordination issue. Issues with this tag will be highlighted in meeting invitations to foster productive discussion. Brainstorming issues shall not be around forever, they will be advertised at most twice and ideally give rise to a infrastructure coordination proposal.

Deprecation#

Decisions on deprecation and removal are also taken in the IC meeting, but described in a different document.

Issue template to start the infrastructure coordination process#

At least answer the following questions:

  • What do you want to develop?

  • Why shall ICON integrate it?

  • How do you want to integrate it? With which code parts does it interfere with?

  • Who is impacted by this development?

Tips and Tricks#

You can subscribe to the labels and filter for infrastructure coordination issues, for infrastructure coordination lean issues, and forinfrastructure coordination brainstorming issues