Research Software Engineering

White Paper by Bernhard Rumpe, Software Engineering, RWTH Aachen

Research Software Engineering (RSE) is a term that was created around 2010 and has started to become prominent in UK, USA, and now also Germany. Wikipedia defines it this way:

Definition: Research software engineering (RSE)

Research software engineering is the use of software engineering practices in research applications.

Software Engineering

Similar to the original term “software engineering”, its creation was felt necessary because there was and is a crisis in software development, especially a “research software crisis”. Software Engineering was made prominent when F. L. Bauer hosted the first conference on Software Engineering in 1968 to address the software crisis. Since then, the Software Engineering discipline has gained a lot of insight into the process of software development cumulating in

  • several introductory and expert books on software engineering,
  • many more books on dedicated sub-fields of software engineering, namely requirements engineering, architecture, design, modeling, testing, development processes, and of course programming,
  • and Software Engineering Body of Knowledge (SWEBOK V3.0) that structures and aggregates what software developers have learned in the last 50 years.
Software Engineering Challenges

Software is rather heterogeneous, ranging from embedded to desktop, to autonomous, to games, to business, and also to research software. While the problems are always the same, namely:

  • How to ensure the quality of the software?
  • How to efficiently develop the software (i.e., preserve developer resources)?
  • How to meet timing deadlines?

The answers and then, in particular, the development techniques are often different in the various sub-areas of software development as the starting situation, the kind of software, the complexity drivers, the needed quality characteristics, the context in which the software is operating in, and the skills and preferences of the developers are different.

Software Engineering: 50 years, culminating in the SWEBOK

The Software Engineering Body of Knowledge (SWEBOK) defines 15 knowledge areas:

  • Software Requirements
  • Software Design
  • Software Construction
  • Software Testing
  • Software Maintenance
  • Software Configuration Management
  • Software Engineering Management
  • Software Engineering Process
  • Software Engineering Models and Methods
  • Software Quality
  • Software Engineering Professional Practice
  • Software Engineering Economics
  • Computing Foundations
  • Mathematical Foundations
  • Engineering Foundations

For business software, we know that the programming activities only make 15% of the overall development time. For skilled people constructing the actual software is a relatively well-understood activity, while many more and more costly errors have been made during the early phases of misunderstanding requirements or defining the architecture wrong.

Software Engineering Areas

Software Engineering doesn’t only have thesubdisciplines mentioned above, which cover the different activities within a software engineering project. Due to the various application areas software engineering is also organized in domain-specific subdisciplines or partners with related domain specific topics so that a holistic, integrative development approach becomes feasible:

  • Automotive Software Engineering,
  • Information Systems,
  • Avionics Systems and Software Engineering,
  • Rail Software Engineering,
  • Embedded Software,
  • Cloud Software Engineering,
  • Data Science and Engineering,
  • Medical Software Engineering,
  • Government Software Engineering
  • Gaming Software Engineering
  • and now also Research Software Engineering.

Other domains, like Quantum Software Engineering, are also emerging.

Characteristics of Research Software

Since 2010 communities have been created, conferences have been organized, and people exchange their findings on how to overcome the challenges of research software development, i.e., how to overcome the research software crisis.

What differentiates the challenges of RSE from other forms of SE?

Kinds of Research Software

First, we recognize that there is no uniform kind of research software. Instead research software mainly falls into one of the following categories (and sometimes combinations):

  • Embedded control software for complex physical or chemical experiments, including many forms of sensor-based data collections
  • Simulation of physical, chemical, social, or biological processes in geometrically distributed spaces
  • Data processing and aggregation

While the embedded control software also has its challenges, the main challenges seem to be the simulation and the data processing codes that are so heavily needed by researchers.

But software consists of different domains and technical parts. While obviously, the mathematical research part receives the main focus in practically useful programs,this often amounts to only 20-30% of the code (estimated, personal insights to some projects, but not thoroughly
validated). The much larger rest has to deal with:

  • App Infrastructure
  • User Interactions
  • Visualization
  • Storage and transaction management
  • Communication between computing and storage nodes
  • Interacting with neighboring systems
  • Providing web services
  • Rights and roles management for access and prevention of undetected changes
  • Technical monitoring
  • Testing
  • Installation, deployment, orchestration

And this is very similar to many other kinds of software. The amount of technical support software tends to grow more quickly than the functional research part.

Characteristics of Research Software

Several common characteristics of research software are:

  • It is used for research in the original research domain and thus originally only a by-product, which also treated like such a by-product.
  • Focus is the publication of the results.
  • Requirements of the software under development are initially unclear and the software development process is heavily intertwined with the research and innovation process, both on the scientific domain and its mapping into software.
  • Developed by researchers of various domains, but mainly not by computer scientists (nor software engineers)
    • An often seen scenario: The complete software is developed by a single Ph.D. candidate and erodes after the Ph.D. is done.
    • A similar scenario: the overall software is complex and has been developed over longer time by many developers, but the newly added packages have also only a single Ph.D. author and erode after he/she leaves.
  • Reuse is difficult, because the software is not designed for reuse.
  • Applicability of software is limited, because it has been designed for a single (or a small set) of use cases and extensibility and flexibility wasn’t designed into it.
  • Rewriting of software is done intensively.
    • Within a Ph.D. project, when the desired outomes (i.e. requirements) change, rewriting is applied.
    • If a new researcher takes the existing software, then a long lasting rewriting takes place to accomodate the new developers preferences. (“Don’t trust foreign code”)
  • Goals of the research institution, respectively their professorial leaders, significantly differ from the goals of the developing researchers, namely: long-lasting, sustainable software programs that can be used as infrastructure for research vs. obtaining quickly publishable insights.

However, the world is changing, the demand for reproducibility of the results enforces also publication of the data and also of the underlying software in a permanently executable form. E.g. Zenodo stores software permanently, but doesn’t itself ensure executability (yet).

Research Software Engineering

As a consequence of the sustainability and reproduceability requirements on research software, the software turns from a by-product into an long-living, sustainable asset if not into a core research infrastructure, where demands for quality of the code heavily increase. This includes understandability, documentation, reuse, ability to evolve, and other typical quality attributes that software engineering has discussed as a consequence of the software crisis in other development domains a while ago.

It therefore is time to put the focus on the question, how to transfer the body of software engineering knowledge to at least the sustainable part of the research software. We remember:

Research software engineering is the use of software engineering practices in research applications.

But which practices are actually useful? Which can be ignored? Which are forgotten, but would be useful?

We observe:

Focus: Processing Efficiency
  • SE: The focus of many SE techniques is the efficiency of the developers during the development project, because complexity of software functions is more demanding than the size of data and the computational effort.
  • RSE: Traditionally, High-Performance Computing (HPC) focusses mainly on execution times. Simulations and various other forms of number crunching and data prcessing force RSE to put focus on efficiency of the programs.
  • RSE tomorrow: complexity of programs steadily increases and probably both, the efficiency of the developers and the program will receive focus in the future.
Reuse
  • SE: Traditionally focuses on the reuse and provides a large set of mechanisms to increase the reusability of software. This starts with language constructs, such as inheritance, import of other modules/classes, explicit definition of interfaces that allow (but not enforce(!)) reuse. This continues on the methodological level, where development processes for frameworks, for features in software product lines, for components, microservices, independently deployable and version level subsystems, etc. are established. Architectural and design patterns are of great help.
    • Reuse needs to be designed for, especially when designing the software architecture.
  • RSE: To some extent uses these techniques, but too often these high-level techniques are not applied. That is natural, as the focus is not on reuse and explicit architectural design activity is not in place. Increasing reusability thus fosters rewriting code.
Modularity
  • SE: Modularity is a core technique to assist in foster reusability. It is applied on various levels, starting the building of classes, and the design and architecture of components, subsystems, etc. Modularity starts with the requirements, but enforces much focus during architectural design, affects the organization of distributed development and modular quality assurance. Achieving modularity is a professional skill that has to be learned and trained.
  • RSE: Technically applied mainly in class design.
Modeling
  • SE: Explicit use of software models using UML (or SysML). Domain specific languages (DSL) are used in various domains with specific characteristics, as well as for configuration and increase of modular use of components. These DSLs include e.g. building information modeling (BIM), control automata, etc.
  • RSE: Sometimes applied too. The core form of models are the mathematical constraint between physical quantities, timing and geometric topics. Many of them are differntial equations.
  • RSE tomorrow: The mathematical constraints, typically defined in scientific papers, and their software implementation needs to have more focus. Especially ensuring the consistency between efficient implementations and the mathematical formula (seens as requirement), tracing changes, explicitly specifying limitations need better assisted by RSE tools, which may include generators, more domain specific languages, but also more math compilers. But also the software itself will need design techniques and such variants of UML-like techniques will be integrated.
Automation With Smart Tools
  • SE: Continuous integration is of great help for developers to get immediate feedback and for project managers to keep track and an overview of the project progress. Tools that help refactoring, detect malicious or dead code, security vulnerabilities, architectural deficiencies, etc.
  • RSE: These techniques are applied e.g. using git capabilities.
  • RSE tomorrow: Assistance seems to be helpful for purposeful use. RSE specific solutions could help more.
Versioning
  • SE & RSE: The git version control systems with its openly accessible github as well as the locally deployable gitlab tooling infrastructures are generally applicable in SE. They provide helpful assistance, e.g. a ticket system, continuous integration, versioning, branching, variant management, etc. Skillfully applied they are improving the development process intensively.
  • SE & RSE tomorrow: Currently many tools are developed around git to even improve their assistance. E.g. detecting security breaches, automatic deployment, artifact management, documentation generation help. Increasingly, projects, companies and sometimes also specific communities are creating their domain specific tools, which know specificities about the domain and such can help developing and maintaining the code even better. The Linux open source project with its kernela, master and lieutenant system may act as good blueprint to build a chain of trust and quality.
Long-Lasting / Sustainable Software
  • RSE: It is true, that in RSE some larger (but by far not all) pieces of programs should be long-lasting and sustainable, to become reusable for various research questions to be addressed based on these programs.
  • SE: This is true for other software is real. Billions have been invested in developing software e.g. in insurances, banks, government, and back-offices of companies.
  • SE & RSE: Same problems. Even though the languages are different, the methods to achieve sustainability for software are potentially similar. This does not only include the software, but also it’s accessibility for example in form of documentation and the people that have knowledge about the software in its complexity. Open source alone is not enough.
Reproducability
  • SE: This challenge occurs for example when legal aspects come into play. Banks and insurances have to provide access to the software e.g. to tax authorities even 10 years after the software was taken out of operation in hot standby and be able to reconstruct the software service for 20 years while the underlying computing infrastructure involves rapidly.
  • RSE: Same problem even though the reason is different and the timing is not constrained by legal laws, but by scientific standards.
Framework Development Strategies

  • SE: SE has created various forms of modularity and reusability, including the concept of frameworks with hot spots, various forms of configuration techniques and tooling. Framework differ from pure libraries, because they our integrated, establish the control flow and allow developers to plug in individual adaptations of functionality.

  • RSE: Mainly has monolithic codes, which are openly accessible, but difficult to understand. Documentation is sometimes available, often as reference to (static, unchangeable) published papers. Libraries of core functions are established.
  • RSE tomorrow: The framework and the product line concepts must be applied more often.
Development Methods
  • SE: Various forms of development methods have been established. These include V-model, Rational Unified Process, Extreme Programming, Scrum, and many other agile or more document oriented development processes.
  • RSE: Hacking still applied too often. Especially the requirements engineering process very much differs from traditional SE. In many Ph.D. projects the candidate is developer and his/her own single customer.
  • RSE tomorrow: dependent on the form of software, the number of people, there are skills, the expected quality and outcomes, timelines, etc. several different forms of methods will be needed. The core need will be that someone in the project know about the activities and their interactions in a development projects, select the right process and ensures that the participants live it in the project.
    • Exploration of new research ideas plays a major role in RSE and must be well integrated into an RSE method. Again: Explicit innovation enabling techniques are already well established in core SE and should be applicable in adapted form.
Consulting
  • SE: In the industrial practice of SE, consulting has become a major driving force. There is a large set of companies that employs consultants of various specific skills, which are working in sometimes large projects to develop various forms of software.
    • Skills may be technical, for example dealing with specific software stacks, or they may be applicable for specific activities, such as architectural design, requirements engineering, testing, database connection, installing software as a service, or tool building and adaptation.
    • Consulting may also consist of various different activities, such as providing templates, giving courses, training on the job, code reviews, tool and software stack assistance, or other kinds of trouble-shooting and problem-solving.
    • Even co-development is doable, when domain specific know how and thus pi-shaped skills (software engineering + domain knowledge) are present.
  • RSE: First research organizations are starting to apply in-house consulting.
  • RSE tomorrow: In-house consulting is a possibility and maybe also a necessity to bring experienced developers with specific skills temporarily to the application projects that need these skills.
    • As a consequence: Groups of skilled software engineers (including developers) will be established at least a larger research organizations to assist various stages of development and potentially also maintenance and curation in the later phases of sustainable software.

RSE Research

For a precise understanding we’ll put this into parentheses: ((research software) engineering) research) is a foundational field of research that focuses on the engineering techniques, tools, methods, frameworks, etc. to develop software quite like software engineering research does.

RSE research has to tackle a number of questions, that do not only address the software as product, but also the domain specific researcher as human being with limited time and skills for software development:

  • How does RSE differ from SE and what specific activities have to be addressed by different methods and tools?

  • How can classic SE approaches, like V-Model, RUP, Scrum, or Xtreme Programming be adapted to RSE needs?

  • Or is an entirely new development process needed?

  • How to run empirical validation in such context? (At least researchers could understand that empirical validation is needed, and would hopefully be assisted?)

  • How to optimize software requirements elicitation which is deeply interlocked with the domain specific research process? (i.e. traditional requirements engineering doesn’t fit)

  • How to defined robust architectures that are extensible, maintainable and allow a reusability for research software, where the forms of extensions, connections to neighbor functionality are initially very unpredictable.

  • How to document and model the desired software with the purpose of (a) effective development, (b) possible code generation, (c) sutainable reproducibility, and (d) comprehension in the sense of FAIR research data management.

  • What tools are needed to easily automate various tedious activities, researches would like to avoid.

  • What is appropriate meta-information about the software artefacts, their states and forms of review, certifications etc. to build trust, enable reusability, and document quality.

  • How do the software analysis tools contribute, e.g. checking consistency between mathematical models in published papers vs. the implemented code, how can tools for dead code, inefficient code and other code analsis techniques effectively help the researchers to improve quality?

  • How to test and what to test in which intensitivity?

  • What are usefuls domain-specific languages (DSLs) for RSE?

  • How to assess and optimize development skills for researchers?

Many of those questions are not specific to RSE, but apply to classic SE as well, but on so the questions are very general, the answers must typically be very domain specific to gain advantage of the domain specific challenges. Classic SE has many (and still optimizable) answers to these questions, but is unclear, how they transfer to RSE.

Recommendations and Concluding Observations

The development of research software faces the same challenges as developing any other kind of software. The software engineering body of knowledge addresses these challenges and is of considerable help. Unfortunately, there is no silver bullet and software engineers are not firefighters who can easily rescue software that has degraded over several years. However, software engineering sometimes provides witchcraft-like techniques, which help best, when applied early.

Research software engineering correctly pushes these techniques into development projects for researchers, but domain-specific adaptations are necessary.

Software development tools for the automation of various activities and wizard-like assistance have enormously increased productivity and simplified the hurdles for newcomers to create pieces of software. Moreover, these tools nowadays enable non-experts to develop significant pieces of software and leverage the knowledge of core software structures, such as persistent storage, communication, compilation, computation orchestration, etc.

Computer science can be very proud of having understood the core technologies, turning scientific knowledge into automatically executable algorithms and frameworks, and embedding these into tools usable by non-experts without having a deep understanding of the internal mechanisms of these tools. This also includes integrated development environments (IDEs) with highly assistive editors, Low Code approaches using scientific or other explicit modeling techniques, automated continuous integration, code analysis, etc.

Computer science can be proud of enabling non-experts to write expert-like software. No one might believe that building even such a simple thing as a one-family house can be done by non-architects, but in software development, we have achieved exactly this. But the tooling infrastructure is still under intense development, and much more can be done.

  • Recommendation 1: It is necessary to build better domain-specific tooling to address the domain-specific challenges of research software. Wizardy-like smart tools help development amateurs and, to some extent, prevent them from having to focus too much on SE skills themselves.

Smart, possibly domain-specific tooling is part of the research infrastructure and needs high quality. It cannot only be a research prototype and thus should be professionally planned, engineered, and managed.

However, there is complexity in research software that doesn’t go away. If the software shall not be a throw-away software, there are additional methodical topics to address that currently cannot be completely automated.

  • Recommendation 2: Researchers that create software for a sustainable, long-lasting infrastructure need to be trained in software engineering skills, which drastically differ from mere programming skills.

Not all software needs to sustain; sometimes simple throw-away experiments are ok, but developers should be aware of the expected outcomes and life expectancies of these outcomes. Their development processes should contain the appropriate measures and mechanisms to achieve their goals.

Researchers need to be aware that software engineering is not only about getting the code right but also involves architectural, design, quality assurance, and management soft-skills to be adopted and lived during a development process.

Because we know that these skills are not easy to be adopted, there is a third relevant possibility:

  • Recommendation 3: Have one or more software engineers be part of your project to get the software and the technical architecture right, adopt the appropriate tools and quality mechanisms. etc.

This can, for example, be done by permanent employment but also by consulting offers, which seems to be an increasingly feasible approach. Various British institutes and Germany’s Helmholtz seems to have started this approach by installing research software engineering consultants with primary software engineering skills and some secondary knowledge about the research domain. This seems to be promising because software engineers are trained to collaborate, and software engineering methods very well assist collaborative approaches.

However, software engineering is a holistic approach, and many strategic decisions have to be taken, therefore:

  • Recommendation 4: Establishing RSE principles is primarily a topic for the management, i.e., the professors and the research institutions, to be addressed adequately.

Finally, we have learned that there are generic software engineering techniques that can be applied in many domains, but due to the domain- specific differences in characteristics, it is also useful to adapt, enhance and possibly create domain-specific techniques, tools, methods, frameworks, etc. This brings us to:

  • Recommendation 5: Establish research software engineering research as a research field over RSE.

RSE research will probably keep us busy for a number of years as a foundational field of research, which obviously is to be executed by SE researchers and not so much by the domain researchers themselves.