The need for scientific software engineering in the pharmaceutical industry

Abstract

Scientific software engineering is a distinct discipline from both computational chemistry project support and research informatics. A scientific software engineer not only has a deep understanding of the science of drug discovery but also the desire, skills and time to apply good software engineering practices. A good team of scientific software engineers can create a software foundation that is maintainable, validated and robust. If done correctly, this foundation enable the organization to investigate new and novel computational ideas with a very high level of efficiency.

Appendix: software processes

Although we didn’t specifically advocate Agile as a software process, one reviewer requested more background on alternative processes that we have had experience with.

The standard cliché is that having some process is better than having no process at all. There are many variables and constraints that lead to the choice of a process: is it a well-defined problem or will scientific research be required, what is the experience level and characteristics of the members of the development team, will team members be working in the same location or even the same time zone, what is the flexibility given by management to try new processes.

We have been involved in very simple processes to very complex processes. One simple process was very effective with highly experienced software engineers. Basically a white-board discussion would take place and then the developer would write a “one-pager” describing what they understood the problem to be, and how they were intended to solve it. One complex process that we investigated was called “clean room” where each developer would have to mathematically prove how a piece of code would behave for all possible input parameters. This is a process we would not recommend because the resulting quality was not worth the amount of time and effort required.

We have experimented with hybrid processes where we mixed-and-matched techniques based on how to most effectively use the development resources. We mentioned Agile above as a process, but even in that case it has been adapted to fit the development teams needs. For example, we do not do pair-programming but instead every check-in is carefully reviewed by another developer. The daily scrums seem to work well, but we no longer make them pure stand-ups limited to 10 min. They are still short (less than 30 min) but they do include longer discussions about over-all design and are not strictly about what is underway or blocking. The sprint transitions are highly effective, they keep the developers motivated to reach the milestones and they give the users a chance to see the product as it grows and make changes and re-prioritize if needed. The drawback to the way we do agile is that it becomes difficult to predict overall timelines which are highly dependent on what changes the users would like. We are still trying to understand this and be able to better predict overall timelines.

Again, we did not advocate the Agile process will work well for all. The process that is most effective for a software team depends on the software teams and the environment.