ELSI by Design

Abstract 

An assessment of ethical, legal, and social implications (ELSI) of digital products is vital to protect users and society against potential damage and misuse. Such assessments are usually conducted before or after product development, thus missing out on the opportunity to address potentially negative implications during the development process. It would be more effective to interlink development and ELSI assessment, especially in agile development projects with their focus on immediate stakeholder feedback.

This research sabbatical focuses on an ELSI-by-Design approach to assess these aspects. I plan to gather requirements, good practices, and pragmatic ideas for making ELSI assessment an integral part of the agile methodology. In addition, I intend to develop patterns and criteria that help to decide if a particular piece of software, like an API or a database, has negative (or positive!) ethical, legal, or social implications.

This way, ELSI assessment can be decomposed in both time (sprint by sprint) and software structure dimension (module by module).

About Myself

I am a full Professor for Software Engineering at Cologne University of Applied Sciences. Before joining the university, I worked for 25+ years in the software industry, mainly as a software architect. My main focus is on methods and patterns for microservice architectures, domain-driven design, and agile development. In this research sabbatical, I intend to widen my scope towards a tight integration of ethical, legal, and social aspects into the development lifecycle.

 

Expert Interviews

In the Research Sabbatical, I plan to conduct a series of expert interviews. As interview partners,I am looking for stakeholders in the development of digital solutions in ELSI-sensitive areas. I am interested in the views of a wide range of roles, like

  • researchers (in academia as well as in companies‘ R&D departments)
  • software develeopers and architects
  • project managers and scrum masters
  • product owners or managers
  • other stakeholders on management and operative level

As I will be in the San Francisco area this summer from July 10 to August 19, I am especially interested in contacts in the Bay area. I would like to compare a European view with the practices found in the US.

Questionnaire

Here you find the questionnaire I am using in the interview,

In order to scientifically evaluate the interview, I will have to record and transcribe it. The transcription will be anonymous, i.e. all references to specific persons, organisations, and projects will be purged. The transcripts will never be published in full. I will use quotes from the interviews in publications, but never in a way that they can be traced to specific person. If you agree to these conditions, please be so kind to sign a declaration of consent that I will bring to the interview (German version).

 

Full Version

Moral vs. Economy: The Challenges Caused by Today’s Digital Innovations

Society often reacts with curious ambiguity to advancements in digital technologies. There seem to be two factions: On the one side, the early adopters, joyfully embracing any new digital gadget, feature, or process; on the other, the sceptics with a deep-rooted mistrust of digital black boxes that send their data into an anonymous cloud, controlled by intransparent AI algorithms.

The gap between these two attitudes seems to widen. Very few people are indifferent towards digital technologies. They either look forward to a brave new world with infinite possibilities – or they are afraid of losing their privacy, their jobs, and their ability to understand the world around them.

As a result, political decision makers, but also science and technology companies themselves, have started to focus on the ethical, legal, and social implications (ELSI) of digital products and processes. Whenever a new digital tool or process is planned to be rolled out in an organization, or a new digital product is about to be launched, several key questions need to be addressed: Is the new digital solution compliant with privacy laws and other legal requirements? Is it ethically justified? What impact will it have on its users and their social community?

How thoroughly these questions are dealt with varies between societies, cultures, and fields of application. In general, societies in Europe tend to worry more than in US and in Asia. An industry sector like medical technology, where human lives are at stake, has a long tradition of reviewing if a new engineering solution is morally sound. Fields like social work or geriatric care are especially sensitive to the social implications of any automation technology.

This growing ELSI awareness has two sides. On the one hand, it is beneficial for society’s control over potential negative implications of new technologies. It is in the nature of liberal western democracies to put human dignity first, and use this principle as a universal yardstick for decision making. Taking ELSI serious means, in essence, taking democratic values seriously.

On the other hand, some countries seem to be more ruthless than others, when it comes to development and deployment of digital products and solutions. Take China and its advancements in Artificial Intelligence to control their own population more efficiently: The Chinese government does not seem overly concerned with the moral implications of their use of digital technologies. A serious ELSI assessment might slow down their ascent to global digital leadership.

Ex-Ante vs. Ex-Post Assessments – a No-Win Situation?

So, are Western democracies doomed to lose in global economic competition, due to their moral qualms about digital transformation?

ELSI assessments can lead to delays and cancellations of digital products and solutions. This might lead to a slowdown of the digital transformation of our society and economy. We seem to stand at a crossroads: Going left, we stick to our beliefs and values, and abstain from developing more advanced digital solutions in order to avoid moral risks. As a consequence, we risk falling behind the global economic leaders. Going right, we compromise on our beliefs and embrace whatever digital solution is possible. In doing so, we are able to compete globally, but run the risk of betraying our values as a society in the long run. So, is this a no-win situation for a liberal democracy?

Maybe we are just need a different, less black-and-white approach. Technology assessments too often follow the pattern of “science proposes, society disposes” (Guston & Sarewitz, 2002, p. 95f.). ELSI is usually checked ex-ante (before a software project even starts), or ex-post (after the digital product development has been completed).

In the case of ex-post assessments, risks or negative implications might have been avoided if the assessment process had started earlier. Whereas, ex-ante assessments often result in projects not being started at all due to excessive caution with regard to potential legal risks or negative ethical or social implications – even if the issues could have been addressed to a satisfactory level during development.

This effect seems to be even more pronounced in domains with low IT affinity and a high degree of ingrained skepticism towards technology, like social work or geriatric care. As a consequence, the gap between “digital” and “analogous” domains is widening. Even where nurses and social workers could benefit from innovative digital solutions, their deployment is often blocked by ELSI constraints.

The Solution: ELSI-by-Design

The ex-ante / ex-post assessment pattern is inherently a waterfall approach. Waterfall development models are based on the assumption that projects go through a development phase, pass some acceptance gateway, and then move on to the next phase – ideally never to return to that previous phase. Today, ELSI assessments often take the role of such a gateway in between development phases – either between initial concept and detailed design (ex-ante), or between acceptance test and rollout (ex-post).

The waterfall approach (“finish one phase for good, and then move on to the next”) frequently leads to a clash with reality. Fortunately, the software industry has proven in the last 15 years that it can do better. Agile development methods focus on stakeholder interaction, short feedback loops, and a rigid adherence to a Definition of Done as a quality yardstick. So the question is, why don’t we integrate the ELSI assessment into this methodology?

Incorporating the ELSI assessment into the agile methodology requires to break it down in two dimensions.

  • Time-wise: Instead of having one comprehensive ELSI check before or after development, the scrum teams need to able to perform “micro-assessments” every two to four weeks, in synchronization with the sprint structure.
  • Structurally: Instead of checking a large, monolithic digital solution (software, hardware) in one piece, assessments should be performed component by component, allowing for a divide-et-impera

An assessment methodology that can effectively and efficiently be broken down in these two dimensions will allow for an organic ELSI integration into the (agile) development process. The effort for the micro-assessments need to be small and manageable – only then can their fulfillment be part of the Definition of Done of a scrum team.

This approach satisfies two requirements by Guston and Sarewitz (2002, p. 95). It allows a continuous reflection on ethical, legal, and social implications of software and hardware, as they a natural part of the non-functional and functional requirements. In addition, it overcomes the separation between those who create technology from those who assess it. The responsibility for these aspects is handed back to the agile development team itself – which is very
agile in nature, as an agile team is supposed to assume a 360° responsibility for its products. This would be a true ELSI-by-Design approach.

Decomposing ELSI Assessments In Time Dimension

Inspiration for the decomposition in time dimension (1) can be gained e.g. from the Double-Diamond Model of Design in Design Thinking (Norman 2013, p. 217-219). It consists of two iterative phases, „finding the right problem“, and „finding the right solution“.

This model is based in a human-centered design approach, which facilitates stakeholder involvement into the development process. With ethical, legal, and social aspects are broken down into assessable categories (like described e.g. in Rohde, 2018), the overall result could be a practical guide with checklists and good practices for a qualitative assessment.

Social aspects, and to some extent also ethical ones, could be also addressed by Participatory Design (Schuler, 1993) or Inclusive Design (Clarkson et al., 2003). This is subject to further and deeper analysis during the research sabbatical.

Decomposing ELSI Assessments In Structure Dimension

An organic integration of ELSI assessments into the software development process will only be possible if the software can be assessed component by component. In that case, each feature planned and developed in a sprint could be checked independently.

If the software application deals with personal data, data protection laws mandate strict rules on privacy protection. Unauthorized access to sensitive information like, say, a person’s health record, must strictly be prevented. Decomposing this goal in the structure dimension means assessing each user interface, database, API with regard to their access rules. For many legal aspects, this principle is fairly obvious to implement, especially for data privacy.

For ethical and social implications of digital products, it is less straightforward. Let’s assume we plan to implement an AAL (ambient assisted living) software solution for elderly patients suffering from dementia. If our software is set up to prevent a demented person from certain activities (e.g. cooking) where she could hurt herself, we have a typical ethical dilemma on our hands: Restriction of personal freedom vs. protection of health and the ability to live in one’s own environment instead of a nursery home.

Is it ethically sound to develop such a digital solution? Under what conditions can it be justified to develop and deploy this software? An ELSI assessment would have to answer these questions. Is it possible to decompose such an assessment in the software structure dimension? Can we decide if, say, an API or a database is ethically sound?

Software Patterns for ELSI-by-Design

ELSI-by-Design requires not only an assessment approach that is organically integrated into the software development process. The developers also need means to deal with possible ELSI concerns, or, even better, to design their software in such a way that it is “automatically” compliant.

This is the core promise of the “by design” approach. It means that ELSI constraints are woven into the basic building blocks of the software. Let’s go back to the above mentioned health monitoring application: If we had a programming language that would allow the programmer to use “data protection entities” instead of Java classes, the entity Patient (being the digital model of a real person) could

  • encapsulate its sensitive data, blocking method calls from unauthorized other entities,
  • have a built-in self-destruction date (e.g. six months after last access), or
  • provide certain (partially) obfuscated views, according to the authorization level of a caller; this way, production data could be used in tests, where name and date of birth are scrambled automatically.

Research Sabbatical Goals

The research sabbatical will focus on gathering requirements and good practices for ELSI assessments, and on compiling ideas for an ELSI-by-Design approach. Besides a thorough literature study, a series of expert interviews is planned. The target group for these interviews consists of …

  • specialists frequently dealing with ELSI aspects, like data protection or compliance officers,
  • scientists researching ELSI assessments and ELSI-by-Design approaches,
  • experts on ethical, social, and legal aspects of digital products (from a business or from an academic background),
  • experienced product owners and scrum masters who have worked in ELSI-sensitive software projects, and
  • decision makers dealing with ELSI assessment of their digital products or IT solutions (product managers, CIOs, IT managers, etc.)

As I will be in the San Francisco area from July 9 to August 20 this summer, I am especially interested in contacts in the Bay area. If the resulting data will be sufficient, I will try a comparison Germany / US (or California) for the focus areas of this research sabbatical.

Key Questions

Literature research and expert interviews will cover the following key questions:

  • What aspects are most relevant in the ethical, legal, and social dimension? How can the ethical, legal, and social dimension be decomposed for assessment purposes?
  • Which domains are specifically ELSI-sensitive with regard to digital products? Which aspects play a prominent role in which domain?
  • What is the experience of software development experts with ELSI assessments? Do they feel supported or hindered in the process? Are there good practices they can share?
  • What are requirements for an ELSI-by-Design methodology? What approaches exist? Are they used in practical software development projects, and to what avail?

Literature

An extensive literature research will be the starting point of the research sabbatical. Therefore, this literature list can only be preliminary and vastly incomplete. Only the references quoted in this program description have been listed.

  • Clarkson, J., Coleman, R., Keates, S., Lebbon, C. (Ed.) (2003): Inclusive design: Design for the whole population, London: Springer.
  • Guston, D. H., & Sarewitz, D. (2002). Real-time technology assessment. Technology in Society, 24(1–2), 93–109. https://doi.org/10.1016/S0160-791X(01)00047-1
  • Rohde, N. (2018): Gütekriterien für algorithmische Prozesse – Eine Stärken- und Schwächenanalyse ausgewählter Forderungskataloge. Bertelsmann Stiftung. Abgerufen 24.08.2018 von https://www.bertelsmann-stiftung.de/index.php?id=5772&tx_rsmbstpublications_pi2%5bdoi%5d=10.11586/2018027
  • Norman, Don (2013). The Design of Everyday Things: Revised and Expanded Edition. Basic Books.
  • Schuler, D. (Ed.) (1993): Participatory design. Principles and practices. Hillsdale NJ u.a.: Erlbaum.