The multilingual contents of the site are the result of an automatic translation.
 

 
 
 
 
 
Français
English
Français
English
 
 
 
View
 
 
 
 
 
View
 
 

Other sources

 
Saut de ligne
Saut de ligne

Towards a ban on autonomous lethal weapon systems?

Cahiers de la pensée mili-Terre n° 47
Science & technology
Saut de ligne
Saut de ligne

While the European Parliament has just adopted a resolution calling for the adoption of civil law rules on robotics[1], the US and Europe are still waiting for the adoption of a new civil law on robotics.Member States of the Convention on Certain Conventional Weapons (CCW) are seeking to regulate the use of 'soldier robots', referred to as Autonomous Lethal Weapons Systems (ALWS). Several States and non-governmental organizations are advocating a ban on SALA.

1] European Parliament resolution of 16 February 2017 containing recommendations to the Commission on civil law rules on robotics.


"Maintaining the soldier's humanity in the future battle will be a major challenge. It will require the army to remain vigilant to the risk of distorting military action and to the risk of the cold gesture of remote action, which is a source of indifference".

Future land action, EMAT, September 2016.

Since 2014 a meeting of experts on Autonomous Lethal Weapons Systems (ALWS) has been held annually under the auspices of the Convention on Certain Conventional Weapons (CCW) [1].

The purpose of the CCW is to prohibit or restrict the use of certain specific types of weapons that are deemed to inflict unnecessary or unjustifiable suffering on combatants or to affect civilians indiscriminately. It is a framework convention to which five protocols have been annexed relating to unexploded ordnance, mines, booby traps and other devices, incendiary weapons, blinding laser weapons and explosive remnants of war.

The last expert meeting on SALA was held from 11 to 15 April 2016 with diplomatic delegations and non-governmental organizations (NGOs) such as the International Committee of the Red Cross (ICRC), Human Rights Watch and the International Committee for Robot Arms Control (ICRAC ). The mission of this group of experts, chaired by Michaël Biontino, Germany's Permanent Representative to the Conference on Disarmament, was carried out by a group of experts from the United States, Germany and the United Kingdom.s Permanent Representative to the Conference on Disarmament, was "to deepen the discussion on issues related to the emergence of technologies in the field of ASALS, in the context and in the light of the CCW" [2].

2] [2] In December 2016, at the Fifth Review Conference of the Parties to the CCW, a recommendation was adopted for the establishment of a Special Commission on the establishment of a European Centre for the Study of the Use of Nuclear Weapons (CCW) in the field of SALW.In December 2016, at the Fifth Review Conference of the Parties to the CCW, a recommendation was adopted to establish a Group of Governmental Experts by 2017 to "explore and make recommendations regarding the emergence of technologies in the field of ASALS". This group should in particular take into consideration two main issues: the definition of ASALS and the application of the law of armed conflict in the context of the use of ASALS[3].

3] This decision reflects the willingness of many states to regulate the use of autonomous weapons systems in theatres of operation.

Attempt to identify criteria for the definition of an ALWS

Described as a "soldier robot" or "killer robot", international law does not define what a SALA is. At the expert meeting in April 2016, several definitions were proposed.

The Vatican defines an "autonomous weapon system" as a "weapon system capable of identifying, selecting and initiating an action on a target without human intervention" [4].

For the French delegation, a SALA must have the following characteristics: total autonomy, implying a total absence of human supervision and capable of adapting to its environment, aiming and firing with lethal effect [5].

5] An "autonomous weapon system" is defined by Switzerland as "a weapon system capable of carrying out missions in response to an emergency situation.A "stand-alone weapon system" is defined by Switzerland as "a weapon system capable of performing missions governed by international humanitarian law by partially replacing the human element in the use of force, in particular in the target identification cycle" [6].

6] NGOs also propose to legally define SALAs. For the ICRC, an "autonomous weapon system" refers to "any weapon system that is autonomous in its critical functions. This implies that the weapon system can select and attack targets without human intervention".

On reading the definitions presented, several characteristics specific to ASALS seem to emerge:

  • Autonomy and the absence of human intervention or supervision;
  • the ability to select and attack a target;
  • the lethal effect of the weapon system;
  • adaptation to its environment;
  • its submission to the law of armed conflict.

A study of these definitions reveals, first of all, that there is no consensus on what the SALA object itself is called. Some refer to it as an autonomous weapon system, others as an autonomous weapon or SALA.

The definition of an ASALA goes through the necessary step of defining the concept of autonomy. Indeed, there are several levels of autonomy, ranging from the existence of human control or supervision ("Human in the loop") to totalautonomy("Human out of the loop"). In addition, there may be different types of autonomy, such as operational autonomy, whereby the system performs low-level control functions, and decision-making autonomy, whereby the system will make substantial decisions without communication with the human operator. Finally, the question also arises as to whether the notion of autonomy should include autonomous learning capabilities. The definition of the concept of autonomy, which is related to a weapon system, is therefore a necessary precondition for any definition of SALA and any reform of international law.

Another issue is raised with regard to the definition of SALA. To date and in the state of technological progress, completely autonomous weapon systems do not exist. Indeed, autonomy is today possible for certain specific tasks, but is not yet usable for an overall mission including orientation, adaptation to the environment and the selection and subsequent attack of a target. It may therefore appear premature to seek a legal, and therefore binding, definition for a non-existent system. The adoption of such a definition in an international convention would, for the time being, have the sole advantage of allowing States to use the same terminology for possible adaptation of the law of armed conflict.

Need to adapt the law of armed conflict?

The law of armed conflict, consisting mainly of the four Geneva Conventions of 1949, the Hague Conventions and the texts relating to arms control, is governed by four fundamental principles: humanity, distinction, proportionality, necessity [7]. 7] This right applies in times of armed conflict and places obligations on members of the armed forces, particularly in the event of the use of autonomous weapons systems. Indeed, there is international consensus on the need for SALAs to respect the law of armed conflict.

The first legal provision directly applicable to SALAs is Article 36 of Additional Protocol I to the Geneva Conventions, which states that "in the planning, development, acquisition or adoption of a new weapon, means or method of warfare, a High Contracting Party has the obligation to determine whether its use would be prohibited in certain circumstances or under any circumstances". Accordingly, States must, from the study phase and before using a SALA, ensure that it complies with the law of armed conflict.

This necessary prior analysis is today the main argument used by many organizations, foremost among which is Human Rights Watch[8 ], to call for the prohibition of SALAs. Some states such as Cuba, Egypt, the Vatican and Pakistan also call for such a preventive ban. Indeed, it appears that, to date, these autonomous systems cannot comply with the fundamental principles of the law of armed conflict. They have not yet demonstrated their ability to show humanity or to distinguish between an armed enemy ready and willing to open fire and an armed enemy that surrenders and against whom it is prohibited to open fire. The difficulty with such an outright prohibition of ASALs lies, as stated above, in the fact that fully autonomous weapon systems are not completely self-contained. While the law must enshrine a use, the approach would therefore consist, as a preventive measure, of prohibiting a system that does not exist and which therefore could not be fully assessed in terms of positive law.

Another possibility for the evolution of the law, apart from the prohibition, would be to adopt a sixth protocol to the CCW which would lay down a specific regime applicable to Sala SALAs. This regime could then deal with the conditions for the design of Sala SALAs, the cases in which Sala SALAs may be used (e.g. by imposing human supervision) and the rules on the use of the SALA.This regime could then deal with the conditions for the design of Salaries, the cases in which Salaries may be used (e.g. by imposing human supervision) and rules on liability for damage caused by Salaries (allowing to specify which of the algorithm designer, the weapon system manufacturer or the military commander could be liable). The essential advantage of this new regulation would be to allow manufacturers to integrate these rules into the heart of the systems, particularly in the algorithms, from the design stage of the SALA. Ethical and legal approaches must not be obstacles to technological progress. In any event, it now seems necessary to keep human beings in the loop of decisions that can be taken by Sala systems.

Need to keep man in the loop

At the last expert meeting on SALA, the Japanese Ministry of Defence took the clear position that the country would not "develop robots, without men in the loop, capable of harming human life" [9].

9] Disconnecting humans from an autonomous weapon system with lethal capabilities presents many dangers. It would dehumanize the battlefield and would amount to the suppression of any moral agent in the most critical situations where human intelligence and intuition cannot be replaced today. Moreover, this dehumanisation would lead to a considerable decrease in the degree to which force could be engaged, thus risking the escalation of conflicts.

Moreover, since SALAs do not have legal personality[10], it would be almost impossible, in the absence of precise legal rules, to determine liability when damage is caused as a result of a decision taken autonomously by a SALA.

Finally, total autonomy of the SALA would lead to the risk of unpredictability of the reactions of the weapon system, the impossibility of controlling the SALA's actions, and the risk of the SALA's being unable to take the necessary measures to prevent damage.Finally, total autonomy of the SALA would lead to the risk of unpredictability of the weapon system's reactions, the impossibility of controlling the effects of the weapon, and would establish a break in the link between that system and the military commander, thus making real operational control impossible.

Some states, such as Poland, argue that the legal link between man and the autonomous weapon system should be based on the concept of "Meaningful human control", implying that certain operational constraints (such as the definition of the target) should be imposed by man. 11] This condition would ensure that war is not totally dehumanized.

However, decision-making autonomy should not be banished from theatres of operation. There are indeed many missions in which autonomous systems can be of real use. This is the case for mine clearance, humanitarian missions, intelligence or intervention in contaminated environments. Combat by robots must remain collaborative and must not be delegated in its entirety to them.

--------------

A lawyer at the Paris Bar specialising in defence and security law, Lieutenant François GORRIEZ serves in the operational reserve of the Paris Bar.after having followed the initial training of reserve officers at the Saint-Cyr Coëtquidan Military School. He is currently assigned to the Ile de France defence and security zone headquarters, where he is conducting legal studies as part of Operation Sentinel. He studied law in France and Canada and was a youth auditor at the National Institute for Higher Studies in Security and Justice (INHESJ) in 2013. He chairs the Military Robotics Commission of the Robot Law Association.

1] Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons Which May Be Deemed to Be Excessively Injurious or to Have Indiscriminate Effects,10 October 1980.

2] Introductory statement by Michaël Biontino, Permanent Representative of Germany to the Conference on Disarmament, Geneva, 11 April 2016 (all the texts cited in this article are available at www.unog.ch).

3] "Advanced Version, Recommendations to the 2016 Review Conference", submitted by the Chairperson of the Informal Meeting of Experts.

4] "ElementsSupporting the Prohibition of Lethal Autonomous Weapons Systems", working papers presented by the Vatican (all the elements cited are freely translated from English to French).

5] "Nonpaper characterization of a laws", working papers presented by France.

6] "Towardsa "compliance-based" approach to LAWS", working papers presented by Switzerland.

7] For an in-depth study of the law of armed conflict, see the Manual on the Law of Armed Conflict, 2012 edition, Directorate of Legal Affairs of the Ministry of Defence.

8] "Making the Case, The Dangers of Killer Robots and the Need for a Preemptive Ban", Human Rights Watch, 9 December 2016.

9] "Japanview's on issues relating to LAWS", working papers submitted by Japan.

10] On the legal personality of robots, see in particular A. Bensoussan, J. Bensoussan, "....Law of Robots", Larcier, 2015.

11] Killer Robots and the Concept of Meaningful Human Control Memorandum to Convention on Conventional Weapons (CCW) Delegates, Human Rights Watch, April 11, 2016.

Séparateur
Title : Towards a ban on autonomous lethal weapon systems?
Author (s) : Lieutenant (CR) François GORRIEZ
Séparateur


Armée