Analysis of OODA Loop based on Adversarial for Complex Game Environments

03/25/2022
by   Xiangri Lu, et al.
139
0

To address the problem of imperfect confrontation strategy caused by the lack of information of game environment in the simulation of non-complete information dynamic countermeasure modeling for intelligent game, the hierarchical analysis game strategy of confrontation model based on OODA ring (Observation, Orientation, Decision, Action) theory is proposed. At the same time, taking into account the trend of unmanned future warfare, NetLogo software simulation is used to construct a dynamic derivation of the confrontation between two tanks. In the validation process, the OODA loop theory is used to describe the operation process of the complex system between red and blue sides, and the four-step cycle of observation, judgment, decision and execution is carried out according to the number of armor of both sides, and then the OODA loop system adjusts the judgment and decision time coefficients for the next confrontation cycle according to the results of the first cycle. Compared with traditional simulation methods that consider objective factors such as loss rate and support rate, the OODA-loop-based hierarchical game analysis can analyze the confrontation situation more comprehensively.

READ FULL TEXT VIEW PDF

Authors

page 2

04/23/2020

GUT: A General Cooperative Multi-Agent Hierarchical Decision Architecture in Adversarial Environments

Adversarial Robotics is a burgeoning research area in Swarms and Multi-A...
09/19/2018

TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

Starcraft II (SCII) is widely considered as the most challenging Real Ti...
04/23/2019

Fuzzy Q-Learning Based Multi-Agent System for Intelligent Traffic Control by a Game Theory Approach

This paper introduces a multi-agent approach to adjust traffic lights ba...
01/12/2018

Simulation of the propagation of a cylindrical shear wave : non linear and dissipative modelling

The simulation of a wave propagation caused by seismic stimulation allow...
11/17/2021

Shifted Brownian Fluctuation Game

This article analyzes the behavior of a Brownian fluctuation process und...
05/04/2020

Construction and Elicitation of a Black Box Model in the Game of Bridge

We address the problem of building a decision model for a specific biddi...
12/06/2021

On the complexity of Dark Chinese Chess

This paper provides a complexity analysis for the game of dark Chinese c...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Modern warfare situations are fluid and require rapid ad hoc countermeasure decisions. Military technology is also advancing with the development of artificial intelligence, and the future warfare can be said to be dominated by the participation of few combatants and the unmanned nature of actual combat. Here to use NetLogo software simulation to build a dynamic projection of both sides tanks in complex terrain confrontation thought

[1], both sides confrontation terrain environment as shown in Figure 1.

Figure 1: Diagram of the Game Environment of both Sides.

The dynamic projection of the two sides’ tank confrontation in complex terrain is conceived as a situation where the Red side is preparing to achieve a strike against the Blue side’s chain of command in the complex terrain of the mountain peaks and valleys, and the Blue side intends to send a significant portion of its armor numbers to intervene in order to achieve the objective of stopping the strike and maintain the intention of containing the Red side’s course. After the Red side learns of the Blue side’s strategic intent, it sends its absolute superior armor numbers to execute counter-intervention operations at the first opportunity. The OODA cycle connects the two sides in the confrontation process, forming a complete game process, and finally using hierarchical analysis to provide a reference for the weight value of the mixed game strategy. -Decision-action” cycle. The ”Orientation” and ”Decision” steps are the most critical in the OODA cycle, and if the enemy’s threat is misjudged or the surrounding environment is misunderstood, then the judgment and decision will be wrong[2, 3, 4, 5]. The goal is to interfere with the enemy’s OODA cycle by taking prompt action after the first OODA cycle is completed by the own side. In constructing a complex combat system adversarial analysis, NetLogo modeling software will be used to model the system dynamics of the game process on both sides[6, 7, 8, 9, 10, 11].

2 Complex system confrontation model construction theory

In the 1970s, the U.S. Air Force Colonel Boyd proposed the ” Loop” theory. is the abbreviation of the four English words Observation, Orientation, Decision, and Action. Its meaning is to observe, Orientation, make decisions, and Action.[11, 12]The OODA Loop theory (Observation, Orientation, Decesion, Action) proposed by John Boyd provides a way to describe conflicts in the cycle of detection, judgment, decision-making, and action. Decision-making processing is widely used in the study of military issues. The OODA loop theory points out that the characteristic of command and control operations is to fight in accordance with the observation-judgment-decision-action loop, which can be divided into the following four stages: The first stage is observation (Observation), mainly through various sensing equipment and The use of the network for intelligence collection, for example, early warning detection information, battlefield environment information, enemy coordinate information, etc. The second stage is Orientation. The battlefield situation is changing rapidly and there is great uncertainty. Judgment is to convert data into useful information. Effective and rapid judgment of the results can assist the commander in making correct decisions and even changes. The battlefield situation. The third stage is Decision, where the commander clearly formulates a mission plan and issues a battle plan. The rapid formation of decision-making is of decisive significance to the battlefield situation. The fourth stage is action. Actions on the battlefield are usually described as taking corresponding operational deployments and performing actions based on the operational plan issued by the superior.

The OODA Loop has periodicity and nesting. Periodicity means that when a loop ends, another new loop begins. The size of the period is related to the scale of combat forces, spatial scope, and combat style; nesting means that the loop and The loops are related in the form of nesting. For example, in the tactical combat system, the smallest OODA Loop is the firepower closed-loop control loop of the close-range weapon system. There are also OODA Loops at the unit level. These command and control loops are nested within each other, and the inner loop The cycle is short and the outer loop cycle is long.

In combat, the opposing parties will continuously observe the surrounding environment, obtain relevant information, judge threats, make immediate adjustments, make decisions, and take corresponding actions. This is specifically reflected in the competition of OODA Loop response capabilities. If they can try to shorten their own side The enemy’s OODA Loop and increase the enemy’s OODA Loop as much as possible. New actions are initiated before the enemy responds to its previous action, causing the enemy to lose the ability to react, so that the initiative can be mastered and a huge advantage can be obtained..The schematic diagram of the Loop is shown in Figure 2.

Figure 2: Diagram of the Game Environment of both Sides.

The first is observation. Before the implementation of command operations, a clear observation plan is required. The two sides started by observing the new situation from the beginning of the war and the reaction of the other side. In the experiment, the red and blue sides were simulated and designed for the Loop, and the deployment was judged by observing the strength of the opposing sides before the war. The second is to judge and observe the situation and make reasonable decisions. In order to reflect the process of dynamically deploying the amount of armor, the slider value adjustment in the judgment and decision-making link in is used. At the end, the actual output of the red and blue sides is used as the execution link, and the amount of armor that needs to be dispatched for the next Loop is made according to the previous three links.[13, 14, 15]

The Loop is a closed ring for decision-making and control actions, and it has the characteristics of a periodic cycle. The law of future warfare is that combat units that make quick judgments and decisions will take the initiative. From the Loop theory, it can be concluded that the timeliness of military operations research. Each time period of the system combat command cycle depends on different factors. and are defined separately as the mobile deployment rhythm and engagement rhythm of the combat system in the physical domain. is the decision-making rhythm of the combat system in the cognitive domain,is the rhythm of interaction between the information domain, the physical domain, and the cognitive domain,is defined as the combat time of the observation, judgment, decision-making and action links of the OODA Loop,then complete the OODA Loop and the time required for each link, as shown in Equation 1 and Equation 2.

(1)
(2)

According to Shannon’s information entropy theory, the probability of occurrence of each part of the observation, judgment, decision-making and action links in the OODA Loop is defined as

.Then the information entropy of each part of the observation, judgment, decision and action links in the OODA Loop is shown in Equation 3.

(3)

In the formula, n indicates that the OODA Loop can be multiple cycles.

The information entropy of all OODA Loop of complex operations systems is shown in formula 4.

(4)

Statement: According to the information entropy of each link of the OODA Loop, the information entropy of the OODA Loop of the complex combat system can be described. An OODA Loop is composed of four links to solve the maximum information entropy of all OODA Loop in a complex combat system.The information entropy of all OODA Loop of the system is ,Then prove .

The information entropy of the complex system is abbreviated as Equation 5, and the constraint condition is Equation 6. The problem of solving the maximum value of the original Equation 4 is equivalent to the minimum value of Equation 5..

(5)
(6)

Then construct the Lagrangian function as formula 7

(7)
(8)

……

(9)
(10)

The above equation can be solved.

(11)

So .

The information entropy value of all OODA Loop of a complex combat system is a system that increases as the number of system cycles increases, indicating that the system is a highly complex entropy increasing system and the entropy value is in a state of divergence. For the highly complex entropy increase system, Swarm, and other software are needed to assist the analysis. has the characteristics of simple model description language and rapid modeling. In the third section, modeling specifically analyzes the process of confronting both parties.

3 Netlogo modeling based on OODA Loop

is a programmable modeling environment used to simulate natural and social phenomena.[16, 17, 18]When the internal operating rules of a complex system are not obvious and it is difficult to grasp the rules, software can be used to simulate the system.[19]The red and blue confrontation experiment simulation is based on the hardware platform of and the software platform of . Taking the construction of the red OODA Loop as an example, the modeling is carried out from four aspects: observation, judgment, decision-making and action.The research of complex nonlinear combat with multi-agent systems in modern combat is an important direction of war simulation. NetLogo is a powerful multi-agent simulation tool, which greatly improves the simple analytical model and analytical combat methods. Due to the simplification of the research, this article limits the combat process to the micro process of OODA Loop combat. In the future, the simulation can be extended and expanded to achieve OODA Loop stacking and cumulative confrontation. Use NetLogo to improve the level of human-computer interaction, establish models and calculation formulas with a higher degree of reduction, and analyze the process and results of simulation operations to draw more credible evaluation conclusions. By analyzing the problems, finding the gaps, laying the foundation for the formulation of combat plans, so that simulation can truly serve the combat decision-making. The functions of each module in the OODA Loop are defined as follows: Based on practical foundation and facts, the difference in the amount of armor between the two opposing parties is relatively large, so the party with the less armor will take a longer time to think and need more time to deploy armor to achieve the goal of winning with less. Select the value of the slider for the judgment and decision-making links of the red and blue parties. For example, 500 armors of the red side and 400 armors of the blue side appear at one of the nodes of the OODA Loop, and Then the red and blue sides in the NetLogo simulation software slider affiliation value is adjusted to 1 and 0.5 respectively. as shown in Figure 3, Figure 4.

Time adjustment module of the judgment link.

Time adjustment module for the decision-making link.

In order to quickly select , , , and values, some judgment and decision time values are now listed, as shown in .

OODA combat Loop part Orientation and Decision-making time value reference table. Red and Blue Armmored Quantity(X,Y) Red Determine the time value Blue Determine the time value Red Decision time value Blue Decision time value (500,200) 0.3 1.0 0.3 1.0 (500,300) 0.4 1.0 0.4 1.0 (500,400) 0.4 0.7 0.4 0.7 (600,200) 0.2 1.0 0.2 1.0 (600,300) 0.3 1.0 0.3 1.0 (600,400) 0.2 0.5 0.2 0.5 (700,200) 0.1 1.0 0.1 1.0 (700,300) 0.1 0.9 0.1 0.9 (700,400) 0.2 0.7 0.2 0.7

3.1 Investigation link

The reconnaissance behavior model mainly describes the combat behavior of various detection, reconnaissance equipment and various sensing devices, which is the main means to obtain intelligence information on the battlefield. In the OODA Loop, the information acquisition of the investigation link is related to the pre-war deployment, command and decision-making, etc. The Red investigation consists of three modules, the speed at which the Red obtains intelligence (), the speed preventing the Red from acquiring intelligence (), and the speed the Red actually obtains intelligence (). The system dynamics module composed of the three modules is as shown in Figure 5.

Dynical model of Red Square investigation link.

3.2 Judgment link

This part of the OODA Loop is to analyze the collected intelligence, determine the authenticity of the intelligence, and study the current situation. It is mainly completed by the red judgment () and the red judgment time () in the combat elements of the command system. It is an accusation The processing center of the information flow in the system. The red side judgment link is composed of two modules, namely, red side judgment () and red side judgment flow input (). The system dynamics module composed of two modules is shown in Figure 6.

Dynical model of red square judgment link.

3.3 Decision link

The decision-making link in the OODA Loop is based on the situation judgment, assigning the action and task plan, mainly by the red decision () module in the combat elements of the accusation system, generating a new command and control information flow. The decision link consists of red temporary storage module () and red decision module (). The system dynamics model of the decision-making link is shown in Figure 7.

Red square decision link dynamics model.

3.4 Action link

The action link in the OODA Loop is the corresponding action behavior based on the decision made and the designated action plan, mainly through the red square armor module (), the red side loss factor module (), the red side actual output module (), and the blue side Decision Module (),It is the specific implementation of the charge system information flow,as shown in Figure 8.

Red square action link dynamics model.

The dynamic model of the Observation, Orientation, Decision, Action link of the blue OODA Loop is shown in Figure 9 below.

(a)Dynical model of blue fang investigation link.

(b)Blue side judgment link dynamics model.

(c)Blue square decision-making link dynamics model.

(d)Blue square action link dynamics model.

The logic pattern of Blue with OODA loop.

4 Traditional methods are compared with OODA Loop

The simple system uses two influencing factors as the OODA Loop of NetLogo simulation. Compared analysis shows the difference between the two methods and whether OODA Loop can improve operational thinking.

4.1 Simple system

Assuming the amount of red and blue armor, the two sides fight according to certain rules of engagement. Both parties then adjust their armor numbers and operational capabilities to simulate their operations. The kinetic model of a simple system is shown in and .The combat system is a complex system composed of personnel, equipment, and organization. The combat process is affected by many factors such as combat missions, technical support, logistical support, suddenness, morale, and command art. It is very difficult and tedious to comprehensively analyze and model the factors in the combat process. Therefore, the combat process is specifically abstracted as the link of dynamic combat interaction, and only the system process of red and blue tank confrontation is analyzed, which can also play a good simulation role. By adjusting the number and combat capabilities of the tanks on both sides, let them conduct simulation operations.

Simulate the number of tanks on both sides, represented by R-Tank and G-Tank. As the confrontation between the two warring parties continues, the number of tanks will show a certain consumption trend. If the number of subsequent tanks is supplemented, it will lead to a certain decrease or increase in the scale of the tanks of both sides. The confrontation between the two sides is a fierce process. The number of tanks is high. In order to ensure the continuity of combat effectiveness, continuous replenishment of tank armor is required. R-Input and G-Input are used to indicate the number of replenishment of the red and blue sides. At the same time, use R- lost and G-lost indicate the amount of equipment lost by both red and blue. There are four variables in this model, namely the loss rate of the red and blue tanks and the replenishment rate of the red and blue tanks. R-lost-rate, G-lost-rate, R-input-rate and G-input-rate are used respectively. Express. In the process of this model test, the Red Army tanks are replenishing (R-input) and losing (R-lost). The confrontation is carried out step by step. Due to the short process of modern warfare and the fierce confrontation, the speed of equipment replenishment is often lower than that of The speed at which weapons and equipment are consumed.

Schematic of simple system dynamics model.

Schematic diagram of the red and blue confrontation results of the simple system.

4.2 OODA Loop system

Through simulation of red and blue OODA Loop by NetLogo in Fig.5-Fig.9, the dynamic model of complex combat system is shown in Figure 12, and the schematic diagram of red and blue confrontation results of OODA Loop system is shown in Figure 13.

Figure 3: Dynamical models of complex combat systems based on OODA Loop.

Schematic diagram of the red and blue confrontation results of the OODA Loop system.

4.3 OODA Loop system feasibility verification

The OODA cycle complex OS scheme is selected according to the Operations Research Hierarchy Analysis Method (The Analytic Hierarchy Process, or AHP). Hierarchical analysis is a systematic and quantitative approach proposed by operations strategist Professor T. L. Saaty in the early 1970s.

The analytic hierarchy process refers to a complex multi-objective decision-making problem as a system, the goal is decomposed into multiple goals or criteria, and then decomposed into multiple indicators (or criteria, constraints) several levels, through the qualitative index fuzzy quantitative method to calculate the level Single ranking (weight) and total ranking are used as a systematic method for target (multi-index) and multi-scheme optimization decision-making.The analytic hierarchy process is to decompose the decision-making problem into different hierarchical structures in the order of the overall goal, the sub-goals of each level, the evaluation criteria, and the specific investment plan, and then use the method of solving the eigenvectors of the judgment matrix to obtain each level The priority of an element to an element of the previous level, and finally the weighted sum method is recursively merged and the final weight of each alternative plan to the overall goal, and the one with the largest final weight is the optimal plan.The analytic hierarchy process is more suitable for decision-making problems with hierarchical and interlaced evaluation indicators, and the target value is difficult to describe quantitatively.

[20]

First OODA Loop complex CS scheme hierarchical hierarchy, as shown in Figure 14. The hierarchical hierarchy includes three levels, target, criterion and scheme. The target layer selects options for complex combat systems; the criterion layer includes four modules: reconnaissance, judgment, decision, and execution; the program layer includes three schemes: support, continuation and retreat.

Hierarchical structure of OODA Loop complex combat system plan.

Secondly, according to the scale value table of the analytic hierarchy process, the consistency matrix of the complex combat system plan to the four criterion modules (as shown in and ) and the consistency matrix of the three combat plans to each part of the OODA Loop are constructed. Due to space limitations, here only the consistency matrix of the four criterion modules of the complex combat system plan and the selection weight matrix of the complex combat system plan (as shown in Table 4) are constructed here, and the consistency of the three combat plans for each part of the OODA Loop The matrix will be explained in the appendix. botrule

The scale value matrix of the four criterion modules of the complex combat system plan. Complex combat system plan selection(A) Observation Orientation Decision Action Observation 1.0000 3.0000 4.0000 7.0000 Orientation 0.3333 1.0000 2.0000 3.0000 Decision 0.2500 0.5000 1.0000 2.0000 Action 0.1429 0.3333 0.5000 1.0000 Sum of scaled values 1.7262 4.8333 7.5000 13.0000

Consistency matrix of complex combat system plan to four criterion modules. Complex combat system plan selection Observation Orientation Decision Action Weights(w) Aw Observation 0.5793 0.6207 0.5333 0.5385 0.5679 2.2933 Orientation 0.1931 0.2069 0.2667 0.2308 0.2244 0.9029 Decision 0.1448 0.1034 0.1333 0.1538 0.1339 0.5357 Action 0.0828 0.0690 0.0667 0.0769 0.0738 0.2967 According to the above table data, the software is used to calculate the value of its natural attributes to judge whether the matrix construction conforms to logical consistency. Among them, the characteristic maximum value =4.0206, , , , since , the consistency matrix is logical. Selection Weight Matrix of Complex Combat System Scheme. Complex combat system scheme selection Comprehensive weight ratio Support weight ratio Continue with the operational weight ratio Retreback weight ratio Observation 0.5679 0.4577 0.1263 0.4160 Orientation 0.2244 0.5571 0.3202 0.1226 Decision 0.1339 0.5571 0.3202 0.1226 Action 0.0738 0.7015 0.2267 0.0718 Weighted weight ratio 0.5113 0.2032 0.2855 According to the selection weight matrix of the complex combat system scheme, it can be concluded that in the simulation combat, it is not the mode described by the simple system. The two sides are simply fighting, and the amount of armor continues to decline; instead, through the description of the combat mode like the OODA Loop, the two sides Confrontation is a reciprocating process. At the same time, the dynamic process of the zigzag fluctuation of the armor quantity shown in the schematic diagram of the confrontation between the red and blue sides of the OODA Loop system is verified from the side.

It can be concluded from that the weight ratios of the three combat plans are respectively, the support weight ratio is , and the continued combat weight and the retreat weight ratio are roughly the same. It shows that when using the OODA Loop to analyze the combat plan, the warring parties should support their own operations as much as possible, but the continued combat and retreat plan also accounted for nearly half of the proportion, and the choice to continue combat or retreat based on the confrontation situation.

5 Conclusions

This paper simulates a complex environment combat model based on OODA loop through NetLogo software and uses hierarchical analysis to embody the game strategy under three scenarios of attack, retreat, and continued reinforcement that the red side should take after the current OODA loop is executed in terms of the number of armor. It can be concluded from the experimental results that the simulation system of OODA loop is more exploratory and feasible compared to the traditional simple system approach simulation. The OODA cycle is a correct practice of modern military theory, and the full use of the OODA cycle can form a closed-loop control of the combat confrontation and make the combat analysis comprehensive and concrete.

Acknowledgments

This research is partially supported by Nature Science Foundation of China with ID 62076028 and by National Key Research and Development Program of China with ID 2018AAA0101000.

References

  • [1] Liu Xiaohu,Zhang Hengwei,Zhang Yuchen,Ma Zhuang,Lu Wenlei. Research on network attack and defense posture based on game model and NetLogo simulation.Journal of System Simulation.2020,32(10):1918-1926.DOI:10.16182/j.issn1004731x.joss.20-fz0290.
  • [2] Nelson J.Build-Measure-Learn as an OODA Loop. Becoming a Lean Library.2016.
  • [3] Kelly M.THE OODA LOOP (OBSERVE, ORIENT, DECIDE, ACT) Applying Military Strategy to High Risk Decision Making and Operational Learning Processes for on Snow Decision Makers. 2014;.
  • [4] Georgakopoulos D.. Information System Engineering Supporting Observation, Orientation, Decision, and Compliant Action. DBLP. 2008;.
  • [5] Leung H.An Integrated Decision Support System Based on the Human OODA Loop.In:1-1; 2018.
  • [6] Brandt C., Gawlick D.. A Modern Approach to Situation Awareness: The Ultimate Challenge for Event Processing. In: ; 2019.
  • [7] Revay Martin, Miroslav. OODA loop in command amp; control systems.In:1-4;2017.
  • [8] Rousseau R., Breton R.. The M-OODA: A model incorporating control functions and teamwork in the OODA loop. Paper presented at the Command and control research and technology symposium. 2004;.
  • [9] Villiers Jpd, Jousselme A. L., Waal A. D., Pavlin G., Costa P.. Uncertainty evaluation of data and information fusion within the context of the decision loop. In: ; 2016.
  • [10] Xiaohu Liu, Hengwei Zhang, Yuchen Zhang, Zhuang Ma, Wenlei Lu. Research on Network Attack and Defense Situation Based on Game Model and NetLogo Simulation. Journal of System Simulation. 2020;32(10):9.
  • [11] Jun Guo Jianguo; Zhou Min; Guo Zongyi; Zhou. Cooperative Combat Technology under Mosaic War. Aviation Weapon. 2021;.
  • [12] Yu Zhang, Jianxin Huang. Using the OODA ring model to study the contribution of equipment to the system. Modern defense technology. 2017;45(002):177-182.
  • [13] Changchun Wang, Junliang Chen, Chao Chen. Modeling and Simulation of Destroy Based on Complex Network Combat System. Journal of System Simulation. 2012;(07):1491-1495.
  • [14] Yang Zhang, Yanzheng Wang, Guangya Si. Analysis and modeling of OODA combat loop of cluster electronic warfare UAV. Firepower and Command Control. 2018;43(08):33-38.
  • [15] Rong Shi, Ke Deng, Jiang Liu, Su Hu. Active active electronic countermeasure reconnaissance for the OODA loop used in cognitive radio. Journal of China Academy of Electronics. 2019;14(5).
  • [16] Jiuyang Tao, Lin Wu, Xiaofeng Hu. Analysis of AlphaGo technology principle and prospects of artificial intelligence military applications. Journal of Command and Control. 2016;2(2):114-120.
  • [17] Xiaofeng Hu, Xiaoyuan He, Dehu Rao, Shengming Guo. Research on Analysis Method of System Combat Command and Coordination Mechanism Based on Complex Network. Journal of Command and Control. 2015;001(001):P.5-13.
  • [18] Liyan Zou, Mingzhi Zhang, Junru Bai. Intelligent unmanned swarm combat simulation modeling framework in OODA-L mode. Journal of National University of Defense Technology. 2021;43(4):163.
  • [19] Day S.. Understanding responses to evolving emergencies: some applications of Boyd’s Observation-Orientation-Decision-Action loop model to the behavior of decision-makers in natural disasters under conditions of uncertainty and incomplete understanding. 2016;.
  • [20] Yuchen Yang. Decision-making evaluation of rear command site selection based on analytic hierarchy process and entropy weight. Ordnance Industry Automation. 2020;(10).