Dynamic safety measurement-control technology for intelligent connected vehicles based on digital twin system
Xingbin Chen1 , Xinhe Min2 , Nini Li3 , Wei Cao4 , Shunren Xiao5 , Guanting Du6 , Peng Zhang7
1, 2, 3, 4, 6, 7Guangzhou Mechanical Engineering Research Institute Co., Ltd., Guangzhou, 510700, China
1, 5, 7School of Mechanical and Automotive engineering, South China University of Technology, Guangzhou, 510641, China
1, 2, 3, 4, 6, 7CRAT Testing and Certification Company Ltd., Guangzhou, 510700, China
1, 7Corresponding authors
Vibroengineering PROCEDIA, Vol. 37, 2021, p. 78-85.
Received 12 April 2021; received in revised form 18 April 2021; accepted 26 April 2021; published 7 May 2021
As the number of connected vehicles increases, the intelligence levels become more and more uneven, so the problem how to determine the dynamic safety of autonomous driving behavior in the mixed-flow traffic system is significantly increased. To solve many difficult problems related to human-vehicle-road perception, decision, control coordination, and reliability evaluation in the intelligent networked scenario, this study establishes a dynamic Game model with multi-source information in the intelligent networked environment, to carry out the measurements and control evaluations for autonomous vehicle dynamics models, forward and backward active safety control, and mixed traffic trajectory optimization planning based on the optimal solution strategy. A digital twin test verification platform with semi-physical environment and hardware-in-the-loop (HIL) simulation to control the accuracy for making dynamic safety decision for the intelligent networked vehicle is developed by combining with the V2X real intelligent transportation system and smart laboratory virtual simulation test technology, which realizes the complex and dynamic safety decision goals for autonomous vehicle in different connected levels of the mixed-flow traffic environments, such as multi-agent perception, multi-source information transmission, vehicle control, vehicle-to-vehicle communication, and vehicle-to-road coordination. The study is conducive to improving the test efficiency and index evaluation integrity of the intelligent networked system, reducing the test costs, proving the behaviors of Game interaction and stress safety response under vehicle-to-road environment, enhancing the robustness and applicability of automatic driving technology, and improving the traffic efficiency and safety of mixed-flow traffic in the intelligent network.
- Game Interactive Behavior Modeling for Dynamic Safety Decision based on surrounding heterogeneous vehicles, traffic environment, objects and even people at different grade intelligence.
- Hierarchical collaboration dynamic in vehicle-road perception, decision, and control are conducted for homogeneous or heterogeneous agents aiming at networked environment.
- Digital twin measurement control evaluation system is studied based on virtual-real combination of intelligent network to set up a multi-dimensional and multi-level dynamic safety evaluation index system for determining the weights of evaluation aspects, elements, factors and indexes at all levels by analytic hierarchy process (AHP).
- Digital twin test and verification platform development for dynamic safety decision and control accuracy with optimized semi-physical environment.
Keywords: intelligent networks, digital twin, dynamic safety decision, measurement and control technology evaluation, game interaction.
The multi-source mixed-flow road environment is composed of heterogeneous vehicles at the level of intelligence and connectivity, which increases the difficulty of vehicle-road safety control decision for intelligent networked autonomous vehicles, and is also one of the main reasons for poor road traffic flow smoothness and low networked environment prediction reliability. At present, there are many theoretical results about the intelligent connected vehicle industry achieved both locally and worldwide, including vehicle ontology structure, road environment and automated and intelligent networked architecture, Refs. [1-3]. However, the engineering application study mainly focuses on the key core technologies, decision control platforms, network communications and infrastructure configurations for intelligent connected vehicles within CA or below grades, Refs. [4, 5]. Regardless of a theoretical study or engineering application, complete and reasonable intelligent networked tests and specially designed evaluation systems are still missing, Refs. [6-8], particularly, it concerns the lack of evaluation methods and specifications for making a dynamic safety decision for the individual test evaluation technology, whole vehicle product, and networked environment system of the HA or FA driving grade, which promotes the development of simulation, road tests and other technologies, then to form comprehensive test verification capabilities for laboratories, closed roads, semi-closed roads and open roads, Refs. [9, 10]. The paper contains a study on the multi-agent Game interactive decision in complex mixed-flow traffic systems with the target as dynamic safety decision of intelligent and connected autonomous vehicles driving trajectory, to solve the difficult problems such as multi-source information identification, Game decision model creation and optimization, dynamic safety target definition and sensitivity parameters characterization relationship identification, etc., and to develop the measurement control technology evaluation and verification platform, which can be applied to the parts and whole vehicle analysis of intelligent networked environment.
2. Game interactive behavior modeling for dynamic safety decision
Due to the complexity of mixed-flow traffic scenes, the connected vehicles when driving autonomously will be affected by surrounding heterogeneous vehicles, traffic environment, objects and even people at different grade intelligence. When carrying out driving behaviors such as path selection or positioning control, the vehicle will inevitably generate Game interaction with different subjects in the mixed-flow system.
In the V2X context, the information communication and sharing takes place between the connected autonomous vehicles and surrounding vehicles, and also cooperative and competitive relationships occur between any two adjacent vehicles. The Game theory can be used to characterize the uncertain strategy of the mixed-flow traffic system, optimize the path selection or positioning control, and then predict the driving behavior to realize the dynamic real-time information interaction between vehicles, or vehicles-roads in all directions.
The intelligent networked cloud can generate Game interaction relationship with CAVs in the mixed-flow traffic, and it is a dynamic Game with multiple reciprocations. It is easy to form a cooperative relationship with CAVs with automatic driving function, thus forming cooperative Game, while it is easy to form non-cooperative Game with CVs due to the selfish behavior of some drivers. The Game phenomenon in the mixed-flow traffic is more complicated, without external intervention, the Game interactions between vehicles are almost non-cooperative. However, from the perspective of the intelligent networked cloud, it is necessary to encourage cooperation between vehicles with different connectivity degrees in the mixed-flow traffic, and realize the “intelligence” for non-networked vehicles through imitating-following connected vehicles, thus, the static Games and non-cooperative Games in the mixed-flow traffic are transformed into dynamic Games and cooperative Games. Furthermore, the intelligent networked environment system is realized as the optimized option, as shown in Fig. 2.
The driving process of regional vehicles can be regarded as a dynamic Game process. The CAV makes safety decisions to pierce safely into the traffic by keeping a proper distance between the front and rear vehicles, which can be regarded as a dynamic repeated Game process with limited times and multiple stages. In the mixed-flow traffic with intelligent network environment, the Game process of the remaining traffic agents can be regarded as N-source information under the specific road grade, lane alignment, driving route and meteorological conditions.
(1) Vehicle agent collection:
where, is the total number of vehicles, is the proportion of different type vehicles.
Fig. 1. Information interaction behavior of traffic system under mixed-flow traffic system
Fig. 2. Dynamic Game interaction mechanism of multi-agent intelligent network in mixed-flow transportation system
(2) Game collection of a single vehicle :
(3) Corresponding Game decision probability:
In Eq. (3), , , .
(4) Mixed Game strategy. represents a vehicle choosing a certain strategy from the Game collection, and the probability in a pure Game strategy is . In the mixed-flow system, the vehicle replaces with another dynamic mixed Game strategy, while keeping the other strategies unchanged:
(5) Single type Game effect. When a certain Game strategy is selected, a collection trend of the Game is formed, where, , for each trend , the Game effect obtained by each different type vehicle is:
(6) Expectation effect of mixed-flow Games. The expected effect of each different type vehicle in the mixed-flow strategy trend is:
(7) Multi-agent Game of mixed-flow traffic in the intelligent network environment:
3. Influencing factors of vehicle dynamic safety decision under Game behavior
Different from most studies for location recognition and trajectory planning of autonomous vehicles with single “real-time motion interaction”, this paper covers hierarchical collaboration dynamic study for vehicle-road perception, decision, and control of homogeneous or heterogeneous agents.
(1) Collision potential. The Game interaction process of multi-agent in the mixed traffic can use time-to-collision (TTC) between vehicles to evaluate the driving safety. Assuming that the relative distance between adjacent vehicles at any time is :
In Eq. (8), , are respectively the speeds of two adjacent vehicles in front and behind (m/s). , are the adjacent acceleration.
Then the collision time is:
In Eq. (9), the relative speed difference is , and is required. The relative acceleration difference is , where, the maximum braking deceleration of the decision vehicle agent to avoid rear-end collision is , and the maximum deceleration of adjacent vehicle at a certain moment is . The relative safety distance is .
(2) Relative driving space and relative speed. The driving space in front of the vehicle (including the front space distance and length of vehicle) and relative speed are important influencing factors for the driving intention of the vehicle agent’s lane selection. In order to better qualitatively analyze the relationship between and , a membership of fuzzy set is introduced as follows:
(3) Planning driving time decision under safe distance. According to the factors such as the boundary value of the collision risk safety degree, adjustment margin of safety guarantee, TTC experience threshold, the driving time of CAV within the limited safe distance is obtained as:
In Eq. (11), , , are respectively the relative displacement, relative speed and relative acceleration of vehicle agent from a certain position to the target safe position selected by the Game.
4. Study on digital twin measurement control evaluation system based on virtual-real combination of intelligent network
The digital twin measurement control evaluation method based on the virtual real combination is helpful to solve these problems, such as real-time calculation performance of HIL test and accurate performance of simplified system, which can save a lot of time, and have high repeatability under the same test conditions.
Based on the multi-dimensional and multi-level dynamic safety evaluation index system of human-vehicle-traffic system, the weights of evaluation aspects, elements, factors and indexes at all levels are determined by the analytic hierarchy process (AHP). The multi-dimensional and multi-level dynamic safety evaluations are studied through the methods of fuzzy comprehensive evaluation and neural network evaluation, as well as the overall evaluation for the passing safety of mixed-flow traffic system, as shown in Fig. 3.
(1) Kinematic and geometric characteristics of networked vehicles. They include collision time, maximum longitudinal acceleration/deceleration, maximum lateral acceleration, braking time, lane post-invasion time, instantaneous collision probability and safety limits, etc.
(2) Mixed-flow traffic environment characteristics. They mainly include road conditions, traffic facilities, landforms, meteorological conditions, and traffic activities of other participants.
(3) Dynamic safety features. The evaluations are mainly from two aspects: driving safety and information safety.
1) Driving safety. Driving safety program mainly evaluates the interactive behavior safety of CAV with surrounding traffic participants (other vehicles, pedestrians, bicycles, etc.) during autonomous driving operations, as well as the safety achieves avoiding and minimizing collision risks when the CAV encounters a collision hazard or anticipates a collision.
2) Information safety. Information safety program conducts corresponding evaluations on protective countermeasures due to information transmission hazards (such as security vulnerabilities), communication network delays or jams, safety threats, and malicious intrusions in the information interaction process between CAVs and outside world.
Generally, the trajectory measurement control evaluations of CAVs are based on complex numerical simulation analysis by taking the whole system as the object, which leads to a lack of stability and engineering operability for dynamic trajectory measurement and control. With focus on establishing a dynamic safety decision model, a test evaluation method is proposed for forward and backward active safety control or mixed-flow traffic trajectory optimization planning based on the optimal solution strategy, the internal correlation between the dynamic safety and traffic stability is revealed through the network hierarchy and multi-source information agent classification, so that the influence law of each sensitivity parameter is explored.
5. Development of digital twin measurement control evaluation platform
Focusing on the problems of dynamic safety decision and trajectory positioning for the Game measurement control of intelligent networked vehicles, it is proposed to conduct a deeper study on the multi-source information identification and Game safety decision with heterogeneous mixed-flow traffic characteristics. Then a digital twin measurement control evaluation platform based on the virtual and real combination of intelligent network is constructed to evaluate the effect of dynamic Game safety decision.
Fig. 3. Multi-dimensional and multi-level dynamic safety measurement control evaluation process
Fig. 4. Digital twin system architecture
(1) Signal input module. The digital twin test system receives the scene information and signal type required by the test device from the HIL and virtual simulation module, which generates the required physical-level signal or true value signal, and input to the digital twin system interface, such as image, RCS echo, LTE-V signals, homogenous or heterogeneous networked vehicle participants in mixed-flow traffic, multi-source information lists.
(2) Scene simulation module.
1) The simulation scene is constructed, and simulated with vehicle position, speed, driving path and positioning selection can be planned globally. 2) Controller sends simulated vehicle information.3) Positioning device is used to determine the current position of vehicle. 4) The signal input device is dependent on the scene, which parameters are dynamically adjusted. 5) Data transmission network. 6) HIL signals support multi-terminal access, and the data can be uploaded through LTE-V/4G/5G networks. 7) A connection is established between the test platform and local test system.
(3) Digital twin system test performance. It can conduct multi-dimensional and multi-level evaluation tests of dynamic safety for the system Game interaction, route planning, positioning selection, lane change decision and other aspects, as well as the overall test of traffic safety in mixed-flow traffic system.
(4) Dynamic safety decision testing and evaluation module. Vehicle information and warning messages uploaded by the local test system, receive simulated vehicle information, and synchronize with system data source, so as to carry out lane Game interactive simulation and trajectory positioning control. It is needed to evaluate the test results of safety distance, relative speed, collision time, etc.
Based on the V2X real intelligent transportation system data, the actual test results are extracted through virtual hardware and virtual sensors. In addition, the intelligent networked virtual simulation technology is integrated, which can set up intelligent networked scenes for the multi-source subjects and information in the complex mixed-flow traffic system. So, the driving status of various levels CVs or heterogeneous vehicles, the trajectory positioning decision are adjusted and optimized through the forward and backward response to obtain a more stable and reliable control strategy.
Focused on the safety driving decision accuracy problems in mixed-flow traffic system, this paper studies a multi-source dynamic Game interaction model, a multi-agent dynamic Game simulation test platform is established under the intelligent networked scene, to test and evaluate the dynamic safety decision control accuracy for autonomous vehicles, thus, a scientific and reasonable evaluation system for intelligent networked dynamic safety standards is established. The results can significantly improve the safety and reliability of connected vehicles, and reduce the delay rate of feedback control response and the error rate of trajectory perception planning.
1) Game interaction relation Construction for heterogeneous mixed-flow traffic
With the goals of dynamic safety decision for autonomous vehicles, the driving path planning can be realized in advance through “Game mutual identification” for homogeneous networked vehicles, the driving trajectory optimization can be realized through “real-time motion interaction” and “Game feedback” between trajectory representations for heterogeneous non-networked vehicles, however, the driving intention can be realized through the “multi-source information collection” and “Game decision” between the safety parameters adjustment decisions for heterogeneous non-networked multi-environment-agent.
2) Digital twin test and verification platform development for dynamic safety decision and control accuracy with optimized semi-physical environment
A test verification method is proposed taking into account the on-site measurement control and virtual simulation evaluation, and a related software and hardware platform is developed, with integrated modules such as network and non-network compatible driving environment, complex mixed-flow traffic system, multi-source information identification and sensing system, autonomous vehicle dynamics, test controller, driving trajectory planning and deviation compensation, components and vehicle dynamic safety decision evaluation modules. The dynamic safety decision and trajectory location control strategy of the intelligent networked vehicle are studied from the perspective of safety and reliability, so as to analyze the trajectory location characteristics of the autonomous vehicle under the complex mixed-flow traffic system with multi-source information and Game interaction between networked and non-networked agents. The dynamic safety decision accuracy is verified through the automatic driving digital twin test technology, the generation mechanism of trajectory positioning deviation and influence law of deviation source sensitivity parameters are summarized, in order to form a multilevel hierarchy deviation compensation mechanism according to the classification compensation method of each deviation source.
3) Study on theories and methods of dynamic safety decision and trajectory positioning reliability evaluation for intelligent networked vehicles
Combined with key technical characterization parameters such as driving reliability, driving control reliability and position recognition reliability, the corresponding evaluation theory was proposed, the intelligent networked digital twin evaluation test platform was improved and perfected, and the trajectory safety reliability testing standard was established. Thus, the evaluation method proved the traceability and verifiability.
This work was supported by the National Key Research and Development Plan of China (Grant No. 2017YFB1301400), the Guangdong Provincial Enterprises Key Laboratory Foundation about Mid-High-End Industrial Robot Technology (Grant No. 2018B030323027), and the Postdoctoral project of Guangzhou Academy of Mechanical Sciences Co., Ltd. (Grant No. 1014300036).
- Vasudev H., Deshpande V., Das D., et al. A lightweight mutual authentication protocol for V2V communication in internet of vehicles. IEEE Transactions on Vehicular Technology, Vol. 69, Issue 6, 2020, p. 6709-17. [Publisher]
- Zargayouna M., Othman A., Scemama G., et al. Multiagent simulation of real-time passenger information on transit networks. Intelligent Transportation Systems Magazine, Vol. 12, Issue 2, 2020, p. 50-63. [Publisher]
- Zeadally S., Guerrero J., Contreras J. A tutorial survey on vehicle-to-vehicle communications. Telecommunication Systems, Vol. 73, Issue 3, 2020, p. 469-89. [Publisher]
- Yan R. D., Yang D. G., Wijaya B., et al. Feedforward compensation-based finite-time traffic flow controller for intelligent connected vehicle subject to sudden velocity changes of leading vehicle. IEEE Transactions on Intelligent Transportation Systems, Vol. 21, Issue 8, 2020, p. 3357-65. [Publisher]
- Younes M. B., Boukerche A. Traffic efficiency applications over downtown roads: a new challenge for intelligent connected vehicles. ACM Computing Surveys, Vol. 53, Issue 5, 2020, p. 15-28. [Publisher]
- Ben Hamida E., Noura H., Znaidi W. Security of cooperative intelligent transport systems: standards, threats analysis and cryptographic countermeasures. Electronics (Switzerland), Vol. 4, Issue 3, 2015, p. 380-423. [Search CrossRef]
- Songchitruksa P., Sunkari S., Ugalde I., et al. Interlinking vissim and ns-3 for connected vehicle simulation case study of intelligent dilemma zone avoidance. Transportation Research Record, Vol. 2619, 2017, p. 36-43. [Publisher]
- Zhang Y., Cassandras C. G., Li W., et al. A discrete-event and hybrid traffic simulation model based on SimEvents for intelligent transportation system analysis in Mcity. Discrete Event Dynamic Systems, Vol. 29, Issue 3, 2019, p. 265-95. [Publisher]
- Huang C., Wang X. C., Cao J. N., et al. HCF: a hybrid CNN framework for behavior detection of distracted drivers. IEEE Access, Vol. 8, 2020, p. 109335-109349. [Search CrossRef]
- Zhang H. C., Meng X., Zhang X., et al. CANsec: a practical in-vehicle controller area network security evaluation tool. Sensors-Basel, Vol. 20, Issue 17, 2020, p. 135-146. [Publisher]