In social network analysis, detecting a large cohesive subgraph is a fundamental and extensively studied topic with various applications. Clique is a classical and ideal model in the field of cohesive subgraph detection. A graph is a clique if there is an edge between any pair of vertices. The Maximum Clique Problem, that is, to find a clique of maximum size in a given graph, is a fundamental problem in graph theory and finds wide application in many fields, such as biochemistry and genomics [Butenko and Wilhelm2006], wireless network [Lakhlef2015], data mining [Boginski et al.2006, Conte et al.2018] and many others.
However, in some real-world applications, the networks of interest may be built based on empirical data with noises and faults. In these cases, large cohensive subgraphs hardly appear as ideal clique. To tackle this problem, many clique relaxation models have been proposed. In this paper, we focus on -plex, a degree-based clique relaxation model. A simple undirect graph with vertices is a -plex if each vertex of this graph has at least neighbors. The maximum -plex problem, that is, to find a -plex of maximum size on a given graph with a given integer , has received increasing attention from researchers in the fields of social network analysis and data mining [Xiao et al.2017, Conte et al.2018].
The decision version of the maximum -plex problem is known to be NP-complete [Balasundaram et al.2011]. Different algorithms have been developed for this problem, including exact algorithms and heuristic ones. balasundaram2011clique balasundaram2011clique proposed a branch-and-bound algorithm based on a polyhedral study of this problem. mcclosky2012combinatorial mcclosky2012combinatorial developed two branch-and-bound algorithms adapted from combinatorial clique algorithms. Recently, xiao2017fast xiao2017fast proposed an exact algorithm which breaks the trivial exponential bound of for maximum -plex problem with . gao2018exact gao2018exact proposed several graph reduction methods integrated them into a brand-and-bound algorithm.
Due to the exponential time complexity of the maximum -plex problem, several heuristic approaches have been proposed to provide a satisfactory solution within an acceptable time. DBLP:series/natosec/GujjulaSM14 DBLP:series/natosec/GujjulaSM14 proposed a hybrid metaheuristic based on the GRASP method. miao2017approaches miao2017approaches improved the construction procedure to provide a better initial solution for GRASP method. zhou2017frequency zhou2017frequency developed a tabu search algorithm named FD-TS which achieved state-of-the-art performance.
Local search is likened to “trying to find the top of Mount Everest in a thick fog while suffering from amnesia” [Russell and Norvig2016]. For a long time, much effort has been devoted to enable a memory mechanism for local search. These works can be roughly divided into two parts. The first part focuses on exploiting the searching history to guide the search into a more promising area. For example, boyan2000learning boyan2000learning proposed the STAGE algorithm to learn an evaluation function from features of visited states which can be used to bias future search trajectory. [Zhou et al.2018]
presented a probability learning based local search algorithm for the graph coloring problem. The other part focuses on reducing the inherent cycling problem of local search. Tabu mechanism[Glover and Laguna1998] maintains a short-term memory of the recent search steps to forbid reversing the recent changes. Configuration Checking strategy [Cai et al.2011] keeps a memory of state change of local structures and reduces cycling problem by prohibiting cycling locally.
When getting stuck in local optima, a good perturbation mechanism can modify the candidate solution and generates a promising search area for the following search steps. Inspired by the multi-armed bandit problem and its algorithms, we propose the bandit learning based perturbation mechanism (BLP), which can learn in an online way to select a good vertex for perturbation. To our best of knowledge, this is the first attempt to combine reinforcement learning and local search for the maximum -plex problem .
Recently, Configuration Checking (CC) and its variants have been successfully applied in various combinatorial optimization problems [Cai et al.2011, Wang et al.2016, Wang et al.2018], revealing the importance of exploiting the structural property of the problems. Different from tabu mechanism, CC is a non-parameter strategy which exploits the circumstance information to reduce cycling problem in local search. However, CC and its variants have the following limitations. Firstly, the use of the configuration information is limited to handling the cycling problem. Secondly, the forbidding strength of the CC and its variants is static and cannot make adjustments to different problem instances. In this paper, we propose a variant of CC, named Dynamic-threshold Configuration Checking (DTCC), to extend the original CC from two different aspects. One is the neighbor quality heuristic which evaluates a vertex with consideration of the community it belongs to. The other is the dynamic threshold mechanism which enables an adaptive forbidding strength for CC.
Based on BLP and DTCC, we develop a local search algorithm, called BDCC, and improve it by a hyperheuristic strategy. The resulting algorithm, named BDCC-H, can learn to select a good heuristic in adding and swapping phase. The experiments show that our algorithms dominate FD-TS on the standard DIMACS and BHOSLIB benchmarks. Not only is our algorithms robust and time-efficient, but also they provide better lower bounds on the size of the maximum -plexes for most hard instances. Besides, our algorithms achieve state-of-the-art performance on massive graphs.
The remainder of this paper is organized as follows. Section 2 gives some necessary background knowledge. Section 3 provides some formal definitions and proposes the BLP mechanism. Section 4 proposes DTCC strategy to extend CC from two aspects. In Section 5, we present the BDCC algorithm and improve it by a hyperheuristic strategy. Section 6 shows the experimental results. Section 7 gives concluding remarks.
2.1 Basic Definitions and Notations
An undirected graph is defined as , where is a set of vertices and is a set of edges. Each edge consists of two vertices, denoted as , where and are the of this edge. Two vertices are neighbors if they belong to an edge. Let denote the set of all neighbors of . The degree of vertex is defined as the . For a vertex set , be the set of neighbors of and be the induced graph of .
Given a graph and an integer , a subset is a -plex, if for all . A vertex is a saturated vertex if . The saturated set of set is the set of all saturated vertices in . A vertex is deficient vertex if . Obviously, any subset containing deficient vertices cannot be a -plex.
A candidate solution is a subset of . Given a graph , an integer and a feasible -plex , a typical 3-phase local search algorithm for the maximum -plex problem maintains a feasible -plex as candidate solution and uses three operators, , and to modify it iteratively [Zhou and Hao2017]. The set is split into three disjoint sets, , and , which contain the objects of the above-metioned operators. Here we give their formal definitions.
Obviously, the vertices in can be added into directly. The vertices in can be added into while removing one vertex in . Adding a vertex into would cause two or more deficient vertices in . Therefore, these vertices should be removed to maintain a feasible -plex.
2.2 Multi-armed Bandit Problem
Multi-armed Bandit Problem
(MAB) is a one-state RL problem. In this problem, there is a set of arms and an agent which repeatedly selects an arm to play at each step with a purpose to maximizing the long-term expected reward. Since the distribution of the reward of each arm is unknown, the agent faces an exploration-exploitation tradeoff. On the one hand, it needs to explore by selecting each arm to estimate the expected reward of them. On the other hand, it needs to exploit the existing knowledge by choosing arms with high expected rewards. The exploration-exploitation trade-off is a fundamental issue in reinforcement learning and the-greedy strategy is widely used to keep a balance of them. With the -greedy strategy, the agent chooses actions randomly for exploration with a probability and makes choices greedily for exploitation with a probability .
2.3 Configuration Checking
Configuration Checking (CC) [Cai et al.2011], is a parameter-free strategy that can exploit the structural property of the problem to reduce cycling problem in local search. The configuration of a vertex is defined as the states of its neighbors. The main idea of CC strategy is that if the configuration of a vertex remains unchanged since its last removal from candidate solution, then it is forbidden to be added back into the candidate solution.
Recently, different CC variants have been proposed and successfully applied to various combinatorial optimization problems [Cai et al.2011, Wang et al.2017a, Wang et al.2017b]. Here we highlight the Strong CC (SCC) strategy which was proposed in [Wang et al.2016] for the Maximum Weight Clique Problem. The difference between SCC and CC is that SCC allows a vertex to be added into candidate solution only when some of ’s neighbors have been added since ’s last removal, while CC allows the adding of a vertex when some of ’s neighbors have been either added or removed.
Due to the similarity of clique and -plex, it is natural to think of applying SCC to local search for the maximum -plex problem. A straighforward SCC strategy for maximum -plex problem can be implemented as follows. We maintain a Boolean array to indicate whether the configuration of each vertex has been changed. Only when a vertex satisfies the SCC condition can it be added in . Initially we set for all . When a vertex is added into candidate solution , for all , is set to . When a vertex is removed from , is set to . As for a swap step, where vertex is added into at the cost of removal of vertex , is set to .
3 Learning from History: BLP Mechanism
According to the definition in Subsection 2.1, as grows, and become smaller because the vertices in them need to satisfy more constraints. Therefore, usually contains most of the vertices in when reaching a local optima. It is difficult to select a good vertex for perturbation from such a large set. We propose bandit learning based perturbation mechanism (BLP) to learn from searching history to select a good vertex for perturbation in an online way. In this section, we give some necessary formal definitions and present the BLP mechanism.
Given a graph and a -plex , an action is a pair where is the operator and is the object. The available action set is defined as . Let denote that applying action to results in a new -plex .
A search trajectory is a finite sequence of -plexes such that for , .
The walk of a search trajectory is an ordered action sequence where and for .
Given a search trajectory , a -plex in is a break-through point if for , and an episode is a subsequence of between two adjacent break-through points.
The underlying consideration of BLP is that all the Perturb actions in the walk of an episode make contributions to the quality improvement at the end of this episode. The BLP treat each vertex as an arm in MAB and reward them according to their contribution when an episode is completed. Therefore, the expected reward of a vertex can reflect the possibility of reaching another break-through point if perturbing the candidate solution with this vertex. In the implementation, the BLP maintains a value for each vertex, initialized to 0 at the start of the search. When an episode is completed, we reward the objects of all the Perturb actions in the corresponding walk. For a vertex to reward, we update with exponential recency weighted average (ERWA) technique [Sutton and Barto1998], as is shown in Equation 1.
Here the is a factor called stepsize to determines the weight given to the recent reward, and where is the number of Perturb actions in the walk of this episode. The intuition behind the reciprocal reward value is that the actions applied in a shorter episode are more valuable than those in a longer one. In the perturbation phase, BLP selects a vertex with -greedy strategy.
4 Dynamic-threshold Configuration Checking
According to our previous experiments, applying CC or other CC variants directly to the maximum -plex problem does not lead to a good performance on graphs with high edge density. The reason is that the configurations of the high-degree vertices in these graphs are very likely to change and CC (or other CC variants) cannot enhance its forbidding strength on these vertices. To make better use of the configuration information and enable an adaptive forbidding strength, we propose a new variant of CC named Dynamic-threshold Configuration Checking (DTCC). The two parts of DTCC are the neighbor quality heuristic and dynamic threshold mechanism.
4.1 Neighbor Quality Heuristic
The neighbor quality of a vertex , denoted by , is defined as , where (resp. ) is the total number of times a vertex in is added into (resp. removed from) the candidate solution. Due to the cohesive characteristics of -plex, a vertex that belongs to a higher-quality community is more likely to appear in a large -plex. In the implementation, DTCC maintains a integer (initialized to ) for each vertex, and update the value with the following rule.
DTCC-NQRule. The value is set to for all . When a vertex is added into candidate solution , for all , . When a vertex is removed from , for all , .
4.2 Dynamic Threshold Mechanism
We extend the to an integer array and maintain an integer array that can adjust the forbidding strength on different vertices. A vertex is allowed to be added into candidate solution only when DTCC condition is satisfied. The following four rules specify the dynamic threshold mechanism.
DTCC-InitialRule. In the beginning of search process, for all , .
DTCC-AddRule. When is added into candidate solution, , , and for all , .
DTCC-SwapRule. When is added into candidate solution at the cost of removal of , .
DTCC-PerturbRule. When is added into candidate solution and a set of vertices is removed from this candidate solution, for all , , and for all .
Note that the SCC strategy is a special case of DTCC strategy whose is a Boolean array and is fixed to . Lemma 1 illustrate their relation.
If a vertex satisfies the DTCC condition, then it satisfies the SCC condition. The reverse is not necessarily true.
According to DTCC rules, for during the search processs. If the the DTCC condition holds, then . So at least one neighbor of must be added into candidate solution since the last time was removed. So the SCC condition is satisfied.
Suppose satisfies the SCC condition , but . In this case the DTCC condition is not satisfied. ∎
According to Lemma 1, we can conclude that DTCC has stronger forbidding strength than SCC. Generally, a frequently operated vertex has a high and is more likely to be forbidden. Thus the algorithm is forced to select other vertices to explore the search space.
5 BDCC Algorithm and A Hyperheuristic
5.1 BDCC Algorithm
Based on the BLP mechanism and DTCC strategy, we develop a local search algorithm named BDCC, whose pseudocode is shown in Algorithm 1. Initially, the best found -plex, denoted as , is initialized as empty set. In each loop (line 3-10), an initial solution is firstly constructed (line 4) as the starting point of the search trajectory, and the search procedure starts. If the best solution in this search trajectory is better than the best solution ever found , is updated by and function (line 8) is called to reduce the graph. If the reduced graph has fewer vertices than , then is returned as one of the optimum solutions. Three major components in BDCC are initial solution construction, search procedure and graph peeling. We describe them in detail in the following.
We adopt the construction function in FD-TS [Zhou and Hao2017]. The function firstly use the vertex with minimum (breaking ties randomly) in a random sample of vertices to create the singleton set , and repeatedly add the vertex with minimum (breaking ties randomly) in into until is empty. Then the final -plex is returned as the initial solution. By giving priority to vertices that are operated less frequently, the construction procedure can generate diversified initial solutions in different rounds.
The function iteratively selects one action to modify the candidate solution until the iterations limit is reached, or the available action set is empty, as is shown in Algorithm 2. The records the best-quality solution in the search trajectory so far, and the record the action sequence since the last break-through point. The algorithm selects vertices with the highest for adding and swapping and selects vertices for perturbation according to their with -greedy strategy. After each iteration, if , that means a break-through point is reached and the episode is completed, then the objects of all Perturb actions in current (if there exist) will be rewarded with the ERWA algorithm and will be cleared out.
If the returned by is better than , then is updated with and the function is called to recursively deletes the vertices (and their incident edges) with a degree less than until no such vertex exists. It is sound to remove these vertices since they can not be included in any feasible -plexes larger than .
5.2 Improving BDCC by A Hyperheuristic
Our previous experiments show that selecting vertices from and greedily can usually lead to a high-quality solution on most hard instances. However, on some problem domains where the optimum solutions are hidden by incorporating low-degree vertices, the search may be misled by the greedy manner and miss the best solutions in some runs [Cai et al.2011]. To enhance the robustness of BDCC, we design a hyperheuristic based on simulated annealing to switch between different heuristics dynamically and select the suitable one for different problem instances. We equip BDCC with this hyperheuristic, developing an algorithm named BDCC-H, as outlined in Algorithm 3.
The difference between BDCC and BDCC-H is whether the heuristics for adding and swapping is fixed. The BDCC-H algorithm adopts three heuristics for adding and swapping, (i), selecting vertex with largest , (ii), selecting vertex with largest , (iii), selecting vertex randomly. The BDCC-H maintains a variable for each heuristic to record the size of the best solution found with . A temperature is used to control the heuristic selection. Before the search procedure begins, the algorithm selects one heuristic under current . The selection probability of each heuristic is defined based on Boltzmann distribution , which is widely used for softmax selection [Sutton and Barto1998]. A relatively high initial temperature can lead to equal selection to force exploration. As the temperature cools down, the algorithm are more inclined to select a heuristic with highest value and exploit with this heuristic.
6 Experimental Result
We evaluate our algorithms on standard DIMACS and BHOSLIB benchmarks as well as massive real-world graphs.
6.1 Experiment Preliminaries
BDCC and BDCC-H and their competitor FD-TS are all implemented in C++ and compiled by g++ with ’-O3’ option. All experiments are run on an Intel Xeon CPU E7-4830 v3 @ 2.10GHz with 128 GB RAM server under Ubuntu 16.04.5 LTS. We set the search depth , the stepsize and for BDCC and BDCC-H. The initial and cooling rate are set to and respectively for BDCC-H. The cutoff time of each instance is set to seconds. All algorithms are executed 50 independently times with different random seeds on each instance with .
6.2 Evaluation on DIMACS and BHOSLIB
We carried out experiments on standard DIMACS and BHOSLIB benchmark to evaluate our algorithms. The DIMACS benchmark taken from the Second DIMACS Implementation Challenge [Johnson and Trick1996]
includes problems from the real world and randomly generated graphs. The BHOSLIB instances are generated randomly based on the model RB in the phase transition area[Xu et al.2005] and famous for their hardness.
Table 1 and Table 2 show the experimental results on these two benchmarks. We report the best size and average size of -plex found by our FD-TS and algorithms, and compare the average time cost of BDCC-H and FD-TS if they have the same best and average solution sizes, shown in column . Most DIMACS instances are so easy that all the three algorithms find the same-quality solution very quickly, and thus are not reported. The result shows that our algorithm not only finds -plexes that FD-TS cannot reach on many instances but usually cost less time than FD-TS on other instances. Particularly, BDCC dominates on the CXXXX.X domain but is not robust enough on the brock domain. With a hyperheuristic, BDCC-H enhances the robustness of BDCC while achieving better performance than FD-TS. Remark that for C1000.9 with , san400_0.7.3 with and DSJC1000.5 with , BDCC-H is the only algorithm that find -plexes of size and respectively in runs.
On the BHOSLIB benchmark, BDCC and BDCC-H dominate FD-TS on most of the instances. We highlight the frb100-40, the hard challenging instance in BHOSLIB. BDCC updates the lower bound of the size of maximum -plex and -plex on frb100-40, indicating its power on large dense graphs. Though BDCC-H does not achieve the same performance as BDCC due to the time-consuming exploration phase of hyperheuristic, it outperforms FD-TS on most of these instances.
6.3 Evaluation on Massive Graphs
We also evaluate our algorithm on massive real-world graphs from Network Data Repository [Rossi and Ahmed2015], Thanks to the powerful peeling technique, most of these graphs are reduced significantly and solved in a short time by BDCC and BDCC-H. For other instances, our algorithms and FD-TS find a solution of the same quality in runs. So we do not report the results. To further assess performance on massive graphs, we choose the state-of-the-art exact algorithm named BnB [Gao et al.2018] for comparison. We run BnB with a cutoff time of 10000 seconds for the optimum solution. For the sake of space, we do not report the instances that can be solved by both BnB and BDCC-H in a few seconds. Table 3 shows the best solution found by BDCC-H and the average time cost of BnB and BDCC-H. An item with a symbol “” in column “max” indicates that this is the size of the optimum solution proved by BnB.
The result in Table 3 shows that BDCC-H can find an optimum solution on most instances while costing much less time. For the instances that BnB fails to solve in 10000 seconds, BDCC-H can return a satisfactory solution within a few seconds.
7 Conclusions and Futrue Work
In this paper, we have proposed two heuristics, BLP and DTCC, for the maximum -plex problem. Based on BLP and DTCC, we develop a local search algorithm BDCC and further improve it by applying a hyperheuristic strategy for the adding and swapping phase. The experimental result shows that our algorithms achieve high robustness across a broad range of problem instances and update the lower bounds on the size of the maximum -plexes on many hard instances. Meanwhile, our algorithm achieve state-of-the-art performance on massive real-world.
In the future, we plan to study variants of CC for other combinatorial optimization problems further. Besides, it would be interesting to adapt the ideas in this paper to design local search algorithms for other clique relaxation model.
- [Balasundaram et al.2011] Balabhaskar Balasundaram, Sergiy Butenko, and Illya V Hicks. Clique relaxations in social network analysis: The maximum k-plex problem. Operations Research, 59(1):133–142, 2011.
- [Boginski et al.2006] Vladimir Boginski, Sergiy Butenko, and Panos M Pardalos. Mining market data: a network approach. Computers & Operations Research, 33(11):3171–3184, 2006.
[Boyan and Moore2000]
Justin Boyan and Andrew W Moore.
Learning evaluation functions to improve optimization by local
Journal of Machine Learning Research, 1(Nov):77–112, 2000.
- [Butenko and Wilhelm2006] Sergiy Butenko and Wilbert E Wilhelm. Clique-detection models in computational biochemistry and genomics. European Journal of Operational Research, 173(1):1–17, 2006.
- [Cai et al.2011] Shaowei Cai, Kaile Su, and Abdul Sattar. Local search with edge weighting and configuration checking heuristics for minimum vertex cover. Artificial Intelligence, 175(9-10):1672–1696, 2011.
- [Conte et al.2018] Alessio Conte, Tiziano De Matteis, Daniele De Sensi, Roberto Grossi, Andrea Marino, and Luca Versari. D2k: Scalable community detection in massive networks via small-diameter k-plexes. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1272–1281. ACM, 2018.
- [Gao et al.2018] Jian Gao, Jiejiang Chen, Minghao Yin, Rong Chen, and Yiyuan Wang. An exact algorithm for maximum k-plexes in massive graphs. In IJCAI, pages 1449–1455, 2018.
- [Glover and Laguna1998] Fred Glover and Manuel Laguna. Tabu search. In Handbook of combinatorial optimization, pages 2093–2229. Springer, 1998.
- [Gujjula et al.2014] Krishna Reddy Gujjula, Krishnan Ayalur Seshadrinathan, and Amirhossein Meisami. A hybrid metaheuristic for the maximum k-plex problem. In Examining Robustness and Vulnerability of Networked Systems, pages 83–92. 2014.
- [Johnson and Trick1996] David S Johnson and Michael A Trick. Cliques, coloring, and satisfiability: second DIMACS implementation challenge, October 11-13, 1993, volume 26. American Mathematical Soc., 1996.
- [Lakhlef2015] Hicham Lakhlef. A multi-level clustering scheme based on cliques and clusters for wireless sensor networks. Computers & Electrical Engineering, 48:436–450, 2015.
- [McClosky and Hicks2012] Benjamin McClosky and Illya V Hicks. Combinatorial algorithms for the maximum k-plex problem. Journal of combinatorial optimization, 23(1):29–49, 2012.
- [Miao and Balasundaram2017] Zhuqi Miao and Balabhaskar Balasundaram. Approaches for finding cohesive subgroups in large-scale social networks via maximum k-plex detection. Networks, 69(4):388–407, 2017.
- [Rossi and Ahmed2015] Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA., pages 4292–4293, 2015.
- [Russell and Norvig2016] Stuart J Russell and Peter Norvig. Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited,, 2016.
- [Sutton and Barto1998] Richard S Sutton and Andrew G Barto. Introduction to reinforcement learning, volume 135. MIT press Cambridge, 1998.
- [Wang et al.2016] Yiyuan Wang, Shaowei Cai, and Minghao Yin. Two efficient local search algorithms for maximum weight clique problem. In AAAI, pages 805–811, 2016.
- [Wang et al.2017a] Yiyuan Wang, Shaowei Cai, and Minghao Yin. Local search for minimum weight dominating set with two-level configuration checking and frequency based scoring function. J. Artif. Intell. Res., 58:267–295, 2017.
- [Wang et al.2017b] Yiyuan Wang, Dantong Ouyang, Liming Zhang, and Minghao Yin. A novel local search for unicost set covering problem using hyperedge configuration checking and weight diversity. Science China Information Sciences, 60(6):062103, 2017.
- [Wang et al.2018] Yiyuan Wang, Shaowei Cai, Jiejiang Chen, and Minghao Yin. A fast local search algorithm for minimum weight dominating set problem on massive graphs. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden., pages 1514–1522, 2018.
- [Xiao et al.2017] Mingyu Xiao, Weibo Lin, Yuanshun Dai, and Yifeng Zeng. A fast algorithm to compute maximum k-plexes in social network analysis. In AAAI, pages 919–925, 2017.
- [Xu et al.2005] Ke Xu, Frederic Boussemart, Fred Hemery, and Christophe Lecoutre. A simple model to generate hard satisfiable instances. arXiv preprint cs/0509032, 2005.
- [Zhou and Hao2017] Yi Zhou and Jin-Kao Hao. Frequency-driven tabu search for the maximum s-plex problem. Computers & Operations Research, 86:65–78, 2017.
- [Zhou et al.2018] Yangming Zhou, Béatrice Duval, and Jin-Kao Hao. Improving probability learning based local search for graph coloring. Applied Soft Computing, 65:542–553, 2018.