Haptic Assembly and Prototyping: An Expository Review

by   Morad Behandish, et al.

An important application of haptic technology to digital product development is in virtual prototyping (VP), part of which deals with interactive planning, simulation, and verification of assembly-related activities, collectively called virtual assembly (VA). In spite of numerous research and development efforts over the last two decades, the industrial adoption of haptic-assisted VP/VA has been slower than expected. Putting hardware limitations aside, the main roadblocks faced in software development can be traced to the lack of effective and efficient computational models of haptic feedback. Such models must 1) accommodate the inherent geometric complexities faced when assembling objects of arbitrary shape; and 2) conform to the computation time limitation imposed by the notorious frame rate requirements---namely, 1 kHz for haptic feedback compared to the more manageable 30-60 Hz for graphic rendering. The simultaneous fulfillment of these competing objectives is far from trivial. This survey presents some of the conceptual and computational challenges and opportunities as well as promising future directions in haptic-assisted VP/VA, with a focus on haptic assembly from a geometric modeling and spatial reasoning perspective. The main focus is on revisiting definitions and classifications of different methods used to handle the constrained multibody simulation in real-time, ranging from physics-based and geometry-based to hybrid and unified approaches using a variety of auxiliary computational devices to specify, impose, and solve assembly constraints. Particular attention is given to the newly developed 'analytic methods' inherited from motion planning and protein docking that have shown great promise as an alternative paradigm to the more popular combinatorial methods.



page 4

page 5

page 9

page 13

page 14

page 19


Haptic Assembly Using Skeletal Densities and Fourier Transforms

Haptic-assisted virtual assembly and prototyping has seen significant at...

Get rid of inline assembly through trustable verification-oriented lifting

Formal methods for software development have made great strides in the l...

Toward Automated Virtual Assembly for Prefabricated Construction: Construction Sequencing through Simulated BIM

To adhere to the stringent time and budget requirements of construction ...

Analytic Methods for Geometric Modeling via Spherical Decomposition

Analytic methods are emerging in solid and configuration modeling, while...

String Diagrams for Assembly Planning

Assembly planning is a difficult problem for companies. Many disciplines...

Peg-in-Hole Revisited: A Generic Force Model for Haptic Assembly

The development of a generic and effective force model for semi-automati...

Shape Complementarity Analysis for Objects of Arbitrary Shape

The basic problem of shape complementarity analysis appears fundamental ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

“Smartness in mechanical devices is often realized through interaction that enhances dumb algorithms so they become smart agents… Interactive systems are grounded in an external reality both more demanding and richer in behavior than the rule-based world of noninteractive algorithms.”

Peter Wegner, 1997 [372]

1.1 Human-Computer Interaction

The role of ‘interaction’ in taking full advantage of computers to support scientific advancements and day-to-day activities is becoming more evident with the rapid growth of hardware and software technologies. Although digital computers have been constantly breaking the records of data storage and processing over the past 50 years, their embarrassingly persistent failure in performing tasks as natural and intuitive to humans as facial recognition, speech recognition, language translation, and similar semantic interpretation problems has raised important questions. Specifically, what are the intrinsic (i.e., theoretical) limitations of the digital computers (and underlying computing models) in automating manual tasks? If computers are great at certain tasks—e.g., storing and processing vast amounts of data in short amounts of time—while humans are better at others—e.g., feeling, learning, and experiencing through the five senses—how can we characterize and leverage those capabilities simultaneously?

Rather than hopelessly trying to entirely automate all tasks that were once entirely manual, is it feasible to realize the ‘best of both worlds’ though effective human-computer interaction (HCI)?222Also called human-machine or man-machine interaction or interface in the literature.

1.1.1 Historical Perspective

Since the birth of the modern theory of computing and the notion of a universal computer by Alan Turing in 1936

[353], there has been no doubt that digital computers can far exceed the human ability to crunch numbers. Ever since, digital computing has revolutionized scientific disciplines, breached engineering limits, and eventually redefined everyday life after the advent of ‘personal computers’ (PC).

Artificial Intelligence (AI).

In spite of the seemingly limitless opportunities of computer-aided automation, the progress in certain directions has been surprisingly difficult. One such example can be seen in the prospects of artificial intelligence (AI) in 1980s. After Turning’s landmark paper in 1950

[354] followed by the famous Dartmouth conference333The Dartmouth conference and summer project is widely considered as the ‘birth of AI’, where the phrase ‘artificial intelligence’ was coined and the famous Dartmouth proposal was asserted: “Every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it.” in 1956 [251], there was a surge of research on AI for about two decades. Although the community’s optimism grew rapidly during those ‘golden years’ of AI, funding in AI suffered a major setback as researchers began to face the difficulties intrinsic to AI problems, and the AI research went through what is now known as the ‘AI winter’ [91]. The question of whether ‘strong AI’444John Searle defined strong AI as the claim that “the appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds.” [317] can be realized at all (or is even desirable anymore) remains unanswered today.

Human-Computer Interaction (HCI).

In a different line of research on theory of computing, Peter Wegner observed that interaction is a fundamentally richer phenomenon than what classical model of algorithms can capture [372]

. He showed that Turing machines cannot model interactive systems

[371] and proposed ‘interaction machines’ as a more powerful and expressive computing paradigm to model features of interactive behavior such as time and dynamically evolving adversaries and oracles [371, 373].555He compares algorithms (without interaction) to ‘sales contracts’ that deliver an output in exchange for an input, while (interactive) objects are like ‘marriage contracts’ that specify the behavior for all contingencies of interaction—i.e., “in sickness and in health [over the lifetime of the object] till death do us part” [372].

With some of the roadblocks being removed by the availability of increasingly more powerful computers and richer computing models, the AI research has began thriving once again. Regardless of whether strong AI and full automation will be made possible in future (or ever at all) and whether or not one day the computers will surpass human cognitive capabilities indiscriminately, an important set of current problems can be solved more effectively by exploiting a collaboration (rather than competition) of humans and computers.

1.1.2 Multimodal Interactions

Any interaction between humans and their environment, including interaction with computers, can be viewed as an information exchange involving a combination of signals received via our five senses—namely, sight, hearing, touch, taste, and smell.666Perhaps with the exception of direct brain-computer interfaces (BCI) made possible for extremely simple tasks using electroencephalographic (EEG) signals collected on the human scalp. Research in this areas is still in the stage of infancy and is limited to few real-world applications. For a recent survey of BCI developments, see [112]. However, the interfaces that we currently use for this type of interaction are often described as an ‘information bottleneck’ [318]. The substitution of punchcards and printers that endured as the primary device for information exchange even into the PC era until mid 1980s with today’s interactive combinations such as keyboards, mice, joysticks, display devices, and more recently, touchscreens, has been a significant progress. However, the extent of the interactive phenomena that can be replicated by these limited set of gadgets is far from being comparable with those of many other human interactions in social and natural environments. These interactions range from those involved in regular day-to-day tasks such as tying one’s shoe laces to specialized skills such as a craftsman’s pottery of an artifact or a musician’s playing of an instrument.

Engaging Multiple Senses.

Significant progress has been made in both research and commercial capacities in incorporating two of the five senses quite effectively. These two senses are, obviously, the sense of hearing engaged by digital audio and the sense of sight engaged by computer graphics technologies. Since the commercialization of the television in late 1940s, the developments in both visual and auditory categories have been driven substantially by the multibillion dollar entertainment (e.g., motion picture and gaming) industry—not to mention other domains such as medical and defense applications.

The next frontier in modeling and implementing human sensorimotor capabilities involves the addition of the sense of touch. Unlike the visual and auditory information that can be more easily replicated, i.e., interpreted, stored, and recreated by (projections of) geometric representations and electrical signals, respectively, it is not very clear what the primitive information units for modeling touch should be. For this and other practical reasons explained in Section 1.2, the touch-enabled HCI has not seen the same rapid growth. The most notable examples of its successful deployment today are either in specific application domains with a restricted and well-understood role for touch—e.g., flight simulation, clinical training, rehabilitation, and alike—or in simplified scenarios that involve a narrow and primitive subset of sensory experience—e.g., touchscreen devices with vibration feedback. In these applications, the type of touch interaction is extremely restrained to facilitate designing special purpose devices that engage only certain parts of the body—e.g., through particular muscles/joints or only fingertips—or involve confined interaction modes—e.g., force feedback at a single point while holding a stylus with a natural grasp like holding a knob, wand, or stylus.777A good example application domain is in clinical training. On the one hand, there is robotic surgery where the interaction modes are quite diverse and challenging to formalize except in a restricted subset of tasks—e.g., cutting tissue using scissors with 1 DOF. On the other hand, there is dental diagnosis which involves little more than what a haptic stylus with a single-point 3 DOF force feedback can replicate. As a result, one observes more mature haptic hardware/software in the latter category currently on the market for training purposes. The reader is referred to [223, 159, 352] for examples of haptic devices on the market.

Figure 1: The evolution of visual aids for mechanical design; from paper-and-pencil sketching (a) to computer-aided tools from 2D drafting (b) to 3D modeling (c).

1.1.3 User Interfaces for CAD

To understand the importance of haptic interfaces, in general, and the implications of developing foundations and algorithms to support touch-assisted engineering tools, in particular, one needs to appreciate the analogous role that the graphical user interface (GUI) has played in the evolution of computer-aided design, analysis, and manufacturing.

Graphical User Interface (GUI).

It is hard to underestimate the crucial role of visual aids (e.g., symbols, sketches, and diagrams) in carrying out even the simplest cognitive processes888“Two minutes with a pencil on the back of an envelope lets us solve problems which we could not do in our heads if we tried for a hundred years.” –Christopher Alexander [5].

let alone its importance in scientific and engineering practice. Architects, engineers, and artists have used sketches for hundreds of years to conceptualize, examine, and communicate their creations. The advent of digital computing and its democratization during the 20th century provided the designers with an incredibly powerful new set of tools. However, to take full advantage of the computational power, there was no other choice but to equip this new machine with visual aids to replace the traditional sketch pad, paper-and-pencil in assisting the designer’s imagination (Fig.


In 1963, Ivan Sutherland developed a revolutionary computer program called Sketchpad: “a man-machine graphical communication system” [338] (Fig. 2). Sketchpad can be viewed as the first computer-aided design (CAD) software and a major breakthrough in the development of CAD and computer graphics as disciplines. It gave birth to the concept of a GUI and spurred a new approach to HCI.

In 1972, Herbert Voelcker and his collaborators started the production automation project (PAP) [358] aiming to develop the solid modeler called part and assembly description language (PADL) [278]. PAP was originally established aiming to provide informationally complete computer representations to support the automation of numerical control (NC) machining. PADL-1 [116] used a combination of constructive solid geometry (CSG) and boundary representation (B-rep) and was made publicly available in 1977. PADL-2 [298] followed closely after in 1981 from which UniSolid was developed by Unigraphics. Around the same time in 1978 the B-rep solid modeling kernel Romulus was released by ShapeData, which influenced the successor kernels Parasolid and ACIS in late 1980s and early 1990s. Other notable CAD software that emerged in those years are AutoCAD, CATIA, and Pro/ENGINEER in 1980s and SolidWorks, SolidEdge, and Autodesk Inventor in 1990s. See [56] for a broad historical review of the evolution of computer graphics and CAD industry, and [374] for a more detailed review of early CAD packages in particular.

In the past two decades, the 3D modeling, simulation, and visualization tools for CAD have evolved into versatile product lifecycle management (PLM) software to manage complete digital product descriptions for development, design, and manufacturing known as digital mock-ups (DMU) [54]. DMU serves as a platform for virtual (or digital) prototyping (VP/DP), which is becoming a commonly adopted design and validation practice in several industrial sectors [32]. Although today’s versatile GUIs allow an interactive visual examination of 3D models at different design and validation stages, a major deficiency can be traced to handling 3D geometry and functions using 2D input/output media [54]. Almost all current CAD software use a combination of widget-based interaction using a pointing device (e.g., a mouse or touchpad) cursor on the screen—namely, the standard windows, icons, menus, and pointer (WIMP) interaction style—and alphanumeric input using a keyboard [46], while research has shown that multimodal interactions enable higher accuracy and lower execution times [162].

Figure 2: Ivan Sutherland showcasing Sketchpad at MIT’s Lincoln Labs.
Haptic User Interface (HUI).

Several general-purpose robotic devices meant for touch-enabled input/output interactions with computers (called ‘haptic interfaces’) ranging from 3D to 6D input (i.e., position and orientation encoding) and output (i.e., force and torque feedback) became widely available in 1990s. Among the most widely used commercial devices one can refer to SensAble Phantom line of devices that originated from MIT Touch Lab [249, 311] (now called Geomagic Touch upon acquisitions in 2012 and 2013), Haption Virtuose6D popular for assembly planning with large workspace requirements, CyberGlove CyberGrasp, Force Dimension Sigma/Omega/Delta, and Novint Falcon.999We hereby emphasize that exemplifying or mentioning device brands that have been used for testing purposes in the reviewed technical papers do not imply recommendation of these devices over any others by the authors. See www.bracina.com/haptichardware.html for a list of haptic devices on the market. Although these general-purpose devices have found numerous applications in research and development labs, as well as special-purpose devices developed in-house and optimized for certain applications, their democratization for the consumer market has been hindered by

  1. the high hardware retail prices due to limited number of units produced;

  2. the lack of common language standards and general-purpose software development platforms to make the technology accessible to non-experts; and

  3. the absence of a clear customer demand—which is both responsible for and aggravated by the above two factors.

The reader is referred to [223, 159, 352] for surveys of the evolution of haptic interfaces and to [330, 107, 356] for the extent of their applications in engineering, medicine, entertainment, and education.

We postpone a more detailed review of the application of haptics to assist VP/VA to Section 2.1. Here it suffices to emphasize that in a similar fashion that developments in ‘computer graphics’ have been essential to achieve the current functional versatility of interactive 3D modeling tools for CAD, the integration of haptic interfaces to achieve their full potential calls for extensive research and development in ‘computer haptics’.101010Haptic studies are typically organized into three subareas; namely, human haptics, machine haptics, and computer haptics [329], which study the physiological, hardware, and software aspects, respectively. Being particularly relevant to this article, computer haptics [164] is defined as algorithms and software associated with integrating touch into (HCI), analogous to computer graphics targeting sight [329]. Developments in both graphics and haptics interaction modes are, in turn, dependent on the advancement of more inclusive mathematical abstractions (e.g., geometric modeling and reasoning) and richer computer representations (e.g., data structures and algorithms).

1.1.4 Practical Significance

Among the numerous applications of graphics- and haptics-enabled HCI, haptic feedback is of particular practical significance in application domains that involve an interplay of shapes and motions. More specifically, when dealing with planning problems that require complex spatial reasoning in higher-dimensional configuration spaces—e.g., assembly planning, path and motion planning, manufacturing process planning, etc.—there are two extreme ends to a computer-aided solution process:

  • The algorithmic approach involves attempting to model a ‘field description’ of the problem [5]—e.g., define a metric to quantify the relative performance of the different plans—followed by devising a search algorithm to find a solution—e.g., an optimization algorithm that navigates through the configuration space and finds local and/or (idealistically) global optima by trial-and-error.

  • The interactive approach involves leveraging the human agent’s expertise and domain knowledge to reject a large subset of trials upfront and eventually select an acceptable (if not optimal) solution by interactive evaluation—e.g., visual and sensorimotor inspection of the configuration space via graphic and haptic feedback, respectively.

Clearly, each of the above approaches has its own advantages and drawbacks. For noninteractive optimization, one needs to come up with a purely quantitative metric to evaluate each and every candidate plan as accurately as possible, which is unrealistic when inevitable heuristics with limited applicability are involved. Even if such a uniform field description is discovered, finding the near-optimal solution requires searching a high-dimensional solution space which is computationally prohibitive, especially in the presence of constraints.

111111Mathematically, a highly constrained search problem—without the knowledge of a parameterization that explicitly guarantees constraint satisfaction—is an attempt to find lower-dimensional manifolds in a higher-dimensional space, which depicts the computational challenge faced in the algorithmic approach by appealing to simple probabilistic arguments. Most search algorithms are doomed to converge to local optima without any global guarantees, and are generally unable to systematically partition the search space to what can be referred to as “qualitatively distinct” solution subspaces or regions with few plausible recent developments [275, 274]. The interactive approach, on the other hand, allows the human agent to identify qualitative distinctions and limit the search space to a few regions while rejecting the rest. The quantitative field description can then assist the user—through proper visual and sensorimotor cues—in finding the local minima in each subspace and further refining the solution.

Besides interactive assembly planning at the focal point of this paper, other related spatial planning problems can also benefit from an energy field description. In particular, automatic robot path planners (including but not limited to assembly planners) that use decomposition methods such as cylindrical algebraic decomposition (CAD) [314, 315, 316] or faster sampling methods such as probabilistic roadmaps (PRM) [208, 209, 28, 131] to construct a combinatorial structure over the configuration space can skip the redundant exploration of a large subset of infeasible solutions by restricting the search space to that swept by the user through interaction. Similarly, protein docking platforms that rely on FFT-based search algorithms [204, 63, 217, 106, 8] to explore the configuration space can exploit the user’s ability to quickly identify qualitative docking sites before making quantitative adjustments to the local configurations.

In the next section, we briefly enumerate the current challenges and promising directions for incorporating haptics into assembly planning applications.

1.2 Research Challenges

The growth in the availability and popularity of the fairly recent haptic technology imposes increasing demands for geometric modeling and computing algorithms, to deliver realistic replication of the real-world experience in virtual environments (VE) as efficiently as possible. We identify the following main challenges in the development of haptic-enabled virtual prototyping tools—with virtual assembly tasks being considered as the particular subarea of interest:

  1. capturing the inherent geometric and spatial complexities faced when assembling objects of arbitrary shape; particularly, the absence of a practically effective force and torque feedback model for assembly and disassembly interactions of general mating pairs; and

  2. abiding by the computation time limitation imposed by the frame rate requirements; particularly, the obligation to carry out all computations per frame within a millisecond to maintain a 1 kHz servo-loop rate for haptic feedback compared to the more manageable 3060 Hz for graphic rendering.

The bigger difficulty is to address both of the above challenges simultaneously. The above two aspects are described in more detail in the following Sections 1.2.1 and 1.2.2, respectively.

1.2.1 Geometric Complexities

The primary challenge in developing generic models for haptic feedback lies in a proper formulation of the guidance forces and torques that effectively assist the user in the exploration of the VE, from repulsing collisions to attracting proper contact. In particular, when two objects (e.g., rigid or flexible parts or subassemblies) are being assembled or disassembled in a VE, the effects of the mechanical forces and torques exchanged during the process are simulated by integrating the equations of motion subjected to the constraints formulated using contact mechanics and friction models. As depicted in Section 2.2

, for the most general type of contact—i.e., a combination of surface, curve, and point contact—collision detection (CD) algorithms are used for computing the resistance forces and torques that are central to the dynamic simulation. However, there are several computational challenges faced when attempting to perform CD in real-time interactive applications. This is especially the case in assembly scenarios that involve tight fits whose ‘nominal’ geometry is described by contact features that reduce the degrees of freedom (DOF) of relative motion—e.g., by restricting the 6D relative translations and rotations of a pair of rigid bodies into lower-dimensional (e.g., 1D or 2D) subspaces of their configuration space. The dynamic simulation along the lower-dimensional contact subspaces for nominal geometry is inherently unstable and difficult to compute due to the extreme sensitivity to infinitesimal perturbations. The natural solution, similar to the case of real physical assembly, is to add finite

clearances according to appropriate geometric dimensioning and tolerancing (GD&T) standards [331] to the nominal geometry for ease of assembly:121212Here we are specifically considering the virtual simulation of the insertion for ‘clearance fits’ that the user can do with bare hands (in both physical and virtual setups alike) and do not require special tools or processes unlike the instances with ‘force fits’, ‘shrink fits’, etc.

  • High-clearance fits have been successfully simulated in VEs—e.g., clearances of 13 mm using approximate CD with resolution of 0.2 mm [320, 321]. However, most real assembly scenarios require much smaller clearances in accordance with GD&T specifications [1].

  • Low-clearance fits, on the other hand, require more accurate CD algorithms—e.g., clearances of 0.001 mm using exact CD on original B-reps [322, 323]. However, exact CD does not exhibit the required performance at 1 kHz with the existing hardware capabilities.

In other words, there is an inevitable trade-off: the practical and efficient CD methods use some approximation—e.g., meshing, voxelization, or bounding volume hierarchies—that compromises the required accuracy for low-clearance assembly, while exact CD is not fast enough to handle numerous parts or complex shapes in real-time. The different CD methods for haptic assembly are reviewed in Section 2.2.2 and in greater detail for general applications in [232, 201, 216].

Sources of Geometric Errors.

It is important to note that the dynamic instability problem of low-clearance fits whose nominal geometry typically characterizes a lower kinematic pair—i.e., contact maintained over a surface, restricting the motion to 1D or 2D—is actually twofold when it comes to interactive VR applications:

  1. the intrinsic geometric representation errors of the approximate CD algorithm—e.g., voxelized or triangulated cylindrical features—which can be eliminated in principle by using exact CD at the expense of computational performance; and

  2. the measurement errors/noise due to the haptic device encoder inaccuracies as well as the ‘jerking’ motion of the hand, which can only be alleviated (e.g., by filtering) at the software level but cannot be completely eliminated.

Unlike what happens in real physical assembly, the dynamic behavior in virtual assembly is simulated using a finite-difference integrator, whose stability is very sensitive to these errors. As a result, CD alone has been found by several researchers [355, 284] to be insufficient for virtual assembly, especially with haptic assistance.

Recognizing Mating Features.

For the lower kinematic pairs that are completely classified into the well-known six classes

[277] (detailed in Section 2.1.3), it is possible to abstract the DOF-limiting contact subspaces in terms of ‘virtual fixtures’ [304] or ‘mating constraints’ [180] between functional surfaces of simple (e.g., planar, cylindrical, spherical, or conical) shapes. The forces and torques for haptic guidance during insertion of these features are then simplified, for example, by using spring-damper models between the current and eventual configurations. Although such approaches provide faster and more effective alternatives to CD for low-clearance assembly, they also depend on at least one of the following simplifications:

  • a priori assumptions on the type of surface features and corresponding kinematic pairs, their explicit semantics, and exact locations on the different parts and subassemblies manually specified either by the CAD designer (pre-importing) or by the VE user (on-the-fly); or

  • heuristic methods to automatically detect the assembly intent and associated mating features as soon as the features are brought to insertion proximity.

The abstraction of the mating features for higher kinematic pairs between features of arbitrary shape, on the other hand, is significantly more difficult since no such finite classification exists for the more general case.

As a consequence of these disadvantages in using each method by itself, the common theme in the recent haptic assembly systems is to use hybrid techniques [355] (detailed in Section 2.4.2) that switch between CD and feature-based constraint resolution. This duality creates extra complications for switching and blending between the two phases [284]. Moreover, when fixated on the low-clearance fit of a particular pair of features, collision events outside the insertion site can be missed. This defeats one of the main purposes of virtual assembly, which is the early detection of design issues such as unaccounted clashes between the different parts.

1.2.2 Computational Limits

The efficiency problem appears more challenging in the case of haptic feedback, when compared to graphic rendering, due to the notorious physiological requirement of (at least) 1 kHz refresh rate necessary for satisfactory tactile experience—especially to acquire the necessary stiffness when manipulating rigid objects [284]—while only 3060 Hz is typically perceived as adequate for appealing to human vision [107].

Frame Rate Requirements.

The human touch perception is typically classified into ‘kinaesthetic’ sensations that are related to muscle control and limb motion, and ‘tactile’ sensations perceived at the skin receptors [107].131313More technically, the term ‘proprioception’ is used for the broad class of perceptions of the position, state, and movement of the body and limbs in space. This includes vestibular, kinaesthetic, and cutaneous sensations. The ‘vestibular’ sensations pertain to the perception of balance, head position, and acceleration/deceleration. The ‘kinaesthesia’ includes the sensation of movement of the body and limbs originating in the muscles, tendons, and joints. The ‘cutaneous’ sensations pertain to the skin itself, including sensations of pressure (from mechanoreceptors) as well as temperature and pain (from nociceptors) the former of which is more specifically referred to as ‘tactile’ [283]. Although a bandwidth of 10 Hz is typically considered adequate for the kinaesthetic sensations [329], it is largely dependent on the task, e.g., 12 Hz for unexpected signals, 25 Hz for periodic signals, up to 5 Hz for internally generated or learned trajectories, and 10 Hz for reflex actions [329, 330]. However, to adequately simulate rigidity with a force feedback device (e.g., in VR-CAD applications), higher frequencies are required due to basic control-theoretic considerations; namely, noting that the maximum stiffness in a closed-loop system is inversely proportional to the square of the regulation period [284]. On the other hand, vibrations of up to 1 kHz can be resolved by the human tactile system, with the highest sensitivity at 250 Hz [330]. To collectively comply with all of these requirements, a response rate requirement of 1 kHz is widely accepted as the standard for most haptic applications—see [55, 329, 330, 107] for more details on the subject. It should be noted, however, that there are studies suggesting that even higher rates of 510 kHz might be necessary for improved haptic performance in certain tasks [218] one example being high-fidelity texture discrimination [75, 76].

2 Literature Review

The rest of this paper is devoted to a survey of research and development in haptic-assisted VP/VA in Section 2.1, a classification of the dominant existing themes for assembly constraint modeling in Sections 2.2 and 2.3, and a proposal for future directions and opportunities in Section 2.4.

2.1 CAD/CAAP Applications

Today, most engineering design tasks are heavily assisted by powerful and widely available computer simulation and visualization tools. Although a large subset of analysis and synthesis tasks have been partially (if not fully) automated, the designer’s decision-making remains central to certain aspects of the design process. This in turn creates a demand for more effective human-computer interfaces to explore more efficient, creative, and cost-effective design solutions in semi-automatic setups.

In particular, haptic assistance has been found useful in several design activities that can benefit from domain expertise and cognitive capabilities of human operators—which are hard to formalize for full automation such as conceptual design [32, 33, 36], aesthetic design [35, 41, 42], design review and functionality validation [265, 391, 113, 97], ergonomics and human factors evaluation [325, 198, 31, 43], and many more reviewed in Appendix A.1. In particular, haptic manipulation has also been leveraged for editing parametric CAD models and freeform surfaces for designing individual parts [94, 95, 237, 238, 239, 123, 124, 125]. Particular attention will be given to methods concerned with applying haptics to the evaluation and planning of assembling and disassembling (already designed) rigid and flexible parts—another critical and costly step in the product design process [29].

A thorough review of research efforts and organization of the literature on graphics- and haptics-assisted virtual assembly can be frustrating, as appreciating its current position and potential implications in the modern product life-cycle management (PLM) requires an overview of a range of different topics. In this section, we provide a sufficiently broad review—brief in text, less so in the number of citations—of some of the relevant research studies in applying virtual reality (VR) and augmented reality (AR) tools to computer-aided design and assembly planning (CAD/CAAP), virtual prototyping (VP) (Section 2.1.1), and virtual assembly (VA) (Section 2.1.2). Of course we do not intend, by any means, to provide a complete survey of VP/VA related research or all published industrial implementations. Instead, we shall focus on providing the reader with clear definitions and classifications of the existing approaches to addressing the difficulties described in Section 1.2 along with introducing common terminologies, comparing advantages and drawbacks, and citing (more than enough) pointers if further details are sought. Particular attention is given to the different conceptualizations of constraints (Section 2.1.3) that emerge in VR-CAD assembly and disassembly problems. Constraint modeling and resolution are central to solving the motion of arbitrarily complex parts in contact in real-time—attempting to simultaneously address both of the challenges presented in Section 1.2—using physics-based (Section 2.2), constraint-based (Section 2.3), or combined (Section 2.4) techniques.

2.1.1 Virtual Prototyping

Recently, an early-stage examination of different product life-cycle aspects related to design, manufacturing, maintenance, service, and recycling has been made possible by integrating VR tools into the modern CAD environments, a practice referred to as virtual (or digital) prototyping (VP/DP) [146, 54, 362, 98]. Such an evaluation results in a significant reduction of time and cost associated with physical prototyping (PP), and allows for the elimination of a large subset of design issues in the earlier stages of the process [89, 90]. Although they cannot yet completely replace physical prototypes, virtual prototypes are less expensive, more repeatable, and easily configurable for different variants, hence provide significant insight into the functionality of the product while eliminating redundant design trials and excessive tests [32].

In the larger context of modern PLM, one often encounters the notion of a digital mock-up (DMU). DMU consists of a complete digital descriptions of the product during its entire life-cycle, serves as a platform for product and process development, and includes geometric, ergonomic, and functional information—with or without human-computer interaction element. DMU construction accounts to realistic computer simulations that are capable of replicating different functionalities ranging from design, manufacturing, maintenance, service, and recycling of the physical mock-up (PMU) [134]. The interactive application of immersive VR tools for a subset of those tasks, including (but not limited to) assembly and disassembly process verification, ergonomics and functional assessment, and other design evaluations are the subset of DMU development technologies referred to as VP [134].141414The notions of physical/digital mock-ups (PMU/DMU) and physcial/digital prototyping (PP/DP) are sometimes used interchangeably, but it is safe to say that a more precise definition puts the latter as a subset of the former that involves VR technologies [134].

For reviews of PP versus VP techniques and their classifications, capabilities, and limitations for product development, we refer the reader to [25, 402, 132]. Currently, the most notable industrial applications of VP are found in the automotive and aeronautic industries [96, 23, 401]. Instructive (although not very up-to-date) surveys of manufacturing applications in general can be found in [327, 265, 269]. In particular, an assessment of the capabilities of VR hardware and software tools available in early 2000s to support VR integration into product life-cycle management (PLM) is given by Jayaram et al. [195].

Appendix A.1 gives a detailed account of several important and relatively recent studies and systems that use haptic support for a variety of design, analysis, validation, and manufacture process planning activities. Table 1 provides a more extensive and chronological list of studies along with their hardware and software components and key features.

2.1.2 Virtual Assembling

Among other VP activities, virtual assembly (VA), defined as a simulated assembly of the virtual representations of mechanical parts in an immersive 3D user interface using natural human motions [211, 212] (Fig. 3), characterizes an important subarea of VP, to which applying haptic feedback has been shown particularly beneficial in terms of task efficiency and user satisfaction by multiple researchers [134, 359, 231, 380]. For instance, a user survey carried out in BMW by Gomes and Zachmann [134] predicted an important role for VR tools in prototyping and assembling activities in the future of the automotive industry. In particular, they showed that VR-enabled DMUs reduce the need for PMUs and facilitate an improvement of the overall product quality. However, the study concluded that a widespread use of VR in manufacturing industries is contingent upon its seamless and complete integration into CAD/CAAP. Volkov and Vance [359] showed that haptic assistance improves the ability of a user for design decision making in VP/VA environments; particularly in terms of task efficiency (e.g., less time taken for user evaluation) and user satisfaction when evaluating automotive design examples. Lim et al. [231] showed that small (i.e., visually insignificant) geometric features—e.g., chamfer or fillets in a simple peg-in-hole pair—can significantly affect user performance in haptic assembly, with measurements showing a similar trend to those of physical assembly. Their results demonstrate that adding haptic feedback to the VP/VA process enables exploiting shape information that are significantly underused when only visual feedback is provided. Wildenbeest et al. [380] conducted experiments to investigate the impact of haptic feedback quality in the performance of teleoperated assembly in the context of four sub-tasks; namely, free-space movement, contact transition, constrained translational, and constrained rotational tasks. They concluded that low-frequency haptic feedback improves overall task performance and control effort in constrained translational and rotational tasks.

Figure 3: Application of haptic technology to virtual assembly and prototyping. Pictures courtesy of Perret et al. [284].

In the past two decades, there have been numerous studies and systems focused on the development of immersive virtual environments for solving assembly and disassembly problems. These systems have used a variety of visualization tools (e.g., stereoscopic displays and goggles) and tracking devices (e.g., head tracking devices and data gloves) to assist the user in virtual object manipulation tasks. More recently, an increasing number of studies have leveraged haptic devices to provide a more realistic assembly experience with force feedback, a thorough account of which would constitute enough material for a full book on the subject. Different survey articles have been authored recently, each focusing on different aspects of the haptic assembly technology:

  • See Seth et al. [324] for an earlier qualitative review of previous studies and existing systems on VP/VA with and without haptic feedback. Here we focus more on the haptic-assisted systems in greater detail, also including the more recent developments.

  • See Xia et al. [385] for a brief overview of the existing challenges and research directions in VR and particularly in leveraging haptics for product assembly with rigid parts and soft cables. Here we intend to identify and analyze the most important challenges in greater depth and propose a precise research agenda.

  • See Leu et al. [226] for a state-of-the-art report on the methods for CAD model generation from digital data acquisition, motion capture, assembly modeling, HCI, and CAD-VR data exchange, to enable assembly simulation, planning, and training. Their report covers different (and important) aspects of CAD-VR integration that we shall not discuss here.

  • See Vance and Dumont [355] for a popular proposal for a promising future direction in haptic assembly—to which we will return in Section 2.4.2.

  • See Perret et al. [284] for a discussion of some of the technical issues faced by haptic assembly in practice along with an assessment of the technical maturity using the technology readiness level (TRL) index—to which we will return in Section 2.4.1.

Other decent reviews on the subject from multiple points of view—followed by presentations of specific systems or methods—can be found in [230, 182].

This survey distinguishes itself by presenting conceptual insight into the back-end abstractions of geometric features that lead to formulating assembly constraints and computing guidance forces and torques for haptic feedback. Although the main theme is of geometric modeling and spatial reasoning techniques, we also provide the most complete catalogue of previous studies and systems to date, which could serve as a useful collection and a comprehensive “beginner’s guide” to the literature.

Appendix A.2 gives a detailed account of several important and relatively recent studies and systems that use haptic support for assembly and disassembly related problems among other prototyping activities. Table 2 provides a more extensive and chronological list of studies along with their hardware and software components and key features.

Computational Commonalities.

As catalogued in the appendices, there is a bewildering variety of VR systems implemented using different software libraries and computational tools for haptic-assisted assembly and disassembly activities. In spite of their major differences in terms of setup (e.g., desktop versus CAVE-like systems), architecture (e.g., PC versus network-based systems), and hardware (wand-type versus glove-type devices), the underlying geometric and physical modeling of most systems are quite similar; namely, they all use the following basic tools:

  • collision detection (CD) to avoid penetration between parts and subassemblies, typically available as part of the physics simulation engines (PSE), which can be conceptualized in terms of ‘physical constraints’;

  • heuristic attraction and/or repulsion forces and torques formulated using spring-damper models or some other simple potential energy field derived from CAD model’s mating pairs called ‘geometric constraints’;

or a blended combination of both. But as we shall articulate in the rest of this section, CD-based physical constraints are difficult to resolve in the vicinity of tight (i.e., low-clearance) fits for a variety of reasons (see Section 2.2), while the artificial geometric constraints are too simplistic to handle arbitrary geometry (see Section 2.3). The following section clarifies the conceptual distinctions between different types of constraints and their implications for VR implementations.

2.1.3 Constrained Motion

The simulation of assembly and disassembly processes for rigid parts can be abstracted as a free motion with 6 DOF (3 for translations and 3 for rotations) per part along with a set of constraints that restrict motion along those DOFs and create interdependencies across different parts. The existing approaches for simulating ‘part behaviour’ in VR systems are typically classified into two groups, with the following definitions [136]:

  • physically-based modelling (PBM), which uses Newtonian/Lagrangian dynamics to solve for the motion trajectories of the virtual objects, under the effect of forces and torques due to physical contact between those objects (e.g., no-penetration impact forces, sliding friction forces, etc.) and environmental effects (e.g., gravity, viscosity, etc.); and

  • constraint-based modeling (CBM), which uses additional geometric constraints to locate the parts in the assembly configuration by artificially reducing the DOF of the manipulated objects, similar to a CAD system.

Although such a distinction is popular in the literature with slightly variant articulations, it is imprecise and often misleading. This becomes clear by noting that motion dynamics is essentially a constrained optimization problem in disguise, hence there is no fundamental distinction between PBM and CBM as defined by the above statements. In particular, rigid body dynamics is given by the Lagrangian formulation of the equations of motion, where the requirement of no-collision between solids is equivalent to a holonomic unilateral constraint [243]. Impulse forces and torques then originate from the Lagrange multipliers associated to these constraints by Gauss’ principle of least constraint [266, 293]. Therefore, there is no fundamental difference, in terms of the underlying physics and mathematics, between finding the motion trajectories under the effect of contact forces and torques (i.e., PBM), on the one hand, and limiting the motion DOF by additional constraints (i.e., CBM) on the other hand. In fact, most implementations use both PBM and CBM, even though one of them might appear as the dominant theme.

Constraining Schemes.

One can still draw a more meaningful classification based on how the constraints are formulated in practice from a knowledge of individual part geometries and their spatial relations. Based on a review of different techniques in the literature, we believe that the following provides a more precise and consistent definition with respect to the existing methodologies:

  • The first approach uses ‘physical constraints’ defined as the set of constraints that arise organically from part geometries (e.g., holonomic constraints due to no-collision condition) and kinematics (e.g., nonholonomic constraints due to sliding motion specifications); whereas

  • The second approach uses ‘geometric constraints’ or ‘kinematic constraints’ introduced artificially to replace the collision response and simplify the solution, ranging from manually specified ‘virtual fixtures’ to heuristically identified ‘mating constraints’ (e.g., co-planarity, co-axiallity, vertex/edge/face coincidence, distance or angle offsets, etc.).

Once again, we do realize that the use of the adjectives ‘physical/geometric/kinematic’ for the classes of constraints still bears the possibility of confusion. The so-called physical constraints are directly imposed due to the interplay between geometry and kinematics of different objects, and the artificial constraints are also solved by appealing to physics-based dynamic simulation.151515It is possible to devise purely kinetostatic constraint solvers by ignoring the dynamic effects—i.e., solving 1st-order, rather than 2nd-order differential equations. However, realistic simulation requires taking dynamic effects into account. Therefore, the keywords “organic” versus “artificial” would perhaps constitute more meaningful adjectives for the different types of constraints classified according to this scheme. Nevertheless, we use the former terminology (with some extra care) for the sake of consistency with the conventions in the literature.

Figure 4: Examples of peg-in-hole assembly problems. Figure adopted from [16, 20].
Example 1.

Consider the simple peg-in-hole assembly examples shown in Fig. 4. One could use a variety of geometric representations including exact (e.g., parametric B-rep) or approximate (polygonal mesh or voxelization) representations to solve for the no-collision constraints. In this case, these ‘physical constraints’ arise organically from the part geometries and are implicitly accounted for by ensuring—through the application of contact forces and torques—that the intersection volume of the two parts remains zero at all times. However, whenever the contact geometry is simple—e.g., cylindrical as in Fig. 4 (a) or prismatic as in Figs. 4 (b, c)—it is possible to simplify the problem by restricting the motion DOF (e.g., from the original 6 DOF to 1, 2, or 3 DOF) by artificially introducing ‘geometric constraints’. In this case, the complex problem of intersection test between arbitrary shapes is reduced to that of simpler geometric abstractions (i.e., virtual fixtures) such as incidence relations between axis lines and corner points. Such incidence relations can be enforced by virtual (axial and torsional) spring-damper couplings whose equilibrium states correspond to proper alignment of virtual fixtures.

Kinematic Pairs.

A more careful examination of the type of assembly problems similar to the above example yields a more rigorous classification of the assembly simulation methods based on the type of ‘kinematic pairs’ [299]:

  • Lower kinematic pairs correspond to surface contact between parts and are classified completely into the well-known six classes [277]—namely, revolute, prismatic, helical, cylindrical, spherical, and planar. Each class corresponds to an automorphism (i.e., symmetry subgroup) of the 6D configuration space (space) of rigid motions.

  • Higher kinematic pairs correspond to curve or point contact between parts, for which no such classification exists. For pair of objects of arbitrary shapes, the type of contact belongs to this class in general.

For lower kinematic pairs, the aforementioned classification enables one to encode all possible interactions between a given pair of parts into well-defined types of ‘mating constraints’ as is customary in many CAD systems. Examples are co-planarity, co-axiallity, co-centricity, and similar conditions. These constraints can be simplified into incidence relations between lower-dimensional geometric constructs that abstract the type of contact. For instance, the cylindrical pair in Fig. 4 (a) can be abstracted by the incidence of the axis of the peg (or at least two points on it) with the axis of the hole, which constrains the motion to a 2D subgroup of ; namely, a translation along and a rotation about the same axis. The enforcement of this constraint in a VE can be realized by a spring-damper coupling between the two cylindrical axes (or certain points on them) that resists an increase in the angle between the two, but is indifferent to the relative rotation around the axes. Similarly, the prismatic pairs in Fig. 4 (b, c) can be captured by additional conditions to lock the rotation around the axes and restrict the motion further into a 1D subgroup of . See also Fig. 7 in Section 2.3.1 for a similar method [343, 344].

An examination of the different existing implementations reviewed in Appendix A.2, reveals that most PBM-based simulations use CD to identify the repulsive effects that resist the penetration of individual parts, making direct use of geometry (i.e., exact or approximate representations). On the other hand, most CBM-based simulations disregard the explicit geometric information and use the additional mating constraint semantics imported from the CAD models to implement the spring-damper couplings that contribute both attractive and repulsive effects. An exception to this theme is the Snap-to-Fit system by Olsson et al. [279] which makes direct use of explicit geometric information to create attractive and repulsive effects between the parts that are close to each other without penetration. More specifically, each point on the moving part’s surface is coupled to the nearest neighbor on the stationary part’s surface using a virtual spring-damper to create the snapping effect. Although it applies to arbitrary geometry and does not depend on simplifying assumptions on the contact features and kinematic pairs, such a simplistic ‘magnetic’ energy model is often counterintuitive and countereffective with regard to the assembly intent. In particular, the underlying energy field merely attempts to bring the parts to proximity by pairing the closest points on their respective surfaces and is indifferent to the geometric constraints like the ones implied implicitly in the peg-in-hole examples of Fig. 4. In contrast, Behandish and Ilieş offered an energy model in [16, 20, 19, 21, 15] that exhibits stronger relationships with both physical (i.e., collision resistant) and geometric (i.e., mating induced) constraints and blends the two in a single formulation. More details on this unified approach are given in Section 2.4.3.

2.2 Physically-Based Modeling

In order to realistically simulate the dynamic interactions between parts and subassemblies (including the user’s interface object/avatar), the majority of haptic-enabled assembly systems perform an explicit real-time integration of the 2nd-order differential equations of motion. These equations are either formulated as Newton+Euler’s equations—e.g., as in [11, 312, 365, 364, 22]—or Lagrange’s equations—e.g., as in [241, 6, 332, 341, 342, 114, 296]—both of which are equivalent in terms of the underlying mathematics, but offer different computational procedures.

The dynamic simulation typically runs at lower rates (e.g., Hz) compared to the haptic rendering loop at the device level, and the two are interfaced using a ‘virtual coupling’ [81, 2, 3, 150, 127, 161], which is essentially a spring-damper model that connects two virtual instances of an object, one residing in the physics simulator and the other assumed to be attached to the user’s interaction point.

Among the earliest attempts for using PBM in haptic assembly were earlier implementations of VEDA [147, 146], VADE [191, 194, 196], and HIDRA [252, 90, 89] systems. Examples of more recent PBM-based systems are SHARP [319, 320, 321, 322, 323] and HAM(M)S [227, 228, 229, 230, 231, 136, 138]. These systems (and a number of others) were reviewed in more detail in Appendix A.2. The most challenging set of computations in PBM are due to solving physical constraints arising from contact between different objects in the scene, including rigid and flexible parts and subassemblies (typically imported from complex CAD models).

2.2.1 Physical Constraints

There are two common approaches for computing the contact forces and torques enforcing the CD-induced physical constraints in real-time:

  1. The first method, referred to as the ‘penalty method’, uses simple force and torque models that make explicit use of collision response—e.g., a linear spring-damper model for computing the normal contact forces proportional to a measure of penetration between objects (or their offset shells) [115, 152, 153] and a proper friction model using the normal pressures and the relative sliding/rolling kinematics to compute the tangential forces [10, 264, 158].

  2. The second method, referred to as ‘constraint-based’—another unfortunate terminology that contributes to confusion with the CBM concepts161616This is a possible ground for confusion (due to the bad terminology of PBM vs. CBM) as the naming similarity suggests that the PBM class is confined to explicit penalty methods only while the implicit constraint-based methods should be classified under CBM. However, this in not the case since CBM deals completely with a different family of constraints added artificially on top of the physical constraints, as pointed out in Section 2.1.3.—instead takes an implicit account of the unilateral contact constraints and solves the more complex set of constrained equations of motion, using non-smooth Lagrangian mechanics [114, 296, 341, 342].

The penalty method is easy to implement and fast to integrate—given an efficient collision response and impact/friction modeling algorithm—due to the simple form of unconstrained Newton+Euler’s or Lagrange’s equations of motion. However, the robustness of the penalty method is heavily dependent on small integration time-steps to ensure minimal violations of constraints and rapid response to correct them. This is difficult to achieve with accurate elasticity models for impact mechanics, inferring normal forces from penetration depth as well as friction models for rolling/sliding mechanics, inferring tangential forces from relative kinematics. The constraint-based method, on the other hand, is more difficult to implement and takes more computing time due to the solution of complicated differential equations, especially as the number of contact features increases. However, it produces more accurate and reliable results, avoids the overhead due to predicting the penetration depth for collision response, and provides straightforward means to model tangential friction forces [241, 332]

. Both methods are dependent on collision detection (CD), although they might use different CD information such as minimum distance, intersection volume, interpenetration depth, contact normal vector, etc.

2.2.2 Collision Detection

Figure 5: Combinatorial CD methods use auxiliary approximate representations ranging from voxel maps (a) to bounding spheres (b–d). Figure courtesy of Nießner et al. [276] and Kavan and Zara [206].

There are several surveys of CD methods (in a general context) for rigid bodies [232, 201, 216] and flexible elements [345].171717For a good collection of collision detection and proximity query packages, visit the website of the GAMMA research group (Lin et al. and Manocha et al.) at the University of North Carolina at Chapel Hill: gamma.cs.unc.edu/research/collision. Here is a list of references in chronological order: I-COLLIDE [80], V-COLLIDE [177], RAPID [139], V-Clip [263], IMMPACT [381], H-COLLIDE [142, 143], PQP [222], SWIFT [104], SWIFT++ [105], PIVOT [165, 166], DEEP [214], CULLIDE [141], DEFORMCD [140], DVD [334], and SELF-CCD [340]. Here we restrict ourselves to a brief review of the most popular methods for real-time computations.

The classical polyhedral CD methods were used in the earliest systems for haptic assembly. Examples are Voronoi-clipping/marching methods—e.g., V-Clip [263], SWIFT [104], and SWIFT++ [105] used in HIDRA [252, 90, 89]. For realistic applications with geometric complexities that require mesh approximations with large polygon counts (i.e., in the order of millions of triangles), these methods are not fast enough to support the 1 kHz haptic rendering.

On the other hand, the bounding volume hierarchy (BVH) methods have been among the most popular CD methods for graphic and haptic rendering purposes. These methods operate by approximating virtual objects recursively with hierarchies of simple bounding shapes (offering fast collision predicates) stored in a tree-like data structure. This allows for quickly ruling out early miss configurations in the broadphase CD as well as a trade-off mechanism between accuracy and running time; namely, by proceeding deep enough down the tree to consume as much of the millisecond per haptic frame as made available for CD budget. Examples are axis-aligned bounding box (AABB) tree-based methods—e.g., a simple algorithm in [267] used in an early haptic training platform [268]—and oriented bounding box (OBB) tree-based methods—e.g, I-COLLIDE [80] and V-COLLIDE [177] giving birth to the H-COLLIDE [142, 143] haptic module, and RAPID [139] used in MIVAS [360, 399]. Among other BVH methods are discrete oriented polytope (DOP) tree-based methods [392, 215] which are generalizations of AABB trees with cubic bounding boxes to convex polytope bounding volumes. A more successful approach is that of hierarchical bounding sphere (HBS) tree-based methods [174, 291, 281, 175, 176, 280, 99, 51], which use bounding spheres (as the name indicates) instead of boxes or other polyhedra, including variants such as the bounded deformation (BD) trees [189]. The spherical symmetry of the primitives in all levels of the HBS trees offers simple and fast radial-based collision predicates that are invariant under rotations, making them popular in more recent haptic implementations [307, 61].

For a long time, uniform volumetric enumeration methods such as the one used in Boeing Corp.’s Voxmap PointShell (VPS) library [361, 259, 260, 261] became very popular for VR applications [202, 297, 211, 212]. VPS works by testing the moving objects represented by a shell of vertices and normals (i.e., the ‘pointshell’) against the stationary obstacles represented by a map of voxels (i.e., the ‘voxmap’), and was used in the earlier versions of SHARP [320, 321]. Several improvements were proposed to the VPS method, ranging from model enhancements—e.g., by using signed distance fields to enhance continuous force and torque response [12, 13]—to implementation speed-ups—e.g., by using improved data structures [308, 309]. Although still being popular due to their simplicity and efficiency, the approximate nature of discrete volumetric representations makes them ineffective for low-clearance assembly [320, 321, 322, 323]. To overcome this, later versions of SHARP [322, 323] employed the Collision Detection Manager (CDM) module of SiemensD-Cubed, which makes direct use of exact B-rep information extracted from the CAD models. Of course, this comes at the expense of slowing the CD process down and making it impractical for large and complex models (e.g., with numerous NURBS patches).

Coutee and Bras [89] compared multiple polygon-based CD toolboxes—namely, V-Clip [263], SWIFT [104], and SWIFT++ [105]—with VPS [361, 259, 260, 261] in terms of their features and capabilities to provide closest point, collision features, penetration depth, geometric constructions, multibody detection and their effectiveness for haptic simulation. They argued that the lone advantage of V-Clip over the other algorithms is that it provides (a not-so-accurate measure of) penetration distance, which can be overcome by using simple tricks via SWIFT(++). On the other hand, VPS and SWIFT++ have the attractive feature of handling arbitrary nonconvex objects, while V-Clip and SWIFT can only handle nonconvex objects as collections of convex pieces. Kim and Vance [211, 212] conducted a more inclusive study comparing a larger number of CD packages—namely, I-COLLIDE [80], V-COLLIDE [177], RAPID [139], PQP [222], and SOLID [24] in addition to the aforementioned four—in terms of their query types and response times. The study concluded that VPS is a better choice due to its ease of CAD preprocessing, faster CD query response, and ability to model physical interactions between parts for force feedback—hence its broad popularity ever since for developing haptic rendering systems.

A promising method was recently developed based on hierarchical inner sphere tree (IST) packing [376, 379, 375] and successfully applied to PBM for haptic rendering [378, 377]. In contrast to the bounding sphere methods described above, the IST packing algorithms use the hierarchy of spheres to pack the interior of the virtual objects—altering the topological structure. Nevertheless, both methods share the computational advantage due to the spherical symmetry in primitive collision predicates. The sphere-packing approximation of the interior can be viewed as a nonuniform extension to the uniform volumetric enumeration approach used in VPS, since it starts from a grid-based discretization of the shape (similar to the voxmap in VPS) over which the distance function is computed and the sphere centers are populated using a simple greedy algorithm. It was shown to outperform the VPS for nonconvex moving objects, but its effectiveness to handle thin objects is yet to be tested [284].

Discrete vs. Continuous Methods.

In contrast to the aforementioned discrete methods that evaluate the collision certificate at intermittent integration time-steps, others have developed continuous methods based on OBB trees [292, 294, 295, 394, 395, 396]

, which attempt to interpolate the first instance of contact in between two subsequent time points along the dynamic time-stepping. This enables speeding up the integration of constrained motion by decreasing the number of unilateral constraints using a fast ‘clash detection’ algorithm based on relative motion during each time-step

[284]. Although this method was used in an operational context in the industry, it was soon abandoned due to a lack of commercial support [284].

Figure 6: Different configurations (a) are evaluated using a gap function (b), based on overlapping indicator functions (c), formulated as a cross-correlation. Compare to Fig. 9.
Combinatorial vs. Analytic Methods.

At a very abstract level, one could classify CD methods into combinatorial techniques (including most, if not all of the aforementioned methods), and analytic methods [243, 18]. PBM applications require not only a collision/non-collision certificate, but a gradient of the constraint function to compute the contact forces and torques. A disadvantage of the combinatorial methods is their indirect approach to infer a gradient-like quantity from the certificate point, which is not trivial for general surface contact [243]. The analytic methods [85, 133, 266], on the other hand, provide a more uniform and robust alternative, which has been popular for a long time in robotics [242].

Recently, Lysenko [243] developed an efficient analytic method for CD based on earlier works in robotics spatial planning [207, 93] and group morphology [303, 244, 245]. The method formulates the so-called ‘gap function’ [151] which measures the penetration volume as a convolution

of nonnegative real-valued defining functions—e.g., the indicator (i.e., characteristic) functions, bump functions, etc.—of the two objects over the configuration space (

space) of their relative motion. By taking advantage of the powerful convolution theorem

, the method applies Fourier transforms to compute the collision response and its general

space gradient (both with respect to translations and rotations) for narrowphase CD. After sampling the defining functions enumerated over a uniform grid (i.e., voxel map), the computations can be carried out rapidly in parallel (on CPU or GPU) using the radix-2 fast Fourier transform (FFT) [88].

More recently, Behandish and Ilieş [18] extended this method to take advantage of the more time- and memory-efficient spherical decompositions—as in HBS and IST methods described earlier. The idea is to replace the grid-based uniform sampling of the defining functions with nonuniform grid-free samplingover a grouping of 3D balls of different sizes, viewed as a 3D slice of a grouping of 4D cones of the same size, which, in turn, can benefit from the convolution paradigm and the nonequispaced fast Fourier transforms (NFFT) [290]. This follows some of the most effective computational methods in state-of-the-art protein docking [8, 7] and enables combining the strengths of combinatorial sphere-trees and analytic convolutions.181818The drawback is that unlike classical FFTs that are implemented as efficient, stable, easy-to-use and widely available open-source packages on the CPU (e.g., FFTW [118, 119]) and the GPU (e.g., cuFFT(W)), similar implementations for NFFTs are scarce [210, 219].

The analytic methods have great potential for haptic applications, as they are indifferent to the topological, geometric, and syntactic complexities of the colliding objects, and provide a graceful trade-off mechanism between fidelity and running time by ‘low-pass filtering’ and ‘anti-aliasing’ of the Fourier transforms of the individual objects [243, 19, 21]. The latter is particularly appealing for haptic application with a sub-millisecond budget for CD and rending altogether, which dictates an upperbound on the number of dominant modes in the frequency domain representations of shape descriptors. Although at very high accuracies the FFT-based CD can become slower than other exact techniques, the CD cost scales with desired accuracy in a predictable and well-define manner, irrespective of geometric complexity. Thus one can fine-tune and calibrate the low-pass filter by specifying the available time budget, querying the hardware capacity, and deciding on the accuracy trade-off without worrying about the actual geometric details. As more efficient high-performance hardware solutions become available, the filter window function can be enlarged systematically to achieve higher accuracies without violating the frame rate requirements.

In principle, one could use the analytic methods based on convolution and Fourier filtering for time-critical narrowphase CD instead of (or in combination with) traditional combinatorial CD methods for guiding haptic assembly and disassembly. However, we are not aware of their implementation into any software library or simulation engine at the time of writing this article. Nevertheless, the difficulties faced in low-clearance assembly will continue to exist as a natural result of input data noise due to device inaccuracies and hand vibration, leading to unstable dynamic response and undesirable ‘buzzing’ in haptic feedback.

Example 2.

Figure 6 (b) illustrates the translational space landscape for a pair of 2D solids shown in panel (a), along with the colormap for the gap function measuring the intersection area. To make the illustration possible, the motion is restricted to translation only, i.e., the landscape in panel (b) is a section (corresponding to zero rotation) through the full 3D space. Each of the relative positions in panel (a) are represented by a point in panel (b). The gap function formulated as the convolution of part indicator (i.e., characteristic) functions penalizes collision as in positions A and B (i.e., ), but does not differentiate point contact in C and separation in D from proper fit/contact in E (i.e., ). This is clearly due to the fact that the convolution integral is only able to measure properties that correspond to full-dimensional (i.e., Lebesgue-measurable) intersections while those of lower-dimensional contact regions vanish during the course of integration.191919This is a major caveat to keep in mind when using analytic approaches in general which only allow a one-to-one correspondence between regularized morphological concepts and measure-theoretic equivalents [244, 245]. A solution paradigm by introducing Dirac delta calculus into the analytic methods is given in depth in [14].

The gap function approach to CD by convolving characteristic functions [244, 245, 243] was extended to formulate a generic ‘geometric energy’ field [15] by convolving skeletal density functions (SDF) [16, 20], which, in addition to accounting for the repulsive effects and resist inter-penetrations, incorporates attractive effects that take over when there is no collision and favor proper fit/contact configurations exhibiting better local shape complementarity [17] (Fig. 8). The method can still benefit from CPU- and GPU-accelerated FFT computation and on-the-fly filtering [19, 21] for sub-millisecond rendering budget in 1 kHz haptic assembly. More details on this method are given in Section 2.4.3.

Rigid vs. Flexible.

A discussion of real-time CD methods would not be complete without a reference to contact modeling between deformable objects, which are prevalent in industrial assembly (e.g., electric cables, hydraulic hoses, rubber seals, leather furnishings, etc.) [284]. Classical methods range from discrete mechanical elements (DME) [52, 53, 120] to finite element method (FEM) [400, 163, 187, 285], which are suboptimal for high frame rate haptic rendering. A few recent studies have successfully applied Signorini’s contact model to haptic assembly of deformable objects [102, 101, 103]. Nevertheless, handling complex deformable shapes and large models remains a challenge [284].

Physics Simulation Engines (PSE).

Combinations of the aforementioned combinatorial algorithms have been implemented into popular physics simulation engines (PSE) for graphic and haptic rendering, such as Ageia PhysX SDK [60] and Bullet Physics SDK [309]. Gonzalez-Badillo et al. [137, 135] recently conducted a comparative performance evaluation of these PSEs and their CD capabilities in practice for haptic assembly. In particular, they measured and compared the task completion time (TCT), mean force feedback (MFF), and physics simulation time (PST) indices in several benchmark examples for the static trimesh/HACD module of PhysX versus the GIMPACT module of Bullet and concluded that in general the latter outperforms the former for haptic assembly tasks. An important exercise would be to

  1. develop robust implementations of analytic methods into the existing PSEs or stand-alone open-source and widely-available libraries; and

  2. conduct similar comparative studies to understand their strengths and limitations, and the opportunities to blend them together with combinatorial techniques.

2.3 Constraint-Based Modeling

Figure 7: Using virtual fixtures for abstracting the mating constraints for cylindrical and prismatic pairs (a, b). At the close vicinity of the insertion site, the virtual fixtures (e.g., guiding planes in (c)) are used to align the peg along the hole. Once the proper alignment is reached, the mating pair’s DOF is limited to model a suitable mechanical joint (e.g., cylindrical pair in (d)). Figure courtesy of Tching et al. [343, 344].

Although implementing physics-based simulation with a combination of CD and impact/friction mechanics seems the most natural choice (at least in theory) for a virtual mimicry of real-world constrained motion, it is not reliable in practice for final insertion of the objects into position [320, 321, 322, 323, 355, 284].

At a fundamental level, this happens due to the degeneracy of the collision-free feasible subspace (i.e., the ‘free space’) in the neighborhood of the final assembly configuration, leading to decreased DOF associated with common mating constraints. Most mating constraints used to model zero-clearance mechanical joints are characterized with multiple compatible unilateral (i.e., inequality) constraints that are critically satisfied during contact leading to one or more bilateral (i.e., equality) constraints, restricting the motion to a surface or a curve.202020An inequality constraint is ‘critically satisfied’ if . Two constraints and are compatible if they define a nonempty subset of the space. If that subset forms a lower dimensional subspace over the intersected boundaries, then they collectively define a bilateral equality constraint . Such a degenerate collection of critically satisfied inequality constraints is extremely unstable with respect to small perturbations, rendering lower kinematic pairs and their constraint resolution practically ‘incomputable’. This is because a small error in constraint specification may lead to major topological changes to the collision-free configuration subspace (i.e., the ‘free space’) that either completely eliminate the constraint or make it theoretically unresolvable unless tolerances are explicitly incorporated.

Example 3.

Take, for example, two inequality constraints and corresponding to non-penetration condition for two planar surfaces between which a planar part is sandwiched without a clearance. The intersection of the two constraints —assuming nonnegative analytic functions —is a 3D subspace of the 6D configurations space defining a lower kinematic pair with two translational and one rotational DOF. Upon introducing a small perturbation (e.g., ) that 3D subspace may widen up to a 6D region (e.g., in ‘clearance fits’) or completely disappear (e.g., in ‘shrink fits’).

At a practical level, it is difficult to stabilize the motion along the degenerate subspace for at least two reasons:

  1. numerical geometric errors due to the approximate representations used in fast CD methods popular for haptic rendering (e.g., voxelized interior or triangulated boundary); and

  2. input noise due to authentic hand vibrations and device encoder reading errors.

The latter is particularly important, implying that even with the improved accuracy of the algorithms that use fine mesh approximations or exact B-rep data, using CD alone for low-clearance insertion is still impractical [322, 323, 355]. One can alleviate the problem by using smaller integration time-steps to keep the unnecessary minor collision events (i.e., violations of the equality constraints) at a minimum, to achieve more stable dynamic response and haptic feedback. However, the finite time-step in real-time applications is lowerbounded by the computation time per frame, which is dictated by hardware capacity.

An alternative solution is to artificially introduce a set of bilateral constraints, rather than relying solely on the group of unilateral constraints organically resulted from CD. Such an approach is mathematically equivalent to a local reparameterization of the configuration space to embed the feasible subspace and to ensure the fulfillment of the constraints explicitly rather than attempting to solve for the original constraints specified implicitly. These artificial constraints are typically classified into geometric (i.e., holonomic) and kinematic (i.e., nonholonomic) constraints.

2.3.1 Geometric Constraints

In most commercially available design and assembly environments such as the modern CAD software (e.g., CATIA, Pro/ENGINEER, NX, etc.) the so-called ‘mating constraints’ are classified into simple spatial relationships between the contact features, such as co-planarity in prismatic features, co-axiallity in cylindrical features, distance and angle offsets, etc. To solve the final insertion problem for haptic assembly, one practical approach is to manually specify the mating constraints in close proximity of the final assembly configuration. The geometric constraints can be either extracted and imported from the CAD model—i.e., manually specified by the CAD user (e.g., the designer)—or specified on-the-fly within the VE—i.e., manually specified by the VE operator (e.g., the inspector)—using a variety of constraint management systems developed for VR-CAD applications [247, 271, 272]. For example, VADE [191, 194, 196] and MIVAS [360, 399] directly imported pre-defined constraint information from Pro/ENGINEER CAD models. The later versions of SHARP [322, 323] used the Dimensional Constraint Manager (DCM) module of SiemensD-Cubed for defining and solving geometric constraints within the VE itself. Rather than using the assembly semantics of the original CAD models, the virtual constraint guidance (VCG) method presented by Tching et al. [343, 344] relied on user-specified ‘virtual fixtures’ [304, 305], which are added abstract and simple geometric elements rigidly attached to the fixed and moving parts—e.g., a pair of perpendicular planes intersecting at the axis of a cylindrical hole, to constrain and guide two points selected along the axis of a cylindrical peg. Figure 7 illustrates the use of virtual fixtures for peg-in-hole examples with cylindrical and prismatic mating pairs.

A few recent studies attempted to automatically identify the assembly intent and associated geometric constraints by analyzing semantic information of individual part geometries [247, 179, 180, 48], referred to by Vance and Dumont [355] as the automatic geometric constraints (AGC) method. This method relies on matching ‘functional surfaces’ [180]—e.g, a cylindrical surface characterized by its axis and diameter, which could be used to predict the intended mating relation and associated trajectories when a peg is brought to the proximity of a hole. However, these methods are limited to matching simple (e.g., planar, cylindrical, spherical, and conical) geometric features. The effectiveness of both VCG and AGC methods relies heavily on either manual specification of the type of mating selected from a finite library of simple constraints, or heuristic models for identifying such mating pairs when the corresponding simple geometric primitives are in proximity. A generic solution that automates the identification and pairing for features of arbitrarily complex surface geometry is missing.

2.3.2 Kinematic Constraints

In a similar fashion to holonomic (i.e., geometric) constraints, one could specify nonholonomic (i.e., kinematic) constraints that depend on the relative linear and angular velocities of parts during assembly. An important caveat is related to proper differentiation of the motion of the haptic proxy from discrete encoder readings in the presence of the added noise and device errors. Our experiments have led to the observation that the data filtering provided by commercial haptic device libraries and APIs might not be adequate for 1st and 2nd differentiation, and additional techniques might be required [190].

2.4 Future Directions

Having outlined current practice and existing methodologies in haptic-assisted VP/VA (including PBM and CBM), this section briefly comments on promising directions for future studies and how the approach presented in this thesis contributes to these advancements.212121Most of Sections 2.4.1 and 2.4.2 are adopted from [284, 355].

2.4.1 Technology Readiness

Perret et al. [284] presented an assessment of the maturity of the technical solutions in haptic-assisted assembly using the technology readiness level (TRL) index originally developed by NASA in the 1980s for space-flight systems and later adopted and expanded by the US Air Force to encompass other technologies. Based on this measure, they assessed the maturity of interactive rigid-body assembly simulation with haptic feedback quoted here as follows:

  • In terms of ‘path finding’: TRL = 8, i.e., “technology has proven to work in its final form and under expected conditions.”

  • In terms of ‘final insertion’: TRL = 5, i.e., “the basic technological components are integrated with reasonably realistic supporting elements.”

  • In terms of ‘human positioning’: TRL = 6, i.e., “representative model or prototype system, which is well beyond TRL = 5, is tested in a relevant environment.”

The ‘path finding’ problem—more commonly referred to as ‘path planning’ in the robotics literature [242]—is of assessing the feasibility of assembly by identifying a collision-free path (or lack thereof) from some initial configuration of a given part or subassembly to the final configuration. If no such path exists, the goal is to identify the bottlenecks and critical collision points to modify the product or process in order to resolve the design problem. The most popular automatic approach to this problem is by using probabilistic roadmap (PRM) planners [209, 208, 28, 131] which randomly sample the configuration space of rigid motions and connect the nearby points to form a graph over which a feasible path can be traced. Although automatic path planning performed in an offline preprocessing step can be used later for haptic guidance as in [77, 78, 79, 154, 155, 156, 220, 221], the true benefit of haptic-assisted VP/VA is realized when it is used as an alternative to automatic path planning by calling upon the human’s cognitive abilities and understanding of spatial relationships—e.g., to feel bottlenecks, evaluate clearances, and explore possible improvements—rather than exploiting the computational power of the machine [284]. Hence it appears that basic CD-based PBM remains to be the most natural toolbox for this purpose. The TRL of 8 indicates an adequate level of maturity in this aspect, even though consistent computational improvements are needed to enable fast CD for increasing size and complexity of models. It has been shown that analytic methods similar to the one presented in this thesis are a promising future direction for both path planning [207, 93] and CD [243, 18].

The ‘final insertion’ problem appears as the least mature component with a TRL of 5, facing multiple challenges to reach effective industrial implementation. It is impossible to rely on CD alone for the final insertion of a part when low-clearance fits are involved [284]. One promising future direction to solve the final insertion problem is to use a hybrid PBM+CBM approach (detailed in Section 2.4.2) where the traditional simulation of contact-constrained motion is confined to a ‘free motion’ phase while fixed mating constraints are artificially added to the PSE solver for a ‘fine insertion’ phase. Once again, these constraints are extracted from CAD model as in [191, 194, 196, 360, 399], defined by the user within the VE as in [322, 323, 343, 344], or determined (i.e., guessed) automatically by the system from an inspection of the geometry as in [247, 179, 180, 48]. One challenge faced by the hybrid approach is developing an algorithm to account for the switch between the two phases, using a variety of remedies such as blending algorithms as in [322, 323, 286] or guiding mechanisms as in [343, 344]. The second challenge is that switching off CD altogether during final insertion might lead to missing contact or collision outside the insertion site [284], to which no plausible solution has been published so far to the best of our knowledge. The alternative analytic approach [15] unifies the two phases (detailed in Section 2.4.3) without having to deal with the challenges presented by such an artificial duality. However, in spite of its theoretical elegance and computational advantages, the latter has not been yet tested for usage in an industrial capacity.

The ‘human positioning’ problem is largely beyond the scope of this article. One of the greatest challenges in this area is of the computational intensity of introducing an avatar into the simulation environment whose realistic model involves hundreds of new rigid bodies to deal with. See [284] for more details.

2.4.2 A Hybrid Approach

Figure 8: Even the artifacts bounded by the simplest lower-order algebraic (e.g., planar, cylindrical, etc.) surfaces can exhibit complex shape complementarity relations that are hard to detect by combinatorial matching of assembly features. Figure adopted from [19, 21].

Although it has been shown that reducing the DOF of motion using geometric constraints supports highly accurate manipulation and positioning in VEs [50], the ad hoc nature of the constraint detection algorithms does not provide sufficient generality to completely replace CD to constrain the motion. Consequently, the state-of-the-art in haptic assembly is a ‘two-phase’ approach [355], i.e., to divide the process into a ‘free motion’ phase accomplished with the help of CD engines, and a ‘fine insertion’ phase using pre-specified or computer-predicted constraints. Vance and Dumont [355] observed such a common theme among a few recent studies and systems:

  1. The automatic geometric constraints (AGC) method by Seth et al. [322, 323] relied on CD between B-rep surfaces during a free motion phase when the parts move freely (i.e., with 6 DOF except for the collision-induced constraints) in the VE. Contact between B-rep elements signaled a switch to the insertion phase of the assembly simulation when the geometric constraints were used to guide the assembly process. These geometric constraints were easily defined based on the B-rep semantics and automatically identified using simple heuristics—e.g., alignment between the cylindrical axes. Once a constraint was identified, the two parts were aligned and the number of DOF allowed for the user motion was reduced to impose the geometric constraint [355].

  2. The virtual constraint guidance (VCG) method by Tching et al. [343, 344] used non-smooth dynamic simulation during an exploration phase when the parts move freely (i.e., with 6 DOF except for the collision-induced constraints) in the VE. The method relied on virtual fixtures [304, 305] to guide the moving object to a specific configuration without making any changes to the underlying CAD geometry. Once the proper insertion alignment was reached, the CD was disabled and the assembly phase was started by modeling the final insertion as a DOF-limited relative motion of simple mechanical joints (e.g., prismatic, ball, hinge, etc.). The transition between the two phases was triggered by the collision of the virtual guides between the moving and stationary objects [355].

  3. The dynamic decomposition and integration of DOF (DIOD) method by Veit et al. [357] divided the assembly task into a ballistic phase and a control phase, and took a different approach based on detecting velocity changes to trigger the switch between the two phases. During the ballistic phase the user could freely move and manipulate an object at a 1:1 ratio between the tracked (i.e., device) and virtual (i.e., object) velocities. During the fine positioning phase, instead of scaling the resultant velocity to increase the dexterity, the total velocity was decomposed along 3 orthogonal directions and the scaling was applied only to the component of the velocity that is below a given threshold [355].

There are two major difficulties faced in this approach. First, it requires developing mechanisms to detect the insertion intent (e.g., using pose or speed clues) and to model the transition between the two phases. The aforementioned implementations typically rely on CD between surface elements associated with insertion constraints [322, 323], CD between the user-defined virtual fixtures [343, 344], or velocity changes that hint on the user’s intent to perform an insertion task [357]. Once the alignment has been reached, part CD is switched off and the number of DOF is reduced to assist the user with final insertion. Second, switching off part CD altogether is not satisfactory as a contact with geometry outside of the insertion area could oppose the movement [284]. To the best of our knowledge, the latter problem is also open.

2.4.3 A Unified Approach

Figure 9: Different configurations (a) are evaluated using a score function (b), based on overlapping skeletal densities (c), formulated as a cross-correlation. Compare to Fig. 6.

Both physical constraints (i.e., captured by CD and used in PBM) and mating (geometric and/or kinematic) constraints (i.e., interpreted from CAD and used in CBM), which govern the part behavior in the two phases of the hybrid approach described in Section 2.4.2, are solutions to the same basic problem that lie on the two extreme ends of an spectrum in conceptual and computational terms:

  • The former methods apply to arbitrary geometry (i.e., general solids with semianalytic surfaces) while the latter intrinsically work only for simple shapes (e.g., assembly features with 1st- or 2nd-order algebraic surfaces).

  • The former methods are numerically complex and time-consuming even using approximate representations (e.g., voxmap enumeration, polyhedral meshing, or BVH tree-based sampling) while the latter are as simple as computing basic and fast geometric predicates.

  • The former methods provide a sense of crisp contact (e.g., impact and friction) at the expense of instability to small perturbations while the latter create a (sometimes unrealistically) more flexible sensation with the additional benefit of robustness to errors and noise.

Is it possible to come up with a single part behavior model that subsumes both of the above and intrinsically enables a smooth transition between them rather than salvaging one to blend the discrete phases?

Behandish and Ilieş proposed in [16, 20, 19, 21, 15] a generic and unified energy model for real-time assembly guidance that applies to objects of arbitrary shape. The formulation starts from the part geometries and directly computes the guidance forces and torques from shape descriptors of interacting features. No simplifying assumption is made on the geometry of the mating features. It was shown that implicit generalizations of the so-called virtual fixtures [304, 305] that are widely popular for haptic assembly [343, 344, 355, 284] automatically appear in the form of interactions between skeletal density function (SDF) (Fig. 10). The SDF shape descriptors are piecewise continuous functions defined over the 3D space for each individual part, whose distributions capture the topological and geometric properties of the surface features that partake in assembly. The spatial overlapping of individual part SDFs—interpreted in the analytic formulation as a convolution—generates an artificial potential energy (called the ‘geometric energy’) field [15] which creates attraction forces and torques towards the proper alignment of assembly features. The same energy model also provides repulsion forces and torques as a natural byproduct, in the case of collisions. Therefore, it unifies the two phases of free motion and precise insertion into a single interaction mode, thus avoids the duality and switch altogether. The method subsumes analytic collision detection (CD) [243], and provides a generalization to analytic feature matching for geometric guidance.

Example 4.

Figure 9 repeats the gap function computation over the space in Example 2 (Fig. 6), this time using more sophisticated shape descriptors than simple indicator functions—such as the skeletal density functions (SDF) shown in panel (c) [16, 20, 19, 21, 15]. This enables assigning higher shape complementarity scores (thus lower haptic energies) to configurations with proper fit/contact, e.g., point E in panel (b). This comes at the expense of disappearance of the clear contact boundary between collision and no-collision regions, i.e., the obstacle, retrieved as the the sublevel set of the gap function , and the free space.

Additionally, the formulation of the energy function as a convolution allows using ideas from multivariate harmonic analysis [205] to streamline haptic feedback computations. The convolution in the physical space (where the part geometries reside) transfers into a pointwise multiplication of the Fourier expansion of the SDFs for the individual parts (i.e., the ‘amplitudes’ of the multi-dimensional SDF signals). Guided by this property, the formulation leads to a straightforward mathematical relationship between the Fourier representations of the SDF shape descriptors and the geometric energy field, which can benefit from the efficiency of the FFT algorithms [88]. Moreover, explicit analytic equations are obtained for computing the gradients of the convolution function (i.e., guidance forces and torques) for arbitrary spatial translations and rotations.

The performance is significantly improved using optimized FFT implementation on the highly-parallel GPU architecture. As a result, haptic-enabled simulation of realistic assembly scenarios with complex CAD models and low-clearance fits is made possible to an adequate fidelity with the application of GPU-accelerated FFT calls.

In addition to its theoretical generality, computational efficiency, and scalability with parallel-computing, the main advantages of this paradigm compared to the existing methods are the following:

  1. The analytic formulation is generic, allowing for a variety of different shape descriptors—ranging from simple PMC to intricate SDF—to be constructed using different kernels in the general formula.

  2. The decomposition of the shape descriptors into their Fourier components (i.e., the ‘modes’ of the 3D signals) allows for a systematic means to trade off the accuracy of computations with the amount of available computation time and resources. In the case of haptic assembly, where there is a limited budget of time (ideally millisecond) available to each simulation frame, one could use truncated Fourier expansions (i.e., apply a ‘low-pass filter’ to the 3D signals) to significantly speed up the convolution in real-time.

  3. The computational performance is indifferent to geometric and syntactic complexity of the objects in the physical domain. Unlike the existing combinatorial approaches to collision detection which scale in computation time with input complexity—e.g., the number of points, triangles, or voxels used in the representation—our method’s efficiency depends solely on the degree of fidelity specified by the low-pass filter (i.e., number of retained dominant modes), and does not scale with the original input complexity.

Figure 10: The interactions between SDF shape descriptors can be viewed as a generalization of virtual fixtures.

In spite of their promising theoretical and computational advantages, there are still important problems that need to be addressed and interesting research avenues to explore in order to realize the full potential of analytic methods:

  • The uniform sampling required by the classical uniform FFT algorithm [88] is not memory-efficient when large models with high levels are detail are rasterized. A promising new direction in this aspect is provided in [18] using ideas from protein docking [8] and tools such as the nonuniform FFT algorithm [290].

  • Although the translational (i.e., commutative) component of the SDF convolution is handled efficiently by the Fourier methods, the rotational (i.e., noncommutative) component still requires sampling and interpolation. An interesting subject to look at in this area is the noncommutative harmonic analysis [74].

  • The preliminary validation results obtained so far are not nearly as strong as their counterparts presented in the literature for the currently popular approaches. In particular, the scalability of these methods from assembling pairs of rigid parts [16, 20, 19, 21, 15] to gigantic assemblies with numerous rigid and flexible elements simulated at hierarchical levels of detail (LOD) is yet to be evaluated. Before this method can be adopted into physics simulation engines (PSE), robust implementations along with extensive testing are imperative.

3 Conclusion

Haptic-enabled assembly planning has been restrained for a long time from achieving its full potential, due to the challenges presented by the competing objectives of handling high geometric complexity while maintaining a response rate of 1 kHz. The current computational models for constraint-based assembly guidance are either

  1. limited to the assembly of solids with very simple geometric features that are automatically detectable; or

  2. heavily dependent on user input for constraint specifications.

Both methods generally presume a priori knowledge of the type of contact surfaces that one deals with, and are not generalizable to support objects of arbitrary shape. The majority of ad hoc solutions start from identifying the simplistic DOF-limiting constraints—e.g., restricting the contact features to planar, cylindrical, spherical, or conical surfaces or their intersection curves—followed by what can be conceptualized as simple energy formulations to enforce those constraints—e.g., spring-damper models to penalize the violation of co-planarity or co-axiallity conditions.

Lately, the dominant direction in this area has been aligned with a hybrid approach [355, 284], separating the simulation into a ‘free motion’ phase, using unilateral (i.e., inequality) ‘physical constraints’ originated from collision detection; and a ‘fine insertion’ phase, using bilateral (i.e., equality) ‘mating constraints’ (e.g., of geometric and/or kinematic types) introduced artificially to limit the DOF. While the former fail to produce dynamically stable guidance for low-clearance insertion, the latter are either dependent on a priori manual specifications by the user, or are limited to simple semialgebraic features (e.g., planar, cylindrical, spherical, or conical) that can be identified automatically from CAD semantics using heuristic algorithms. The identification of the switch criteria between the two phases, on the one hand, and modeling the insertion constraints for different contact surface features, on the other hand, remained an open problem for objects of arbitrary shape.

An alternative promising direction is that of analytic methods [85, 133, 266] that have been popular in robotics for several decades and have found applications more recently in narrowphase CD [243, 18]. These methods are advantageous over combinatorial techniques in their indifference to topological, geometric, and syntactic complexities, as well as the meaningful trade-off mechanism they provide between accuracy and performance for time-critical applications “on a budget” by Fourier filtering. An extension of the Fourier CD method is only recently being applied to haptic assembly [16, 20, 19, 21, 15], which formulates a ‘geometric energy’ field to unify the collision response and geometric guidance into a single interaction and applies to arbitrarily complex shapes. Although this unified paradigm provides a promising alternative direction for haptic assembly, its practical application to real-world assembly planning, simulation, and verification scenarios at an industrial capacity is yet to be tested, which, in turn, is contingent upon its integration into existing PSEs.

4 Acknowledgements

This work was supported in part by the National Science Foundation grants CMMI-1200089, CMMI-1462759, and CISE-1526249.

Appendix A Literature Catalogue

In this appendix we present examples of several important and relatively recent studies and systems that use haptic support for a variety of VP/VA activities, catalogued on the basis of authoring research groups.

Different applications of haptics to design, analysis, validation, and manufacture process planning are reviewed in Section A.1. The applications of haptics to assembly and disassembly related problems are exclusively reviewed in greater extent in Section A.2. The enumerated collection is by no measure comprehensive, but presents a broad selection of a range of different research developments—the most complete collection in one place to the present date, to the best of our knowledge.

a.1 Haptic-Enabled VP

Hollerbach et al. at the University of Utah developed a ‘haptic display’ for grasping and manipulating virtual mechanisms (e.g., linkages and chains) [246, 273] using an exoskeleton haptic device Sarcos Dextrous Arm Master—later upgraded to Sarcos DTS Master Exoskeleton for subsequent work [117]. They also integrated the haptic interface with Utah’s Alpha-1 geometric modeling system to enable manipulation of both polygonal (i.e., mesh) and freeform (i.e., parametric) surfaces, particularly using direct parametric tracing (DPT) [348] for tracing untrimmed and trimmed NURBS surfaces [347, 346] and physics-based models—e.g., stick-slip friction [310] and nonlinear viscosity [248]—for rapid virtual prototyping [167, 168]. Among other related works of the group is nonlinear device modeling for VR applications [83, 84].

max width=0.95 References Year System Methods Software Hardware Key Features [168] 1996 PBM, Utah’s Alpha-1, Sarcos Dextrous exoskeleton haptic interface for CAD; [167, 347, 348] 1997 CD. ControlShell, Arm Master. uses mesh and DPT for elastic CD; [246, 273] 1998 TCP and UDP. asynchronous networking with device. [346] 2005 [95] 1999 PBM, GHOST API. Phantom 1.0. freeform sculpting using B-splines; [94] 2001 MSS, mass-spring discretization of surfaces; CD. dynamic optimization of control points. [257, 258] 2001 Digital PBM, GHOST API. Phantom 1.0. ‘virtual clay’ using subdivision solids; [62, 255] 2002 Sculpture DSS, mass-spring discretization of lattice; [254] 2005 CD. dynamic optimization of mass points. [256] 2007 [109] 2000 PBM, FreeForm. Phantom Desktop. concept generation via sketch elevation; [73] 2001 CD. form approximation; surface shaping; [108] 2005 wire-cutting, smoothing, and mirroring. [236] 2003 Virtual CBM, GHOST API, Phantom Desktop, COM-based CAD-VR interoperability; [238, 237] 2004 DesignWorks CD. MS COM+. Unspecified OST-HMD, freeform-based B-rep surface operations; [239] 2005 5DT FOB OTs. shape control functions and SVD. [387] 2003 PBM, FreeForm, Phantom Desktop, uses mass-spring model for elastic CD; [66, 68, 388] 2004 RLE, GHOST API, ABB IRB 1400. uses S-RLE description for plastic CD; [69, 391, 389] 2005 FEM, VTK Toolkit. both additive and subtractive forming; [67] 2007 CD. models milling and path generation. [65] 2004 HVCMM PBM, GHOST API, Phantom Desktop. uses S-RLE and custom model for CD; [70, 390] 2005 RLE, models CMM inspection path planning; [363] 2006 CD. models CMM accessibility analysis. [367] 2009 [122] 2004 PBM, GHOST API. Phantom Premium. freeform sculpting using B-splines; [123, 124] 2005 MSS, mass-spring discretization of surfaces; [125, 121] 2006 CD. dynamic optimization of control points; [126] 2007 implicit modeling of prob/tool heads; [86] 2004 VRAD PBM, EVI3d Drivers, CAVE-like System, allows implicit edition of CHG data; [286, 287, 288] 2008 CD. VEserver, head tracking devices, models haptic selection and extrusion; [328] 2009 OpenCASCADE, WTP haptic devices, enables multimodal interactions; [46] 2010 OpenGL. IBM ViaVoice. future developments aimed at CATIA. [30] 2004 T’nD PBM, Device APIs. FCS-HapticMaster, uses tesselated models for CD; [32, 33, 34] 2006 CD. nVisor ST OST-HMD, uses chip removal theory for PBM; [92] 2007 Vicon M2-460. conceptual design and ergonomics; [36] 2010 creates NURBS for downstream. [30] 2004 VeRVE PBM. Device APIs, Haptic Knob(s), uses ‘smart’ haptic feedback (tacton); [32] 2006 UGS Jack. nVisor ST OST-HMD, models ergonomic interactive testing. [31, 43] 2007 Vicon M2-460. [38] 2008 PROGIMM PBM, 3DVIA Virtools, KUKA KRC and KR3, mixed reality and mixed prototyping; [39, 40] 2009 CD. KRL+XML. nVisor ST OST-HMD, ergonomic assessment of driver seats; [57] 2010 Vicon M2-460. haptic tools for automotive industry. [58, 59] 2011 [35] 2008 SATIN PBM, SML Manager, FCS-HapticMaster, haptic strip for curve approximation; [41, 113, 97] 2010 CD. ThinkCore API, DVHDS Components, various haptic modules and knobs; [42] 2012 OpenIVI, nVisor ST OST-HMD, CAD modeling and shape analysis; TCP/IP. Vicon M2-460. conceptual and aesthetic design. [233, 233] 2012 DesignWorks CBM, GHOST API, Phantom Desktop, variational B-spline editing techniques; [235] 2014 CD. MS COM+, Unspecified HMD, real-time energy minimization. [240] 2015 VBS Kernel. 5DT FOB OTs.

  • Abbreviations used for generic terms:

    physically-based modeling (PBM), constraint-based modeling (CBM), collision detection (CD), virtual environment (VE), virtual reality (VR, augmented reality (AR), computer-aided design (CAD), application programmer’s interface (API), software development kit (SDK), transmission control protocol (TCP), user datagram protocol (UDP), internet protocol (IP), direct parametric tracing (DPT), mass-spring system (MSS), dynamic subdivision solids (DSS), optical see-through (OST), head-mounted display (HMD), boundary representation (B-rep), optical tracker (OT), singular value decomposition (SVD), run-length encoding (RLE), finite element method (FEM), spatial run-length encoding (S-RLE), coordinate measuring machine (CMM), wand-type pointer (WTP), construction history graph (CHG), nonuniform rational B-splines (NURBS), extensible markup language (XML).

  • Abbreviations used for brand names: general haptic open software toolkit (GHOST), haptic virtual coordinate measuring machine (HVCMM), virtual reality aided design (VRAD), cave automatic virtual environment (CAVE), touch and design (T’nD), virtual reality system for validation of equipment controls (VeRVE), la progettazione immersiva multisensoriale (PROGIMM), sound and tangible interfaces for novel product design (SATIN), Microsoft (MS), Component Object Model (COM), Flock of Birds (FOB).

  • means two-handed haptic device or a pair of one-handed devices.

Table 1: A chronological review of haptic-enabled virtual prototyping platforms for various product development tasks.

Qin et al. at the State University of New York at Stony Brook developed a variety of haptic sculpting systems by applying physics-based modeling based on lumped mass-spring networks—made of ‘control points’ and ‘mass points’ connected within a control mesh and a network of springs—and Newtonian dynamics to different solid representations. The representations include B-spline surfaces discretized with linear springs over the control mesh [94, 95]; dynamic subdivision solids [253] discretized with both linear and angular springs over the control lattice (called the ‘virtual clay’ method)222222Rossignac et al. at the Georgia Institute of Technology used a similar project name (the ‘digital clay’ project) [128, 306, 45] for the development of a new type of haptic interface for finger sculpting; namely, a computer-controlled physical surface that deforms in response to the pressure changes exerted by bare hands [128, 306], built using ‘formable crust architectures’ [45]. Ishii et al. at the MIT Media Lab recently developed a similar concept called InForm [224, 225] for ‘physical telepresence’ and remote collaboration. [255, 257, 258]; volumetric implicit functions (i.e., ‘density fields’) used to define semialgebraic sets bounded by a finite number of B-spline patches discretized into a grid of ‘density springs’ [171, 172, 173]—which are also capable of performing CSG operations and knot insertion; and dynamic pointset surfaces by fitting implicit functions to local distance fields and applying ideas from implicit modeling [144]. The group later implemented the ideas from both volumetric subdivisions and implicit modeling into a system called DigitalSculpture [254] for interactive surface editing. Among their other relevant works are direct mesh editing using PDE-based geometric surface flow in a system called HapticFlow [100] and applying FEM to incorporate flexibility into subdivision solid geometry for haptic sculpting [256]. A Phantom 1.0 device (3 DOF input, 3 DOF output) was used for all applications.

Liu et al. form the Queen’s University of Belfast developed the first VR-CAD system called Virtual DesignWorks [236, 238] that used MicrosoftCOM+ technology for real-time exchange of models between a CAD module (e.g., NURBS-based B-reps for flexible editing) and a haptic module (e.g., polygon/voxel-based for fast rendering) for freeform surface editions (e.g., pulling, pushing, and dragging). The COM-based implementation enabled real-time interoperability between a typical CAD kernel (e.g., used in SolidWorks, Unigraphics, etc.) and an approximate representation of geometry for haptics. In a follow-up study [237, 238] they used ‘shape control functions’ to simulate surface deformations which yield a linear system of constraints solved by the singular value decomposition (SVD) method. Later in Hebei University of Technology, Liu also implemented variational B-spline (VBS) techniques and real-time energy minimization (using Hebei’s VBS kernel) for surface hole filling [234, 240] and for interactive surface editing [233, 235] into DesignWorks.232323The later versions of Virtual DesignWorks [236, 238] was called DesignWorks [233, 235] in subsequent publications. A Phantom Desktop device (6 DOF input, 3 DOF output) was used for all applications.

Chen et al. at the University of Hong Kong developed a product development platform with a wide range of haptic functionalities [67, 68, 69] including machine tool path planning [387]—e.g., for 5-axis milling based on the method used in MIT Suzuki haptic system [9]; real-time mechanical property analysis [391]—e.g., using a hierarchical finite element method (FEM) from [203]; reverse engineering and shape digitizing [388, 389]—e.g., using haptic-guided volume sculpting method from [188]; and a module called HVCMM [65, 70, 390, 363, 367] for inspection path generation for coordinate measurement machines (CMM). The platform uses their own volumetric enumeration data structure called spatial run-length encoding (S-RLE) [66] for geometric rasterization and haptic rendering. The group’s subsequent works also include haptic-guided repair of triangular meshes (e.g., hole-filling) [160], haptic-assisted evaluation of compliant mechanisms [339], and surface texture and friction modeling for tactile feedback [64]. A Phantom Desktop device (6 DOF input, 3 DOF output) was used for all applications.

Gao et al. form the University of Hong Kong and Gibson et al. from the National University of Singapore developed a haptic sketching system for manipulating 3D B-spline curves [122] and a haptic sculpting system [123, 124, 125] to create and modify B-spline surfaces using a variety of prob/tool head geometries. The sculpting system enables intuitive pushing and pulling operations on freeform surfaces to relate the virtual modeling experience to the physical world experience. The model uses a mass-spring discretization of the surface checked against the implicit representation of the prob profile for elastic collision response. The implementation also enables wavelet-based multiresolution representations, which in turn enables sweep editing and 3D texture reuse in the frequency domain [125]. The system was later added with functionalities to design and evaluate multimaterial products [121] and to work with point clouds and NURBS [126]. A Phantom Premium device (6 DOF input, 6 DOF output) was used for all applications.

The VENISE research team (Bourdot et al.) at the CNRS/LIMSI laboratory in partnership with Universite Paris-Sud developed an integrated and immersive VR-CAD system called VRAD [46] that supports modification of CAD semantics directly within the VE. The system enables direct modification of construction history graphs (CHG) by selection of primitive elements on the evaluated B-reps [86]. The haptic functionality [286] was added using a variety of force models—namely, spring elasticity and attraction force models for haptic selection [287], ‘reference-based’, ‘haptic-grid’, and graduated virtual fixtures (GVF) methods for haptic extrusion [288], and potential field approaches for haptic selection [328]. The implementation supports multimodal interactions using previous tools developed by the group such as EVI3d [47, 350] including multimodal ‘fusion’ using their own VEserver [349] concurrently running on multiple computers, and multimodal ‘fission’ for managing large and complex data in future work [49]. Among other works of the group is flexible model rendering [87]. The device brands were not specified.

The KAEMaRT research group (Cugini et al. and Bordegoni et al.) at the Politecnico di Milano developed a system called T’nD [32, 33, 36] to perform conceptual design activities using virtual tools attached to a 6DOF FCS-HapticMaster device to simulate physical form-making activities—e.g., surface scraping using rakes [34] and finishing using sandpaper [92] modeled using chip removal theory [262]. The group also developed other prototypes such as VeRVE [32, 31, 43] for ergonomic validation using haptic knobs [82], and PROGIMM [38, 39, 40] as a mixed-reality platform for validating prototypes in the automotive industry in collaboration with Caruso et al. [57, 58, 59]. These conceptual design and ergonomic validation functionalities were also integrated, in addition to aesthetic design tools [35, 41, 42] into a multimodal and multisensory system called SATIN [113, 97] using a deformable strip attached to two FCS-HapticMaster devices installed in a parallel configuration and a display system called DVHDS composed of projectors and mirrors for superimposing the virtual scene over the user’s hands.

Table 1 presents a more inclusive list (including studies and systems not described above) and a summary of their key features.

a.2 Haptic-Enabled VA

Gupta et al. at the MIT Media Lab developed one of the earliest multimodal desktop VA systems (i.e., with visual, auditory, and touch feedback) called VEDA [147, 145, 146] that used physically-based modeling (PBM) for part behavior, with the end-goal of integrating design evaluation techniques such as design for assembly (DFA) into the existing CAD systems. The shapes were limited to either convex or concave rigid 2D polygons whose contact was modeled using Coulomb’s laws of (static and dynamic) friction. Dual Phantom 1.0 devices (6 DOF input, 3 DOF output) were used. The system’s performance was evaluated for simple peg-in-hole examples.

Jayaram et al. at the Washington State University in a research partnership with Lyons and Hart from the National Institute of Standards and Technology (NIST) developed the immersive assembly planning and evaluation platform called VADE [191, 194, 196]. The system directly imported CAD models from Pro/ENGINEER and performed assembly using constraint-based modeling (CBM) [366] in addition to some basic PBM. It was further augmented with additional functionalities including interactive swept volume generation and modification using synchronous links between the VE (providing trajectory data) and the CAD (providing shape data) sub-systems [386], inclusion of hand-held tools (e.g., screwdrivers) and the corresponding assembly operations and tool/hand/part interactions [193], and distributed environments for collaborative assembly [199]. Additionally, workplace ergonomic evaluation tools—e.g., the rapid upper-limb assessment (RULA) algorithm [250]—were integrated later into the system [325, 326]. Subsequent industrial case studies [197, 192] concluded the feasibility of VA methods for deployment in the actual PLM cycle, and identified the key issues to be resolved in terms of ease of use, portability of the applications, and preparation of the evaluation models. Both one- and two-handed assembly was enabled using CyberGrasp haptic gloves (5 DOF input, 5 DOF output).

Bras et al. at the Georgia Tech. developed another early haptic-enabled assembly and disassembly simulation environments called HIDRA [252, 90, 89]. The implementation used two concurrent loops (one for graphics and one for haptics+CD) to comply with the difference in frame rate requirements. To allow for fast CD, they used University of Minnesota’s Qhull library for computing the convex hull of parts (or an obvious decomposition of the parts). For the CD itself between convex elements, Mitsubishi Electric Research Laboratory (MERL)’s V-Clip library [263] was used in the earlier versions of HIDRA [252, 90]. Later, to enable faster multibody processing, sweep generation, and prune sorting, the use of SWIFT [104] and SWIFT++ [105] was investigated in [89], where the ++ version enables nonconvex CD by decomposing into convex elements. The study discusses optimization techniques such as part anchoring, dynamic loading of haptic representations, and partitioning of part updates to minimize the effects of inherent computational limitations. The results of the study indicated limited usability of haptics for simple peg-in-hole examples. A Phantom 1.0 device (6 DOF input, 3 DOF output) was used.

Borro et al. at the CEIT research group (CEIT-IK4 research alliance as of 2004) developed the system called REVIMA [313, 44] for haptic-assisted maintainability simulation in aeronautics (e.g., assembling aircraft engine mock-ups). A specialized large haptic device called LHIfAM (6 DOF input, 3 DOF output) was developed for this purpose.

Wan et al. at the Zhejiang University developed a multimodal VE system for assembly named MIVAS [360, 399] with voice input, stereo visual feedback (CAVE-like system), sound feedback, and force feedback. To facilitate an automatic translation of the assembly model from Pro/ENGINEER CAD models, an interface was developed using Pro/TOOLKIT which automatically imports geometry and topology (for PBM) as well as assembly constraints (for CBM) into the VE. It also incorporated models of virtual hand kinematics and grasping heuristic patterns for realistic user interaction. Virtual hand/part CD for grasp feedback was carried out using RAPID [139] while fast part/part CD was implemented using VPS [361, 259, 260, 261]. CyberGrasp haptic gloves (5 DOF input, 5 DOF output) were used to enable force feedback from the virtual hand.

max width=0.95 References Year System Methods Software Hardware Key Features [147] 1995 VEDA PBM, GHOST API, Phantom 1.0, ‘vertex-edge’ pair contact modeling; [145, 146] 1997 CD. SGI Indigo. uses Coulomb’s law of friction; limited to 2D convex/concave polygons. [191] 1997 VADE PBM, Device APIs. CyberGrasp, one- & two-handed dexterous assembly; [194, 196, 386] 1999 CBM, SGI Onyx2, approximate line-polygon intersections; [193, 199] 2000 CD. InfiniteReality Pipes, maintains link between VE and CAD; [195, 364] 2001 5DT FOB OTs, imports “assembly intent” from CAD; [326, 366] 2003 Unspecified HMD, sweep generation and trajectory editing; [195, 325] 2004 ImmersaDesk. simulates fasteners and screwdrivers; [198] 2006 ergonomic evaluation tools; [192] 2007 collaborative/distributed assembly. [252] 1999 HIDRA CD. GHOST API, Phantom 1.0. CD-based assembly and disassembly; [90] 2001 OpenGL, Motif uses Qhull for convex hull generation; [89] 2002 Qhull, V-Clip, uses V-Clip and SWIFT(++) for CD; SWIFT(++). limited use to simple peg-in-holes. [313] 2002 REVIMA PBM, Device APIs, CEIT LHIfAM. maintenance of aircraft engines; [44] 2004 CD. OpenGL. accessibility and interference analysis. [360, 399] 2004 MIVAS PBM, OpenGL Optimizer, CyberGrasp, multimodal VE for grasp+assembly; CBM, Pro/TOOLKIT SGI Onyx2, direct constraint import from CAD; CD. IBM ViaVoice, CAVE-like System allows assembly sequence generation; TCP/IP. CrystalEye SG, allows assembly trajectory generation; 5DT FOB. optimization techniques for assembly. [319] 2005 SHARP PBM, VR Juggler, Phantom Omni, uses VPS for high-clearance CD; [320] 2006 CBM, OpenGL PSG, Barco Baron PT, uses D-Cubed for low-clearance CD; [322] 2007 CD. OpenHaptics, CAVE-like System, swept volumes for maintainability; [321] 2008 VPS, D-Cubed, Unspecified HMD. capable of creating subassemblies; [323] 2010 TCP/IP. capable of networking with others. [227, 228] 2006 HAM(M)S PBM, OpenHaptics, Phantom Omni, TCT evaluations for performance; [229, 231] 2007 CD. VTK Toolkit, Phantom Desktop. EMG evaluations for motor control; [230] 2010 PhysX SDK, MTL & therblig analysis of motion; [136] 2013 Bullet Physics, multiple PSEs for PBM+CD; [138, 137, 135] 2014 MFC library. assembly process modeling & planning. [184, 183] 2006 (C)HAS CBM, OpenHaptics, Phantom Omni, distributed & collaborative assembly; [186] 2006 CD. Labein’s DATum, Phantom Premium, combines assembly & haptic simulators; [185] 2006 RAPID. PERCRO GRAB. tested on an aeronautical assembly. [178] 2007 CVE PBM, VirtuoseAPI, Virtuose6D 35-45, collaborative VE; modular behavior; [179] 2008 CBM, VTK Toolkit, 2D Wall Display, automatic contact constraint detection; [180] 2011 CD. ODE. Crystal Eye SG, mobility trajectory characterization; [182] 2012 Christie HD3 SP. classifies simple ‘functional surfaces’; [181] 2013 uses dual-quaternion representation. [26] 2008 HIIVR PBM, Device APIs, Phantom Omni, used for procedural skills development; [27, 200] 2009 CD. SmartCollisions. 5DT FOB & DGs, different difficulty levels for training; eMagin Z800 HMD, assists the user by visual cues; NEC SP. evaluated using SE & PVE scales; [78] 2008 CBM, CHAI3D, Phantom Omni. optimization of path & sequence; [79] 2009 CD. AABB CD for 2D polygonal shapes; [77] 2011 improved assembly (time & distance). [127, 393] 2009 VEDAP-II PBM, VHT Toolkit, CyberGlove, models grasp+move+locate+secure; CD. PhysX SDK, CyberGrasp, focuses on virtual coupling dynamics; OpenGL, V-Clip. CyberForce, models ‘visual dynamic behavior’. [37] 2009 PBM, VirtuoseAPI, Virtuose6D 35-45, evaluation of two-handed assembly; CD. Cyviz Viz3D. WiiRemote, heuristic criteria for quality assessment; Dassault Virtools, Unspecified OTs. low- vs. high-cost device assessment. ARTrack, IPP. real-scale projection and tracking. [333] 2009 CD. Novint SDK, Novint Falcon telepresence for in-space assembly; uses ‘virtual walls/boundaries’ for CD; tested on NASA’s SPHERES testbed.

Table 2: A chronological review of haptic-enabled virtual prototyping platforms for assembly and disassembly tasks.

max width=0.95 References Year System Methods Software Hardware Key Features [221] 2009 PBM, VirtuoseAPI, Virtuose6D 35-45, PRM PP methods (A-star, RDT/RRT); [220] 2010 PP, guides parts along ‘following zones’; CD. updates the path when user deviates; [397] 2009 DPVAE PBM, Device APIs, CyberGlove, collaborative assembly environment; [382] 2012 CBM, TCP/IP. CyberTouch, supports data conversion from CAD; CD. 5DT FOB & DGs. parallel rendering based on PC-cluster; ‘high-efficient’ CD and HLA/RTI. [170, 398] 2010 GCVAE PBM, Device APIs, CyberGlove, collaborative assembly environment; CBM, TCP/IP. CyberTouch, network grid‐based support platform; CD. 5DT FOB & DGs. supports large and complex scenes; [149] 2010 IMA-VR CBM, OpenHaptics, Phantom Omni, multimodal assembly training system; CD. Device APIs. CEIT LHIfAM, cognitive and motor skills transfer; PERCRO GRAB. spring-damper model for CD response. [343, 344] 2010 PBM, VirtuoseAPI. Virtuose6D 35-45, VCG method for hybrid PBM+CBM; CBM, uses nonsmooth rigid body dynamics; CD. uses virtual fixtures for DOF-limiting; [154] 2010 MAD PBM, OpenHaptics, Phantom Omni. aircraft assembly and maintenance; [155, 156] 2011 CD. OpenInventor CACO path & sequence optimization; [157] 2014 GLUT. active and passive haptic guidance. [370, 369, 130] 2011 PBM, Device APIs. Haptic bracelet, AR for maintenance & assembly; [148] 2012 CD. Generic webcam, enables skills acquisition & transfer; [368] 2013 Generic tablet. AVA and vibrotactile feedback; [129] 2015 low-cost AR training platform. [383] 2011 HVAS PBM, OpenHaptics. Phantom Premium, combined PBM+CBM for assembly; CBM, WTK Toolkit, Unspecified SG. automatic data transfer from CAD; CD. PhysX SDK. hierarchical constraint data model. [384] 2012 HITsphere PBM, OpenHaptics. Phantom Premium, motion simulator for free walking; CBM, WTK Toolkit, Cybersphere System, combined PBM+CBM for assembly; CD. PhysX SDK, Unspecified SG. automatic data transfer from CAD; TechViz XL. hierarchical constraint data model. [270] 2013 Poster PBM, PTAMM, ODE, HapticGEAR, wearable backpack-type haptic device; CD. ARToolKit. Unspecified HMD, markerless AR with large workspace. [279] 2013 Snap-to-Fit PBM, H3DAPI, Unspecified Device, point-to-point attraction force model; CD. applied to facial surgery & archaeology. [17, 16] 2014 PBM, OpenHaptics, Phantom Omni, unified CD and geometric guidance; [20, 19] 2015 CD, VirtuoseAPI, Virtuose6D 35-45. automatic GE for arbitrary geometry; [21] 2016 GE. OpenGL, Havoc3D. does not depend on CAD constraints; GLUT, Win32 API. does not scale with syntactic size;

  • Abbreviations used for generic terms: physically-based modeling (PBM), constraint-based modeling (CBM), collision detection (CD), virtual environment (VE), virtual reality (VR, augmented reality (AR), computer-aided design (CAD), application programmer’s interface (API), software development kit (SDK), transmission control protocol (TCP), user datagram protocol (UDP), internet protocol (IP), optical see-through (OST), head-mounted display (HMD), boundary representation (B-rep), shutter glasses (SG), stereo glasses (SG), projection table (PT), task completion time (TCT), electromyography (EMG), motion timeline (MTL), physics simulation engine (PSE), data glove (DG), stereo projector (SP), self-efficacy (SE), perceived virtual environment (PVE), axis-aligned bounding box (AABB), optical tracker (OT), probabilistic roadmap (PRM), path planning (PP), rapidly-growing deterministic trees (RDT), and rapidly-exploring random trees (RRT), high level architecture (HLA), runtime infrastructure (RTI), virtual constraint guidance (VCG), degrees of freedom (DOF), combined ant colony optimization (CACO), adaptive visual aids (AVA), geometric energies (GE).

  • Abbreviations used for brand names: virtual environment for design for assembly (VEDA), general haptic open software toolkit (GHOST), Silicon Graphics, Inc. (SGI), virtual assembly design environment (VADE), Flock of Birds (FOB), haptic integrated dis/reassembly analysis (HIDRA), open graphics library (OpenGL), Voronoi clip (V-Clip), speedy walking via improved feature testing (SWIFT), realidad virtual para el estudio de mantenibilidad en sistemas aeronàuticos (REVIMA), large haptic interface for aeronautic maintainability (LHIfAM), multimodal immersive virtual assembly system (MIVAS), cave automatic virtual environment (CAVE), system for haptic assembly and realistic prototyping (SHARP), Voxmap PointShell (VPS), haptic assembly, manufacturing, and machining system (HAMS), haptic assembly and manufacturing system (HAMS), visualization toolkit (VTK), Microsoft Foundation Class (MFC), perceptual robotics (PERCRO), haptic assembly simulator (HAS), collaborative haptic assembly simulator (CHAS), open/object dynamics engine (ODE), haptically enabled interactive and immersive virtual reality (HIIVR), computer haptics and active interfaces 3D (CHAI3D), virtual environment for the design and assembly planning (VEDAP), virtual human toolkit (VHT), interactive physics pack (IPP), National Aeronautics and Space Administration (NASA), synchronized position hold, engage, reorient, experimental satellites (SPHERES), distributed parallel virtual assembly environment (DPVAE), grid‐enabled collaborative virtual assembly environment (GCVAE), industrial maintenance and assembly with virtual reality (IMA-VR), maintenance assembly/disassembly (MAD), graphics library utility toolkit (GLUT), haptic-based virtual assembly system (HVAS), world toolkit (WTK), parallel tracking and multiple mapping (PTAMM).

  • means two-handed haptic device or a pair of one-handed devices.

Vance et al. at the Iowa State University developed a series of VA tools with and without haptic support: Johnson and Vance developed VEGAS [202], an assembly simulator that used Boeing Corp.’s Voxmap PointShell (VPS) library [361, 259, 260, 261] for CD between high-polygon parts without PBM for haptic feedback. Kim and Vance investigated different CD and part behavior algorithms [211, 212] and modified VEGAS to include PBM but no haptics. Kim and Vance also developed NHE [213] to facilitate collaborative assembly over the web using a combination of peer-to-peer and client-server models. Howard and Vance [169] developed a prototype desktop system for haptic assembly using PBM. From the same group, Seth et al. later developed SHARP [319, 320, 321, 322, 323] which expanded the VA functionality to include dual-handed haptics, swept volume representation, subassembly modeling, and more realistic part behavior via PBM. The earlier versions of SHARP [319, 320, 321] used VPS for approximate CD between a ‘voxmap’ representation of an stationary part and a ‘pointshell’ representation of the moving part, which was inadequate for low-clearance assembly. The subsequent versions of SHARP [322, 323] implemented a combination of PBM and CBM using the automatic geometric constraints (AGC) method [355]; it used D-Cubed’s CDM module for exact CD between original CAD B-rep data and D-Cubed’s DCM module for constraint management. Faas and Vance [110] proposed a hybrid method to tie the B-rep and voxel representations for simultaneous collision response and constraint-based guidance. Dual Phantom Omni devices (6 DOF input, 3 DOF output) were used.

Cheng-Jun et al. at the Qingdao Technological University proposed the use of dynamically constructed oriented bounding box (OBB) tree-based CD [139] for haptic assembly [72]. They also proposed a ‘increment-along-constraint’ (IAC) method [71] to solve the separation problem between the haptic proxy and the parts under constrained motion. A Phantom Desktop device (6 DOF input, 3 DOF output) was used, for which the specialized OBB and IAC algorithms were implemented.

Lim et al. at the Heriot-Watt University developed a system called HAM(M)S [227, 228, 229, 230, 231, 136, 138] as a testbed to investigate and measure user interactions and response while performing various engineering tasks in a haptic-enabled VE including assembly.242424The system was called haptic assembly, manufacturing, and machining system (HAMMS) in the earlier publications [227, 136]. The authors chose to simplify the system’s name to haptic assembly and manufacturing system (HAMS) in a subsequent paper [138]. Different physics simulation engines were utilized; namely Ageia PhysX SDK [60] was used in the earlier versions [229] and Bullet Physics SDK [309] was added into the later versions [136, 138], whose pros and cons were evaluated and compared in [137, 135]. They conducted peg-in-hole assembly experiments (using insertion routines with and without chamfers) on both real and virtual setups for comparison in terms of task completion times (TCT) [231] and motor control measured via muscle Electromyography (EMG) [230]. Further investigation of the user-object interaction was made with the objective of assembly plan generation by analyzing chronocycle-graph motion timelines (MTL) and “therblig” units [336] for both peg-in-hole examples and a more realistic pump assembly [300, 301, 302]. Phantom Omni and Desktop devices (6 DOF input, 3 DOF output) were used interchangeably. The group also developed a cable harness VR system called COSTAR [335, 336, 337] (without haptic feedback) for user-logging in order to analyze the design process, capture design knowledge, and produce assembly plans.

The Fundación LABEIN research team (Iglesias et al.) at Tecnalia developed the distributed haptic assembly system HAS [183] using their own geometric modeler DATum for creation of 3D virtual scenes from CAD models. An assembly simulator was developed based on RAPID [139] for CD and simple semantics for constraint detection and enforcement; namely, along co-axial axes and co-planar planes. However, it did not include real-time physics-based simulation. The collaborative extension called CHAS [184, 186, 185] using a peer-to-peer architecture and specific consistency maintenance schemes was developed later and tested on an aeronautical assembly test-case. Phantom Omni and Premium devices (6 DOF input, 36 DOF output) and a PERCO GRAB device providing two points of contact (4 DOF input, 4 DOF output) were used.

Christiand et al. at the Gyeongsang National University developed an assembly simulation system [77, 78, 79]

with haptic guidance along an optimized path. The sequence identification and path planning were carried out offline using an optimization genetic algorithm (GA) that took into account the part geometries and gripper data. Once a sequence was identified and a path was generated for each part in the sequence, a potential field method was used in real-time to combine repulsive and attractive forces to avoid obstacles and guide each part to its final position along the known path. The implementation was limited to the assembly of 2D polygonal parts whose collisions were detected using an axis-aligned bounding box (AABB) method. The experiments concluded better result with haptic guidance compared to those without haptics, both in terms of assembly time and travel distance. Hassan and Yoon from the same group developed the assembly and disassembly maintenance system called

MAD [154]. The system imported CAD files from CATIA using OpenInventor API and performs sequence and path planning using parallel GA-based planners. Given the initial and final configurations, the GA minimizes the number of required gripper exchanges and orientation changes to reduce the assembly time according to the findings in [282]. To accommodate the geometric complexity and high-DOF that led to the failure of GA, Hassan and Yoon later developed a two-stage combined ant colony optimization (CACO) algorithm [156], which performed sequence optimization in the first stage followed by path planning in the second stage. The CACO algorithm was introduced to MAD [155] to investigate the effects of active and passive haptic guidance to improve the user performance. A Phantom Omni device (6 DOF input, 3 DOF output) was used for all experiments.

The KAEMaRT research group (Cugini et al. and Bordegoni et al.) at the Politecnico di Milano evaluated haptic-assisted manual assembly in a mixed reality environment for the case study of grabbing, holding, and positioning pairs of mechanical components relative to each other [37]. A Virtuose6D 35-45 (6 DOF input, 6 DOF output) and a Nintendo WiiRemote were used together in a two-handed setup. They used a set of heuristic criteria for assessing the quality of the application and identifying usability problems to be fixed in future studies.

Ladeveze et al. at the Université de Toulouse developed a system for haptic assembly and disassembly task assistance [220, 221] using probabilistic roadmap (PRM) path planners—namely, tools such as the A-star algorithm, rapidly-growing deterministic trees (RDT), and rapidly-exploring random trees (RRT) [4]. The planner first identified a collision free path in the 6D configuration space of a rigid part. The haptic control loop then used that information to guide the user into and along a so-called ‘following zone’ formed around the free path, which was discretized for simplifying and stabilizing force and torque computation. A Virtuose6D 35-45 device (6 DOF input, 6 DOF output) was used.

Wu et al. at the Shanghai Jiao Tong University developed a grid-based VA server called GVAS [398] for large and complex (e.g., automotive and ship) assemblies, which used parallel computing and network resources for demanding VA computations such as model rendering, image processing (i.e., fusion), and CD. To fulfill security requirements, product data was managed independently using the concept of role-based access control (RBAC). The group later developed two systems for automobile VA; namely: 1) a grid-based collbaorative VA environment called GCVAE [170], which comprised of a grid-based support platform, a service-based parallel rendering framework, and a multi-user collaborative VA environment; and 2) a distributed parallel VA environment called DPVAE [397, 382], which used high-level architecture and runtime infrastructure (HLA/RTI) event synchronization mechanisms. The systems made extensive use of parallel processing to enable complex assembly scenarios with intensive rendering and CD requirements at multiple levels of detail (LOD). CyberTouch haptic gloves (with small vibrotactile feedback) were used.

Tching et al. at the IRISA–Bunraku developed the virtual constraint guidance (VCG) method [343, 344] for haptic guidance. The method decomposes a task into 1) a guiding step which used virtual fixtures [304] to guide the objects into position; and 2) a functional step which used kinematic constraints via mechanical joints to restrict the DOF for insertion. The idea is to use both PBM-based exploration of the VE (using nonsmooth dynamics [341]) and CBM-based execution of fine insertion (using virtual fixtures [304]) while CD is locally deactivated. The method has proved very effective for peg-in-hole test-cases with simple contact constraints (i.e., lower pairs) whose virtual fixture abstractions and mechanical joint equivalents are obvious. The constraints were extracted from CAD mating pair semantics in a preprocessing step. A Virtuose6D 35-45 (6 DOF input, 6 DOF output) was used.

Xia et al. at the University of Porto developed a multithreaded haptic assembly system called HVAS [383]. The system had an automatic data integration interface to transfer geometry, topology, assembly, and physics information from CAD systems (e.g., Pro/ENGINEER and SolidWorks) to the VE. A hierarchical constraint-based data model and scene-graph structure was designed to construct the VA environment. The system employed a combined PBM and CBM approach for haptic guidance; namely, a spring-damper model for collision response and dynamic simulation when parts penetrate into each other, and a geometric guidance model to generate attractive and repulsive forces to guide the mating parts that are close to each other. The architecture was embedded into another system called HITsphere [384] for VA training which, unlike the traditional desktop or CAVE-like systems, also enabled natural human walking motion—similar to the Cybersphere system [111]—using a low-cost motion simulator. Haptic feedback was enabled by a Phantom Premium device (6 DOF input, 6 DOF output).

The G-SCOP group (Noël et al.) at the Grenoble Institute of Technology and Iacob et al. at the University Politehnica of Bucharest developed a collaborative VE called CVE [351, 289] for design and assembly activities, which contained several modules—such as viewer, recorder, editor, object dynamics engine (ODE), analysis, and haptic models—integrated together for different sub-behaviors. The system performed automatic constraint detection using a contact identification process between ‘functional surfaces’—restricted to planar, cylindrical, spherical, and conical surfaces—between CAD models (exported in STEP format) [178, 179, 180, 181, 182]. The system was evaluated for usability, efficiency, user experience, and feedback quality. A stereoscopic display and a Virtuose6D 35-45 device (6 DOF input, 6 DOF output) was used for haptic feedback.

Table 2 presents a more inclusive list (including studies and systems not described above) and a summary of their key features.


  • [1] ASME Y14.5-2009: Dimensioning and tolerancing: Engineering drawing and related documentation practices. An international standard, American Society of Mechaninical Engineers (ASME), 2009.
  • [2] R. J. Adams and B. Hannaford. Stable haptic interaction with virtual environments. IEEE Transactions on Robotics and Automation, 15(3):465–474, 1999.
  • [3] R. J. Adams and B. Hannaford. Control law design for haptic interfaces to virtual reality. IEEE Transactions on Control Systems Technology, 10(1):3–13, 2002.
  • [4] I. Aguinaga, D. Borro, and L. Matey. Parallel RRT-based path planning for selective disassembly planning. The International Journal of Advanced Manufacturing Technology, 36(11):1221–1233, 2007.
  • [5] C. Alexander. Notes on the Synthesis of Form. Harvard University Press, 1964.
  • [6] M. Anitescu, F. A. Potra, and D. E. Stewart. Time-stepping for three-dimensional rigid body dynamics. Computer Methods in Applied Mechanics and Engineering, 177(3-4):183–197, 1999.
  • [7] C. L. Bajaj, B. Bauer, R. Bettadapura, and A. Vollrath. Nonuniform Fourier transforms for rigid-body and multidimensional rotational correlations. SIAM Journal on Scientific Computing, 35(4):B821–B845, 2013.
  • [8] C. L. Bajaj, R. Chowdhury, and V. Siddahanavalli. F2Dock: Fast Fourier protein-protein docking. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 8(1):45–58, 2011.
  • [9] M. Balasubramaniam, S. Ho, S. Sarma, and Y. Adachi. Generation of collision-free 5-axis tool paths using a haptic surface. Computer-Aided Design, 34(4):267–279, 2002.
  • [10] D. Baraff. Fast contact force computation for nonpenetrating rigid bodies. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, pages 23–34, New York, NY, USA, 1994.
  • [11] D. Baraff. Interactive simulation of solid rigid bodies. Journal of Computer Graphics and Applications, 15(3):63–75, 1995.
  • [12] J. Barbič and D. James. Time-critical distributed contact for 6-DOF haptic rendering of adaptively sampled reduced deformable models. In Proceedings of the ACM SIGGRAPH’2007/Eurographics Symposium on Computer Animation, pages 171–180, Aire-la-Ville, Switzerland, Switzerland, 2007.
  • [13] J. Barbič and D. L. James. Six-DoF haptic rendering of contact between geometrically complex reduced deformable models. IEEE Transactions on Haptics, 1(1):39–52, 2008.
  • [14] M. Behandish. Analytic Methods for Geometric Modeling. Ph.d. dissertation, 2016.
  • [15] M. Behandish. Geometric Energies for Haptic Assembly. M.sc. thesis, 2016.
  • [16] M. Behandish and H. T. Ilieş. Peg-in-hole revisited: A generic force model for haptic assembly. In Proceedings of the 2014 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2014), number DETC2014-35290, pages V01BT02A041:1–10. American Society of Mechanical Engineers (ASME), 2014.
  • [17] M. Behandish and H. T. Ilieş. Shape complementarity analysis for objects of arbitrary shape. Technical Report CDL-TR-14-01, University of Connecticut, 2014.
  • [18] M. Behandish and H. T. Ilieş. Analytic methods for geometric modeling via spherical decomposition. Computer-Aided Design, 70:100–115, 2015. 2015 SIAM/ACM Symposium on Solid and Physical Modling (GD/SPM’2015).
  • [19] M. Behandish and H. T. Ilieş. Haptic assembly using skeletal densities and Fourier transforms. In Proceedings of the 2015 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2015), number DETC2015-47923, pages V01BT02A060:1–11. American Society of Mechanical Engineers (ASME), 2015.
  • [20] M. Behandish and H. T. Ilieş. Peg-in-hole revisited: A generic force model for haptic assembly. Journal of Computing and Information Science in Engineering (JCISE), 15(4):041004:1–11, 2015.
  • [21] M. Behandish and H. T. Ilieş. Haptic assembly using skeletal densities and Fourier transforms. Journal of Computing and Information Science in Engineering (JCISE), 16(2):021002:1–11, 2016.
  • [22] Jan Bender, Kenny Erleben, and Jeff Trinkle. Interactive simulation of rigid body dynamics in computer graphics. Computer Graphics Forum, 33(1):246–270, 2014.
  • [23] G. R. Bennett. The application of virtual prototyping in the development of complex aerospace products. Aircraft Engineering and Aerospace Technology, 69(1):19–25, 1997.
  • [24] G. van den Bergen. Efficient collision detection of complex deformable models using AABB trees. Journal of Graphics Tools, 2(4):1–13, 1997.
  • [25] A. Bernard and A. Fischer. New trends in rapid product development. CIRP Annals - Manufacturing Technology, 51(2):635–652, 2002.
  • [26] A. Bhatti, Y. B. Khoo, D. Creighton, J. Anticev, S. Nahavandi, and M. Zhou. Haptically enabled interactive virtual reality prototype for general assembly. In Proceedings of the 2008 World Automation Congress (WAC’2008), pages 1–6, 2008.
  • [27] A. Bhatti, S. Nahavandi, Y. B. Khoo, D. Creighton, J. Anticev, and M. Zhou. Haptically enable interactive virtual assembly training system development and evaluation. In Proceedings of the 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (SIMTECT’2009), pages 1–6. SIMTECT, 2009.
  • [28] V. Boor, M. H. Overmars, and A. F. van der Stappen. The Gaussian sampling strategy for probabilistic roadmap planners. In Proceedings of the 1999 IEEE International Conference on Robotics and Automation, volume 2, pages 1018–1023. IEEE, 1999.
  • [29] G. Boothroyd and P. Dewhurst. Product Design for Assembly. Boothroyd Dewhurst Incorporated, 1987.
  • [30] M. Bordegoni. Deliverable D1 hardware and software technology and cognitive ergonomics update. Technical Report at KAEMaRT group, Politecnico di Milano, 2004.
  • [31] M. Bordegoni, G. Caruso, F. Ferrise, and . Giraudo. A mixed environment for ergonomics tests: Tuning of stereo viewing parameters. In Proceedings of the 5th Eurographics Italian Chapter Conference. The Eurographics Association, 2007.
  • [32] M. Bordegoni, G. Colombo, and Formentini. Haptic technologies for the conceptual and validation phases of product design. Computers and Graphics, 30(3):377–390, 2006.
  • [33] M. Bordegoni and U. Cugini. A conceptual design tool based on innovative haptic interfaces. In Proceedings of the 2006 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2006), pages 897–904. American Society of Mechanical Engineers, 2006.
  • [34] M. Bordegoni and U. Cugini. Haptic modeling in the conceptual phases of product design. Virtual Reality, 9(2-3):192–202, 2006.
  • [35] M. Bordegoni and U. Cugini. The role of haptic technology in the development of aesthetic driven products. Journal of Computing and Information Science in Engineering (JCISE), 8(4):041006, 2008.
  • [36] M. Bordegoni and U. Cugini. A CAD system based on haptic modelling for conceptual design. International Journal of Product Development, 11(1-2):25–46, 2010.
  • [37] M. Bordegoni, U. Cugini, P. Belluco, and M. Aliverti. Evaluation of a haptic-based interaction system for virtual manual assembly. In Randall Shumaker, editor, Virtual and Mixed Reality, volume 5622 of Lecture Notes in Computer Science, pages 303–312. Springer Berlin Heidelberg, 2009.
  • [38] M. Bordegoni, U. Cugini, G. Caruso, and S. Polistina. Mixed prototyping for rapid design review of information appliances. In Proceedings of IDMME’2008 Virtual Concept, Beijing, China, 2008.
  • [39] M. Bordegoni, U. Cugini, G. Caruso, and S. Polistina. Mixed prototyping for product assessment: A reference framework. International Journal on Interactive Design and Manufacturing (IJIDeM), 3(3):177–187, 2009.
  • [40] M. Bordegoni, U. Cugini, G. Caruso, and S. Polistina. The role of mixed prototyping in product design assessment. In Proceedings of the 2nd International Conference on Research into Design (ICoRD’2009), Beijing, China, 2009.
  • [41] M. Bordegoni, U. Cugini, and F. Ferrise. Analysis of designers manual skills for the development of a tool for aesthetic shapes evaluation. In Proceedings of IDMME’2010 Virtual Concept, 2010.
  • [42] M. Bordegoni, U. Cugini, and F. Ferrise. Requirements for an enactive tool to support skilled designers in aesthetic surfaces definition. International Journal on Interactive Design and Manufacturing (IJIDeM), 6(2):83–91, 2012.
  • [43] M. Bordegoni, U. Giraudo, G. Caruso, and F. Ferrise. Ergonomic interactive testing in a mixed-reality environment. In R. Shumaker, editor, Virtual Reality, volume 4563 of Lecture Notes in Computer Science, pages 431–440. Springer Berlin Heidelberg, 2007.
  • [44] D. Borro, J. Savall, A. Amundarain, J. J. Gil, A. Garcia-Alonso, and L. Matey. A large haptic device for aircraft engine maintainability. Computer Graphics and Applications, 24(6):70–74, 2004.
  • [45] P. Bosscher and I. Ebert-Uphoff. Digital clay: architecture designs for shape-generating mechanisms. In Proceedings of the 2003 IEEE International Conference on Robotics and Automation (ICRA’2003), volume 1, pages 834–841, 2003.
  • [46] P. Bourdot, T. Convard, F. Picon, M. Ammi, D. Touraine, and J. M. Vezien. VR-CAD integration: Multimodal immersive interaction and advanced haptic paradigms for implicit edition of CAD models. Computer-Aided Design, 42(5):445–461, 2010.
  • [47] P. Bourdot, M. Krus, and R. Gherbi. Cooperation between reactive 3D objects and a multimodal X window kernel for CAD. In H. Bunt, R. J. Beun, and T. Borghuis, editors, Multimodal Human-Computer Communication, volume 1374 of Lecture Notes in Computer Science, pages 188–212. Springer Berlin Heidelberg, 1998.
  • [48] F. Boussuge, J. C. Léon, S. Hahmann, and L. Fine. An analysis of DMU transformation requirements for structural assembly simulations. In Proceedings of the 8th International Conference on Engineering Computational Technology, 2012.
  • [49] G. Bouyer, P. Bourdot, and M. Ammi. Supervision of task-oriented multimodal rendering for VR applications. In Proceedings of the 13th Eurographics Conference on Virtual Environments (EGVE’2007), pages 93–100. Eurographics Association, 2007.
  • [50] D. Bowman, D. Johnson, and L. Hodges. Testbed evaluation of virtual environment interaction techniques. Presence, 10(1):75–95, 2001.
  • [51] G. Bradshaw and C. O’Sullivan. Adaptive medial-axis approximation for sphere-tree construction. ACM Transactions on Graphics (TOG), 23(1):1–26, 2004.
  • [52] R. Bridson, R. Fedkiw, and J. Anderson. Robust treatment of collisions, contact and friction for cloth animation. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pages 594–603, New York, NY, USA, 2002.
  • [53] R. Bridson, R. Fedkiw, and J. Anderson. Robust treatment of collisions, contact and friction for cloth animation. ACM Transactions on Graphics (TOG), 21(3):594–603, 2002.
  • [54] H. J. Bullinger, R. Breining, and W. Bauer. Virtual prototyping–state of the art in product design. In 26th International Conference on Computers and Industrial Engineering, pages 103–107, 1999.
  • [55] G. C. Burdea, G. C. Burdea, and C. Burdea. Force and Touch Feedback for Virtual Reality. Wiley New York, 1996.
  • [56] W. Carlson. A critical history of computer graphics and animation. Online course, Ohio State University, 1999.
  • [57] G. Caruso and P. Belluco. Robotic arm for car dashboard layout assessment in mixed reality environment. In Proceedings of the 2010 IEEE International Conference on Robot and Human Interactive Communication (RO-MAN’2010), pages 62–68, 2010.
  • [58] G. Caruso, S. Polistina, and M. Bordegoni. Collaborative mixed-reality environment to support the industrial product development. In Proceedings af the 2011 ASME World Conference on Innovative Virtual Reality, pages 207–215. American Society of Mechanical Engineers, 2011.
  • [59] G. Caruso, S. Polistina, M. Bordegoni, and M. Aliverti. Collaborative mixed-reality platform for the design assessment of cars interior. In R. Shumaker, editor, Virtual and Mixed Reality - Systems and Applications, volume 6774 of Lecture Notes in Computer Science, pages 299–308. Springer Berlin Heidelberg, 2011.
  • [60] L. S. H. Chan and K. S. Choi. Integrating PhysX and OpenHaptics: Efficient force feedback generation using physics engine and haptic devices. In Proceedings of the 2009 Joint Conferences on Pervasive Computing (JCPC’2009),, pages 853–858, 2009.
  • [61] J. W. Chang, W. Wang, and M. S. Kim. Efficient collision detection using a dual OBB-sphere bounding volume hierarchy. Computer-Aided Design, 42(1):50–57, 2010.
  • [62] Y. S. Chang, K. T. McDonnell, and H. Qin. A new solid subdivision scheme based on box splines. In Proceedings of the Seventh ACM Symposium on Solid Modeling and Applications (SMA’2002), pages 226–233, 2002.
  • [63] R. Chen and Z. Weng. A novel shape complementarity scoring function for protein-protein docking. Proteins: Structure, Function, and Bioinformatics, 51(3):397–408, 2003.
  • [64] Y. H. Chen, L. L. Lian, and X. J. He. Haptic rendering of three-dimensional heterogeneous fine surface features. Computer-Aided Design and Applications, 5(1-4):1–16, 2008.
  • [65] Y. H. Chen, Y. Z. Wang, and Z. Y. Yang. Towards a haptic virtual coordinate measuring machine. International Journal of Machine Tools and Manufacture, 44(10):1009–1017, 2004.
  • [66] Y. H. Chen and Z. Y. Yang. Haptic rendering based on spatial run-length encoding. Robotics and Computer-Integrated Manufacturing, 20(3):237–246, 2004.
  • [67] Y. H. Chen and Z. Y. Yang. Haptic modeling as an integrating tool for product development. The International Journal of Advanced Manufacturing Technology, 33(7-8):635–642, 2007.
  • [68] Y. H. Chen, Z. Y. Yang, and L. Lian. Haptic modeling in rapid product development. Computer-Aided Design and Applications, 1(1-4):577–584, 2004.
  • [69] Y. H. Chen, Z. Y. Yang, and L. Lian. On the development of a haptic system for rapid product development. Computer-Aided Design, 37(5):559–569, 2005.
  • [70] Y. H. Chen, Z. Y. Yang, and y. z. Wang. Haptic modeling for a virtual coordinate measuring machine. International Journal of Production Research, 43(9):1861–1878, 2005.
  • [71] C. Cheng-Jun, W. Yun-feng, and L. Niu. Research on interaction for virtual assembly system with force feedback. In Proceedings of the 2010 International Conference on Information and Computing (ICIC’2010), volume 2, pages 147–150, 2010.
  • [72] Chen Cheng-Jun, A.Y.C. Nee, S.K. Ong, Zhou Yi-qi, and Qu Bin. An improved haptic interface for virtual assembly operations. In Second Workshop on Digital Media and its Application in Museum Heritages, pages 410–415, Dec 2007.
  • [73] D. G. Cheshire, M. A. Evans, and C. J. Dean. Haptic modeling - an alternative industrial design methodology? In Proceedings of Eurohaptics 2001, volume 2001, pages 124–128, 2001.
  • [74] G. S. Chirikjian and A. B. Kyatkin. Engineering Applications of Noncommutative Harmonic Analysis: with Emphasis on Rotation and Motion Groups. CRC Press, 1st edition, 2010.
  • [75] H. I. S. Choi and H. Z. Tan. Effect of update rate on perceived instability of virtual haptic texture. In Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2004), volume 4, pages 3577–3582, 2004.
  • [76] H. I. S. Choi and H. Z. Tan. Discrimination of virtual haptic textures rendered with different update rates. In Proceedings of the 2005 Joint Conference on World Haptics, the Eurohaptics 2005, and the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 114–119, 2005.
  • [77] Christiand. Assembly simulations in virtual environments with optimized haptic path and sequence. Robotics and Computer-Integrated Manufacturing, 27(2):306–317, 2011.
  • [78] Christiand and J. Yoon. A novel optimal assembly algorithm for the haptic interface application of a virtual maintenance system. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation (ICRA’2008), pages 3612–3617, 2008.
  • [79] Christiand, J. Yoon, M. Auralius, and Wonpil Y. An enhanced haptic assembly simulation system for the efficiency of assembly tasks. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2009), pages 1241–1246, 2009.
  • [80] J. D. Cohen, M. C. Lin, D. Manocha, and M. Ponamgi. I-COLLIDE: An interactive and exact collision detection system for large-scale environments. In Proceedings of the 1995 Symposium on Interactive 3D Graphics (I3D’1995), New York, NY, USA, 1995.
  • [81] J. E. Colgate, M. C. Stanley, and J. M. Brown. Issues in the haptic display of tool use. In Proceedings of the 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 3, pages 140–145, 1995.
  • [82] G. Colombo, F. De Angelis, and L. Formentini. A mixed virtual reality-haptic approach to carry out ergonomics test on virtual prototypes. In Proceedings of TMCE’2006, pages 18–22, 2006.
  • [83] M. B. Colton and J. Hollerbach. Automated modeling of nonlinear mechanisms for virtual prototyping. In ASME 2004 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pages 1349–1355. American Society of Mechanical Engineers, 2004.
  • [84] M. B. Colton and J. M. Hollerbach. Identification of nonlinear passive devices for haptic simulations. In Proceedings on the 1st Joint Eurohaptics Conference 2005, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 2005, and World Haptics 2005, pages 363–368, 2005.
  • [85] P G. Comba. A procedure for detecting intersections of three-dimensional objects. The ACM Journal, 15(3):354–366, 1968.
  • [86] T. Convard and P. Bourdot. History-based reactive objects for immersive CAD. In Proceedings of the 9th ACM Symposium on Solid Modeling and Applications, pages 291–296. Eurographics Association, 2004.
  • [87] T. Convard, P. Bourdot, and J. M. Vezien. Managing deformable objects in cluster rendering. In V. S. Sunderam, G. D. van Albada, P. M. A. Sloot, and J. J. Dongarra, editors, Computational Science (ICCS’2005), volume 3515 of Lecture Notes in Computer Science, pages 290–297. Springer Berlin Heidelberg, 2005.
  • [88] Tukey J. Cooley, J. W. An algorithm for the machine calculation of complex Fourier series. Mathematics of CSomputation, 19(90):297–301, 1965.
  • [89] A. S. Coutee and Bras. Collision detection for virtual objects in a haptic assembly and disassembly simulation environment. In Proceedings of the 2002 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2002), pages 11–20, 2002.
  • [90] A. S. Coutee, S. D. McDermott, and B. Bras. A haptic assembly and disassembly simulation environment and associated computational load optimization techniques. Journal of Computing and Information Science in Engineering, 1(2):113–122, 2001.
  • [91] D. Crevier. AI: The tumultuous history of the search for Artificial Intelligence. Basic Books, Inc., 1993.
  • [92] U. Cugini and M. Bordegoni. Touch and design: Novel haptic interfaces for the generation of high quality surfaces for industrial design. The Visual Computer, 23(4):233–246, 2007.
  • [93] B. Curto, V. Moreno, and F. J. Blanco. A general method for C-space evaluation and its application to articulated robots. IEEE Transactions on Robotics and Automation, 18(1):24–31, 2002.
  • [94] F. Dachille IX, H. Qin, and A. E. Kaufman. A novel haptics-based interface and sculpting system for physics-based geometric design. Computer-Aided Design, 33(5):403–420, 2001.
  • [95] F. Dachille IX, H. Qin, A. E. Kaufman, and J. El-Sana. Haptic sculpting of dynamic surfaces. In Proceedings of the 1999 Symposium on Interactive 3D Graphics (I3D’1999), pages 103–110. ACM, 1999.
  • [96] F. Dai, W. Felger, T. Frühauf, M. Göbel, D. Reiners, and G. Zachmann. Virtual prototyping examples for automotive industries. volume 96, 1996.
  • [97] B. R. de Araujo, T. Guerreiro, M. J. Fonseca, J. A. Jorge, J. M. Pereira, M. Bordegoni, F. Ferrise, M. Covarrubias, and M. Antolini. An haptic-based immersive environment for shape analysis and modelling. Journal of Real-Time Image Processing, 5(2):73–90, 2010.
  • [98] T. Deviprasad and Kesavadas. Virtual prototyping of assembly components using process modeling. Journal of Manufacturing Systems, 22(1):16–27, 2003.
  • [99] J. Dingliana and C. O’Sullivan. Graceful degradation of collision handling in physically based animation. Computer Graphics Forum, 19(3):239–248, 2000.
  • [100] Y. Duan, J. Hua, and H. Qin. HapticFlow: PDE-based mesh editing with haptics: Research articles. Computer Animation Virtual Worlds, 15(3-4):193–200, 2004.
  • [101] C. Duriez, C. Andriot, and A. Kheddar. A multithreaded approach for deformable/rigid contacts with haptic feedback. In Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 272–279, 2004.
  • [102] C. Duriez, C. Andriot, and A. Kheddar. Signorini’s contact model for deformable objects in haptic simulations. In Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2004), volume 4, 2004.
  • [103] C. Duriez, F. Dubois, A. Kheddar, and C. Andriot. Realistic haptic rendering of interacting deformable objects in virtual environments. IEEE Transactions on Visualization and Computer Graphics, 12(1):36–47, 2006.
  • [104] S. A. Ehmann and M. C. Lin. Accelerated proximity queries between convex polyhedra by multi-level Voronoi marching. In Proceedings of the 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2000), volume 3, 2000.
  • [105] S. A. Ehmann and M. C. Lin. Accurate and fast proximity queries between polyhedra using convex surface decomposition. Computer Graphics Forum, 20(3):500–511, 2001.
  • [106] M. Eisenstein and E. Katchalski-Katzir. On proteins, grids, correlations, and docking. Comptes Rendus Biologies, 327(5):409–420, 2004.
  • [107] A. El Saddik. Haptics Technologies: Bringing Touch to Multimedia. Springer, 1st edition, 2011.
  • [108] M. Evans, D. Wallace, D. Cheshire, and B. Sener. An evaluation of haptic feedback modelling during industrial design practice. Design Studies, 26(5):487–508, 2005.
  • [109] M. A. Evans, D. G. Cheshire, and C. J. Dean. An investigation into the use of haptic modelling during industrial design activity. Technical report, Loughborough University, 2000.
  • [110] D. Faas and J. M. Vance. BREP identification during voxel-based collision detection for haptic manual assembly. In Proceedings of the 2011 ASME World Conference on Innovative Virtual Reality, pages 145–153. American Society of Mechanical Engineers, 2011.
  • [111] K. J. Fernandes, V. H. Raja, and J. Eyre. Immersive learning system for manufacturing industries. Computers in Industry, 51(1):31–40, 2003.
  • [112] A. Luiz S. Ferreira, Leonardo C. de M., E. Esteves C. de Miranda, and S. G. Sakamoto. A survey of interactive systems based on brain-computer interfaces. SBC Journal on Interactive Systems, 4(1):3–13, 2013.
  • [113] F. Ferrise, M. Bordegoni, and J. Lizaranzu. Product design review application based on a vision-sound-haptic interface. In S. Nordahl, R.and Serafin, F. Fontana, and S. Brewster, editors, Haptic and Audio Interaction Design, volume 6306 of Lecture Notes in Computer Science, pages 169–178. Springer Berlin Heidelberg, 2010.
  • [114] R. Fetecau, J. Marsden, M. Ortiz, and M. West. Nonsmooth Lagrangian mechanics and variational collision integrators. SIAM Journal on Applied Dynamical Systems, 2(3):381–416, 2003.
  • [115] S. Fisher and M. Lin. Deformed distance fields for simulation of non-penetrating flexible bodies. In Computer Animation and Simulation, pages 99–111. Springer Vienna, 2001.
  • [116] W. B. Fisher, E. E. Hartquist, A. A. G. Requicha, and H. B. Voelcker. The PADL-1.0/n processor: Overview & system documentation. Production Automation Project, System Document No. 01 (PADL/SD-01), University of Rochester, 1977.
  • [117] M. Frey, D. E. Johnson, and J. M. Hollerbach. Full-arm haptics in an accessibility task. In Proceedings of the 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 405–412, 2008.
  • [118] M. Frigo and S. G. Johnson. FFTW: An adaptive software architecture for the FFT. In Proceedings of THE 1998 IEEE International Conference on Acoustics Speech and Signal Processing, volume 3, pages 1381–1384, 1998.
  • [119] M. Frigo and S. G. Johnson. The design and implementation of FFTW3. Proceedings of the IEEE, Special issue on Program Generation, Optimization, and Platform Adaptation, 93(2):216–231, 2005.
  • [120] A. Fuhrmann, G. Sobotka, and C. Groß. Distance fields for rapid collision detection in physically based modeling. In Proceedings of the GraphiCon’2003, pages 58–65, 2003.
  • [121] Z. Gao, Y. Chen, and L. Lian. Haptic aided soft-touch multi-material product design. In Proceedings of 2006 IEEE International Conference on Virtual Environments, Human-Computer Interfaces, and Measurement Systems, pages 36–41, 2006.
  • [122] Z. Gao and I. Gibson. Haptics-based 3D B-spline curve sketch using multi-resolution and two-handed techniques. Proceedings of EuroHaptics 2004, 2004.
  • [123] Z. Gao and I. Gibson. Haptic B-spline surface sculpting with a shaped tool of implicit surface. Computer-Aided Design and Applications, 2(1-4):263–272, 2005.
  • [124] Z. Gao and I. Gibson. A novel haptic interface and its application in CAD. Virtual Modelling and Rapid Manufacturing, pages 225–232, 2005.
  • [125] Z. Gao and I. Gibson. Haptic sculpting of multi-resolution B-spline surfaces with shaped tools. Computer-Aided Design, 38(6):661–676, 2006.
  • [126] Z. Gao and I. Gibson. A 6-DOF haptic interface and its applications in CAD. International Journal of Computer Applications in Technology, 30(3):163–170, 2007.
  • [127] S. Garbaya and U. Zaldivar-Colado. Modeling dynamic behavior of parts in virtual assembly environment. In Proceedings of the 2009 ASME-AFM World Conference on Innovative Virtual Reality, pages 89–99, 2009.
  • [128] J. Gargus, B. Kim, J. R. Rossignac, and Christopher D. Shaw. Finger sculpting with digital clay. GVU Technical Report No. GIT-GVU-02-22, 2002.
  • [129] N. Gavish, T. Gutiérrez, S. Webel, J. Rodríguez, M. Peveri, U. Bockholt, and F. Tecchia. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments, 23(6):778–798, 2015.
  • [130] N. Gavish, T. Gutiérrez, S. Webel, J. Rodriguez, and F. Tecchia. Design guidelines for the development of virtual reality and augmented reality training systems for maintenance and assembly tasks. In Proceedings of the International Conference (SKILLS’2010), BIO Web of Conferences, volume 1, page 00029. EDP Sciences, 2011.
  • [131] R. Geraerts and M. H. Overmars. Algorithmic Foundations of Robotics V, chapter A Comparative Study of Probabilistic Roadmap Planners, pages 43–57. Springer Berlin Heidelberg, 2004.
  • [132] I. Gibson, Z. Gao, and I. Campbell. A comparative study of virtual prototyping and physical prototyping. International Journal of Manufacturing Technology and Management, 6(6):503–522, 2004.
  • [133] E. G. Gilbert and D. W. Johnson. Distance functions and their application to robot path planning in the presence of obstacles. Journal of Robotics and Automation, 1(1):21–30, 1985.
  • [134] A. Gomes de Sa and G. Zachmann. Virtual reality as a tool for verification of assembly and maintenance processes. Computers and Graphics, 23(3):389–403, 1999.
  • [135] G. Gonzalez-Badillo, H. I. Medellin-Castillo, , T. Lim, J. M. Ritchie, R. C. W. Sung, and S. Garbaya. A new methodology to evaluate the performance of physics simulation engines in haptic virtual assembly. Assembly Automation, 34(2):128–140, 2014.
  • [136] G. Gonzalez-Badillo, H. I. Medellin-Castillo, and T. Lim. Development of a haptic virtual reality system for assembly planning and evaluation. Procedia Technology, 7(0):265–272, 2013. 3rd Iberoamerican Conference on Electronics Engineering and Computer Science (CIIECC’2013).
  • [137] G. Gonzalez-Badillo, H. I. Medellin-Castillo, T. Lim, and V. E. E. López. Physics engines evaluation based on model representation analysis. In Proceedings of the 2014 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2014), 2014.
  • [138] G. Gonzalez-Badillo, H. I. Medellin-Castillo, T. Lim, J. M. Ritchie, and S. Garbaya. The development of a physics and constraint-based haptic virtual assembly system. Assembly Automation, 34(1):41–55, 2014.
  • [139] S. Gottschalk, M. C. Lin, and D. Manocha. OBBTree: A hierarchical structure for rapid interference detection. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pages 171–180, New York, NY, USA, 1996.
  • [140] N. K. Govindaraju, D. Knott, N. Jain, I. Kabul, R. Tamstorf, R. Gayle, M. C. Lin, and D. Manocha. Interactive collision detection between deformable models using chromatic decomposition. ACM Transactions on Graphics, 24(3):991–999, 2005.
  • [141] N. K. Govindaraju, S. Redon, M. C. Lin, and D. Manocha. CULLIDE: Interactive collision detection between complex models in large environments using graphics hardware. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware (HWWS’2003), pages 25–32, 2003.
  • [142] A. Gregory, M. C. Lin, S. Gottschalk, and R. Taylor. H-COLLIDE: A framework for fast and accurate collision detection for haptic interaction. In Proceedings of the 1999 IEEE Virtual Reality Conference (VR’1999), pages 38–45, 1999.
  • [143] A. Gregory, M. C. Lin, S. Gottschalk, and R. Taylor. Fast and accurate collision detection for haptic interaction using a three degree-of-freedom force-feedback device. In Proceedings of the ACM SIGGRAPH’2005 Courses, 2005.
  • [144] X. Guo, J. Hua, and H. Qin. Touch-based haptics for interactive editing on point set surfaces. IEEE Computer Graphics and Applications, 24(6):31–39, 2004.
  • [145] R. Gupta, T. Sheridan, and D. Whitney. Experiments using multimodal virtual environments in design for assembly analysis. Presence: Teleoperators and Virtual Environments, 6(3):318–338, 1997.
  • [146] R. Gupta, D. Whitney, and Zeltzer. Prototyping and design for assembly analysis using multimodal virtual environments. Computer-Aided Design, 29(8):585–597, 1997.
  • [147] R. Gupta and Zeltzer. Prototyping and design for assembly analysis using multimodal virtual environments. In Proccedings of the 1995 ASME Computers in Engineering Conference and the Engineering Database Symposium, 1995.
  • [148] T. Gutiérrez, N. Gavish, S. Webel, and F. Tcchia. Multimodal Virtual Environments, chapter Training Platforms for Industrial Maintenance and Assembly, pages 227–239. CRC Press, Taylor & Francis, 2012.
  • [149] T. Gutiérrez, J. Rodriguez, Y. Velaz, S. Casado, A. Suescun, and E. J. Sanchez. IMA-VR: A multimodal virtual training system for skills transfer in industrial maintenance and assembly tasks. In Proceedings of the 2010 IEEE International Symposium on Robot and Human Interactive Communication, pages 428–433, 2010.
  • [150] B. Hannaford and J. H. Ryu. Time-domain passivity control of haptic interfaces. IEEE Transactions on Robotics and Automation, 18(1):1–10, 2002.
  • [151] P. T. Harker and J.-S. Pang. Finite-dimensional variational inequality and nonlinear complementarity problems: A survey of theory, algorithms and applications. Mathematical Programming, 48(1-3):161–220, 1990.
  • [152] S. Hasegawa and N. Fujii. Real-time rigid body simulation based on volumetric penalty method. In Proceedings of the 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 326–332, March 2003.
  • [153] S. Hasegawa and M. Sato. Real-time rigid body simulation for haptic interactions based on contact volume of polygonal objects. Computer Graphics Forum, 23(3):529–538, 2004.
  • [154] S. Hassan and J. Yoon. Haptic based optimized path planning approach to virtual maintenance assembly/disassembly (MAD). In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2010), pages 1310–1315, 2010.
  • [155] S. Hassan and J. Yoon. Intensified haptic guidance for a 3D virtual maintenance environment optimized by an intelligent assembly planning algorithm. In Proceedings of the 11th IEEE International Conference on Control, Automation, and Systems (ICCAS’2011), pages 676–679, 2011.
  • [156] S. Hassan and J. Yoon. Virtual maintenance system with a two-staged ant colony optimization algorithm. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA’2011), pages 931–936, 2011.
  • [157] S. Hassan and J. Yoon. Haptic assisted aircraft optimal assembly path planning scheme based on swarming and artificial potential field approach. Advanced Engineering Software, 69:18–25, 2014.
  • [158] V. Hayward and B. Armstrong. A new computational model of friction applied to haptic rendering. In Experimental Robotics VI, pages 403–412. Springer-Verlag, 2000.
  • [159] V. Hayward, O. R. Astley, M. C. Hernandez, D. Grant, and G. Robles-De-La-Torre. Haptic interfaces and devices. Sensor Review, 24(1):16–29, 2004.
  • [160] X. J. He and Y. H. Chen. A haptics-guided hole-filling system based on triangular mesh. Computer-Aided Design and Applications, 3(6):711–718, 2006.
  • [161] X. J. He and K. Choi. Stable haptic rendering for physics engines using inter-process communication andremote virtual coupling. International Journal of Advanced Computer Science and Applications (IJACSA), 4(1), 2013.
  • [162] D. Hecht, M. Reiner, and G. Halevy. Multimodal virtual environments: Response times, attention, and presence. Presence: Teleoperators and Virtual Environments, 15(5):515–523, 2006.
  • [163] Gentaro Hirota, Susan Fisher, and Ming Lin. Simulation of non-penetrating elastic bodies using distance fields. Technical report, University of North Carolina at Chapel Hill, 2000.
  • [164] C. H. Ho. Computer Haptics: Rendering Rechniques for Force-Feedback in Virtual Environments. Ph.D. dissertation, Massachusetts Institute of Technology (MIT), 2000.
  • [165] K. E. Hoff III, A. Zaferakis, M. C. Lin, and D. Manocha. Fast and simple 2D geometric proximity queries using graphics hardware. In Proceedings of the 2001 Symposium on Interactive 3D Graphics (I3D’2001), pages 145–148. ACM, 2001.
  • [166] K. E. Hoff III, A. Zaferakis, M. C. Lin, and D. Manocha. Fast 3D geometric proximity queries between rigid and deformable models using graphics hardware acceleration. UNC-CH Technical Report TR02-004, 2002.
  • [167] J. M. Hollerbach, E. C. Cohen, W. B. Thompson, R. Freier, D. Johnson, A. Nahvi, D. Nelson, T. V. Thompson II, and S. C. Jacobsen. Haptic interfacing for virtual prototyping of mechanical CAD designs. In Proceedings of the 1997 ASME Design for Manufacturing Symposium, 1997.
  • [168] J. M. Hollerbach, E. C. Cohen, W. B. Thompson, and S. C. Jacobsen. Rapid virtual prototyping of mechanical assemblies. In Proceedings of the NSF Design and Manufacturing Grantees Conference, 1996.
  • [169] B. M. Howard and J. M. Vance. Desktop haptic virtual assembly using physically based modelling. Virtual Reality, 11(4):207–215, 2007.
  • [170] Y. Hu, D. L. Wu, X. Fan, and X. J. Zhen. Grid-enabled collaborative virtual assembly environment. Assembly Automation, 30(4):352–364, 2010.
  • [171] J. Hua and H. Qin. Dynamic implicit solids with constraints for haptic sculpting. In Proceedings of the 2002 Shape Modeling International Conference, pages 119–129, 2002.
  • [172] J. Hua and H. Qin. Haptics-based volumetric modeling using dynamic spline-based implicit functions. In Proceedings of the 2002 IEEE Symposium on Volume Visualization and Graphics (VVS’2002). IEEE, 2002.
  • [173] J. Hua and H. Qin. Haptics-based dynamic implicit solid modeling. IEEE Transactions on Visualization and Computer Graphics, 10(5):574–586, 2004.
  • [174] P. M. Hubbard. Interactive collision detection. In Proceedings of the 1993 IEEE Symposium on Research Frontiers in Virtual Reality, pages 24–31, 1993.
  • [175] P. M. Hubbard. Collision detection for interactive graphics applications. IEEE Transactions on Visualization and Computer Graphics, 1(3):218–230, 1995.
  • [176] P. M. Hubbard. Approximating polyhedra with spheres for time-critical collision detection. ACM Transac, 15(3):179–210, 1996.
  • [177] T. C. Hudson, M. C. Lin, J. Cohen, S. Gottschalk, and D. Manocha. V-COLLIDE: Accelerated collision detection for VRML. In Proceedings of the 2nd Symposium on Virtual Reality Modeling Language, New York, NY, USA, 1997.
  • [178] R. Iacob, P. Mitrouchev, and J. C. Léon. A simulation framework for assembly/disassembly process modeling. In Proceedings of the 2007 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2007), number DETC2007-34804, pages 1017–1027. American Society of Mechanical Engineers (ASME), 2007.
  • [179] R. Iacob, P. Mitrouchev, and J. C. Leon. Contact identification for assembly-disassembly simulation with a haptic device. The Visual Computer, 24(11):973–979, 2008.
  • [180] R. Iacob, P. Mitrouchev, and J. C. Leon. Assembly simulation incorporating component mobility modelling based on functional surfaces. International Journal on Interactive Design and Manufacturing, 5(2):119–132, 2011.
  • [181] R. Iacob and D. Popescu. Generation of disassembly directions based on component mobility. Studies in Informatics and Control, pages 1220–1766, 2013.
  • [182] R. Iacob, D. Popescu, and P. Mitrouchev. Assembly/disassembly analysis and modeling techniques: A review. Strojniški vestnik-Journal of Mechanical Engineering, 58(11):653–664, 2012.
  • [183] R. Iglesias, S. Carrillo, S. Casado, T. Gutiérrez, and J. I. Barbero. Virtual assembly simulation in a distributed haptic virtual environment. In Proceedings of the International Conference on Advanced Design and Manufacture, 2006.
  • [184] R. Iglesias, S. Casado, T. Gutiérrez, A. García-Alonso, K. M. Yap, W. Yu, and A. Marshall. A peer-to-peer architecture for collaborative haptic assembly. In Proceedings of the 10th International Symposium on Distributed Simulation and Real-Time Applications (DS-RT’2006), pages 25–34, 2006.
  • [185] R. Iglesias, S. Casado, T. Gutiérrez, A. García-Alonso, W. Yu, and A. Marshall. Simultaneous remote haptic collaboration for assembling tasks. Multimedia Systems, 13(4):263–274, 2008.
  • [186] R. Iglesias, E. Prada, A. Uribe, A. García-Alonso, S. Casado, and T. Gutiérrez. Assembly simulation on collaborative haptic virtual environments. In Proceedings of the 15th International Conference in Central Europe on Computer Graphics, 2007.
  • [187] G. Irving, J. Teran, and R. Fedkiw. Invertible finite elements for robust simulation of large deformation. In Proceedings of the ACM SIGGRAPH’2004/Eurographics Symposium on Computer Animation, pages 131–140, Aire-la-Ville, Switzerland, Switzerland, 2004.
  • [188] H. Iwata and H. Noma. Volume haptization. In Proceedings of the 1993 IEEE 1993 Symposium on Research Frontiers in Virtual Reality, pages 16–23, 1993.
  • [189] D. L. James and D. K. Pai. BD-tree: Output-sensitive collision detection for reduced deformable models. ACM Transactions on Graphics (TOG) and Proceedings of the ACM SIGGRAPH’2004, 23(3):393–398, 2004.
  • [190] F. Janabi-Sharifi, V. Hayward, and C. S. J. Chen.

    Discrete-time adaptive windowing for velocity estimation.

    IEEE Transactions on Control Systems Technology, 8(6):1003–1009, 2000.
  • [191] S. Jayaram, H. I. Connacher, and K. W. Lyons. Virtual assembly using virtual reality techniques. Computer-Aided Design, 29(8):575–584, 1997.
  • [192] S. Jayaram, U. Jayaram, Y. J. Kim, C. DeChenne, K. Lyons, K. W. andLyons, and T. Mitsui. Industry case studies in the use of immersive virtual assembly. Virtual Reality, 11(4):217–228, 2007.
  • [193] S. Jayaram, U. Jayaram, H. Tirumali, and K. Lyons. A tool/part/human interaction model for assembly in virual environment. 2000.
  • [194] S. Jayaram, U. Jayaram, Y. Wang, H. Tirumali, K. Lyons, and P. Hart. VADE: A virtual assembly design environment. Journal of Computer Graphics and Applications, 19(6):44–50, 1999.
  • [195] S. Jayaram, J. M. Vance, R. Gadh, U. Jayaram, and H. Srinivasan. Assessment of VR technology and its applications to engineering problems. Journal of Computing and Information Science in Engineering, 1(1):72–83, 2001.
  • [196] S. Jayaram, Y. Wang, U. Jayaram, K. Lyons, and P. Hart. A virtual assembly design environment. In Proceedings of the IEEE Virtual Reality Annual International Symposium (VRAIS’1999), pages 172–179, 1999.
  • [197] U. Jayaram, S. Jayaram, C. DeChenne, Y. J. Kim, C. Palmer, and T. Mitsui. Case studies using immersive virtual assembly in industry. In Proceedings of the 2004 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2004), pages 627–636, 2004.
  • [198] U. Jayaram, S. Jayaram, I. Shaikh, Y. J. Kim, and Palmer. Introducing quantitative analysis methods into virtual environments for real-time assembly and continuous ergonomic evaluations. Computers in Industry, 57(3):283–296, 2006.
  • [199] U. Jayaram, S. Jayaram, Y. Yang, and K. Lyons. CORBA-based collaboration in a virtual assembly design environment. In Proceedings of the ASME Design Engineering Technical Conferences (DETC’2000), number DETC2000/CIE-14585. American Society of Mechanical Engineers (ASME), 2000.
  • [200] D. Jia, A. Bhatti, and S. Nahavandi. Design and evaluation of a haptically enable virtual environment for object assembly training. In Proceedings of the 2009 IEEE International Workshop on Haptic Audio visual Environments and Games (HAVE’2009), pages 75–80, 2009.
  • [201] P. Jimenez, F. Thomas, and Torras. 3D collision detection: A survey. Computers and Graphics, 25(2):269–285, 2001.
  • [202] D. C. Johnson and J. M. Vance. The use of the Voxmap PointShell method of collision detection in virtual assembly methods planning. In Proceedings of the 2001 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2001), 2001.
  • [203] H. Kang and A. Kak. Deforming virtual objects interactively in accordance with an elastic model. Computer-Aided Design, 28(4):251–262, 1996.
  • [204] E. Katchalski-Katzir, I. Shariv, M. Eisenstein, A. A Friesem, C. Aflalo, and I. A. Vakser. Molecular surface recognition: Determination of geometric fit between proteins and their ligands by correlation techniques. Proceedings of the National Academy of Sciences, 89(6):2195–2199, 1992.
  • [205] Y. Katznelson. An Introduction to Harmonic Analysis. Cambridge University Press, 3rd edition, 2004.
  • [206] L. Kavan and J. Žára. Fast collision detection for skeletally deformable models. In Computer Graphics Forum, volume 24, pages 363–372. Wiley Online Library, 2005.
  • [207] L. E. Kavraki. Computation of configuration-space obstacles using the fast Fourier transform. IEEE Transactions on Robotics and Automation, 11(3):408–413, 1995.
  • [208] L. E. Kavraki, M. N. Kolountzakis, and J. C. Latombe. Analysis of probabilistic roadmaps for path planning. IEEE Transactions on Robotics and Automation, 14(1):166–171, 1998.
  • [209] L. E. Kavraki, P. Svestka, J. C. Latombe, and M. H. Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4):566–580, 1996.
  • [210] J. Keiner, S. Kunis, and D. Potts. Using NFFT 3–a software library for various nonequispaced fast Fourier transforms. ACM Transactions on Mathematical Software (TOMS), 36(4):19, 2009.
  • [211] C. E. Kim and J. M. Vance. Using VPS (Voxmap PointShell) as the basis for interaction in a virtual assembly environment. In Proceedings of the 2003 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2003), pages 1153–1161, 2003.
  • [212] C. E. Kim and J. M. Vance. Collision detection and part interaction modeling to facilitate immersive virtual assembly methods. Journal of Computing and Information Science in Engineering, 4(2):83–90, 2004.
  • [213] C. E. Kim and J. M. Vance. Development of a networked haptic environment in VR to facilitate collaborative design using Voxmap PointShell (VPS) software. In Proceedings of the 2004 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2004), number DETC2004-57648, pages 19–25. American Society of Mechanical Engineers (ASME), 2004.
  • [214] Y. J. Kim, M. C. Lin, and D. Manocha. DEEP: Dual-space expansion for estimating penetration depth between convex polytopes. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation (ICRA’2002), volume 1, pages 921–926, 2002.
  • [215] J. T. Klosowski, M. Held, J. S. B. Mitchell, H. Sowizral, and K. Zikan. Efficient collision detection using bounding volume hierarchies of k-DOPs. IEEE Transactions on Acoustics Speech and Signal ProcessingVisualization and Computer Graphics, 4(1):21–36, 1998.
  • [216] S. Kockara, T. Halic, K. Iqbal, C. Bayrak, and R. Rowe. Collision detection: A survey. In Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics (ISIC’2007), pages 4046–4051, 2007.
  • [217] J. A. Kovacs, P. Chacon, Y. Cong, E. Metwally, and W. Wriggers. Fast rotational matching of rigid bodies by fast Fouriertransform acceleration of five degrees of freedom. Acta Crystallographica Section D: Biological Crystallography, 59(8):1371–1376, 2003.
  • [218] D. Kubus, I. Weidauer, and F. M. Wahl. 1kHz is not enough; how to achieve higher update rates with a bilateral teleoperation system based on commercial hardware. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’2009), pages 5107–5114, 2009.
  • [219] S. Kunis and S. Kunis. The nonequispaced FFT on graphics processing units. Proceedings in Applied Mathematics and Mechanics (PAMM), 12(1):7–10, 2012.
  • [220] N. Ladeveze, J. Y. Fourquet, and B. Puel. Interactive path planning for haptic assistance in assembly tasks. Computers & Graphics, 34(1):17–25, 2010.
  • [221] N. Ladeveze, J. Y. Fourquet, B. Puel, and M. Taix. Haptic assembly and disassembly task assistance using interactive path planning. In Proceedings of the 2009 IEEE Virtual Reality Conference (VR’2009), pages 19–25, 2009.
  • [222] E. Larsen, S. Gottschalk, M. C. Lin, and D. Manocha. Fast distance queries with rectangular swept sphere volumes. In Proceedings of the 2000 IEEE International Conference on Robotics and Automation (ICRA’2000), volume 4, pages 3719–3726, 2000.
  • [223] S. D. Laycock and A. M. Day. Recent developments and applications of haptic devices. Computer Graphics Forum, 22(2):117–132, 2003.
  • [224] D. Leithinger, S. Follmer, A. Olwal, and H. Ishii. Physical telepresence: Shape capture and display for embodied, computer-mediated remote collaboration. In Proceedings of the 27th annual ACM symposium on User Interface Software and Technology, pages 461–470, 2014.
  • [225] D. Leithinger, S. Follmer, A. Olwal, and H. Ishii. Shape displays: Spatial interaction with dynamic physical form. IEEE Computer Graphics and Applications, 35(5):5–11, 2015.
  • [226] M. C. Leu, H. A. ElMaraghy, A. Y. C. Nee, S. K. Ong, M. Lanzetta, M. Putz, W. Zhu, and A. Bernard. CAD model based virtual assembly simulation, planning and training. CIRP Annals - Manufacturing Technology, 62(2):799–822, 2013.
  • [227] T. Lim, M. Calis, J. M. Ritchie, J. R. Corney, R. G.e Dewar, and M. P. Desmulliez. A haptic assembly, machining and manufacturing system (HAMMS) approach. In Proceedings of the 1st International Virtual Manufacturing Workshop (VirMan’2006), 2006.
  • [228] T. Lim, R. Dewar, M. Calis, J. M. Ritchie, J. R. Corney, and M. Desmulliez. A structural assessment of haptic-based assembly processes. In Proccedings of the 1st International Virtual Manufacturing Workshop (VirMan’2006), volume 26, 2006.
  • [229] T. Lim, J. M. Ritchie, J. R. Corney, R. G. Dewar, K. Schmidt, and K. Bergsteiner. Assessment of a haptic virtual assembly system that uses physics-based interactions. In Proceedings of the 2007 IEEE International Symposium on Assembly and Manufacturing (ISAM’2007), pages 147–153, 2007.
  • [230] T. Lim, J. M. Ritchie, R. Sung, Z. Kosmadoudi, Y. Liu, and A. G. Thin. Haptic Virtual Reality Assembly - Moving Towards Real Engineering Applications, chapter Advances in Haptics, pages 693–722. InTech, 2010.
  • [231] T. Lim, J.M. Ritchie, R.G. Dewar, J.R. Corney, P. Wilkinson, M. Calis, M. Desmulliez, and J.-J. Fang. Factors affecting user performance in haptic assembly. Virtual Reality, 11(4):241–252, 2007.
  • [232] M. Lin and S. Gottschalk. Collision detection between geometric models: A survey. In Proceedings of the 1998 IMA Conference on Mathematics of Surfaces, volume 1, pages 37–56, 1998.
  • [233] X. Liu. Designing B-spline surfaces with haptics based on energy minimization method. In Proceedings of the 2012 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2012), pages 1509–1516. American Society of Mechanical Engineers, 2012.
  • [234] X. Liu. Filling N-sided holes using trimmed B-spline surfaces. In Proceedings of the 2012 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2012), number DETC2012-70735, pages 611–618. American Society of Mechanical Engineers, 2012.
  • [235] X. Liu. Designing B-spline surfaces with haptics based on variational technique. Computer-Aided Design and Applications, 11(4):369–380, 2014.
  • [236] X. Liu, G. Dodds, B. K. Hinds, and J. McCartney. Virtual DesignWorks: Designing 3D CAD models via touch interaction. In Proceedings of the 2003 ASME International Mechanical Engineering Congress and Exposition, pages 39–45. American Society of Mechanical Engineers, 2003.
  • [237] X. Liu, G. Dodds, J. McCartney, and B. K. Hinds. Design and deformation of CAD surface models with haptics. In Proceedings of the 2004 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2004), pages 883–890. American Society of Mechanical Engineers, 2004.
  • [238] X. Liu, G. Dodds, J. McCartney, and B. K. Hinds. Virtual DesignWorks–designing 3D CAD models via haptic interaction. Computer-Aided Design, 36(12):1129–1140, 2004.
  • [239] X. Liu, G. Dodds, J. McCartney, and B. K. Hinds. Manipulation of CAD surface models with haptics based on shape control functions. Computer-Aided Design, 37(14):1447–1458, 2005.
  • [240] Xiaodong Liu. Filling N-sided holes with trimmed B-spline surfaces based on energy-minimization method. Journal of Computing and Information Science in Engineering (JCISE), 15(1):011001:1–9, 2015.
  • [241] P. Lötstedt. Numerical simulation of time-dependent contact and friction problems in rigid body mechanics. SIAM Journal on Scientific and Statistical Computing, 5(2):370–393, 1984.
  • [242] T. Lozano-Perez. Spatial planning: A configuration space approach. IEEE Transactions on Computers, C-32(2):108–120, 1983.
  • [243] M. Lysenko. Fourier collision detection. International Journal of Robotics Research, 32(4):483–503, 2013.
  • [244] M. Lysenko, S. Nelaturi, and V. Shapiro. Group morphology with convolution algebras. In Proceedings of the 2010 ACM Symposium on Solid and Physical Modeling (SPM’2010), pages 11–22, New York, NY, USA, 2010.
  • [245] M. Lysenko, V. Shapiro, and Nelaturi. Non-commutative morphology: Shapes, filters, and convolutions. Computer Aided Geometric Design, 28(8):497–522, 2011.
  • [246] H. Maekawa and J. M. Hollerbach. Haptic display for object grasping and manipulating in virtual environment. In Proceedings of the 1998 IEEE International Conference on Robotics and Automation (ICR’1998), volume 3, pages 2566–2573 vol.3, May 1998.
  • [247] L. Marcelino, N. Murray, and T. Fernando. A constraint manager to support virtual maintainability. Computers and Graphics, 27(1):19–26, 2003.
  • [248] D. W. Marhefka and D. E. Orin. Simulation of contact using a nonlinear damping model. In Proceeding of the 1996 IEEE International Conference on Robotics and Automation, volume 2, pages 1662–1668, 1996.
  • [249] T. H. Massie. Initial Haptic Explorations with the Phantom: Virtual Touch through Point Interaction. PhD thesis, Massachusetts Institute of Technology (MIT), 1996.
  • [250] L. McAtamney and E. N. Corlett. RULA: A survey method for the investigation of work-related upper limb disorders. Applied Ergonomics, 24(2):91–99, 1993.
  • [251] J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon. A proposal for the Dartmouth summer research project on artificial intelligence. AI Magazine, 27(4):12–14, 2006. Originally published in August 31, 1955.
  • [252] S. D. McDermott and B. Bras. Development of a haptically enabled dis/re-assembly simulation environment. In Proceedings of the 1999 ASME Design Engineering Technical Conferences (DETC’1999), 1999.
  • [253] K. T. McDonnell. Dynamic Subdivision-based Solid Modeling. Research proficiency examination, State University of New York at Stony Brook, 2000.
  • [254] K. T. McDonnell, Y. S. Chang, and H. Qin. DigitalSculpture: a subdivision-based approach to interactive implicit surface modeling. Graphical Models, 67(4):34–369, 2005.
  • [255] K. T. McDonnell and H. Qin. Virtual clay: Haptics-based deformable solids of arbitrary topology. In F. Perales and E. R. Hancock, editors, Articulated Motion and Deformable Objects, volume 2492 of Lecture Notes in Computer Science, pages 1–20. Springer Berlin Heidelberg, 2002.
  • [256] K. T. McDonnell and H. Qin. A novel framework for physically based sculpting and animation of free-form solids. The Visual Computer, 23(4):285–296, 2007.
  • [257] K. T. McDonnell, H. Qin, and R. A. Wlodarczyk. Virtual clay: A real-time, haptics-based sculpting system. Technical report, State University of New York at Stony Brook, 2001.
  • [258] K. T. McDonnell, H. Qin, and R. A. Wlodarczyk. Virtual clay: A real-time sculpting system with haptic toolkits. In Proceedings of the 2001 Symposium on Interactive 3D Graphics, pages 179–190, 2001.
  • [259] W. A. McNeely, K. D. Puterbaugh, and J. J. Troy. Advances in voxel-based 6-DOF haptic rendering. In Proceedings of the ACM SIGGRAPH’2005 Courses, 2005.
  • [260] W. A. McNeely, K. D. Puterbaugh, and J. J. Troy. Six degree-of-freedom haptic rendering using voxel sampling. In Proceedings of the ACM SIGGRAPH’2005 Courses, New York, NY, USA, 2005.
  • [261] W. A. McNeely, K. D. Puterbaugh, and J. J. Troy. Voxel-based 6-DOF haptic rendering improvements. Haptics-e, 3(7), 2006.
  • [262] M. E. Merchant. Mechanics of the metal cutting process i. orthogonal cutting and a type 2 chip. Journal of Applied Physics, 16(5):267–275, 1945.
  • [263] B. Mirtich. V-Clip: Fast and robust polyhedral collision detection. ACM Transactions on Graphics (TOG), 17(3):177–208, 1998.
  • [264] B. Mirtich and J. Canny. Impulse-based simulation of rigid bodies. In Proceedings of the 1995 Symposium on Interactive 3D Graphics (I3D’1995), pages 181–217, New York, NY, USA, 1995.
  • [265] G. Moreau, P. Fuchs, and P. Stergiopoulos. Applications of virtual reality in the manufacturing industry: From design review to ergonomic studies. Mécanique and Industries, 5:171–179, 2004.
  • [266] J. J. Moreau. Standard inelastic shocks and the dynamics of unilateral constraints. In G. Del Piero and F. Maceri, editors, Unilateral Problems in Structural Analysis, volume 288, pages 173–221. Springer Vienna, 1985.
  • [267] D. Morris. Algorithms and data structures for haptic rendering: Curve constraints, distance maps, and data logging. Technical report, Standford University, 2006.
  • [268] D. Morris, Hong Tan, F. Barbagli, T. Chang, and K. Salisbury. Haptic feedback enhances force skill learning. In Proceedings of the 2007 Joint Conference on World Haptics, the Eurohaptics 2005, and the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 21–26, March 2007.
  • [269] T. S. Mujber, T. Szecsi, and M. S. J. Hashmi. Virtual reality applications in manufacturing process simulation. Journal of Materials Processing Technology, 155–156:1834–1838, 2004. Proceedings of the International Conference on Advances in Materials and Processing Technologies: Part 2.
  • [270] K. Murakami, R. Kiyama, T. Narumi, T. Tanikawa, and M. Hirose. Poster: A wearable augmented reality system with haptic feedback and its performance in virtual assembly tasks. In Proccedings of the 2013 IEEE Symposium on 3D User Interfaces (3DUI’2013), pages 161–162, 2013.
  • [271] N. Murray and T. Fernando. An immersive assembly and maintenance simulation environment. In International Conference on Advanced Research in Virtual and Rapid Prototyping, 2003.
  • [272] N. Murray and T. Fernando. An immersive assembly and maintenance simulation environment. In Proceedings of the 2004 IEEE International Symposium on Distributed Simulation and Real-Time Applications (DS-RT’2004), pages 159–166, October 2004.
  • [273] A. Nahvi, D. D. Nelson, J. M. Hollerbach, and D. E. Johnson. Haptic manipulation of virtual mechanisms from mechanical CAD designs. In Proceedings of the 1998 IEEE International Conference on Robotics and Automation (ICR’1998), volume 1, pages 375–380, 1998.
  • [274] S. Nelaturi, G. Burton, C. Fritz, and T. Kurtoglu. Automatic spatial planning for machining operations. In 2015 IEEE International Conference on Automation Science and Engineering (CASE’2015), pages 677–682, 2015.
  • [275] S. Nelaturi, A. Rangarajan, C. Fritz, and T. Kurtoglu. Automated fixture configuration for rapid manufacturing planning. Computer-Aided Design, 46:160–169, 2014.
  • [276] M. Nießner, C. Siegl, H. Schäfer, and C. T. Loop. Real-time collision detection for dynamic hardware tessellated objects. In Eurographics (Short Papers), pages 33–36, 2013.
  • [277] M. O’Connor, V. Srinivasan, and A. Jones. Connected Lie and symmetry subgroups of the rigid motions: Foundations and classification. Research Report 20512, IBM, 1996.
  • [278] Staff of the Production Automation Project. An introduction to PADL. Production Automation Project, Technical Memo. No. 22 (TM-22), University of Rochester, 1974.
  • [279] P. Olsson, F. Nysjo, J. M. Hirsch, and I. Carlbom. Snap-to-Fit, a haptic 6 DOF alignment tool for virtual assembly. In Proceedings of the 2013 World Haptics Conference (WHC’2013), pages 205–210, 2013.
  • [280] C. O’Sullivan and J. Dingliana. Real-time collision detection and response using sphere-trees. In Spring Conference on Computer Graphics, pages 83–92, 1999.
  • [281] I. J. Palmer and R. L. Grimsdale. Collision detection for animation using sphere-trees. Computer Graphics Forum, 14(2):105–116, 1995.
  • [282] C. Pan and S. Smith. Case study: The impact of assembly reorientations on assembly time. International Journal of Production Research, 44(21):4569–4585, 2006.
  • [283] M. Paterson. The Senses of Touch: Haptics, Affects and Technologies. Bloomsbury Academic, 2007.
  • [284] J. Perret, C. Kneschke, J. M. Vance, and G. Dumont. Interactive assembly simulation with haptic feedback. Assembly Automation, 33(3):214–220, 2013.
  • [285] G. Picinbono, H. Delingette, and N. Ayache. Non-linear anisotropic elasticity for real-time surgery simulation. Graphical Models, Special Issue on SMI’2002, 65(5):305–321, 2003.
  • [286] F. Picon, M. Ammi, and P. Bourdot. Case study of haptic methods for selection on CAD models. In Proceedings of the 2008 IEEE Virtual Reality Conference (VR’2008), pages 209–212, 2008.
  • [287] F. Picon, M. Ammi, and P. Bourdot. Force model for CAD selection. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (VRST’2008), pages 283–284, 2008.
  • [288] F. Picon, M. Ammi, and P.k Bourdot. Haptically-aided extrusion for object edition in CAD. In M. Ferre, editor, Haptics: Perception, Devices and Scenarios, volume 5024 of Lecture Notes in Computer Science, pages 736–741. Springer Berlin Heidelberg, 2008.
  • [289] D. Popescu, R. Iacob, F. Noel, C. Masclet, and T. Louis. Evaluation of a haptic environment for assembly task simulation. Studies in Informatics and Control, 24(3):329–338, 2015.
  • [290] D. Potts, G. Steidl, and M. Tasche. Fast Fourier transforms for nonequispaced data: A tutorial. In Modern Sampling Theory, pages 247–270. Springer-Verlag, 2001.
  • [291] S. Quinlan. Efficient distance computation between non-convex objects. In Proceedings of the 1994 IEEE International Conference on Robotics and Automation (ICR’1994), volume 4, pages 3324–3329, 1994.
  • [292] S. Redon, A. Kheddar, and S. Coquillart. Fast continuous collision detection between rigid bodies. Computer Graphics Forum, 21(3):279–287, 2002.
  • [293] S. Redon, A. Kheddar, and S. Coquillart. Gauss’ least constraints principle and rigid body simulations. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation (ICR’2002), volume 1, pages 517–522, 2002.
  • [294] S. Redon, Y. J. Kim, M. C. Lin, and D. Manocha. Fast continuous collision detection for articulated models. In Proceedings of the 9th ACM Symposium on Solid Modeling and Applications, pages 145–156, 2004.
  • [295] S. Redon, M. C. Lin, D. Manocha, and Y. J. Kim. Fast continuous collision detection for articulated models. Journal of Computing and Information Science in Engineering, 5(2):126–137, 2005.
  • [296] M. Renouf, V. Acary, and G. Dumont. 3D frictional contact and impact multibody dynamics. a comparison of algorithms suitable for real-time applications. In ECCOMAS Thematic Conference on Mutlibody Dynamics, 2005.
  • [297] M. Renz, C. Preusche, M. Pötke, H. P. Kriegel, and G. Hirzinger. Stable haptic interaction with virtual environments using an adapted Voxmap-PointShell algorithm. In Proceedings of the Eurohaptics 2001, 2001.
  • [298] A. A. G. Requicha, N. M. Samuel, and H. B. Voelcker. Part and assembly description languages–II. Production Automation Project, Technical Memo. No. 20a (TM-20a), University of Rochester, 1974.
  • [299] F. Reuleaux and E. S. Ferguson. Kinematics of Machinery: Outlines of a Theory of Machines. Dover Publications, 2012.
  • [300] J. M. Ritchie, T. Lim, H. Medellin, and R. S. Sung. A haptic based virtual assembly system for the generation of assembly process plans. In Memorias Del XV Congreso Internacional Anual De La Somim, pages 23–25, 2009.
  • [301] J. M. Ritchie, T. Lim, R. S. Sung, J. R. Corney, and H. Rea. The analysis of design and manufacturing tasks using haptic and immersive VR–some case studies. In D. Talaba and A. Amditis, editors, Product Engineering, pages 507–522. Springer Netherlands, 2008.
  • [302] J. M. Ritchie, T. Lim, R. S. Sung, and H. Medellin. Generation of assembly process plans and associated Gilbreth motion study data. In Proceedings of the 2nd Virtual Manufacturing Workshop, EU Intuition VR Network Conference Intuition, pages 6–8, 2008.
  • [303] J. B. T. M. Roerdink. Group morphology. Pattern Recognition, 33(6):877–895, 2000.
  • [304] L. B. Rosenberg. Virtual fixtures: Perceptual tools for telerobotic manipulation. In Proceedings of the 1993 IEEE Virtual Reality Annual International Symposium, pages 76–82, 1993.
  • [305] L. B. Rosenberg. Virtual Fixtures. PhD thesis, Stanford University, 1994.
  • [306] J. R. Rossignac, M. Allen, W. J. Book, A. Glezer, I. Ebert-Uphoff, C. Shaw, D. Rosen, S. Askins, J. Bai, P. Bosscher, J. Gargus, M. Kim, I. Llamas, A. Nguyen, G. Yuan, and H. Zhu. Finger sculpting with digital clay: 3D shape input and output through a computer-controlled real surface. In Shape Modeling International, 2003, pages 229–231. IEEE, 2003.
  • [307] E. Ruffaldi, D. Morris, F. Barbagli, K. Salisbury, and M. Bergamasco. Voxel-based haptic rendering using implicit sphere trees. In Proceedings of the 16th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pages 319–325, 2008.
  • [308] M. Sagardia, T. Hulin, C. Preusche, and Hirzinger. Improvements of the Voxmap-PointShell algorithm–fast generation of haptic data-structures. In 53rd Internationales Wissenschaftliches Kolloquium Technische Universit t Ilmenau, 2008.
  • [309] M. Sagardia, T. Stouraitis, and J. L. e Silva. A new fast and robust collision detection and force computation algorithm applied to the physics engine Bullet: Method, integration, and evaluation. In Proceedings of the 2014 Conference and Exhibition of the European Association of Virtual and Augmented Reality (EuroVR’2014), 2014.
  • [310] S. E. Salcudean and T. D. Vlaar. On the emulation of stiff walls and static friction with a magnetically levitated input/output device. Journal of Dynamic Systems, Measurement, and Control (JDSMC), 119(1):127–132, 1997.
  • [311] J. K. Salisbury and M. A. Srinivasan. Phantom-based haptic interaction with virtual objects. IEEE Computer Graphics and Applications, 17(5):6–10, 1997.
  • [312] J. Sauer and E. Schömer. A constraint-based approach to rigid body dynamics for virtual reality applications. In Proceedings of the 5th ACM Symposium on Virtual Reality Software and Technology (VRST’1998), pages 153–162, New York, NY, USA, 1998.
  • [313] J. Savall, D. Borro, J. J. Gil, and L. Matey. Description of a haptic system for virtual maintainability in aeronautics. In Proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 3, pages 2887–2892, 2002.
  • [314] J. T. Schwartz and M. Sharir. On the “piano movers” problem i. the case of a two-dimensional rigid polygonal body moving amidst polygonal barriers. Communications on Pure and Applied Mathematics, 36(3):345–398, 1983.
  • [315] J. T. Schwartz and M. Sharir. On the “piano movers” problem. ii. general techniques for computing topological properties of real algebraic manifolds. Advances in Applied Mathematics, 4(3):298–351, 1983.
  • [316] J. T. Schwartz and M. Sharir. On the “piano movers” problem: Iii. coordinating the motion of several independent bodies: The special case of circular bodies moving amidst polygonal barriers. The International Journal of Robotics Research, 2(3):46–75, 1983.
  • [317] J. R. Searle. Mind, Language and Society: Philosophy in the Real World. Cambridge University Press, 1998.
  • [318] ELI Educational Series. 7 things you should know about haptics. Technical report, Educause Learning Initiative (ELI), September 2007.
  • [319] A. Seth, H. J. Su, and J. M. Vance. A desktop networked haptic VR interface for mechanical assembly. In Proceedings of the 2005 ASME International Mechanical Engineering Congress and Exposition, pages 173–180, 2005.
  • [320] A. Seth, H. J. Su, and J. M. Vance. SHARP: A system for haptic assembly and realistic prototyping. In Proceedings of the 2006 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2006), pages 905–912, 2006.
  • [321] A. Seth, H. J. Su, and J. M. Vance. Development of a dual-handed haptic assembly system: SHARP. Journal of Computing and Information Science in Engineering, 8(4):1–8, 2008.
  • [322] A. Seth, J. M. Vance, and J. H. Oliver. Combining geometric constraints with physics modeling for virtual assembly using SHARP. In Proceedings of the 2007 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2007), 2007.
  • [323] A. Seth, J. M. Vance, and J. H. Oliver. Combining dynamic modeling with geometric constraint management to support low clearance virtual manual assembly. Journal of Mechanical Design, 132(8):081002–1–7, 2010.
  • [324] A. Seth, J. M. Vance, and J. H. Oliver. Virtual reality for assembly methods prototyping: A review. Virtual Reality, 15(1):5–20, 2011.
  • [325] I. Shaikh, U. Jayaram, S. Jayaram, and C. Palmer. Participatory ergonomics using VR integrated with analysis tools. In Proceedings of the 2004 Winter Simulation Conference, volume 2, pages 1746–1754, 2004.
  • [326] I. Shaikh, Y. J. Kim, S. Jayaram, U. Jayaram, and H. Choi. Integration of immersive environment and RULA for real-time study of workplace related musculoskeletal disorders in the upper limb. In Proceedings of the 2003 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2003), pages 1163–1171. American Society of Mechanical Engineers (ASME), 2003.
  • [327] C. Shukla, M. Vazquez, and Chen. F. F. Virtual manufacturing: An overview. Computers & Industrial Engineering, 31(1-2):79–82, 1996. Proceedings of the 19th International Conference on Computers and Industrial Engineering.
  • [328] J. Simard, M. Ammi, F. Picon, and P. Bourdot. Potential field approach for haptic selection. In Proceedings of Graphics Interface (GI’2009), pages 203–206. Canadian Information Processing Society, 2009.
  • [329] M. A. Srinivasan. What is haptics? Technical report, Massachusetts Institute of Technology, 1995.
  • [330] M. A. Srinivasan and Basdogan. Haptics in virtual environments: Taxonomy, research status, and challenges. Computers and Graphics, 21(4):393–404, 1997.
  • [331] V. Srinivasan. Standardizing the specification, verification, and exchange of product geometry: Research, status and trends. Computer-Aided Design, 40(7):738–749, 2008. Current State and Future of Product Data Technologies (PDT).
  • [332] D. Stewart. Rigid-body dynamics with friction and impact. SIAM Review, 42(1):3–39, 2000.
  • [333] E. Stoll, M. Wilde, and C. Pong. Using virtual reality for human-assisted in-space robotic assembly. In Proceedings of the 2009 World Congress on Engineering and Computer Science, volume 2, 2009.
  • [334] A. Sud, N. Govindaraju, R. Gayle, I. Kabul, and D. Manocha. Fast proximity computation among deformable models using discrete Voronoi diagrams. ACM Transactions on Graphics, 25(3):1144–1153, 2006.
  • [335] R. C. W. Sung, J. M. Ritchie, T. Lim, and H. Medellin. Assembly planning and motion study using virtual reality. In Proceedings of the 2009 ASME-AFM World Conference on Innovative Virtual Reality, pages 31–38, 2009.
  • [336] R. C. W. Sung, J. M. Ritchie, G. Robinson, P. Day, J. R. Corney, and H. J. Rea. The role of therbligs in automated design process mapping. In Proceedings of the KIM Project Conference, 2008.
  • [337] R. C. W. Sung, J. M. Ritchie, G. Robinson, P. N. Day, J. R. Corney, and T. Lim. Automated design process modelling and analysis using immersive virtual reality. Computer-Aided Design, 41(12):1082–1094, 2009.
  • [338] I. E. Sutherland. Sketchpad: A Man-Machine Graphical Communication System. Ph.D. dissertation, Massachusetts Institute of Technology (MIT), 1963.
  • [339] L. B. Tang, Y. H. Chen, and X. J. He. Multi-material compliant mechanism design and haptic evaluation. Virtual and Physical Prototyping, 2(3):155–160, 2007.
  • [340] M. Tang, D. Manocha, and R. Tong. Fast continuous collision detection using deforming non-penetration filters. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, pages 7–13. ACM, 2010.
  • [341] L. Tching and G. Dumont. Haptic simulations based on non-smooth dynamics for rigid-bodies. In Proceedings of the 15th ACM Symposium on Virtual Reality Software and Technology (VRST’2008), pages 87–90, New York, NY, USA, 2008.
  • [342] L. Tching and G. Dumont. Interactive simulation based on non-smooth contact dynamics: Application to haptic rigid-body simulations. In Contecpt and Integrated Design and Manufacturing in Mechanical Engineering (C&IDMME’2008), pages 87–90, New York, NY, USA, 2008.
  • [343] L. Tching, G. Dumont, and J. Perret. Haptic assembly of CAD models using virtual constraint guidance. In Proceedings of the 2010 ASME World Conference on Innovative Virtual Reality, pages 11–19, 2010.
  • [344] L. Tching, G. Dumont, and J. Perret. Interactive simulation of CAD models assemblies using virtual constraint guidance. International Journal on Interactive Design and Manufacturing, 4(2):95–102, 2010.
  • [345] M. Teschner, S. Kimmerle, B. Heidelberger, G. Zachmann, L. Raghupathi, A. Fuhrmann, M. P. Cani, F. Faure, N. Magnenat-Thalmann, W. Strasser, and P. Volino. Collision detection for deformable objects. Computer Graphics Forum, 24(1):61–81, 2005.
  • [346] T. V. Thompson II and E. Cohen. Direct haptic rendering of complex trimmed NURBS models. In Proceedings of the ACM SIGGRAPH’2005 Courses, SIGGRAPH’2005, 2005.
  • [347] T. V. Thompson II, D. D. Nelson, E. Cohen, and J. Hollerbach. Maneuverable NURBS models within a haptic virtual environment. In Proceedings of the 6th Annual Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, volume 61, pages 37–44, 1997.
  • [348] Thomas V. Thompson II, T. V., D. E. Johnson, and E. Cohen. Direct haptic rendering of sculptured models. In Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D’1997, pages 167–176, 1997.
  • [349] D. Touraine and P. Bourdot. VEserver: A manager for input and haptic multi-sensorial devices. In Proceedings of the 10th IEEE International Workshop on Robot and Human Interactive Communication, pages 2–7, 2001.
  • [350] D. Touraine, P. Bourdot, Y. Bellik, and L. Bolot. A framework to manage multimodal fusion of events for advanced interactions within virtual environments. In Proceedings of the 2002 Workshop on Virtual Environments (EGVE’2002), pages 159–168. Eurographics Association, 2002.
  • [351] C. Trakunsaranakom, F. Noël, and P. Marin. Assessment of virtual reality environments for design activities. In G Zachmann, J. Perret, and A. Amditis, editors, Conference and Exhibition of the European Association of Virtual and Augmented Reality (EURO-VR’2014), 2014.
  • [352] M. Y. Tsalamlal, N. Ouarti, and M. Ammi. Non-intrusive haptic interfaces: State-of-the art survey. In I. Oakley and S. Brewster, editors, Haptic and Audio Interaction Design, volume 7989 of Lecture Notes in Computer Science, pages 1–9. Springer Berlin Heidelberg, 2013.
  • [353] A. M. Turing. On computable nnumbers, with an application to the entscheidungsproblem. In Proceedings of the London Mathematical Society, volume 42 of 2, 1936.
  • [354] A. M. Turing. Computing machinery and intelligence. Mind, pages 433–460, 1950.
  • [355] J. M. Vance and G. Dumont. A conceptual framework to support natural interaction for virtual assembly tasks. In Proceedings of the 2011 ASME World Conference on Innovative Virtual Reality, pages 273–278, 2011.
  • [356] B. D. Varalakshmi, J. Thriveni, K. R. Venugopal, and L. M. Patnaik. Haptics: State of the art survey. International Journal of Computer Science Issues, 9(5):234–244, 2012.
  • [357] M. Veit, A. Capobianco, and D. Bechmann. Dynamic decomposition and integration of degrees of freedom for 3-D positioning. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology (VRST’2010), pages 131–134, New York, NY, USA, 2010.
  • [358] H. B. Voelcker and A. A. G. Requicha. Research in solid modeling at the university of rochester: 1972–1987. In L. Piegl, editor, Fundamental Developments of Computer-Aided Geometric Modeling, pages 203–254. Academic Press Ltd., London, UK, 1993.
  • [359] S. Volkov and J. M. Vance. Effectiveness of haptic sensation for the evaluation of virtual prototypes. Journal of Computing and Information Science in Engineering, 1(2):123–128, 2001.
  • [360] H. Wan, S. Gao, Q. Peng, G. Dai, and F. Zhang. MIVAS: A multi-modal immersive virtual assembly system. In Proceedings of the 2004 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2004), pages 113–122, 2004.
  • [361] M. Wan and W. A. McNeely. Quasi-static approximation for 6 degrees-of-freedom haptic rendering. In Proceedings of the 14th IEEE Visualization 2003 (VIS’03), pages 257–262, Washington, DC, USA, 2003.
  • [362] G. G Wang. Definition and review of virtual prototyping. Journal of Computing and Information Science in Engineering, 2(3):232–236, 2002.
  • [363] Y. Wang, Y. Chen, Z. Nan, and Y. Hu. Ieee international conference on accessibility analysis for cmm inspection planning using haptic device. In Robotics and Biomimetics (ROBIO’2006), pages 1239–1243, 2006.
  • [364] Y. Wang, S. Jayaram, and U. Jayaram. Physically based modeling in virtual assembly. International Journal of Virtual Reality, 5:1–14, 2001.
  • [365] Y. Wang, S. Jayaram, U. Jayaram, and K. Lyons. Physically Based Modeling in Virtual Assembly. PhD thesis, Washington State University, 1998.
  • [366] Y. Wang, U. Jayaram, S. Jayaram, and S. Imtiyaz. Methods and algorithms for constraint-based virtual assembly. Virtual Reality, 6(4):229–243, 2003.
  • [367] Y. Wang, W. Zhang, Y. Chen, D. Liu, and R. Zhang. Accessibility analysis for CMM inspection planning by means of haptic device and STL representation. In IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurements Systems (VECIMS’2009), pages 174–178, 2009.
  • [368] S. Webel, U. Bockholt, T. Engelke, N. Gavish, M. Olbrich, and C. Preusche. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems, 61(4):398–403, 2013. Models and Technologies for Multi-modal Skill Training.
  • [369] S. Webel, U. Bockholt, T. Engelke, N. Gavish, and F. Tecchia. Design recommendations for augmented reality based training of maintenance skills. In Recent Trends of Mobile Collaborative Augmented Reality Systems, pages 69–82. Springer, 2011.
  • [370] S. Webel, U. Bockholt, T. Engelke, M. Peveri, M. Olbrich, and C. Preusche. Augmented reality training for assembly and maintenance skills. In Proceedings of the International Conference (SKILLS’2011), BIO Web of Conferences, volume 1, page 00097. EDP Sciences, 2011.
  • [371] Wegner. Interactive foundations of computing. Theoretical Computer Science, 192(2):315–351, 1998.
  • [372] P. Wegner. Why interaction is more powerful than algorithms. Communications of, 40(5):80–91, 1997.
  • [373] P. Wegner, E. Eberbach, and M. Burgin. Computational completeness of interaction machines and Turing machines. In Turing-100, pages 405–414, 2012.
  • [374] D. E. Weisberg. The Engineering Design Revolution: The People, Companies and Computer Systems That Changed Forever the Practice of Engineering. Cyon Research Corporation, 2008.
  • [375] R. Weller. Inner sphere trees. In New Geometric Data Structures for Collision Detection and Haptics, pages 113–144. Springer International Publishing, 2013.
  • [376] R. Weller and G. Zachmann. Inner sphere trees for proximity and penetration queries. In Proceedings of the 2009 Robotics: Science and Systems Conference (RSS’2009), volume 2, 2009.
  • [377] R. Weller and G. Zachmann. Stable 6-DOF haptic rendering with inner sphere tees. In Proceedings of the 2009 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2009), 2009.
  • [378] R. Weller and G. Zachmann. A unified approach for physically-based simulations and haptic rendering. In Proceedings of the ACM SIGGRAPH’2009 Symposium on Video Games, pages 151–159, New York, NY, USA, 2009.
  • [379] R. Weller and G. Zachmann. Inner sphere trees and their application to collision detection. In G. Brunnett, S. Coquillart, and G. Welch, editors, Virtual Realities, pages 181–201. Springer Vienna, 2011.
  • [380] J. G. W. Wildenbeest, D. A. Abbink, C. J. M. Heemskerk, F. C. T. van der Helm, and H. Boessenkool. The impact of haptic feedback quality on the performance of teleoperated assembly tasks. IEEE Transactions on Haptics, 6(2):242–252, 2013.
  • [381] A. Wilson, E. Larsen, D. Manocha, and M. C. Lin. Partitioning and handling massive models for interactive collision detection. In Computer Graphics Forum, volume 18, pages 319–330. Wiley Online Library, 1999.
  • [382] D. L. Wu, X. J. Zhen, X. Fan, Y. Hu, and H. Zhu. A virtual environment for complex products collaborative assembly operation simulation. Journal of Intelligent Manufacturing, 23(3):821–833, 2012.
  • [383] P. Xia, A. Lopes, and M. Restivo. Design and implementation of a haptic-based virtual assembly system. Assembly Automation, 31(4):369–384, 2011.
  • [384] P. J. Xia, A. M. Lopes, M. T. Restivo, and Y. X. Yao. A new type haptics-based virtual environment system for assembly training of complex products. International Journal of Advanced Manufacturing Technology, 58(1-4):379–396, 2012.
  • [385] P. J. Xia, A. Mendes Lopes, and M. T. Restivo. A review of virtual reality and haptics for product assembly: From rigid parts to soft cables. Assembly Automation, 33(2):157–164, 2013.
  • [386] Y. Yang, S. Jayaram, U. Jayaram, K. Lyons, and P. Hart. Representation of swept volumes in a parametric CAD system using trajectory information from virtual environments. In Proceedings of the ASME Design Engineering Technical Conferences (DETC’1999), number DETC99/CIE-9109. American Society of Mechanical Engineers (ASME), 1999.
  • [387] Z. Y. Yang and Y. H. Chen. Haptic rendering of milling. In Proceedings of the EuroHaptics, 2003.
  • [388] Z. Y. Yang and Y. H. Chen. Reverse engineering based on virtual volume sculpting. Computer-Aided Design and Applications, 1(1-4):63–72, 2004.
  • [389] Z. Y. Yang and Y. H. Chen. A reverse engineering method based on haptic volume removing. Computer-Aided Design, 37(1):45–54, 2005.
  • [390] Z. Y. Yang and Y. Z Chen. Inspection path generation in haptic virtual CMM. Computer-Aided Design and Applications, 2(1-4):273–282, 2005.
  • [391] Z. Y. Yang, L. Lian, and Y. Chen. Haptic function evaluation of multi-material part sesign. Computer-Aided Design, 37(7):727–736, 2005.
  • [392] G. Zachmann. Rapid collision detection by dynamically aligned DOP-trees. In Proceedings of the 1998 IEEE Virtual Reality Annual International Symposium, pages 90–97, 1998.
  • [393] U. Zaldivar-Colado and S. Garbaya. Virtual assembly environment modelling. In Proceedings of the 2009 ASME-AFM World Conference on Innovative Virtual Reality, pages 157–163, 2009.
  • [394] X. Zhang, M. Lee, and Y. J. Kim. Interactive continuous collision detection for non-convex polyhedra. The Visual Computer, 22(9-11):749–760, 2006.
  • [395] X. Zhang, S. Redon, M. Lee, and Y. J. Kim. Continuous collision detection for articulated models using taylor models and temporal culling. In Proceedings of SIGGRAPH’2007, volume 26, 2007.
  • [396] X. Zhang, S. Redon, M. Lee, and Y. J. Kim. Continuous collision detection for articulated models using taylor models and temporal culling. ACM Transactions on Graphics (TOG), 26(3-15), 2007.
  • [397] X. J. Zhen, D. L. Wu, X. Fan, and Y. Hu. Distributed parallel virtual assembly environment for automobile development. Assembly Automation, 29(3):279–289, 2009.
  • [398] X. J. Zhen, D. L. Wu, Yong Hu, and X-M Fan. A real-time simulation grid for collaborative virtual assembly of complex products. International Journal of Computer Integrated Manufacturing, 23(6):500–514, 2010.
  • [399] Z. Zhu, S. Gao, H. Wan, Y. Luo, and W. Yang. Grasp identification and multi-finger haptic feedback for virtual assembly. In Proceedings of the 2004 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE’2004), number DETC2004-57718, pages 661–669. American Society of Mechanical Engineers (ASME), 2004.
  • [400] Y. Zhuang and Canny. Real-time simulation of physically realistic global deformation. Technical report, University of California at Berkeley, 1999.
  • [401] P. Zimmermann. Virtual reality aided design. a survey of the use of VR in automotive industry. In D. Talaba and A. Amditis, editors, Product Engineering, pages 277–296. Springer Netherlands, 2008.
  • [402] F. Zorriassatine, C. Wykes, R. Parkin, and N. Gindy. A survey of virtual prototyping techniques for mechanical product development. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 217(4):513–530, 2003.