Microsoft demonstrates the tactile feedback controller pivot, picking apples in VR is more realistic

When you try to pick an apple from an apple tree, you will get a series of different sensory experiences, such as the hardness of holding the apple, the resistance of the branch when pulling the apple, the weight of the apple on the palm after picking the apple, and the smoothness and roundness of the fingertips when touching the apple. < p > < p > in recent years, Microsoft research has made steady progress in controller, and the team has been committed to achieving a tactile feedback that can match the realistic vision of head display. For devices such as normal touch, the team can simulate the surface tilt texture and texture of virtual objects at the fingertips; claw can give you a feeling of grasping objects with your fingers to explore their compliance and elasticity; and torc brings a level of flexibility close to the real world. Using the prototype, you can sense the virtual apple skin, squeeze the virtual fruit, and move with your hands. However, in order to realize the complete interaction with virtual apple in virtual environment, we must also consider the spatial dynamics of objects. Now, Microsoft has come up with a new controller, haptic pivot, which brings the physical properties of force to the virtual reality controller. Pivot is a wrist device, which is a portable component containing a tactile handle that can be moved back and forth on the hand as needed. < / P > < p > if Newton needed to find inspiration for the laws of motion and gravity from a virtual apple that fell off a virtual tree, he would need a pivot like controller. By pinning pivot to the wrist, the team can render the momentum and resistance of the toss object. All of this is governed by Newton’s laws, including the speed at which the simulated object reaches the hand: the robotic tactile handle unfolds when needed, approaches and finally reaches the hand, creating the first touch feeling from the empty hand to holding the object, and simulates our natural interaction with physical objects in a way that traditional handheld controllers cannot. A team from the University of Potsdam, Microsoft Research, Stanford University and the Federal Institute of technology in Zurich studied the performance and limitations of pivot, and co authored and published a paper entitled. In addition, the team will present haptic pivot at the ACM 2020 Symposium on user interface software and technology this week. < / P > < p > pivot provides control and tactile rendering of virtual objects on demand. It includes a tactile handle that is unfolded and retracted by an electric hinge. The passive radial hinge allows for natural hand tilt. The core of pivot is hinge mechanism and tactile handle. The tactile handle belongs to a replaceable component and can be replaced by an existing controller. But in the pivot study, the team equipped the prototype handle with a capacitive touch sensor that detects object contact and release; a voice coil actuator to provide vibration feedback; and a trigger switch to control input. The tactile handle works through an improved servo motor and can be released to the palm of the hand as needed, and retracted when not needed to keep hands free. This ability makes pivot an ideal choice for augmented reality or mixed reality scenarios. For example, you can operate the keyboard normally, use the mouse, or work with other physical objects. As long as you need to, you can quickly swing your wrist to start pivot, and rotate and release the handle to the palm so you can interact with the dummy. Swing your wrist again and the handle will retract. Both calls are detected by an internal accelerometer. < / P > < p > pivot allows the user’s hands to move freely and provides support when a controller is needed. The rapid swing of the wrist turns the tactile handle into the hand. This on demand capability is very useful in augmented reality or mixed reality scenarios. < / P > < p > based on the perception of capacitive sensors, the motor stops running when the user holds the handle. The passive radial hinge allows the user to move the wrist left and right and up and down while continuing to hold the handle. In order to prevent the tactile handle from hitting the thumb when it is moving between the rest position and the active position, the electric hinge is inclined to the hand rather than perpendicular to the hand. In addition, the team set a 190 degree working range to prevent the handle from interfering with users when not in use. < / P > < p > when interacting with virtual objects, pivot will shine its true power. Take picking apples from trees. From the moment Apple enters the working range, machinery, electronics, firmware and software work together until Apple stops in the palm of the user’s hand. < p > < p > head displays such as Microsoft hololens can track their hands with computer vision, so that the control system can detect when a user starts touching a target, such as apple. When the apple is within a working radius of 30 cm, pivot will move the tactile handle to a ready position. When the user’s hand is less than 10 cm close to the apple, the handle will approach proportionally and fall on the palm when the user holds the virtual apple with his finger. The handle moves as fast as the user reaches for an apple, providing a very realistic impact simulation. Four capacitive touch areas along the surface of the handle register the user’s contact with the handle, and send signals to the virtual hand through the serial communication interface, which makes the virtual hand close around the apple when the user’s real hand holds the handle. Using less than 1 millisecond polling, the interface provides tactile response instantaneity that meets user expectations. < / P > < p > when pulling an apple, you will encounter the expected resistance from the branch, because pivot will try to pull the tactile handle away from the user’s hand through the motor. This resistance will continue until the apple separates from the branch, and the voice coil actuator will feel a “bang” feeling, and then the user will feel the impact and weight of the apple falling on the palm. Pivot does not pull the handle away, but presses it into the palm, creating a sense of impact and weight. Pivot can render these forces in the palm and fingers, because it is based on the wrist rather than the palm. By simply turning your hand and releasing the handle, you can throw a bad apple on the ground or put a good apple in the basket. In addition, pivot promotes hand to hand interaction when worn on both wrists, such as picking up apples from a basket with a handle. The device can present the feeling of grasping by synchronizing their respective tactile feedback. < p > < p > wearing pivot on both hands can realize tactile feedback of two hands interaction. Here, participants are stretching and compressing the basket, and the device will render push and pull to both hands simultaneously < / P > < p > when designing pivot around the wrist, the team first considers baseball. From pitching to first base, the arm goes through a lot of things. The same is true of other sports. By incorporating a wrist shape parameter or design, the device can provide a wider range of mobility without interfering with the player’s physical environment. < / P > < p > with pivot, you can drop virtual objects. The reaction time of catching the virtual object in flight is shorter than that of grasping the static virtual object. The simulation of catching a virtual object requires the pivot and visual input to be aligned correctly, so as to accurately render the object when it touches the hand. It can achieve a larger collision radius for high-speed motion, so as to improve the response ability of the equipment. < / P > < p > just like throwing an apple into a basket, throwing relies on pivot to sense the movement of the hand and the release of the tactile handle. This is consistent with the release of virtual objects. When released, the motor attempts to pull the handle away from the palm at a physically correct angular speed. In other words, the handle can be gripped to fully retracted in 340 milliseconds, which is the time required to blink. This kind of simulation can be used not only in sports games, but also in virtual workplace. For example, before actually manufacturing a product, a worker or designer can feel the force of a virtual design or product in a new way. < / P > < p > pivot can not only help you grasp the virtual apple, but also allow you to throw the apple into the basket, throw the apple out, and catch the flying apple. Touch is the Ultimate Frontier of rendering. Once you’re able to achieve incredibly realistic visual effects in virtual reality and augmented reality, you’re going to simulate the natural interaction with this virtual object. At this point, the sense of touch will come to the center of the stage. < / P > < p > nowadays, the visual effects of virtual reality have been so immersive and fascinating that when you put on a virtual reality head display and enter a virtual world with an apple hanging on a tree branch, you can’t help but reach for it. But when you reach for the apple and don’t feel its smoothness and hardness, the resistance of the branch to pull back, or its weight in the palm of your hand, the illusion will be shattered. With a touch controller like pivot, Microsoft researchers are trying to solve this problem. Continue ReadingYueshang group has become the third social e-commerce service platform listed in China after being gathered in pinduoduo