AR and VR aren’t just for gaming; they’re also for amazing entertainment experiences that immerse you in the stuff you love like never before. Our team is at Mobile World Congress this week, and we shared a few updates. Let’s dive in.

Mobile VR: The best way to watch

We’ve built Daydream with lessons learned from Cardboard. It’s more comfortable, interactive, and immersive; as a result, people are spending more time using it—about 40 minutes per week. With six Daydream-ready phones and 100 Daydream apps to explore, there are lots of devices and experiences to choose from.

Announcing Sky VR

Welcoming The Sims, Chelsea Kicker and WSJ AR to Tango

Tango technology powers devices that help you bring virtual objects into your world, and now three new augmented reality (AR) experiences are available. The Sims app lets you use your phone with Tango technology to travel around the Sims house; Chelsea Kicker puts a Chelsea football player right in your space, so you can take your picture with him or even try to best him at a few soccer tricks; and WSJ AR lets you visualize stock trends.

So whether you’re one of millions using Cardboard, or kicking back with the best games and 360 videos on Daydream, or playing soccer with Chelsea stars on Tango, you can have incredible experiences in virtual and augmented reality that are hassle-free, comfortable, and fun.

AR and VR aren’t just for gaming; they’re also for amazing entertainment experiences that immerse you in the stuff you love like never before. Our team is at Mobile World Congress this week, and we shared a few updates. Let’s dive in.

Mobile VR: The best way to watch

We’ve built Daydream with lessons learned from Cardboard. It’s more comfortable, interactive, and immersive; as a result, people are spending more time using it—about 40 minutes per week. With six Daydream-ready phones and 100 Daydream apps to explore, there are lots of devices and experiences to choose from.

Announcing Sky VR

Welcoming The Sims, Chelsea Kicker and WSJ AR to Tango

Tango technology powers devices that help you bring virtual objects into your world, and now three new augmented reality (AR) experiences are available. The Sims app lets you use your phone with Tango technology to travel around the Sims house; Chelsea Kicker puts a Chelsea football player right in your space, so you can take your picture with him or even try to best him at a few soccer tricks; and WSJ AR lets you visualize stock trends.

So whether you’re one of millions using Cardboard, or kicking back with the best games and 360 videos on Daydream, or playing soccer with Chelsea stars on Tango, you can have incredible experiences in virtual and augmented reality that are hassle-free, comfortable, and fun.

AR and VR aren’t just for gaming; they’re also for amazing entertainment experiences that immerse you in the stuff you love like never before. Our team is at Mobile World Congress this week, and we shared a few updates. Let’s dive in.

Mobile VR: The best way to watch

We’ve built Daydream with lessons learned from Cardboard. It’s more comfortable, interactive, and immersive; as a result, people are spending more time using it—about 40 minutes per week. With six Daydream-ready phones and 100 Daydream apps to explore, there are lots of devices and experiences to choose from.

Announcing Sky VR

Welcoming The Sims, Chelsea Kicker and WSJ AR to Tango

Tango technology powers devices that help you bring virtual objects into your world, and now three new augmented reality (AR) experiences are available. The Sims app lets you use your phone with Tango technology to travel around the Sims house; Chelsea Kicker puts a Chelsea football player right in your space, so you can take your picture with him or even try to best him at a few soccer tricks; and WSJ AR lets you visualize stock trends.

So whether you’re one of millions using Cardboard, or kicking back with the best games and 360 videos on Daydream, or playing soccer with Chelsea stars on Tango, you can have incredible experiences in virtual and augmented reality that are hassle-free, comfortable, and fun.

With Daydream, you can get your game on with a mobile VR headset that's comfortable to wear and a controller that's simple to use. Whether you love to play first-person battles in space, solve complex puzzles, or make real friends in virtual spaces, Daydream has you covered. Here at the Game Developers Conference (GDC) in San Francisco, we just announced more great VR titles joining our gaming lineup on Daydream from top developers.

Ubisoft and some of the original creators of Rabbids are teaming up to bring you Virtual Rabbids. For the first time ever, you’ll be able to communicate and interact directly with the larger-than-life Rabbids embarking on new wacky adventures. This lighthearted game makes full use of the controller to show what Daydream can do, while capturing the humor and antics of the original Rabbids series.

Next we have Beartopia from Spry Fox, who won Standout Indie at the inaugural Google Play Awards in 2016 for their word puzzle game, Alphabear. Beartopia is a fun co-op village game that will bring you together with other players so you can form friendships and build a bustling and prosperous society. It will be Spry Fox’s first foray into VR, and we love indie games like this that push the envelope of what’s possible and bring new ideas to growing platforms. Indies have long been a key pillar of Google Play, and we’re excited to see that continue on Daydream.

Find your true calling in Virtual Virtual Reality by Tender Claws. Try on different VR headsets, and the Virtual Labor System will help you identify a suitable artificial intelligence client match.

Finally, Along Together by Turbo Button is a game about being an imaginary friend. Using the Daydream controller, you’ll help your pal safely navigate treacherous 3D puzzles.

These titles are all coming to Google Play soon. In the meantime, to celebrate GDC, some of the top Daydream games will be on sale for a week starting on March 1: grab Gunjack 2: End of Shift, Need for Speed: No Limits VR, and Keep Talking and Nobody Explodes for half price.

Anyone can make apps and games on Daydream, and there are more than 100 experiences available on Google Play. If you’re here at GDC, check out our sessions and stop by our Daydream booth #1424 for demos and a chance to learn more about our developer tools.

Virtual reality lets you experience amazing things—from exploring new worlds, to painting with trails of stars, to defending your fleet to save the world. But, headsets can get in the way. If you're watching someone else use VR, it's hard to tell what's going on and what they’re seeing. And if you’re in VR with someone else, there aren’t easy ways to see their facial expressions without an avatar representation.

Daydream Labs and Google Research teamed up to start exploring how to solve these problems. Using a combination of machine learning, 3D computer vision, and advanced rendering techniques, we’re now able to “remove” headsets and show a person’s identity, focus and full face in mixed reality. Mixed reality is a way to convey what’s happening inside and outside a virtual place in a two dimensional format. With this new technology, we’re able to make a more complete picture of the person in VR.

Using a calibrated VR setup including a headset (like the HTC Vive), a green screen, and a video camera, combined with accurate tracking and segmentation, you can see the “real world” and the interactive virtual elements together. We used it to show you what Tilt Brush can do and took Conan O’Brien on a virtual trip to outer space from our YouTube Space in New York. Unfortunately, in mixed reality, faces are obstructed by headsets. 

The first step to removing the VR headset is to construct a dynamic 3D model of the person’s face, capturing facial variations as they blink or look in different directions. This model allows us to mimic where the person is looking, even though it's hidden under the headset.

Next, we use an HTC Vive, modified by SMI to include eye-tracking, to capture the person’s eye-gaze from inside the headset. From there, we create the illusion of the person’s face by aligning and blending the 3D face model with a camera’s video stream. A translucent "scuba mask" look helps avoid an "uncanny valley" effect.

Finally, we composite the person into the virtual world, which requires calibrating between the Vive tracking system and the external camera. We’re able to automate this and make it highly accurate so movement looks natural. The end result is a complete view of both the virtual world and the person in it, including their entire face and where they’re looking.

Our initial work focused on mixed reality is just one potential application of this technology. Seeing beyond VR headsets could help enhance communication and social interaction in VR. Imagine being able to VR video conference and see the expressions and nonverbal cues of the people you are talking to, or seeing your friend’s reactions as you play your favorite game together.

It’s just the beginning for this technology and we’ll share more moving forward. But, if you’re game to go deeper, we’ve described the technical details on the Google Research blog. This is an ongoing collaboration between Google Research, Daydream Labs, and the YouTube team. We’re making mixed reality capabilities available in select YouTube Spaces and are exploring how to bring this technology to select creators in the future. 

Virtual reality lets you experience amazing things—from exploring new worlds, to painting with trails of stars, to defending your fleet to save the world. But, headsets can get in the way. If you're watching someone else use VR, it's hard to tell what's going on and what they’re seeing. And if you’re in VR with someone else, there aren’t easy ways to see their facial expressions without an avatar representation.

Daydream Labs and Google Research teamed up to start exploring how to solve these problems. Using a combination of machine learning, 3D computer vision, and advanced rendering techniques, we’re now able to “remove” headsets and show a person’s identity, focus and full face in mixed reality. Mixed reality is a way to convey what’s happening inside and outside a virtual place in a two dimensional format. With this new technology, we’re able to make a more complete picture of the person in VR.

Using a calibrated VR setup including a headset (like the HTC Vive), a green screen, and a video camera, combined with accurate tracking and segmentation, you can see the “real world” and the interactive virtual elements together. We used it to show you what Tilt Brush can do and took Conan O’Brien on a virtual trip to outer space from our YouTube Space in New York. Unfortunately, in mixed reality, faces are obstructed by headsets. 

The first step to removing the VR headset is to construct a dynamic 3D model of the person’s face, capturing facial variations as they blink or look in different directions. This model allows us to mimic where the person is looking, even though it's hidden under the headset.

Next, we use an HTC Vive, modified by SMI to include eye-tracking, to capture the person’s eye-gaze from inside the headset. From there, we create the illusion of the person’s face by aligning and blending the 3D face model with a camera’s video stream. A translucent "scuba mask" look helps avoid an "uncanny valley" effect.

Finally, we composite the person into the virtual world, which requires calibrating between the Vive tracking system and the external camera. We’re able to automate this and make it highly accurate so movement looks natural. The end result is a complete view of both the virtual world and the person in it, including their entire face and where they’re looking.

Our initial work focused on mixed reality is just one potential application of this technology. Seeing beyond VR headsets could help enhance communication and social interaction in VR. Imagine being able to VR video conference and see the expressions and nonverbal cues of the people you are talking to, or seeing your friend’s reactions as you play your favorite game together.

It’s just the beginning for this technology and we’ll share more moving forward. But, if you’re game to go deeper, we’ve described the technical details on the Google Research blog. This is an ongoing collaboration between Google Research, Daydream Labs, and the YouTube team. We’re making mixed reality capabilities available in select YouTube Spaces and are exploring how to bring this technology to select creators in the future. 

Whether you’re a first time doodler tracing lines of fire and stars against the night sky or a concept artist designing a set for a film, the possibilities are endless when you paint in virtual reality with Tilt Brush.

Starting today, Tilt Brush is available on the Oculus Rift in addition to the HTC Vive. We brought it to Oculus Rift so more of you with PC-powered systems can create and experience works of art in VR. No matter what you decide to make in Tilt Brush, painting should be natural, comfortable and immersive. So, we thought a lot about how to customize the app for Rift’s platform, hardware, and Touch controllers:

• In order to make it more convenient to paint, we recently added features that let you rotate and resize your work. 
  
• We redesigned interactions to take advantage of the Oculus Touch controllers. For example, you can easily highlight which button you're touching on the controller and get an indication of what it does just by resting your finger on it. This makes it easy to see exactly what button you're about to press while using Tilt Brush.
  
• Painting isn’t just visual. Thanks to the Rift’s built in headphones, you’ll be fully immersed from the moment you enter Tilt Brush's virtual canvas. Different brushes create different sound effects, and they become a vivid part of the experience through your headphones. We love using audio reactive mode with Rift headphones and seeing strokes come to life with light and sound.

So, if you have an Oculus Rift and Touch controllers, Tilt Brush is available now. And if you need inspiration for getting started, have a look at some of the creations from our Artist in Residence (AiR) program, many of which you can access right from the app. Happy painting!

Whether you’re a first time doodler tracing lines of fire and stars against the night sky or a concept artist designing a set for a film, the possibilities are endless when you paint in virtual reality with Tilt Brush.

Starting today, Tilt Brush is available on the Oculus Rift in addition to the HTC Vive. We brought it to Oculus Rift so more of you with PC-powered systems can create and experience works of art in VR. No matter what you decide to make in Tilt Brush, painting should be natural, comfortable and immersive. So, we thought a lot about how to customize the app for Rift’s platform, hardware, and Touch controllers:

• In order to make it more convenient to paint, we recently added features that let you rotate and resize your work. 
  
• We redesigned interactions to take advantage of the Oculus Touch controllers. For example, you can easily highlight which button you're touching on the controller and get an indication of what it does just by resting your finger on it. This makes it easy to see exactly what button you're about to press while using Tilt Brush.
  
• Painting isn’t just visual. Thanks to the Rift’s built in headphones, you’ll be fully immersed from the moment you enter Tilt Brush's virtual canvas. Different brushes create different sound effects, and they become a vivid part of the experience through your headphones. We love using audio reactive mode with Rift headphones and seeing strokes come to life with light and sound.

So, if you have an Oculus Rift and Touch controllers, Tilt Brush is available now. And if you need inspiration for getting started, have a look at some of the creations from our Artist in Residence (AiR) program, many of which you can access right from the app. Happy painting!

Can't get enough dunks, three pointers, and last-second jumpers? Experience the NBA in a whole new way with the new NBA VR app, available on Daydream.

Catch up with highlights in your own virtual sports lounge or watch the NBA’s first original VR series, “House of Legends,” where NBA legends discuss everything from pop culture to the greatest moments of their career. The series tips off today with seven-time NBA Champion Robert Horry. New episodes featuring stars like Chauncey Billups and Baron Davis will debut regularly.

Daydream gives sports fans a new way to connect to the leagues, teams and players they care about most. The NBA VR app joins a lineup that already includes:

• NFL VR: Get access to the NFL Immersed series featuring 360° behind-the-scenes looks into the lives of players, coaches, cheerleaders, and even fans themselves as they prepare for game day.

• MLB.com Home Run Derby VR: Hit monster home runs with the Daydream controller in eight iconic MLB ballparks and bring home the ultimate Derby crown.

• NextVR: From NBA games and the Kentucky Derby, to the NFL and the US Open, experience your favorite sporting events live or revisit them through highlights.

You're just a download away from being closer than ever to the sporting events and athletes you love!

Can't get enough dunks, three pointers, and last-second jumpers? Experience the NBA in a whole new way with the new NBA VR app, available on Daydream.

Catch up with highlights in your own virtual sports lounge or watch the NBA’s first original VR series, “House of Legends,” where NBA legends discuss everything from pop culture to the greatest moments of their career. The series tips off today with seven-time NBA Champion Robert Horry. New episodes featuring stars like Chauncey Billups and Baron Davis will debut regularly.

Daydream gives sports fans a new way to connect to the leagues, teams and players they care about most. The NBA VR app joins a lineup that already includes:

• NFL VR: Get access to the NFL Immersed series featuring 360° behind-the-scenes looks into the lives of players, coaches, cheerleaders, and even fans themselves as they prepare for game day.

• MLB.com Home Run Derby VR: Hit monster home runs with the Daydream controller in eight iconic MLB ballparks and bring home the ultimate Derby crown.

• NextVR: From NBA games and the Kentucky Derby, to the NFL and the US Open, experience your favorite sporting events live or revisit them through highlights.

You're just a download away from being closer than ever to the sporting events and athletes you love!