How We Created Back2Pack, Our Holiday VR Game

Unfortunately, we faced some problems with tracking gestures through the controller right from the start. The thing is, the quality of accelerometers in phones usually leaves much to be desired. The data we received from them varied for different devices and had a lot of noise. So, we had to stick to a single gesture that was easy to track - a hand swing.

This made us change the game concept and rename it as Back2Pack. Wondering where the name comes from? In the new version of the game, the player had to throw elves into the runaway toys to quickly pack them and then send them to kids as Christmas presents.

But the result was still far from what we expected it to be. There was a delay between swinging the controller and throwing an elf in the game since we tracked the full gesture from beginning to end and then sent a message about the event via web sockets.

But often the easiest solutions are the best. So, instead of tracking the full gesture, we simply tracked the acceleration value of a phone swing. When the acceleration acquired a certain value (we picked them by experimenting with different phones) on one of three axes, the system understood that the user had swung an arm to throw an elf and sent a message about the event in advance. That’s how we achieved good gameplay and got rid of the laggy feeling between swinging the controller and throwing an elf in the game.

WebGL

In WebGL everything is about performance and memory. You plan to do a lot of cool things, but very soon you can reach the GPU limit. So, you have to be aware of optimization every single moment, regardless of whether you think a certain part will be “heavy” for the processor or not. You also have to constantly improve everything you can: the less you calculate and render, the higher frame rate you get.

Luckily, the WebGL development part of the project was fairly trivial: simply create an engaging holiday VR game with a fascinating presence effect environment and make it all work on 60 fps in mobile browsers on WebGL-compatible devices (at least on more or less modern ones) - no biggie.

The first step was to create the winter environment and landscape. We decided to make it realistic with a parallax effect. But how do you make it detailed and maintain the polygon count? Yep, by cheating. We made two cylinders of different widths and generated textures for them: with an alpha channel and low mountains for the narrower one and with sky gradient and blurred higher mountains for the wider one. Next, we added a parallax effect by slightly rotating one cylinder along with the camera. The view from above was not that impressive…

…but it was quite nice from the player’s point of view:

We needed to create a relief. We did not have much time to make models of levels in a CAD program; plus, they would look the same every time the game started - b-o-o-o-ring! So, we decided to generate them dynamically.

First of all, the landscape height changes had to be smooth. It sounds like a kind of mathematical function. We used the CatmullRomCurve3 component for generating user path, and positioned all objects including the relief according to this curve’s points. So, we created a big plane geometry, calculated its triangles vertices positions according to the curve point coordinate and set narrow range colors for those triangles. That is how we received a relief with smooth hills:

And... it looked rather ugly. A kind of a parquetry landscape, not the snowy view we were looking for. Plus, the unrealistic colors... We needed light to calculate shadows for a more realistic view. But calculating light for that huge amount of triangles is costly. Soooo… Yup, cheating again. We calculated triangle normals and used that data to set the colors range. In that way we created a more natural view and avoided resource intensive light calculation:

Then we needed some objects – trees, rocks, branches, stumps, etc. We used some 3D models, locating them according to the path coordinates. In this case, we did not need to fill up the whole plane, but only the area near the path the user would run through.

To reduce loading time, we decided to use only one model of a tree and then randomly rotate and scale it in some diapason to make it look different. To provide some more visual variations, we changed the color of the tree. Using various colors and textures, we managed to make the forest look less “flat” and more realistic.

To avoid light mapping and all the other additional calculations, we “baked” shadows into textures. We also had to fill the horizon area, because there were some gaps there. To do this, we created a “fake” ground - just a white plane area outside the user path.

As a result we received a winter forest landscape with a clear user path:

The next step was about adding the user’s movement. There were two options: moving the camera through the space, or moving all the objects instead, with the camera left in its initial position. We decided to choose something in between these two variants: the plane moved towards the user, simulating the run, while the camera rotated and slid along the width and height axes, following the path’s coordinates and angle.

To provide an experience of infinite movement along the plane, we created an additional plane with a randomly set user movement path. We adjusted the relief accordingly, placed some landscape objects on it and set the plane lengths to match the environment size. To hide the joint between the two planes, we set their beginning and end relief heights to the same values. We also adjusted plane positions for every frame. If the plane was behind the horizon at the user’s back, we simply put it behind the horizon in front of the user as well, re-generating the relief and landscape objects. The objects were positioned randomly, and we decided to reuse them instead of generating new ones.

Next, we needed to add animated toys. First, we decided to load DAE animation files, because they were lightweight. But there was one problem. According to our gameplay concept, we needed to duplicate animated objects. In other words, we needed to have multiple similar toys; however, the DAE animations did not duplicate correctly. In three.js, DAE animation is a group of objects. After we tried to create duplicated characters, the animation failed to play correctly and there was also a problem with the dynamic texture changes. We did not find the solution to this issue, so we changed the animation format to JSON. The only one animation left in the DAE format was the deer animation, because it did not need to be duplicated.
At that point we were ready with all the objects we needed for the game, the toys and all the other visual stuff:

It was time to develop the catch algorithm. According to our gameplay scenario, the user had to throw an elf to hit a running toy, so the elf could catch and pack it. The common technique for implementing this interaction was ray casting. But this method is rather resource consuming, especially for multiple objects. So, we calculated the distance to every toy instead, and if it was small enough, we knew, that the elf was really close enough to catch that toy. We added some cool “fight cloud” animation and then the animation to show the process of packing to make the game more interactive and exciting.

At last, ladies and gentlemen, our holiday VR game was ready!

We were about to say: watch out, Angry Birds, we are coming! But, unfortunately, we faced one more problem with reference to the display performance for the iPhone 6. The reason was that we had too many landscape objects. If you remember, we tried to reduce the memory footprint and the loading time. Therefore, we loaded only one model geometry and randomized it.

At the same time we created over 400 objects and added them to the scene. And every single object presupposed an extra draw call. We ended up with the additional 400 draw calls for every frame while the objects did not change. As a result, we had to merge the objects to reduce the number of draw calls. Simply packing objects into container-objects did not work. We had to merge geometries. In our case, we had a single tree geometry, but we needed a forest geometry instead.

So, we created an empty geometry and copied our tree geometry into it with the corresponding offset, rotation and scale. After that, we set the tree geometry properties and copied them to the forest geometry. We had to do this as many times as there were trees on the plane. Next, we received one big forest geometry, drawn by a single draw call. As a result, we managed to double the display performance indicators.

And here is a valuable tip for developers. When the game starts, the player looks forward through the deer’s antlers. But it looks different on Android and iOS devices. On iOS, every time you start a game and initialize the accelerometer controls component, the player looks straight just like you need. But on Android the initial rotation angle depends on the phone’s rotation. So, on Android you have to save the initial rotation value on the first frame of your game into a variable and then count it for every frame afterward.

3D

All character models for the game were created in Maya using simple polygonal modeling tools. But due to technical limitations of our game engine and target devices, the models were very low-poly - less than 1,000 triangles and had very small textures of 128x128 pixels. The most detailed texture parts were the characters’ faces.
(Monkey Model)

Most characters were created in the “T” pose to make the rigging and skinning processes smoother.
(Skinning process)

Inside the game engine, we used morphing for character animation. But before baking morph targets, we created skeleton and control rigs in Maya for all characters.

After that, we created motion cycles for the characters and baked animation into the format acceptable for the game engine.

Animation Hacks

Although the WebGL environment had the leading part in developing this game, everything started with the intro markup. Adding HTML and CSS was not a big deal, even though we did have a lot of uncommon elements such as polygon borders. However, the tough part was having to handle plenty of animations; so, we had to think about them from the very beginning.

As you know, every animatable element usually has a different starting position, does its thing and takes another position. We wanted to have full control over that. That is why we decided not to use native CSS animations and instead work with the JavaScript animation library Velocity.js. Using it, we could tune animations much easier and not care about any changes that could influence other relevant elements. You see, the big problem with CSS animations is their sequence. Particularly, the lack of it. There are techniques that help you solve the issue, but you still will not get as much flexibility as you can have with Velocity.js. Clearly, that is why we used it to animate almost all of the game’s HTML stuff.