Project Caliper: the journey of prototyping optimal controllers for VR


Today is a special day in Italy when families prepare the Christmas Tree, so I decided to publish a very special post that is a big gift to the whole XR community. But the gift is not mine, but from the XR ergonomics expert Rob Cole, that has written an INCREDIBLE guest post about his experiments in trying to build from scratch the prototype of a SteamVR controller with a modular design and optimal ergonomics. The post is very long, but also super-interesting, especially if you are a maker or love DIY stuff.

Rob loves exploring the technology always starting from the human perspective, putting the human at the center of his experiments and analysis, and he did this also while working on Project Caliper, his experiment in building a better version of the Valve Index Controllers alone in his spare time. I think the final result of this post is pretty intriguing, and I invite you all to get in touch with Rob over Twitter, or e-mail (asking me for an introduction) to discuss potential collaborations (or funding?) with him. I even hope some big brands can take inspiration from some of the features he has introduced in the Caliper controllers.

That said, inviting you to support this blog on Patreon, so that I can keep hosting these amazing articles, I leave you to Rob’s post, that is like a tiny book, on the journey of developing Project Caliper. Enjoy your read

(Image by Rob Cole)

Introduction

Following my article on the Tundra SteamVR HDK published by Skarredghost earlier this year, I embarked on the development of a new motion controller by launching start-up Immersion Mechanics. Using my background in Industrial Design, years of experience in ergonomics, plus the past five years spent evaluating and hacking XR equipment, I decided to give it a good shot.

Following the sage advice “don’t give up the day job” I continued to work full time managing a technical servicing facility for a Global manufacturer in an unrelated industry. Which proved incredibly useful, giving me 24-hour access to fully equipped service workshops, plus a secure outdoor space for paint spraying or working with adhesives and plastics.

Finding a balance between working full time whilst trying to find as many spare hours as possible for development work and still finding time for family, proved very tricky. I quickly learned that at least 3 hours needed putting aside to get anything meaningful done, otherwise you need to finish up just as you get going.

Going into the workshop at 4 am to carve wooden grip forms, spending my day off moulding plastics, or learning Blender at 5am in the morning was really challenging. It took firm discipline to keep moving forward as it was all too easy to slack off by working too infrequently to make any real progress.

(Image by Rob Cole)

The ‘ground-hog day’ vibe of lockdown life during the Pandemic caused the days to blur together… our facility was classed as an essential service so we stayed open, giving me ongoing access to the workshops, but at the cost of working super long hours whilst many were parked at home on paid furlough.

Initially, I had planned to put my money into building a working prototype and then seek further funding to bring my new SteamVR controller “Caliper” to the consumer market alongside Index and Wand controllers; but unfortunately, the World had other plans.

Lockdowns and shortages

The ongoing impact of Covid, huge demand for electronics, labour shortages, power cuts in China leading to 3-day-working in factories, and the ongoing container shipping lottery have broken many “just in time” supply lines. This has led to long manufacturing delays and component shortages, especially for the silicon processors used in everything from electric cars and smartphones to game consoles and VR equipment.

Recently, Valve announced delays to their new Steam Deck handheld, whilst Megadodo Games announced a delayed, limited production run of their DecaGear before leaving the hardware market completely. New PlayStation and Xbox consoles are still very thin on the ground, whilst the PC graphics card shortage has no end in sight, with gamers still suffering overinflated prices on Nvidia and AMD products.  It wasn’t even possible to buy another Tundra HDK, and months later they are still showing out of stock.

(Image provided by Rob Cole)

Even the big players like Sony, Apple, and Microsoft have substantially reduced their production estimates for 2022, whilst processor manufacturers talk of delays into 2024. It’s no secret that the largest players have paid handsomely to secure component supplies and manufacturing capacity in a bid to insulate their businesses from further shocks.

This problem is hitting everyone, from start-ups to well-established players with deep pockets; as an independent hardware developer quickly burning through my money, I had to make the hard decision during the middle of this year, deciding to stop before I got any deeper into what was becoming an unwinnable situation.

The financial reality of hardware development quickly became evident; it was very easy to spend huge amounts of money on planning, development, and pre-production before even making anything to sell. Throwing thousands away on an attempt unlikely to succeed was starting to look like a grim reality, I didn’t want to be another one of those stories…

Old hardware joke: How to make one million dollars? Start with two million…

Just to register the PID license (required for any new USB connected device) was a mere £5000; the cost of custom flexes, circuit boards, and injection mould tooling meant remortgaging the house, whilst licensing Immersion Corporation’s haptics technology wasn’t inexpensive.

People had not been wrong about hardware being expensive and difficult to develop; console manufacturers often claw back hardware costs through software sales, when they own the storefronts and take a cut of everything sold digitally (often 30%).  

With Meta and Steam dominating VR software sales in the consumer market, software sales weren’t really a viable source of income for other hardware manufacturers – hence HTC pivoting to enterprise hardware with higher pricing, whilst potentially making more lucrative profit from their digital Enterprise services and Viveport consumer subscription model.

This could explain why the Wand and Index motion controllers are still the only choices for SteamVR, despite the Vive launching to consumers back in 2016.  The Etee and Pimax Sword SteamVR controllers are long delayed, whilst other attempts have come and gone without further trace.

(Image by Rob Cole)

SteamVR tracking components do add a noticeable cost to the bill of materials, compared to camera and algorithm-based solutions, yet you don’t have any profitable software to sell to offset your hardware costs. I considered the Valve Index controllers a very reasonable price at £129.50 each (£259 for pair with free shipping), considering the insane level of complexity and the sheer number of sensors packed into each device.

With LG charging me £60 for a simple TV remote control produced at mass scale, it was hard to see if Valve was making any profit selling their Index controllers? Their business-to-consumer (B2C) sales channel was eliminating any middleman and their margin, somewhat supporting this observation. As Valve makes its money from software sales, it’s possible the Index was covering its costs, but not particularly profitable as it didn’t actually need to be. 

It’s been said that hardware is merely the delivery system for great content, and a necessary evil in many ways, that many would prefer to let someone else deal with.  As Sumner Redstone repeatedly said about television programming during his time leading Viacom, “Content is king!”

Time for my reality check

Frank discussions with a mentor (who has a portfolio of patents and experience bringing products to market) led to us looking at the estimated setup costs, potential sales numbers based on best-known data, current Global risks to supply and manufacturing, escalating transport costs.

Established players with deep pockets were really struggling with the ongoing uncertainty, whilst in my real job we were seeing months of delays for stock and warranty replacements, compounded by ongoing QA issues caused by third-party component defects or from factory assembly errors; everyone was struggling to get stuff out, and some stuff was arriving broken.

We also looked at how the rapid growth of the Meta Quest platform was siphoning software development money and hence future consumer interest away from using SteamVR hardware, whilst Valve maintained their customary radio silence with little said about VR since the HLA launch.

YouTuber Bradley Lynch (Sadly Its Bradley) did painstaking research, confirmed by Ars Technica using contacts at Valve, of a new Valve VR system referred to as “Deckard”. A big tranche of patents was released earlier this year following the termination of a lawsuit against an optics supplier they’d made a $9 million investment into back in 2018.

(Image by Rob Cole)

But as with all these things: just a legal action, some patents about onboard processing, polarising optics, and different head straps, several code leaks containing references to Deckard, but radio silence from Valve (nothing unusual here then), the hype train had left the station and run out of…Steam (sorry!)

The VR community continued waiting for any further news on an Index successor: the 2017 BOE display panels are overdue for an update, and some improved joysticks and 802.11ay wireless module would be truly appreciated by many.

Sadly, our numbers for bringing a new SteamVR controller to production just didn’t add up, I’d already had a strong suspicion this was to be the case, but needed to hear it from someone with more experience. Following my welcome to reality check, I thanked my mentor and shut down my little venture before I spent any more of my own money.

XRCaliper

I’d had too much fun already to completely quit, and decided to just start building fun stuff at a more relaxed pace as I’d already been burning the midnight oil for a couple of months now.

Why not give it a name, so XRCaliper it was. I set up a simple website www.xrcaliper.com and a Twitter account @XRCaliper to connect with the World.

(Image by Rob Cole)

As this year continued, I tackled different development areas (Tracking, Ergonomics, Haptics, Mechanical, Software, etc.) resulting in the construction of a number of development “mules” before starting work on Caliper itself. Here’s some of the story…

Steamworks

After receiving my Tundra HDK earlier this year, my second step was to sign up with Steamworks as a licensee courtesy of Valve, at no additional cost. I was now officially a SteamVR developer, and free to build games or hardware for release on the Steam platform.

Due to the terms of the license, I am not permitted to discuss certain technology in detail, but if any readers are interested there is nothing to stop them from also signing up to Steamworks at this link: https://partner.steamgames.com/.

Becoming a licensee gave me access to the SteamVR Tracking HDK and Steam SDK, which also contained the original 2016 SteamVR training course documentation which was run for a couple of years in conjunction with Valve partner Synapse in Seattle.

This training course cost about $3,000 and was mandatory for anyone wanting to use SteamVR. This fee had included a suite of useful reference hardware to build tracked objects with, which could also be disassembled and reused to build prototypes.  

(Image by Rob Cole)

However, the decision was made in 2017 to drop the course requirement (it was still made available if required), and open up the training materials and examples for free to encourage the growth of the SteamVR ecosystem.

Whilst this had the benefit of dramatically lowering the cost of entry ($3,000 for the course versus $149.99 for the HDK without a dongle), the lack of any direct support and reliance on outdated course materials (base stations evolved to 2.0 and there has been the introduction of SteamVR Input System) meant it was more difficult to understand building with the technology, especially for a novice hardware developer like myself. 

Tundra HDK RMA

After reading everything I could several times and back to front (just in case) I decided to get on with firing up the Tundra HDK. This is done using the “Lighthouse Console” which is a command-line interface with a list of commands used for identification, pairing, diagnostics, etc.

(Image by Rob Cole)

I tried to connect the HDK to my PC, and then to pair the wireless module with a Vive wireless dongle (USB) I purchased with the HDK (TL448K6D-GP-HDK with dongle $179.99). Wireless communication is on the 2.4Ghz band using a modified version of Nordic’s Enhanced Shock Burst platform.

However, I kept on getting connection problems, with the HDK searching for the wireless connection, always refusing to pair. Concerned I had done something wrong I went through all my information and tried setting it up many times, trying different USB ports for the Dongle and holding the HDK right next to the Dongle. The HDK connected every time through USB, but just wouldn’t work with the wireless connection.  

I reached out to Tundra, and after some back and forth they advised it was probably a fault with the wireless module itself on the HDK. Tundra Labs apologised and arranged a replacement HDK which was personally tested by Tundra boss Luke Beno (!). Within a week I had a brand new Tundra HDK, whilst keeping the faulty HDK as shipping it back to the USA was pointless. Overall it was a great customer service experience! 

(Image by Rob Cole)

What had originally left me feeling like a bit of a hardware dummy and had temporarily paused proceedings, now offered a bonus opportunity for further hardware tinkering. Using a long USB cable would let me continue to use the faulty HDK with 6DoF tracking but tethered to the PC around the desktop.

It was now possible to build:

A wired SteamVR object using the faulty HDK, for testing a tracked object model with one base station around a desktop environment

A wireless SteamVR object using the new Tundra HDK, for testing a working controller prototype with two base stations across a Roomscale environment.

The tracked object

I figured I might as well have some fun with my tracked object, as it was possible to 3D print the reference design (UFO) used on the SteamVR training course, but I preferred to go my own way figuring I could learn more by starting from scratch. In hindsight, it’s probably worth doing it their way to save yourself a headache, unless you like maths and hacking.

After having used many different motion controllers in the past 5 years, often variations on a theme, I wondered if there was a different solution. Why had they settled on this shape? Why not try something different, how about a gun grip? Instead of building a grip, I could purchase something readily available online. Amazon sold me a moulded nylon grip for an AR-15 airsoft rifle… it was time to start work building my first development “Mule”.

Mules” are testbeds often equipped with working components to allow direct evaluation during early development. They might be dumb – fitted with switches and sticks that are mechanically operable, or live – with inputs powered up and connected through GPIO (General-Purpose Input/Output).

Now I had the wired HDK for testing a tracked object and the new wireless HDK for an eventual controller prototype. I would need some input components such as buttons, triggers, trackpads, joysticks, and eventually a microcontroller to run them when all wired up for prototyping. Some haptic motors and light-emitting diodes would also be useful to provide users with feedback from the controller.

COTS to the rescue

Why not use existing parts, already in mass production? Using COTS (commercial off-the-shelf) parts can save huge amounts of time and money, compared to using proprietary parts which are made specifically for the design with high setup costs. As the old saying goes…no point in reinventing the wheel?

Delving into the world of gamepads I found several companies offering “replaceable” parts for their products, some aimed at the budget end of the market, though a couple of the reputable big brands were targeting the high-end gamers.

Thrustmaster’s X Pro E-Swap gamepad looked very suitable, despite costing an outrageous £220 with shipping and tax from brand owner Guillemot’s warehouse in France. As a bonus, a thriving ecosystem of spare input parts was readily available for consumers, with “hop up” kits offering different input components, colours, textures, and styles.

This gamepad is modular, made of easily replaceable units (Image by Thrustmaster, provided by Rob Cole)

A replacement joystick module was less than £20, which could be swapped with a D-Pad or Touchpad to allow the user to fully customise their setup. I also opted to buy a blue hop-up kit for £40 which came with a replacement joystick, d-pad, a pair of side grips, and a pair of trigger covers. 

Immediately this modular design brought a big benefit: it could be possible to set up the left controller with a joystick, the right with a touchpad, or however you wanted, you could use D-Pad on both controllers if you really wanted.

After weeks of shipping delays (thanks Brexit), my Thrustmaster gamepad arrived, and it was a beautiful-looking product, no doubt.  Before pulling it all apart (!!) I had a couple of VR sessions in Project Cars 2, House of the Dying Sun, and Aircar to test the quality of the input components. I was not disappointed, with the triggers, joysticks, and buttons easily the best I’d ever tried, the metal joysticks making my other gamepads and VR motion controllers feel like cheap toys in comparison.

The E-Swap system used strong magnets to retain the modules inside special receptacles with spring-loaded contacts and worked very well with no free movement felt at the joystick or d-pad.  Thrustmaster included software with button mapping, E-Swap input setups, and full adjustment of the joysticks, making me think about software support for a future VR controller with similar levels of customisation.

(Image by Rob Cole)

However, all was not rosy, with the gamepad feeling very heavy and a little small in the hands (causing it to try and slide down) exacerbated by a long braided USB cord (it was not wireless) which made the gamepad feel even heavier when moving around.

Most noticeable though, which was a big distraction in VR, was constant small movements and clicking noises from magnetic “side grips” on the controller body. The grips did not line up very well with their mounts, and the magnets were much too weak to arrest any movement.

I’d seen this problem before with the Valve Index headset, which also used magnets to retain the face gasket – it wasn’t uncommon for the face gasket to fall from the headset as the magnets needed to be much stronger.

As a paying gaming customer, I would have sent it straight back for a refund, but I was more interested in ripping out its input components so it was time to strip the Thrustmaster gamepad down; I got busy with my screwdrivers, cutting tools, and soldering iron. It was super interesting to study their manufacturing and design, I took lots of photographs and made notes.

(Image by Rob Cole)

Now armed with a sketch pad and pen, my wired Tundra HDK, a head full of ideas, a box full of spare Thrustmaster input parts, and my AR15 grip, I got to work trying to figure out how to build my tracked object, which I had already decided should be a handheld controller of some sort.

“Gungrip”

This was built to work out the space required for a joystick, buttons, and trigger, offering 12mm of fore / aft adjustment between the grip and joystick to allow the user to set the ideal reach distance to suit their hands. It had no sensors, just a simulated sensor bar in front of the handgrip.

(Image by Rob Cole)

The amazing Thrustmaster X-Pro trigger and S5 joystick with adjustable position already made this feel really cool to try despite it just being a dumb old mule. Now that I had something that felt good in the hand, I needed to work out how to integrate the HDK, but for that, I needed extra space so I temporarily ditched the joystick.

“T-Bar”

This was quickly built to understand any practical problems of working with the HDK, specifically sensor placement and flex management. Locating the sensor “centroid” accurately was challenging; I used a number of commercial adhesive products trying to find the best solution that was stable but not permanent as I planned to remove the sensors at some point in the future.

The HDK was mounted underneath the main body, powered by a large rechargeable lithium-ion battery in a quick-release mount at the rear, so I could try hooking up a battery to the HDK  though the wireless connection was dead requiring the controller to be connected and powered via USB. The battery also balanced the weight of the front tracking bar making the controller feel neutrally weighted in the handgrip. Overall it had a good amount of weight making it feel more like a real pistol.

(Image by Rob Cole)

This was connected by a USB cable to the Lighthouse command-line console on my PC providing IMU data and light sensor hits to be viewed in real-time… it was very cool to see this live and understand something was actually happening here! It was enjoyable moving my controller around and seeing the reaction in the IMU data, or obscuring different sensors to see the data stream drop out. 

At this point, no sensors were mapped to the model and the IMU (gyro) was not calibrated so the data was quite random, but it was encouraging as a starting point and let me start learning what the different Lighthouse console commands did when testing tracked objects. Some examples of commands:

identifycontroller: Trigger haptic pulses on the active serial number to identify it

imu:Toggle IMU data packet dumping

imustats: Print IMU statistics

period: Print sync statistics

The mule felt very balanced in the hand... perhaps could it be a good choice to use as a tracked gun controller in shooting games like Pistol Whip? Intensive gun play over several weeks in Pistol Whip had done a great job cracking the grip mounting plate on my right Index controller, so a dedicated peripheral could be most welcome. 

(Image by Rob Cole)

A large haptic motor mounted inside the empty grip would provide substantial feedback whilst the large-capacity battery would ensure long run times between charges.

Built with a single trigger to allow a simple input, I soon realized the horrible complexity of calculating the physical sensor locations and somehow translating them into a 3D object that I still needed to learn the software to even start building, so moved on to a much simpler idea.

“Guncube”

After stripping down the T-bar mule, I tried using several small cardboard boxes before finding just the right size to use as a base for building a simple tracked cube object. I took measurements and placed sensors in those locations so I knew roughly where the sensors were located.

The box was thoroughly taped to stabilise the surface against damp or movement as much as possible. The excess flexes for each sensor were allowed to hang freely inside the box, removing the headache of cable routing, channeling, or taping on the exterior.

Handle of the cube, sensors installed on the cardboard cube, and interior of the cube (Image by Rob Cole)

This ensured the most realistic chance of actually getting something working built, though I soon realised I needed more sensors and importantly more “non-co-planar” sensors (not all mounted on the same plane or surface) to get it to boot quickly and then maintain accurate 6DoF tracking across a range of useful interaction angles.

The SteamVR course literature contained some great design tips championing the use of facets to break up surfaces and using wide spacing between sensors on the same plane (base line); ideally to cover the object in multiple sensors at many angles, as far apart as possible.

However I really needed to go through the development process even making a big mess just to understand what was needed, so here it was, incoming tracking problems or not.

Skarredghost then suggested the name “Guncube” which sounded ideal, thanks Tony!

The Guncube. The name was of course inspired by “The Unity Cube” application (Image by Rob Cole)

After building my physical model, this time around I also built a basic CAD model and used Valve’s SteamVR HDK tools for the first time to analyse sensor placement and develop a deeper understanding of the process from start to finish.

The process started with first building a CAD model: I built a simple model with my tracking cube and a shape representing the volume of the handgrip. I used a free application recommended by Valve called OpenSCAD, after going through the tutorial to get a basic understanding of how some of it worked, as it uses text-based language:

OpenSCAD is a descriptive, text-based programming language that uses a familiar syntax, if statements, for loops, variables, and arithmetic to combine simple geometric primitives into complex objects. https://programmingwithopenscad.github.io/

The Guncube inside Openscad (Image by Rob Cole)

After creating my Guncube object, I created a simple CAD model of the Triad TS4112 optical sensors used by the Tundra HDK. The sensor model is used when placing many identical sensors across the surface of the object during simulation, a little later in the process. 

A broken sensor from my HDK is shown below, the shiny black area is the “sensor window” which receives incoming laser light projected from the base stations in the SteamVR environment. These sensors are bonded with adhesive or mechanically secured (there is a small locating or mounting hole), whilst the flexes are generally secured inside the controller body. Production hardware used custom flexes to reduce physical packaging and enable a wider spread of sensor locations across the entire structure.

Sensor and two sensors locations onto the Valve Index (Image by Rob Cole)

When building a tracked object, the sensors are located on the outside of the body, whilst on production controllers, the optical windows are made from a special filtering plastic to allow the laser light through whilst minimising the effect of visible light and reflections. These optical windows were a proud design feature on the HTC Vive headsets, but are much more subtle on the Index equipment making them harder to identify. Two are seen in the image above right.

The next step was creating a mask for the handle, so that the simulation programme would not attempt to place sensors on the handle, for obvious reasons! The mask was built in a different colour to make it more obvious when setting its position relative to the centre of the controller model itself.

The handle area is colored in grey because it should not contain sensors (Image by Rob Cole)

Once I had my OpenSCAD model ready to go (with the sensor and mask models) I used Valve’s special simulation tool, which creates the sensor cones and then tries to determine where it thinks the sensors would be best placed.

The results can sometimes be a little wonky, but it was a good starting point as I had no idea how to do this myself at this point. The programme thankfully created the sensor cones and attached them to exposed surfaces with correct orientations.

(Image by Rob Cole)

Once the simulation tool is used, it creates some output files, one of which contains the sensor geometry with “Model points” and “Model normals” which are the three-dimensional (X,Y,Z) locations and orientation of each sensor in Metres, relative to the origin point of the model.

It also contains the “Channel Map”: each sensor has a physical connection through the FPGA to the HDK, and the channel map is a directory of these wiring connections. My missing sensor was just ignored by deleting the channel from the file, as there are normally 25 sensors (25 channels) in total on the Tundra HDK, this left me with 24.

For my Guncube model, I had actually limited myself to using just 8 sensors (8 channels) to keep it simple, with hindsight I needed more sensors but also a better shape.

Simulations and “the JSON”

The simulation tool also outputs a series of very useful files which show the result of the sensor locations and model shape, in terms of SteamVR tracking performance.

Will the device boot? Will it boot at different angles (relative to Lighthouse base stations)? How many sensors are visible as the device moves around a Roomscale environment? If too many sensors are obscured it falls back to IMU-derived 3DoF tracking.

This tool shows the number of visible sensors, Initial Pose possible, Pose rotation, Pose translation and allows changes to be made to the model and sensor placement to optimize the tracking. It showed my cube was not ideal, but I pressed ahead anyhow as it should boot if held at an angle.

(Image by Rob Cole)

These output files are the start for building what Valve likes to call “the JSON” file, which is unique to each controller with a high degree of accuracy due to the number of decimal points registered after calibration. This accounts for manufacturing variances in the sensors or slight misplacement on the physical model. These are two example lines of the file for a single sensor:

modelpoints [ 0.0705242157, 0.0206667259, 0.0303570442 ]
modelNormals [ 0.850198984, 0.440032184, -0.289020866 ]

The JSON file is carefully constructed during development and contains all the information the controller needs to interface with SteamVR. At the most basic level, the controller JSON contains the channel map, model normals, model points and tells Steam what type of device it is. The processor in each SteamVR device (in my case on the Tundra HDK) generates a unique serial number used to ID the device, for example:

LHR-18DE32B3

Shown in the image below is part of the JSON for the Guncube model.

(Image by Rob Cole)

Now I had my simulation, and it was time for further work as my simulated sensors were located in very different places to the optical sensors I had already stuck onto the physical model. 

I managed to alter the locations of the sensors in OpenSCAD by giving each sensor a different colour to identify it clearly, and then tinkering with the numbers until each sensor location matched my physical model. This was quite time-consuming but worked out well in the end.

(Image by Rob Cole)

Now I had the OpenSCAD model with all the sensors correctly positioned, the next step was importing a Vive Wand controller model into Blender to prepare for alignment with my Guncube model to ensure the orientation was correct. The Vive wand is recommended as an alignment tool as it’s a well-known entity in SteamVR used by many existing applications.

I never used Blender before, so I had to take a crash course using one of many free tutorials available from the super helpful Blender community. This proved useful as I would need to come back to Blender a little later in the process.

(Image by Rob Cole)

After preparing the Wand in Blender, I imported it back into OpenSCAD and then carefully aligned it with my Guncube model as seen in the image above. This would ensure that my hand would be in the right place when holding the object and that the cube would be orientated correctly.

Further work on the JSON included setting up the IMU with its physical location and orientation within the object calculated, before using a IMU calibration tool to generate a very accurate setup. As an example:

“imu” : {
“acc_bias” : [ -0.137274206, -0.173890889, 0.0727910548 ],
“acc_scale” : [ 1.00237846, 1.00400507, 1.00108767 ],
“gyro_bias” : [ 0.0518387705, -0.0783547238, 0.0147437099 ], 
“plus_x” : [ -0.17015972188657003, -0.76451300222496954, 0.62174398114313334 ], 
“plus_z” : [ 0.98356390608984867, -0.093095425418226796, 0.15471041664190466 ], 
“position” : [ -0.0093050003051757813, -0.058168049901723862, 0.093742668628692627 ]
}

Next stage was setting the “Head”, aligning the object coordinate system and the reference model coordinate system. As an example:

“head” : {
“plus_x” : [ 0.97946963207312565, 0.021247903815272418, 0.20046886629933758 ],
“plus_z” : [ -0.17364797890248995, -0.41619795493819628, 0.89253887385233843 ],
“position” : [ 0.010757000185549259, 0.021780999377369881, -0.042139999568462372 ]
}

Some important information to add about the device itself and its identity:-

“device_class” : “controller” (what it is? HMD or Controller)
“device_pid” : 8192 (USB product identification number)
“device_serial_number” : “LHR-18DE32B3” (unique to device)
“device_vid” : 10462 (USB vendor identification number)

Finally, I went through the JSON to remove any syntax errors, as misplaced “,” or “{” can completely throw it out with it refusing to upload or function.

As my Guncube didn’t currently have any input components connected, I was just concerned with tracking, so there was no need to work on that aspect of the JSON at this stage. This is something I planned to explore in the future, as I’d already received all the connecting wires from Digikey for wiring input components to the HDK’s 25pin FPC connector for Input/Output (IO) expansion.

All SteamVR devices (headset, controllers, tracked objects) have their own JSON which can be downloaded for use or reference. It was very useful to study the different JSON files for Index headset, Index controllers, and Vive Wands when building my own JSON. You may notice the additional entries in the complete JSON for the Index controller shown below, there are lots of firmware references for the finger tracking, grip, and other sensors.

(Image by Rob Cole)

The Render Model

Finally, I needed to work on the Render Model, that is what is actually seen by the user in VR, and its located in the SteamVR folder on the desktop.

C: Program Files(x86)SteamsteamappscommonSteamVRresourcesrendermodels

The render model consists of three components: a Wavefront (OBJ) object, a material (MTL), and a texture (PNG or TGA). This required further work in Blender to set the material, texture and do the UV unwrapping to create a UV map. I wasn’t really sure what this all meant so quickly looked it up online at https://en.wikipedia.org/wiki/UV_mapping and found its definition:

UV mapping is the 3D modelling process of projecting a 2D image to a 3D model’s surface for texture mapping. The letters “U” and “V” denote the axes of the 2D texture because “X”, “Y”, and “Z” are already used to denote the axis of the 3D object in model space.

(Image by Rob Cole)

Eventually, after lots of reading the training materials, head-scratching, and failed attempts I somehow ended up with a finished “Guncube” render model, sat in the SteamVR render models folder on my PC.

For production, a much higher quality render model would be used, but for my tracked object this basic model was all I needed.

(Image by Rob Cole)

Finally, I had “the JSON” and the render model completed and breathed a sigh of relief. However, when I uploaded the JSON to the HDK, it stopped registering any optical sensor hits during sensor check. I went through the JSON many times using all my course information and examples from other controllers but couldn’t find a fault, perhaps I had missed something critical but really hard to spot?

Unfortunately, I didn’t manage to finish my Guncube object as I couldn’t get the HDK to work despite multiple attempts. But some time later, I found a message from Tundra to another HDK user on the SteamVR community boards:

Hi, This is Luke from Tundra Labs. No you did not break the kit. There is a firmware issue that I’m working on to solve this where certain JSON keys cause TL448K6D to jump to bootloader for now the easiest fix is to use lighthouse watchman update to clear out the JSON from the bootloader. The command for this is:

lighthouse_watchman_update.exe –target=default empty.fw

You can download empty.fw from here:

https://github.com/tundra-labs/SteamVR_Documentation/raw/master/docs/files/empty.fw

To view the Guncube render model in VR, I cheated by downloading the JSON file from a faulty Index controller, making sure to save a copy first as it’s unique to that controller.

After adjusting the render model information in the JSON to reference the Guncube folder on my PC instead of the Index folder, I uploaded the modified JSON to the Index controller using the Lighthouse console.

(Image by Rob Cole)

Wow, OK! This allowed me to see my somewhat primitive-looking Guncube object using my Index headset, somewhat misaligned, but at least this aspect was working. I quickly reset my Index controller with its original JSON which thankfully worked without any issues; it was being boxed the following day for shipping to the Netherlands for a warranty replacement.

Satisfied that the tracking was a solvable problem that just needed a little further time and tinkering with a better shape containing more sensors, there was no point in doing any further work on Guncube. I applied the “empty.fw” file to remove my suspect JSON and once reset checked that the HDK was again receiving optical hits from the Lighthouse base station above my desk.

After stripping down the Guncube model and carefully removing the HDK with its 24 remaining channels of sensors and flexes, it was shipped to Skarredghost in Italy for tinkering, following his uber successful “Unity Cube” project. It was time that I moved on to getting some physical modeling underway.

There were many mules over many months…

(Image by Rob Cole)

Following on from the headaches of building a tracked object and clumsily making my way through OpenSCAD and Blender, I was happy to get back into doing some physical modelling work.

During my training as an industrial designer back in the mists of time, we were trained in model making to build highly finished appearance models and working prototypes. It had been a long while but after buying lots of new tools, protective equipment, and materials I soon got back into the swing of building things.

Starting with a thorough investigation into hand grips, I made many different grips and controller bodies cut and carved from wood, used heat moulded plastics, and sometimes used the two materials combined.  One piece of advice, don’t try to use the finger scanner to unlock your phone after a morning of carving wood, you may not have fingerprints for a few days…

(Image by Rob Cole)

I tried lots of different shapes, different volumes, straight grips, left and right specific grips, curved grips, modular grips with bolt-on plates; tried them on work colleagues with carefully sanitised hands of different sizes and noted preferences. I copied the Index body shape and the Wand body space to better understand their design choices; I ended up with so many grips that eventually most went into the recycling bins.

(Image by Rob Cole)

Skeletal approaches were tried with minimal components at different orientations, which felt good in the hand, easily lending itself to a modular design with bolt-on components for ergonomic adjustment or damage replacement (and colourful “hop up kits”).

(Image by Rob Cole)

Work started on a “control module” using the Thrustmaster joystick unit with A / B buttons and switches mounted inside a modified plastic housing which had been cut from the X-Pro controller body. The Thrustmaster trigger was also rebuilt, fixed onto a mounting plate to allow attachment to different types of mule, still containing a small haptic motor and power cable. 

Adjustable slides were then fitted to the control module and joystick plate, to allow the user to independently adjust the reach distances for each. Next, these parts were integrated into a moulded plastic hand grip to see how it would actually feel and work for different size hands.

One consideration was using a software setting to adjust the Render Model to align the virtual and physical once the controller has been set for the user hands size, otherwise the user would see the joystick in a different position to what they were feeling with their thumb!

(Image by Rob Cole)

Having spent lots of time trying out many different “held grip” shapes, I moved on to exploring the “hands-free” style recently championed by Valve with the Index controller and its elasticated hand strap.

Going hands free…

After lots of holding grips, the relationship changed with the introduction of a dorsal strap (back of hand) to pull the palm of the hand against the controller body. I started with a funky ergonomic grip in a vertical orientation, because why not?

Of course it was terrible for free movement with the peaks catching my fingers, but felt really good when closed and gripped tightly. The next mule was built using an offset oval body to allow the user to rotate the grip to suit their hand size, and secured with adjustable strap tension for further fine-tuning.

(Image by Rob Cole)

The dorsal strap progressed through several iterations, resulting in a rigid plastic plate with soft backing, that introduced extra stability during the hands-free use, perhaps useful for a heavier controller or during really active gaming movement.

Finally, I took several handgrips and made straps for them just to see what would happen, also fitting them with different input components as I continued to experiment, this included controllers with a vertical orientation, and controllers with a slanted orientation.

(Image by Rob Cole)

I’d already started thinking about integrating sensor tracking rings with the mules, to figure out the right shape to use which wouldn’t interfere with opening my hands, whilst maintaining good sensor visibility for optimum tracking. Mock-up tracking rings were fitted to a couple of different mules, simply to see what would happen. Which sensors would be occluded by the presence of my hand around the controller body?

(Image by Rob Cole)

After having lots of fun trying different combinations of straps, plates, grips, and input parts, I selected my second hands-free mule as the base model for my next investigation into haptics.

“Squirrel”

The E-Swap Pro gamepad had a total of 4 ERM haptic motors; I poached the 2 large motors from the controller body and 1 of the small motors from the triggers. When I tested the gamepad the haptic triggers were brilliant in use, the small motors providing a surprising punch.

ERM are “eccentric rotating mass” motors, the most basic type of haptics commonly used, with an ability to make blunt noises in 2 axis but lacking finesse or the fastest response (50-100ms) and more power hungry which could quickly flatten a smaller battery. There were other choices for haptics, but the ERM I’d pulled from the gamepad were available to use and in my hands, so to speak.

To get this working, I built mounts for the big motors, a mount for the small motor on the hand strap, and then wired it up with a variable power supply, so I could test the Haptic response across the controller body. The long power leads coming off the controller actually looked quite cool, like I was testing one of those clunky haptic gloves often seen on Kickstarter campaigns.

(Image by Rob Cole)

The two large motors had a maximum torque rating of 15nm on full power, whilst the small motor had a maximum torque of 3nm. Trying different combinations of motors was very interesting, running both big motors on full power gave a great wrist workout, like fighting an angry squirrel!

More interesting was the tiny motor on the dorsal strap; I thought this might work well because the back of the hand is very sensitive, but this little motor driving the plastic strap plate made for a powerful sensation that felt like it was actually inside my hand.

The next stage would have been to purchase a haptics development kit with the superior LRA (linear resonant actuator) motors or the latest Piezo haptics, but unfortunately, I didn’t have $400 spare for a haptics HDK. However, I finished with more of an understanding of haptics and the important discovery of using a haptic motor in the hand strap. 

Following on from my “interesting” attempt to build a tracked object, and much time spent building many different mules to explore controller design, I turned my attention to the “Caliper” concept that had been in my head for some time.

Why Caliper?

It started with reference to a type of measuring device I’ve used many times during my professional career, called Caliper.

(Image by Rob Cole)

Caliper is a device used to measure the dimensions of an object. It is used in many fields such as mechanical engineering, metalworking, forestry, woodworking, science, and medicine.

Many types of caliper permit reading out a measurement on a ruled scale, a dial, or a digital display. Some calipers can be as simple as a compass with inward or outward-facing points, but no scale.

In this case, I was very interested in flipping this around with a concept of “measuring” human hands using a fully adjustable controller, to increase accommodation for a wider range of hand sizes and common human asymmetry. Two aspects that I felt could add some real value to the existing controller ecosystem, whether coming to market or not:- 

1 – Ergonomic adjustment to accommodate different hands

As mentioned in my first ever article for Skarredghost back in 2018, human beings are wonderfully wonky displaying all manner of physiological asymmetry, and catering for this is no easy task!

Lots of questions started to emerge…how many times does a player pull the trigger during a two-hour session in Pavlov? What is this repetition doing with a poor hand fit? Does it cause discomfort? Could this cause a repetitive strain injury in the long term? Can this be accommodated through a modular or adjustable design?

How about an adjustable trigger position? Can the reach be adjusted between the hand grip and the joystick? Could the joystick and trigger reach be adjusted independently? What if I need a different size controller body to suit my hands? Can the trigger cover be changed for different shapes? What if I prefer a smooth texture, or rough texture grip? Can I adjust the left and right controllers separately, as hands are often slightly different sizes?

If it was possible to provide a good range of fitting adjustments in a hand controller, it would benefit the widest range of users. this seemed like a worthwhile pursuit that would add genuine value when building any new controller.

2 – Replaceable input parts to reduce E-waste and minimise VR downtime

No secret; I suffered ongoing issues with the somewhat fragile Index controllers during the 2 years that my EU warranty covered, with many replacement Index controllers provided by Steam Support, a grand total of 11 pairs (22 controllers) replaced during those 2 years. I also had several headsets, tethers, and ear speakers, but that’s another story.

(Image by Rob Cole)

Steam Support people did what they could and were as helpful as possible, but stock shortages were common as the Index continued to sell as quickly as they could manufacture it, and this included replacement controllers. When they worked they were fantastic fun to use and added a huge sense of immersion, but many weeks were lost waiting for replacement controllers, which really spoiled my ownership.  

Later, I even bought a second pair to minimise VR downtime, but these also soon required replacement…and so it went on; as soon as my warranty expired I quickly sold my Index before something broke again.

Uncomfortable E-waste

The first couple of controller returns (for joysticks not clicking properly) were a novelty where I appreciated great customer service. I was provided with free returns paperwork which was attached to the box and taken to a local courier office for return to Valve’s European partner Micro Ingram in the Netherlands. The booking agent at the office also started recognising me, which became a running joke between us, “another RMA?”

But as time ticked on, these regular trips to the courier office started making me feel uncomfortable about adding to the growing pile of e-waste I kept hearing about in the news.

The UK generated the second most waste electrical and electronic equipment (WEEE) per capita in the world in 2019, behind only Norway, according to a report published by a global e-waste grouping. The Global E-waste Monitor report says the UK generates 23.9kg per capita – compared with 26kg in Norway 

July 2020 Global E-waste Monitor 2020

(Image provided by Rob Cole)

As well as the environmental cost of manufacturing the new controllers, there were all those extra shipping miles to consider. Faulty controllers were shipped from the customer in the UK back to the Netherlands for inspection; replacement controllers were shipped from China to the Netherlands, before being shipped out to the customer in the UK.

Surface shipping by container = 10 to 40 g CO2 (in grams) emitted per metric ton of freight and per km of transportation

Timeforchange.org

As someone who never received a reconditioned controller – it was box-fresh each time (very obvious if you know what to look for) –  what happened to my old ones, did they just scrap them? From looking at images of broken controllers, they seemed very complex and probably not cost-effective to repair, even if there was a repair facility with factory trained technicians in Europe (there wasn’t) stocked with controller spare parts (there weren’t any).

I was aware of technically capable Index users fitting their own replacement joysticks but at the expense of losing capacitive sensing on the joystick and obviously invalidating any return (Steam Support was often generous with out-of-warranty replacements).

Having gone through the RMA process time after time, I wanted something tougher that was rebuildable to minimise VR downtime and dramatically reduce e-waste, allowing the user to swap out a joystick, trigger module, grip, or perhaps even replace a diminished battery to ensure long term function of the controller.

So why did these keep failing?

(Image provided by Rob Cole)

A couple of weeks of playing In Death, Boneworks and Pavlov seemed to have a really detrimental effect on my joysticks and triggers, and occasionally the grip. Running around in Boneworks with my left trigger jammed shut wasn’t really the best way to play, whilst remapping buttons to avoid broken controls also broke muscle memory, making for a less fluid play experience.

I started thinking more about what was potentially causing these issues, soon identifying 3 areas of concern that needed addressing lest my new controller design was to suffer a similar fate:-

1. Underbuilt inputs

The Index controllers definitely seemed a bit underbuilt, as I reported in my second Index ergonomics article for Skarredghost back in 2019:

When my Valve Index arrived on launch day (28th June), the first item I removed was the right Index Controller, the headset didn’t even get a look in. “Knuckles” had arrived and were now in my hands…

It felt reassuringly heavy (196 grams) and looked well built with a premium look, although both A/B buttons and the trigger felt a little wobbly, and perhaps a little out of place here.

Talking of quality, despite the joysticks being a big improvement on the Vive Wand trackpad, the Index joysticks did not have the tight, precision feel of the sticks on the official gamepads for Xbox and PS4; for the high price of the Index controllers I expected better quality joysticks – something that would come back to haunt Valve?

Something important to consider, the joysticks were a late addition to the “Knuckles” design, which originally had large trackpads; something Valve has been reported many times as preferring to do the locomotion with a joystick.  Due to developer pressure, a decision was made late during Index development to incorporate a joystick whilst retaining a trackpad; but for a lack of space this decision ended up compromising both with a small joystick and a small pill-shaped trackpad.

(Image by Rob Cole)

This was in direct contrast to the Oculus Touch controllers which had launched with a joystick and great ergonomics courtesy of the Carbon Design team who were also behind the Xbox controller design.

Unfortunately, small joysticks just degrade quickly when high loads are applied. No manufacturer can build a small joystick with the same load capacity as a large joystick; there is a physical limitation within potentiometers.

The image right above clearly shows the big difference in size between the Index joystick and Thrustmaster’s S6 joystick used in their gamepad.

Nintendo had already been through this drama with their “Joycons” which used a similar small joystick module prone to quickly wearing causing drifting or stick failure, this ended up in Nintendo owners taking legal action after Nintendo tried to ignore the problem.  

Something also noticeable when using the Index joystick was a soft “mushy” feel compared to using larger joysticks on high-quality consoles and PC gamepads. Several replacement controllers had arrived with loose feeling joysticks “out the box” which only got worse over a couple of play sessions and soon required replacement again.

This soft, somewhat vague feeling was evident during free locomotion and flying, quickly swapping to my Xbox or Steam controllers for Aircar showed the immediate difference in precision and control feel. Essentially, Index needed a larger, high-quality joystick to provide a better feel and long-term durability, but this would come at the cost of losing the trackpad completely, or perhaps there was an alternative (more on this later)?

2. Input misalignment

Sharing a limited space between the two discrete inputs requires that the Index user deflect their thumb off its natural axis, to use either the trackpad or joystick, with the stick requiring considerably more deflection as seen in the image below. 

(Image by Rob Cole)

This deflection was something I always felt during joystick locomotion, maintaining true forward direction was difficult without slight diagonal gain, it was also uncomfortable causing my thumb to quickly fatigue.  It’s also possible that this deflection and constant adjustment caused unwanted side load on the joystick module, further accelerating wear in the potentiometer module.

To determine the ideal joystick position on the controller, I placed a spare joystick cover onto an Index controller with some adhesive tape;  I was not surprised to see the big differences in both lateral alignment and the forward (reach) position. The original stick was a long way away from where my thumb needed it to be. If you look carefully, it would have been possible to mount a larger central joystick and still have a small trackpad (perhaps bigger) to the right side.  

(Image by Rob Cole)

3. Strong hands hurt controllers

As a working technician, I’m hands on tools all day during assembly, servicing, or moving boxes of stock which develops a strong hand grip –  known to ergonomists as “Hand Grip Strength” (HGS). 

Admittedly this doesn’t give motion controllers an easy life; though I always treat them with the respect any expensive device deserves, playing in a large empty space devoid of furniture and computer monitors, and carefully storing after use. I’d also never hit the controllers together with enough force to do any damage, but had seen plenty of pictures on Reddit of controllers destroyed during Beat Saber sessions!

The trigger cover was a dome-shaped moulded piece, it seemed to be made from a relatively soft plastic that started deforming with repeated hard trigger pulls against the metal controller body it’s mounted onto. The shape of the Index controller body meant I was not able to pull the trigger straight, but from the side causing a side load to be applied to the cover; the trigger cover needed to be made from a much stronger and stiffer material to resist these bending loads.

(Image by Rob Cole)

Repeatedly pulling the triggers for weapon firing, or using the left trigger for shield bash playing “In Death” would quickly do its damage with an ever deforming cover starting to bind against the inside of the controller body, causing audible clicking and finally grinding noises.

Over time the covers got more readily jammed, requiring considerable pressure to release; finally requiring a flat blade screwdriver to prise back open, before it was time to box the controller up for another RMA.  

After my latest RMA, I started looking into HGS data to see what hand load I was potentially putting into the Index controllers. The image below shows some HGS data captured using a Camry Electronic Hand Dynameter, from a sample of 500 adult males with left and right hands averaged.

(Image provided by Rob Cole using data and images by Topendsports)

From looking at this data it’s not unreasonable to surmise that adult male Index players can repeatedly subject their controllers to a grip strength of 40-50 kg. The reason I haven’t included female players is that their average HGS is considerably lower than males of similar ages, and I wanted to see the upper limits as this is where input component damage should occur. 

HGS tends to peak around the mid-to-late twenties in both sexes, before declining as people age. It could also explain why some users have few problems with their Index controllers, as they don’t have a strong hand grip and won’t cause physical damage in the same way that stronger hands easily can.

Data suggests high grip strengths are found in people regularly exercising using gym equipment, playing racket and ball sports, or in people working with their hands such as mechanics, builders, and farmers, whilst the NHL recorded a maximum HGS of 75.8kg at their 2012 Scouting combine.

(Image by Rob Cole)

Back in virtual reality with the Index running at 144hz….a spine-tingling rush of adrenaline floods my body, ramping up my grip strength to joystick drifting, trigger bending, grip cracking levels and for that, I don’t apologize… the parts need to be tougher, or at least replaceable – I took notes.

The need for higher quality input components, the need for ergonomic alignment with primary inputs, and the need to cater for strong hands using replaceable parts gave me a good starting point of what “not to do” when designing Caliper.

Building Caliper

(Image by Rob Cole)

Taking everything I’d learnt so far, all the different grip shapes and mules I’d built, and months of design scribblings, I selected the best shape and got to work moulding plastic. A nylon tube representing the combined volume of the Li-Ion battery and haptic motor was embedded into the centre of the body, and plastic material added to build up the controller around an aluminium alloy frame.

Magnets were embedded into the body so I could use magnetic grip covers to suit 3 different hand sizes on a single size controller, also making it easy for 3D printing of grip covers for customisation to suit individual hands for a really precise fit. At this point, the controller body itself was sized as small with no grip cover fitted and had a comfortable shape, whilst medium and large hands used size-specific grip covers and an adjustable hand strap for sizing setup.

After noticing the magnetically held grip covers tended to squirm when people gripped them really tightly, I took inspiration from the ‘picatinny rail’ used for mounting optics onto military firearms. Subtly reshaping the controller body in a more angular fashion gave the grip covers a secure interface, in conjunction with magnets embedded inside the grip cover and controller body.

Evolution of the Caliper controller (Image by Rob Cole)

This would have left small-handed users holding an uncomfortable feeling controller body, but since reshaping had also slightly reduced the controller body volume it meant a grip could always be used for all size ranges.

Small-handed users now had the opportunity of also using customised grip covers without making the controller too large for them to hold. Using a slightly raised, rougher texture could provide increased handgrip during a sweaty workout in a fitness game, whilst a smooth surface allowed the hand to move more easily on the controller, perhaps better for finger tracking.

No further movement was felt, with the grip covers only taking a small extra effort (hard tug) to remove when required. For a production controller, the design would change to use 2 small countersunk mounting bolts through the grip cover and matching threaded inserts embedded inside the controller body, to provide a much more secure fit with a smaller packaging volume as paired magnets are bulky.

Once I had a good body / grip shape and a working strap with tension adjustment, my work turned back towards the tracking ring, integrating it carefully to give clearance for opening the hands and using controls, whilst presenting the sensors in the best orientations for good tracking performance.

(Image by Rob Cole)

Experiments continued with different strap materials, adjustment methods (single side, dual side), seeing how dorsal plates could fit inside the limited confines of the tracking ring interior. Size-specific dorsal plates were built, and I made myself a custom dorsal plate using a heat moulded strip of material troughed inside a rigid backing plate – it felt really supportive against the back of my hand. 

Finally, attention was shifted back to the “control module”, finishing it properly by installing a System button to give parity with the industry-standard Touch layout. Thrustmaster’s E-Swap X-Pro input components were rated for 2 million + activations, with the joystick boasting “increased return to centre precision” which boded well for long-term use until a replacement was eventually required. 

The spectre of adjustment reared its head, working out how to provide independent reach adjustments for the joystick and trigger within the limited volume of the controller body was really difficult. Mounting an adjustable trigger slide underneath the control module was the solution, with many iterations as the mule slowly took shape.

(Image by Rob Cole)

For the grip sensor, I built a simulated pressure plate sensor into the underside of the controller body, to replace the grip button found on the side of many current Touch layout controllers. Originally I tried using a grip button on the side of the trigger module but found it didn’t feel as natural as a squeeze grip after 2 years of using Index controllers. 

(Image by Rob Cole)

Using squeeze to grip is very instinctive but can be less reliable for frictionless input as it’s prone to unintended activations unless really well adjusted to the user’s HGS. Two different methods of activating a grip function on the controller, both with advantages and disadvantages, something I could always revisit in the future.

Paying special attention to the trigger cover, I custom built a special shape with internal reinforcement, which was very comfortable to use, and detachable from the removable trigger module if required – this could allow for custom trigger covers of different sizes, shapes, and materials, or just for replacement if the stock cover was ever damaged.

Eventually, I got most of the physical packaging figured out, integrating a minimalist tracking ring, representative cabling runs, and multi-faceted sensor tracking head, whilst finishing my Caliper mule to a higher standard than any of the previous mules.  The blue cable on the left of the controller body connects the tracking head (containing the SteamVR electronics and front optical sensors) to the main body where the battery, body haptic motor, and tracking ring with additional sensors would be based.

(Image by Rob Cole)

In terms of replaceable parts, both the control module and trigger module quickly unbolt and unplug from the controller body. If the right bolt of the control module is completely removed then the joystick module can be removed and a new one fitted in a few seconds, as seen in the image above.

The S6 module is commercially available for less than $20, boasts a substantially longer operating life than anything currently used for VR, and most importantly has a great feel whether locomoting or flying. It’s also available with convex and concave rubber covers (that simply unscrew) whilst the joystick module can be swapped with a D-Pad or Touchpad from the same E-Swap ecosystem.

(Image by Rob Cole)

Tracking sensor placement on the mule was explored using 25 small fluorescent decals, allowing me to photograph the model at different angles and note the number of sensors visible. This led to many adjustments to sensor placement to improve visibility and ensure non co-planar sensors were available during different poses. Looking closely at occlusion caused by hand movement inside the controller tracking ring, I relocated the sensors a number of times making sure any hand movement wouldn’t block a sensor window.

Though this was still a “dumb” mule, it had operable input components so that testers could move the joystick, pull the trigger, press buttons, try different grip covers, adjust the strap length and strap tension whilst setting 4 different adjustments with an Allen key, as seen in the images below.

(Image by Rob Cole)

Control module (Joystick, A/B, and System buttons)

Left image: +/- 30 degrees angle adjustment relative to body horizontal
Middle image: 12mm of granular reach adjustment from the handgrip to the module

Trigger module

Right image: 15mm of reach adjustment using 5 positions
Right image: +30 degrees angle adjustment relative to body horizontal.

After some proper hands-on assessment, it was felt to be comfortable, well balanced, and easy to use the controls with good alignment and control feel.

The range of adjustments was easy to use, proving very useful when fitting people with different-sized hands. Some extra work was still required on the dorsal plate for the strap (it’s not seen here in the images) and integrating the strap adjuster into the base of the controller body to make it easier to use one-handed.

(Image by Rob Cole)

Overall, I was pleased with the outcome of this first Caliper mule, meeting my requirements for adjustable ergonomics to accommodate different sized hands with input alignment, and the use of replacement parts to minimise controller E-waste whilst minimising VR downtime.

Customisation options include magnetic grip covers, a replaceable trigger cover, and a replaceable joystick cover, in addition to the joystick module and socket being parts of the E-Swap ecosystem with options for coloured replacement joysticks, D-Pads, or Touchpads.

(Image by Rob Cole)

The future of Caliper?

It feels like the hard early work has been done, resulting in practical solutions to some problems I identified with my previous controllers. Perhaps the Caliper design is something that could be manufactured in a more refined version, if funding was actually available. Personally, I’d prefer to work on the ergonomics and adjustments and leave the mechanical engineering and electronics to people who have experience and more knowledge.

Already my Caliper design has moved on from these photos, using a one-sided adjustment arm and more compact control module, allowing the body to come even closer to the controls for the smallest hands. Improvements to the tracking ring, an updated strap design with a haptic plate, and a better wiring solution for the adjustable control module and trigger modules are all being worked on at the moment.

I’m currently learning Fusion360 to start building my new design in CAD; once it’s finished I plan to build a working prototype as my new Tundra HDK and single base station are still waiting to be used for testing. Hopefully, by then Tundra will have a new HDK to purchase so it would be possible to build a second controller, for the other hand!

In the meantime, I’ll keep doing my day job, working on the XRCaliper prototype, and building stuff as I’ve already got some new ideas…

Final thoughts

Well, the past year was quite an adventure, maybe not one to repeat; thankfully I didn’t break the bank after deciding to quickly shut down my start-up. Many companies are still struggling, with long delays and component shortages, in hindsight making me very glad I stopped. 

The Caliper project moved further along than I thought would be possible during the limited time I had for development (it was easily a full time job in itself) though there is still much to do, despite the lack of any commercial end goal.

It was very interesting learning new skills with a number of really rewarding “a-ha!” moments especially in software, whilst maintaining a strict discipline to keep things moving.

I was always short on spare time and often out of my depth with the more technical problems but somehow carried on moving forward. Many problems and roadblocks were overcome, making my first foray into hardware development not a complete disaster. 

During development, I started looking with fresh appreciation at the Index controllers: it’s too easy to criticise until you try doing it yourself. I found it incredible that they had managed to design them in the first place, and then actually get them into production for people to purchase at a reasonable price. Bravo, Valve!

Finally, a great quote had kept me going during difficult development times and reminded me to not take it all too seriously, it is from a Valve round table interview back in 2017 (reported by Polygon):

“We’re optimistic. We think VR is going great. It’s going in a way that’s consistent with our expectations,” says Newell. “We’re also pretty comfortable with the idea that it will turn out to be a complete failure.”

Thanks for reading, Rob Cole.

The post Project Caliper: the journey of prototyping optimal controllers for VR appeared first on The Ghost Howls.

Read More