Categories
Experiences

Gnomes and Goblins preview

Well this was completely unexpected, and a very welcome surprise return for Jon Favreau and Jake Rowell’s Wevr title that seemed to have slipped into oblivion. I had played it again when I got my Valve Index last year but there had been no further updates or information since, like many early Vive demos that are still available but appear dormant in terms of development.

I always had fond memories of the Vive demo from 2016, but 4 years was a long time ago, especially in terms of VR software. So to find a “Venice” preview available at the festival was fantastic, and much more rewarding when I discovered in-headset that it wasn’t an updated Vive demo but something entirely new.

One problem I found with the original Vive demo was the requirement for a big room (to create a big play space) to get its roomscale VR to work properly. Without going into software and moving my playscape, an important trigger event could be just out of reach, often at home I’d find myself trying to flatten my Vive controller against the physical wall to try and reach something.

During the new Venice preview I used free locomotion, shook a fairy (don’t ask!), transferred through portals, paddled down a river on a raft, lowered myself on a rope platform down into a mine, explored a series of tunnels in an illuminating way, attended a meeting of many in a cave and finally released lanterns into the sky. Hundreds of NPC’s roamed about, some singing, many doing their own stuff, with gorgeous detail and animation. Overall an incredibly rich world, which felt like being inside a big budget Hollywood animation movie.

Performance took a hit, with strange microstutters in headset, despite my frame timing not averaging over 9ms/11.1ms (90hz) though the developer graph showed very thin yellow spikes in places. I tried adjusting super resolution without it resolving the issue. Textures seemed muddy and very “2016” in places, yet sharper in other places; close up interaction with NPC’s were richly detailed, longer distances were blurrier.

Something odd was going on, could be compatibility with the Index headset, though I suspect Viveport was not playing nicely with the new Windows update, or with SteamVR. Its not the first time I’ve had problems after installing Viveport, and was glad to uninstall it at the end of the festival.

I’ll need to play the Steam release at the end of this month to see if this problem reoccurs. The recommended specification demonstrates that this title will be perfect for the new wave of Nvidia RTX 3xxx graphics cards as its going to require some serious grunt to run well. For reference I’m using 8086K @ 5.2Ghz / RTX 2080Ti.

  • i7-9700K or equivalent
  • Memory: 32 GB RAM
  • Graphics: Nvidia Geforce GTX 2080 or equivalent

I’m not going to say any more as its better discovered first person in your headset. The full game releases on 23rd September 2020 and promises many hours of gameplay in its rich environment.

It’s not an exaggeration to say that Gnomes & Goblins is the biggest project Wevr has developed. Where the G&G Preview was a quick introduction to Buddy and his world, a bit like a tasting spoon of ice cream, this all-new, full-featured, multi-hour VR simulation game is like a full pint of your favorite flavor with new interactions and story moments to discover including new characters to meet, forest areas to explore, magical collectibles to discover and a range of open-world gameplay from climbing to paddle boarding to farming and brew-making.”Wevr

Thanks for reading! Rob Cole

Categories
Experiences

Rembrandt Reality

Use your smartphone to travel to the year 1632 and step into Rembrandt’s painting ‘The Anatomy Lesson of Dr. Nicolaes Tulp’.

Place the gate and walk round in the Anatomical Theatre.

See through Rembrandt’s eyes how Doctor Tulp and his fellow doctors are examining the body of the criminal Aris Kindt. Discover all the stories behind the painting.

Developer description on Google play store

This amazing ARCore application for Android smartphones has been a pleasure of mine since discovering it over a year ago. It’s great to load it up every once in a while and enjoy the 6DoF environment in all its glory (it has a huge playspace).

Scanning the floor to create a ground plane

Best used outdoors in a quiet place, with good quality audio headphones. I use my Pixel 3aXL which has a good quality display, and sennheiser HD 461 headphones which provide some isolation from background noise. The application can be used with your phone display in portrait or landscape with auto rotation.

Placing the gate on the ground plane

The application boots up quickly and takes you through an introduction to the scenario and how to interact.

You then scan the floor to create a ground plane for the application to sit on, once ARCore has worked its magic a “gate” appears which can be placed precisely by tapping on your screen.

Initial appearance of the gate

Once the gate has been placed it materializes first as an archway with a stone finish, before opening the “gateway” into the Rembrandt environment.

An open invitation… what lies beyond?

You are then invited to walk through the gate, and this is where a genuine sense of physical space is generated from physically walking forward into a rich black backdrop, with the scene itself set further back from the entrance gate.

The anatomy lesson in progress

In front is the anatomy lesson of Dr. Tulp, and the surgeons keenly watching the dissection. I am able to walk forward another 5 metres before reaching the centre, the sense of scale is very impressive as it’s using 1:1 mapping.

With ARCore providing a solid geospatial anchor, I can freely walk around inside the environment, and get as close as I want with high quality assets showing rich detail in 6DoF.

The sense of presence is rewarding ‘despite’ this being presented on just a smartphone display rather than inside AR glasses.

During the experience, ‘hotspots’ can be clicked on which provide very useful insights into the original Rembrandt painting using images and audio description.

Up close and personal

Once you have selected the different hotspots you feel well informed, yet the real treat for me is always looking around the environment, with the ceiling a particular highlight – this is cleverly mentioned during one of the hotspot activated informationals.

Amazing ceiling with animated bird flying about

My other favourite element of this AR experience is exploring the boundaries of the environment and looking back through the entrance gate (to the “real world”) which causes a strong impression that you are inside the environment of the painting!

The participants appear to be taking note of the gate….

I haven’t calculated the available space inside the environment but it’s very large, I always finish by walking back through the gate.

The persistent nature of the application means that the gate straddling the boundary between the environment and real world can be carefully inspected, walked through, back through, the students and doctor remaining in place, a great example of a “Portal mechanic” in action.

Mind is blown by seeing the real world back through the gate! It’s fun to stand to one side and look around the thick edge of the gate out into the “real” street outside.

It’s always a pleasure to use Rembrandt Reality, the developers did a great job building this using ARCore. High quality experiences like Rembrandt Reality demonstrate the potential of augmented reality even on smartphones (I’d like AR glasses,but 2030?)

Rembrandt Reality is available as a free download on both the Google play store and Apple store (there is an ARkit build for Apple devices).

Thanks for reading! Rob Cole.

Categories
Experiments

Valve ear speaker teardown

Ever wondered what was inside those neat off-ear speakers on the Valve Index?

“Balanced Mode Radiator” (BMR) ear speakers use custom drivers made by Tectonic for Valve. Valve list their audio solution as having these characteristics:-

Built-in: 37.5mm off-ear Balanced Mode Radiators (BMR), Frequency Response: 40Hz – 24KHz, Impedance: 6 Ohm, SPL: 98.96 dBSPL at 1cm.

In use onboard the Valve Index headset, the BMR ear speakers are unrivalled in terms of sound quality and sense of spatial soundstage for VR headsets, Emily Ridgway and her team at Valve certainly worked some magic here!

After experimenting with the BMR ear speakers and different audio headphones, I kept using the ear speakers as their excellent audio combined with quality of life (off-ear, on-board) was a great combination.

During the past year I did need to RMA a number of ear speakers, Steam support were very supportive and shipped them all as advanced replacement, and didn’t ask for the defective ones back. So I obtained some spares…

2 problem developed:-

1. Unwanted speaker movement. This seemed to worsen after lots of active gaming in Pistol Whip and Best Saber. Over time the ear speakers stopped holding the set position, drooping during a session or sudden movement. It appeared that the mechanism spring force degraded over time/use.

2. Vibrating. A slow developer but eventually the speaker pods started to vibrate at higher volumes or on bass hits. Not a malfunction of the driver but the physical connection between the speaker pod and speaker arm. This is felt as a looseness (slop) with light finger pressure, it’s easy to wobble the speakers pods.

Despite these problems I continued using the BMR ear speakers and gave Valve some feedback to help with further iterations.

How do they attach?

These attach to the Valve Index headset using a circular ‘pogo pin’ mounting system retained by a single torx T6 bolt through the headstrap.

Pogo pin mounting socket on Index headstrap

I decided to teardown one of my faulty BMR ear speakers to have a good look inside.

BMR Ear Speaker removed from Index, ‘Pogo Pin’ mounting system on the right
Carefully removed plastic cover (glued), I pushed it open using a flatblade screwdriver through the open slot for the height adjuster
Sliding height adjuster at minimum, power cable routing accommodates movement of height adjuster
Sliding height adjuster at maximum
Pogo pin mechanism taken apart showing pogo pin springs, bolts, sliding plate and circular pogo mounting

Looking closely at the “pogo pin” system, it’s cleverly designed using the springs to apply pressure to the pogo pins (to ensure contact with headstrap audio pads) and also allow vertical adjustment of the speaker with enough resistance to prevent unwanted movement.

Perhaps these springs are stretching over time/use, as springs do tend to stretch, to a reduced clamping force allowing the speaker pod to droop. It may be possible to tighten the small bolts to increase spring pressure, or pad the spring with steel washers to achieve the same.

The Circlip in the image above locks the speaker pod axle to the speaker arm.

This axle socket has gone sloppy (flogged out) on several of my ear speakers allowing the speaker pod to vibrate at higher volumes or during bass heavy audio. I’m unsure how this can be resolved without a different type of fitting, or perhaps a polymer bushing.

Circlip removed from axle
Speaker pod removed from arm, at this point I cut the power cables
Carefully working blade around speaker basket to break glue seal
Finally! Speaker assembly coming apart…
Wire basket removed, showing foam damper covering rear of driver unit
Wire basket removed from ear speaker pod
Thick foam damper from rear of ear speaker
Inside face of foam damper with moulding detail
BMR Ear Speaker pod stripped as far as possible, the plastic moulding was heavily glued to the front ring and resisted considerable force
Power wires to driver hanging inside speaker assembly
Detail showing driver and some type of baffling
Carefully cutting away the plastic moulding to access the driver
Driver core and baffling
Outside wire basket and diaphragm of driver
Cut apart driver showing magnet, copper coil, power leads
Close-up of coil

More information on the Valve Index audio is found in this blog article

https://immersivecomputing.org/2020/02/25/valve-index-ear-ergonomics/

Thanks for reading! Rob Cole.

Categories
Experiences

Microsoft Hololens

I’ve been fortunate enough to have had several sessions with Microsoft’s Hololens AR standalone headset; it’s always been impressive to use despite the obvious limitations of current AR technology.

Talking of technology, Microsoft list the Hololens with these specifications:

Optics See-through holographic lenses (waveguides)

Holographic resolution 2 HD 16:9 light engines producing 2.3M total light points

Holographic density >2.5k radiants (light points per radian)

-Eye-based rendering

-Automatic pupillary distance calibration

In addition, the Hololens has a fully loaded sensor array:


    1 inertial measurement unit (IMU)
    4 environment understanding cameras
    1 depth camera
    1 2MP photo / HD video camera
    Mixed reality capture
    4 microphones
    1 ambient light sensor

Compute

  • Intel 32-bit architecture with TPM 2.0 support
  • Custom-built Microsoft Holographic Processing Unit (HPU 1.0)
  • 64 GB Flash
  • 2 GB RAM

Regarding pricing, I’d only heard of them being sold to enterprise and big business (i.e. Microsoft partners) but I once saw a Hololens for sale in computer exchange (CEX) for a cool £3,200.

From my somewhat limited understanding of augmented reality technology, there is a long roadmap of development still ahead.

VR is almost seen as a solved problem with further iterations only set to improve on what is already a very immersive experience in terms of ‘presence’ (feeling of being there). Wider field of view, varifocal, eyetracking, HDR, etc. These features will be introduced to consumer headsets as costs are reduced.

VR experiences are very effective even with current consumer level technologies.

But AR has a much harder set of technical challenges and problems to solve before we find ourselves wearing the “AR glasses” seen in a number of films and television shows over many years. 2 great examples of AR glasses and contact lenses in media are Hulu’s Mars mission television show “The First” (Sean Penn), and Clive Owen’s recent film “Anon”.

AR overlay in movie “Anon” with Clive Owen and Amenda Seyfried

Facebook Reality Labs, Apple and Microsoft are amongst those companies employing lots of very smart people to try and figure it out as the race to replace the smartphone with AR glasses is underway. Of course Microsoft had their kinect sensor technology from the gaming console business, which was further developed for Hololens.

Welcome to the future….

Microsoft’s Hololens AR standalone headset has available since October 2016 in the UK, with a new version shipping right now. Being a special order device aimed at enterprise customers, it’s been difficult to get any hands-on, until Microsoft did a launch party for their new London experience store.

And of course I went back several times in the following weeks to use it again, including a quiet morning where I had a full hour using the Hololens 😘

Interesting form factor and ergonomics:- rotate the headband, push it back, adjust the wheel on the rear of the headband.
Sensors galore and awesome looking waveguides

The device was reasonably light (reported at 579 grammes) and comfortable to wear with easy adjustment system using an rotating headband which is pushed back to fit, and then a simple adjustment wheel on the rear of the headband to change the circumference.

The holographic display was surprisingly impressive with the limited field of view not as severe as I had been led to believe. Yes it was limited especially compared to my VR headsets, but after all…it was using holograms 🤯

Holographic resolution and brightness were sufficient to create a convincing illusion, it was better than I had expected from reading many reviews prior to trying it myself.

I first did an experience focused on the current London location but with an AR overlay showing a historical scene with horse and carriages rolling past outside, which felt really magical.

Then I used several applications which were already onboard, with one showing how to use hand gestures; it was here the limitations of hand tracking were evident with it sometimes requiring several gestures actions to trigger. Despite that, it was great fun when it worked with the freedom of hands free computing.

However the lighting conditions were not optimum with lots of sunlight and people moving about,vso it would need testing in another location to determine the reliability of the gesture recognition.

Microsoft list the device capabilities as follows:-

Using the following to understand user actions:

    Gaze tracking
    Gesture input
    Voice support

Using the following to understand the environment:

    Spatial sound

Having an amazing time playing with Hololens

Overall I found Hololens to an impressive demonstration clearly signalling the huge potential for AR glasses.

Most importantly, it passed the “WOW!!” test, which is the potential of any HMD to make you pull the wow! face. This is clearly seen in the image below, wow!

Having now used Hololens several times, I’m really looking forward to trying it’s successor the Hololens 2.

I’m also very interested in following the development of augmented reality glasses as the successor to the smartphone. Google glass, Microsoft, Magic Leap, Apple, and many more to follow…

Making the transition to a “head up, hands free” computing platform has substantial benefits for skeletal posture, reduction of repetitive strain injuries, increased spatial and environmental awareness, and hand freedom to interact with the computing interface and the real world.

My experiences with the Hololens and Magic Leap has firmly convinced me of AR’s potential to change our world.

However, these 2 devices remind me of early VR headsets from the 1990’s, where potential was clear to see despite the technology being immature.

I don’t expect to see really competent AR glasses until the early 2030’s, but do look forward to trying further developments as AR technology continues to improve.

Big thanks to the people at Microsoft London for letting me use the Hololens. And thanks to you for reading! Rob Cole

Categories
Experiments

Experiments with ette…

“ette is the first of its kind finger-tracking controller. Powered by TG0’s patented technology, etee allows user to control VR without gloves, camera, or other encumbering equipment. Etee is lightweight and intuitive to use. The controller has a battery life of 8 hours that enables the user hours of time to build, explore and immerse themselves into the world of VR”

Product Description, TG0

After reading Tony’s (a.k.a Skarredghost’s) recent review of the ette controllers I was intrigued and wanted to know more – especially in light of some ergonomic concerns he raised in his article:

This was a comment I posted on his blog post after reading the review:

“Hi Tony, Great review, very detailed. The ergonomics aspect is an interesting problem, due to wide variation in hand sizing. Typically we have 3 blunt measurements:

1. length: measured from tip of the longest finger to crease under the palm.
2. breadth: measured across widest area where fingers join the palm.
3. circumference: measured around the palm of the dominant hand below the knuckles excluding the thumb.

Creating a gripped or held object to cater for the wide variation is an interesting challenge.

This challenge effect many objects whether pistol grip on weapon, household item like cutlery or cooking tool, and of course VR controller; the Valve Index controllers (Knuckles) have the ability to use clip-on Palm boosters.

I’d be interested to try the TG0 Etee controller, without getting my hands into them it’s impossible to make any valid judgement; however some physical adjustment within their structure (a clip on spacer or sliding component with locking) may be beneficial to optimise the fitting for different hands.” 

Happy XR! Cheers. Rob Cole.


Tony introduced me to TG0 who are also based in London, UK. They quickly arranged a courier delivery and within a couple of days a small box containing an Etee dev kit had arrived; many thanks to TG0.

Packaging was neat and minimalist, although I must admit I damaged the cardboard box trying to remove the controllers as their handles were very firmly wedged into the foam. I resorted to pulling the entire foam slab upwards to release them from the box, which freed the controllers but also separated the adhesive tape holding the box together.

Physical examination

The etee controllers are wonderfully simple, yet very sophisticated in terms of material technology compared to the mechanical switches and sensors we commonly see in motion controllers.

TG0’s “secret sauce” is their patented material technology used to build the sensing strips, which I understand is already used in different commercial applications – this being their first foray into motion controllers for XR.

TG0 list these as the key features of their “Etee” motion controllers:-

-Multi function TG0 thumb-pad technology

-Wireless connection

-5 finger 100 level of sensing

-6hr continue using battery life

-Gesture sensing

-Magic Trackpad with swiping, scroll, rotation and pressure sensing

-Soft silicone shore A 60 touching surface

-Weight: 75g

-Size: L136 x W60 x H30 mm

Despite coming in a little heavier than advertised at 88g vs.75g, a single Etee controller is still considerably lighter than one of my Valve Index controllers which are 197g each.

I’m also using 3D printed Valve Index ‘Palm Booster’ clip-ons, which I have to use to put controls at correct “reach” and to increase controller body “volume” to suit my medium sized hand fit, but this takes a single Index controller up to a weighty 224g.

Releasing the Palm Booster was a smart move by Valve’s designers to widen the range of hand fits possible on a single device, and many Index owners placed orders with 3D printing shops soon after Valve released all the Index mod files for free under the Creative Commons License.

From nearly a year’s experience of using “naked” Index controllers and “Palm Booster” Index controllers, the clip-on added an important tactile enhancement as the additive printing process creates a warmer, more textured surface.

This helped balance the “material mismatch” I’d experienced as a proprioceptive challenge between the soft, warm fabric strap and cold, hard plastic controller body in the weeks following the Index launch.

As well as the weight difference between the Etee and Index controllers, there is a big difference in physical size as can be seen in the side by side image below:

The Valve Index is used for comparison here as its the most advanced VR motion controller currently available to the consumer, whilst Etee offers a new approach to hand input championing a revolutionary material technology and different way of thinking.

Unlike many controllers past and present, Etee lacks physical buttons or triggers, instead providing full finger tracking across a rubberised controller body – an oval shape 26mm deep facing the palm (palmer side) and 32mm wide across the hand, combined with a large tracked thumb pad up top.

The low durometer (60a shore) controller skin has moulded vertical ribs which curve around a glowing status window, giving a soft, tactile feel with a small moulded ridge to separate the index and ring finger.

This moulded ridge protrudes 6mm from the surface, what I took to calling the “index” ridge (not to be confused with Valve Index!) to assist in locating and separating the index and ring fingers.

Controllers are marked “L” (left) and “R” (right) at the top of each handle as they are orientated specifically for each hand.

Perhaps the standout piece of the controllers is a rose gold coloured “finger bar” which provides a stiff brace for a sculpted foam cushion which is glued with adhesive tape to a channel along the inside face of the bar.

This foam bar “sandwiches” the finger against the rubber controller body, effectively capturing Etee to the hands without requiring any adjustable straps or safety lanyards which is a good “quality of life” improvement if regularly putting on and taking off.

This foam cushion is punched through with 8 triangular holes which increases its flexibility, perhaps helping to accommodate larger fingers. The foam is a relatively high density, possibly a requirement to maintain structural integrity as its only 12mm wide and 10mm deep at maximum.

The foam also has a forward extension protruding 9mm at the same vertical height as the “index” ridge on the controller skin, although rotated approximately 30 degrees around a vertical axis to the side of each ridge.

This locates the index finger of each hand and secures firmly in place by encircling that finger, whilst the remaining fingers are left “open” especially the little finger (pinky) which floats about unencumbered.

FITTING NOTES

My first quick setup attempt at fitting Etee was strange, being in a rush I pushed them on horizontally through the finger bar foam, finding a comfortable position across my “middle phalanxes”. I had tried pushing the controllers further up my fingers but found the opening between the body and foam a bit too small, I was concerned to snap or bend the finger bar if I forced them.

They felt odd but seemed to fit in some way, until both Tony and TG0 pointed out from a photo I’d emailed them, that I had them incorrectly fitted! So much for reading the instructions….

The hand anatomy image below shows the names of the 27 different bones that make up each human hand. From this image we can see the Middle Phalanx where I’d incorrectly first fitted them, and the Proximal Phalanx which was the correct location.

Image: Paul Jarrett, Murdoch Orthopaedic Clinic

TG0 advised me to fit by “sliding” the controllers vertically down over stretched hands, aiming for the foam to make contact with the proximal phalanxes of the index, middle and ring fingers. I watched the videos on their website again and it started to make more sense.

I tried fitting them again, and despite being a tight squeeze that felt sure to rip the foam off the finger bar, I finally got them on and gave my fingers a quick outstretched wiggle which felt very liberating as I now had “hands free” controllers firmly clamped in place.

I also found it a lot more comfortable to remove my wedding ring as this was being uncomfortably squashed into my ring finger during the fitting attempts.

SETUP

Setup was very easy with a USB dongle, an email I had earlier received from Tg0 contained a download link for their software.

These dev kit controllers have 3 degrees of freedom (3DoF) tracking like the older Samsung GearVR and Google Daydream headsets and remote controllers.

This means only rotational motion is tracked around a fixed location:- pitch, yaw, and roll, but not able to move forwards, backwards, side to side or up and down.

All contemporary PCVR systems and the Oculus Quest stand-alone use 6 DoF tracking for headsets and controllers; 6DoF allows movement forwards and backwards, up and down, left and right (translation in three perpendicular axes) combined with the rotation (pitch, yaw, and roll) of 3DoF systems.

I was interested to see how 3 DoF motion controllers would mesh with a 6DoF headset, in my case the Valve Index. My previous experience of using two different tracking systems together was limited to brief sessions with Lenovo’s Mirage Solo and HTC’s Vive Focus.

Both of those devices 6DoF headsets felt comfortable and immersive but compromised by a “fixed point” 3DoF controller which acted like a strange laser pointer fixed to my hip.

Is 6DoF possible?

It is possible to attach an HTC Vive Steam VR tracking puck to a special bracket, but after speaking to TG0 they advised me to wait for the new 6DoF SteamVR version of ette which is due later this year.

For the purposes of these experiments I limited myself to using the Etee visualiser whilst viewing the desktop through my Index headset, as I was more interested in the direct “hand feel” of using the controllers rather than interacting with virtual worlds.

After charging the controllers using 2 USB leads provided with the dev kit, I installed the software and took them for a test drive after starting steamVR alongside the Etee visualiser.

Calibration and rotation are easily handled through the Etee application, with a simple keyboard input starting each process, and on-screen instructions which indicate the position the controllers should be orientated against.

After getting them calibrated and setting rotation, I started my journey…

SO WHAT ARE THEY LIKE?

They feel surprisingly light in the hands, though unstable as they roll inwards towards the palm as the fingers are closed.

For quick comparison I looked at what happened when squeezing an Index controller and what happened when squeezing an Etee controller

Because Etee is clamped to the upper bones (proximal phalanxes) of the fingers rather than across the palm like index, closing my hands causes the controller to rotate as my phalanxes change angle, a large gap of 30mm before making contact with my palm.

This also causes the thumb to pivot backwards, if using the thumbpad whilst trying to close your hand! The image below shows the immediate difference in how the Etee and Index controllers are clamped to the hands.

Moving my fingers produced an immediate response in the Visualiser application and I started noticed haptic feedback loudly buzzing inside the controllers as “gestures” were made using combinations of fingers.

The most impressive aspect was their finger tracking, which I found to be very reliable once the controllers were fitted correctly and calibrated.

Individual finger tracking was almost flawless, even when lifting fingers off the body surface and placing back down in a slightly different pose; unlike my Valve Index controllers which sometimes struggle to maintain accurate per-finger tracking despite the “drum roll” recalibration and hardware reset tricks.

Etee’s pinky (little finger) tracking was especially reliable, and allowed me to finally include my little finger as a reliable participant in my experiments

Another impressive feat was their sensitivity to finger pressure, as mentioned in their literature “100 levels” per finger and its easy to understand this is no idle boast but a reflection of the precision of their material sensing technology.

This sensitivity allows controlled ramping up of pressure, and it was great fun “driving” my fingers and watching the finger levels rising from green to orange on the Visualiser.

I found the calibration process which uses outstretched hands created a strange finger input pressure if I tried to relax my hand into a neutral pose; try relaxing your arm and hand, and watch your fingers curl inwards as if holding a drink.

I tried to recalibrate from this neutral position but it caused the calibration to go a little wonky so I reset it to the outstretched fingers position again. This meant unless I kept my fingers outstretched (an unnatural pose) the input pressure rose typically on the pinky and ring fingers.

In terms of hand fit, I immediately found the foam finger bar too close to the controller body, causing my fingers to feel compressed to the point of discomfort, limiting my session time.

This sensation felt much more pronounced as Etee are securing the controller to the hand by clamping a foam block across the dorsal side (backside) of the Proximal Phalanxes.

The dorsal side of the hand is unused to pressure as its pretty much unused in everyday life, unlike the palmer side which is familiar with contact as we hold objects by using fingers to pull objects against our palms (try picking up your smartphone).

I refer back to my ergonomics article on the Valve Index controllers:-

The skin of the palmer side including the fingers is tough, thick and hairless with your fingerprints (double rows of papillae) help you grip objects as well as protect the skin from ripping.

Straight away with the Index controller I have the odd sensation of having something clamped across the back of my hand, it’s unusual as it goes against a lifetime of not having anything clamped across the back of my hand!”

The Etee controllers go one further than the Index controllers in clamping to a more sensitive part of the hand with thinner skin, and clamping using a high density foam of narrow width which is readily felt as it localises clamping pressure causing a proprioceptive challenge like Index.

Where Etee is less problematic is that the hand has soft rubber on side and foam on the other with Index having fabric on one side and hard plastic on the other which causes a material mismatch.

There wasn’t any way to adjust the position of the foam, as the bar was a fixed length to the controller body, so I turned my attention to the foam itself; thankfully this was only secured with adhesive tape so it quickly came loose.

Now with a blank finger bar, I searched through my fitting toolbox and found a number of different pads from bicycle helmets as well as foam pipe lagging of different thickness. Getting to work I trimmed different pads and stuck them in place, quickly realising the difference between ‘just too tight’ and ‘just too loose’ was critical to hold the controllers firmly in place without discomfort.

The alternative cushioning was interesting to try out, but made the controllers less stable as the material was a little too soft allowing free movement of the fingers, it needed to be a higher density like the original Etee foam cushion.

Moving further I removed the finger bars completely and tried using the controllers “naked” which was interesting, though a bit frustrating as without clamped fingers the sensitivity was all over the place making reliable inputs less frequent.

After spending a while trying the Etee controllers without their finger bars I decided to strip them down further and had a good look inside after peeling off the rubber controller body. The sensor strips were clearly visible, and some internal hardware.

Before putting the Etee controllers back together, I wanted to examine how changing the shape of the controller body would alter the hand fit so cut and taped together a larger diameter foam shape with a forward offset to create a relaxed hand pose.

Without the finger bar the controller felt like an oversized GearVR or Daydream remote and surprisingly comfortable to hold though the sensors did not register buried far underneath the foam. The big difference was that the controller did not “roll” in the palm when pulling the fingers as there was no gap with my hand “filled” by the new shape, creating a more stable grip pose.

Looking at other grips used for different sports equipment, handlebar grips on bicycles were an immediate comparison.

Round, constant diameter rubber grips are used on flat-bar sports and mountain bikes to allow the hand to move dynamically as the rider moves their body about on the bicycle, especially when standing up or sitting down. This instability of grip in the hand is what helps give a sports bicycle their agility, but at expense of stability and comfort.

In contrast, shaped ergonomic grips are commonly used for leisure bikes which are typically ridden in a seated position; another use for ergonomic grips is for riders with ulnar nerve damage whom benefit from a reduction in pressure on the wrists by having the palm fully supported.

After my quick test with the foam the solution to adjust the shape of the controller body and rubber skin would be the decision of TG0 as it would require a new design of controller (though using the same technology). Satisfied that adjusting the body shape could prove very beneficial. I refitted the rubber skins and thought more about the finger bar.

Adjustment time

I needed to create an adjustable distance finger bar; using the original high density foam piece and accommodating for different hands by moving the finger bar in or out, rather than trying to adjust a fixed distance using different thickness foam pieces…which hadn’t proven successful so far.

After sketching out the existing design and looking at the measurement I realised that creating a method of adjustment was going to be tricky due to the lack of physical space around the finger bar and body junction.

Following a number of iterations in my workshop using different cog parts (commercial off the shelf goods) like metal brackets from bicycle mudguards, plastic light fixings and different sized metric fitting bolts I finally managed to create a working prototype which actually surpassed my expectations in terms of its adjustment range.

By removing the finger bar’s attachment block (which has 2 angled keys that lock into slots on the body) I created an empty volume of space in which to fit an adjustment mechanism; 2 steel plates with oval slots to allow “infinite” sliding adjustment within its range.

A fixing bolt was captured with a knurled plastic knob with its own internal screw thread, and various bolts, nuts, grub screw used to attach the controller body to one of the steel plates, and to attach the finger bar to the other steel plate.

Finally I cleaned up the finger bar with isopropyl alcohol and firmly stuck some male velcro in place, and female velcro on the rear of the original finger bar foam so that the foam cushion could be adjusted from left to ride along the chanel of the bar – this would allow fine tuning of the foam ridge relative to the index finger on each hand.

After assembling my prototype I sent an email to TG0:

Previously I found the controller squashed my fingers which was quite uncomfortable and limited my time using them, unfortunately no way to adjust the clamping force though I tried a number of different cushions inside the finger bar (always slightly too loose or too tight). 

I quickly realised a method of adjustment was required. 

I went through many iterations to try and get this right as space was very limited whilst it needed to be easy to adjust whilst wearing, and offer “infinite” adjustment within the range of movement (clicks/notches are often just too tight or too loose).


This modification allows 3 adjustments to accommodate a wider range of hands/fingers. 

1. Distance (depth) from controller body to finger bar 

2. Angle of finger bar relative to controller

3. Lateral position of foam relative to index finger

Adjustments 1 and 2 are made using the black knurled knob which loosens and tightens a bolt, easy to do whilst wearing. 

Loosening the knob allows forward and backwards sliding movement to set the distance and adjust angle if required; tightening the knob locks the position. 

Adjustment 3 is made by repositioning the foam on the finger bar, because the foam is now velcro backed it’s easy to adjust left to right(can also swap to different shapes or materials)”.

The proof is always in using something practically, so I slipped both controllers on and started the Visualiser. For the purpose of this experiment I had modified only the left controller, as I didn’t want to risk damaging both if it didn’t work properly.

Thankfully, this was not the case, with the new adjustable left controller allowing me to find the ideal depth and best angle for the finger bar foam to maximise comfort whilst maintaining enough clamping pressure to retain the controllers in place.

This adjustment was easily done whilst wearing both controllers, with the knurled knob very easy to turn due to its vertical orientation.

The adjustment angle of the finger bar foam was limited by its square cross section, perhaps cutting a slight chamfer on each outside edge would soften its presence and allow more angle adjustment whilst providing enough flat contact against the load bearing bone to stabilise the controller.

Having the adjustable left controller and fixed right controller to compare against each other allowed easy assessment of the differences in comfort; the adjustable controller now accomodating my hand fit without squashing my fingers.

In terms of function, still very similar with the controller body rotating towards my palm every time I closed my hand.

Clamping the controllers to the proximal phalanxes will always cause this rotation but without a 6DoF version to assess using steamVR applications, I was unable to properly assess the etee controllers as virtual reality input devices.

The question of whether the mechanical switches, for example the trigger common to existing motion controllers, can be replaced by TG0 sensing technology; wasn’t something I managed to properly assess during my experiment.

The day following the completion of the adjustable left controller, the rigid plastic finger bar of the right controller snapped across its middle whilst I was removing the controller, putting an end to my experiments with Etee.

I suspect it fractured from being overloaded by my fingers being too large to accommodate, stressing the plastic over time and failing at the weakest part of the finger bar.

After speaking to TG0 they advised me the plastic was not production strength but a weaker 3D printed piece; this failure should not occur on production controllers and so my failure with the Dev kit should not be considered representative.

Immediate improvements to my adjustable concept would be a more rigid adjustment structure as the steel plates were rigid enough for my testing purposes but would benefit from being stiffer for day to day use especially for larger hand users with more strength and finger leverage.

Additionally, a “captive” bolt for the sliding adjuster plate to make it even more responsive to adjustment (the existing bolt can start turning if the knurled knob is unscrewed too far).

Finishing my time with the Etee controllers, I thought about the improvements that could be made to their ergonomic shape, and adjustment to suit a wider percentile of users.

The sensing technology itself was very impressive, and with a successful Kickstarter now backed and a SteamVR version due towards the end of the year there is a lot of potential waiting to be unlocked from this interesting product.

Thanks to TG0 for supplying this sample, I wish them luck in their developments. And thanks to you for reading! Rob Cole, immersivecomputing.org

Categories
Experiments

VR Dolly

My latest experiment…

VR Dolly for my PCVR rig

So what is a VR Dolly?

Dolly’s have been used for many years for moving heavy furniture, pianos and also a variant is used on film sets where the camera is mounted on a Dolly to allow easy movement.

Film dolly image: Eugen Puckovsky

It’s also a great piece of equipment for hosting and moving a heavy PC.

Since moving away from the standard desktop mounted on a desk, I’ve been experimenting with different methods to mount the PC whilst allowing it to be easily moved inside a room, or out of the room if required.

Coming soon….

Categories
Experiences

Immersivecomputing @ Vertigo VR, Milton Keynes

Visited May 2019

I needed to travel to Milton Keynes for a family gathering but managed to sneak in a visit to Vertigo VR which is located next to the Central shopping centre.

Arriving early, I had a good look at the building which was quite substantial, the VR centre seemed to occupying the entire upper floor.

Soon enough the doors opened, as the first visitor I had the place to myself so had a good look at the artwork across the walls.

Heading upstairs I found the staff behind a bar, and bought some tickets to try out the solo experiences.

Ferocious motion platform

I started with a funfair ride simulator hosted on a standing motion platform and using Rift CV1. The ride was a platform on a long arm lifting me about 100 feet into the air before swinging back down, above a carpark.

This was brutual with the simulator almost flooring my legs everytime the ride swung around, I managed to hang on (for life!) until thankfully it ended. Probably not one to try again…

Cool space pods

The second experience was slightly more tame, taking place inside what looked like a space pod from the 1970’s. The experience itself involved rapid flight with lots of banking and turning around big buildings in a city.

I enjoyed this much more than the funfair simulator, and had a long session until a family member arrived, thankfully they agreed to try VR for the first time so we booked a multiplayer session in the HTC Vive booths at the rear of the venue.

We had a great session across several games ending with cooperative mode Arizona Sunshine – the first time I’d tried it, it was awesome protecting each others backs as the zombie horse advanced.

Soon enough, our time finished; thanking the staff we walked away talking about their first experience of VR (basically a wow!)

Thanks for reading! Rob Cole, immersivecomputing

Categories
Experiences

Immersivecomputing @ Raindance Immersive Festival 2019

Following my first visit to Raindance in 2018, I was keen to go again and waited patiently for the organisers to open up their website ticketing for the 2019 event.

It was tricky working out which experiences to visit during each timeslot so I tried different combinations until I had a full day of experiences booked for the final Sunday.

The venue was the same at The Oxo Tower in London, it felt comfortable to be back at a familiar venue as I walked up the stairs.

After signing in I received an armful of coloured wristbands and a programme of events with a floor layout showing where each of the experiences were located. The event was starting to fill up with guests eager to get into VR.

HTC Vive Pro Eye

I started off with “Gloomy Eyes” on the HTC Vive Pro Eye headset, my first time seeing one or using one since HTC launched the new model.

With the eye tracking inoperative it was the same Vive Pro I’m familiar with, generally a decent headset with a robustness ideal for public or enterprise use where they don’t always get well treated!

The experience itself was very…gloomy…but very cinematic and awe inspiring. Hopefully it will get a home release very soon!

I was getting some light leakage at certain angles due to the large windows facing the River Thames, so tried to find a better pose in which to enjoy the experience. I was glad to be using an OLED display headset as it was very dark inside Gloomy Eyes.

Eye tracking module inside Vive Pro Eye

After Gloomy Eyes, I tried more experiences, all using Vive Pro Eye.. they were everywhere at the festival alongside some older headsets.

Doctor Who: The Edge of time was fun though I managed to get snagged in some virtual scenery which prevented me completing the demo.

No man’s Sky VR was my next experience, I failed to get out of the crater before being irritated as I grappled with the Vive wands. 🥴

My final experience from my first session was “The curious case of the stolen pets” which was on the new Oculus Rift S; a headset I’d not been available to demo anywhere in London.

Oculus Rift S in all its glory!

This was a great opportunity to see how different it was from the Rift CV1 which I’d owned in the past, it was reportedly using the single display panel (overclocked) and new Fresnel hybrid lenses from the Oculus Go.

The fixed IPD and lower refresh rate (80hz) were something that had concerned me, though my IPD at 63.5 was right in the sweet spot with the lenses proving Oculus’s prowess with lens design (often requiring a lot of ray simulation involving a supercomputer).

Pet rescue on the Rift S

I was initially impressed with the Rift S, the display was clear with minimal screen door and the lenses were clear of artifacts. The experience was very fun with a large puzzle I spun around as I tried to rescue the pets, I solved the first two before running out of time.

However after months of using the Valve Index, the lower 80hz frame rate on the Rift S was very noticeable and felt sluggish in VR.

Great lenses in the Rift S

It didn’t give me a great feeling of presence, partly the lower frame rate and partly the smaller FOV, which felt like scuba goggles again after Index and actually seemed slightly smaller than CV1.

Touch insight (left), Touch CV1 (right)

The Touch insight motion controllers worked well, but felt a bit creaky and unbalanced compared to the sublime Touch CV1 controllers which perhaps shall remain the gold standard.

Oculus Touch insight controller

Overall the Rift S felt like a sidegrade in some ways with steps forward and backwards, certainly very good value at £399 although the frame rate remains a concern – 90Hz should be the minimum for PCVR with Index proving 120Hz the new standard.

As my first session finished I felt very satisfied having tried the Vive Pro Eye and Rift S, but these were quickly forgotten when it dawned on me that my next session was using something even more rare, that I didn’t think would be available to demo.

What could that be??

“Rise of the Animals with David Attenborough” immersive AR experience on the Magic Leap was awaiting, of course I needed to have a really good look at the hardware 🤯

Sensor array on Magic Leap
The ML goggles
See the waveguides?
Compute unit on belt
Remote

After looking very thoroughly at the entire kit, it was time for my first Magic Leap AR experience.

The lady running the experience helped me fit the Magic Leap on my head, I hung the compute unit belt on my shoulder but didn’t need the remote as the hand tracking was being used.

Making gestures using hand tracking to start the experience

The experience was…. very impressive. I’d read so much negatively and poor reviews about Magic Leap I wasn’t sure how it would work and how effective it would be?

I’m not going to spoil the experience itself with any spoilers; the field of view was limited (as many had mentioned in reviews) but serviceable, the 2 depth focus planes were very welcome after several years of fixed focus VR headsets.

I had dinosaurs crawling about everywhere in a huge space so I went wandering about following the creatures…much to the amusement of the other guests!

Overall I was impressed with Magic Leap which gave a really good impression of how powerful AR can be once the technology develops.

It certainly got me moving about and using the entire space, the image quality was good and effective. Environmental tracking and hand tracking also impressed. I’ve used Hololens in the past, recently only a week before Raindance, and thought Magic Leap was a superior device in many ways.

After Magic Leap it was time for lunch, before another round of experiences. I spent my lunch talking XR with developers, volunteers and people from different companies and university’s, including some people from Bose who were showing their new AR sound glasses.

These were interesting to try and quite effective, I tried a couple of different audio experiences. The glasses provided good audio presence but were a bit overpowered by background noise.

After Bose, I had some fun using the Oculus Go which has always been good for shorter sessions as it’s a bit front heavy like the Quest.

Many Oculus Go charging alongside a Vive Focus

Oculus Go experiences included Anonymous, Playing God and Afterlife. This was pretty creepy showing the aftermath of a child’s death and it’s impact on remaining family members specifically the mother.

Go is still a great device for media consumption and 3DoF experiences especially when paired with good audio headphones. Its ideal for festivals being inexpensive and easy to deploy in volume.

Probably the strangest thing about going to immersive festivals is lifting off a headset to realise you are in a room full of people wearing headsets! It’s a bit odd to witness, and I had the thought that many headset owners would appreciate the opportunity to try the experiences and demos at home.

I spoke to the organisers and proposed an idea for the next festival; offer ticketed access so headset owners could participate online during the festival with time limited access to the different experiences.

Perhaps the last time I’ll see Oculus’s Rift CV1 at a festival? This solitary headset did it usual stellar duty!

I thoroughly enjoyed my time at Raindance Immersive Festival 2019, huge thanks to the organisers, the developers and hard working volunteers assisting the guests.

Thanks for reading! Rob Cole, immersivecomputing

Categories
Experiences

Immersivecomputing @ Raindance Immersive Festival 2018, London

Visited October 2018

I heard that Raindance were holding an immersive festival in London at The Oxo Tower, right at the end of their month long film festival.

I managed to book an entire day’s worth of experiences before they sold out.

The festival had taken the entire first floor, providing a large long space ideal for multiple VR demos.

I arrived on the Sunday morning for opening and it quickly filled up with people eager to try their booked experiences.

Walking around I noticed many different experiences on offer across a wide variety of headsets.

A big draw of these festivals is the opportunity to try experiences and applications which are not generally available as home releases, or are exclusive to the festival.

A big draw for me was the World premiere of HTC’s “7 Miracles” on the Vive Focus; the second time I would get to use the Focus and the first opportunity to use it for any length of time.

The Archivist :VR

I found my first experience which was “Transference” by Spectrevision which was super creepy in the Oculus Rift CV1; Spectrevision is partly owned by actor Elijah Wood who is apparently heavily into VR!

Transference on Rift CV1…thanks Elijah Wood!
Vive Pro – Awake part 1
Oculus Go being used by many people
7 miracles on HTC Vive Focus
HTC Vive Focus standalone headset

The HTC Vive Focus – 7 Miracles

I’d enjoyed using the Vive Focus briefly at the HTC launch event but only had 5 minutes of playtime so couldn’t get a sense of comfort.

One reason for booking the 7 Miracles demo was the offer of 1 hour and 20 minutes runtime, available in one sitting. The experience was extremely well put together and not overly religious (I do not practise any faith). I found the story effective, acting was excellent and you could tell some serious money had been spent on the production. The resolution of the headset was impressive compared to the Vive and Rift which I was more familiar with.

I featured in “The Verge’s” coverage of Raindance

After about 30 minutes, I started feeling a bit uncomfortable in the headset, with pressure building on my forehead in the sinus area. I tried shifting the headset around which provided a temporary solution. The headset felt front heavy due to onboard compute, and the halo style band didn’t really work for my head shape. This left my forehead supporting the weight, after 45 minutes I started getting a mild headache, but struggled on to the end.

I thoroughly enjoyed the actual VR experience, but found the focus incompatible with my headshape; I would need to modify the face cushion or headstrap for it to work. Despite the seemingly plush looking face cushion, it didn’t have enough density to support the headset weight causing a hard plastic part to press through the cushion against my head.

I ended my morning session with a headache and sore forehead which was cured with some painkillers and a quick freshen up with warm water in the bathroom before taking an hour break for lunch.

Still, i was very glad to try it out and my afternoon session started with the comfortable Oculus Go, so no further headaches!

A gaggle of Go’s?
7 Miracles on the Vive Focus
Oculus Go – Grenfell experience
“The Apple” – Backpack PCVR with 4 x steamVR 2.0 lighthouses
“The Apple” – Backpack PCVR with Leap Motion

I had a great day at Raindance Immersive Festival, and thoroughly enjoyed the different experiences as well as getting to use a number of different Headsets.

Something going on in there!

It’s always great meeting new people in the immersive computing space, and getting to talk with them about their applications and experiences, and how people are receiving them.

Doing something intricate in VR

Many thanks to the organisers and all the developers for bringing their applications and experiences to Raindance Immersive Festival.

Thanks for reading! Rob Cole, immersivecomputing.org

Categories
Experiences

Immersivecomputing @ EGX 2018

Visited September 2018

After earlier accepting an invite from my friend Callum to be on the discussion panel for “The Future of VR” at EGX 2018, I travelled by train to Birmingham’s NEC.

Having never been to EGX before, I wasn’t sure what was happening across the show, but was pleasantly surprised to find a good number of VR demos and experiences on offer.

Sony in particular had made a huge effort for PSVR with numerous demo booths and events occurring.

With several hours free until the panel discussion started I hit the show floor and set out to demo everything I could find.

It’s always a pleasure to use PSVR with its clear display, comfortable fit and wonderful software library, “The Persistence” was very impressive. Sony had a neat booth setup taking digital photos and giving out colour prints so I obliged!

After trying several PSVR demos I moved onto PCVR and found multiplayer game “Skyfront”.

Skyfront on the HTC Vive was great fun and very challenging. After talking to the developers I understood more about their development process and the difficulty of optimization for large maps.

Moving on, I found “Titanic VR” running on the Oculus Rift CV1 with a gamepad instead of Touch controllers.

The eyewitness scene in the lifeboat was unnerving and effective with some great character animation and voicework.

The ROV section gave a great feeling of presence, but started making me feel motion sick due to the disconnect between the ROV movement and my seated position; enjoyable to witness but good to know when to stop.

I then found an Oculus Go with a starfighter game (cannot remember the name) which was great fun and banished any feelings of motion sickness as I engaged in combat until I’d hogged the demo for too long!

Walking around the show I also found lots of simulator rigs being test driven with some very expensive and exotic setups as well as more affordable offerings.

After finding some lunch, I came back to the show with an hour to spare and wandered about looking at all the shiny stuff until I met this chap, legendary overclocker Ian “8 Pack” Parry.

Talking tech was interesting and he said he’d sort out a nice overclocked 8086K motherboard bundle if I got in touch with him at Overclockers.uk (I took him up on this offer a month later)

As usual, all kinds of ridiculous PC were on display showing off liquid cooling, RGB and custom modifications.

It was time to head over to the theatre where the discussion was taking place so I threaded my way back through the show watching many people having fun in VR.

“The future of VR” discussion started with a good number of people in the audience, and 3 other panel members including a prominent YouTuber, a VR specialist from The VR Concept and a Public Relations manager from Skyfront VR. Callum did a great job in hosting the event and keeping the discussion going forward as there was much to discuss.

I had a great day at EGX and made to check out some other VR experiences I’d missed before lunch; of course I had to go and look at the crazy PC’s again before getting a train home!

Thanks for reading! Rob Cole, immersivecomputing