In the previous part, we’ve re-compiled Spectator View using the latest HoloToolkit sources. At the time of writing, this resulted in a compile error, as the HoloToolkit for Unity has seen a breaking change since Spectator View was released.
In this part, we’ll first fix the Spectator View code, and then set it up correctly in our own Unity scene. Then, we’ll need to check several other things: how to launch the Sharing Service, adding Internet Connectivity and the Holographic Camera prefab.
Fixing the Spectator View Code
From the two choices at the end of part 4, of course, we go with the adventurous road to fix the code 🙂
Turns out that the AddSurfaceObject() method from the base class SpatialMappingSource was changed. Instead of directly creating a Surface Object from a mesh, this process was now split up into two parts. There is one method to create the surface object, and another one to add it.
The current “wrong” code in the Addons\HolographicCameraRig\Scripts\RemoteSpatialMappingSource.cs file (line 53+):
GameObject surfaceObject = AddSurfaceObject(mesh, "RemoteSpatialMapping", transform); surfaceObject.transform.localPosition = Vector3.zero; surfaceObject.transform.localRotation = Quaternion.identity; meshCollider = surfaceObject.GetComponent<MeshCollider>(); meshCollider.sharedMesh = null; meshCollider.sharedMesh = mesh;
Replace that with the following code:
var surfaceObject = CreateSurfaceObject(mesh, "RemoteSpatialMapping", transform);
AddSurfaceObject(surfaceObject);
surfaceObject.Object.transform.localPosition = Vector3.zero;
surfaceObject.Object.transform.localRotation = Quaternion.identity;
meshCollider = surfaceObject.Object.GetComponent<MeshCollider>();
meshCollider.sharedMesh = null;
meshCollider.sharedMesh = mesh;
I’ll probably create a pull request for that in the near future. That obviously includes moving the whole sample project to the latest HoloToolkit.
Setting up the Spectator View Manager
Now, our project should compile again. This time, without any errors! The only remaining step from the instructions is to enter the IP addresses of the sharing service PC, as well as of the HoloLens mounted to the DSLR. Turns out, that’s not all. Three game object references need to be defined:
So, where to get these three references from in our own custom project? The help is in the tooltips of the properties. You only need to drag & drop the prefabs directly from your Project view. No need to add them to your scene hierarchy. (Except for the Anchor! But more on that later. For now, drag it there anyway.)
Internet Connectivity for HoloLens
This blog post series turned out a lot longer than I hoped it to be. Is it now finally time to test Spectator View with our custom app?
Unfortunately, there are still several things to do. When executing the app on the HoloLens, it fails with the following error:
SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection on port 20608. Error code: 11
Well, obviously for the HoloSharing to work, the app now needs to have the Internet capabilities added. That’s simple to add, but easily forgotten 🙂
Go to File > Build Settings. Make sure you have Windows Store selected (with the Unity symbol next to it, otherwise select “Switch Platform”). Next, select “Player Settings…” and in the inspector panel on the right scroll down to activate the InternetClient capability.
Locating SharingService.exe: the Sharing Server
With that fixed, we of course also need to launch the Holo Sharing Server. In Unity, this should work through HoloToolkit > Sharing Service > Launch Sharing Service. In our custom app, that results in the error message:
Sharing service does not exist at location: External\HoloToolkit\Sharing\Server\SharingService.exe Manually copy SharingService.exe to this path from HoloToolkit-Unity\External
The HoloToolkit repository indeed contains this .exe file at this location: HoloToolkit-Unity\External\HoloToolkit\Sharing\Server\SharingService.exe
The SharingService.exe file shouldn’t go to the Assets folder of your project. Instead, you need to create the folder hierarchy directly in the root of your project. For example:
D:\Source\Unity\HoloLensPlayground\External\HoloToolkit\Sharing\Server\SharingService.exe
Holographic Camera
When we launch our app on one of our HoloLens devices now, it already connects to the HoloLens SharingService server. Great news! Still not done, though.
Running on the HoloLens outputs countless of the following errors to the debug output in Visual Studio:
Waiting for HolographicCamera to join and create the anchor (Filename: C:/buildslave/unity/build/artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)
This is coming from the HolographicCameraRig\Scripts\SV_ImportExportAnchorManager.cs script.
In the InitRoomApi() method, it wants to join a room. As there is no room yet, it would create a room – but only if a Holographic Camera is present. That camera is now actually the HoloLens that is mounted to our DSLR (identified via the IP that we’ve set for the SpectatorViewManager). So we have to really deploy the app to both HoloLenses in this step. That’s a bit more work, but you have to get used to that from now on.
It’s a bit tricky to launch your app on the HoloLens mounted to the camera, as you can’t really see what it’s showing with the mount in front of the glasses. Therefore, I’ve usually deployed to the mounted HoloLens second, so that the app is launched through Visual Studio and I don’t need to manually interact.
By the way, to switch the deployment target in Visual Studio to another HoloLens IP for remote deployment, go to Debug > [projectname] Properties … > Debug > Start options > Remote machine.
To see the Spectator View output, launch the compositor again in Unity through Spectator View > Compositor.
The good news is that both HoloLens devies, as well as the PC with Unity now join the Sharing server. On the downside, the position of the hologram doesn’t get synchronized yet.
How to extend our app for full Sharing support will be part of the next blog post!
HoloLens Spectator View blog post series
This post is part of a short series that guides you through integrating HoloLens Spectator View into your own Mixed Reality app: