OpenXR/OpenVR (virtual reality support), new Python workbench WiP

Have some feature requests, feedback, cool stuff to share, or want to know where FreeCAD is going? This is the place.
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
The Modern Wizard
Posts: 4
Joined: Fri Oct 09, 2020 12:56 am

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by The Modern Wizard »

Quick question - Does the current newest release support Monado? The .19-preXR mentions using either SteamVR or Monado, but the .20-preXR only has SteamVR listed.

I'm not in a position to actually test the build yet (I don't have a VR headset or a Linux machine (for Monado)) but I plan to get both in the near future.
User avatar
kwahoo
Posts: 680
Joined: Fri Nov 29, 2013 3:09 pm
Contact:

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by kwahoo »

The Modern Wizard wrote: Tue Sep 21, 2021 5:05 pm Does the current newest release support Monado?
It should work with Monado, but I haven't tested it yet.
The Modern Wizard
Posts: 4
Joined: Fri Oct 09, 2020 12:56 am

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by The Modern Wizard »

Alright, thank you!
User avatar
kwahoo
Posts: 680
Joined: Fri Nov 29, 2013 3:09 pm
Contact:

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by kwahoo »

kwahoo wrote: Tue Sep 21, 2021 9:27 pm
The Modern Wizard wrote: Tue Sep 21, 2021 5:05 pm Does the current newest release support Monado?
It should work with Monado, but I haven't tested it yet.
Small update. I got FreeCAD working with Monado, without a single line of code changed.

Few things:
1. If Monado is not set as default OpenXR runtime, you have to tell FreeCAD, where it is, eg.:

Code: Select all

XR_RUNTIME_JSON=/usr/share/openxr/1/openxr_monado.json ./bin/FreeCAD
2. I run Monado with libsurvive to get positional tracking of Valve Index and controllers. It still works best with 1 lighthouse, tracking is unstable with 2 lighthouses.
3. Unlike SteamVR, closing and reopening XR session works fine. Also no glitches in the mirror window.
4. My Index does not want work at 144Hz, I had to use 120Hz MODE 2:

Code: Select all

XRT_COMPOSITOR_DESIRED_MODE=2 monado-service
catman
Posts: 412
Joined: Fri Jan 11, 2019 10:42 pm

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by catman »

@kwahoo thanks for your excellent work.
I am trying to get the Rift running offline in Linux. Making progress but not there yet.

Maybe you want to add the info on your first post to compile your branch like this

Code: Select all

 cd build
 cmake .. -DBUILD_OPENXR
 make
I was wondering why I did not get a VR menu anywhere. ;)
catman
Posts: 412
Joined: Fri Jan 11, 2019 10:42 pm

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by catman »

Whow! First time an offline Linux VR Image on the Oculus Rift!

Here is a short Summary of the not-so-straightforward way to get it running. All software was current at 2022-07
* Standard PV with Ubuntu 22.04 installation
* Nnvidia GTX1050TI Graphics card
* Oculus Rift CV1, connected to only HDMI port. Adapters and extension cables all lead to black HMD image for me.
* xrandr --output HDMI-0 --props --set non-desktop 0
note that this is different from the infos on the OpenHMD site. With non-desktop=1 the HMD remains black with orange LED.
I added the 99-HMD.conf setting as descriped to the xorg folder.
* nvidia drivers are very very very buggy for this HMD. The standard case for me was "no image on the HMD". Like 99.9% of all trials... The current reoccomended for 22.04 is nvidia-driver 515, got me no HMD image. Ubuntu repository driver "nvidia-driver 450" eventually did. Only drivers 440 or newer work, because we need Vulkan.
* OpenHMD 3.0 compiled from Source. The Repository binaries did not work.
* Monado compiled from Source. The Repository version has the OpenHMD driver not compiled in and is useless for the rift ( :roll: ). Make sure to install all dependencies, or cmake delelects important parts of monado.

Monado is the runtime service that must be started manually each time. Unfortunately in my setup I only found one combination where it worked and got me an image on the HMD. I started it with two environment variables set (start the service in a separate terminal):

Code: Select all

  export  XRT_COMPOSITOR_FORCE_XCB=1 
  export  XRT_COMPOSITOR_XCB_FULLSCREEN=1 
  monado-service                                                             # note, no sudo!
* openXR-SDK I compiled from source, but I think it should work with the repository version, except that you must not install the package openxr-monado, because this again overwrites the monado service with the version without OpenHMD support.
FreeCAD needs /use/local/lib/libopenxr_loader.so to be presen, a symlink from the default monado location will work.
* compilation of FreeCAD VR
** I downloaded the Tag "0.20_preVR"
** install as per FreeCAD wiki (again with all dependencies), except you must use this line for cmake

Code: Select all

    cmake .. --DBUILD_OPENXR
** "sudo make install" puts it in /usr/local. Not sure how to uninstall it again ...
FirstRift-FeeCAD_screne.png
FirstRift-FeeCAD_screne.png (146.89 KiB) Viewed 1949 times


The procedure to get started is still a bit involved, because the chain nvidia/monado/openHMD does not fully work.
I first start the service in an own terminal. It will open a full screen window with two distorted eye views, fully gray. Then I open the nvidia-settings tool (should be installed with the driver), the Rift is visible in the Configuration section, usually overlapped with the other monitor(s). Grabbing the monitor and moving it around to the left so it will take the full screen window with it. Now the HMD view is not visible on the desktop but on the HMD. Its a workaround until direct mode works.

Then I start the VR. Version of FreeCAD.
build/bin/FreeCAD
Since I have a symlink from the default monado location to the compiled driver in /usr/local I do not need to set an environment variable each time.

At startup the menu entry VIEW/FreeCAD-XR is greyed. When a part is loaded it can be selected and opens a new overlay window on top of FreeCAD and outputs to the monado-service window. The overlay window has no real use, maybe its gone in direct mode. The HMD gets me into the current scene. Key motions WASD are a bit limited, cursor keys crash the application (bug).

So this allows to use the VR with the Oculus Rift in OpenXR without the need to install StreamVR

So this is really a big milestone.
Thanks again to @kwahoo to make this possible!


Next steps:
I want to see my own models.
* Need to get Assembly3 running with this version, failed so far - OK
* need to get a way of moving with keys/mouse or SpaceNavigator

Edit: the Assembly3 issue is a 22.04 problem due to Python3.10. See [https://forum.freecadweb.org/viewtopic.php?t=68555]. The easierst fix was as suggested in that post adding a line to the src/Gui/ApplicaltionPy.cpp file and recompile. After that including the "Assembly3" Workbench and "A2plus" workbench to the Mod folder made it working again.

Any tips on changeing the start position of the VR view?
Last edited by catman on Thu Aug 11, 2022 6:13 pm, edited 1 time in total.
catman
Posts: 412
Joined: Fri Jan 11, 2019 10:42 pm

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by catman »

The key functions needs some fixing. I guess it was partly disabled in favour of the much more complex 3D handles.

Gladly its quite easy to enable it again. All fixed are limited to the the class in the file src/Gui/View3DInvetorXrViewer.cpp is simple to understand and extend. The crash on the cursor key(s) was due to a dangling pointer. The keys only work when the new addon-window is selected which is very good since it prevents one from typing commands by accident while under the hood.

catman wrote: Wed Jul 27, 2022 10:22 pm Edit: the Assembly3 issue is a 22.04 problem due to Python3.10. See [https://forum.freecadweb.org/viewtopic.php?t=68555]. The easierst fix was as suggested in that post adding a line to the src/Gui/ApplicaltionPy.cpp file and recompile.
The fix for Assembly3 does not really solve the problem. It allows to start Asm3, but when loading more complex files there are still many coin.py errors of the kind

Code: Select all

 <built-in function SoGroup.getChild> returned a result with an exception set
User avatar
kwahoo
Posts: 680
Joined: Fri Nov 29, 2013 3:09 pm
Contact:

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by kwahoo »

catman wrote: Wed Jul 27, 2022 10:22 pm ** I downloaded the Tag "0.20_preVR"
I'd recommend compiling the latest git version. It was rebased recently, so issues like the Python3.10 issue should not exist anymore
catman wrote: Wed Jul 27, 2022 10:22 pm Any tips on changeing the start position of the VR view?
worldTransform stores the total transformation of the world. So you can change initial position, by adding start values during initialization https://github.com/kwahoo2/FreeCAD/blob ... t.cpp#L683

Note: if you changing the translation field, change the center field too. Otherwise, the rotation will be off. https://www.coin3d.org/Coin/html/classSoTransform.html

Do you have Oculus Touch controllers? Are they supported by Monado?
There is a recent video showing OT with Monado.

Bindings are here, if you need some tuning https://github.com/kwahoo2/FreeCAD/blob ... t.cpp#L545
catman
Posts: 412
Joined: Fri Jan 11, 2019 10:42 pm

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by catman »

kwahoo wrote: Fri Jul 29, 2022 9:52 am I'd recommend compiling the latest git version. It was rebased recently, so issues like the Python3.10 issue should not exist anymore
That will be greate. You wrote your master is based on the link version, correct? When I modify models with that version, is there a risk they get incompatibilities to standard FreeCAD?

kwahoo wrote: Fri Jul 29, 2022 9:52 am worldTransform stores the total transformation of the world. So you can change initial position, by adding start values during initialization https://github.com/kwahoo2/FreeCAD/blob ... t.cpp#L683
Note: if you changing the translation field, change the center field too. Otherwise, the rotation will be off. https://www.coin3d.org/Coin/html/classSoTransform.html
Yes I saw that bit. Can you say what is the difference to setting the transformation here https://github.com/kwahoo2/FreeCAD/blob ... er.cpp#L45, seems more convenient for a keyboard interface. In my tests it seemed to work.
I need to experiment with different key functions to get something practical, e.g. some models have different orientations. I guess a simple relative model with turning and fwd/bckwrd might be easier to use but needs some form of presistence between the calls.


kwahoo wrote: Fri Jul 29, 2022 9:52 am Do you have Oculus Touch controllers? Are they supported by Monado?
Bindings are here, if you need some tuning https://github.com/kwahoo2/FreeCAD/blob ... t.cpp#L545
Actually that file was really helpful to understand how the 3D view and the VR integration works. I admired how much effort is required to cover the controllers part.
I do not have OTs and I must admit that I am not convinced that they are useful in a pure CAD workflow for me, so I do not plan to get some. I am sure they are great for arcitect customers or customers to interact with special prepared products. My use cases is to verify models during design - especially life-size models - to get a feel for dimension, position, placement, etc.
So I am looking for a seamless integration into the normal workflow - as little changes as possible: switch to VR for a minute and then back to 3V view. Throughout using the same keyboard/mouse tools all the way through. The framerates are really low but more than adequate for this use case. Also motion sickness impact is only minor.


Have you been thinking to put this into a pull request for the master (OpenXR part)? Seems to to me like a well separatable section with little risk to impact other areas. Maye with a Preference option "experimental" it could be disabled by default. Basically thats what Blender is doing as well.
catman
Posts: 412
Joined: Fri Jan 11, 2019 10:42 pm

Re: OpenXR/OpenVR (virtual reality support), arch-friendly update

Post by catman »

btw, here https://github.com/kwahoo2/FreeCAD/blob ... er.cpp#L86 and 2 lines below it should read "rotation1->...". Thats the crash.
Post Reply