The special lenses of the Oculus Rift require the image to be distorted. This can be done with the official SDK. Its documentation is quite good, but luckily I’ve found the upcoming Oculus Rift in Action
book with its nice source code examples
. For my code I used the Hello Rift example
as a base.
The application has two threads. On the main thread the OpenGL part happens – the camera image is loaded to the texture etc. The second thread is used for getting both camera images by OpenCV. To ensure that the two threads don’t mess each other up, a mutex
and two conditional variables are used.
The cameras have two compression modes: YUV and MJPEG. With the standard OpenCV camera implementation, only YUV is available. For 60fps, however, the MJPEG mode is necessary. Therefore the DirectShow implementation in OpenCV has to be used. That can be accomplished by using a camera id from 700 – 799
and then setting FOURCC to MJPEG.
Our tests have shown that accessing the two cameras in separate threads results in worse performance than using one thread. In order for the camera thread to already request the new image while the current image is uploading to a graphics card in the main thread, the image is copied to dedicated image buffers.
For each image there are two buffers. The pointers of them are swapped in the main thread as soon as the camera thread is finished. That way, it can be ensured that no screen tearing
occurs. After the swapping, the camera thread is told to resume.
In the meantime, the visible part of image is uploaded to the image OpenGL texture. Then the texture is mapped on a rectangle. This rectangle will be positioned and zoomed as necessary.
Finally the Oculus Rift SDK distorts the stereo image and the finished frame will be displayed on the screen.
You can find the whole code on GitHub
Oh, and the audio: At the moment the microphone will just pass through to the HDMI with standard Windows settings
The next article
will cover the gyrating camera rig resp. head.
If you have any suggestions or questions, please use the comment form. I am always happy to learn something new.