TechEnablement recently spoke with David Wiltz, product manager of Intel Software and Services Group, about the early release RealSense 3D camera and SDK that we tested. Intel is currently providing early access to the RealSense developer kit that contains a 3D camera and associated SDK for software development. Devices with integrated Intel® RealSense™ 3D Cameras are shipping on Ultrabook™ devices, notebooks, 2 in 1s, and all-in-one PCs from major OEMs.
David Wiltz (Image courtesy Intel)
Succinctly, we like the technology and see a big potential upside for both users and application developers. RealSense is designed for the mass market, which means that application developers have the opportunity – right now – to establish themselves with innovative applications that exploit the natural use of hands and speech for user interactions. David summarized the mass market potential for developers by noting RealSense, “enables a new type of interaction” that will be integrated into many form factors from many different OEMs.
What this means for developers is a ‘first to market’ opportunity to design a 3D gesture interface that can potentially replace the computer mouse and eliminate the messiness of using a computer touch screen (which is lovely but generally results in a smeared and dirty screen – especially with kids).
However, RealSense is much more than a touchless über-mouse as the current SDK includes modules using the front facing camera for hand tracking, gesture recognition, face detection and speech recognition.
A simplified version of the RealSense class hierarchy (Image courtesy Intel from https://software.intel.com/en-us/articles/intel-realsense-sdk-architecture)
Significant effort has already been put into the early release of the SDK to recognize joints in the hands. In our evaluation, this module worked very well and was quite responsive.
Hand joints recognized by the RealSense SDK (Image courtesy Intel from https://software.intel.com/en-us/articles/intel-realsense-sdk-architecture)
See it in action:
The SDK is straight-forward. The following example from Tips and Tricks shows how to read the hands stream:
High-resolution facial tracking is also part of the SDK. The RealSense camera also provides 3D point-cloud data, which opens the door to high-accuracy facial recognition and recognition of emotions. Computers are now better at recognizing faces than humans. Imagine what can be done with real-time 3D data!
- Available Intel tutorials:
- Unity_Getting_Started
- Face Tracking tutorial includes face location, pose, landmark, and expression detection, face recognition, alerts and code samples
- Hand/Joint/Blob Tracking tutorial covers gesture, hand and joint tracking, blob and contour extraction, data smoothing & code samples
- Raw Stream Capture tutorial :Catching and rendering color and depth streams with code samples
- Unofficial – Intel developer Amit M has Unity Toolkit Demo and RS/Unity RFBDepth Alignment videos
While we did not work with the speech synthesis and recognition APIs, the Intel flashcard demo shows the potential as does the speech controlled drone in the video below.
The Intel Scavanger bot demo show an application of 3D scanning:
Following is a case study video for a game “Head of the Order”:
Have some fun mixing technology and music:
How about drone ping pong, self-navigation down a wooded path and more?
Excited? Dave Wiltz noted that the path to market provided by Intel is the app showcase and app challenge.
Look at the current entries, create your own software, and see if you too can get your own RealSense apps recognized!
Leave a Reply