Mapping Physical Human Joint-motion in Physical World with Virtual Reality 3D Skeleton Through Multi-Sensory Environment


·  Two thirds of the patients do not perform exercises correctly at home.

·  A patient needs constant coaching from the therapist while performing exercises.

·  A therapist has no way of knowing if a patient performed exercises at home correctly or not.

·  There is no way for a therapist to correct a person performing exercise at home.


·  We have developed a sensor based motion capture environment that allows a therapist to record a therapy.

·  The therapist can make a video recording of the therapy and upload it to a cloud storage area for the patient to download and view while performing the exercise.

·  The patient can see her avatar/stick figure superimposed onto the stick figure/avatar of the patient while performing exercise.


·  Powerful gesture detection sensors record minute differences in joint movements and report full details of the exercise performed at home to the therapist.

·  Advanced analytics can provide detailed reports, graphs and charts regarding recovery.

·  Visual feedback allows the patient to correct exercise immediately.

·  Social media integration can help people recover quickly by sharing their experiences and supporting each other.

·  Big Data back end can store data generated by millions of users simultaneously.

·  Computational Intelligence can be used to study and discover new recovery parameters and patterns based on Big Data analysis.

·  The 3D skeleton displayed on the screen allows the user to explain to the patient the exact details of her disability which may help her in focusing on the right group of muscles to perform the exercise properly.

Value to the Kingdom of Saudi Arabia:

·  The 3D skeleton can be used for pedagogical purposes and different parts of the skeleton can be labeled in Arabic to make it easy to understand for both patients and students.