Multimedia, Virtual Reality, and Augmented Reality Explained
Multimedia
Multimedia is a form of communication that uses multiple media types, including audio, images, text, videos, and animations.
- Linear media is a type of multimedia that starts at one point and runs until the end. Examples include movies, television, and cinema.
- Nonlinear multimedia allows users to control and interact with the content.
Virtual Reality (VR)
Alternative Names for VR: Synthetic Environment, Virtual World, Cyberspace, Virtual Environment, Virtual Presence, Artificial Reality.
Definition: VR is a sophisticated user interface that simulates a computer-generated environment in real-time. It uses devices for multisensory interaction.
The VR Triangle: Immersion, Interaction, and Imagination
- Immersion: The user should feel immersed in the virtual environment, giving them the impression that it is real.
- Interaction: A key requirement of a VR environment. Users must interact with the system, enhancing the immersive experience. Tracking systems capture user interactions.
- Imagination: The user must use their imagination for the virtual environment to be perceived as real.
Immersive vs. Non-Immersive Virtual Environments
An immersive virtual environment is a dynamic three-dimensional scene (animation) stored in the computer and displayed using computer graphics techniques (such as a Head-Mounted Display or HMD) in real-time. This makes the user believe they are immersed in the environment. A non-immersive virtual environment uses a screen to display the virtual environment.
VR System Components
A VR system consists of I/O devices, software, a database (supporting communication, I/O, and model import), tasks (applications), the user (considering human factors like discomfort, convenience, and perceived limitations), and the engine (handling geometric modeling, kinematics, physics, and intelligent agents).
Human Factors in VR
- Will the user feel uncomfortable?
- Will they realize the limitations of the system?
- Which tasks are more convenient for users in VR?
Augmented Reality (AR)
AR Display Technologies
- Monitor: Mixes real and virtual, but has limited interaction.
- Optical See-Through: The real world is observed in real-time, and only the virtual channel is rendered. Simpler but with more difficulty controlling delays.
- Video See-Through: Both the real and virtual worlds are processed electronically, allowing for delay alignment. It is always slightly delayed but more complex and manageable.
Interaction in AR
Interaction techniques are the methods by which interaction can occur. Physical devices are used to perform the interaction, and interaction tasks are the applications of these interactions.
3D Reconstruction
Camera Model
A 2D representation of 3D objects. It sets the pose of an object according to a frame of reference (position and orientation). Using homogeneous coordinates, the projection is represented by a linear transformation: P = K[R|t], where:
- K is the camera calibration matrix.
- R is the rotation matrix describing the camera’s orientation.
- t is the translation matrix describing the camera’s origin relative to a coordinate system.
Reconstruction Pipeline
This process recovers a 3D surface from a sequence of images acquired by the free movement of the camera around the object. It involves camera movement setup and texture obtained from the images.
Basic Elements of Reconstruction
- Generation of Keyframes: Epipolar geometry is measured between frames, requiring a minimum distance (baseline) and speed (about 15). The first set of poses and 3D points can only be performed after selecting the first two keyframes.
- Structure from Motion: This technique calculates the fundamental or essential matrix of a pair of images. It can use 5, 7, 8, or more points.
- Dense Reconstruction: Techniques based on photometric discrepancy, stereoscopy, and rectification.