[MRTK 기능 공부] Eye-Tracking

Eye-Tracking 은 문자 그대로 눈으로 무언가 상호작용을 하고 기능을 하는 것을 의미함.

 

1. Eye Tracking - Target Selection 

 

내가 처다보게 되면 기능이 작동하고,

클릭하면 파괴되는 데모임

 

데모 자체 스크립터, TargetGroupCreateRadial 에서  생성되는 구슬의 템플렛을 넣고

동그런 모양으로 만들어주는 기능을 하는 것 같음

 

---

 

 

구슬 마다 EyeTrackingTarget 스크립터를 가지고있고

Eye를 처다보면 작동하는 스크립터,

지금은 처다보면 로테이션이 되는 스크립터를 작동하고있음

 

 

구슬 안에  Hit하면 사라지게 하는 스크립터 (데모에 있는 스크립터)가 있음

 

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using Microsoft.MixedReality.Toolkit.Input;
using UnityEngine;

namespace Microsoft.MixedReality.Toolkit.Examples.Demos.EyeTracking
{
    /// <summary>
    /// Destroys the game object when selected and optionally plays a sound or animation when destroyed.
    /// </summary>
    [RequireComponent(typeof(EyeTrackingTarget))]
    [AddComponentMenu("Scripts/MRTK/Examples/HitBehaviorDestroyOnSelect")]
    public class HitBehaviorDestroyOnSelect : MonoBehaviour
    {
        [Tooltip("Visual effect (e.g., particle explosion or animation) that is played when a target is selected.")]
        [SerializeField]
        private GameObject visualFxTemplate_OnHit = null;

        [Tooltip("Audio clip that is played when a target is selected.")]
        [SerializeField]
        private AudioClip audioFx_CorrectTarget = null;

        [Tooltip("Audio clip that is played when a wrong target is selected.")]
        [SerializeField]
        private AudioClip audioFx_IncorrectTarget = null;

        [Tooltip("Manually indicate whether this is an incorrect target.")]
        [SerializeField]
        private bool is_a_valid_target = true;

        [Tooltip("Associated TargetGridIterator to check whether the currently selected target is the correct one.")]
        [SerializeField]
        private TargetGroupIterator targetIterator = null;


        private EyeTrackingTarget myEyeTrackingTarget = null;

        private void Start()
        {
            myEyeTrackingTarget = this.GetComponent<EyeTrackingTarget>();
            if (myEyeTrackingTarget != null) // Shouldn't be null since we use RequireComponent(), but just to be sure.
            {
                myEyeTrackingTarget.OnSelected.AddListener(TargetSelected);
            }
        }

        /// <summary>
        /// Internal audio source associated with the game object.
        /// </summary>
        private AudioSource audioSource;

        private void SetUpAudio()
        {
            audioSource = gameObject.EnsureComponent<AudioSource>();

            audioSource.playOnAwake = false;
            audioSource.enabled = true;
        }

        /// <summary>
        /// Play given audio clip.
        /// </summary>
        private float PlayAudioOnHit(AudioClip audioClip)
        {
            // Set up audio source if necessary
            SetUpAudio();

            // Play the given audio clip
            float audiocliplength = 0;
            if ((audioSource != null) && (audioClip != null))
            {
                audioSource.clip = audioClip;
                audioSource.PlayOneShot(audioSource.clip);
                audiocliplength = audioSource.clip.length;
            }
            return audiocliplength;
        }

        /// <summary>
        /// Show given GameObject when target is selected. 
        /// </summary>
        private void PlayAnimationOnHit()
        {
            if (visualFxTemplate_OnHit != null)
            {
                GameObject visfx = Instantiate(visualFxTemplate_OnHit, transform.position, transform.rotation);
                visfx.SetActive(true);
                Destroy(visfx, 2);
            }
        }

        public void TargetSelected()
        {
            if (!is_a_valid_target)
            {
                PlayAudioOnHit(audioFx_IncorrectTarget);
                return;
            }

            if (!HandleTargetGridIterator())
            {
                return;
            }

            // Play audio clip
            float audiocliplength = PlayAudioOnHit(audioFx_CorrectTarget);

            // Play animation
            PlayAnimationOnHit();

            // Destroy target
            gameObject.SetActive(true);
            gameObject.GetComponent<MeshRenderer>().enabled = false;
            Destroy(gameObject, audiocliplength);
        }

        /// <summary>
        /// Check whether the selected target is the intended one based on the referenced 'targetIterator' object.
        /// </summary>
        private bool HandleTargetGridIterator()
        {
            if (targetIterator != null)
            {
                if ((targetIterator.PreviousTarget != null) && (targetIterator.PreviousTarget.name == name))
                {
                    return true;
                }
            }
            else
            {
                return true;
            }

            PlayAudioOnHit(audioFx_IncorrectTarget);
            return false;
        }
    }
}

 

오디로와 사라질때 나오는 fx 등등 나오게하는 기능을 구현하고 있고

 

핵심은 눈에 닿았을 때 그 오브젝트를 Target 으로 잡는 것이다 .

 

 

 

EyeTrackingTarget 이라는 기능을 써서 눈에 닿고 있는 오브젝트를 타겟으로 넣고

 

클릭을 하면  OnSelected 리스터 이벤트를 실행하여서 TargetSelected 메서드를 던져줘서 오브젝트를 없애준다.

 

 

핵심은 EyeTrackingTarget.cs 와 삭제되는 스크립터에서 EyeTrackingTarget 이벤트들 인듯

 

--------

 

Eye-Gaze-supported Scroll, Pan, Zoom

 

Eye-Gaze 는 내가 바라보고있는 곳에 특정 기능을 넣는 것 인듯 

 

 

내가 처다보고있으면 로테이션을하고, 조금 더 오른쪽 또는 왼쪽을 보면 가속이 붙는다.

처다보고 클릭하면 커지는 기능을 함 

 

1. 로테이션

 

 

최상위 오브젝트의 

EyeTrackingTarget 스크립터를 넣어주고,

 

OnLookAtRotateEyeGaze.cs 를 달아준다. (샘플 씬에 있는 스크립터)

 

이 스크립터는 BaseEyeFocusHandler 를 상속하고 있음

BaseEyeFocusHandler의 OnEyeFocusStay 를 작동하면 (처다보면 이벤트가 작동되는 것 인듯)

 

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using Microsoft.MixedReality.Toolkit.Input;
using Microsoft.MixedReality.Toolkit.Utilities;
using UnityEngine;

namespace Microsoft.MixedReality.Toolkit.Examples.Demos.EyeTracking
{
    /// <summary>
    /// The associated game object will turn depending on which part of the object is looked at: 
    /// The currently looked at part will move towards the front facing the user.
    /// </summary>
    [RequireComponent(typeof(EyeTrackingTarget))]
    [AddComponentMenu("Scripts/MRTK/Examples/OnLookAtRotateByEyeGaze")]
    public class OnLookAtRotateByEyeGaze : BaseEyeFocusHandler
    {
        #region Serialized variables
        [Tooltip("Horizontal rotation speed.")]
        [SerializeField]
        private float speedX = 0.0f;

        [Tooltip("Vertical rotation speed.")]
        [SerializeField]
        private float speedY = 4.0f;

        [Tooltip("To inverse the horizontal rotation direction.")]
        [SerializeField]
        private bool inverseX = false;

        [Tooltip("To inverse the vertical rotation direction.")]
        [SerializeField]
        private bool inverseY = false;

        [Tooltip("If the angle between 'Gaze to Target' and 'Camera to Target' is less than this value, do nothing. This is to prevent small jittery rotations.")]
        [SerializeField]
        private float rotationThreshInDegrees = 5.0f;

        [Tooltip("Minimum horizontal rotation angle. This is to limit the rotation in different directions.")]
        [SerializeField]
        private float minRotX = -10.0f;

        [Tooltip("Maximum horizontal rotation angle. This is to limit the rotation in different directions.")]
        [SerializeField]
        private float maxRotX = 10.0f;

        [Tooltip("Minimal vertical rotation angle. This is to limit the rotation in different directions.")]
        [SerializeField]
        private float minRotY = -180.0f;

        [Tooltip("Maximum vertical rotation angle. This is to limit the rotation in different directions.")]
        [SerializeField]
        private float maxRotY = 180.0f;
        #endregion

        protected override void OnEyeFocusStay()
        {
            // Update target rotation
            RotateHitTarget();
        }

        private void RotateHitTarget()
        {
            Vector3 TargetToHit = (this.gameObject.transform.position - CoreServices.InputSystem.EyeGazeProvider.HitPosition).normalized;
            Vector3 TargetToCam = (this.gameObject.transform.position - CameraCache.Main.transform.position).normalized;

            float angle1x, angle1y, angle1z, angle2x, angle2y;

            angle1x = Mathf.Atan2(TargetToHit.y, TargetToHit.z);
            angle1y = Mathf.Atan2(TargetToHit.x * Mathf.Cos(angle1x), TargetToHit.z);
            angle1z = Mathf.Atan2(Mathf.Cos(angle1x), Mathf.Sin(angle1x) * Mathf.Sin(angle1y));

            angle2x = Mathf.Atan2(TargetToCam.y, TargetToCam.z);
            angle2y = Mathf.Atan2(TargetToCam.x * Mathf.Cos(angle2x), TargetToCam.z);

            if ((angle1y > 0) && (angle1z < 0))
            {
                angle1y -= Mathf.PI;
            }
            else if ((angle1y < 0) && (angle1z < 0))
            {
                angle1y += Mathf.PI;
            }

            float rotx, roty;
            rotx = angle1x - angle2x;
            roty = angle1y - angle2y;
            float newRotX = transform.eulerAngles.x, newRotY = transform.eulerAngles.y;

            // Restrict the rotation to a given angle range for x.
            if (Mathf.Abs(rotx) > (Mathf.Deg2Rad * rotationThreshInDegrees))
            {
                float stepx = speedX * ((inverseX) ? -1 : 1) * rotx;
                newRotX = ClampAngleInDegree(transform.eulerAngles.x + stepx, minRotX, maxRotX);
            }

            // Restrict the rotation to a given angle range for y.
            if (Mathf.Abs(roty) > (Mathf.Deg2Rad * rotationThreshInDegrees))
            {
                float stepy = speedY * ((inverseY) ? -1 : 1) * roty;
                newRotY = ClampAngleInDegree(transform.eulerAngles.y + stepy, minRotY, maxRotY);
            }

            // Assign the computed Euler angles.
            transform.eulerAngles = new Vector3(newRotX, newRotY, transform.eulerAngles.z);
        }

        /// <summary>
        /// Clamps angle within the range of a given min and max value and maps it to the range of -180 to +180.
        /// </summary>
        private float ClampAngleInDegree(float angleInDegree, float minAngleInDegree, float maxAngleInDegree)
        {
            // Angle is not constricted
            if ((minAngleInDegree == -180f) && (maxAngleInDegree == 180f))
            {
                return angleInDegree;
            }

            // Wrap around angle to stay within [-180, 180] degrees
            if (angleInDegree > 180)
            {
                angleInDegree -= 360;
            }

            if (angleInDegree < -180)
            {
                angleInDegree += 360;
            }

            // Final checks on min and max range
            if (angleInDegree > maxAngleInDegree)
            {
                return maxAngleInDegree;
            }

            if (angleInDegree < minAngleInDegree)
            {
                return minAngleInDegree;
            }

            return angleInDegree;
        }
    }
}

 

 

핵심은 BaseEyeFocusHandler와   로테이션 기능을 할 때 사용하는 EyeGazeProvider 인듯

 

 

 

https://docs.microsoft.com/ko-kr/windows/mixed-reality/mrtk-unity/features/input/eye-tracking/eye-tracking-eye-gaze-provider?view=mrtkunity-2021-05 

시선 추적 시선 공급자 - Mixed Reality Toolkit

 

참고

 

2. 클릭하면 오브젝트 커지기

 

최상위 오브젝트의

 

EyeTrackingTarget.cs 가 있고

 

클릭하면 발생하는 On Selected 이벤트가 있음.

클릭하면

TargetMoveToCamera.OnSeclect 메서드가 실행됌

 

OnSelect 는 만약  커져있다면 다시 작게 만들고

아니면 커지게하는 메서드를 호출함

 

--------

 

Eye-supported Target Selection & Positioning

 

 

 

내가 처다보고있는 것을 그랩(집으면) 움직일 수 있께하고, 위치를 바꿀 수있다. 

 

 

잡을 때 쓰는 핸들러,