Abstract: Commonly, survey instrument reliability is achieved through Cronbach’s alpha and Exploratory Factor Analysis (EFA). However, the existence of different values rule of thumb often creates confusion for novice researchers. Thus, the study aims to investigate the crucial role of Exploratory Data Analysis (EDA) in assisting the reliability testing of a survey instrument via. box plot to clearly understand the instrument’s items. The instrument was adapted and adopted from well-established instrument of the National Survey of Student Engagement (NSSE) which consists of 4 constructs, 13 dimensions and total of 59 items. The study focuses on student-faculty interaction dimension which comprises 6 items. The finding shows that item 1a is causing the internal inconsistency which jeopardizing the reliability of the student-faculty interaction dimension. Hence, by using EDA in depth understanding is obtained and cross checked with Cronbach’s alpha and EFA values in removing the item from the instrument. Thus, emphasizing the crucial role of EDA in assessing survey instrument reliability where in depth understanding regarding the item offers alternative solution to overcome the problem of different rule of thumb of Cronbach’s alpha and EFA values.