Stiffness of Hot Mix Asphalt (HMA) in flexible pavement is largely dependent of temperature, mode of testing and age of pavement. Accurate measurement of HMA stiffness is thus quite challenging. This study determines HMA stiffness based on Finite Element Model (FEM) and validates the results using field data. As a first step, stiffnesses of different layers of a pavement section on Interstate 40 (I-40) in New Mexico were determined by Falling Weight Deflectometer (FWD) test. Pavement temperature was not measured at that time due to lack of temperature probe. Secondly, a FE model is developed in ABAQUS. Stiffness of the base, subbase and subgrade were taken from the FWD test output obtained from the first step. As HMA stiffness largely varies with temperature it was assigned trial and error approach. Thirdly, horizontal strain and vertical stress at the bottom of the HMA and temperature at different depths of the pavement were measured with installed sensors on the whole day on December 25th, 2012. Fourthly, outputs of FEM were correlated with measured stress-strain responses. After a number of trials a relationship was developed between the trial stiffness of HMA and measured mid-depth HMA temperature. At last, the obtained relationship between stiffness and temperature is verified by further FWD test when pavement temperature was recorded. A promising agreement between them is observed. Therefore, conclusion can be drawn that linear elastic FEM can accurately predict the stiffness and the structural response of flexible pavement.