Abstract: The DCT 8x8 transform mode has been adopted for high profile encoding in H264/AVC in addition to 4x4 integer transform. We propose an efficient DCT 8x8 transform mode decision method to reduce the computational complexity of the transform mode decision. We observed that the average variance of the Direct Current (DC) coefficients is quite different according to the best DCT kernel size. We use the amount of variation of the DC coefficients after DCT 4x4 procedure as a classification feature for the proposed algorithm and make a adaptive threshold using the average of variance of DC coefficients. We verify that the proposed algorithm can reduce the 8x8 transform mode decision by amount of 47.35-65.53%.