In this research two automatic video annotation techniques are considered. The first technique uses ontology to reduce the semantic gap during video retrieval and other performs a group based image retrieval using video files. The proposed algorithm uses GIR algorithm to create similar image group. From this refined set of images, SIFT features are extracted and the steps used by ASVA algorithm is performed to annotate the video in a semantic fashion. The Automatic Semantic based Video Annotation algorithm performs annotation in three steps. The first step calculates the video similarity using SIFT features, sentence and synonym analysis is performed in the second to find similar meaning annotations and finally the conjunction of the sentences are analyzed to increase the certainty of each annotation using Concept Net.