  The most common malware detection approaches which are based on signature
matching and are not sufficient for metamorphic malware detection, since virus
kits and metamorphic engines can produce variants with no resemblance to one
another. Metamorphism provides an efficient way for eluding malware detection
software kits. Code obfuscation methods like dead-code insertion are also
widely used in metamorphic malware. In order to address the problem of
detecting mutated generations, we propose a method based on Opcode Graph
Similarity (OGS). OGS tries to detect metamorphic malware using the similarity
of opcode graphs. In this method, all nodes and edges have a respective effect
on classification, but in the proposed method, edges of graphs are pruned using
Linear Discriminant Analysis (LDA). LDA is based on the concept of searching
for a linear combination of predictors that best separates two or more classes.
Most distinctive edges are identified with LDA and the rest of edges are
removed. The metamorphic malware families considered here are NGVCK and
metamorphic worms that we denote these worms as MWOR. The results show that our
approach is capable of classifying metamorphosed instances with no or minimum
false alarms. Also, our proposed method can detect NGVCK and MWOR with high
accuracy rate.
