All-in-One Framework for Detection, Unpacking, and Verification for Malware Analysis

Algorithm 2

Dynamic unpacking algorithm.

	Input: A PE file
	Output: OEP //the start address of the unpacked original file.
	begin
(1)	Measure the entropy value of each section;
(2)	while the process is not reached to the end do
(3)	IP the current instruction pointer;
(4)	if IP’s address  current section’s last address then return Not-Found;
(5)	else if IP is JMP or CJMP or RETN then
(6)	Execute instructions until IP;
(7)	DstAddr the next instruction address;
(8)	if IP  IP-Histroy and DstAddr  DstAddr-History then
(9)	Store IP and DstAddr into each history;
(10)	if DstAddr is out of the file then return Not-Found;
(11)	else
(12)	Re-calculate the entropy value of each section;
(13)	if Entropy change is stable and DstAddr is in the different section then
(14)	return DstAddr;
(15)	end-if
(16)	end-if
(17)	end-if
(18)	end-while
(19)	return Not-Found;
	end