CSC 471/583 Malware Analysis

2021-Spring Course Website

Instructor: Si Chen

Course Logo

About Maleware and Malware Analysis

Malware is a catch-all term for various malicious software, including viruses, adware, spyware, browser hijacking software, and fake security software.
Once installed on your computer, these programs can seriously affect your privacy and your computer's security. For example, malware is known for relaying personal information to advertisers and other third parties without user consent. Some programs are also known for containing worms and viruses that cause a great deal of damage. As a result, the ability to detect, analyze, understand, control, and eradicate malware is an increasingly important issue of economic and national security.

Course Description

This course will introduce students to modern malware analysis techniques through lectures and hands-on interactive analysis of real-world samples, including explore various recent attacks. These examples and studies will help the students develop a foundation and a well-rounded view of cybersecurity research. Participants in the course will also read and discuss research papers, as well as conduct independent project in a topic related to cyber risk and malware analysis.
After taking this course students will be equipped with the skills to analyze advanced contemporary malware using both static and dynamic analysis.

Expected Background

No prerequisite for graduate students, although sufficient security background is expected. For undergraduate students, please make sure you completed CSC 242.
My expected demographic for Malware Analysis was students with zero reverse engineering experience. That said, to be able to take this course you will probably need at least the following skills.
  • Basic programming concepts
  • Knowledge with the C programming language, including pointers, arrays, loops, function calls, etc.
  • Familiar with Unix/Linux including the command-line shell and gdb
  • Familiar with Intel x86 assembly language and architecture
  • Familiar with web programming concepts (HTML, HTTP, TCP, network communications)


No Textbook

Reference book:

  1. Monnappa K A, Learning Malware Analysis: Explore the concepts, tools, and techniques to analyze and investigate Windows malware , ISBN 978-1788392501
  2. Michael Sikorski, Andrew Honig, Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software, 1st Edition, ISBN 978-1593272906

Course Content

# Date Topic Slides Supporting Materials
Class 1 Jan 28, 2020 Introduction [Video] ch01.pdf
Class 2 Feb 03, 2021 Basic Concepts, DLL Injection (1) [Video] ch02.pdf
  • Windows XP Environment Disclaimer
  • [backup link]
  • [backup link]
  • Lab 1
    (10 points)
    Feb 03, 2021 Lab1: OllyDbg and DLL Injection
  • Windows XP Environment Disclaimer
  • [backup link]
  • Class 3 Feb 11, 2021 Static Analysis, PE Format (1)[Video] ch03.pdf
  • [backup link]
  • flare-floss
  • LittleEndian.exe
  • LittleEndian.cpp
  • Class 4 Feb 19, 2021 PE Format (2), Real-world Malware Analysis[Video] ch04.pdf
  • Download PEview
  • Lab 2
    (10 points)
    Feb 19, 2021 Lab2: Build a heuristic malware detection system
  • How to connect to Badger CTF (Windows)
  • How to connect to Badger CTF (MacOS, Linux)
  • password:infected
  • pefile usage examples (Python)
  • Class 5 Feb 26, 2021 IA32 Registers & X86 ASM[Video] ch05.pdf
  • abexcm1-voiees.exe
  • Class 6 Mar 6, 2021 Stack and Stack Frames[Video] ch06.pdf
  • Stack.exe
  • StackFrame.exe
  • StackFrame.cpp
  • HelloWorld.exe
  • Lab 3
    (10 points)
    Mar 07, 2021 Lab3: Stack, Stack Frame & CrackMe
  • lab3.exe[backup link]
  • A reversing tutorial for newbies by lena151 part10.avi
  • Class 7 March 22, 2021 Hooks [Video] ch07.pdf
  • password:infected
  • password:infected
  • Class 8 April 4, 2021 API Hook, Stealth process, Kernel Mode Rootkit [Video] ch08.pdf
  • password:infected
  • password:infected
  • Class 9 April 15, 2021 Rootkit [Video] ch09.pdf
  • stuxnet.vmem
  • stuxnet source code
  • volatility
  • Lab 4 + Reading Question 1
    (20 points + 5 points bonues)
    April 15, 2021 Lab4: Memory Forensics - Stuxnet
  • stuxnet.vmem
  • How to connect to Badger CTF (Windows)
  • How to connect to Badger CTF (MacOS, Linux)
  • Class 10 April 21, 2021 Anti-virus Software, Dynamic Heuristic Analysis [Video] ch10.pdf
  • (Experiment 1) password:infected
  • (Experiment 2)
  • Lab 5
    (10 Points)
    April 21, 2021 Lab 5: Build a Dynamic Heuristic Analysis Tool for Detection of Unknown Malware
  • password:infected
  • Windows XP Environment Disclaimer
  • Final Project
    (40 Points)
    April 21, 2021 Malware Analysis: Zeus
    Analysis zeus malware with Volatility and write a malware analysis report
  • zeus.vmem
  • zeus source code -- client side
  • How to connect to Badger CTF (Windows)
  • How to connect to Badger CTF (MacOS, Linux)
  • Final project Guidline:

    Our final project is to analyze a notorious Trojan malware -- Zeus). You need to use the provided memory dump file (zeus.vmem) and use Volatility to analyze it. This part is similar to lab4. Then you need to use the leaked source code to justify it.
    This is a group project and each group can have at most 5 students. You can find your teammate via our Discord channel. You can also work individually if you want. If you decided to work as a team then please send me an email before 05/01, 2021, and let me know your group member's name and WCU ID. If your team needs a private discord channel (text or voice), please let me know, I can create one for your team.
    You cannot claim that you are in a team after 05/02, 2021. You can still work on this project individually.
    Feel free to discuss with other students in our "#finalproject" channel, but do not share your report. You can use/share any online materials (e.g., tech blog, GitHub, ...).

    About the Final Report:

    • Each team needs to submit one detailed malware analysis report (in PDF format) on D2L. The team leader needs to submit the report using his/her student account to D2L, in the first page of your group report, please write down how your team divides the tasks for each member.
    • In your report, please use both text and figure (e.g., screenshot) to show what you've done. Do NOT copy or reuse other groups/student's screenshots.
    • Malware analysis report is a document that provides an in-depth breakdown of the functionality and risk of a new or evolving cyber threat. Your malware analysis report should include the following information:
      • Introduction -- give a brief introduction of the malware
      • Findings -- describes what the malware is designed to do by using the following two methods:
        • Use Volatility to analyze the malware memory dump and report how Volatility could be used to find relevant artifacts of the activity within memory (e.g., find all API Hooks of Zeus).
        • Use the leaked malware source code to justify such activity (e.g., identify which function/line of code that does the API Hooks). (Note that this part is very important, your group needs to pinpoint the functions/line of codes for each malware activity that you find in the memory dump).
      • Summary -- Summary of your work and give some best practices for preventing infections and recovering from them.

    Report Grading Rubric:

    • Technical Content (50%)
      • Find an adequate amount of malware activities/behaviors from the memory dump and justify it using the leaked source code
      • Appropriate level of detail and thoroughness of documentation
    • Organization (15%)
      • Clearly identified purpose and approach
      • Content is clearly organized and supports the objective
    • Presentation (15%)
      • Easy to read
    • Layout/Visuals (20%)
      • Consistent presentation of graphics

    Tutorials and Supporting Materials