CSC 497/583 Advanced Topics in Computer Security

Modern Malware Analysis

2019-Spring Course Website

Instructor: Si Chen

Course Logo

About Maleware and Malware Analysis

Malware is a catch-all term for various malicious software, including viruses, adware, spyware, browser hijacking software, and fake security software.
Once installed on your computer, these programs can seriously affect your privacy and your computer's security. For example, malware is known for relaying personal information to advertisers and other third parties without user consent. Some programs are also known for containing worms and viruses that cause a great deal of damage. As a result, the ability to detect, analyze, understand, control, and eradicate malware is an increasingly important issue of economic and national security.

Course Description

This course will introduce students to modern malware analysis techniques through lectures and hands-on interactive analysis of real-world samples, including explore various recent attacks. These examples and studies will help the students develop a foundation and a well-rounded view of cybersecurity research. Participants in the course will also read and discuss research papers, as well as conduct independent project in a topic related to cyber risk and malware analysis.
After taking this course students will be equipped with the skills to analyze advanced contemporary malware using both static and dynamic analysis.

Expected Background

No prerequisite for graduate students, although sufficient security background is expected. For undergraduate students, please make sure you completed CSC 242.
My expected demographic for Malware Analysis was students with zero reverse engineering experience. That said, to be able to take this course you will probably need at least the following skills.
  • Basic programming concepts
  • Knowledge with the C programming language, including pointers, arrays, loops, function calls, etc.
  • Familiar with Unix/Linux including the command-line shell and gdb
  • Familiar with Intel x86 assembly language and architecture
  • Familiar with web programming concepts (HTML, HTTP, TCP, network communications)

Textbook

No Textbook

Reference book:

  1. Monnappa K A, Learning Malware Analysis: Explore the concepts, tools, and techniques to analyze and investigate Windows malware , ISBN 978-1788392501
  2. Michael Sikorski, Andrew Honig, Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software, 1st Edition, ISBN 978-1593272906

Course Content

# Date Topic Slides Supporting Materials
Class 1 Jan 22, 2019 Introduction ch01.pptx
Class 2 Jan 24, 2019 Basic Concepts, DLL Injection (1) ch02.pptx
  • Windows XP Environment Disclaimer
  • hack_dll.zip
  • hack_dll_src.zip
  • Class 3 Jan 29, 2019 DLL Injection (2), Static Analysis ch03.pptx
  • Manjaro Linux (ArchLinux) EnvironmentUsername:csc495 password:csc495
  • hack_dll.zip
  • hack_dll_src.zip
  • fingerprinting.py
  • vt_query.py
  • flare-floss
  • myhack_packed.dll
  • Lab 0
    (Non-graded)
    Jan 29, 2019 Lab0: "Hello World" -- DLL Injection
    lab0.pdf
  • Windows XP Environment Disclaimer
  • hack_dll.zip
  • hack_dll_src.zip
  • Class 4 Jan 31, 2019 Static Analysis, PE Format ch04.pptx
  • hack_dll.zip
  • flare-floss
  • notepad_xp.zip
  • Download HxD
  • display_sections.py
  • Download PEview
  • myhack_packed.dll
  • Class 5 Feb 5, 2019 PE Format (2): NT Header, IAT, EAT ch05.pptx
  • notepad_xp.zip
  • Download HxD
  • display_sections.py
  • enum_imports.py
  • enum_exports.py
  • Download PEview
  • myhack_packed.dll
  • Class 6 Feb 7, 2019 Static Analysis: Real-world Case Study ch06.pptx
  • display_sections.py
  • enum_imports.py
  • enum_exports.py
  • malware_sample_0.zip
  • Lab 1
    (8 points)
    Feb 7, 2019 Lab1: Build a heuristic malware detection system
    lab1.pdf
  • Manjaro Linux (ArchLinux) Environment Username:csc495 password:csc495
  • malware_lab_1.zip password:infected
  • enum_exports.py
  • pefile usage examples (Python)
  • Class 7 Feb 19, 2019 Assembly Language and Disassembly Primer ch07.pptx
    Class 8 Feb 21, 2019 X86 ASM, Dynamic Analysis (1), Stack ch08.pptx abexcm1-voiees.exe Stack.exe stack.py
    Class 9 Feb 26, 2019 Stack Frame ch09.pptx Stack.exe stack.py StackFrame.exe StackFrame.cpp HelloWorld.exe
    Class 10 Feb 28, 2019 Stack Frame (Review), Calling Convention, Dynamic Analysis(2) ch10.pptx StackFrame.exe StackFrame.cpp cdecl.exe stdcall.exe HelloWorld.exe hack_dll.zip
    Class 11 Mar 5, 2019 Dynamic Analysis(2) ch11.pptx hack_dll.zip
    Lab 2
    (8 points)
    March 7, 2019 Lab2: Stack and Stack Frame in Linux
    lab2.pdf
  • Manjaro Linux (ArchLinux) Environment Username:csc495 password:csc495
  • lab2.c
  • Class 12 Mar 19, 2019 Hooks ch12.pptx
  • Hook.zip password:infected
  • IATHookMsgBox_x86.zip password:infected
  • Class 13 Mar 19, 2019 Anti-virus Software, Dynamic Heuristic Analysis ch13.pptx
  • VirusShare_00001.md5
  • ransomware.zip password:infected
  • monitor.py
  • Lab 3
    (8 points)
    April 2, 2019 Lab3: Build a Dynamic Heuristic Analysis Tool for Detection of Unknown Malware
    lab3.pdf
  • ransomware.zip password:infected
  • Windows XP Environment Disclaimer
  • R1
    (6 points)
    April 4, 2019 Reading Question 1
    ReadingQuestion1.pdf
    Class 14 Apr 9, 2019 API Hook, Stealth process (Rootkit) ch14.pptx
  • StealthProcess1.zip password:infected
  • StealthProcess2.zip password:infected
  • Class 15 Apr 16, 2019 Kernel Mode Rootkit ch15.pptx
  • stuxnet.vmem
  • stuxnet source code
  • laqma.vmem
  • volatility
  • R2
    (6 points)
    April 16, 2019 Reading Question 2
    ReadingQuestion2.pdf
    Lab 4
    (16 + 6 points)
    April 23, 2019 Lab4: Stuxnet
    lab4.pdf
    R3
    (12 points)
    April 25, 2019 Reading Question 3
    ReadingQuestion3.pdf
    R4
    (6 points)
    May 6, 2019 Reading Question 4
    ReadingQuestion4.pdf

    Schedule for Presentations

    DOODLE LINK

    Malware Analysis Tools

  • volatility
  • VirtualBox
  • Get Manjaro (Arch Linux) ISO
  • Manjaro Linux (ArchLinux) Environment Username:csc495 password:csc495
  • Windows XP Environment Disclaimer
  • ProcessExplorer
  • Dependence Walker
  • PEiD
  • DebugView
  • WireShark
  • OllyDbg for Windows
  • Online Tutorials

  • Volatility Command Reference new!
  • A Crash Course in x86 Assembly for Reverse Engineers
  • Manjaro User Guide
  • RMS's gdb Tutorial
  • PE Format
  • Overview of IA-32 assembly programming
  • x86 Assembly Language Reference Manual