CSC 495/583 Topics of Software Security

2018-Fall Course Website

Advisor: Si Chen

Course Logo

Course Overview


  • The legal aspects of reverse engineering.
  • Assembly language for IA-32 compatible processors and how to read compiler-generated assembly language code.
  • The general principles behind malicious software and how reverse engineering is applied to study such program.

Expected Background

  • Basic programming concepts (e.g. complete Java I, II)
  • Knowledge with the C programming language, including pointers, arrays, loops, function calls, etc.
  • Familiar with Unix/Linux including the command-line shell and gdb
  • Familiar with Intel x86 assembly language and architecture
  • Familiar with web programming concepts (HTML, HTTP, TCP, network communications)


No Textbook

Reference book:

  1. Randal E. Bryant, Davie Richard O'Hallaron, Computer Systems: A Programmer's Perspective, 3rd Edition, ISBN 978-0134092669
  2. Kris Kaspersky, Hacker Disassembling Uncovered, 2nd Edition, ISBN 978-1931769648
  3. Eldad Eilam, Reversing: Secrets of Reverse Engineering, 1st Edition, ISBN 978-0764574818

Course Content

# Date Topic Slides Supporting Materials
C1 Aug 28, 2018 Introduction ch01.pptx
C2 Aug 30, 2018 IA-32 Register, Byte Ordering, x86 ASM ch02.pptx abexcm1-voiees.exe LittleEndian.exe LittleEndian.cpp HelloWorld.exe
C3 Sep 4, 2018 x86 ASM, Stack, Stack Frame ch03.pptx Stack.exe StackFrame.exe StackFrame.cpp
L0 (Non-graded) Sep 4, 2018 Lab: Hello World
Change the text in pop-up window from "Hello World!" to "Hello Reversing", take a screenshot and upload the image to D2L.
Bonus Instead of changing it to "Hello Reversing," can you hack and change the text to "Hello Reversing World!!!" You'll get 1 bonus point.
HelloWorld.exe Windows XP Environment Disclaimer
C4 Sep 6, 2018 Stack Frame, Calling Convention ch04.pptx StackFrame.exe StackFrame.cpp
C5 Sep 11, 2018 Calling Convention, System Call, Introduction to PEDA and Pwntools ch05.pptx cdecl.c stdcall.c cdecl.exe stdcall.exe helloworld.asm shell.asm 1_sample.c 2_interactive.c 3_reversing.c
C6 Sep 13, 2018 Stack Overflow (1) ch06.pptx buffer.c buffer2.c overflow.c
C7 Sep 18, 2018 Stack Overflow (2) ch07.pptx hello.asm test.c shellcode.asm Shellcode overflow2.c
L1 Sep 18, 2018 Lab: Buffer Overflow
lab1.pdf lab1.c VM image for Lab1
Username: quake0day Password: chensi
C8 Sep 25, 2018 Stack Overflow Review: Classic Exploitation Technique (with PEDA, Pwntools) & Linux Binary Protections (ASLR, DEP, Stack Canaries) ch08.pptx hello.asm test.c shellcode.asm Shellcode overflow2.c overflow3.c
C9 Sep 27, 2018 Return-oriented programming (ROP) ch09.pptx rop.c
C10 Oct 2, 2018 Return-oriented programming (ROP) (2) ch10.pptx rop.c rop2.c
C11 Oct 4, 2018 Return-oriented programming (ROP) (3) & Dynamic Linking & Return-to-libc Attack & ASLR ch11.pptx reveal_address.c ret2lib.c niklasb/libc-database
L2 Oct 9, 2018 Lab: Return-oriented programming (ROP)
  • lab2.c
  • Manjaro (Arch Linux) 64 Environment
    Username: csc495 Password: csc495
  • C12 Oct 9, 2018 Web Security (1) ch12.pptx PHP Exercise (1) PHP Exercise (2) PHP Exercise (3) PHP Exercise (4) PHP Exercise (5)
    R1 Oct 9, 2018 Reading Question 1: BlueBorne
    homework1.pdf blueborne technical paper Video (smartwatch takeover)
    C13 Oct 11, 2018 Web Security (2) ch13.pptx SQL Injection Exercises
    C14 Oct 16, 2018 Web Security (3) & PLT, GOT & Return-to-plt Attack (Bypassing ASLR/NX) ch14.pptx ret2plt.c
    C15 Oct 18, 2018 GOT Overwrite Attack (1) ch15.pptx bypassGOT.c
    C16 Oct 23, 2018 GOT Overwrite Attack (2) ch16.pptx event1.c
    L3 Oct 23, 2018 Lab: Multi-Stage Exploits
    Target IP:
    Target Port: 8888
    Vulnerable program: lab3 (lab3.c)
    Target File (flag): flag.txt
    ASLR/NX is on, StackGuard and PIE is off
    Hint Libc version:libc6-i386_2.27-3ubuntu1_amd64 [Link]
  • lab3.c
  • Ubuntu 18.04 LTS
    Username: csc495 Password: csc495
  • C17 Oct 25, 2018 Multi-Stage Exploits ch17.pptx multi_stage.c
    C18 Nov 1, 2018 Stack Guard & Format String Bug ch18.pptx easy_canary_32.c easy_canary_64.c fmt_write.c fmt_wrong.c fmtstr.c
    R2 Nov 6, 2018 Reading Question 2: Hacking Blind
    ReadingQuestion2.pdf Hacking Blind paper Project Website
    C19 Nov 6, 2018 Format String Bug (2) & Heap ch19.pptx fmt_write2.c fmt_test.c fmt_test2.c heap1.c
    C20 Nov 8, 2018 Heap Exploitation (1) ch20.pptx Kali-Linux-2017.2-vbox-i386.ova heap0.c heap1.c
    C21 Nov 13, 2018 Heap Exploitation (2): Unlink ch21.pptx unlink unlink.c
    L4 Nov 13, 2018 Lab: Heap Exploitation: Unlink
    lab4.pdf unlink unlink.c
  • Manjaro (Arch Linux) 64 Environment
    Username: csc495 Password: csc495
  • C22 Nov 29, 2018 Heap Exploitation (3): House of Force & The future of Software Security ch22.pptx house_of_force.c bamboobox bamboobox.c
    R3 Nov 29, 2018 Reading Question 3: Blockchain
    homework3.pdf Blockchain papers
    Final Project Nov 28, 2017 Final Project
    IP: Port:9999
    Exploit this server, and show me the secret stored in file flag.txt
    ASLR on
    Canary found
    NX enabled
    No PIE
    Hint Gadget Infomation:[Link]
    1. Use format string bug to leak Canary value [Walkthrough]
    2. Use write@plt to leak information about libc and then find the memory address for get@plt
    3. Use ROP to launch system call execve() to open a shell. You can use get@plt to take user input and write the user input “/bin/sh” to an empty memory address.

    Tutorials and Supporting Materials