Announcement.

  • 關於開學上課方式 Q & A
    • 上課方式：
      前三周的資訊理論與編碼技巧會以遠端授課方式進行，未來視情況以現場授課的方式進行，
      但不論線上或現場上課內容都會錄製成影片，可以自行觀看，所以整學期都不一定要到現場。
    • 作業繳交方式：
      會在課程網頁和NTU COOL公布，之後會有完整的遠端繳交方式說明
    • 加簽資格：
      為2類加選，有加簽或旁聽需求請寄信給助教
    • 先備知識：
      修習這門課的話，建議可以（不是必要）先預讀過「機率與統計」、「數位訊號處理」及「通訊原理」等課程，會對老師的授課內容更加有了解!
    • 遠端連線方式：
      遠端連線觀看目前會使用 Google Meet 進行連線，之後會視疫情情況調整
    • Google Meet連結：https://meet.google.com/eog-zevh-yso

    • Course Information

  • Lecturer: Prof. 吳家麟 (wjl@csie.ntu.edu.tw) R214
  • TA: 李旻倫 R505
  • TA Hour : By appointment via email.
  • itct@cmlab.csie.ntu.edu.tw

    • Assignments

  • HW1
    •           Deadline 11/8, 23:59
              請交pdf檔到NTU COOL作業區

  • HW2
    •           Deadline 12/20, 23:59
              請交pdf檔到NTU COOL作業區


  • A Brief Survey of your interested IT-related Research topic
    •           1. A4, less 6 pages
              2. Write in Chinese or English, clear and easily readable, No restriction on typesetting.
                  Please upload in pdf file.
              3. Please fill a topic in https://forms.gle/XWuup6M5MZP6Mwim9
                  before 11/19, 提醒大家星期五前先填上要做的topic供老師檢視， 如果有問題也可以寄信跟老師討論
              4. Deadline 23:59, 11/29


    Outline & videos

    Lecture 1

    1.1 Information Theory and Coding Techniques.mp4
    1.1 Information Theory and Coding Techniques.pdf
    1.2 Information Theory and Coding Techniques.mp4
    1.2 Information Theory and Coding Techniques.pdf

    Lecture 2
    2. Entropy,Relative Entropy and MI.mp4
    2. Entropy,Relative Entropy and MI.pdf

    Lecture 3
    3. Asymptotic equipartition property and Entropy rate.mp4
    3. Asymptotic equipartition property and Entropy rate.pdf

    Lecture 4
    4.1 Channel and Channel Capacity.mp4
    4.1 Channel and Channel Capacity.pdf
    4.2 Channel Capacity and BSC.mp4
    4.2 Channel Capacity and BSC.pdf
    4.3 Markov Process and Source with Memory.mp4
    4.3 Markov Process and Source with Memory.pdf

    Lecture 5
    5.1 Mutual Information and Channel Capacity.mp4
    5.2 Introduction to Data Compression.mp4
    5.2 Introduction to Data Compression.pdf

    Lecture 6
    6.1 Huffman Codes.mp4
    6.1 Huffman Codes.pdf
    6.2 Research Topics on Huffman Codes.mp4
    6.2 Research Topics on Huffman Codes.pdf

    Lecture 7
    7.1 Arithmetic Coding.mp4
    7.1 Arithmetic Coding.pdf
    7.2 Implementation of Arithmetic Coding.mp4
    7.2 Implementation of Arithmetic Coding.pdf
    7.3 Secure Arithmetic Coding.mp4
    7.3 Secure Arithmetic Coding.pdf

    Lecture 8
    8.1 Lempel-Ziv Coding
    8.2 Dictionary Codes and LZ Coding
    8.3 Adaptive Dictionary Compression Algorithm
    new : How to Speed up Machine Learning Using LZW Coding? Suggested References for Advanced Study in LZ and Tunstall Codes

    Lecture 9
    9.1 Image Data Compression (1)
    9.2 Image Data Compression (2)
    9.3 Predictive Coding

    Lecture 10

    10.1 Transform Coding Techniques
    10.2 Discrete Cosine Transform
    10.3 Modified Discrete Cosine Transform
    10.4 Fast Algorithm for DCT

    Lecture 11
    11.1 Overview of Video Coding Algorithms
    11.2 Motion Estimation for Video Coding Standards
    11.3 Video Compression ( Cited from Yao Wang)

    Lecture 12
    12.1 Rate Distortion Function and Optimal Bit Allocation
    new: Data Compression, data Security and Machine Learning

    Optional Topics

    Lecture 13
    13.1 On Compression Encrypted Data (1)
    13.2 On Compression Encrypted Data (2)
    13.3 On Compression Encrypted Images

    Lecture 14
    Introduction to Distributed Video Coding
    new: Perceptual DVC – Highly Parallel Codes

    Lecture 15
    15.1 Information Theory in Data Visualization (1)
    15.2 Information Theory in Data Visualization (2)

    Grading

    1. Homeworks (20%)

    2. A Brief Survey of your interested IT-related Research topic (Minimum: A4 6 pages 15%)

    3. Programming Assignments (25%)
    • Variable Length Codes
    • Real-Time JPEG Compressor/Decompressor
    • An Optimal JPEG Quantization Table wrt. a given Image

    4. Final Project: (40%)
    • A group (with Maximum 3 members) tries to realize/investigate an Information Theory related final project proposed by the group members.
    • The grading depends on the satisfaction of the group’s 30 mins. recorded pptx-based Oral presentation associate with the Final hand-in report.

    Resource & Reference

    • 1. Elements of Information Theory, Thomas M. Cover & Joy A. Thomas, Wiley 2nd Edit. 2006
    • 2. Introduction to Data Compression, Khalid Sayood, 1996.
    • 3. JPEG/ MPEG-1 coding standard
    • 4. Introduction to Information Theory and Data Compression, Greg A. Harris & Peter D. Johnson, CRC Press, 1998
    • 5. The Minimum Description Length Principle, Peter D. Grunward, the MIT Press, 2007.
    • 6. IEEE Trans. on Information Theory, Circuits and Systems for Video Technology, Signal Processing, Image Processing, Multimedia, Information Forensics and security, Communications, and Computers
    • 7. Open Journal : Entropy

    QA