Reading a PNG by hand
File formats are where hexdump skills pay off outside of executables. PNG is a perfect first format: simple, well documented, and big-endian, which makes a nice contrast with the little-endian ELF header.
The file
A 2×2 yellow image, 79 bytes in total:
00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR|
00000010 00 00 00 02 00 00 00 02 08 02 00 00 00 fd d4 9a |................|
00000020 73 00 00 00 16 49 44 41 54 78 9c 63 fc ff 9f 81 |s....IDATx.c....|
00000030 81 81 81 89 81 81 81 81 81 01 00 19 11 02 02 9d |................|
00000040 ce 17 c3 00 00 00 00 49 45 4e 44 ae 42 60 82 |.......IEND.B`.|The signature: 8 bytes
89 50 4e 47 0d 0a 1a 0a. Every byte has a job:
89: a non-ASCII byte, so tools that only handle 7-bit text notice corruption50 4e 47:PNG, readable in a hexdump0d 0a: a Windows line ending, and later0aa Unix one: a transfer that converts line endings breaks the signature, so the damage is detected immediately1a: Ctrl-Z, which stops thetypecommand on old DOS systems from printing binary garbage
It's a nice example of a format designed to fail loudly.
Chunks
After the signature, the file is a list of chunks, each with the same layout:
4 bytes length of the data (big-endian)
4 bytes type, four ASCII letters
n bytes data
4 bytes CRC-32 of type + data (big-endian)IHDR at 0x08
- length
00 00 00 0d= 13. Big-endian: read the bytes in order, no reversing. - type
49 48 44 52=IHDR - data, 13 bytes:
- width
00 00 00 02= 2 - height
00 00 00 02= 2 - bit depth
08 - colour type
02= RGB, no alpha - compression
00, filter00, interlace00 - CRC
fd d4 9a 73
IDAT at 0x21
- length
00 00 00 16= 22 - type
IDAT: the image data - data starts with
78 9c: a zlib header. The pixels are compressed with DEFLATE, so you can't read them directly here, but78 9cis a pattern worth recognising: you'll find zlib streams inside many formats. - CRC
9d ce 17 c3
IEND at 0x43
- length 0, type
IEND, CRCae 42 60 82. Every PNG ends with exactly these 12 bytes:00 00 00 00 49 45 4e 44 ae 42 60 82. Handy when you need to find where an image ends inside a bigger file.
Check the CRC yourself
python3 -c "import zlib; print(hex(zlib.crc32(bytes.fromhex('49484452000000020000000208020000 00'.replace(' ','')))))"It prints 0xfdd49a73, the same as in the file: CRC-32 over the type and data, not the length.
Decompress the pixels
import zlib
data = open('tiny.png', 'rb').read()
raw = zlib.decompress(data[0x29:0x29 + 22])
print(raw.hex(' '))01 ff ff 00 00 00 00 02 00 00 00 00 00 00Two rows of 7 bytes: a filter byte, then 2 pixels × 3 bytes. The encoder filtered each row to make it compress better:
- row 1, filter
01(Sub): each byte is stored as the difference from the byte one pixel to the left. The first pixel isff ff 00(yellow); the second is stored as00 00 00, meaning "same as the previous pixel". - row 2, filter
02(Up): each byte is the difference from the pixel above, so an identical row is all zeros.
Undo the filters and you get four ff ff 00 pixels: a 2×2 yellow square. Filters are why PNG data compresses well, and why you can't read pixels straight out of a decompressed stream without applying them.
Lessons that carry over
- Check the byte order in the spec before reading any number: PNG, JPEG and network protocols are big-endian; ELF on x86, PE and most in-memory structures are little-endian.
- Length + type + data is one of the most common file structures you'll meet (RIFF, PNG, many game and firmware formats). Once you spot it, you can walk the file even without a spec.
- Data that looks random usually is compressed or encrypted. Look for known headers like
78 9c(zlib) or1f 8b(gzip) at its start.