Game Boy Test ROMs

If you’re building an emulator, you can hardly avoid relying on automated tests to prevent regressions. A common approach is unit tests. They verify individual parts of your code in isolation and therefore need to be tailored to the specific emulator. To complement that, integration tests validate the behavior of the emulator as a whole.

Test ROMs as integration tests

A particularly interesting form of integration testing is test ROMs. They run directly on the emulated platform and probe specific behavior. You could say they test the hardware “from the inside out.”

A Game Boy test ROM could, for example:

• examine timer behavior by timing the exact cycle that increments the TIMA register.
• check PPU mode 3 duration for different SCX, window, and sprite configurations.
• demonstrate the effects of the DIV register on the APU.

Well-known test suites

There are entire collections of Game Boy test ROMs, called test suites. One of the best known is the Mooneye Test Suite, developed by Joonas Javanainen (Gekkio). Also worth your time are Lior Halphon’s SameSuite and Matt Currie’s Mealybug Tearoom Tests. There are plenty of others as well, too many to list here.

I’ve put together a collection of test suites by various authors in one of my GitHub repositories.

Pros and cons of test ROMs vs. unit tests

Compared to unit tests, the barrier to entry for Game Boy test ROMs is significantly higher. As integration tests, they are more involved to run: You have to start the full emulator, load the test ROM and execute it, and then determine the outcome through an appropriate evaluation (e.g., comparing screen output, memory contents, or CPU registers). Creating test ROMs is also more complex: you need a dedicated toolchain, compile ROMs separately, and they’re typically written in assembly — not exactly a mainstream language.

On the other hand, test ROMs come with clear advantages:

1. They are independent of the emulator’s implementation. Any emulator for the platform can use them.
2. They run on real hardware, so anyone can verify correctness there.

Developing Game Boy test ROMs

You’ll need a suitable toolchain to develop Game Boy test ROMs. My favorite is RGBDS combined with a Makefile. RGBDS is tailored to Game Boy development and includes an assembler, linker, and utilities for graphics conversion and header patching. For small projects, rgbds-live even lets you write and try code directly in the browser.

Popular alternatives to RGBDS include WLA DX (a cross-assembler for various CPUs) and GBDK (a C toolchain for Game Boy software).

While RGBDS and WLA DX are assembly-only, GBDK also supports C. For folks new to Game Boy development, GBDK might be the easier path. However, when using C you don’t directly control which CPU instructions the compiler emits. For test ROMs that often rely on cycle-exact timing, assembly is the better choice.

Running on real hardware

To try your software on a real Game Boy, you’ll need a suitable flash cartridge. I currently use the EZ-FLASH Junior, which you load via microSD card. There are other options of this type as well. A quick search for “Game Boy cartridge adapter” on Google or Amazon will turn up viable choices.

Almost all of these adapters can host multiple ROMs at once. On boot, you choose the ROM to run from the Game Boy’s menu.

A simple example test ROM

To illustrate, let’s look at a (very simple) test ROM for DIV timing. It verifies the cycle of the very first DIV increment after a DIV reset, and consists of three parts:

1. Initialize VRAM and BGP
2. Run the actual test
3. Display the result (✓ / ╳)

To try it out, copy the code from Listing 1 and run it on rgbds-live (replace the prefilled main.asm entirely). To make the test fail, you can change the numeric value in line 48 or 54, for example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

; main.asm
;
; This intentionally simple test ROM checks the timing of the
; first DIV increment after a DIV reset.

INCLUDE "hardware.inc"

SECTION "Header", ROM0[$100]
  jp EntryPoint

SECTION "EntryPoint", ROM0[$150]
EntryPoint:
  ; ----- 1. Initialize VRAM and BGP
  ;
  ; wait for V-Blank to disable the LCD
: ldh a, [rLY]
  cp LY_VBLANK
  jr c, :-
  ; turn LCD off
  ld a, LCDC_OFF
  ldh [rLCDC], a
  ; initialize VRAM
  ld de, VramData
  ld hl, STARTOF(VRAM)
  ld bc, VramData.end - VramData
: ld a, [de]
  inc de
  ld [hli], a
  dec bc
  ld a, b
  or a, c
  jr nz, :-
  ; set BG palette
  ld a, %11_10_01_00
  ldh [rBGP], a

  ; ----- 2. Run the test
  ;
MACRO NOPS
  REPT \1
    nop
  ENDR
ENDM
  ; read rDIV one M-cycle before the increment
  ldh [rDIV], a ; reset rDIV, restart counter
  NOPS 60
  ldh a, [rDIV]
  cp a, 0       ; rDIV == 0 right before the first increment
  jr nz, .fail
  ; read rDIV during the increment cycle
  ldh [rDIV], a ; reset rDIV, restart counter
  NOPS 61
  ldh a, [rDIV]
  cp a, 1       ; rDIV == 1 on the first increment
  jr nz, .fail

  ; ----- 3. Display the result (✓ or ╳)
  ;
  ; Test passed: ✓
  ld a, 1
  jr .finish
  ; Test failed: ╳
.fail:
  ld a, 2
.finish:
  ; Show result: ✓ or ╳
  ld [STARTOF(VRAM) + $1800], a
  ld a, LCDC_ON | LCDC_BLOCK01 | LCDC_BG_ON
  ldh [rLCDC], a
: jr :-

SECTION "VRAM data", ROM0
VramData:
  ; Tile 0: empty
  ds 16, 0
  ; Tile 1: ✓
  dw `00000000
  dw `00000033
  dw `00000333
  dw `00000330
  dw `03303300
  dw `03333300
  dw `00333000
  dw `00033000
  ; Tile 2: ╳
  dw `00000000
  dw `03300033
  dw `03330333
  dw `00333330
  dw `00033300
  dw `00333330
  dw `03330333
  dw `03300033
  ; remaining VRAM
  ds $2000 - 3 * 16, 0
.end

Implementing a test framework

As you can see, the test ROM above is largely initialization and result display. The actual test logic takes about 20 of the roughly 100 lines. In a full test suite, this boilerplate code would repeat in every single test ROM. It’s therefore worth moving those parts into a shared test framework that each test ROM can include.

Typically, a test ROM includes the framework code before its own test code. The framework handles program startup and initialization and provides helpers commonly used across the suite. The test suites mentioned above — Mooneye Test Suite, SameSuite, and Mealybug Tearoom Tests — use this approach.

Evaluating the result in the test ROM

If we pull initialization and result display out of the example test ROM into a test framework, we’re left with Listing 2. After initialization by the framework, the test ROM runs its checks as usual. It also still performs its own evaluation: on failure it calls the TestFail routine to finish (lines 16 and 23), and on success it calls TestSuccess (line 26).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

; main.asm

INCLUDE "test-framework.inc"

MACRO NOPS
  REPT \1
    nop
  ENDR
ENDM

  ; read rDIV one M-cycle before the increment
  ldh [rDIV], a ; reset rDIV, restart counter
  NOPS 60
  ldh a, [rDIV]
  cp a, 0
  jp nz, TestFail

  ; read rDIV during the increment cycle
  ldh [rDIV], a ; reset rDIV, restart counter
  NOPS 61
  ldh a, [rDIV]
  cp a, 1
  jp nz, TestFail
  
  ; Test passed
  jp TestSuccess

The test framework (Listing 3) provides program start and initialization as well as the TestSuccess and TestFail routines. It also takes care of displaying the result.

This makes individual tests leaner and easier to maintain. At the same time, the test ROM remains completely free in how it evaluates its results and can abort early if needed (“fail fast”).

The Mooneye Test Suite is implemented this way.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

; test-framework.inc

INCLUDE "hardware.inc"

SECTION "Header", ROM0[$100]
  jp EntryPoint

SECTION "Utilities", ROM0[$150]
; Test passed: display ✓
TestSuccess:
  ld a, 1 ; ✓
  jr Finish

; Test failed: display ╳
TestFail:
  ld a, 2 ; ╳
  jr Finish

Finish:
  ld [STARTOF(VRAM) + $1800], a
  ld a, LCDC_ON | LCDC_BLOCK01 | LCDC_BG_ON
  ldh [rLCDC], a
: jr :-

VramData:
  ; Tile 0: empty
  ds 16, 0
  ; Tile 1: ✓
  dw `00000000
  dw `00000033
  dw `00000333
  dw `00000330
  dw `03303300
  dw `03333300
  dw `00333000
  dw `00033000
  ; Tile 2: ╳
  dw `00000000
  dw `03300033
  dw `03330333
  dw `00333330
  dw `00033300
  dw `00333330
  dw `03330333
  dw `03300033
  ; remaining VRAM
  ds $2000 - 3 * 16, 0
.end

SECTION "EntryPoint", ROM0
EntryPoint:
  ; wait for V-Blank to disable the LCD
: ldh a, [rLY]
  cp LY_VBLANK
  jr c, :-
  ; turn LCD off
  ld a, LCDC_OFF
  ldh [rLCDC], a
  ; initialize VRAM
  ld de, VramData
  ld hl, STARTOF(VRAM)
  ld bc, VramData.end - VramData
: ld a, [de]
  inc de
  ld [hli], a
  dec bc
  ld a, b
  or a, c
  jr nz, :-
  ; initialize BG palette
  ld a, %11_10_01_00
  ldh [rBGP], a

  ; from here on, control passes to the actual test

Evaluating results in the test framework

Our example test ROM only checks two data points: the value of DIV immediately before and exactly during its first increment. That’s manageable to evaluate inside the ROM itself. But if a test produces hundreds or even thousands of data points, doing the comparison in the ROM quickly becomes unwieldy.

An alternative is to perform the comparison in the test framework. To do that, the framework reserves a region in WRAM. Each test ROM stores its measured values there and also provides the expected results. The framework compares both and then shows pass or fail.

The advantage is that test ROMs can become even slimmer, since they no longer have to evaluate their own results.

The downside is that the test must always run to completion before you get an outcome. A “fail fast” approach isn’t possible, as that would require evaluation within the ROM.

This is how SameSuite and Mealybug Tearoom Tests operate.

Using test ROMs in automated integration tests

To run the example test ROM as an automated integration test, you load it into the emulator under test and let it run for a certain time. After that, you verify the result automatically. To make this reliable, two questions need answers:

1. How long does the example test ROM need to run before a result is available?
2. How is the result checked automatically — i.e., pass or fail?

Of course, these questions apply to every test ROM, not just our example.

Current convention

Mooneye Test Suite, SameSuite, and Mealybug Tearoom Tests all follow the same convention here. The Mooneye Test Suite likely led the way, originally being part of the Mooneye GB emulator.

1. A test is considered complete once the instruction ld b, b executes. Since that instruction has no effect other than taking one cycle, it’s not used in normal code.

2. A passing test is signaled by a specific set of CPU register values — a slice of the Fibonacci sequence:

   • B = 3
   • C = 5
   • D = 8
   • E = 13
   • H = 21
   • L = 34

   If a test finishes with different values, it is considered failing.

Our example test ROM adheres to this convention once you adjust the test framework as shown in Listing 4.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

; test-framework.inc

; < ... >

; Test passed: display ✓
TestSuccess:
  ld a, 1 ; ✓
  ld b, 3 ; signal “test passed” via CPU registers
  ld c, 5
  ld d, 8
  ld e, 13
  ld h, 21
  ld l, 34
  jr Finish

; Test failed: display ╳
TestFail:
  ld a, 2 ; ╳
  ld b, 0 ; signal “test failed” via CPU registers
  jr Finish

Finish:
  ld b, b ; signal “test finished”
  ld [STARTOF(VRAM) + $1800], a
  ld a, LCDC_ON | LCDC_BLOCK01 | LCDC_BG_ON
  ldh [rLCDC], a
: jr :-

; < ... >

Visual/analog test results

Not every test ROM can evaluate its own result. A classic example is video output: the test ROM can render a picture, but it can’t verify whether it’s displayed correctly.

In such cases, the test ROM simply signals completion with ld b, b. The actual evaluation is done by the integration test harness. To do that, the test ROM is bundled with a reference screenshot, which is compared to the emulator’s actual output.

There’s also a convention for the colors in such screenshots:

• DMG screenshots use these colors:
  #000000, #555555, #AAAAAA, #FFFFFF
• CGB screenshots convert 15-bit CGB colors to 8-bit per channel:
  (X << 3) | (X >> 2)
• “Non-CGB mode” screenshots use these colors:
  • BGP: #000000, #0063C6, #7BFF31, #FFFFFF
  • OBP: #000000, #943939, #FF8484, #FFFFFF

Conclusion

• Test ROMs are ideal as integration tests for emulators: They verify the behavior of the emulated hardware and run both on emulators and on real hardware.
• Compared to unit tests, they’re more effort to create and run, but once written they can be reused across emulators.
• For precise timing, assembly is the better choice over C.
• A test framework cuts boilerplate and makes it much easier to build larger test suites.
• With tools like RGBDS, WLA DX, or GBDK and a suitable flash cart (e.g., EZ-FLASH Junior), you can cover everything from development to testing on real hardware.