Beginning Game Development - A Dreamcast Dev Guide

Guides to jumpstart your Retrogaming lifestyle
Post Reply
User avatar
Krejlooc
128-bit
Posts: 525
Joined: Mon Feb 27, 2012 4:27 pm

Beginning Game Development - A Dreamcast Dev Guide

Post by Krejlooc »

NOTE: I wanted to have this all completed for the Dreamcast's birthday this year, but it looks like the entire thing will not be complete. But in an effort to get something out by the birthday, I'm releasing the first 2 chapters in their entirety, plus half of chapter 3. I write these things mainly on the weekends, and am still working on chapter 3 (of a planned 7 chapters). The size of this tutorial has ballooned greatly. My first chapter used to be all of 3 pages long, now my introduction alone takes up that much space. Currently, I'm sitting at 32 pages for this tutorial.

What we have here is enough to get you through the process of setting up your toolchain, learning how to compile and build Dreamcast programs, a primer on some topics you will need to know for Dreamcast development, and enough to get you to plotting pixels on the screen for your first ever program. Consider this a work-in-progress tutorial


BEGINNING GAME DEVELOPMENT
A Dreamcast dev tutorial

INTRO
Developing a video game can be a deeply rewarding experience. It is a demonstration of technical proficiency in a number of disciplines that makes it rewarding to engineering types, while simultaneously being an expression of creativity and personality that makes it rewarding for artistic types. If you hang around on gaming forums long enough, you'll undoubtedly see people express the desire to try their hand at making games. There is a creative spark in amateurs that manifests in many art forms, from fan films to cover bands.

I've been doing computer programming focusing on game development for over 20 years now. One of the things I did that immensely helped my understanding of game development was picking a classic console and learning how to program for it. Since about 1998, I've been studying the Sega Genesis, called the Sega Megadrive outside North America, as a hobby. I picked up m68k microprocessor assembly in the process and played along the ROM hacking scene for many years. Learning how the genesis worked gave me a deep understanding of how video games actually interact with hardware, and how those interactions fuel design choices. I've often recommended people pick up a classic system if they are serious about learning game development. Unfortunately, it's pretty hard for a complete beginner to jump into that kind of system due to comparatively weak hardware. We live in an era where programmers have essentially unlimited resources if they are developing a 2D game, which makes jumping back to an era where you are literally counting clock cycles enormously difficult. A better system to recommend people pick up is the Sega Dreamcast.

The Dreamcast is a wonderful system. I consider it the last classic console - in terms of 3D, it offers no shader support (although you can perform bump mapping), nor does it have an incredible CPU that can brute force shader effects. In terms of 3D, it's still of the fixed function pipeline era of OpenGL. It's GPU, the Holly chip, is part of an exotic design - the PowerVR 2 core. It even has a real dedicated sprite mode for 2D graphics. All this means that the Dreamcast has a pretty unique visual style. In terms of design, the Dreamcast is sort of the opposite of the Playstation 2. The PS2 had a powerful CPU - The Emotion Engine - to offset a crummy graphics chip, with only 4MB of fast ram. The Dreamcast, by contrast, had a weak CPU even for the time, but a pretty incredible graphics chip that punched way above its weight to go along with a huge 8MB of slow ram. The PVR Chip is pretty fun to play with - it offers tile based deferred rendering, and has hardware functions that greatly improve it's performance and ways to get even more out of your already-large pool of ram - sometimes even 32 times more.

What all that amounts to is a system that is still unique enough to affect your approach to game design, like you have to do with mega drive programming, but with a lot more power. You need to tap the Dreamcast hardware in a specific way to make it sing, and doing so is pretty fun. I've researched online for a long time and there are many great resources for Dreamcast development out there, but no one singular guide to bring everything together.

The goal of this tutorial is to introduce you to Dreamcast game development. It's meant to be a small slice of what game development can be like, concentrating more on the technical side of the equation than the creative side. With fans who want to dabble in homebrew, there never seems to be a loss for creativity anyways. This guide will walk you through making a breakout clone for the Sega Dreamcast. Breakout was the very first game I ever made using QBasic, several decades ago. Breakout is such a simple game to get your feet wet with - there is really no AI, the rules and mechanics are simple, and we can concentrate most of our energy on actually getting the to know the Dreamcast environment.

This guide will walk you through every step, from you having no tools on your PC (Windows or Linux) at all to running the game on your Dreamcast. Along the way, it will discuss optimization, and teach techniques that can apply to more than just Dreamcast programming. It will also walk you through the creation of tools we will need to make our Dreamcast game, that help us interact with the hardware on deeper levels.

Be mindful, though, that while this is a guide for beginners, a degree of basic programming knowledge will be assumed, but the point of this topic is for beginners to ask questions as we progress through this project to help understand what very basic development is like. I'll also try to explain the logic of what is going on as best as possible. Additionally, I make no claim to being the world's best coder and I'm still pretty new to Dreamcast Development in general, so if my code is embarrassing or I'm doing something all wrong, please feel free to correct me. In the end, this tutorial is for my own benefit as well.

TABLE OF CONTENTS
-LESSON 1: CREATING THE TOOLCHAIN
-LESSON 2: BUILDING OUR FIRST EXAMPLE PROJECT
-LESSON 3: SOFTWARE RENDERING

FILES

RECOMMENDED RESOURCES
Official KallistiOS reference manual
http://cadcdev.sourceforge.net/docs/kos-2.0.0/
The official online manual for KallistiOS, with reference pages for every function and file in KallistiOS.

DC Emulation
https://dcemulation.org/
DC Emulation is the best resource for Dreamcast programming I found. The board is active, as is the IRC chat (chat.freenode.net, #Dreamcastdev). Many of it's moderators have a deep understanding of KOS, and you get answers to questions pretty quickly.

Dreamcast Programming by Marcus Comstedt
http://mc.pp.se/dc/
Marcus is one of the people who figured out the bin scrambler. His website is old, but is full of great Dreamcast knowledge that is still relevant today

Sega Dreamcast Hardware Specification Outline
http://hwdocs.webs.com/Dreamcast
Sega's official Hardware outline for the Dreamcast, good for a basic understanding of the Dreamcast architecture.

Lazy Foo's SDL Tutorials
http://lazyfoo.net/tutorials/SDL/index.php
Much of our PC software will use SDL 2.0. This guide is not intended to be a tutorial for SDL 2.0, but Lazy Foo's tutorials are where I learned SDL 2.0. You can use his pages if you wish to learn more about SDL 2.0

Gamasutra - Image Compression with Vector Quantization
http://www.gamasutra.com/view/feature/131499/image_compression_with_vector_.php
While I briefly cover Vector Quantization in this tutorial, this is a better, fuller read on the subject. In fact, I use images from this article to help with the concept.

SPECIAL THANKS TO
I couldn't have possibly learned all I have learned or compiled this guide without the following people or places. It is with a huge debt of gratitude to the following that this guide exists:

Code: Select all

BlackAura
BlueCrab
MetalliC
Tvspelsfreak
Bogglez
tonma
The Taxman
Lazy Foo
Marcus Comstedt
Dan Potter
Light-Dark
Chilly Willy
SWAT
Ivan-Assen Ivanov
Gecko Yamori
Zeboyd Games


PLEASE NOTE: ALL FILES WITHIN THIS TUTORIAL HAVE EITHER BEEN CREATED BY ME, OR USED WITH PERMISSION FROM THE ORIGINAL AUTHORS. ALL CODE IS APPLICABLE UNDER GPL.
Last edited by Krejlooc on Fri Sep 09, 2016 10:17 pm, edited 1 time in total.
User avatar
Krejlooc
128-bit
Posts: 525
Joined: Mon Feb 27, 2012 4:27 pm

Re: Beginning Game Development - A Dreamcast Dev Guide

Post by Krejlooc »

LESSON 1: CREATING THE TOOLCHAIN

TABLE OF CONTENTS
-BEGINNING DEVELOPMENT
-STEP 1: CONFIGURING YOUR PC
-STEP 2: INSTALLING KOS

BEGINNING DEVELOPMENT
For a large number of people reading this tutorial, this will likely be their first experience with homebrew development. Homebrew development can be at times much different than working with official libraries. In any job, especially when you are just beginning to work with a new tech, actually beginning is usually one of the most difficult steps involved. This is exasperated with homebrew development, because homebrew tools have been hobbled together over the years and thus up-to-date documentation can be hard to find. Luckily, Dreamcast homebrew remains vibrant and robust, thanks in large part to KallistiOS, also called KOS (pronounced "Chaos").

Modern development is often done with the help of a powerful suite called an IDE, an integrated development environment. An IDE is sort of a one-stop programming suite that contain everything you need to make a program. The most popular IDE is Visual Studio by Microsoft. Our goal in this first lesson is to, essentially, build an IDE for Dreamcast development. Now, that task is less daunting when you understand the toolchain process and how an IDE functions. An IDE, in actuality, isn't a single program but rather hundreds and hundreds of small programs, packaged together, and wrapped up in a nice UI that makes everything look like one big, integrated program. When you, for example, call upon your IDE to build your project, what is actually happening is that you are sending your source code to another program called a compiler, which is fed options from the IDE, that takes the source and ultimately pumps out an executable object file. The hundreds of programs working in concert to make development of your program possible is known as a toolchain.

Setting up your tool chain is often the biggest stumbling block for beginning Dreamcast developers. This is because each tool in the toolchain needs to be configured per system. ISO images of a windows XP installation with a Dreamcast toolchain already installed exist, but should be avoided because the toolchain updates pretty regularly and thus those tools are already out of date, and because of the hardware differences between your computer and the computer the windows XP installation first occurred on in the first place. To give a real world example - when this tutorial began in 2014, the process of setting up KOS was actually different than it is today. I had to update the install process between the two years it took for this tutorial to come to completion.

The only real way to setup a Dreamcast toolchain is to do it yourself, locally, and set by step. We first need to set our PC up. This guide works with all versions of windows to my knowledge - I have set it up most recently on Windows 10 Pro x64. Additionally, it will also work with Linux - I have tested it with the most up-to-date Ubuntu distro. When appropriate, I will provide additional steps and tips, should there be differences between operating systems.

STEP 1: CONFIGURING YOUR PC
Most of what we do in this tutorial will use either a text editor or a terminal. For Linux, any text editor or terminal program will do. I personally use gedit and GNOME Terminal, although others assuredly would work.

For windows, we will need CYGWIN as our terminal, and Notepad++ as our text editor. CYGWIN is a UNIX-like environment for windows. Notepad++ is a very lightweight text editor with lots of functions built in to aid with coding, like highlighting syntax. More importantly, notepad++ lets us change our encoding style in ways that Word Pad or notepad won't. See, those programs will use windows encoding randomly and it can screw up our gnu tools, which expects a UNIX encoding style. An example is how return characters in Word Pad will appear as the character '/r' to gnu, which causes all sorts of problems. Notepad++ is what we will be writing our Dreamcast source codes in, as well as editing our Makefile scripts and building additional scripts to help us compile.

First, download Cygwin from here: https://www.cygwin.com/

Please note that, even if using a 64-bit operating system, the x64 version of Cygwin is unstable and not suitable for this project. Using Cygwin64 will cause problems. Even if you are on a 64-bit operating system, download the x86 32-bit version of Cygwin instead.

Begin by installing Cygwin onto your PC. When you install Cygwin, make sure the following packages are installed as well:

Code: Select all

libgmp-dev
libmpfr-dev
libmpc-dev
gettext
wget
libelf-dev
texinfo
bison
flex
sed
make
tar
bzip2
patch
gawk
git
gcc
g++


In practice, this amounts to making sure the following boxes are checked at install:

Image

When Cygwin is installed, run it and you will see what looks like a DOS terminal. This works much like a standard UNIX file structure. Located within C:\Cygwin is a number of files and folders that seem to correspond to the typical files and folders you'd find in UNIX: usr, home, lib, var, etc. Any time I refer to the terminal, I will be referring to Cygwin when running windows.

Now, install Notepad++. Any version will do: http://notepad-plus-plus.org/download/v6.6.9.html

Once you have your terminal and text editors set up, you'll also need a graphics editing program. I recommend GIMP for both Windows and Linux. In fact, many of the examples I write in this program will be using GIMP. Install GIMP from the website (or apt-get install in Linux): https://www.gimp.org/

Now, open a terminal window. If you are running windows, you need to make sure to give your terminal window Administrator privileges for many of the steps we are doing. To do this, right click on your Cygwin.exe icon instead of left clicking it, and click "Run as Administrator." If you'd like, you can make a shortcut to your Cygwin.exe and set it to automatically launch as administrator by right clicking and going into properties, then clicking "always run as administrator." For Linux, you can grant super user privileges by using the sudo prefix with your commands. I will try to show use of sudo when necessary.

If you are running Windows and using Cygwin, we need apt-cyg installed. Apt-cyg is the CYGWIN equivalent of apt-get for Linux. Apt-get is a program that makes downloading and installing tools, libraries, and programs into 1 command line, simplifying the process. To install apt-cyg, type the following into your terminal:

Code: Select all

lynx -source rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg
install apt-cyg /bin


If you are using Linux, you need to make sure the packages above are installed on your system. These packages can be obtained with the following command:

Code: Select all

sudo apt-get install libgmp-dev libmpfr-dev libmpc-dev gettext wget libelf-dev texinfo bison flex sed make tar bzip2 patch gawk git gcc g++


Additionally, the last time I built the toolchain in Ubuntu, a few graphics libraries weren't installed. Just in case, do the following command:

Code: Select all

sudo apt-get install libpng12-dev libjpeg8-dev libpng12 libjpeg8


We also need mkisofs manually installed in Linux. mkisofs is a program that makes our iso file system, needed later. Run the following code:

Code: Select all

sudo apt-get install mkisofs


Both windows and Linux need cdrecord installed. It's a program we will use to record our Dreamcast disks. In windows, run the following command from the terminal:

Code: Select all

apt-cyg install cdrecord


in Linux, the command is:

Code: Select all

sudo apt-get install cdrecord


In addition, windows users can use a special version of IMGBurn to burn Dreamcast images. I am providing a mirror to this special version of IMGBurn, both here by itself, and packaged into a rar containing all the files we need for this project in the opening post. To grab this version of IMGBurn by itself, click here: {img burn link}

Finally, we need an emulator to let us rapidly test our projects as we work on them. It's perfectly fine to work entirely by SD Card or burning CDs using a real console, of course, but it becomes frustrating when each build of your project turns into a several minute-long process. In normal development, you might need to make dozens of builds within minutes to debug a problem. Emulators let us skip the burning process of the build. Note that Dreamcast emulation is not perfect by a long shot, and as a result it's nice to periodically check your stuff on a real Dreamcast to make sure it's not working differently. We will be using NullDC for our testing purposes. I have packaged a version of NullDC for windows in the link in the opening post, or you can grab it separately here: [NullDC link]

NullDC will run in Linux, too, but we need to run it through Wine. Wine is a program that lets us run some windows applications in Linux. To install wine, send the following command through the terminal:

Code: Select all

sudo apt-get install wine


It'll run for a while then require you agree to an EULA for some Microsoft products. Tab over to OK and click "OK", then "Yes."

Finally, we will need SDL 2.0 installed and set up on your PC. SDL 2.0 is a cross-platform development library that handles PC multimedia, including graphics, sound, and controller input. We will use SDL 2.0 to make tools for our game later on. Learning about SDL 2.0 could be an entire project in and of itself, so I will limit my discussion of SDL 2.0 as much as possible. I don't intend to reinvent the wheel when it comes to SDL, so rather than walking you through setting up SDL 2.0, I'll point you to the best tutorial on the net I've found: http://lazyfoo.net/tutorials/SDL/01_hello_SDL/index.php

For the sake of preservation and the general completeness of this tutorial, the above pages and files from the Lazy Foo SDL 2.0 tutorial have been archived, both in the pack in the first post and individually here: [Lazy Foo SDL Tutorials]

Follow the above guide. Note that the PC I'm working off of for this tutorial already was configured to build SDL projects with Visual Studio 2013, so that's the IDE I use for developing SDL 2.0 applications. The above tutorial is short and walks you through setting SDL 2.0 for a variety of free and commercial IDEs in both Linux and windows (as well as OSX). Setting up SDL with the IDE of your choice shouldn't take very long at all. Once you have it set up and can run his test program, you can return to this tutorial and go to the next step.

STEP 2: INSTALLING KOS
Open your terminal. The folder it opens to should be your home folder. In Cygwin, this is usually "/home/" while it can vary depending on your Linux distro. We will refer to this home folder with a tilde in the future. I.e. "~/" is usually "/home/" in Cygwin.

first, we want to make a dc (Dreamcast) folder within our home folder. type the following command and hit enter:

Code: Select all

mkdir dc


now, let's navigate inside that folder, this folder is where our Dreamcast toolchain will be setup. Type the following to change directories to the dc folder:

Code: Select all

cd dc


Now, within the dc folder, we will use git to download and configure KallistiOS, a collection of homebrew tools to develop Dreamcast applications with. Use the following command to download KallistiOS:

Code: Select all

git clone git://git.code.sf.net/p/cadcdev/kallistios kos


This will run for a while. When it finishes, we will have a folder called KOS inside of dc, this is where our KallistiOS tools will reside. Now, use the following command to grab some additional KallistiOS porting tools:

Code: Select all

git clone git://git.code.sf.net/p/cadcdev/kos-ports


This will create a folder inside of dc called KOS-ports. Let's navigate to that directory:

Code: Select all

cd kos-ports


Let's use git to initiate an update from within this folder:

Code: Select all

git submodule update --init


Now we are finally ready to actually install and configure KallistiOS (we'll call it KOS from here on out to keep things short). Use the following to navigate to the KOS folder:

Code: Select all

cd ../kos/utils/dc-chain


As a note to beginners, the folder ".." represents a parent path. You can use ".." in any path to indicate a recursive return to a higher parent. So, parsing that code, we change directories to parent (we were in /dc/kos-ports, so the parent is dc), then kos, then utils, then dc chain, meaning our final location is /dc/kos/utils/dc-chain.

From within this folder, run the following command:

Code: Select all

./download.sh --no-deps


This will run the download script. sh files are scripts which send commands to Cygwin (or UNIX or whatever) that we will be using extensively to compile our programs later on. We'll go into more detail about how to build and setup scripts later on, right now the above script will download the dc-chain within KOS.

An additional note, the path "." means "current working folder." Because you need to indicate the exact location a file resides, you can use "." to avoid having to type out the full directory. In this case, "." represents "dc/kos/utils/dc-chain" meaning the full command we issued was "dc/kos/utils/dc-chain/download.sh --no-deps"

this is going to run for a long time. Once it finishes, we will setup the tool chain using the following command:

Code: Select all

./unpack.sh --no-deps


Let this run a bit, once it unpacks all the files, we will make the KOS toolchain. Let's explain a sec what the process of make is. make is a program that builds executable and libraries from scripts called Makefiles. You can think of make as sort of a command-line tool you use to feed your compiler source code to, and you select what source and build options for make to use by supplying it with a Makefile. In this instance, make is going to configure our libraries for us.

Execute make with the following command:

Code: Select all

make


this will run the Makefile that your unpack command generated. This will run for a while, at which point you now have your libraries configured. Go up to the kos folder like so:

Code: Select all

cd ../..


That's "parent" of "parent," remember. Now, we need a script to configure our environment, KOS has an example included, copy it like so:

Code: Select all

cp doc/environ.sh.sample ./environ.sh


This copies environ.sh.sample (it's a script) from doc to kos and renames it environ.sh. We can now set aside cygwin, as we need another piece of software.

Now that notepad++ is installed, we need to edit our environ.sh inside of our KOS folder. Environ.sh is a script that sets up a bunch of variable names we will use when compiling our projects. We need to edit environ.sh with our Text editor. On windows, this means you have to actually navigate into the Cygwin folder. Open up windows explorer or my computer or whatever and navigate to C:\Cygwin or wherever you installed Cygwin. Now, inside of Cygwin, find your KOS folder. On my PC, it's

Code: Select all

D:\Cygwin\home\Dreamcast\dc\kos\


Where [windows user name] is the name of the account you're logged in as under windows. On my PC, my user name is "Dreamcast". Locate environ.sh and open it with notepad++. Likewise, you can use an editor like gedit to open it in Linux.

[KOSPATH image]

The highlighted line in the above image, "# KOS main base path" is what we need to change. We need it to point it to our KOS folder. On windows, this path is relative to Cygwin, and we need to use a POSIX style folder location, which involve simply changing the backslashes to forward slashes. That means that if the folder was at "D:\Cygwin\home\Dreamcast\dc\kos\" then we would only consider "home\Dreamcast\dc\kos\".

Now, we need to run our environ.sh script. To do that, we need to make it an executable script. Do this with the following command:

Code: Select all

chmod u+x environ.sh


Now, we prep our script for make:

Code: Select all

source environ.sh


Finally, run make:

Code: Select all

make


make executes our script. We've now built our environment for our compiler. Next, we need to compile and set up our KOS ports. First, navigate to the kos-ports folder:

Code: Select all

cd ../kos-ports


The KOS Ports model has changed from my first tutorial. It is recommended that you pick and choose which ports to install and use. For the sake of simplicity in this tutorial, we will install all of them, although you can later pick and choose which ports to install if you'd like. Navigate to the folder inside kos-ports called utils:

Code: Select all

cd utils


from within here, run the following:

Code: Select all

./build-all


You need to do those last steps - make environ.sh executable, source environ.sh, then make, then building kos-ports - every time you first run a fresh terminal if you want to compile your source code. These steps, what we've done, is primed our compiler to be able to generate Dreamcast executable files. To make this easier, let's set these steps to occur automatically. In Linux, this means you want to edit your bash configuration script located at "~/.bashrc" and Cygwin has a similar file. In "~/.bash_profile"

Open your configuration script in a text editor and put the following at the bottom of your script:

Code: Select all

cd dc
cd kos
chmod u+x environ.sh
source environ.sh
make
../kos-ports/utils/build-all.sh


Now, every time we open our terminal, it'll be ready to compile Dreamcast code.

Finally, we will eventually use a program called Scramble during our build process. It is located in ~/dc/kos/utils/scramble as a source file. You have to build it for windows. To make things simple, I have included an already built copy of scramble.exe in the file pack in the opening post, and individually here:

[scramble file]

You need to extract the executable into the "\bin\" folder of Cygwin. On my computer, that is "D:\Cygwin\bin\".

Our PC is finally set up to start building Dreamcast projects from source. In our next tutorial, we will do just that, building an example included with KOS.
User avatar
Krejlooc
128-bit
Posts: 525
Joined: Mon Feb 27, 2012 4:27 pm

Re: Beginning Game Development - A Dreamcast Dev Guide

Post by Krejlooc »

LESSON 2: BUILDING OUR FIRST EXAMPLE PROJECT

TABLE OF CONTENTS
-WHAT DOES IT MEAN TO COMPILE A DREAMCAST PROJECT? WHAT DOES IT MEAN TO BUILD A DREAMCAST PROJECT?
-STEP 1: CREATING DIRECTORIES AND FILES FOR THE PROJECT
-EXPLAINING THE KOS FILESYSTEM
-STEP 2: CONFIGURING OUR MAKEFILE
-STEP 3: BUILDING THE PROJECT
-STEP 4: RUNNING OUR PROGRAM


WHAT DOES IT MEAN TO COMPILE A DREAMCAST PROJECT? WHAT DOES IT MEAN TO BUILD A DREAMCAST PROJECT?
So, now that our toolchain is all set up, let's try building a project and running it, both in an emulator and on a real Dreamcast. Before we begin, let's first talk about what compiling actually is. Colloquially, compiling has become a catch all term for "Turn source code into executables." More specifically, the compiler we are using, gcc, does two things - first it converts files into object code (called objects, denoted as *.o files), and then it links objects together into a single binary that is executable. Since Windows is the dominant operating system, most people assume an executable is an .exe file, but a more common executable format outside of windows is .elf (Executable and Linkable Format).

When we compile our Dreamcast projects, we will output an elf file that is capable of being executed by the Dreamcast. There are multiple ways to deploy an elf file on a Dreamcast - for example, there are windows-like shells you can run on the Dreamcast, like Dreamshell, that will let you mount an SD card and execute elf files by clicking and opening them, and emulators like NullDC are capable of loading an elf file like a rom. But, ultimately, we are interested in running our program on a real, stock Dreamcast, which means we will have to burn our program into a bootable CD-ROM. To understand how we build a bootable CD-ROM, we must first understand what a CD-ROM is. From the perspective of our computers, a CD-ROM is a continuous stream of binary - 1's and 0's, which are further arranged into sectors, which are further arranged into tracks (owing to the format's roots in audio). There is no inherent file system when talking about data stored at this level (yes yes, ignore sector segmenting for a moment); rather data is stored sequentially, meaning if I had 3 files each 1 byte big, they would be stored as 24 bits back to back with no discernible distinction between the files. File systems, interfacing, and data formats make sense of this data at a higher level, but on the disk the best we can do is tell which specific bits we are reading in a stream of them. We identify what area we are currently examining via a seek offset. The Dreamcast boot sequence is tuned to look for two files at two specific regions of the CD in order to boot - IP.bin and 1st_read.bin.

IP.bin is the bootstrap loaded from a reserved sector area on the last track of a CD, and contains a checksum (which, funnily enough, is not checked), a Sega license screen graphic (which is required to boot - an old trick console makers used to do for legal reasons to allow them to classify non-licensed software as trademark violations), an area that defines the region of the disk - Japan, US, or Euro. More importantly, IP.bin also contains meta information defining 1st_read.bin. IP.bin's bootstrap will execute code to display the Sega License Screen graphic (and check that it's not modified), then move to a second bootstrap that sets up some hardware registers. Many homebrew applications will use this second bootstrap to load a second graphic under the Sega License Screen graphic explaining that the licensing terms from the Sega graphic should be ignored, like so:

Image

After which, the bootstrap transfers control over to 1st_read.bin. 1st_read.bin is actually our ELF, stripped of all the headers it gains when compiling, and converted into binary. It is placed at offset 0x8C010000 which IP.bin branches to in execution. The Dreamcast reacts differently from here on out depending on whether it is reading a GD-ROM or a CD-ROM. If the disc is a GD-Rom, the Dreamcast will read 1st_read.bin linearly into memory, as would be expected. To prevent people from running programs from CD-ROMs (as one of their anti-piracy measures), if one is detected the Dreamcast will instead read 1st_read.bin using a pseudo-random scattered pattern generated from a specific arbitrary seed, scrambling the binary out of order and destroying it in the process. We can defeat this process by reverse-scrambling our elf when converting it to 1st_read.bin by using the opposite of this pseudo-random scatter. This way, the scatter cancels out and our ELF is preserved when it enters memory for execution.

Many assume the Dreamcast had poor anti-piracy measures, but the reality is that the Dreamcast was broken wide open early on by a pretty heinous and serious breech of security. The only reason Dreamcast piracy and homebrew were ever possible is because a Katana SDK - the official Sega SDK for the Dreamcast, was stolen by the warezgroup Utopia sometime around late 1999. The Katana SDK contained a scrambling program that would automatically reverse scramble your binary for you, intended to let developers burn non-game related tools on CD-ROM instead of GD-ROM. Access to the Katana SDK did not mean piracy groups could freely author software, as it still took a few years of research to reverse engineer the pseudo-random scatter algorithm (which they did by examining pre-scattered code against post-scattered code), but rather it allowed them to create one single program which could load other programs, which they dubbed the utopia boot CD. Had that Katana SDK never been stolen, the entire Dreamcast piracy and homebrew scenes would likely never have materialized.

Anyways, because these files (IP.bin and 1st_read.bin) need to be in precise locations on the CD, we cannot simply burn our elf like a PC data disk and expect the program to execute. Rather, we need to pack both files into a binary CD image, so we can place them at precise offsets, and then burn that image. Luckily, KOS provides utilities to do that manually (mkisofs), and there are also easier gui utilities to do that automatically (bootdreams). Additionally, both of those utilities will also let us add other external files to our CD-ROM image beyond our executable binary into the other unused sectors of the CD-ROM, like music, graphics, data, and so forth. From there, we can finally burn our disc and boot our program on a real Dreamcast.

to sum up, the entire process looks like this:

Image

All that looks gnarly, but in practice, it comes down to just a few commands after setting up our Makefile. It's not a painful process once you do it a few times, and shouldn't be a problem to build in rapid succession multiple times during development. Additionally, we can use emulators at times to cut out the steps involving actually burning the CD. For the sake of clarity, going forward, use of the term "Compile" will refer to the process of creating the ELF, while use of the term "Build" will refer to the entire process of creating the elf and converting it to a CD-ROM image along with external files. I.e. "Compile the project" = "create an elf file"; "build the project" = "create a booting CD that includes our program and external files."

STEP 1: CREATING DIRECTORIES AND FILES FOR THE PROJECT
With all that said, let's first do some house keeping to keep our directory lists nice and organized. I prefer to keep all my active projects in a folder located within the KOS folder in our tool chain. To do this, in cygwin, navigate to:

Code: Select all

cd ~/dc/kos/


The tilde '~' represents a path to our home directory in cygwin. Now that we are in the KOS folder, make a folder called 'Projects':

Code: Select all

mkdir Projects


To make sure our tool chain ws set up correctly and KOS is installed, we will try compiling the example project called "png." I prefer to leave the projects untouched in their original folder and copy them into my projects folder so I can have an original copy if I need to revert. The example project we want is located at ~/dc/kos/examples/Dreamcast/png. First, make a directory in our Projects folder called png

Code: Select all

mkdir ~/dc/kos/Projects/png


then copy the contents of ~/dc/kos/examples/Dreamcast/png into this the folder we created:

Code: Select all

cp -r ~/dc/kos/examples/Dreamcast/png/* ~/dc/kos/Projects/png/


the "-r" flag we used in the copy command means we are copying all the files (indicated by *) and folders in ~/dc/kos/examples/Dreamcast/png/ recursively, which is a long way of saying we will copy folders as well as files.

Before we continue, I like to set my terminal to automatically open to the Projects folder when I open them. I also like to clear the screen of any verbose info. To do this, open your ~/bashrc file or ~/bash_profile file in the text editor, and add the following command to the bottom of the file:

Code: Select all

cd Projects
clear


Now that we have our project copied, navigate over to the png folder in our Projects folder:

Code: Select all

cd ~/dc/kos/Projects/png/


If we have a look around in this directory like so:

Code: Select all

dir


we see we have a few files:

Code: Select all

example.c
Makefile
wfont.bin
romdisk_boot


First, we have our source code file, example.c When we program for the Dreamcast, we can use either C++, which are .cpp files, or we can use C, which are .c files. Next, we have our Makefile, which is where we do the actual compiling of our projects. Makefiles are scripts that we feed to a GNU program called Make to automate the process. Within our Makefiles we can set up compiler options and do more, such as calling external tools we may need during the compiling process. Finally, we have a binary file called wfont.bin that holds the graphics data for a font we will use to render text in this example. In a later lesson we'll dive further into data formats and reading binary.

We also see we have a folder, called romdisk_boot, and if we navigate into that folder:

Code: Select all

cd romdisk_boot


and look in the directory

Code: Select all

dir


we see we have two files:

Code: Select all

background.png
text.gz


The first file is a 512x512 picture encoded in png format (you can open it on your PC like any other png), and a gzip compressed file. If we open the gzip file on our PC (using a tool like winzip or winrar), we see inside that is another file called text (without any extension). If we open that file in notepad, we see it is a plain text file containing the following text:

Lincoln's Gettysburg Address, given November 19, 1863
on the battlefield near Gettysburg, Pennsylvania, USA


Four score and seven years ago, our fathers brought forth
upon this continent a new nation: conceived in liberty, and
dedicated to the proposition that all men are created equal.

Now we are engaged in a great civil war. . .testing whether
that nation, or any nation so conceived and so dedicated. . .
can long endure. We are met on a great battlefield of that war.

We have come to dedicate a portion of that field as a final resting place
for those who here gave their lives that this nation might live.
It is altogether fitting and proper that we should do this.

But, in a larger sense, we cannot dedicate. . .we cannot consecrate. . .
we cannot hallow this ground. The brave men, living and dead,
who struggled here have consecrated it, far above our poor power
to add or detract. The world will little note, nor long remember,
what we say here, but it can never forget what they did here.

It is for us the living, rather, to be dedicated here to the unfinished
work which they who fought here have thus far so nobly advanced.
It is rather for us to be here dedicated to the great task remaining
cd-rombefore us. . .that from these honored dead we take increased devotion
to that cause for which they gave the last full measure of devotion. . .
that we here highly resolve that these dead shall not have died in vain. . .
that this nation, under God, shall have a new birth of freedom. . .
and that government of the people. . .by the people. . .for the people. . .
shall not perish from this earth.


This lesson isn't going to dive into the source code for this example, as we'll eventually cover everything it does in future lessons. It's not important to understand exactly how it does everything, but for the curious, this program will use a library called zlib to decompress the text file above into plain text in memory, then load a font graphic into memory along with a png file that is loaded as a texture. It then draws the png file in the background of the program and renders our decompressed text file using our font on the screen in scrolling text - a pretty basic and classic style demo.

EXPLAINING THE KOS FILESYSTEM
Let's take a minute to talk about how the Dreamcast and KOS file system works. KOS is more than just a set of tools we use when building our programs, KOS is actually a full blown operating system for the Dreamcast, like Windows or Linux. KOS can do many things, like interacting with the Dreamcast hardware, executing program threads, manage memory, etc. It even utilizes a virtual file system with mount points. Our program's working directory is the Root of this virtual file system, "/". When we build a Dreamcast project using the KOS SDK, KOS sets up and mounts multiple directories that our programs can access. One directory maps to the VMU devices in the controllers, "/vmu/". There is also the CD-ROM drive, which is mounted as "/cd/". Files in here are located on the CD itself and must be loaded into memory by our program to be used. For example, if we were to open a text file on the CD using something like this:

Code: Select all

FILE* f;
f = fopen("/cd/example.txt", "r");


then KOS would allocate some memory for us in the Dreamcast's 16MB main memory, and then tell the GD-ROM drive to read data off the CD into that memory so we can access it locally. This has a few downsides - first, this is slow, as it takes time for the GD-ROM to be read and moved into memory. Secondly, this has the obvious downside of needing to actually load files to use them in the first place. That might not sound like something that we need to worry about, but it can be problematic. For example, when you run a program on a real Dreamcast, we have no console to output text to. If we output text to the screen on a real Dreamcast console, that means we are rendering text itself, which means we need to have a font graphic loaded in memory. But, say our command to open the font graphic from the CD was bugged - we couldn't actually output whatever error we had in the first place. For this reason, it becomes necessary to have data that is bundled with our program itself, as part of the program binary, so that, as soon as we load the program, this data is loaded with it.

This particular example we are compiling is good because it demonstrates two different ways to do this - one that applies no only to Dreamcast development but general programming, and one that is specific to the KOS SDK. The font itself is a binary graphics file called wfont.bin. We first use a utility bundled with KOS called bin2o to turn the binary graphics file into an object that can be linked into our executable. By linking the file as an object in our executable, we can access it in our program source with a global declaration like so:

Code: Select all

extern char wfont[];


extern, for those unfamiliar, is a reserved word indicating that the variable is defined elsewhere in the program. You usually see this in programs that share a variable across files (i.e. source1.c and source2.c both share a variable), but in this case rather than pointing to a later declared variable, we're pointing to an actual block of data in our binary. So long as our label matches the file name, our compiler will figure it out for us. This way, if we needed to use a font to output console information to the screen, it would already be available in memory as our program loads. This method of packing data into your executable isn't limited to KOS or the Dreamcast, and can be used in normal PC development, btw. The downside to this is that, should you want to unload that file to free memory, you're going to have to manually deallocate your memory. Further, you're going to have to essentially manually refernce every file you want packed into your binary in your source code. If you're packing multiple objects into your binary, this could get very long.

Luckily, KOS provides an automated way to manage data packed into your binary. We do this by creating a romdisk directory on our development PC. When we compile our program using our Makefile, we will first use a utility bundled with KOS called genromfs to turn our romdisk directory (in this example, ~/dc/kos/Projects/png/romdisk_boot/) into a binary image file that we can be bundled into our program executable. With a single command, KOS will automatically mount a portion of a program's executable that maps to this romdisk directory as "/rd/". If we wanted to later free up memory, we can unmount the "/rd/" folder through KOS. The only thing we need to do is make sure KOS can point to the romdisk folder correctly, which we do in this example with the following code in our source:

Code: Select all

extern uint8 romdisk_boot[];
KOS_INIT_ROMDISK(romdisk_boot);


Just make sure the label matches the file name for our romdisk object that we generated with genromfs. You can actually use romdisks to load multiple chunks of files by creating multiple romdisk objects packed into your executable. This can make for an easy way to load in and out data in your program; i.e. packing all of "level 1" assets into "level_one_romdisk", all of "level 2" into "level_two_romdisk", etc.

We can also mount an SD Card adapter through KOS, although it's more involved and not automatic. We will cover mounting an SD Card in a later lesson. SD Cards are read through a homebrew adapter that connects to the serial port of the Dreamcast. We can launch an ELF from the SD Card as though it were launching from the GD-Rom Drive, but in doing so we lose the ability to write to the SD Card adapter. However, if we boot from a CD-ROM, we can access the SD Card in much the same way you'd access as USB drive - both with the ability to read and write. This can be helpful for logging or swapping out assets without rebuilding your entire program.

STEP 2: CONFIGURING OUR MAKEFILE
Open the Makefile in our png folder (~/dc/kos/Projects/png/) with our text editor. You'll note that Makefiles don't have a file extension, but they are still plain text files. Makefiles will vary from project to project, but the Makefile in this example is pretty general and can be used as template going forward. Makefiles are scripts that control the way our make commands work.

The first line of the Makefile is

Code: Select all

all: rm-elf example.elf


This is basically the default action when running make. You can actually make individual parts of the Makefile, but this opening part labeled “all” is what gets run if you run make by itself. In this case, it runs a macro called “rm-elf” then another macro called “example.elf”

Code: Select all

include $(KOS_BASE)/Makefile.rules


This line tells our Makefile to include a general set of rules. You'll note we use a symbol, $(KOS_BASE). This is a label we declared when we ran our environ.sh script. If you go into environ.sh, see exactly what $(KOS_BASE) maps to, in this case, the location of our KOS folder (~/dc/kos).

Code: Select all

OBJS = example.o wfont.o


This line declares a variable string called OBJS that we can use when sending commands. Anytime we use OBJS in our commands, it will be replaced with the text to the right of the equals sign. To use this variable in a command, we use a dollar sign then call the name of our variable in parenthesis, like this: $(OBJ).

Code: Select all

KOS_LOCAL_CFLAGS = -I$(KOS_BASE)/addons/zlib


This is another variable string. This one is for flags we want to use later on. It uses the $(KOS_BASE) symbol, then goes to the /addons/zlib/ folder in KOS.

Code: Select all

clean:
 -rm -f example.elf $(OBJS)
 -rm -f romdisk_boot.*


This is another macro defined in the Makefile, called “clean:”. To call this macro, when we launch make from our terminal, we add the label as a flag, like so:

Code: Select all

make clean


This macro sends two commands to the console. The first removes (deletes) files, namely example.elf, and whatever we have defined in $(OBJS). The “-f” flag after rm is for “force,” it deletes the files without complaining, whether they exist or not. It also deletes any files named “romdisk_boot” regardless of extension. “*” is a wild card symbol, it means “all” or “any.” You can use “*” on both sides of the file name – before the period and after. “*.o” would mean “any object file,” “object.*” would mean “any file named “object” of any type, and “*.*” would mean “any file, period.”

This means that the clean command is deleting any files we might generate during a compile. In this case, the following files are deleted: example.elf, example.o, wfont.o, and any files named romdisk_boot. That is the general purpose of any clean command, and it's generally a good idea to have one that deletes all files you produce in compile.

Code: Select all

dist:
 -rm -f $(OBJS)
 -rm -f romdisk_boot.*
 $(KOS_STRIP) example.elf


This is a macro to prep a built project for distribution. You see it deletes the object files we produce during compile (example.o and wfont.o), and it deletes any files named “romdisk_boot” regardless of extension.

This command then calls $(KOS_STRIP). KOS_STRIP is defined in environ_base.sh, which is called in environ.sh. It is a link to a program in Cygwin's bin folder called sh-elf-strip, which is located in opt/toolchains/dc/sh-elf/bin/ sh-elf-strip is a program that removes symbols from files, which we do to our elf when we want to distribute it.

Code: Select all

rm-elf:
 -rm -f example.elf
 -rm -f romdisk_boot.*


this is a macro to manually remove the elf file we generate from compile, along with all our romdisk_boot files regardless of extension.

Code: Select all

example.elf: $(OBJS) romdisk_boot.o 
 $(KOS_CC) $(KOS_CFLAGS) $(KOS_LDFLAGS) -o $@ $(KOS_START) $^ -lpng -lz -lm $(KOS_LIBS)


This is a macro to create example.elf. To the right of the macro name, we have two per-requisites that make will check. First, all the objects we defined as $(OBJ) must exist, and also a file named romdisk_boot.o must exist.

This is our Linker function. The actual command begins with $(KOS_CC). KOC_CC is defined in environ.sh, it is a link to our compiler, at “/opt/toolchains/dc/sh-elf/bin/sh-elf-gcc”. Sh-elf-gcc is a special variant of gcc that generates elfs for the sh family of processor by linking object files. The Dreamcast uses an sh4 processor.

Before this macro will run, the prerequisite files must exist. If the files do not exist, they will check our environ.sh, environ_base.sh, and any files “included” with the Makefile, along with the rest of the Makefile itself for instructions on how to create them. Let's examine our prerequisites and see how they are generated before examining how our linker works.

The first prerequisite is $(OBJS). That is a string meaning example.o and wfont.o. So when we call $(OBJS) it equates to two separate prerequisites. Example.o, as you might recall, is the compiled object file. Since the name of the source is Example.c, it makes sense to call it's compiled object output Example.o. Example.o doesn't exist yet, and the rest of the Makefile doesn't define how to create it. Neither does environ.sh or environ_base.sh. But we included $(KOS_BASE)/Makefile.rules, and in there we define how to create a .o file:

Code: Select all

%.o: %.c
 kos-cc $(CFLAGS) -c $< -o $@


“%” in this case is a wild card input, kind of like “*”. In this case, there is no specific instruction for “Example.o” but there is a general rule for any object file. You see this macro has a prerequisite itself, %.c has to exist, in this case “Example.c”. In this case, that file does exist in our folder, so we pass the prerequisite.

It calls kos-cc from our KOS_BASE folder (because that's where Makefile.rules is), which is our KOS C compiler. We feed our C Compiler $(CFLAGS) which is set automatically depending on our version of KOS. The -c flag tells the compiler to take in a source file. The -c flag takes in a parameter, which pass as $<. $< is a symbol that refers to our prerequisites themselves. $< means the first prerequisite on the list, which is “%.c” which is also “Example.c”.

The -o flag tells the compiler we want to output a file. The -o flag takes in a parameter which we pass as $@. $@ is a symbol that refers to the name of the macro itself. Since our macro is called “%.o” this means we pass the compiler an output of “%.o” which, in this case, means “example.o”. In short, we are telling the compiler to take in Example.c and output Example.o.

Going back to our Linker, the second prerequisite from $(OBJS) is wfont.o. Again, this file doesn't exist, but the next defined macro in our Makefile tells us how to make wfont.o:

Code: Select all

wfont.o: wfont.bin
 $(KOS_BASE)/utils/bin2o/bin2o $< wfont $@


In this case, wfont.bin needs to exist first. You may remember that we said wfont.bin was the graphic for our font in binary form. In this case, rather than calling our KOS-CC compiler, we call a program in ~/dc/kos/utils/bin2o called bin2o. This program rather obviously converts a binary file into an object file suitable for linking. Again, we feed the first prerequisite as an input, recognized as the symbol $<. Object files contain a label, which we set by passing the “wfont” flag. This labels our object file as “wfont.” The output file is passed as the macro name recognized by the symbol $@. In short, this runs bin2o and takes in wfont.bin and outputs a object labeled “wfont” as wfont.o.

Back at our Linker, the final prerequisite is romdisk_boot.o, which also does not exist. But later in the Makefile, we define it as:

Code: Select all

romdisk_boot.o: romdisk_boot.img
 $(KOS_BASE)/utils/bin2o/bin2o $< romdisk_boot $@


You see that we first need romdisk_boot.img to exist. We create that in the Makefile here:

Code: Select all

romdisk_boot.img:
 $(KOS_GENROMFS) -f $@ -d romdisk_boot -v


$(KOS_GENROMFS) is a link to a program called genromfs located at “~/dc/kos/utils/genromfs/”. By passing -f to genromfs we force it to generate the file without complaining, even if it already exists. We use the macro name, romdisk_boot.img as our output recognized by $@, then tell genromfs we are passing a directory by passing “-d”. “-d” takes a parameter which we pass as the string “romdisk_boot”. The -v flag tells it to be verbose, so we can see the generation of the folder. What this does is take our romdisk_boot folder and turns it into a binary image of the folder, called romdisk.img.

With romdisk_boot.img created, we can continue creating romdisk_boot.o. This time it continues just like we did with wfont. We take romdisk.img as the input ($<), label it romdisk_boot, then name the output file romdisk.o ($@)

With Example.o, wfont.o, and romdisk_boot.o all created, we can finally continue with our Linker call. Recall that our linker called a program named sh-elf-gcc. When calling sh-elf-gcc, we use $(KOS_CFLAGS), which is defined in environ.sh. These are general settings we want to use for gcc. According to environ_base.sh, $(KOS_LDFLAGS) actually isn't set. Whether or not it's set depends on which version of the kos-cross compiler we are using. we don't really need to worry about it. It disables stdlib when set, which we don't necessarily want.

The next part of the command, “-o” tells gcc that we want to output a file. The -o flag takes a name as a parameter, which we pass $@. Our macro is example.elf, so the name we passed -o is “example.elf”

$(KOS_START) is a macro to call a startup script. Whether or not it's set depends on which version of the kos-cross compiler we are using. we don't really need to worry about it.

$^ refers to our prerequisites, which we are feeding to gcc to generate our output file. Though we don't use it in this example, $< would refer to the first prerequisite we listed, $(OBJ), while $> would refer to the last, romdisk_boot.o. $^ refers to all the prerequisites, so we wind up passing gcc $(OBJ) and romdisk_boot.o, which equates to example.o, wfont.o, and romdisk_boot.o.

Finally, we pass our Linker library files that we need. When compiling in unix, library files usually begin with -l, for example the png library is -lpng. In windows, typically library files are .lib files, for example png.lib may exist. Because we are using Cygwin in windows, we adhere to UNIX style rules. In this case, we pass 4 libraries: -lpng, -lz, and -lm, and $(KOS_LIBS). $(KOS_LIBS) are all the libraries we need to compile KOS projects.

Thus, in all, our Linker calls sh-elf-gcc and links example.o, wfont.o, romdisk_boot.o, and uses libraries -lpng, -lz, and -lw, to output example.elf.

The final line in the Makefile is a macro called run. This should compile our project then attempt to run it through a special serial cable you can build for your Dreamcast. We won't be using this style of running programs in our tutorial, but this is a valid avenue for deploying ELFs. To use this, we would call

Code: Select all

make run


STEP 3: BUILDING THE PROJECT
Now that we understand our Makefile, it's finally time to build our project. Open a terminal and go to the png folder. From within this folder, type:

Code: Select all

make


This will call make using the Makefile. Before we called make, we should have had the following files in the directory:

Code: Select all

example.c
Makefile
romdisk_boot (folder)
wfont.bin


After running make, we should have the following:

Code: Select all

example.c
example.elf
example.o
Makefile
romdisk_boot (folder)
romdisk_boot.img
romdisk_boot.o
wfont.bin
wfont.o


If we were to run:

Code: Select all

make clean


we would go back to our first file list. Make creates those files, and make clean deletes them. Now that we have generated our elf file, let's turn it into a binary:

Code: Select all

sh-elf-objcopy -R .stack -O binary example.elf output.bin


this calls a program called sh-elf-objcopy. Objcopy is a program available on many platforms that lets you copy objects into a binary. In this case, we are turning our elf into a binary called output.bin. After we have our output.bin, we need to scramble it. If you are on Linux, use the following command:

Code: Select all

../../utils/scramble/scramble output.bin 1st_read.bin


or, if you are windows, you run this command:

Code: Select all

 scramble output.bin 1st_read.bin


This will produce a binary file called 1st_read.bin. Now, we need to grab an appropriate IP.bin. I have included one in the file pack in the OP, and likewise you can grab it individually right here: [IP.bin link]

From here, if you are running windows, you can use a program called Bootdreams. I have included Bootdreams in the filepack in the opening post, and also individually here: [Bootdreams link]

Boot dreams makes it easy to turn your 1st_read.bin and IP.bin into a bootable Dreamcast disk with a graphical interface. Open it in windows:

Image

Click on Browse to open a file browser. Navigate to your png folder in Cygwin:

Image

This folder we select will be turned into the “/cd/” folder from the perspective of our program. Bootdreams will find 1st_read.bin and IP.bin in this folder and assign them to the appropriate spot on the cd accordingly. Make sure you have diskjuggler selected at the top of bootdreams and give your program a label in the label section. Keep your disk format as Audio/disk, then click process. It will ask you if you want to create a diskjuggler image, click yes. If, for some reason, it cannot find 1st_read.bin, it will ask you what binary file you want to associate with that region of the disk. Also, if it doesn't find IP.bin, it was ask if you want it to create it. With all that said and done, save your file as “example.cdi” file.

Next, burn your .cdi file with the special version of IMGBurn you installed on your PC prior. To do this, open IMG Burn:

Image

Click “write image file to disc” and next to “Please select a file” click the folder button. Navigate to your example.cdi file, insert a blank CD-ROM into your CD burner, then click the burn button. Sit back and wait for it to burn.

If you aren't on windows, or don't want to use these GUI programs, you can burn your disc using the following commands instead.

Code: Select all

mkisofs -G IP.bin -o session1.iso ~/dc/kos/Projects/png/


This uses mkisofs to pack IP.bin and an entire folder, “~/dc/kos/Projects/png/” into an output file called session1.iso. After we create session1.iso, use the following commands to burn the disc:

Code: Select all

dd if=session1.iso bs=1024 count=36 > session2.iso
cdrecord dev=1,0,0 speed=8 -multi -xa1 session1.iso
cdrecord dev=1,0,0 speed=8 -eject -xa1 session2.iso


This creates a second iso that will pad our CD, then burns our session1.iso on session 1, then burns session2.iso on the second session.


STEP 4: RUNNING OUR PROGRAM
With this burned disc, we can place it in our Dreamcast and it will boot:

Image

Otherwise, we can load either the cdi image we create with boot dreams, or the sesssion1.iso we created otherwise in NullDC. To do this, open NullDC and click “File → Normal Boot” then select your CDI image and click ok. Assuming your emulator is set up correctly, you will boot the program as intended.
User avatar
Krejlooc
128-bit
Posts: 525
Joined: Mon Feb 27, 2012 4:27 pm

Re: Beginning Game Development - A Dreamcast Dev Guide

Post by Krejlooc »

LESSON 3: SOFTWARE RENDERING

TABLE OF CONTENTS
-SOFTWARE RENDERING VS HARDWARE RENDERING
-PIXEL FORMATS
-REFRESHER ON BIT MANIPULATION
-STEP 1: PREPROCESSOR DEFINITIONS
-STEP 2: SIMPLE MAIN ROUNTINE

SOFTWARE RENDERING VS HARDWARE RENDERING
Now that we understand how to build a project, we can embark on writing our very first Dreamcast program. As mentioned before, you need to tap the Dreamcast in a specific way to make it perform well. For our first program, we will intentionally not tap the Dreamcast that way. We will instead do things as inefficiently as possible. I have chosen to start off by showing the wrong method first, because the right methods build upon these fundamentals.

What we will be doing is called software rendering. If you've played PC games like Quake, you might be familiar with the term. Software rendering means what we will be using our CPU exclusively to draw our scene, rather than using the video hardware inside the Dreamcast to speed things up.

our code will measure our frame rate, so we can observe how much of a speed increase we get from writing code that utilizes the Dreamcast hardware better. The Dreamcast hardware is interesting compared to modern systems. In a modern system, the GPU is essentially a tiny media-oriented computer within your larger computer. Modern GPUs are pretty amazing, they not only interface with the video device we are outputting to, but they also are capable of running tiny programs, called shaders, that operate on every pixel or vertex being drawn to the screen. They are also terrific at crunching math. Today all 3D math is handled by your GPU in massively parallel fashion.

The Dreamcast is way different. Technically, it doesn't have a GPU, it has a graphics core which a part of another chip called Holly. The graphics core of the Holly chip is a Power VR2 core. The Holly chip is sort of a gate keeper that handles input and output for the Dreamcast, and communicates to the SH4 CPU. The holly chip handles polling from the controllers, for example. The PVR core has it's own quirks that make it better at handling certain tasks, which we'll get into in a later chapter. But one task the PVR core and Holly chip do not handle is 3D math – there is no logical hardware in the PVR core to handle mathematical calculations. All 3D transformations on the Dreamcast are done by the CPU!

Image

There is a bus between the CPU and Main ram to the Holly chip, and separate busses between the graphics core and the VRAM. This means writing to VRAM from the CPU is very slow – When the CPU accesses VRAM, it must first send the data to the Holly chip, which is then sent to the PVR Core, which then gets sent to the VRAM. By constrast, access from the Holly chip to the VRAM is fast – the VRAM of the Dreamcast is split into two contiguous memory blocks, each made up of two 16MBit SDRAM banks that are also contiguous. Holly can access these banks either 64-bits at a time, 32-bits at a time, or 16-bits at a time. This is because each bank actually has it's own bus to the PVR Core.

This will all get explored in more depth later on. For now, the main thing to know is that every time you access your VRAM directly through the CPU, it has to travel a long distance, and no matter how small your operation is, it will be padded out to 16-bits.

What we will be doing with our program is directly accessing a part of VRAM called the frame buffer from our CPU, for every single pixel we draw, for every frame we draw. Again, this is purposefully inefficient. Your CPU should typically be used for other things instead of plotting pixels in VRAM. But we can learn a variety of concepts this way, and it makes for a good starting point.

PIXEL FORMATS
Computers work in binary, which, taken on it's own, is meaningless. As an example, try decipher my intended meaning behind this string of binary:

Code: Select all

0000-1110 1010-0100


You could read this as a straight single digit, in which case it would be 3,815, but that wasn't my intended meaning. Rather, that string is supposed to mean: 7, 5, 4. To get that reading of the string, I separated out the string into segments, and read each segment separately like so:

Image

in this case, reading from left to right, the first four bits are ignored, then the next three bits of the string are taken in as one digit – 111, which is binary for 7. The eighth bit is ignored. The ninth, tenth, and eleventh bits are next read as one digit, 101, which is binary for 5. The twelfth and thirteenth bits are ignored, then the fourteenth, fifteenth, and sixteenth bits are read as 100, which is binary for four.

The rule that I used to read the string correctly is called a format. Formats help us make sense of data that is otherwise meaningless. A popular format is ASCII, the American Standard Code for Information Interchange, which maps arbitrary digits 0-255 to English characters.

You saw above how we had wasted space in our data string – 7 different bits were entirely unused. We are bound by the size limits of bytes as the lowest level we can access data (typically, there is no single bit type). Remember, 1 byte is 8 bits. Since I intended 9 bits to be interpreted, I need at least 2 bytes that allocate 16 bits total.

When you set up a program for the Dreamcast, you need to select from a few different pixel formats that the Dreamcast can draw with. Modern systems typically output in what is known as True-color format 24-bit color, i.e. 8 bits dedicated to Red, blue, and green sub-channels individually, giving each color channel 255 different values (8 bits = max value of 255). This yields a total of 16,777,216 colors. There is also sometimes 32-bit color, which adds an 8-bit alpha channel that controls transparency, although the end color selected is still one of those 24-bit colors in the end.

The ideal pixel format for drawing to the screen 16-bits long, although it offers a 24-bit pixel mode. These pixel formats define the type of image the Dreamcast itself outputs to the television. The Dreamcast offers color modes in a variety of formats:

RGB565 mode
For a variety of reasons, this is the fastest drawing mode for the Dreamcast. It devotes 5 bits to the red sub-channel, 6 bits to the green sub-channel, and 5 bits to the blue sub-channel. The reason for this division is because the human eye is slightly more perceptible to the color green, allowing us to see greater variations of that color than others. Hence, there are 32 values of red and blue available, and 64 values of green, for a result of 65,536 maximum colors. Each pixel is 2 bytes big.

RGB555 mode
A 15-bit derivative of RGB565 mode. This discards a bit from the green index. Each pixel is 2 bytes big.

RGB888 mode
This is the much slower 24-bit pixel mode. Though it offers better color selection precision, each pixel is 3 bytes big, making drawing to the screen much slower.

Additionally, internally, the Dreamcast offers texture formats. Textures are what we refer to when an image resides in VRAM. These texture formats describe the way the texture pixels reside in VRAM internally. The native Dreamcast internal pixel format is 32-bit color, which is converted to 16-bit color with dithering when drawn if not using the 24-bit color mode. The Dreamcast offers the following texture formats:

ARGB8888 mode
This is the native Dreamcast internal pixel format. It is 8 bits per channel, for a maximum of 2,147,483,647 colors. For the most part, this is the texture pixel format you will want to use when working with the Dreamcast.

ARGB4444 mode
ARGB4444 mode derives it's resultant output from ARGB8888. In this mode, 4 bits are devoted to an alpha channel, with the remaining 12 bits going to RGB respectively. This yields only 16 values for R, G, and B each, for a result of 4,906 colors, that can be modified by the alpha channel (and what they blend with) to ultimately output a possible 65,536 max colors.

YUV442 mode
An entirely different method of representing colors is YUV mode. Rather than describing a color by the amount of red, blue, and green used to produce the color, YUV describes color in terms of a luma or brightness index (Y) applied to an array of U-V color planes. It is, in essence, a three dimensional representation of color, working like this:

Image

This mode devotes 4 bits to the luma channel, 4 bits to the U-axis (x-direction), and 2-bits given to the V-axis (y-direction). This yields 4^4^2 possible color positions for a max value of 65,536 colors.

4-BIT quantized mode
The Sega Genesis pixel format. The Dreamcast actually has a 4-bit pixel format that works with a palette. 4-bits per pixel yields a maximum possibility of 16 index positions. What 16 colors those index positions refer to depends on how the Dreamcast palette is set up. The Dreamcast will set aside 1024 palette entries for colors. Because 4-bits only allows for 16 colors to be selected at once, those 1024 colors are split up into 64 different palettes that we can draw with (1024/16=64). Each of those 1024 color entries can be any color. The main upside of this mode is that the size of your image in VRAM becomes extremely tiny, needing only 4-bits per pixel to be represented. That means that each byte represents two pixels. Contrast to the other modes, where 1 pixel was represented by two bytes each.

8-BIT quantized mode
Much like the 4-bit quantized mode, the 8-bit quantized mode works with a palette. But instead of using 4-bits per pixel for a max of 16 color index positions, this mode dedicated 8-bits per pixel, yielding a max o 256 color index positions. Because each palette can now have 256 entries, this means that our 1024 color entries is divided up into 4 palettes instead of 64 (1024/256=4). Like 4-bit quantized mode, each palette color can be any color. The main advantage of this mode is a smaller VRAM footprint. 1 byte represents 1 pixel in this mode.


REFRESHER ON BIT MANIPULATION
When working with retro consoles, doing things efficiently becomes a must, which often means you need to waste as little space as possible as often as possible.

Say you have two variables in your game, like number of bombs and number of arrows. But say these numbers have a limit you want to impose – nobody should be able to carry more than 15 bombs, and nobody should be able to carry 15 arrows. If you were to allocate each of these variables as an integer, you would be allocating 64 bits for these two variables (32-bits each). Considering you can represent every digit between 0 and 15 in only 4 bits, that means each variable is wasting 28 bits on meaningless data! That also means to pass these variables to functions, you must send 28 wasted bits of data. We only need 8 bits total, 1 byte. You are using 8 bytes to store what could be stored in 1 byte.

While we are bound to the size of bytes when allocating memory, we are not bound to the size of bytes when interpreting data. In our example from before, recall that we only considered 3 bits per value. In order to read individual bits in a byte, you must use bit manipulation.

Bit manipulation works on the principles of Boolean Algebra. The concept is to apply a mask using logical operations to trap the values we want to examine in a string of bits.

Explaining Boolean algebra would be literally an entire course on its own, but a quick crash course for those looking to dive into Dreamcast programming – Boolean algebra is the study of logical truths. The area of Boolean algebra that we are concerned with is the definition of a few concepts: AND and OR primarily.

Boolean algebra explains, logically, how these concepts work. Given any two binary inputs, Boolean algebra explains a logical output of a specific operation, be it AND or something else. Every permutation of each input is calculated for this operation and displayed in what is called a truth table.

For example, this is the truth table for the operation AND:

Code: Select all

 AND:
input1 input2 output
0    0     0
1    0     0
0    1     0
1    1     1


It is perhaps easier to understand what boolean algebra is explaining by using a real world example. Say we have a door in our game, and it takes both a blue key AND a red key to open. Boolean algebra explains the behavior of the door for every possible situation. That is to say, if we have no blue keys, and no red keys, then we can't enter the door. Similarly, if we have 1 blue key, and no red keys, we can't enter the door either. If we have no blue keys, and 1 red key, we also cannot enter the door. But, if we have 1 blue key, and 1 red key, we can enter the door.

Likewise, this is the truth table for the operation OR:

Code: Select all

 OR:
input1 input2 output
0    0     0
1    0     1
0    1     1
1    1     1


Applying it to our door analogy, in this case the door opens if we have either a blue key OR a red key. So if we have no blue key or red keys, it won't open. But if we have 1 blue key, and no red keys, it opens. And if we have no blue keys, and 1 red key, it opens. And if we have 1 blue key, and 1 red key, it also opens.

There are logical operators to explain a variety of concepts. Other logical operators you might run into include XOR (exclusively OR) and NOT.

In C style programming, the logical operator AND is represented with &, and the logical operator for OR is represented with |. When evoked in code, it will return a comparison between two given strings of bits.

To give an example, say I define 2 variables 8 bits long that equates to this:

Code: Select all

Variable1: 11001100
Variable2: 11000011


if I used a logical operator AND on them, like so:

Code: Select all

(Variable1 & Variable2)


that would equate to:

Code: Select all

 11001100
&11000011
---------
 11000000


reading from left to right, we see that the first bit of variable 1 is a 1, and the first bit of variable 1 is a 1. 1 & 1 = 1. We see the same for the second bits of each variable. The third and fourth bits are both 0's, and 0 & 0 = 0. The fifth and sixth bit of variable 1 is a 1, and the fifth and sixth bit of variable 2 is a 0. 1 & 0 = 0. The inverse is true for the seventh and eight bits.

By creating a variable that sets the correct bits that we want to read, known as a mask, and comparing it to data we want to read using a logical operator, we can isolate individual bits.

When creating these masks, it is often beneficial to work in Hexadecimal. Hexadecimal is base 16, which means that instead of being able to represent 10 digits (0-9) with 1 character as we can in base 10, you can represent 16 digits with 1 character. The digits beyond 9 are represented with the characters A-F. When we use hexadecimal, we denote this in a few ways. Typically, hexadecimal is represented with either a $ symbol preceding the digits, or the 0x symbol preceding the digits. That means 0x10 and $10 should not be read as the digit 10 (as in coming after 9) but rather the hexadecimal digit 10, I.E. the digit coming after 15, or 16 in decimal.

Hexadecimal is useful when working with bits because it naturally breaks bytes up into half-bytes (4-bits). This is because the numbers 0-F take 4-bits to be represented. That means that 1 byte in hexadecimal holds a maximum value of 0xFF.

Each Hexadecimal digit maps out to a specific 4-bit binary pattern, as follows:

Code: Select all

HEX   BINARY
0   0000
1   0001
2   0010
3   0011
4   0100
5   0101
6   0110
7   0111
8   1000
9   1001
A   1010
B   1011
C   1100
D   1101
E   1110
F   1111


So, let's take a Dreamcast pixel in RGB565 format like so:

Code: Select all

1100-1110 1101-1011


Let's first try and read the red sub-channel of this pixel. Remember, the red sub-channel is the first five bits. That means we need to examine the first byte of the 2 bytes needed. We want to create create a mask that is also 1 byte big, that has the first five bits set to 1, and the other bits set to 0. We work 4 bits at a time. Using the pattern chart above, we see the first character we need is 0xF:

Code: Select all

HEX   BINARY
0xF   1111


Next we set second half of the byte so that the first bit is set to 1, and the other 3 are set to 0. In the pattern table above, that maps to 0x8:

Code: Select all

HEX   BINARY
0xF8   1111-1000


Thus, our mask is 0xF8. Say we retrieved the pixel like so:

Code: Select all

Uint8 Pixel[2] = {code to get pixel}; //two bytes to store our pixel


Then we could isolate the red sub-channel of the pixel like so:

Code: Select all

Uint8 MASK = 0xF8;      //our mask to isolate the Red sub-channel
Uint8 Red = (Pixel[0] & MASK);   //new byte holding only Red sub-channel


You see in the second line we allocate a new byte that holds the product of (Pixel[0] & MASK). Since our first byte, stored at Pixel[0] was:

Code: Select all

1100-1110


and our MASK was:

Code: Select all

1111-1000


Then Red holds the following:

Code: Select all

 1100-1110
&1111-1000
----------
 1100-1000


Now only the values of the Red sub-channel are shown. But there is a problem. Our Red sub-channel should be 11001 which equates to decimal 25. But we read in bytes, and the 0's at the end are still read. This means our value is read as 11001000 which is decimal 200. To fix this, we use bit-shifting. Bit-shifting lets us push bits over either left or right, discarding bits that are not needed. Bit-shifting to the left is done with the operator << and bit-shifting right is done with the operator >>. Since there are 3 bits we need to discard to the right, we can append our previous command like so:

Code: Select all

Uint8 MASK = 0xF8;      //our mask to isolate the Red sub-channel
Uint8 Red = (Pixel[0] & MASK) >> 3; //new byte holding only Red sub-channel


This means that the result of (Pixel[0] & MASK) is still 1100-1000, but that is then shifted over to the right 3 times like so:

Code: Select all

SHIFT      RESULT
0      1100-1000
1      0110-0100
2      0011-0010
3      0001-1001


Now when we read the value, it equates correctly to decimal 25. Continuing on, we could read the green sub-channel by creating two masks for each byte. First, we need a mask that reads only the last 3 bits of the first byte, like so:

Code: Select all

/*HEX      BINARY
0x07      0000-0111*/

Uint8 Mask1 = 0x07;
Uint8 Green = (Pixel[0] & Mask1);


Next, we need to move those 3 bits all the over to the left so they comprise the first three bits of 6 bits total using bit-shifting, like so:

Code: Select all

Green = Green << 3;

/*SHIFT      RESULT
0         0000-0110
1         0000-1100
2         0001-1000
3         0011-0000*/


Then you could create a second mask for the second byte, that read only the first 3-bits, then shifted over 5 bits so they are the last 3 bits of the byte like so:

Code: Select all

/*
HEX   BINARY
0xE0   1110-0000*/

Uint8 Mask2 = 0xE0;            //our second mask
Green += (Pixel[1] & Mask2) >> 5;      //result shifted right 5 bits

/*SHIFT      RESULT
0         1100-0000
1         0110-0000
2         0011-0000
3         0001-1000
4         0000-1100
5         0000-0110*/


You see we added this to Green. This works like so:

Code: Select all

 0011-0000
+0000-0110
----------
 0011-0110


This maps to decimal 54. With both the red and green sub-channels isolated, we only need to find the blue sub-channel, which is the last 5 bits of the second byte. We grab that like so:

Code: Select all

/*
HEX   BINARY
0x1F   0001-1111*/

Uint8 Mask3 = 0x1F;            //our second mask
Uint8 Blue += (Pixel[1] & Mask3);      //result shifted right 5 bits

/*SHIFT      RESULT
0         0001-1011*/


This maps to decimal 27. Thus, our two byte binary string: 1100-1110 1101-1011
maps to

Code: Select all

RED: 25
GREEN: 54
BLUE: 27


Make sure you understand these operations well, as we'll use bit-masking and bit-shifting quite a bit going forward.

STEP 1: PREPROCESSOR DEFINITIONS
With all that finally said, let's start a new project. Open your terminal and make a new folder for our project then navigate to it:

Code: Select all

mkdir Lesson3
cd Lesson3


Open our text editor, we are going to create a Makefile. Configure your Makefile so that it reads like so:

Code: Select all

all: rm-elf main.elf

include $(KOS_BASE)/Makefile.rules

OBJS = main.o
   
clean:
   -rm -f main.elf $(OBJS)
   
clean-all:
   -rm -f main.elf $(OBJS) main.iso output.bin Program.cdi 1st_read.bin

dist:
   -rm -f $(OBJS)
   $(KOS_STRIP) main.elf
   
rm-elf:
   -rm -f main.elf

main.elf: $(OBJS)
   $(KOS_CC) $(KOS_CFLAGS) $(KOS_LDFLAGS) -o $@ $(KOS_START) $^ -lm $(KOS_LIBS)


save this file in our Lesson3 folder as Makefile. Make sure you specify that the “save as” type is “All (*.*)” and that you don't add an extension to the file name.

Now create a new file in your text editor. Save this empty file as Main.cpp in your Lesson3 folder.

Lets begin by giving some Preprocessor commands. Preprocessor commands are denoted by # symbols, and are commands intended for the compiler. We will begin by #defineing a macro:

Code: Select all

#define PACK_PIXEL(r, g, b) ( ((r & 0xF8) << 8) | ((g & 0xFC) << 3) | (b >> 3) )


This is a nice little macro that takes 3 bytes (24-bits) and automatically formats them into the Dreamcast's 16-bit packed pixel format. Anytime we use the command “PACK_PIXEL(r, g, b)” the compiler will replace that code with the code to the right of the parenthesis, with the r, g, and b variables replaced by the digits we put in their place. The command on the right looks complex, but it's just a bunch of masking and bit-shifting to place the values in the correct locations in our packed pixel. Very handy shortcut.

Next, we will include some external files necessary for our program.

Code: Select all

#include <kos.h>
#include <stdio.h>


The first file is kos.h, which is necessary to access KallistiOS. The second is the standard input/output library that we use for console output.

STEP 2: SIMPLE MAIN ROUNTINE

Next, start our main routine like so:

Code: Select all

int main(void) {
 int quit = 0;
 int i = 0; int j = 0;
 
 //init kos
 pvr_init_defaults();
 
 //set our video mode
 vid_set_mode(DM_320x240, PM_RGB565);


You can see we define a few variables, i and j will be used in a nested for loop in a bit, and quit is a variable we use to keep our program running.

The first bit of interesting code is pvr_init_defaults();. This sets up the pvr core with a bunch of default settings. Next, we define our video mode with vid_set_mode(DM_320x240, PM_RGB565);. You can see the function takes two parameters. The first is the Display Mode, which can be selected from the following:

Code: Select all

DM_320x240         320 x 240, 60Hz (or VGA)
DM_640x480         640 x 480, 60Hz (or VGA)
DM_800x608         800 x 608, 60Hz (or VGA)
DM_256x256         256 x 256, 60Hz (or VGA)
DM_768x480         768 x 480, 60Hz (or VGA)
DM_768x576         768 x 576, 60Hz (or VGA)
DM_640x480_PAL_IL   640 x 480, 50Hz
DM_256x256_PAL_IL   256 x 256, 50Hz
DM_768x480_PAL_IL   768 x 480, 50Hz
DM_768x576_PAL_IL   768 x 576, 50Hz


The second parameter selects the Pixel Mode:

Code: Select all

PM_RGB555   15-bit (xRRRRRGGGGGBBBBB)
PM_RGB565   16-bit (RRRRRGGGGGGBBBBB)
PM_RGB888   24-bit (RRRRRRRR GGGGGGGG BBBBBBBB)


We're gonna work with 320x240 mode and RGB565. Next, let's define a loop that will draw to the screen:

Code: Select all

 /* keep drawing frames */
 while(!quit) {
   for(i = 0; i< 10; i++)
   {
      for(j = 0; j <10; j++)
      {
          vram_s[i + (j * 320)] = PACK_PIXEL(0, 0, 255);      
      }
   }
}


We have defined a nested for loop that goes 10 x 10, and we use those numbers, represented by i and j, to offset where we draw. You'll see inside the nested loops we use an array pointer called vram_s. This is automatically defined by KOS as a shortcut to the location of the framebuffer in VRAM. The framebuffer is stored as a 1-D array – recall that in C and C++, 2D arrays are not arrays of arrays, but rather a contiguous 1D array.

Thus, while you may be expecting the screen to be represented by this:

Image

it is actually represented by this:

Image

Which is to say that each pixel of the screen is stored sequentially in the array. When you reach the right edge of the screen denoted by n, the left edge of the screen 1 row of pixels down is n+1. We can access individual pixels using (X, Y) notation using the following formula:

Code: Select all

x + (y * width)


where width refers to the x resolution of our 2D array, in this case 320. Hence, when we plot pixels to the frame buffer, using our nested for loops as X and Y, we arrive at:

Code: Select all

vram_s[i + (j * 320)]

You see we assign each pixel we access in the frame buffer a value that we create using our preprocessor macro. In this case, we are setting the red value to 0, the green value to 0, and the blue value to 255 (max).

Finally, we can finish our main routine by returning a value:

Code: Select all

 return 0;
}


Go ahead and save this main.cpp file and build it. Make sure you include the bundled IP.bin file in your Lesson3 directory when building the project, or else your disc will not boot.

If everything has gone correctly, you should be greeted with this:

Image
Post Reply