Difference between revisions of "A look inside The Link @ wiki:About"

Revision as of 22:44, 12 February 2008

Some info is needed about the project itself. This page reports a bit of history and lists people who contributed to make the project possible.

Why such an effort?

For fun. The so-called reverse engineering can turn huge amounts of brain work into huge amounts of satisfaction (in a way similar to science, if you like). You start with just a big file, knowing almost nothing about it, and eventually you have an almost complete description of its structure, coming just from observation and reasoning. Like science, it's a challenge: a way of proving the power of our mind. When I completely decoded SFXE resources, which were originally unknown in both format and purpose, people thought I had gone mad. I was running everywhere and telling everyone "Hey, I've decoded SFXEs!".

Another reason is allowing people to extend the Riven experience to other platforms and operating systems. At the beginning I was far from seeing this as a serious reason, it was more like a dream; but as fields and structures were decoded, I started wondering if someone had noticed my efforts and exploited them for porting Riven to other platforms. Eventually, it happened with two projects: Fissure and Riven X.

How was it performed?

Hmm, quite a difficult question. First of all, you have to be extremely calm and relaxed...

Seriously, I first made some guesses and hypoteses based on the type of game and what I already knew about Myst: the data files were probably some sort of archives containing game pieces, so the goal was un-archiving them. I first directed my efforts toward understanding Mohawk archives. I did hex dumps of the smaller data files, printed pages and pages and carefully searched the numbers for any possible pattern. I finally managed to write a small program to "unroll" Mohawk archives and reveal their content, which was a bunch of "resources". Each resource was identified by a 4-char type, an integer ID and optionally a name. An approach very similar to old MacOS resources. These resources were the pieces of the game: pictures, movies, sounds, unknown data. They were mostly encoded in proprietary formats, so I had to start again with the research: make hex dumps, print, analyze and search for patterns, for every resource type. For example, in pictures I first looked for values matching the bitmap dimensions, while in sounds I looked for common sample rate values like 44100. Decoding resources like SFXE and RMAP also involved understanding what the resource contained; I guessed this by observing the context in which these resources were invoked. With new info constantly coming in and so many resource types to handle, at some point I decided to make some order in my head, pause the research and write a big Mohawk archive browser, a program with a nice graphical interface to explore Mohawk files "by click" and, more important, test whether hypotetic data structures really matched all of the available data. It was a good idea and greatly speeded up further research.

More or less, the decoding process can always be viewed as an initial observation of the numbers, formulation of hypotetic data structures and creation of test programs to match the data structures to all available data. If even a single mismatch is found, the data structure has to be reviewed. Clearly, some of the steps can be skipped when you already have enough information. Another useful approach is brutally altering the data and then running the original game engine to see what happens. However this is like bruteforcing and is only useful to provide hints when you are really desperate about a single unknown field. If you alter something you know nothing about, you'll mostly get a hard, spectacular crash filling the screen with random pixels.

As for the tools, the whole thing has been performed on Linux, using mainly hexdump, hexedit and the good compiler gcc. I used MPlayer to watch tMOV resources and Baudline to check decoded tWAV audio signals.

Who contributed?

I didn't decode everything myself, and I want to thank people who contributed. Gabriel Bauman figured out most of the Mohawk archive format when I was still enjoying the game, and then founded the Fissure port; his work was my initial inspiration. Arthur Muller provided an immense help by understanding tBMP compression after my vain efforts. Ron Hayter provided details on the DVD version of tWAV resources. Jean-Francois Roy, author of Riven X, constantly keeps my interest active and provides reasons to go on. Mario Carneiro was the one that definitely convinced me to port the original documentation to this wiki, and he is contributing to the Myst research. Many thanks to all!