Another interesting development...

Since Novus pointed it out, I decided to play around with DOSBox's built-in debugger, and using a HEAVY_DEBUG build, I obtained the following (excerpt from the debugger):

Code:

    462876: EXEC:Execute sc3.EXE 0
    462876: FILES:file open command 0 file sc3.EXE
    464196: FILES:file open command 0 file C:\SC3.EXE
    464407: FILES:file open command 0 file C:\SC3.ETX
    465760: BIOS:INT15:Unknown call BFDE
    465811: DOSMISC:DOS:Multiplex Unhandled call 1687
    465834: BIOS:INT15:Unknown call BFDE
    470291: BIOS:INT15:Unknown call BF01
    470349: BIOS:INT15:Function 0x88 Remaining 0000 kb
   1503533: FILES:file open command 0 file C:\SC3.EXE
   9000353: FILES:file open command 0 file LEGEND.INI
   9023854: PIT:PIT 0 Timer at 18.21 Hz mode 3
   9190030: PIT:PIT 0 Timer at 99.99 Hz mode 3
   9192092: FILES:Special file open command 10 file AUTORUN.LOC
   9222210: FILES:file open command 0 file MDI.INI
   9248830: FILES:file open command 0 file MPU401.MDI
   9258415: FILES:file open command 0 file MPU401.MDI
   9266178: FILES:file open command 0 file MPU401.MDI
   9281409: MISC:MPU-401:Reset FF
   9281907: MISC:MPU-401:Set UART mode 3F
  10903521: PIT:PIT 0 Timer at 120.00 Hz mode 3
  10904754: FILES:file open command 0 file GM.XMI
  10992615: FILES:file open command 0 file DIG.INI
  11016330: FILES:file open command 0 file SB16.DIG
  11026586: FILES:file open command 0 file SB16.DIG
  11035030: FILES:file open command 0 file SB16.DIG
  11051512: SBLASTER:DSP:Reset
  11051718: PIC:1 mask EC
  11051722: PIC:0 mask 78
  11052043: SBLASTER:Short transfer scheduling IRQ in 0.023 milliseconds
  11052043: SBLASTER:DMA unmasked,starting output, auto 0 block 1
  11052043: SBLASTER:DMA Transfer:16-bits PCM Stereo Single-Cycle freq 22050 rate 44100 size 1
  11052069: IO:Writing 04 to port 00ED
  11052268: SBLASTER:Single cycle transfer ended
  11052268: SBLASTER:Raising IRQ
  11136859: PIC:1 mask EC
  11136863: PIC:0 mask F8
  11140162: PIC:1 mask EC
  11140166: PIC:0 mask 78
  11140181: SBLASTER:MIXER:Read from unhandled index 32
  11140187: SBLASTER:MIXER:Read from unhandled index 33
  11140205: SBLASTER:MIXER:Read from unhandled index 32
  11140208: SBLASTER:MIXER:Write 0 to unhandled index 32
  11140213: SBLASTER:MIXER:Read from unhandled index 33
  11140216: SBLASTER:MIXER:Write 0 to unhandled index 33
  11143464: PIT:PIT 0 Timer at 200.00 Hz mode 3
  11155239: FILES:file open command 0 file STAR001.VOC
  11187488: INT10:Function 6F not supported
  11188096: INT10:Function 5F not supported
  11188338: IO:Read from port 03CD
  11188342: IO:Writing 55 to port 03CD
  11188737: INT10:Function 12:Call 80 not handled
  11378620: INT10:Set Video Mode 101
  11378620: VGA:Blinking 0
  11378620: MOUSE:Unhandled videomode 69 on reset
  11387796: FILES:file open command 0 file STAR000.PIC
  11398032: MOUSE:Unhandled videomode 69 on reset
  11404208: FILES:file open command 0 file STAR000.FNT
  11421087: FILES:file open command 0 file SC3STR.DAT
  11442600: MOUSE:Define Hortizontal range min:0 max:639
  11443241: MOUSE:Define Vertical range min:0 max:479
  11449370: FILES:file open command 0 file .\Q\STAR909.Q
  11858361: VGA:H total 100, V Total 525
  11858361: VGA:H D End 80, V D End 480
  11858361: VGA:Width 640, Height 480, fps 70.007141
  11858361: VGA:normal width, normal height aspect 1.000000
  13196077: FILES:file open command 0 file .\Q\STAR923.Q
  13497437: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
  13497437: SBLASTER:DMA Transfer:16-bits PCM Stereo Auto-Init freq 22050 rate 44100 size 1024
  13708984: SBLASTER:Raising IRQ
  13939022: SBLASTER:Raising IRQ
  14168409: SBLASTER:Raising IRQ
...
  19784215: SBLASTER:Raising IRQ
  20004240: SBLASTER:Raising IRQ
  20234289: SBLASTER:Raising IRQ
  30696035: FILES:file open command 0 file STAR200.PIC
  35391734: FILES:file open command 0 file STAR220.PIC
  35771776: FILES:file open command 0 file STAR400.PIC
  37563673: FILES:file open command 0 file STAR000.PIC
  41174719: FILES:file open command 0 file STAR800.PIC
  41195646: FILES:file open command 2 file RESTART.DAT
  69015087: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
  69224460: SBLASTER:Raising IRQ
  69444582: SBLASTER:Raising IRQ
  69674713: SBLASTER:Raising IRQ
  69904075: SBLASTER:Raising IRQ
  70481783: FILES:file open command 0 file STAR200.PIC
  75455670: FILES:file open command 0 file STAR220.PIC
  75863420: FILES:file open command 0 file STAR400.PIC
  77734434: FILES:file open command 0 file STAR000.PIC
  81504589: FILES:file open command 0 file STAR800.PIC
  81529665: FILES:file open command 0 file RESTART.DAT
 101593877: FILES:file open command 0 file .\Q\STAR905.Q
 102020600: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
 102231354: SBLASTER:Raising IRQ
 102471391: SBLASTER:Raising IRQ
 102701423: SBLASTER:Raising IRQ
...

The ... indicates more lines of the same (i.e. Sound Blaster raising an IRQ, which just plays audio in this case). What I found was that Star Control 3's Q files, which were accessed before playing a video (STAR909.Q, STAR923.Q, STAR905.Q), also contain WAVE headers. STAR905.Q seems to be the first intro video, for which I posted a link containing the audio portion from STAR001.VOC earlier in this discussion. I poked around in some of the other Q files, and not all of them contain WAVE headers. The ones that do contain headers will extract properly with MRIP, but the data they contain is useless as-is. BAH! More riddles.

What we do now is watch the ASM instructions to see what SC3.EXE does with this data. While we can see the code accessing the sound driver (SB16.DIG) in the listing above, I can't tell if, or when, DOSBox's debugger is displaying the instructions executed by the driver, though we can probably assume that most of the "SBLASTER:" commands are coming from it. This may not be particularly relevant to the decoding process anyway, but I thought it was worth mentioning. I do know that SoftICE will tell you when switching to and displaying a driver's code as soon as it is called, though I still haven't had the time to set up a box for this yet. With luck, the calls will be documented (right, haha) and our solution will be found. I'm even not going to try and assert the probability of this, heh.

I did some poking around with DOSBox. Whatever compression SC3 is using, it's not using the Sound Blaster's ADPCM decoder to play it.
...
I managed to find a copy of the source code to Miles Audio Interface Library (AIL) 2.14 on John Miles's homepage, but the only ADPCM support there seems to rely on the SB's decoding hardware.

That makes sense, considering that if they did use SB hardware to decode, the audio wouldn't work with any of the other supported sound cards. It would have to be decoded before passing data to the sound card driver(s). I'm going over some decompiled ASM code now, but I think the best route at this point will be with a low level debugger, such as SoftICE. I might fall back to a real DOS box for this.

Had another chance to play with the files today. Did a quick and dirty XOR on WAV00000.WAV using every single-byte key from 0-255 (included 0 as a check of the program's integrity). Just got through listening to all 256 files, and guess what: no results. Er, no GOOD results

Oh well, I'm posting some BASIC code if anyone wants to modify it to use a larger key. i.e. Maybe there is a key somewhere in the file data, like those 3-4 bytes before each WAVE header. There are also quite a few other not-so-random looking bytes near the end of the 'padding' area of each WAVE file. Might play with it some more if I can get some free time.

Remember, this code is quick and dirty; no error trapping or file creation confirmation, etc. Use at your own risk. I don't know if this board supports code snippets, so everything following this line is code:

Code:

'Brute force XOR decrypt (single byte key)
'
'Notes:
'  First output file is XOR 0 (output file = input file), verifies program is working properly
'  Modified to leave a 44 byte WAVE header intact

DEFINT A-Z

DIM CurrentByte AS STRING * 1
DIM NewByte AS STRING * 1

INPUT "File to process: ", infile$
OPEN infile$ FOR BINARY AS #1

FOR XorLoop = 0 TO 255

  outfile$ = "XOR_" + LTRIM$(RTRIM$(STR$(XorLoop))) + ".WAV" 'format (text) output file name
  PRINT "Output file: ", outfile$ 'show what file we are currently operating on
  OPEN outfile$ FOR BINARY AS #2

  FOR HeaderLoop = 1 TO 44 'leave header intact
    GET #1, HeaderLoop, CurrentByte
    PUT #2, HeaderLoop, CurrentByte
  NEXT HeaderLoop
 
  FOR ReadLoop& = 45 TO LOF(1) 'XOR remaining bytes with current key
    GET #1, ReadLoop&, CurrentByte
    NewByte = CHR$(ASC(CurrentByte) XOR XorLoop)
    PUT #2, ReadLoop&, NewByte
  NEXT ReadLoop&

  CLOSE #2

NEXT XorLoop

CLOSE
END

Edit: ^^^ Hey look, a code snippet! Heh, thanks Novus.

Report: Initial bit inversion yielded no results. Tried all bits and even bits, similar to A-Law and mu-Law types, even tried odd bits. I only tried it on a couple files so far and have no reference to check these against, so don't let that stop anyone else from trying the same thing. After observing the mixed bag earlier, who knows? Maybe they used a slightly different encoding on each file haha, wouldn't that be fun...

Well, isn't this interesting... I was going over the 150 'good' files for clues, and noticed that they are a mixed bag of 8 AND 16-bit 22,050 Hz Mono files. Don't know if this will have any impact on the format we are seeking, but thought it was worth mentioning. This should make it harder to figure out, yay!

Novus: Ok, here is a link to one of the extracted (compressed) files as well as the DOSBox output of that file, reduced to 22,050 Hz 8-bit Mono (Left channel):

http://www.oblyvaeon.com/TEMP/WAV03545.zip

The DOSBox output was 22,050 Hz 16-bit Stereo, but it looks like it only contained 8 bits worth of data anyway, and each channel contained the same data, other than the stereo image being out of phase by 1 millisecond. It is interesting to note (I mentioned this earlier with regards to the intelligibility of the audio) that downsampling the compressed file to 11,025 Hz makes the audible portion of the file closely match the length that the file would be if properly played / uncompressed. This suggests a 1:2 format. How closely matched in this case depending on human error while pressing CTRL-F6 in DOSBox

Can't get the quotes to work on this forum, hmm, but regarding the bit inversion / XOR: yeah, I've encountered this before as well. Even used it myself on some "archives" attached to the end of executable files. The problem here being, what did they do this time? Haha, but well, I could try a few simple swaps for fun anyway.

Side note: I guess I was wrong about intro/cutscene audio being among the 150 'good' files I could extract, because this file happens to be audio from the first segment of the intro, heh. Should have given them a good listen again. It's been a while since I actually played the game all the way through as well.

meep-eep: Thanks for the information. I was just now reading about the SDL environment variables on the DOSBox Wiki. I'll have to try that since VDMSound is not being kind to me on this game.

Novus: Did you manage to get a pure digital sampling of the audio output? I'm assuming not since you mentioned artifacts, but I wonder if it would be possible with DOSBox or some kind of signal analysis software. Thinking about this reminded me of a software called "Virtual Audio Cable" (VAC) which lets you reroute a digital audio signal from the output of one program to the input of another. That's assuming you can actually select the output and input drivers on both softwares, though. I've been using Virtual PC (Microsoft version) with MS-DOS 6.22 installed, but there seems to be no way to get a recording without resorting to sampling the sound card's main outs (you can't select an output driver like VAC), which puts us in the analog domain. I thought about trying an optical out, but not sure that would be enough to get a pure signal since we're emulating DOS and relying on two layers of drivers for output (emulation driver passed to the host driver). But if you could select an output driver with DOSBox, that would give us an excellent option for analyzing the signal. I haven't had a chance to play with DOSBox yet, so I'll have to look into that. Thanks for the idea.

Another possibility for a pure signal would be VDMSound, which operates as a driver emulating DOS sound hardware within the native OS, and has a "Wave Writer" option. Though every time I try to enable that option with Star Control 3, it refuses to work on my system. I just opens a file for writing, but never writes anything to it, heh.

I suppose, if nothing else, I could get one of these options working (or even go analog) and meticulously go through and record all the audio I want; but this, of course, doesn't answer the question of the format. It becomes academic at this point, but hey, that's what I'm here for, right?

From reading K.Death's post, it sounds like he could not extract all the files; he never posted a follow-up to tell us if he did finally get them. So, for those who want to work on this, you can get all 3,592 files out of STAR001.VOC with Multi-Ripper (MRIP) v2.80 by specifying the /N switch on the command line. If you don't use the /N switch, it will only extract the 150 'good' files mentioned earlier (the intro/cutscene/credits and sound effects audio).

Syntax:

MRIP STAR001.VOC /N

You can use the default Windows WAVE search, or for a cleaner result use:

User Defined Pattern: RIFF
File Extension: WAV
Offset: 0

Novus: That's about as far as I've gotten with the archive itself as well. One theory I had was that STAR001.VOC is a pseudo-compressed archive (or contains pseudo-compressed files), such that the 'pseudo' part would be referring to the WAV data itself being compressed while the headers become accurate for the files when Star Control 3 (or Miles) decompresses them at runtime, in which case we're at a dead-end if they used a proprietary compression algorithm. Pretty much what you said with regards to the WAV header referring to the unpacked size. It sounds like you've played with this file before But at any rate, I'm still hoping it is not a proprietary or in-house derived format. This is my 3rd attempt over the years to crack this thing, heh.

On a side note: I am trying to contact some of the old Legend programmers to see if they will let anything slip, though they are hard to track down.  I found one on the Gamasutra (sp?) staff, but no response yet. Also tried to contact a user (Kohr-Ah Death) on this forum who claims to know the format in this post: http://uqm.stack.nl/forum/index.php?topic=2748.0 Maybe he's had the answer all along...

Novus: You calling me a Dak?! Haha.

I have been researching Miles version 3.x actually, but I don't know if that will point to a specific format. Doesn't look like it so far, unfortunately. I agree about ADPCM being a likely format, since when you force most ADPCM formats to load as uncompressed PCM (Unsigned Little-endian 8-bit in this case) they do sound similar to the files in STAR001.VOC.