|
Pages: 1 [2] 3
|
 |
|
Author
|
Topic: Star Control 3 STAR001.VOC archive WAV file format? (Read 20194 times)
|
Novus
Enlightened
 Offline
Gender: 
Posts: 1938
Fot or not?
|
Studying the DOSBox output from SC3 seems to confirm that the speech is 8-bit PCM mono at 22 kHz (DOSBox output at 16 bits/44 kHz (with linear interpolation) has every other sample half way between its neighbours and 22 kHz output exhibits about 240 distinct signal levels). If that part of the header information checks out, the uncompressed length is probably correct, too.
Further analysis of the DOSBox output supports the theory of an ADPCM variant; prime numbers (e.g. 53) are heavily underrepresented in the differences between two successive samples (the fact that 1 or 2 instances show up may be an artefact of the playback process in SC3 or DOSBox). LucasArts VIMA seems to be similar in principle.
|
|
|
|
|
Logged
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Novus: Did you manage to get a pure digital sampling of the audio output? I'm assuming not since you mentioned artifacts, but I wonder if it would be possible with DOSBox or some kind of signal analysis software. Thinking about this reminded me of a software called "Virtual Audio Cable" (VAC) which lets you reroute a digital audio signal from the output of one program to the input of another. That's assuming you can actually select the output and input drivers on both softwares, though. I've been using Virtual PC (Microsoft version) with MS-DOS 6.22 installed, but there seems to be no way to get a recording without resorting to sampling the sound card's main outs (you can't select an output driver like VAC), which puts us in the analog domain. I thought about trying an optical out, but not sure that would be enough to get a pure signal since we're emulating DOS and relying on two layers of drivers for output (emulation driver passed to the host driver). But if you could select an output driver with DOSBox, that would give us an excellent option for analyzing the signal. I haven't had a chance to play with DOSBox yet, so I'll have to look into that. Thanks for the idea. 
Another possibility for a pure signal would be VDMSound, which operates as a driver emulating DOS sound hardware within the native OS, and has a "Wave Writer" option. Though every time I try to enable that option with Star Control 3, it refuses to work on my system. I just opens a file for writing, but never writes anything to it, heh.
I suppose, if nothing else, I could get one of these options working (or even go analog) and meticulously go through and record all the audio I want; but this, of course, doesn't answer the question of the format. It becomes academic at this point, but hey, that's what I'm here for, right? 
|
|
|
|
|
Logged
|
|
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
meep-eep: Thanks for the information. I was just now reading about the SDL environment variables on the DOSBox Wiki. I'll have to try that since VDMSound is not being kind to me on this game.
|
|
|
|
|
Logged
|
|
|
|
Novus
Enlightened
Offline
Gender:
Posts: 1938
Fot or not?
|
Novus: Did you manage to get a pure digital sampling of the audio output? I'm assuming not since you mentioned artifacts, but I wonder if it would be possible with DOSBox or some kind of signal analysis software.
The problem is that DOSBox is not dumping the raw data sent to the sound card; it's resampling everything to the output frequency, applying mixer settings, adding OPL output (unless disabled) and so on (as well as accurately emulating any gaps in the signal). Setting the DOSBox output frequency to the Sound Blaster's approximation of 22050 Hz (22222 Hz, I think) and turning up the PCM volume to max ought to fix the resampling and scaling problems, but doesn't affect any timing-related problems.
Of course, since DOSBox is open source (just like UQM), it's possible to hack it to output the sound data sent to the emulated SB16 instead of the emulated SB16's output.
Recording from DOSBox is dead easy; just make sure you have a "capture" directory (or whatever your DOSBox configuration calls it) and press CTRL-F6 to record (this has the advantages over meep-eep's method that you can start/stop recording while running and hear what's playing). The only problem is that DOSBox is that DOSBox is emulating the sound a bit too realistically as mentioned above.
I suppose, if nothing else, I could get one of these options working (or even go analog) and meticulously go through and record all the audio I want; but this, of course, doesn't answer the question of the format. It becomes academic at this point, but hey, that's what I'm here for, right?
It would be easier if I had both the compressed version of a sound and a good recording of the same sound. The compressed sounds are a bit hard to identify, especially since it was quite a while since I actually played SC3 last, so I'm having a hard time finding the corresponding sound in game. As far as I can tell, silence tends to be encoded as repetitions of 7f 7f... which seems to correspond to silence showing up as sequences of 0, 1, 0, 1... That looks like 4-bit ADPCM with the sign bit in the MSB of each nibble to me and some sort of inverted magnitude part. However, I'm just guessing at this point. Quick and silly hypothesis: Creative VOC data with all the bits inverted.
Edit: Actually, that hypothesis might not be as silly as it initially sounds; Creative's Sound Blasters (and therefore also VOC files) support 3 different ADPCM formats, with compression ratios of 1:4 (2-bit), 1:3 ("2.6-bit") and 1:2 (4-bit). AIL/Miles Sound System has had VOC support since the beginning as far as I can tell, which ought to include compressed VOC files. Inverting all bits would be a good example of the "encryption" game programmers like to do to standard file formats (e.g. LucasArts archives make a lot more sense if you XOR everything with 0x69). Writing a decoder and seeing whether it works might be a good idea.
|
|
|
|
« Last Edit: May 12, 2006, 10:21:29 pm by Novus »
|
Logged
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Novus: Ok, here is a link to one of the extracted (compressed) files as well as the DOSBox output of that file, reduced to 22,050 Hz 8-bit Mono (Left channel):
http://www.oblyvaeon.com/TEMP/WAV03545.zip
The DOSBox output was 22,050 Hz 16-bit Stereo, but it looks like it only contained 8 bits worth of data anyway, and each channel contained the same data, other than the stereo image being out of phase by 1 millisecond. It is interesting to note (I mentioned this earlier with regards to the intelligibility of the audio) that downsampling the compressed file to 11,025 Hz makes the audible portion of the file closely match the length that the file would be if properly played / uncompressed. This suggests a 1:2 format. How closely matched in this case depending on human error while pressing CTRL-F6 in DOSBox
Can't get the quotes to work on this forum, hmm, but regarding the bit inversion / XOR: yeah, I've encountered this before as well. Even used it myself on some "archives" attached to the end of executable files. The problem here being, what did they do this time? Haha, but well, I could try a few simple swaps for fun anyway.
Side note: I guess I was wrong about intro/cutscene audio being among the 150 'good' files I could extract, because this file happens to be audio from the first segment of the intro, heh. Should have given them a good listen again. It's been a while since I actually played the game all the way through as well.
|
|
|
|
« Last Edit: May 12, 2006, 11:42:02 pm by Daktaklakpak »
|
Logged
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Well, isn't this interesting... I was going over the 150 'good' files for clues, and noticed that they are a mixed bag of 8 AND 16-bit 22,050 Hz Mono files. Don't know if this will have any impact on the format we are seeking, but thought it was worth mentioning. This should make it harder to figure out, yay! 
|
|
|
|
|
Logged
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Report: Initial bit inversion yielded no results. Tried all bits and even bits, similar to A-Law and mu-Law types, even tried odd bits. I only tried it on a couple files so far and have no reference to check these against, so don't let that stop anyone else from trying the same thing. After observing the mixed bag earlier, who knows? Maybe they used a slightly different encoding on each file haha, wouldn't that be fun...
|
|
|
|
|
Logged
|
|
|
|
Novus
Enlightened
Offline
Gender:
Posts: 1938
Fot or not?
|
Thanks, that ought to help.
The DOSBox output was 22,050 Hz 16-bit Stereo, but it looks like it only contained 8 bits worth of data anyway, and each channel contained the same data, other than the stereo image being out of phase by 1 millisecond. It is interesting to note (I mentioned this earlier with regards to the intelligibility of the audio) that downsampling the compressed file to 11,025 Hz makes the audible portion of the file closely match the length that the file would be if properly played / uncompressed. This suggests a 1:2 format. How closely matched in this case depending on human error while pressing CTRL-F6 in DOSBox
Can't get the quotes to work on this forum, hmm, but regarding the bit inversion / XOR: yeah, I've encountered this before as well.
The "insert quote" links in the "Post reply" page have been broken for so long I forgot they existed. The "Quote" button on the main thread view works fine, but only quotes one post.
If you want to obfuscate your data but not sacrifice performance, you have to use a fast algorithm. XOR is quick. That makes it common. However, in that case, I would have expected the entire file to be XORed, not just the data.
|
|
|
|
|
Logged
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Had another chance to play with the files today. Did a quick and dirty XOR on WAV00000.WAV using every single-byte key from 0-255 (included 0 as a check of the program's integrity). Just got through listening to all 256 files, and guess what: no results. Er, no GOOD results
Oh well, I'm posting some BASIC code if anyone wants to modify it to use a larger key. i.e. Maybe there is a key somewhere in the file data, like those 3-4 bytes before each WAVE header. There are also quite a few other not-so-random looking bytes near the end of the 'padding' area of each WAVE file. Might play with it some more if I can get some free time.
Remember, this code is quick and dirty; no error trapping or file creation confirmation, etc. Use at your own risk. I don't know if this board supports code snippets, so everything following this line is code:
'Brute force XOR decrypt (single byte key)
'
'Notes:
' First output file is XOR 0 (output file = input file), verifies program is working properly
' Modified to leave a 44 byte WAVE header intact
DEFINT A-Z
DIM CurrentByte AS STRING * 1
DIM NewByte AS STRING * 1
INPUT "File to process: ", infile$
OPEN infile$ FOR BINARY AS #1
FOR XorLoop = 0 TO 255
outfile$ = "XOR_" + LTRIM$(RTRIM$(STR$(XorLoop))) + ".WAV" 'format (text) output file name
PRINT "Output file: ", outfile$ 'show what file we are currently operating on
OPEN outfile$ FOR BINARY AS #2
FOR HeaderLoop = 1 TO 44 'leave header intact
GET #1, HeaderLoop, CurrentByte
PUT #2, HeaderLoop, CurrentByte
NEXT HeaderLoop
FOR ReadLoop& = 45 TO LOF(1) 'XOR remaining bytes with current key
GET #1, ReadLoop&, CurrentByte
NewByte = CHR$(ASC(CurrentByte) XOR XorLoop)
PUT #2, ReadLoop&, NewByte
NEXT ReadLoop&
CLOSE #2
NEXT XorLoop
CLOSE
END
Edit: ^^^ Hey look, a code snippet! Heh, thanks Novus.
|
|
|
|
« Last Edit: May 21, 2006, 07:26:59 pm by Daktaklakpak »
|
Logged
|
|
|
|
Novus
Enlightened
Offline
Gender:
Posts: 1938
Fot or not?
|
I did some poking around with DOSBox. Whatever compression SC3 is using, it's not using the Sound Blaster's ADPCM decoder to play it. Also, the intro speech seems to be using all 16 bits, while the dialogue is in 8 bit, meaning that they are probably in different, unknown, compressed formats. 
I managed to find a copy of the source code to Miles Audio Interface Library (AIL) 2.14 on John Miles's homepage, but the only ADPCM support there seems to rely on the SB's decoding hardware.
On the plus side, I did manage to extract the subfiles apparently cleanly without any trailing garbage using the following:
#include <stdio.h>
#include <stdlib.h>
void abend(char *s) {
fprintf(stderr, "%s\n", s);
exit(1);
}
int main(int argc, char *argv[]) {
unsigned short files;
FILE *in,*out;
unsigned int *offset;
int a;
unsigned int size;
char *buffer=NULL;
char filename[200];
if ((in=fopen(argv[1], "rb"))==NULL)
abend("Input file open failed.");
if (fread(&files, sizeof(unsigned short), 1, in)<1)
abend("Number of files read failed.");
offset=malloc(sizeof(unsigned int)*files);
fprintf(stderr, "Offsets in index: %d\n", files);
if (fread(offset, sizeof(unsigned int), files, in)<1)
abend("Offsets read failed.");
for(a=0;a<files;a++)
if (offset[a]) {
fseek(in, offset[a]+2, SEEK_SET);
fread(&size, sizeof(unsigned int), 1, in);
fprintf(stderr, "File %d: %d bytes\n", a, size);
buffer=realloc(buffer, size*sizeof(char));
fread(buffer, sizeof(char), size, in);
sprintf(filename, "out%d.wav", a);
out=fopen(filename, "wb");
fwrite(buffer, sizeof(char), size, out);
fclose(out);
}
fclose(in);
free(buffer);
free(offset);
return 0;
}
|
|
|
|
|
Logged
|
|
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
I did some poking around with DOSBox. Whatever compression SC3 is using, it's not using the Sound Blaster's ADPCM decoder to play it. ... I managed to find a copy of the source code to Miles Audio Interface Library (AIL) 2.14 on John Miles's homepage, but the only ADPCM support there seems to rely on the SB's decoding hardware.
That makes sense, considering that if they did use SB hardware to decode, the audio wouldn't work with any of the other supported sound cards.  It would have to be decoded before passing data to the sound card driver(s). I'm going over some decompiled ASM code now, but I think the best route at this point will be with a low level debugger, such as SoftICE. I might fall back to a real DOS box for this.
|
|
|
|
|
Logged
|
|
|
|
|
|
Daktaklakpak
Zebranky food
Offline
Posts: 11
|
Another interesting development...
Since Novus pointed it out, I decided to play around with DOSBox's built-in debugger, and using a HEAVY_DEBUG build, I obtained the following (excerpt from the debugger):
462876: EXEC:Execute sc3.EXE 0
462876: FILES:file open command 0 file sc3.EXE
464196: FILES:file open command 0 file C:\SC3.EXE
464407: FILES:file open command 0 file C:\SC3.ETX
465760: BIOS:INT15:Unknown call BFDE
465811: DOSMISC:DOS:Multiplex Unhandled call 1687
465834: BIOS:INT15:Unknown call BFDE
470291: BIOS:INT15:Unknown call BF01
470349: BIOS:INT15:Function 0x88 Remaining 0000 kb
1503533: FILES:file open command 0 file C:\SC3.EXE
9000353: FILES:file open command 0 file LEGEND.INI
9023854: PIT:PIT 0 Timer at 18.21 Hz mode 3
9190030: PIT:PIT 0 Timer at 99.99 Hz mode 3
9192092: FILES:Special file open command 10 file AUTORUN.LOC
9222210: FILES:file open command 0 file MDI.INI
9248830: FILES:file open command 0 file MPU401.MDI
9258415: FILES:file open command 0 file MPU401.MDI
9266178: FILES:file open command 0 file MPU401.MDI
9281409: MISC:MPU-401:Reset FF
9281907: MISC:MPU-401:Set UART mode 3F
10903521: PIT:PIT 0 Timer at 120.00 Hz mode 3
10904754: FILES:file open command 0 file GM.XMI
10992615: FILES:file open command 0 file DIG.INI
11016330: FILES:file open command 0 file SB16.DIG
11026586: FILES:file open command 0 file SB16.DIG
11035030: FILES:file open command 0 file SB16.DIG
11051512: SBLASTER:DSP:Reset
11051718: PIC:1 mask EC
11051722: PIC:0 mask 78
11052043: SBLASTER:Short transfer scheduling IRQ in 0.023 milliseconds
11052043: SBLASTER:DMA unmasked,starting output, auto 0 block 1
11052043: SBLASTER:DMA Transfer:16-bits PCM Stereo Single-Cycle freq 22050 rate 44100 size 1
11052069: IO:Writing 04 to port 00ED
11052268: SBLASTER:Single cycle transfer ended
11052268: SBLASTER:Raising IRQ
11136859: PIC:1 mask EC
11136863: PIC:0 mask F8
11140162: PIC:1 mask EC
11140166: PIC:0 mask 78
11140181: SBLASTER:MIXER:Read from unhandled index 32
11140187: SBLASTER:MIXER:Read from unhandled index 33
11140205: SBLASTER:MIXER:Read from unhandled index 32
11140208: SBLASTER:MIXER:Write 0 to unhandled index 32
11140213: SBLASTER:MIXER:Read from unhandled index 33
11140216: SBLASTER:MIXER:Write 0 to unhandled index 33
11143464: PIT:PIT 0 Timer at 200.00 Hz mode 3
11155239: FILES:file open command 0 file STAR001.VOC
11187488: INT10:Function 6F not supported
11188096: INT10:Function 5F not supported
11188338: IO:Read from port 03CD
11188342: IO:Writing 55 to port 03CD
11188737: INT10:Function 12:Call 80 not handled
11378620: INT10:Set Video Mode 101
11378620: VGA:Blinking 0
11378620: MOUSE:Unhandled videomode 69 on reset
11387796: FILES:file open command 0 file STAR000.PIC
11398032: MOUSE:Unhandled videomode 69 on reset
11404208: FILES:file open command 0 file STAR000.FNT
11421087: FILES:file open command 0 file SC3STR.DAT
11442600: MOUSE:Define Hortizontal range min:0 max:639
11443241: MOUSE:Define Vertical range min:0 max:479
11449370: FILES:file open command 0 file .\Q\STAR909.Q
11858361: VGA:H total 100, V Total 525
11858361: VGA:H D End 80, V D End 480
11858361: VGA:Width 640, Height 480, fps 70.007141
11858361: VGA:normal width, normal height aspect 1.000000
13196077: FILES:file open command 0 file .\Q\STAR923.Q
13497437: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
13497437: SBLASTER:DMA Transfer:16-bits PCM Stereo Auto-Init freq 22050 rate 44100 size 1024
13708984: SBLASTER:Raising IRQ
13939022: SBLASTER:Raising IRQ
14168409: SBLASTER:Raising IRQ
...
19784215: SBLASTER:Raising IRQ
20004240: SBLASTER:Raising IRQ
20234289: SBLASTER:Raising IRQ
30696035: FILES:file open command 0 file STAR200.PIC
35391734: FILES:file open command 0 file STAR220.PIC
35771776: FILES:file open command 0 file STAR400.PIC
37563673: FILES:file open command 0 file STAR000.PIC
41174719: FILES:file open command 0 file STAR800.PIC
41195646: FILES:file open command 2 file RESTART.DAT
69015087: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
69224460: SBLASTER:Raising IRQ
69444582: SBLASTER:Raising IRQ
69674713: SBLASTER:Raising IRQ
69904075: SBLASTER:Raising IRQ
70481783: FILES:file open command 0 file STAR200.PIC
75455670: FILES:file open command 0 file STAR220.PIC
75863420: FILES:file open command 0 file STAR400.PIC
77734434: FILES:file open command 0 file STAR000.PIC
81504589: FILES:file open command 0 file STAR800.PIC
81529665: FILES:file open command 0 file RESTART.DAT
101593877: FILES:file open command 0 file .\Q\STAR905.Q
102020600: SBLASTER:DMA unmasked,starting output, auto 1 block 2047
102231354: SBLASTER:Raising IRQ
102471391: SBLASTER:Raising IRQ
102701423: SBLASTER:Raising IRQ
...
The ... indicates more lines of the same (i.e. Sound Blaster raising an IRQ, which just plays audio in this case). What I found was that Star Control 3's Q files, which were accessed before playing a video (STAR909.Q, STAR923.Q, STAR905.Q), also contain WAVE headers. STAR905.Q seems to be the first intro video, for which I posted a link containing the audio portion from STAR001.VOC earlier in this discussion. I poked around in some of the other Q files, and not all of them contain WAVE headers. The ones that do contain headers will extract properly with MRIP, but the data they contain is useless as-is. BAH! More riddles.
What we do now is watch the ASM instructions to see what SC3.EXE does with this data. While we can see the code accessing the sound driver (SB16.DIG) in the listing above, I can't tell if, or when, DOSBox's debugger is displaying the instructions executed by the driver, though we can probably assume that most of the "SBLASTER:" commands are coming from it. This may not be particularly relevant to the decoding process anyway, but I thought it was worth mentioning. I do know that SoftICE will tell you when switching to and displaying a driver's code as soon as it is called, though I still haven't had the time to set up a box for this yet. With luck, the calls will be documented (right, haha) and our solution will be found. I'm even not going to try and assert the probability of this, heh.
|
|
|
|
|
Logged
|
|
|
|
|
Pages: 1 [2] 3
|
|
|
|
|
