The Ur-Quan Masters Discussion Forum

Good day. Any plans to port UQM onto Sony Playstation Portable console - as a firmware 1.50 homebrew, perhaps? Its screen resolution is big enough to fit the graphics, it's powerful enough even for some scaling and resampling, and all the big MemorySticks will hold all the content with no problems.

People often suggest porting UQM to this console or that console. Most of the time, the response is simply "Good idea! Tell us when you're finished." or something to that effect.

The problems with developing for consoles are (off the top of my head):

• Most of the development team don't have the console you want a version for.
• As programming environments go, consoles are quite different to general-purpose computers.
• Consoles come and go, while the major workstation operating systems (Windows and Unix derivatives) stay pretty much the same or at least mostly backwards compatible.
• Many console makers actively discourage homebrew development through technical or legal limitations on home development.

A port of SDL for the PSP (http://svn.ps2dev.org/filedetails.php?repname=psp&path=%2Ftrunk%2FSDL%2FREADME.PSP&rev=0&sc=0) apparently exists, so porting UQM to the PSP shouldn't be too hard. Still, there are a lot of issues to be ironed out, and I haven't heard anyone mention an interest in doing a PSP port.

Yes, I heard of SDL port for PSP.

Thank you for a very consistent reply. Unfortunately, I'm no coder (although I quite undestand the basics as I'm employeed in related industry, I severely lack skill), so hmmm... perhaps some donations might help. I'm willing to donate - it's not like "let me pay to get a PSP port in a near future", it's more like "let's bring some incentive to the topic".

FYI,

I'm about halfway into a port for the GP2X console.

http://en.wikipedia.org/wiki/GP2X

I'll write a thread about it once it runs.

The game will run on QVGA to begin with at 16-bit.  Then I'll start adding more features, like 32-bit and two player support.

CVS now contains support for Tremor which you can use instead of libogg/libvorbis when floating point support is not available or slow.

I am working on a PSP port.  The UQM code is good, so most of the "porting" has involved improving and fixing the pspsdk and PSP SDL.  My current version is partially playable, although there are some memory issues during the game, and I get occasional hangs during Super Melee that I still need to track down.  The libTremor support will be very helpful, thanks.  Once I get things a bit more stable I'll let you know.

Unfortunately I don't actually have a PSP to develop on at the moment, as I upgraded mine to 2.00 to play the new GTA game, but I'll probably downgrade back to 1.50 when I finish that, or maybe just buy another PSP for development work.

As I understand it, the PSP has native floating point support (unlike the GP2X). So why would you chose Tremor over libvorbis/libogg? Are there other advantages? Memory usage?

The PSP only supports single-precision floats, and libvorbis uses doubles a lot.  Also, my eventual goal would be to move the ogg decoding to the second CPU, which lacks a FPU.

Man,

I think you'll make me buy the GP2X sooner than I expected... :-)

Too bad GP32 never had a port (wich I believe would have been too slow).

Someone suggested pooling money together - perhaps this could be used to buy them PSPs?  The trick is getting them a PSP that has 2.0 or earlier firmware so it can be downgraded to 1.5.


Not if they can all be programmed in C.  C is the same if it's on a console or on a 'general-purpose' computer.

For PSP firmware revision 1.5, there is PSPSDK, which contains a minimal port of the standard C library, as well as other underlying support tools available here:

http://wiki.pspdev.org/psp:pspsdk


Who ever said we have to get the console makers involved?

You're missing the point. The console makers are getting themselves involved by modifying firmware to prevent homebrew software from running (as in the case of the PSP you mention) or by taking legal action against non-licensed developers (e.g. Nintendo).

i would love to see this topic show up on the ZSNES forums ;D

Unfortunately ZSNES in its current state will never be ported to the PSP (or to any gaming platform except perhaps the XBox) due to its reliance on Intel assembly code.

The assembly code would have to be ported to C (which would kill low-end system speed which is what ZSNES has always been about), or to platform-specific assembly (which takes a lot of patience and man-hours).

Jimparis, I want to help you in porting Ur-Quad Masters to the psp. I have had fond memories playing it on the pc a few years back and having it on the psp would be spectacular. I have the latest sdk installed along with most of the necessary libraries.

I have a question though...what was the build configuration that you used to compile the game. From what I can tell in the source, it isn't quite straightfoward to just "make" it.
This could be a good experience for me to port other games also. I wanted to port Abuse also but again am stumped by the ./configure type stuff.

./build.sh uqm

right, but I'm cross compiling for the psp platform. Using build.sh uqm initiates a compilation for my machine (cygwin).

The file build/unix/README.crossbuild should contain all the info you need.

yea, I should have gotten my lazy butt to do more research before saying anything. I'll get on it now.

I'm currently using the 0.4 build that is available in the ur-quan masters web page (not cvs).
After much modifying to the CFLAGS, LDFLAGS, PKGconfig, and then modifying the config_unix.h and build.vars, I was able to get it to compile successfully. However I ran into some initial problems.

One was:
    Warning: could no mount 'uqm-04.0-3domusic.zip'; No such device
    Warning: could no mount 'uqm-04.0-content.zip'; No such device
    Warning: could no mount 'uqm-04.0-voice.zip'; No such device

I read from another post that this is because zip mounting was not supported in an earlier version.
The version I've downloaded was from http://sc2.sourceforge.net/downloads.php. I'm just wondering if
there was no zip support in that version or probably something wrong on my end.

Also, another error came about...
 
   Fatal errror: Couldn't mount temp dir 'temp/43a744ef': No such file or directory.

I've checked and found the directory not to be found anywhere, so it could have been a directory creation problem.
In case this helps, this is what i use in config_unix.h

/* Directory where the UQM game data is located */
/* #define CONTENTDIR "/uqm/content" */
#define CONTENTDIR "/psp/game/uqm/content"

/* Directory where game data will be stored */
/* #define USERDIR "~/.uqm/" */
#define USERDIR "/psp/game/uqm/config"

/* Directory where config files will be stored */
#define CONFIGDIR USERDIR

/* Directory where supermelee teams will be stored */
/* #define MELEEDIR "${UQM_CONFIG_DIR}teams/" */
#define MELEEDIR "/psp/game/uqm/config/teams"

/* Directory where save games will be stored */
/* #define SAVEDIR "${UQM_CONFIG_DIR}save/" */
#define SAVEDIR "/psp/game/uqm/config/save"


I'm suspecting both of these problems are related to the unofficial psp sdk, but if you have any insights to the problem, help is much appreciated.

* I've also tried downloading a recent build from cvs (via the web interface) and tried to build it using the same settings and have gotten the error :

Fatal: $MENU_debug_ITEM_graphics_TYPE is not defined!

Any ideas of this also?


P.S.  I've spent some hrs porting OpenAL and then found out I didn't need it to compile :-p, aww well. I'm secretly hoping someone would finish the porting before I do so I can play soon :-)  Never finished the game because I deleted the pc version by accident.

The "no such device" errors are caused by .zip support not being included in your executable. The 0.4 source does support it, but it looks like it wasn't activated in the executable for you.
When you ran "./build.sh uqm config" (or just "./build.sh uqm" from a fresh source), you have an option "Supported file i/o methods".  It should be set to "Direct & .zip file i/o". Note that you can only use this setting if build.sh can find zlib.

I'd also advice you to work from the CVS version. There are some bug fixes and changes that will make your life a lot easier.

The menu item error is an indication that some of your files in build/unix/ are of a different version than others. If you're going to work from CVS, download your files with an actual CVS client, not through the web interface. Or you can download a cvs snapshot from http://uqm.stack.nl/files/snapshots.

As for OpenAL, perhaps you'll make someone else happy with your port.

I'm very happy to see someone is working on a PSP port. Thanks for your hard work! If there is anything a non-coder can help with.. Let me know :)

it was a joke, because anybody who asks to have the program ported to psp usually gets flamed to hell.

I'm not sure how Jimparis's build goes, but so far I've hit a stump and that is memory. The psp has around ~20-22 MBs of user memory available. I've read from the sc2 faq that the memory footprint is around 30MBs. Is there anything I can easily modify to lower the memory footprint?

There's probably a lot of room for improvement. The original PC and 3DO code would unload resources when they are no longer needed, to reload them when they are needed again.I think that currently many resources are kept in memory once they have been loaded. The resource and memory management is something which we intend to redo in the future. But that's not the easy fix you're looking for.

I think the number in the FAQ is a bit on the large size; with a normal debug build, UQM's resident memory usage on this Linux PC I'm typing on gets no larger than 20MB (and the kernel may be able to swap out more if it has a reason).

Also, that number may have been based on a similar debug build and an SDL and SDL_image with many features. So if you build UQM and its dependencies without debugging info, and compile SDL with only the specific drivers you will be using, and SDL_image with only .png support, you could get a long way. (Although I think SDL loads a lot of stuff dynamically).

Hi cooleyandy,
Glad to see someone else working on this.  I can certainly share my code with you if you'd like; no reason to duplicate effort.  Send an e-mail and I'll give you access to my svn repository.  I personally still won't have a chance to work on this until later in the month at the earliest.  Most of the work that I did was to fix up the pspsdk and SDL port, as I mentioned, so you should hopefully have found that the UQM code just works with only minor build changes.  As for RAM, I did get a crash when exploring a planet that I assumed was memory-related, but didn't explore it further.  I was hoping it was just a simple fix like removing the rotating planet animation.  One problem that I was seeing is that zipped content is preferable on the memory stick (due to small files and a large FAT cluster size), but the frequent unbuffered seeks and small reads in the zip are a killer for performance and it's very slow.  Perhaps a buffering layer would be useful, at least for the zipfile directory.

A few comments:

• reading is buffered. Buffers of 1K are used, but you can change uio_Stream_BLOCK_SIZE in src/sc2code/libs/uio/uiostream.c if you want larger buffers.
  
• no seeking within compressed data is done. And seeking within a single file is a much cheaper operation than opening files. Even more so when you can do random access. And a memory stick is RAM, after all.
• when using a CD-ROM, using zipped content is actually very noticably faster, because it takes less time to decompress than it would take to read the extra bytes. I wouldn't be surprised if using compression would turn out to be cheaper for USB memory sticks too (although it may not be as noticable).

I don't know why reading from .zip files would be so slow for you. You're not repacking the .uqm zip files, are you? The ones provided by us have the .ogg files stored uncompressed. Working with "compressed" .ogg would be a slowdown (and pointless). That's even very noticable on "normal" PCs. Supply '-n .ogg' to zip to tell it to store .ogg files uncompressed, and supply '-8' for maximum compression while you're at it (not '-9', as zip would ignore the '-n' flag in that case). Use 'zipinfo' or 'unzip -v -l' the check the zip files.

The zip contains a ton of tiny files.  Every time UQM tries to open or read from one, it seeks at least to the centeral zipfile directory at the end, then back to where the actual file is.  Individual files may be buffered from UQM's point of view, but it's the initial finding and getting those files out of the zipfile that appears to be taking so much time.  I agree that using non-zipped content would only be worse in that regard.

The memory stick isn't really RAM.  It's accessed serially, and more like a disk in that sequential access is faster than having to keep changing the address (from what I understand from publically available info out there).  On the PSP in particular, benchmarks show that reading in large blocks (at least 64k) is important for getting decent performance.


But now you're talking about a system where the OS provides a generous page cache that will easily retain things like the zipfile directory, and coalesce lots of small requests into larger ones before sending them to the hardware.  The PSP OS has nothing like that, and I think that's the real source of the performance difference here.  A user-level version of a buffer cache like this is exactly what I'm proposing.


I haven't touched oggs yet.  I was seeing times of something like 1-2 minutes just to get past the initial splash screen, almost entirely due to I/O.  Working on this was going to be my next goal once I have time.

Jimparis, my email is cooleyandy at hotmail.com
The latest cvs version of UQM has support for libTremor. I'll take a look into that as a replacement for libogg.
Can't wait to get this game working. I just love the pc version.

Great work on getting homebrew running on 2.01+ firmwares btw.

Not true. UQM reads the central directory at startup, and it will never need to read it again.

UQM never reads from more than one file at once. So opening a file means a single seek. Then everything is read in sequence. (Unless you rewind an .ogg speech).

If it has a constant seek time, I call it random access. I don't expect that seeking is the problem. Reading in short blocks however sounds like a good reason. If the OS doesn't read in pages at a time, each read would mean a new request.

Do you have an URL to those benchmarks?
64k sounds like rather a lot to me btw.

There's a separate buffer for the decompression, 64k in size. Reading files shouldn't be a problem.
I do however have an idea where the performance problems may be coming from (see below).

Do you have console output? Is the delay right at the start or is it spread over all files?

While there shouldn't be any short (low level) reads for reading files, there are quite a lot of them when the zip file is mounted, when the central directory is read in.
I actually have a todo remark somewhere to add read-ahead buffering. I'll see if I can do that somewhere in the next few days. It shouldn't take much time.

Does the PSP have memory mapped IO btw? In that case there would be room for even more optimisation.

no memory mapped io. The memory stick is treated as just a disk io. The psp does not have a memory manager.

OK, I was almost certain I had been seeing tons of back-and-forth calls to stdio_seek, but perhaps I'm misremembering.  I'll have to look at that again when I get time; I no longer have any of my logs.

As far as I am aware the OS does no buffering, so yeah, it would have to be a new request each time.

Sure, try this (http://noisy.compsoc.man.ac.uk/~inomine/psp_ms_benchamark_usb2.JPG).
I believe this particular test was over the USB interface, but I was told that we get similar results with code on the PSP.  I should probably test it for myself rather than just trust them I suppose.

I think it was spread all over.  For example: selecting my next ship in Super Melee would take 5-10 seconds to load.  I do have a console and can attach gdb to the process, although that's over a serial link and slows things down quite a bit itself.

OK, that would be interesting to try out, but take your time, I'm quite busy myself for the next few weeks  :-\

Nope, there's only a rudimentary hardware MMU that appears to have fixed mappings.

Hmm... when a file is opened, the local file header is read. This is done with some short reads. Not many in itself, but it could add up when whole font dirs are loaded.
This would benefit from the same read-ahead optimisation.

I'm not sure if the latest pspsdk has faster stdio operations but I did get a 2-3x speed increase when switching to the sceXXX file io functions a while ago when linking newlibc. Perhaps we can replace the sc2 with those instead for some loading speedup.

UQM uses unbuffered functions (open, read, lseek) that are just simple wrappers in newlib to the underlying sceIo* stuff.  If you're seeing that sceIoRead() is faster than read(), it's a bug, and if you could post an example to ps2dev.org we'll fix it.

fstat() may be slower than expected because we need to emulate it with stat().

Buffered I/O (fopen, fread, fseek) may be slower for other reasons (which may or may not be fixable) but that shouldn't affect UQM.

Ah okay, that makes sense.

In a related note, I was waiting at the loading screen for about a minute and during that minute I had a thought. Since the initial loading is pretty much the same every single time, how about we have a binary file that is a byte copied version of all the loaded items. With a struct and one fread, we can get everything in a couple seconds. Having a max 20meg file doesn't sound too bad to be able to bypass all the loading. Perhaps it could be generated after the first run.

As a side note, I've played around the uio_Stream_BLOCK_SIZE define and the difference is pretty much unnoticible.

While uio (the UQM I/O lib) doesn't use the C stream io functions (fopen, fread, etc.), it provides its own buffered variants which use the unbuffered uio functions (open(), read(), etc). uio_Stream_BLOCK_SIZE specifies the size of this buffer.

As for the binary file that is a byte copied version, I was already planning to profile the files used and order the files in the .uqm files as they are used. With a large enough look-ahead buffer, the effect would be the similar.

I ran a quick benchmark just to verify what we thought.  I create a 4 KB file, then lseek to sequential offsets within it and read 32 bytes each time.  It's measuring how many lseek/read pairs it can do per second.  On my PC it's easily 1,200,000, while the PSP does 3,000.  On a PC, just starting UQM causes at least 20,000 lseeks/reads within the content zip by the time the title screen loads.

What if you don't seek at all, but still read in small steps? I'd expect the same thing to happen, as the RAM itself is... well... random access.

If not, things could easilly be sped up by comparing the new value with the old one when seek()ing, and only passing it to the lower layer when they differ.

If I take out the seeks it's closer to 5,000 reads/sec on the PSP.

About the RAM usage, I ran UQM under valgrid's "massif" tool on Linux.  The PDF here (http://psp.jim.sh/uqm/) shows memory usage versus time.  I started a new game at around 50 seconds, visited a planet at 90 seconds, and left about 10 seconds later without scanning or landing.

Hmm. I didn't expect that. But I guess that offers one easy way to speed things up a bit.
What's the result if you do larger reads?

Interesting graphs. I didn't know Valgrind could do that.
We really haven't done much memory profiling so far.

See http://psp.jim.sh/uqm/benchmark.txt.  The rated speed for my memory stick is 15MB/sec.  As you can see, the PSP hardware is quite capable of reaching that, but only with very large reads.

And rightly so, as it's really not that bad for a desktop PC to warrant wasting much time on it.  For the PSP, It looks like there are some pretty obvious killers in there (like 8 megs of SDL surfaces when you visit a planet); I think Andy is looking into those.

Hmm? The speed on the PC seems to be 127 times higher than that.

Ah, good. Don't forget to send in patches. :)

Sure, but my hard drive unfortunately can't really read 2 gigabytes per second, so that's clearly the result of having a real OS providing a buffer cache. :)  The PSP's kernel is basically just giving direct access to the hardware.

Will do.. I might have some stuff already I could give to you, I'll take a look.

Here's to report recent successes. With the latest uqm snapshot and Jimparis' patches, the planet orbiting part works great now. Now most of the parts are functional now. All we need to do now is to fix the super melee stabiliy issue and we'll have a nice PSP port :-)

The planet orbiting code generates a tonn of SDL_Surfaces to cache the 3D rotating planet frames. I am not entirely sure if it is necessary at all, much less on a modern PC. You could certainly dispose of those and regenerate each rotation frame all the time. That should take the memory consumption down about 6 Megs  ;D
Also, the pre-generated 4x scaled planet surface for the lander is pretty large. I do not know what you can do about that, other than going back to blocky scaling on demand (needs much CPU), or at least making the 4x scaled surface 8bpp paletted, because it really does not need to be 32bpp RGB.

Any new updates? Very interested in this..  :D

Gaa you try and do something original and you find out someone else is already working on it :). I have just installed cyygwin and all the appropriate toolchains sdk and librarys and started to work on a UQM port. Right now I am in the middle of updating config_unix.h. It seems silly now to duplicate the work of what people have already done so I am hoping to help out a little. It seems that some people on here have a farther head start than I on the port so I am wondering if I might be able to have access to the cvs or svn source. Even if it is only read for now.

What I can focus on is getting some decent keyboard-like support and not the rubbish the psp comes with. Anyway my email is meining@fastmail.fm.

well... i've been working on it too though, one of the previous post says theres a working port with only little difficultys running super melee if I read it right.. I don't know why they wouldn't release a demo version or something to keep us happy instead of just keeping us waitng... I'de be fine without super melee for now if I can straight up just play the main game while i'm witing... anyways, should anyone need an extra team member to help out a bit with the port i'm available... I'm not to great with programing.. I don't know coding really.. though unlike a lot of other people here I do infact have "2" development PSPs.. that is 2 PSPs that are firmware version 1.5... ( I just had to get the white one, heh ) so I can try to run any ports that people may have.... besides that I have access to some new 1.5 psp should I some how manage to break mine....  so feel free to send me an email or something if you want... staarx@yahoo.com ;D

I got tired of waiting so I gave it a shot.

You can download it and find more details here:

http://www.dcemu.co.uk/vbulletin/showthread.php?t=20763

-Sharkus

well i posted on the other forum that i would like to take a peak at the source, but i don't really wanna keep checking a forum that i don't goto often so i'll post here.  :D

Where would I get the modified source (or a patch)? (Remember the GPL?)

I haven't forgotten...   :)

The changes were pretty minor.  Mostly just some changes in the fileio (directory handling) and getting the build.vars.psp just right... Give me a day or two to remove all of my debug/hacks and generate some build instructions.

-Sharkus

Hey guys,
Sorry for being slow with this; been really busy lately.  Fortunately, most of what I did to port UQM involved writing and improving SDL and newlib in the pspsdk, so hopefully you didn't spend too much time duplicating work.  You can get my work-in-progress here via Subversion:


The last time I merged with upstream was snapshot 20060110.1200, and you can see the relatively short diffs easily enough:


To build, just run "pspbuild.sh".  Remember to run psp-fixup-imports and strip the resulting binary before transferring to the PSP.

Thanks.  I'll take a look.  I didn't have to make any changes to the PSP SDL or newlib libraries so things seem to have improved since you did your initial work.

I'm also pulling a patch together today for my 001 release for those that want to take a look at my source changes.

-Sharkus

I committed all my fixes back in October, so yeah, they should be OK now.

i dont think you understand. the GPL license requires you to released the source at the time of the program, not days later.
if you didnt want this, shouldnt of released it yet.

and yeah, nice work. its a kickass game

Slightly more stable version posted at original link with source changes included (and very rough build instructions).

-Sharkus

Actually, it doesn't. Section 3.b) of the GPL essentially states that he can distribute the program in executable form as long as he is willing to provide the source code (in machine-readable form) to the executable to anyone who asks, charging only the costs of copying, and makes a written offer valid for 3 years stating this. I'm not a lawyer, but I think this means that writing something like "mail me at foo@example for sources" in the documentation of the binary is enough to satisfy the GPL.

You really should take a look at how I did things in my port.  The majority of your patch is dealing with path issues, which I fixed in a much cleaner way by just providing a modified getcwd() that returns a standard Unix-style path.  Also, I integrated the PSP as a normal cross target into the build system, rather than just providing a static build.vars.  Finally, creating a separate PSP-specific startup function as I did (src/psp/main.c) lets you easily do other initialization and keeps you from having to make PSP-specific changes to the default options in src/starcon2.c. 

If I get a chance I'll see about syncing my code with a current snapshot (some patches have already been applied upstream).

Sounds like some good ideas.  I'll take a look at your changes when I have a chance.  I was not familiar with UQM or PSP development when I started so there is no doubt better ways to handle the changes!

The major issues that still exist:

Using -O2 compiler flag causes instability (Super Melee mode crashes instantly).  -O works much better.
Sometimes UQM runs out of memory (allocating a surface) and crashes.  Normally seen during first pluto trip.
Long load times

If you have any suggestions on the above please let me know!

For our thoughts on memory and load times, just read this thread.

Just took a look throught the thread and took a look at the diffs from jimparis.

jimparis, a couple comments:

1. I seem to be getting pretty good performance running at the standard 222 mhz CPU speed (typically 46 FPS as reported in periodic console statements).  What is the "ultimate" UQM framerate?

2. While load times are definitely too long they are nowhere near the 1-2 minutes as reported in this thread.  Perhaps 0.5.0 is more efficient in how it uses content files.

3. One of the disadvantage of the cwd changes you made is that you loose some flexibility.  For example, for development I use PSPLINK and run the elf file from the host0:/ drive but content/config directories from ms0:/.

4. It looks like most of your changes should integrate nicely.  I'll try to pull in as many as possible for my next release.

222 MHz should be fine, and I certainly wouldn't release my version at 333 MHz, but was just doing that for testing.  I wanted to make sure threading issues weren't speed issues (the thread priorities should be correct in SDL now).  SDL graphics speed should also be faster than when I was working on it, as there has been a lot of work done on the GU acceleration since then.
Wouldn't surprise me.
psplink and hostfs didn't exist when I wrote that code :).  You still might be able to get around it differently, by e.g. mounting the directories to /ms0/ and /host0/ instead.  Or, at least, handle the "string:/path" format in such a way that the same code is used for Windows and PSP.

There have been no significant changes to uio since 0.4.0, so don't expect any speedups from that.

Can you tell me something about the paths as used on the PSP? Is it just that some "string:" is prepended? What letters may occur in that string? Is it handled like DOS drives, with a concept of the "current drive"?

All PSP "drives" are formatted like so:

ms0:/
ms0:/
host0:/
host1:/
etc...

(The length of the drive name is variable.)

It ends up being sort of a mix between unix and win32...

The other problem is that if you access the a string like "ms0:" or "host0:" (without the "/" at the end) the PSP barfs.  This happens with the unmodified code when it walks the directory paths.  The PSP also doesn't like it if you access a path like "/ms0:/"

-Sharkus

Hmmm.  It looks like the instability is caused by not serializing SDL_FreeSurface() and SDL_CreateRGBSurface() function calls.  A crash is especially likely if the SDL_FreeSurface() routine is interrupted by a SDL_CreateRGBSurface().  Most likely this is due to critical data structures being corrupted.

Before I go and do a bunch of debugging, is this non thread-safe issue expected in UCM or should I start looking at the PSP semaphore/mutex implementation?  I looked through the UQM and did not see any obvious mutex or semaphore that would ensure these calls are serialized.

Here's the log (I have found a very consistent failure).  Every time I've seen it crash it seems to fail when transitioning from one screen or another (in this case from the trader character back to the navigation screen).  I believe this is when UQM is handling off from one thread to another, so perhaps there is some overlap when both threads are making SDL calls (bad)?

-Sharkus

Doh! Might have been barking up the wrong tree.  I hacked a mutex in the SDL code to serial the calls to these two functions and the crash still happens.

Serializing?

Serialising as in "forcing to run one at a time sequentially; ensuring mutual exclusion", not "converting an object into a byte stream for storage or transmission".

FYI: I just verified with the PSPSDK folks that the PSP malloc and free functions are not thread-safe at this point so reentrant calls to SDL are bound to fail right now.  This probably explains where the instability is coming from...

Hi guys,
Need any help with the port? How do I get your code?

See earlier posts, they describe how to get both ports

Is this Project still active? Sharkus if it is i am willing to join your team and help your port.

I've ported 0.6.2 to psp. There is still some problems with hangs and performans in melee and planet landing.... Sound, keyboasrd, video is working fine. Any ideas how to speed the things up? I am not C guru, so, any help is welcome.

ok.. after some playing with dependencies i managed to compile 6.0.2 in eclipse. ;)))) got  half mega smaller executable, no slow downs anymore. ;))))

I am playong for an hour now.. so far so good ;)))) i am happy ;)

What kind of tricks were necessary?  Just linking against Tremor instead and such, or did you need to tweak the configuration some way?

My daughter is kinda testing the game at the moment.... I will release patch soon. If there will be people who are interested.

AWESOME NEWS DUDE! Oh man thats awesome. Please release a download soon so i can help test it out :D

Strong interest in seeing this port release on my part.  I cannot wait for updates.

Fedor i would be very interested in testing the new PSP version you have as well.  :)

I just got my PSP Slim from eBay today (it was a used Daxter bundle, and while the PSP itself has numerous scrapes and wear marks, it functions perfectly and cost me 45 US$ less than new). I'm looking forward to running UQM on it (especially since my hx4700 is my last WM device and its controls are very unsuitable for a game like UQM), though I'll need custom firmware first (the memory stick should be magic now; I just need a Pandora/tool/JigKick battery, and I don't want to hardmod my existing battery).

It's been several months since this refined UQM port was mentioned...I hope it's been released somewhere. (I especially hope that I can just drop it in the MS and fire it up in 3.90 M33 without much fuss...)

is it easy to play UQM in a psp w/o experience in coding?

BUMP! BUMP! BUMP!

Fedorus, what happened? Are you alive? :( Your post dates back to February, no updates? No releases?
Hope the guy's ok.

Please reply. Him or anyone who could update Sharkus's version of UQM-PSP.

---

As for the previous poster's question:

Yes, it is.

Though right the only version playable is the one made by Sharkus and it only works on 1.5 (on PSP SLIM you'd have to use DA's TIME MACHINE).

Grab it here.

http://www.dcemu.co.uk/vbulletin/showthread.php?s=&threadid=20763

Newer gamefiles work with the release as well, just rename them to 0.5.0. Hell, even the remixed OST works :).

The latest Subversion source has been updated to allow for much faster load times on devices like the PSP.

I got bored on a long flight to Japan and updated my old patches for 0.5.0  to 0.6.2 with support for the new firmware revs.  I might spend a bit more time getting OpenGL to work (since that is new and interesting to me).  You can grab it here (patch file with source changes are included in zip file):

http://www.dcemu.co.uk/vbulletin/showthread.php?t=20763

Thanks again to the UQM team! 

-Sharkus

P.S. (Sorry about the hacked and slashed makefile.)

Hi Meep-eep,

I just read your post carefully and see that you made some speed improvements in the SVN version.  Originally I thought it meant the latest release version (i.e. 0.6.2)...  Unfortunately, I just finished wading through the code and making my own.   I was able to bring down the startup time from well over 1.5 minutes to under 20 seconds...

Here's a summary of what I did:

1. When using central headers do not read local headers when performing stat operations.
2. Updated fileblock.c with some caching functions.
3. Only read zip data (including local headers) one time, and keep it in the fileblock object.
4. No longer use zip_INPUT_BUFFER_SIZE, and instead use the compressed size to determine how much data to read.

My changes are included in diff form in the release zip posted in the thread above.  Let me know if you want to take a look at them and I can send you the source directly.

Also, I was able to get OpenGL working under the PSP (required a couple minor changes, also included in diff file), but it is  really slow, so I don't include it in my build.

-Sharkus

I created the uio library from scratch, and I want to be able to use it in other projects, including non-GPL projects. So for copyright reasons, I can't even look at your changes (unless you were to agree that I can use and keep using your changes as if they were my own).

That said, if these modifications you made are worth the effort, I might reimplement them myself. I'll comment a bit on all of them:
1. The central headers don't contain all of the information which stat() returns, so this wouldn't be a general solution. But UQM isn't interested in most of that information, so I guess I could make it a compile-time option. An alternative would be to add separate uio_filesize() and uio_access() functions, as those are all what stat() is used for in UQM. I don't think this change wins you anything significant though, as the use of stat() in UQM will usually (always?) precede the use of open(), which needs the local header information anyhow. That said, there will be very little work involved to make this change, so if there is any noticable gain, it's probably worth the effort to make that change.
2. This is what I did in my own speed improvements, which are in SVN. I expect that this change accounts for most of your speed gain. It drastically reduces the amount of short reads.
3. Are you talking about the metadata? That is already only read once, if I recall correctly. If you're also talking about the actual compressed data, that would make 2 pointless. (Or, if you wish, 2 makes 3 pointless; at any rate, doing both only increases your memory usage).
4. I don't think that this will win you much when 2 is in place, but it will cost a lot in terms of memory usage.

I would be interested in seeing what kind of times you get if you just use the SVN version of uio.

The OpenGL fixes are probably good for inclusion in UQM. Could you send me that in a separate patch? Or better yet, put it in Bugzilla?

Hi Meep-eep,

Bugzilla submitted for the OpenGL fix.

In regards to performance...  I actually implemented the cache functionality initially, including sequential read ahead, partial hit detection, etc.  Surprisingly, it did very little to improve the performance, and actually hurt in some cases.  The real performance gains came from making sure data is read once and only once (including local headers) across the entire stat, open, and read sequence, and that reading extra data is completely eliminated (i.e. intelligently only read what is needed based upon compressed file size, and avoid hard coded buffer sizes).  In the end, I removed pretty much all of the fancy cache stuff as it was overkill and not required once the efficiency issues where addressed.

I must say that there was quite a bit of code to weed through...  The codebase must look a lot different today than when this game was running on the 3DO.  :)

-Sharkus

I'm pretty sure that all meta-data is only read once as it is, in 0.6.2. And I think that the same holds for other data, although that depends on other parts of the UQM source.
And UQM in 0.6.2 doesn't pre-buffer; the hard-coded buffer sizes only offer an upper limit to the amount read in one go. There is no extra data read. The SVN version does pre-buffer, but only based on hints given, based on the size of the data needed. So there's no extra data read there either.

Have you tried uio as it is in SVN? If not, could you please do so, and tell me what the effect on performance is?

Edit: I don't see your bug report in Bugzilla.

That's strange.  My wife was rushing me off to a soccer game so I must have clicked the wrong button.  The change is pretty minor:

Just change the first GL_RGB to be RGBA in the following function call (OpenGL ES specified that both formats must match when calling this function):

original:
glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, texture_width, texture_height,
             0, GL_RGBA, GL_UNSIGNED_BYTE, 0);

fixed for PSP:
glTexImage2D (GL_TEXTURE_2D, 0, GL_RGBA, texture_width, texture_height,
             0, GL_RGBA, GL_UNSIGNED_BYTE, 0);

I also hard coded the ActualScreenSize to be 480x272 for proper full-screen operation.

Here are the benchmark results.  However, I only updated zip.x and fileblock.x from SVN into my 0.6.2 build, so if there were other modules in SVN that would have an effect on performance they were not tested:

PSP Rev 001 (Baseline 0.6.2) - 58 sec
PSP Rev 002 (my zip and fileblock) - 22 sec
PSP Rev 002 (latest  SVN zip and fileblock) - 40 sec

-Sharkus

for sharkus
i have tried your port and i can say that this is a great game for the PSP fomat,though i have never heard of or played this game befor it works realy well for this system because it is easy to look at on a small screen like the psp and there is no button bashing needed
BUT...i found a bug , a real nasty one too
i am using a psp fat running 5.00 m33 4 CF
when ever i enter the atmophere of an inhabitted "PLANET" no matter what planet or inhabittants (save for the UR-QUAN base on your EARTHS MOON) the game gives the option of converse or fight. no matter what i chose, the psp freezes then shuts down
for example the ZOG-FOT-PIK home world and ORZ playground and well,ambush planets.....other planets and worlds no problems,hyperspace encounters no problems
i have yet to succesfully visit another races world to accuire allies thereby leaving my human captain alone in this univers and his strugle against the ur-quan
to be honnest i have choke emulators and a gagle of roms and ports galore but out of all this is the one game i can think of that realy makes sence for the psp
and i would realy like to play it further but cannot

p.s  for any one annoyed by this post.......BLOW ME!
i don't care if bug reports go here or not. i've done my testing and thats what i found
peace out bitches :P

Sounds like an out of memory problem.  UQM isn't all that friendly to the PSP when it comes to allocating and freeing memory.  I'll take a look when I have a chance.

-Sharkus

thank you
sorry about the "bitches" i had low blood sugar at 4 a.m.
it took too many hours just to find a place to report this bug ,any other ports need testing i'd be happy to help

peace out man!

You may be interested in reading why we don't like bug reports here. We've even got a FAQ item about it here (http://wiki.uqm.stack.nl/The_Ur-Quan_Masters_Technical_FAQ#I_don.27t_want_to_make_a_Bugzilla_account._Is_it_ok_to_let_you_know_about_my_bug_in_another_way.3F).

Note that in this case you're reporting a bug in an "unofficial" port, in which case the forum might actually be more likely to reach the maintainer than our bug database.

yeah NO!!!
not interested
my only intention was to point out an unseen flaw in someones hard work and to get one messege to one guy and im not joing 20 forums to do so!
reporting a bug should not be this much trouble it took way to long to find some place to report this thing last night and just when i was about to give up i stumble upon this place.wich i did not want to join i might add but had to just to make a post.a post not as incriminating as others !
so befor i unsubscribe here i'm going to look for this said "bug database" if its an easy thing to find i'll eat crow,however if it starts to look like more hassle then the P.S. is for you.

my appologies for attempting to help


p.s  for any one annoyed by this post.......BLOW ME!
i don't care if bug reports go here or not. i've done my testing and thats what i found
deal with it bitches !

yeah... ooh... no.. looks like.. oh.... YOU SUCK!!
no crow for me you still suck and as usual blow me! im out

Ignoring the outraged retard, why is guest posting disallowed here? I remember that t used to be allowed, is it an anti-spam measure, or for some other reason?

It was an anti-spam measure, iirc. As we have a captcha system now, I'll try to reactivate it and see how that works now.

Key piece of info, this game was tested only on firmware version 1.50 AFAIK
I have played successfully on CFW's 3.XX with no issues except for extreme lag.

I Don't have a PSP ATM but getting a new one soon so I can get "back in the game".
Once I get my new one up and running with 5.00 M33-4 I'll see if there is any validity to this dudes "bug"

. : . UPDATE . : .

Well I finally got my psp up and running and have tried UQM with no major issues besides lag and controllability
honestly the control layout is THE most annoying thing about it now.

I've went to a multitude of planets that were "in habited" and "un-inhabited" but with no freezing occuring.
The only freezing i got is when i tried to screw around with the content files :D

But now I'm really happy that this got the port which means i can further mod it to my desires and take it with me  ;D

just made a custom icon & BG for the eboot and thought I would share..enjoy!!

(http://img23.imageshack.us/img23/7767/icon0n.png) (http://img23.imageshack.us/my.php?image=icon0n.png)

(http://img17.imageshack.us/img17/9103/pic1j.png) (http://img17.imageshack.us/my.php?image=pic1j.png)

Wow, that's simple yet incredibly awesome  ;D

Too bad there is no official "box art" for UQM itself so you could use that instead of the original PC box art. 8)

Bug Arne to finish this.. (http://forum.uqm.stack.nl/index.php?topic=2142.msg27571#msg27571) :P

3 years to the day that topic hasn't had a post  ;D

Blood sugar's a bit low today? Eat... crow? lostlinfoot kind of owned, you guys.

I think the point here is, if you act like a retard none of the crew will even pay attention to ya.

From what I see that person didn't get much of a response after he/she started acting like a spoiled child.

I only answered because I don't want anybody else to be discouraged by his/hers rant.