The Ur-Quan Masters Home Page Welcome, Guest. Please login or register.
Did you miss your activation email?
August 14, 2020, 10:02:13 pm
Home Help Search Login Register
News: Paul & Fred have reached a settlement with Stardock!

+  The Ur-Quan Masters Discussion Forum
|-+  The Ur-Quan Masters Re-Release
| |-+  Technical Issues (Moderator: Death 999)
| | |-+  Sony PSP port?
« previous next »
Pages: 1 2 [3] 4 5 ... 7 Print
Author Topic: Sony PSP port?  (Read 58712 times)
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #30 on: January 03, 2006, 01:45:10 am »

Not true. UQM reads the central directory at startup, and it will never need to read it again.
OK, I was almost certain I had been seeing tons of back-and-forth calls to stdio_seek, but perhaps I'm misremembering.  I'll have to look at that again when I get time; I no longer have any of my logs.

Quote
Reading in short blocks however sounds like a good reason. If the OS doesn't read in pages at a time, each read would mean a new request.
As far as I am aware the OS does no buffering, so yeah, it would have to be a new request each time.

Quote
Do you have an URL to those benchmarks?  64k sounds like rather a lot to me btw.
Sure, try this.
I believe this particular test was over the USB interface, but I was told that we get similar results with code on the PSP.  I should probably test it for myself rather than just trust them I suppose.

Quote
Do you have console output? Is the delay right at the start or is it spread over all files?
I think it was spread all over.  For example: selecting my next ship in Super Melee would take 5-10 seconds to load.  I do have a console and can attach gdb to the process, although that's over a serial link and slows things down quite a bit itself.

Quote
While there shouldn't be any short (low level) reads for reading files, there are quite a lot of them when the zip file is mounted, when the central directory is read in.  I actually have a todo remark somewhere to add read-ahead buffering. I'll see if I can do that somewhere in the next few days. It shouldn't take much time.
OK, that would be interesting to try out, but take your time, I'm quite busy myself for the next few weeks  Undecided

Quote
Does the PSP have memory mapped IO btw? In that case there would be room for even more optimisation.
Nope, there's only a rudimentary hardware MMU that appears to have fixed mappings.
Logged
meep-eep
Forum Admin
Enlightened
*****
Offline Offline

Posts: 2847



View Profile
Re: Sony PSP port?
« Reply #31 on: January 03, 2006, 04:44:20 am »

Quote
Do you have console output? Is the delay right at the start or is it spread over all files?
I think it was spread all over.  For example: selecting my next ship in Super Melee would take 5-10 seconds to load.
Hmm... when a file is opened, the local file header is read. This is done with some short reads. Not many in itself, but it could add up when whole font dirs are loaded.
This would benefit from the same read-ahead optimisation.
« Last Edit: January 03, 2006, 11:16:02 pm by meep-eep » Logged

“When Juffo-Wup is complete
when at last there is no Void, no Non
when the Creators return
then we can finally rest.”
cooleyandy
Zebranky food
*
Offline Offline

Posts: 10



View Profile
Re: Sony PSP port?
« Reply #32 on: January 03, 2006, 08:35:05 am »

I'm not sure if the latest pspsdk has faster stdio operations but I did get a 2-3x speed increase when switching to the sceXXX file io functions a while ago when linking newlibc. Perhaps we can replace the sc2 with those instead for some loading speedup.
Logged
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #33 on: January 03, 2006, 05:21:28 pm »

I'm not sure if the latest pspsdk has faster stdio operations but I did get a 2-3x speed increase when switching to the sceXXX file io functions a while ago when linking newlibc.
UQM uses unbuffered functions (open, read, lseek) that are just simple wrappers in newlib to the underlying sceIo* stuff.  If you're seeing that sceIoRead() is faster than read(), it's a bug, and if you could post an example to ps2dev.org we'll fix it.

fstat() may be slower than expected because we need to emulate it with stat().

Buffered I/O (fopen, fread, fseek) may be slower for other reasons (which may or may not be fixable) but that shouldn't affect UQM.
Logged
cooleyandy
Zebranky food
*
Offline Offline

Posts: 10



View Profile
Re: Sony PSP port?
« Reply #34 on: January 03, 2006, 06:06:35 pm »

Ah okay, that makes sense.

In a related note, I was waiting at the loading screen for about a minute and during that minute I had a thought. Since the initial loading is pretty much the same every single time, how about we have a binary file that is a byte copied version of all the loaded items. With a struct and one fread, we can get everything in a couple seconds. Having a max 20meg file doesn't sound too bad to be able to bypass all the loading. Perhaps it could be generated after the first run.

As a side note, I've played around the uio_Stream_BLOCK_SIZE define and the difference is pretty much unnoticible.
« Last Edit: January 03, 2006, 07:00:11 pm by cooleyandy » Logged
meep-eep
Forum Admin
Enlightened
*****
Offline Offline

Posts: 2847



View Profile
Re: Sony PSP port?
« Reply #35 on: January 03, 2006, 11:29:14 pm »

While uio (the UQM I/O lib) doesn't use the C stream io functions (fopen, fread, etc.), it provides its own buffered variants which use the unbuffered uio functions (open(), read(), etc). uio_Stream_BLOCK_SIZE specifies the size of this buffer.

As for the binary file that is a byte copied version, I was already planning to profile the files used and order the files in the .uqm files as they are used. With a large enough look-ahead buffer, the effect would be the similar.
Logged

“When Juffo-Wup is complete
when at last there is no Void, no Non
when the Creators return
then we can finally rest.”
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #36 on: January 04, 2006, 08:38:24 pm »

I ran a quick benchmark just to verify what we thought.  I create a 4 KB file, then lseek to sequential offsets within it and read 32 bytes each time.  It's measuring how many lseek/read pairs it can do per second.  On my PC it's easily 1,200,000, while the PSP does 3,000.  On a PC, just starting UQM causes at least 20,000 lseeks/reads within the content zip by the time the title screen loads.
Logged
meep-eep
Forum Admin
Enlightened
*****
Offline Offline

Posts: 2847



View Profile
Re: Sony PSP port?
« Reply #37 on: January 04, 2006, 10:37:31 pm »

What if you don't seek at all, but still read in small steps? I'd expect the same thing to happen, as the RAM itself is... well... random access.

If not, things could easilly be sped up by comparing the new value with the old one when seek()ing, and only passing it to the lower layer when they differ.
Logged

“When Juffo-Wup is complete
when at last there is no Void, no Non
when the Creators return
then we can finally rest.”
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #38 on: January 05, 2006, 04:01:35 am »

If I take out the seeks it's closer to 5,000 reads/sec on the PSP.

About the RAM usage, I ran UQM under valgrid's "massif" tool on Linux.  The PDF here shows memory usage versus time.  I started a new game at around 50 seconds, visited a planet at 90 seconds, and left about 10 seconds later without scanning or landing.
Logged
meep-eep
Forum Admin
Enlightened
*****
Offline Offline

Posts: 2847



View Profile
Re: Sony PSP port?
« Reply #39 on: January 05, 2006, 07:25:03 am »

If I take out the seeks it's closer to 5,000 reads/sec on the PSP.
Hmm. I didn't expect that. But I guess that offers one easy way to speed things up a bit.
What's the result if you do larger reads?

Quote
About the RAM usage, I ran UQM under valgrid's "massif" tool on Linux.  The PDF here shows memory usage versus time.  I started a new game at around 50 seconds, visited a planet at 90 seconds, and left about 10 seconds later without scanning or landing.
Interesting graphs. I didn't know Valgrind could do that.
We really haven't done much memory profiling so far.

Logged

“When Juffo-Wup is complete
when at last there is no Void, no Non
when the Creators return
then we can finally rest.”
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #40 on: January 06, 2006, 01:13:22 am »

What's the result if you do larger reads?
See http://psp.jim.sh/uqm/benchmark.txt.  The rated speed for my memory stick is 15MB/sec.  As you can see, the PSP hardware is quite capable of reaching that, but only with very large reads.

Quote
We really haven't done much memory profiling so far.
And rightly so, as it's really not that bad for a desktop PC to warrant wasting much time on it.  For the PSP, It looks like there are some pretty obvious killers in there (like 8 megs of SDL surfaces when you visit a planet); I think Andy is looking into those.
« Last Edit: January 06, 2006, 01:24:20 am by jimparis » Logged
meep-eep
Forum Admin
Enlightened
*****
Offline Offline

Posts: 2847



View Profile
Re: Sony PSP port?
« Reply #41 on: January 06, 2006, 01:54:56 am »

What's the result if you do larger reads?
See http://psp.jim.sh/uqm/benchmark.txt.  The rated speed for my memory stick is 15MB/sec.  As you can see, the PSP hardware is quite capable of reaching that, but only with very large reads.
Hmm? The speed on the PC seems to be 127 times higher than that.

Quote
Quote
We really haven't done much memory profiling so far.
And rightly so, as it's really not that bad for a desktop PC to warrant wasting much time on it.  For the PSP, It looks like there are some pretty obvious killers in there (like 8 megs of SDL surfaces when you visit a planet); I think Andy is looking into those.
Ah, good. Don't forget to send in patches. Smiley
Logged

“When Juffo-Wup is complete
when at last there is no Void, no Non
when the Creators return
then we can finally rest.”
jimparis
Zebranky food
*
Offline Offline

Posts: 16



View Profile
Re: Sony PSP port?
« Reply #42 on: January 06, 2006, 02:14:27 am »

The rated speed for my memory stick is 15MB/sec. As you can see, the PSP hardware is quite capable of reaching that, but only with very large reads.
Hmm? The speed on the PC seems to be 127 times higher than that.
Sure, but my hard drive unfortunately can't really read 2 gigabytes per second, so that's clearly the result of having a real OS providing a buffer cache. Smiley  The PSP's kernel is basically just giving direct access to the hardware.

Quote
Ah, good. Don't forget to send in patches. Smiley
Will do.. I might have some stuff already I could give to you, I'll take a look.
Logged
cooleyandy
Zebranky food
*
Offline Offline

Posts: 10



View Profile
Re: Sony PSP port?
« Reply #43 on: January 13, 2006, 03:05:38 am »

Here's to report recent successes. With the latest uqm snapshot and Jimparis' patches, the planet orbiting part works great now. Now most of the parts are functional now. All we need to do now is to fix the super melee stabiliy issue and we'll have a nice PSP port :-)
Logged
fossil
Core Team
Frungy champion
*****
Offline Offline

Gender: Male
Posts: 94



View Profile
Re: Sony PSP port?
« Reply #44 on: January 13, 2006, 10:32:56 pm »

Quote
We really haven't done much memory profiling so far.
And rightly so, as it's really not that bad for a desktop PC to warrant wasting much time on it.  For the PSP, It looks like there are some pretty obvious killers in there (like 8 megs of SDL surfaces when you visit a planet); I think Andy is looking into those.

The planet orbiting code generates a tonn of SDL_Surfaces to cache the 3D rotating planet frames. I am not entirely sure if it is necessary at all, much less on a modern PC. You could certainly dispose of those and regenerate each rotation frame all the time. That should take the memory consumption down about 6 Megs  Grin
Also, the pre-generated 4x scaled planet surface for the lander is pretty large. I do not know what you can do about that, other than going back to blocky scaling on demand (needs much CPU), or at least making the 4x scaled surface 8bpp paletted, because it really does not need to be 32bpp RGB.
Logged
Pages: 1 2 [3] 4 5 ... 7 Print 
« previous next »
Jump to:  


Login with username, password and session length

Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!