The Ur-Quan Masters Discussion Forum

I know many people out there are willing and ready to play net matches… Of course that means we need to start some sort of list. Like

–LightwaveArtist_71.163.81.133 3:00pm-6:00pm EST

Perhaps the sys-admins could set something like this up? The “UQM Net Players List” ???
Unless someone has a software solution in the works. 

The current solution is to look for likely opponents on #uqm-arena on the Freenode IRC network. A list of times is not going to be much help for people with irregular schedules.

I think  that the current big issue really is not a meeting place more than finding times when people are actually awake and are there willing to play net-uqm.

So how about a software solution built in the game? Could a subroutine be written to detect ready players and manage connection? Anyone thinking about it?

Although I know what IRC is, I have never been on it before.  :-\  It all seems like too much trouble.

I just downloaded an IRC plugin for my Webbrowser... Easy enough...

There are plans to implement some sort of multiplayer "lobby" or metaserver. This would probably consist of a server or set of servers to which players connect to get information on active players. From here, UQM would allow players to talk to each other and start games against each other.

This is mostly at the ideas stage right now, so a lot of things may change or simply not pan out. This also means that it's a good time to make suggestions.

Well, the only suggestion I want to add to the lobby is to make it serverless and just automagically log on an IRC channel. Servers have a tendency to consume bandwidth/get hacked/get deactivated by the cops etc. etc. A  built in "IRC" client would work better, and I'm guessing it's a bit easier to code as well. Make it automatically log on the uqm-arena channel for now, with options to change the channel/server...

On the other hand, logging onto IRC like we do now might be easier and less time consuming  ;D

Oh well, dump this one in the "great ideas from someone who ain't gonna write it himself" bin (aka trashbin).

You'd still need a server for the IRC channel.  Unless you're going to have a ultra-packet-spammy protocol (WINS, NetBeui, Appletalk, anyone?  usually non-routable), you can't have a serverless auto-connection/detection technique.  Even if you house your own server (in this case, running UQM on your machine), without a meta-server (same example, the irc daemon), no one would know where your server was...

Adding an IRC client to UQM would be unnecessarily complex, since IRC contains a lot of features that aren't relevant to UQM (e.g. DCC, multiple servers for a single channel, much of the administration and a bunch of minor features like MOTD), which would have to be supported even though they are not required by UQM. Integrating an existing IRC client is likely to be messy and non-portable (and not much easier on the devs); this is the reason for the current "bring your own IRC client" policy.

In contrast, a custom protocol can easily be extended to transmit custom information (e.g. fleet setup) relevant to UQM netplay, in order to make it easier to find opponents.

From a bandwidth point of view, it makes sense to transmit player-to-player communications (messages, gameplay) directly between players and use the server only to find players and for global messages (administration, general chat). This should keep metaserver traffic quite low. Load balancing between multiple servers can also be built into the protocol, if needed.

From a security and legal perspective, a simple and specialised metaserver is likely to be less of a problem than e.g. a public web-based discussion forum; the UQM metaserver does not need to retain non-admin messages at all nor accept large amounts of traffic; this makes copyright violation difficult and abusive messages should disappear quickly (unless archived by the client, of course). Similarly, it is usually easier to ensure the security of a simple system than a complex one. This makes the UQM metaserver quite resilient against most threats on the Net.

Based on the above, running a UQM metaserver on the server providing this forum does not seem unrealistic, although load considerations may make a seperate machine advisable.

Aah. I was under the impression that writing up an IRC client that just automagically logs you on a specific channel on a specific server (#uqm-arena on freenode for example) would be easier to conjure up, thus eliminating the use of an own server.

My bad.  ::)

So I've been tossing around ideas about how to code a reasonably good "master server" + client code for publishing and finding UQM net games.  Here are the notes so far:

Right.  So what's the required feature set? 

Assumptions (please correct if any are wrong!):
* UQM's netplay uses a peer-to-peer model with two peers, rather than a client-server model.  Thus player A needs to know B's IP and port, and B needs to know player A's IP and port. 
* Moreover, currently A needs to be waiting for B to connect, and B needs to be waiting for A to connect.  (is this correct?)
** This is kinda strange.  A suggestion is to have A wait for a connection, then when B connects, we're done.  A and B talk via the same port, at this point they know each other's IP address.  If for some reason A needs to know B's port, it can be the first item sent.  This means only A needs to have it's port open for incoming connections: usually firewalls can talk freely once an outbound connection has been made.  (is this correct?)
* Also there's currently a bit of quirkiness in that you have to determine who is the top player and who is the bottom player (is this correct?).  I recommend that in the future this gets phased out in some way.  Since it's peer-to-peer though, there has to be some way to distinguish between player slots.  My first instinct is for one side to just force the issue:
* There is currently no way to host a game server where you are not playing but others can join (ie a "dedicated server" so two other players can play; is this correct?). 
* I'm also assuming here that even though you must currently specify all these details on the command line, in the future it will be possible to change these parameters at run-time, and they will change many times in one program run to play complete games on different servers. 
* An integrated solution for finding games online would be cleaner than using some outside tool to launch the game via commandline parameters.  For example, GameSpy had such a tool.  Writing (or using) an outside tool could work as an intermediate measure.  From my experience, temporary solutions end up being permanent more often than not.  Google "lazy as a virture."  :)
* This solution requires the creation of master server software.  This also requires a dedicated server box set up someplace.  If money's an issue, I recommend a "Slug (http://www.nslu2-linux.org/)" running on someone's dedicated home, business or school network and using a free dns service.  Or paying for an actual DNS entry. :)  Note that sourceforge, berlios etc are not candidates for running such a server. 
* This solution should not require any other external software or service.  As in, you don't need a seperate ping daemon / web server / etc running. 

Required:

* (REQ 1000) Players can start a game server to host a game with themselves as a player.
** Starting such a game server advertises its availabilty to a master server. 
** By virtue of sending this message the sender's external IP address is revealed
** The sender may also have to specify which port to use. 
** This request also specifies which player positions are in use and which are available. 
** For future use, "max players" might be better than available player positions.
** For future use, we might specify the "map", "gametype", existing players in the game (enumerated by their names).  Console games typically have a "matchmaking service" whereby they don't show a list of servers in the UI; instead it picks a server that matches the player's chosen criteria.  Here is where such clues are given. 
** This master server may also return some info from the request. 
*** For example, a generated unique public game server ID.  This is useful for distinguishing between multiple servers behind the same network, using the same external IP and port.  May also be used for updating the server's stats. 

* (REQ 1001) Players can get a list of available game servers by querying the master server.
** Players can send in a request to the master server.  For now, it doesn't look like there's much to specify in this outbound request.  The master server doesn't really care about the player's IP, port, player position etc. 
** Should Filters be done on the requester's end or the master server's end?  So if the player specified in a UI someplace that they do not want to hear about full servers or something, do we make that part of this outbound request (at the expense of the master server's CPU time) or do we do that on the
** The return from this request is a list of servers.  For each, we have:
*** Public IP and port
*** Number of players currently in the game
*** Number of available slots, and for each, their "position."  This position should be phased out at some point.   
** Curiously, the server's ping is *not* returned here.  The master server has no idea about the ping time between anything; instead it's up to the player's software to ping the game servers. 

* (REQ 1002) Game servers can update their stats on the master server. 
** Same as REQ 1000.  The main reason this exists is to prevent the server from reporting multiple instances of the same server (by using that unique server ID).

* (REQ 1003) Ping from one game server to another. 
** Useful for weeding out bad game candidates.  Presumably the UI can sort a game list by ascending ping time. 
** This should be done by UQM code. 

* (REQ 1004) UI update for finding online games.
** Think Quake 3's browser list.  You can sort and filter results. 

* (REQ 1005) UI update for creating / hosting an online game.
** Presently there's not much to adjust: your port is about it. 
** For future expansion, this should be made a seperate UI page.  There may be things like choosing maps, gametypes, specifying max players, etc.


Sounds Great!  Now what technology is appropriate here?

Raknet.  This is a network library, which includes among other features, a fairly quick and easy way of implementing a master server and client code. 

This lib used to be available as GPL code; now it's an "Attribution-NonCommercial 2.5" creative commons license.  This is GPL-compatible, but it adds the additional restriction that the resulting binaries must be non-commercial.  Now it might just be possible to work something out with the author such that we pay for the commercial license, but we still end up with a GPL-compatible license... I dunno.. this kinda licensing makes my head spin.  Or maybe it was the Sambuka.  :)

Anyhow: great tech, but licensing will have to get sorted out. 

Use SOAP calls.  Simply put all the stuff talked about above into neat XSD-formatted SOAP requests, use some GPL-compatible lib to generate your code from the XSDs, and Bob's your uncle.  Or Toy. 

This is actually a fairly clean solution.  It means adding a build target to generate this code, plus adding the XML parser libs etc, but once that's out of the way this is really cheap maintenance, even if we add crazy features like easy matchmaking services like XBox etc have.  I was actually going to try doing this using a Java client front-end that just calls the commandline + a Java server, just because it happens to be familiar. 

Note that we are *not* going to hand-roll our own XML parsers etc.  *sigh* many a programmer has gotten overzealous and coded such solutions, only to pull out their hair when the spec changes...

Hand-roll our own code.  Nope, not a good idea.  Aside from the idea of which slots are available and the fact that the two players must know each other's IP addresses, this is definitely not a unique problem.  Someone's bound to have code for this, and this is what open source is for! :)

Go find someone else's code that already does something similar and use it.  Someone want to spend a few minutes doing a google "site:sourceforge.net ..." so see if there's something already out there? :)  There has to be an open source "gamespy" equivalent.

Ok, so are those assumptions even slightly right?  Is this the required feature set? 

Not entirely true. Hosting player works as a server and gets opponent's IP and port from the incoming TCP connection.

No. See above.

This is essentially how it works, except TCP keeps track of the client's IP address and port on the server, so UQM doesn't need to know that.

Right now the game works fine irrespective of who's top and who's bottom; the top player on one machine can be bottom on the other and vice versa with no ill effects.

I haven't tested this recently, but it should be possible.

Hello? UQM already has full in-game netplay configuration as you describe. What version are you using?

I believe Meep-eep is already working on an integrated custom solution. Finding a decent server shouldn't be a problem (unless UQM netplay becomes too popular for Meep-eep's bandwidth).

This matches my conceptions so far.

This makes sense.

I don't think designing options for possible future use is a good idea. Better to either force everyone to use the latest version (easy but annoying; current design) or make the protocol backwards compatible so that unrecognised messages can be ignored (trickier).

While having multiple servers on the same port would be possible (as long as their clients use different IP addresses and/or ports), making this work properly would require some major changes to how UQM handles network games. There is no particular reason to do this, either, apart from trying to circumvent firewalls.

Of course it cares. How can it sends lists of these to people otherwise?

For large data sets, filtering on the server is going to be a lot faster than transmitting everything, from a CPU and especially a bandwidth point of view.

While it's kind of pointless to calculate ping from the metaserver to the players, I'm a bit concerned about generating an amount of pings proportional to the square of the amount of players. Based on the behaviour of similar software like BitTorrent, contacting 100 servers should be OK.

Also, instead of using ICMP ping, it would be a good idea to connect to the server and then measure round-trip time for a message through this connection.

Furthermore, I think it would be a good idea for the metaserver to track stuff like names of each server or defined fleet.

Is this necessary? I'd just leave a TCP connection open between the server and metaserver; when it's gone, the server is gone.

See my comments above.

Sounds good, but what do we want to filter and sort on? Fleet size/points and "ping" spring to mind. Searching for a specific player/fleet name sounds useful, too. Anything else?

Already done, more or less. Again, the "future expansion" part makes little sense in the context of UQM (although some minor rule changes have been discussed).

Looks to me like a bunch of wrappers and frameworks that don't really do anything for a lot of stuff that we mostly don't need (although the connection security bit sounds impressive, assuming it works). The way UQM's designed, most of the useful stuff that RakNet does (object replication, remote function calls, compression, routing, et.c.) is simply not relevant. Adapting something like this is not necessarily easier than rolling your own code.

Considering that we have little reason to attempt to integrate with anything else that uses SOAP, I'd classify this as a no-go; you're just adding complexity to the communications protocol for no good reason.

True, but is this problem so complex on the inside and simple on the outside that we can easily reuse someone else's code written for similar, but not the same, requirements? I doubt it.

Most of this stuff is pretty simple to implement, really. It's designing it that's the hard part; once you have an unambiguous and sufficiently detailed specification I don't think it'll take many days to get the whole thing running.

Checking that someone else's solution does what you need and adapting it to fit your needs can easily be time-consuming. Finding it in the first place is also kind of hard. There are tons of open source games that have code to co-ordinate multiplayer games, but they all have different requirements and thus do different things.

Before I respond to your remarks, I first want to say that I'm a bit irritated that you go ask questions without doing your home work first. Had you started just a single net game in the released version of UQM, you would have known that top/bottom position does not matter, that one side awaits a connection while the other side tries to connect to it, and that the network parameters can be set from inside the game.

You seem to be guided in your design by the possibility of having fights involving more than 2 players. This is something which is not likely to happen. But if it woud happen at some point, the amount of work that would be "wasted" by not supporting it from the beginning would be small compared to the work needed to be done to support more than 2 players.
And doing things incrementally would hugely increase the chances of anything getting completed at all.

So let's throw out the server-client design for now. What's left for the "meta"-server (you can leave out the "meta", as there wouldn't be any other level servers anymore) is to pair up players.
What I am thinking of, to start with, is a simple list of players, and a list of "challenges". A player could challenge another player, or do an "open" challenge, to which anyone can respond. Some info about the type of challenge ("100 points no doubles") would be included. Some chat ability would be nice too.

Then it's up to the two players themselves to configure their game. A post-connect, pre-fleet-setup game configuration would come in handy there.


More to follow later, I've got to go now.

If you want to look at GPL code that already has a lobby, chat and such you can take a look at TA-Spring (http://taspring.clan-sy.com/) for ideas.

Continuing my note from earlier.


The pre-game configuration and the lobby would be two separate issues, which can be implemented individually.


One approach to doing the lobby is to have it done almost entirely web-based. Finding an opponent and even negotiating the game type can would be done in the browser. When the players are ready to start the game, the necessary information is sent to the player's UQM program. This could either be done through a minimal webserver built into UQM (using existing libs), or (less portably) by registering an "uqm://" url with the desktop environment. For an example of this method, you can look at The Bub's Brothers (http://bub-n-bros.sourceforge.net/) (or "Bub-n-Bros" or "Bub & Bob Brothers" depending where on the site you are looking).

This method has a number of big advantages:
- The web part would have no dependance on UQM code. This means that whoever implements it does not have to know anything about the UQM code, and can implement this in any language he/she chooses, which in turn means that there might be more people interested in creating this, and I could spend my time on other things (like the long-expected resource system).
- It's very easy to make the gui, as it would be done entirely using standard web languages.

Disadvantages:
- When something on the server end changes (e.g. someone says something in the chat) it would have to be reported back to the clients, which is tricky using javascript. The easiest solution would probably be to have the client continuously polling for changes. But it should be noted that I am no expert in web development.
- You would be doing your setup in an external program (your browser). This is not very slick. And it would complicate things for those who want to have netmelee support on mobiles.


Hosting the server would not be a problem. uqm.stack.nl can handle it.


Determining the network latency between clients should not be a problem as long as we have few players. If it ever gets really big we can split up the players into "rooms".


When there is a lobby, you don't want to pick your opponent from a list of network names, so there would have to be player names. You could use any available name at any time, but it should also be possible to register a name so that others can't misrepresent you.


If there are additional options introduced that affect the gameplay, you can retain backwards compatibility by -- instead of checking for the opponent's version -- having UQM reject any option it doesn't recognise (like telnet options, for those to whom that means anything).
There would still be required UQM updates, but at least their number could be minimised this way.


I agree with Novus that sometimes reinventing the wheel can be the easiest solution, but it may not be necessary, and it isn't an all-or-nothing thing.

Well, first, I really appreciate all the responses to this thread. I don’t know what solution is best. I agree with the statement that “temporary solutions often become permanent ones” I know there is someone out there ready and able to code a solution. I thank Novus and Meep-eep for their effort in responding this thread. All I can do is offer money or CGI to the person or who makes my dreams of escorting several unsuspecting SC2 players to their deaths, a reality. Best of luck to you all,
        Metallic Medium CGI
                                                                 

I'm very interseted in playing against other people...but the channel is always empty when i get in there.

If it's empty you're probably using the wrong channel or the wrong network (which means you're actually creating a channel).
The correct channel is #uqm-arena, and the network is Freenode, and you can use irc.freenode.org to connect to.
There are about 10 people on average nowadays, but frequently many are idle. Some are just waiting for an opponent, so don't wait until someone invites you to play; just ask whether anyone wants to. If noone responds, don't give up, just leave your IRC session open and go something else for a bit and tell your IRC client to give a signal when someone says something.

That must be it. I'll try again this evening. Thanks for the info.

I don't see a 'freenode' network in the list. I see 'freeworld' and 'freedomchat'... If I pick it as a standalone server is that a problem?

edited to add: apparently, no.

"Open Projects Network" (OPN) is the old name.

Hi UQM team. I would love to know what the latest is with the netplay lobby development (if any). Is this something you guys are doing right now? Is it something that can accommodate contributions from others before its completed?

No work is being done on the lobby atm.
As it is something which is only marginally connected with the game source, my hope is that someone else steps up to work on this, while I work on other things that netplay needs.

I suppose I could take a shot at it if we can agree on a feature set and division of labour (i.e. where does my part end?). We've gone over the basic idea already, but here's my take on how to do it:

Each player connects to the server on entering the lobby (from the SuperMelee menu) and gets a list of available challenges on the server (IP address, player/fleet name, fleet composition). A connected player can:

• Add (or, if he already has one, remove it) a challenge (which all the other players are told about).
• Respond to a challenge, in which case the other player is contacted directly and asked to confirm; if affirmative, both players are removed from the lobby and returned to the SuperMelee menu connected to each other, as in the existing system.
• Exit the lobby (drop the connection, removing the challenge if any).

The above is pretty much the bare minimum lobby functionality. The obvious next step to me is to add a public chat to the lobby and a private chat between connected players, allowing for more detailed discussion of fleet setups. At a minimum, this would require the option to write single lines of plain text and send them in between the messages sent to/from the lobby or other player.

As far as connection setup goes, it should be possible to estimate a good value for netdelay from round-trip time between the players in the game (this could then be tweaked at connection confirmation time).

I haven't worked out the protocol in detail yet, but I'd probably do it as newline-separated textual commands over TCP; easy to implement, test and debug. I think I'd do the server in Java, where insufficient validity checks on net traffic are more likely to get the offender hung up on than result in code injection.

Am I making any sense here? :-\

It would also be awesome if non-players could connect and watch games.

I have a a bunch of design ideas kicking around but because I have 0.01 computer knowledge I don't know whether they're plausible, improbable, or just laughable.

I will take a chance and float some in the anticipation that I get laughed out of UQMville

The basics

• A prominent link from the PoNaF home page
• A lobby that works off the same log-in details as the UQM forum. This way people retain the same identity in both the lobby and the forum(s). Perhaps if you're already logged in to UQM forum you don't need to log-in to the melee lobby.
• Once you've entered the Netplay lobby you're confronted with a list of 'arenas' where battle can take place. You know how battle takes place around a planet? Same principle here! The arenas would be labeled Pollux II, Jupiter, Beta Brahe IA etc. Ideally the actual planet graphic could be used as the planet around which battle takes place. That would be neat!
• Ability to join arenas to watch a melee - with no limit to how many people can view. Imagine everyone watching the final of a league game - cool!
• Maybe even the ability to comment during game play in the arena. Full crowd atmosphere.

Tournament capability (my baby)

• Okay, this might be far fetched:  I would like this lobby to magically administer tournaments and record results and statistics
• Automatic registration of league entrants and updates of league results and tables
• Ratings system for players
• And other such exciting features

The underlying mechanics, which have been discussed on this thread are not something I really understand at all.

My aim is to persuade my brother to have a crack at this, if no-one else is. Clearly I need to be made aware of the varying degrees of impossibility for the above!

My initial design is purely intended to aid in setting up games, not keep track of any statistics or so. However, some form of authentication would be necessary if any meaningful tracking of player performance is to be added. As the netplay server would probably run on the same machine as the forum, using the same user database would save the trouble of keeping track of separate user accounts (and allow us to use the forum's features for much of the password administration).

If we have large amounts of users, splitting them up will become necessary, in which case dividing them up into channels or arenas or realms or planets or whatever becomes necessary (and the "planet" theme actually sounds kind of cool).

Wild idea: use the starmap to select a planet when creating a new arena.

Melee recording and observing are important ideas that should definitely be taken into account (why use a complicated video recording system when you can record the control data transmitted by the netplay system more or less for free?). Is anyone working on this?

One way to approach this is to extend the lobby system to include games in progress to which one can connect as an observer (some sort of peer-to-peer chaining of observers would probably be nice for major events; even though the amount of data is small, thousands of small packets per second can still clog up your link pretty bad, and latency is not really an issue for an observer).

Support for voice and text chat during game sounds like a good feature, although some sort of ability to control whom you listen to would be necessary. This gets complicated to use (and implement) fast. I'd start off by allowing players to talk and leave observers (with the possible exception of an official commentator ("... And Elvish Pillager afterburns right into the planet. Again. What do you have to say about that, EP?" ">:(!")) mute.

If the server keeps track of players, keeping track of game results (and even replays!) would be relatively easy, as would organising tournaments. A tournament or league would merely be an arena with special rules on who plays who next and a few lines of extra code to handle scoring.

I've already said a lot about this earlier in this thread, but  here are a few more comments:
I agree with a TCP text-based protocol; easy to implement, debug, and maintain. I already had the start of a protocol written down; perhaps you're interested in seeing it.

As for the "where does my part end?", for a start, the lobby could be a separate program which starts UQM. Then you wouldn't have to touch the UQM code at all (if you want), and we'll add a few command-like options to start the game in Netplay mode. And as I mentioned before, that "separate program" could just be something which communicates between the game and a web server, so that the entire lobby could be done in the browser. (It doesn't have to work like this; this is just one relatively easy way).

As for Java, it would not have been my first choice, but if you're making the lobby, it will be your choice. I'm sure we can install java on uqm.stack.nl.

Using the forum login for authentication is an interesting idea, which I hadn't considered before. But in Open Source spirit, it should be possible for anyone to run a server, and then this method would suddenly be a lot less interesting.

As for rooms; star/planet names might be a nice default for room names, but it would be better to make them user-definable. If you have a contest, a room named "Kohr-Ah Cup" beats "Beta Copernicus IV".

Cool ideas. While you're posting your wish list, I wish that your "arena" idea included not a single planet, but a system of planets, spinning in a 3D starmap where your fleet can be strategicly moved so as to allow you force or avoid encounters to give yourself a chance. Like chess in space, only when the ships meet they fight.

An optional starbase where you could deploy new ships would be cool too.

...And then you end up with the game that Spathi captain are playing...

Comparing notes would be a very good idea.

I distinctly dislike half measures, and would prefer to construct something with a practical user interface that doesn't compromise the portability of UQM or add unnecessary dependencies. I'm not too familiar with the widget set in UQM, but I believe that, as long as the user interface is kept simple enough, it should be a satisfactory approach.

This is also why I'm eliciting discussion about the user interface here.

Considering what I'm comfortable with, the options are (in descending order of preference):

• Java application (note that designing to allow compiling to native code with gcj is an option)
• C application (input validation and memory management is going to require a lot more care)
• PHP with a database backend (I prefer not to write long-running applications in languages with unclear memory management)

Using Java or C makes it easy to implement a protocol based directly on TCP; PHP would integrate nicely with HTTP. The former would minimise the client-end stuff (no need to rely for an HTTP client), the latter would make web integration easier to add.

What was your first choice, Meep-eep?

It doesn't work well as a sole authentication mechanism, but it would have the advantage of avoiding confusion between members of the UQM community as seen on the forum and UQM players with the same name.

Obviously.

I think my lobby system below might address your first point.  The star map to select an arena sounds very funky


Novus, it sounds to me like you are the man. I have expanded out my initial ideas below:

The Basics

I envisaged the structure to look as follows:

PoNaF
     Netplay website
           Rules/FAQ
           Stats/results archive
           Hall of fame/Melee profiles
           Lobby system
              Lobby 1
                 Arena 1 (Beta Copernicus IV)
                 Arena 2 (Pluto)
                 Arena 3 (Beta Leporis II)
                 Arena 4 (Alpha Squidii VI)
                 ...Arena 50
              Lobby 2
                 Arena 1 (Gamma Ceti IA)
                 Arena 2 etc

Any more than one lobby would only be required if this became amazingly popular. Currently there’s a maximum of 5 games going on at once uqm-arena. I foresee one lobby managing up to around 50 battle arenas. So jumping up to multiple lobbies assumes a ten-fold increase in popularity. Not likely any time soon, but would be good to have capacity for this. Or it may be that initially this extra layer will be cumbersome and not required.

Hopefully the uqm-forum log-in can be used to get into NetPlay lobby. This would keep identities consistent. People already do this but it would be nice to validate it and have forum avatars for NetPlay.

Chat could take place inside lobbies OR arenas. Arenas would feature the host and guest combatants plus any spectators. I am quite excited about the possibility of having crowd atmosphere at melees. Not so fussed on voice chat as it may disturb the combatants. There’s no reason they have to look at the text chat window though.

Host combatants are responsible for establishing battle rules:

• Official/unofficial game (rated or not)
• Max points
• Max ships per type (dupes)
• Max time limit (is a timer possible?)

I would say that an official game must have no duplicates and between 150 and 225 points. Official game results would be recorded in the results archive.

The results archive would record the following:

• Players (winner/loser)
• Date
• Tournament/stage (if applicable)
• Margin of victory
• Fleets used/ships left

The above would generate some pretty interesting stats on players (How  often does Galactus use an Orz?). Anyone think of any other things to include?

Okay, so it’s sounding like a pretty involved database already but I’m sure it’s doable. I’m happy to take responsibility for this part of it.


Tournaments

Okay, this is where it gets fanciful.

I want people to be able to create their own tournaments. Level one tournaments will be ones agreed by the community to be premier – created by a Lobby Administrator. Level two will be those casually created by users (can still contain official melees).

Once you’ve selected this you choose the following:

• Type
             League: Select number of round robins (any more than one is unwieldy really). Select number that go through to playoffs.
             KO: Tennis style (1 plays 32, 2 plays 31 etc). Seeding based on player rating
             Hybrid: think Soccer world Cup (pool play then KO). Could be tricky to implement if a nice number of participants is not registered
• Maximum participants (max 16 for league, 64 for KO)
• Dates/times on which tournament takes place – uh oh, this will require a calendar system! First date is set in concrete – other dates provisional on number of entries. Clearly if you get only 4 entries for your KO tournament only one day is required. However, if you get 16 entering your league they you will need round 6 dates. After entries close, the provisional dates become concrete. You cannot schedule a tournament to clash with someone elses.
• Registration: Default open - immediately. Default close - 10 min before first melee
• Rating restriction (e.g. only accept entries from those below a certain rating for a newby-friendly tournament)

Novus, I will write a proper document up for this with even more detail.

My biggest reservation is latency. I know nothing of solving/improving this probem how the above ideas will make it worse.

I didn't have any plans to implement the server myself at all. As I mentioned before, it is the perfect occasion for someone else to contribute, as it doesn't require knowledge of the UQM code, or of C even, and I had enough on my plate already.

I had thought about it a bit, and my thoughts would go to something light-weight, web-based. You would get a lot for free that way, as the layout could be done entirely in HTML, and not much interaction with UQM itself would be needed (only to start the game). I thought it would be a lot more likely to get finished that way.

But a client built into UQM, with a non-web-based server would be nicer. If I were to do that myself, I'd probably either make the server in C, based on the network stuff I wrote for UQM, or in Pike, which I'm rather partial to. The reason that I said Java wouldn't be my first choice is that I still think of Java as rather resource-heavy, and the computer serving uqm.stack.nl serves many other sites. But I guess that idea is outdated, especially if you compile to native code. And uqm.stack.nl is a decent machine nowadays, which can spare a few MB.

As the lobby would be full of activity (chat mainly), it's probably not such a good idea to start a new process for each command, which makes a CGI script a bad solution. So for a web-based solution, I'd be thinking of putting the static data on the uqm.stack.nl webserver, and letting the dynamic stuff be handled by a dedicated process with a minimal webserver built in, to which a persistent connection is maintained. I guess something like Java EE might be used, but I have zero experience with that myself.

How would the web browser communicate with UQM in a way that wouldn't become a tech support nightmare? Most web browsers are set up specifically so that web sites can't start external programs without the user setting up file associations. Building the UI into UQM is the way to go, simply because problems getting the web thingy (which we don't control) and UQM to work together don't exist.

Even when using Sun's VM (with the usual just-in-time compilation), the overhead is pretty much just a second or two at startup and about 20 MB of memory for VM and libraries; once the system is running, it's quite efficient. I'm not familiar enough with Pike to comment on it or write anything in it.

Well, we agree on the basic design; one single server process is definitely the way to go. One major question is data storage. While chat messages are in through one ear (or network connection) and out through the other(s), some sort of persistent data storage (maybe a database) would probably be nice for things like league states (although just keeping everything in memory would be very fast and easy to implement, I'd hate to be the one to explain that, due to power failure, all active leagues have been reset). Putting stuff like that in a database would probably be a good idea in the long run (you already have one for the forum, right?).

The easiest way to group things would be to divide the lobby into rooms, each of which would have a chat channel and rules determining who you can play against and how the server tracks the results (free-for-all: challenge anyone, no tracking of games; various types of leagues and tournaments: results tracked by server and only opponents allowed by league/tournament type shown, registration allowed as determined by tournament type and organiser and independent of which room you are in at the time (i.e. you can be entered in several leagues at the same time and  play one-off games without losing your place in the cup)).

The minimum client functionality is then: login/logout, select room (getting list of possible opponents), receive chat (and private) messages and opponent status updates and exit room, and, of course, start a game against an opponent (reporting status back when game over, if necessary).

The lobby UI would be split into a room/opponent list and a chat window, essentially replacing (or working as an alternative to) the current netplay connect dialog (in other words, you could set up your fleet before or after entering the lobby).

A simple way to do this is running a mini-webserver in UQM itself, and then in the web page send a request to some port on localhost.

I think 20 MB is still quite a lot of overhead. Still, it is acceptable.
Pike is just my interpreted language of choice. You can substitute Python or Ruby or *shudder* Perl or something else.

I think you should focus on chat and initiating games first. But a MySQL database on uqm.stack.nl is not a problem.

I was thinking along those lines too. A bit like IRC, but without making the same mistakes that they did: a specified character encoding (UTF-8), allowing spaces in various places where it is not allowed in IRC, extensible design, no lost messages, and no combination commands (like adding a message to "/leave"; if you want to say something, say it before you leave; it's just used for spamming now).

I have two unfinished documents regarding the protocol I had in mind. But feel free to do it your own way. Files: protocol (http://www.stack.nl/~svdb/sc2/lobby) and rationale (http://www.stack.nl/~svdb/sc2/lobby-rationale).
I tried to keep it simple; the client is always in control. It decides when to check for remote events; the server doesn't send anything on its own. But no polling; a client can initiate a "wait for updates" state. Also, each confirmation of a command is accompanied by a result code that includes whether there are events waiting, for increased responsiveness.

While these have a lower start-up memory overhead, I'm not sure how well they handle long-running processes. A few years back at least, Python had issues with freeing memory; this may have been fixed already.

Agreed.

No arguments here.

I was thinking pretty much on the same lines, except that I'd intended the connection to be in event listen mode all the time (i.e. the server can send an event any time after authentication except in the middle of a response to a command). In fact, my way is roughly what you get by wrapping almost every command in an ABORT and a (repeated) LISTEN. You might get a few unnecessary chat messages before leaving a room (for example) instead of a simple "messages available", but there's a lot less flow control messages going back and forth adding extra states to the protocol (and forcing the server to buffer events possibly indefinitely instead of until it has finished responding to a command). I'd prefer to design the protocol so that the server doesn't suffer if the client is behaving badly.

Memory leaks are bugs, and one interpreted language is not another. And even if these bugs are not fixed yet they might not even show up under our usage, or may be easilly worked around.

My protocol only has two states, sending, and listening. All the state information you have to keep is in which one you are. And in return this gives you the ability to know exactly what kind of information to expect, which makes the client very easy to write. It also allows the client to determine the rate that the messages come in, and even tell the server ad hoc to discard specific messages without having them sent over to the client.

And yes, it adds a few extra messages to the protocol, but only at a time when there's nothing to report. As every reply includes a status of whether there are more messages available, you don't even have to LISTEN if you know there are events available.
Though maybe adding a "count" argument to the POP command (or add a POPN command) to get multiple events at once would be useful, to do away with the repeated POPs (although you can safely queue POPs).

Yes, eventually you have to start discarding events, if they aren't POPped. But this approach is also an advantage, as it allows for clients that have been momentarilly disconnected to catch up on messages they would otherwise have missed, something which I wish IRC had.

Here's something for you to look at
http://homepages.paradise.net.nz/~ejrh/uqm/lobby.png (http://homepages.paradise.net.nz/~ejrh/uqm/lobby.png)

Just before you all go "oh my god he's gone ahead and built a fully functioning netplay lobby!", there is no substance behind that image at all.

My point was that little-known languages have been subjected to less testing and are thus more likely to have bugs in their implementations. Java, for example, is too widespread to have (many) major problems in core functionality.

I'm not saying that having a send/receive distinction is a bad idea, I'm just saying that I didn't think of having one at first, because it isn't necessary. It may be desirable, however, and I understand your reasoning for having it.

The overhead may be small in practice; TCP has to send acknowledgement packets anyway; if I remember TCP correctly, they are often delayed to allow applications to add data to them.

Assuming the protocol is built on TCP, having the server transmit events freely means that TCP on the server is going to have to buffer the remaining events (or the server performs a blocking write or a failure to write without blocking). If events are requested one by one (and event sizes are limited to less than the TCP implementation buffer), guaranteeing non-blocking writes (from the server app's point of view) becomes possible, which is a useful property in the server (at the expense of having to buffer messages in the server).

Hmm. I think you've convinced me.

Python and Perl are hardly little-known languages.

Good point; I didn't think of that.

Another good point.

Nice mockup. Not quite what I was thinking of though.

• Naming rooms after planets is confusing. Just let the creator of the room decide on the name.
• I would make it tabbed, with a tab for each room the player is in.
• On a tab, you'd see a list of players, with indication of whether they are in a game, idle, and what their ping time is.
• There would be a list of challenges, with challenges to you specifically on top, and challenges to an entire channel below that.
• The chat window would occupy a large part of the screen.

Compared to C, C++ or Java, yes (academia seems to be moving to Java as the primary language, with C and C++ still holding their own in performance-critical or low-level stuff). Compared to pretty much anything else, no. Besides, I was mostly referring to Pike, which I don't remember hearing of before you mentioned it.

Agreed. Furthermore, individual games don't need names; giving rooms names should be enough. The guest/host distinction is irrelevant to the user, so games can be described simply as "Elvish_Pillager vs Novus", or, if you want to show how the game is progressing: "Elvish_Pillager (170/200) vs Novus (154/198)".

I originally intended the player to be in only one room at a time to keep the UI simple. I can see why you'd want to be in several at a time, though.

Again, for simplicity, I'd do this on a room-by-room basis (using invite-only rooms).

Generally speaking, I think Cedric's solution is a bit too cluttered, although this is because I've approached this from an ingame UI point of view (the 320x240 screen forces you to keep the UI very simple, but has the advantages of being portable to consoles and handhelds and maintaining a consistent appearance; the downside is that manipulating large amounts of information gets really painful).

Haha...The chat there is especially amusing.

I want to make it as easy as possible to start a game. So my idea would be something like this:

• you enter a room named "newbies"
• you press the "challenge channel" button
• In the popup you enter as the challenge description "just for fun", you leave all the checkboxes for type of gameplay to the default, and press "ok". The challenge is added to the list.
• someone presses the "accept challenge button" next to the "just for fun" challenge
• the challenge disappears from the list and the players go to pre-game negotiation

Agreed, give or take some minor details. A blank challenge description is OK; the player name is sufficient to identify a challenge. I'd also require the challenger to confirm the game instead of unceremoniously yanking him out of the room (message in chat pane with a button: "Cedric6014 has accepted your challenge: [PLAY]"; in the mean time, the person accepting the challenge stares at a "Waiting for confirmation" message with a cancel button)

In tournaments, appropriate challenges would be automatically created: "Semifinal: Novus vs Shiver (200 pt, nodups, fixedfleet)", "League round 2: Novus vs Cedric6014 (150 pt, randfleet, randsel)".

Net play is fun. Everyone should try net play.

I can imagine circumstances where one would want to issue multiple challenges. Though I guess it could default to the player's name.

Agreed.

That would be nice, but of secondary concern (though something to keep in mind).

Multiple challenges would have different challengers, requirements or rooms, right (no sense in having two identical ones)? You could display those and enforce uniqueness that way.

Check.

I think there could be a lot of possible parameters, too many to show in the list of challenges.
I think it would be better to have some sort of description; "tournament rules" says a lot more than a list of parameters.