(Dev's please review this.) How PS2 Works and Where it can be Optimized

Call me snake..

hey projected plan corporate guy, okay what you mean is:
1. if they cannot make a game properly, their company and their game is a piece of ****
2. if they cannot upgrade the game properly due to the above, than it's really a pos
3. it's a pos.
4. pos
Case closed no objections your honor.

Lets keep the weak counters and politics out of this, the op is apparently contributing
more to this than the rest of you whiners, yes you reading this right now. Jerk.

Cheerio chap sucks to be youuu

@ Xae, Ive worked a bit with that "corporate world", concerning game development. With most highly successful game dev companies (Blizzard, Insomniac, Bioware, etc) its NOTHING like how you described. Its a helluva lot more leisurely. At least, in terms of what they work on, and how many hours they put on what. Insomniac specifically, have regular casual dress, and go all out on Casual Tuesdays. Riot Games, of League of Legends fame, lets their devs do whatever they want as long as they get their work done. Furthermore, a little research will show you the game industry usually has people work extra hours to get stuff done, even outside of deadlines.

TL;DR - The good game companies know how to have fun and push out results.

I suggested an optimization to my outfit mates, here is the complicated instructions to achieving it.

Step 1: Spawn all vehicles with headlights off.
Step 2: Enjoy better FPS in big battles while the game renders far less light sources.

PS. before you start, yes I always turn my headlights off (at night anyway). It's still rendering them during the day and for all the nabs that leave them on.

The thread is interesting, and the findings well presented, howevre Toxic does have a point, if you analyze the CPU usage even when the engine is bottlenecked by the CPU the engine does a horrible job of using all of the resources of the system, by quite a significant degree.

Making optimizations in the engine is nice (do more with the same resource) but right now what will bring us the largest improvements is actually increasing the amount of resources the engine can tap in the first place.

The major issue is the multi-threading support, the engine is doing a very poor job of tapping more than 1 core, while it does use all 4 cores on quad core machines it does not use more than about 80% of the primary core and no more than about 40% on the other cores, this leads to total usage including hyperthreaded cores (8 logical cores) of way less than 20% of the total resources.

Quite often core game logic is run on the master thread and it farms off tasks to worker threads, in a lot of games in CPU bottlenecked circumstance the master thread will run maxed out while the worker threads have varying degrees of load. However in PS2 even the master thread isn't maxing out its core which must mean the threads in the game engine are entering wait states for each other, this is due to sloppy multi-threaded implementation in the core game engine.

They need to focus on getting the core game logic threads running without ever having to wait for other threads and at least push the primary thread to use the maximum resources of that core, that should give a healthy boost in frame rate for a lot of people. After that the focus should be on optimizing process of handing off work to the slave threads so they too can better use their cores, this is always more tricky because workload fluctuates rapidly depending on what is happening in the game world so balancing it perfectly isn't possible but generally getting to around the 70-80% load mark per core is do-able.

Going from usage of something like 80%-40%-40%-40% to 100%-80%-80%-80% would offer massive improvements in frame rates, only once that has been achieved is it worth looking at optimizations such as merging scripting to one common format which is not likely to bring much of an improvement.

With a 2600k @ 4.9Ghz I can get about 40-50fps in large fights which is creeping up on acceptable, but that is performance that isn't actually available off the shelf the average consumer, you need to push the latest and greatest very hard to achieve those kind of speeds which only a very small percentage of people are willing to do, so this is a very real problem for PS2.

Thanks for the laugh!

Interesting.

Maybe they are fixing all of the stated issue and that is why they said the update may require a re-download of the game? Changing the way everything is done as stated would be very messy to just download updated files and could and probably would cause many issues. Hopefully this theory is correct and all these issues are being fixed.

I don't understand anything that is said here but it all sounds cool to me

And yet again I ask:

How is stating the obvious going to help the devs? Do you expect the devs somehow have forgotten that they use scaleform and you have to remind them? You think they don't know they're using physx? What? Listing imports is of no value and takes zero effort

Worst thing is, you know that. The only reason for you to do this is getting attention from people who aren't that computer-savvy

FFS, stop giving him attention, for a developer this thread is as useful as saying "the sky is blue" using complicated words
Edit: Now that I think about it.. it's more like saying "the sky is red", considering some things that have been said are flat out wrong

The findings stated here concerning Scaleform coaling pretty well with what I found in Tera (Also an MMO with Scaleform UI).

There disabling UI also gaved a boost and grabbing a window resulted in fps drops (like grabbing the scrollbar in PS2).

While it is true that many current AAA titles use Scaleform, how many of them are MMOs? PS2, Tera and WoT (not shure but it is flash). There we have 2/3 MMOs (I have knowledge of) that have reported problems with it.

The scaleform UI is pretty CPU heavy, it seems that it runs not GPU acellerated and the CPU is doing all the work. Considering that MMOs are traditionally more CPU heavy it may be the cause of problems on the lower end of Processors.
It would also be intresting to know if the UI runs in it own thread, if not that should be done.

agree

True. I kinda said the same thing. Don't think anyone here actually expects devs to take "notes." It's just interesting to see what people dig up. Granted, you won't find anything major, much less provide usable data to help with optimization. I mean, if you have whole project in front of you and struggle optimizing it then what chances someone has with disassembled code? It would take ages to turn it into an organized pseudo-code. You could at best track down some memory leak if there's any, but that's as far as it goes. Rest is just...sh*t and giggles.

So again, none of this will ever help any developer unless you bump into an obvious mem leak or such (very unlikely).

@Archey

Unlikely. Game engine is probably fine. Take UE3 for example. Some games run great, some don't. Why? Because game devs don't know how to properly use UE3, its features and limitations. Optimization is most likely kept within the engine (scripting, clever math, etc). This is the PS2 team. Another team is working on the engine; bug fixing, implementing new features (if there are any) and doing general improvements. Most changes, fixes and improvements won't force game devs to rework large chunks of scripts/code. Changes that could potentially affect the current game will be put in, but it's game dev's their responsibility to use it properly, test it and fix any problems that might occur.
It's doubtful you'll see any ground-breaking core improvements as it would probably break many things used by both PS2 and EQ3. In few years you'll get Forgelight 2 (assuming they keep the name), etc with better core features and what not.

This may be slightly off-topic but since there's a very knowledgeable gathering here I figured I would give it a shot. The localization files have id's followed by four characters and then the localized string. The ID's appear to be a hash of some sort but I can't seem to locate them in the client data. So, it seems like the ID's are from the server which is seems very strange. Can anyone else confirm/deny?

I think the Dev's know how their game works...

@SpiffyJr
By localization files, I'm assuming your referring to the files in ~/Planetside 2/Locale/
In analyzing these. I've gathered that the .dat file is where all of the strings displayed on the screen are kept and the .dir files are a md5 hash used to verify the strings in the .dat. Given that these are strings rendered to the screen, I'd say that the client executable reads the ID numbers appended to the start of the string.

Another interesting discovery is that the game has parts of T4Lib, such as is used for a macro system. Given that the engine is based on FreeRealms, a RPG, seeing a macro system is expected.. T4 is good for quickly making something, however it is also a generation of unoptimized code. T4lib is part of Visual Studio as code generator. More information on T4 can be found here. Link 1 Link 2 Link 3

@Elo1
@Cysquatch
Of course the dev's know their engine. However, how many of the dev's are GM's or players themselves? Optimization is the result of a recurring cycle of 'Programming -> Debugging -> repeat' The dev's are doing their part on the Programming side, however, on the debugging side there isn't much evidence. For example, I've been refraining from mentioning this so as to not cause a lot of negative commotion. However, if you open planetside2.exe in IDA Pro you get this message upon open. You can test this yourself by using IDA Free or IDA Demo.

@Kitsune94

Yes, those are the files I'm referring to. The only MD5 I'm seeing is the checksum used in the .dir file. The entries in the .dir file appear to be an id, starting byte, and length for quick-reference to the .dat file. The .dir file has an id, some string, then the localized string. The only issue I'm having is linking the ID from, say, Weapons.txt to the localization ID present in the .dir and .dat files.

Take, for example, NC14 Bolt Driver which appears to be WEAPON_ID 31 in Weapons.txt (from the .pack files). The ID listed in the en locale file is 1462496766. I'm assuming that 1462496766 is some sort of hash but I've got no clue where to start looking.

*EDIT* How bout it devs - wanna tell me where to find this info so I can stop searching? :-D

Brilliant thread. Hope the devs fix up the performance and security issues. Also, I hate Flash...Games shouldn't use it.

So much ado about nothing.

Devs know how to fix the game guys. They aren't fixing it because there is no money in fixing it. They are being funded to create content to help monetize the game before it tanks, that's about all they're working on now.

I concur.

You are kidding right....how are they going to make any money creating content and monetizing if the game is full of critical bugs? Kitsune is right and sony needs to fix this. I'm surprised that a next gen game like this is only running on 1 thread!

Personally I'm ok with having him discuss this here. I mean if a dev happens to glance at it and remember oops i forgot that(insert random code or engine tweak here) thanks guys. would it not be worth this entire little thread? even if its just a tiny improvement?

Now it is all ifs and maybes but it seems you would want to shut that all down before they even had a chance?

@kitsune
Btw i seem to remember a kitsune on the SINS forums always giving a helping hand to those in need......................