STEAM GROUP
Special K - "Kaldaien's Mod" Special☆K
STEAM GROUP
Special K - "Kaldaien's Mod" Special☆K
271
IN-GAME
2,572
ONLINE
Founded
May 23, 2016
Language
English
All Discussions > Development > Topic Details
 This topic has been pinned, so it's probably important
Kaldaien Apr 12, 2019 @ 6:20pm
Special K - v 0.10.3 - [HDR stuff probably]
What's New? (Current Test Build :: 0.10.3)


I've opted to add a new form of frametime analysis to Special K's framepacing graph.
    The old data (which still remains), was based on higher-level statistics and standard deviation; an uncommon representation of data not easily understood or comparable to expected benchmark performance and limited to a very small sample window of 120 frames. In short, it was useful to me for performance optimization and analysis but those numbers had no meaning to anyone else!
      Fun bit of trivia, however, the color of the framepacing graph is based entirely on the standard deviation.

What are Percentiles and Why Should I care?

To put it simply, when we speak of frametime percentiles, we mean either the bottom N-% or top N-% performing frames recorded. Since I don't care about high framerates and only care about stuttering, Special K's analysis focuses on the low percentiles.

    When numbers like "10% Low FPS" are being thrown about, it's another way of saying 90% of all frames drawn were faster than xyz FPS.

Common numbers used for frametime analysis are 10% / 1% and 1% / 0.1%.

It is my strong belief that 1% / 0.1% is not appropriate for measuring and tweaking game performance. 1% / .1% should be reserved only for people publishing benchmark scores because that's really all they're good for :P

    I would suggest configuring the framepacing widget manually and selecting something more reasonable such as 5% / 0.5% or even 10% / 1%, but it is pre-configured for 1% and 0.1% since this is the most common percentile ranking published in online reviews of games.

How large is the dataset used?
There are two different datasets for percentile analysis.

There's one that keeps track of the last 1,000 frames and then there's an even longer-term one that gets a summary of all 1,000 frames every time the short-term history fills up with data. Effectively this gives a 1,000,000 frame history and that ought to be enough for anybody :P

    If you're trying to find optimal in-game settings, I suggest focusing on the smaller 1,000 frame history. That's at least 8 seconds worth of data even at 144 Hz.

Conclusion
Knowing what this new feature is designed to measure (how poorly the worst-performing part of the game is performing), you should be able to use it to find the perfect set of graphics settings that keeps the two percentile charts in the green. Failing the ability to find any settings in-game that stop stuttering, you can always fallback to Special K's framerate limiter.
Last edited by Kaldaien; Jun 6, 2020 @ 4:40am
Originally posted by Aemony:
The new discussion forum, http://special-k.info/, is now up and running with working mail notifications as well. Feel free to sign up for an account there if anyone want -- it's the only place to reach Kaldaien right now. We still haven't landed on what categories to set up and such, but that will probably be figured out soon enough.
< >
Showing 1-15 of 8,121 comments
Kaldaien Apr 12, 2019 @ 6:21pm 
Update Corner (2020 Apocalypse)
I already asked about the screenshots when I went through the store page check, they said it was fine since there is no way to demonstrate the software in anything but a game. They double-backed on that decision 6+ months later.

Now I cannot even get the fee for publishing refunded because it has been over 6 months. They offered to give me a credit toward another application, but I do not have any other applications. Even if I did, what about this whole experience makes Valve think I would want to put myself through this a second time?

They basically lured me into their van with the promise of Steam candy, robbed me and then sent me back onto the street naked :-\

I am not pleased with Valve.
Last edited by [SK] Kaldaien; Jul 29, 2020 @ 9:17am
Kaldaien Apr 12, 2019 @ 6:23pm 
On the Subject of Universal Windows Platform (UWP)
    @Aemoney:
      Are you really bothered by that issue with DLLs getting stuck in Application Frame Host or Chromium processes (i.e. Steam Web Helper)?


I have been reading up on the documentation for UWP, and I discussed this a little bit before, but the reason for the DLLs getting stuck has to do with some features only exposed to (and heavily used by current common UWP software) designed to save energy by suspending entire processes (annoyingly, while they hold locks on DLLs they did not load !!!!).

Effectively, Microsoft took various concepts from Windows Phone and noticed they didn't quite make sense in regular Win32 but had an opportunity to start adding weird features that aren't normal for Windows software into UWP.


SKIM is capable of injecting code into UWP, it happens quite a bit actually, but Special K is based on Win32, Nt kernel and C code and once SK accidentally injects into a UWP program, stuff goes downhill in a hurry ;)

UWP is some truly weird combination of COM and stuff that resembles .Net and all that jazz Microsoft invented that I've never used before (COM aside, you cannot easily write a PC game without using COM). I have been reading up on the docs, but don't know whether I can throw together code for improved UWP compatibility in any reasonable timeframe.


That stuff is all oriented at C# and .Net programmers and I stubbornly never took those things seriously until suddenly the documentation for using UWP from what I consider a "real programming environment" (lol, I am such an ass when it comes to anything not written in C or C++) does not exist.



Tl;Dr:

    I could make UWP interop a design goal for 0.10.x or push it off until 0.11.x.
    • I need to know how relevant you, the users, expect UWP to be before I commit to anything.

    I would also love to hear from any developers who might be reading this thread who have UWP experience, am I way off base? Finding UWP documentation that is intuitive to a C programmer seems like a fools errand at the moment.
    • I don't want to write any C# code, the same way I refused to write Objective-C or Java for OS X / Android development; I want to keep using my beloved C and just a smidgen of C++ dammit :P
Darktalon Apr 12, 2019 @ 7:11pm 
I can't particularly speak for the importance of UWP, but I do wonder if there is a way to be able to force flip model presentation in DX9. Recently I have been having extreme troubles getting DX9 games such as Borderlands 2, to play nicely with g-sync and v-sync. Both with fullscreen optimizations enabled and disabled. G-sync won't work no matter what I do, and if I have it enabled it breaks v-sync as well, no matter if its forced in game or in control panel. I think that the fullscreen optimizations isn't working as it previously did, or its failing to actually convert to flip model.
Edit: I discovered the errors of my ways, if the launcher of a game has fullscreen optimizations disabled it also can force the game to be disabled too despite what it has selected. Thus, when I made the Borderlands 2 launcher set to FSO on, suddenly gsync works.
Last edited by Darktalon; Apr 12, 2019 @ 7:58pm
Aemony Apr 12, 2019 @ 8:37pm 
Reposting here since the old thread have been locked:

---
I doubt all UWP-based games are Dx12, as UWP is really only a container that everyone can convert an app into. That UWP “powers up” article also only relates to Xbox One, where UWP have historically been gimped as it was used mostly for apps and non-games. You can create an UWP-converted app quite easily using Microsoft’s official app converter as well, which even works for regular users. For example, I have a UWP app variant of Google Picasa Photo Viewer, which is a tiny GPU accelerated image viewer that saw its final release back in 2015, and most definitely does not use Dx12.

That said, while I am bothered by the SK DLL file getting stuck in processes and whatnot, it isn’t an ultra major critical issue at the moment as I’ve learned to rely on local injection for the most part, so pushing it further back for now is fine, I guess.

Long term UWP support is of interest because of the insight it can give in non-Dx12 UWP-based games, from the perspective of Special K.

Also, there’s some 400 games currently available from the Microsoft Store, including games that have historically never had a peep of Dx12 support such as ARK, Sundered, Superhot, The Wolf Among Us, etc.

Meanwhile there’s only 32 confirmed Dx12 games.
---


Originally posted by Darktalon:
I can't particularly speak for the importance of UWP, but I do wonder if there is a way to be able to force flip model presentation in DX9. Recently I have been having extreme troubles getting DX9 games such as Borderlands 2, to play nicely with g-sync and v-sync. Both with fullscreen optimizations enabled and disabled. G-sync won't work no matter what I do, and if I have it enabled it breaks v-sync as well, no matter if its forced in game or in control panel. I think that the fullscreen optimizations isn't working as it previously did, or its failing to actually convert to flip model.
Edit: I discovered the errors of my ways, if the launcher of a game has fullscreen optimizations disabled it also can force the game to be disabled too despite what it has selected. Thus, when I made the Borderlands 2 launcher set to FSO on, suddenly gsync works.

Glad you figured it out. It would’ve also maybe been worth trying out the latest dgVoodoo WIP versions which apparently supports wrapping Dx9 to Dx11, which then allows Special K to hook and affect the game using its Dx11 feature set.

I had Prince of Persia: Sands of Time using “native” flip model though that combination.

That said, it is still a pretty new development for dgVoodoo so compatibility differs.
Last edited by Aemony; Apr 12, 2019 @ 10:29pm
Aemony Apr 12, 2019 @ 10:32pm 
Eh, apparently it's *really damn hard* for moderators/admins to spot that a thread had been locked, so I added a new post to the end of the old thread explaining it.

I didn't even realize it was locked until Erebus mentioned it, at which point I had to scroll up and down and question how the fuck I was supposed to spot that. Only way turned out to spot the "Unlock thread" option under Moderator Tools signifying the thread being locked.
Wastelander121 Apr 13, 2019 @ 4:08am 
[Posting here for notifications, sometimes subscribing alone doesn't work]
HolyDeath3000 Apr 13, 2019 @ 8:30am 
[Posting here for notifications, sometimes subscribing alone doesn't work]
daxxy Apr 13, 2019 @ 12:40pm 
Well, "don't randomly break innocent UWP apps" seems kind of important :P

My experience with UWP is pretty limited - I had a "mobile Windows" C# project ages ago, but I think that was still the Win8.1 platform which is similar but not equivalent to UWP, and I mostly forgot that stuff already. That said:

You're right that .NET is where UWP is at home. That doesn't mean C++ support is an afterthought - Microsoft surely needs it for Windows-internal UWP crap and porting their own code, and it's obviously unavoidable for flagship games - but people working with it are required to have some familiarity with how things work in .NET land.

It's not like they completely threw away Windows.h and require you to use the .NET UWP APIs for everything. I think most of your stuff will just keep working if the functionality is available in UWP (for the process you're running in). See MSDN's WinAPI reference, UWP support and limitations are noted on each function page.

Today's .NET is very different from last decade's .NET. C# has been getting incremental additions, in principle like all other current languages, but as I see it it's moving much faster. More importantly, the developer toolchains, runtimes and standard libraries have seen complete overhauls/rewrites.

For someone with working knowledge of a typical C-style object-oriented language, going from "0 C# experience" to "can build a calculator in UWP" shouldn't take more than a weekend at most. Microsoft makes approachable video courses for some of their core technologies, often involving a primary developer of those technologies, and they like to give an overview of the design/rationale in their getting started material (both written and video). There's almost certainly something like that for UWP which should be quite helpful.

The bigger problem by far than figuring out how to write a UWP app - after all, you're not doing much that would involve invoking the new mobile-platform-style APIs, and you're not making an application so you don't have to bother with app packaging etc. - is gonna be dealing with application sandboxing and hacking on the stuff you're not intended to touch. UWP is not an effective walled garden in the face of an elevated Win32 program next door, but it sure seems annoying and awkward to screw with. There's the good old "Basic and Intermediate Techniques of UWP App Modding" tutorial (google it, IIRC people have had problems linking to it here before because lol multiplayer cheating sites), apparently you can get IPC by giving the UWP app permissions from the outside[github.com], and there's probably lots more arcane ways to poke holes that certainly aren't supported but the platform was never designed to prevent.
Kaldaien Apr 13, 2019 @ 8:40pm 
Oh, neat. I actually am considering re-writing SKIM to be (or leverage at least) UWP features since that's the one component of Special K that really and truly is an application. It's the thing injecting my code into UWP applications by mistake as well :P I need to give it the ability to find a UWP program that's been known to suspend itself for power saving reasons and then prevent injection into that process.

The problem is Microsoft didn't add that stuff to the normal WIn32 API sets, it's in the kernel if I want to spend time reverse engineering that (nobody else has as this poiint, not even the wonderful System Internals book). I can only get at that functionality from a UWP context at the moment, and the logical place to control all of this would be through SKIM. It'll gain the UWP brains and everything else will remain C / Win32 / (Documented) User-Kernel code.
Kaldaien Apr 13, 2019 @ 8:44pm 
I am going to ship an updated version of ImGui when 0.10.0 goes live:

https://steamcommunity.com/sharedfiles/filedetails/?id=1712082325

It has various new usability features, including window names in the window selector (Ctrl + Tab). The default color scheme is also a bit different, but I'll probably get all that back to normal before release.

A huge number of specialized custom ImGui features I developed are now core in ImGui, so I decided to gut my custom code and go with the new official stuff instead. It remains to be seen whether that is a mistake :P


Once I get up to speed with the v 0.70 ImGui codebase (previous versions of Special K used v 0.50, which is from 2016), I should be able to hack together basic UI support for D3D12 and Vulkan. The text-based OSD and achievement popups are rendered using CEGUI, so they won't be usable, but this is better than nothing.
Last edited by Kaldaien; Apr 13, 2019 @ 9:11pm
Kaldaien Apr 13, 2019 @ 9:20pm 
I'll also probably issue a pull request for HDR support in ImGui at some point. I fixed Valve's Steam overlay shader and gave them a patch to support HDR for HDR10 and scRGB 6 or 7 months ago but they just ignored me :-\

ImGui's open source and used by tons of software such as ReShade, so even though Valve's being an ass I can still help other projects :P
Last edited by Kaldaien; Apr 13, 2019 @ 9:21pm
Aemony Apr 13, 2019 @ 10:13pm 
It’s a shame they don’t have a public issue tracker for Windows and the overlay as they do for e.g. the Linux version of Steam.

You could always try post about your findings on the Linux tracker, as that will almost certainly guarantee that /someone/ at Valve might see and forward it to whomever is appropriate? :conwayshrug:
Kaldaien Apr 14, 2019 @ 4:20am 
That wouldn't help a whole lot, considering ... I've only implemented HDR stuff in D3D11 right now :)

Maybe after I get a SPIR-V shader backend for Vulkan, then I'll go ahead and do that. I expect OpenGL will be easy to implement an HDR fix for, but OpenGL software that uses HDR is rarer than a JRPG that uses HDR correctly.

----
Speaking of HDR (in OpenGL, D3D11/12 and Vulkan):

I just got through making ImGui 0.7x compatible with HDR the right way. That means floating-point colors instead of RGB [0,255] throughout the entire ImGui render pipeline.

Internally it now renders to 16-bit fp vertex color, then (ideally) the framebuffer is also 16-bit fp (HDR10 throws away some precision like the hackjob it is, but DolbyVision and scRGB retain 16-bit color all the way to the RAMDAC).

This produces really nice alpha blending because translucent UI elements now have 65,536 gradiations instead of 256. It also gives finer control over the HDR luminance control. Color levels don't have nearly as many quantization headroom (at some point the 16-bit color has to be range compressed to 10- or 12-bit), but they're still wide / tall color gamut friendly with an additional 2 - 4-bits of post-blending precision retained on HDR screens.


It looks fantastic now, not a dim white line of text anywhere and the bright whites contrast against the rest of the UI much better. Not to mention, anti-aliasing the UI works better with that much alpha channel precision on tap. That may carry over to non-HDR display users.
Last edited by Kaldaien; Apr 14, 2019 @ 6:29am
Kaldaien Apr 14, 2019 @ 6:27am 
BTW, does anyone have an AZERTY, QWERTZ or ... otherwise unusual keyboard layout available? :)

I've been convinced by a few French users to try and detect AZERTY and establish a few default keybinds differently (i.e. 'ZQSD' instead of 'WASD' for any code that establishes left-handed arrow keybinds) if I detect a common alternate keyboard layout.

I need help testing this though because I'm a silly American and we only have one kind of keyboard here and a bunch of measurement units nobody else uses because we're separated from the rest of the world by oceans and I guess that means we get to do whatever stupid things we want and pretend nobody else exists :P
Last edited by Kaldaien; Apr 14, 2019 @ 6:29am
daxxy Apr 14, 2019 @ 8:02am 
Originally posted by Kaldaien:
BTW, does anyone have an AZERTY, QWERTZ or ... otherwise unusual keyboard layout available? :)

I've been convinced by a few French users to try and detect AZERTY and establish a few default keybinds differently (i.e. 'ZQSD' instead of 'WASD' for any code that establishes left-handed arrow keybinds) if I detect a common alternate keyboard layout.

I need help testing this though because I'm a silly American and we only have one kind of keyboard here and a bunch of measurement units nobody else uses because we're separated from the rest of the world by oceans and I guess that means we get to do whatever stupid things we want and pretend nobody else exists :P

QWERTZ user here.

To test, you can just switch keyboard layouts in Windows. In Windows 10, Settings -> Time and language -> Region and language -> Preferred languages -> English (United States) -> Options -> Add a keyboard. Once you do that you'll get a switcher in the notification area.

Don't make multiple keybinds in an attempt to cover all layouts, that's just a mess. The preferred way to deal with this problem in game input is to bind functionality to scan codes (referring to physical locations on the keyboard, usually named after their label in US QWERTY) rather than (virtual-)key codes (referring to the labels). When you get a "Y" scancode, it means the key in the place occupied by the Y key in QWERTY, regardless of what label that has on the user's keyboard. When you get a "Y" keycode, it means the key labeled "Y" on the user's keyboard, regardless of where that may be.

Arguably the ideal solution for games is to support both, so e.g. you can bind to WASD scancodes for directional movement, or the I keycode for "inventory on the i key", and to convert all your binds to keycodes for purposes of display (so the appropriate labels show up in binding editors, tutorials etc.)

In SDL this is trivial and their documentation explains the idea quite well[wiki.libsdl.org]. Never had to do it in plain Win32, I imagine it's a bit more complicated, but MapVirtualKey[docs.microsoft.com] should be a good starting point.
< >
Showing 1-15 of 8,121 comments
Per page: 1530 50

All Discussions > Development > Topic Details