E
%Begin AMD FidelityFX Super Resolution: the Digital Foundry interview Eurogamer.net If you click on a link and make a purchase we may receive a small commission. Read our editorial policy. AMD FidelityFX Super Resolution: the Digital Foundry interview
Team Red talks FSR upscaling.
visibility
410 views
thumb_up
3 likes
V
Interview by Richard Leadbetter Technology Editor, Digital Foundry Updated on 14 Mar 2022 39 comments Everyone wants extra performance from their PC hardware, right? And that's where technologies like AMD's FidelityFX Super Resolution (FSR), Nvidia's DLSS and Intel's upcoming XeSS are all about - essentially allowing the GPU to render at a lower resolution, then either upscaling or reconstructing to the native output of the display.
thumb_up
26 likes
J
In a sense, this is new territory for PC, where native resolution rendering was for a long time considered the only way forward. However, in console land, 'smart upscaling' isn't new, really coming to the fore with the launch of PS4 Pro in 2016. PC has embraced similar techniques, but has also spawned its own blend - and AMD's FSR in particular stands apart.
thumb_up
26 likes
A
So when Team Red asked us if were interested to talk FSR, we jumped at the chance. AMD's approach is different, both in terms of technology and philosophy.
thumb_up
14 likes
S
To set the scene, what we've traditionally defined as 'smart upscalers' have all had one thing in common: the use of prior frames as a reference for improving the quality of the next one to render. All of the effort the GPU has spent in generating a previous image, working in combination with motion vectors that inform the game of where those pixels will end up in the future, allows for extra detail to be injected into a freshly rendered frame. The two key technologies to use this initially were checkerboard rendering and temporal super-sampling.
thumb_up
21 likes
E
These techniques what made PS4 Pro's 4.2TF GPU capable of producing a pretty convincing 4K output and they've been used on all consoles by this point. Similar techniques are now found in a range of engines supported on PC too - and with an added component from machine learning, that's how DLSS 2.x works as well, and additionally, it's how we expect Intel's XeSS to play out. When AMD revealed that FSR would be a spatial-only upscaling solution, we were sceptical, because rather than drawing on data from prior generated frames to increase detail, it has no visibility on data from anything other than the current frame which inevitably leads to temporal discontinuities.
thumb_up
28 likes
A
AMD was effectively pursuing techniques that the industry in general had basically left behind - FSR isn't a reconstruction technology like TSSAA or checkerboarding, rather an upscaling one. However, there is more to it than upscaling alone: it is augmented with the ability to detect and refine edges and to mitigate the aliasing artefacts that would otherwise result - and that's what makes it intriguing.
thumb_up
43 likes
D
Then there is the philosophical angle too. Nvidia DLSS is closed source - a black box for developers, if you like, and only capable of running on ML-equipped cards. Intel XeSS aims to be more open but certainly to begin with, there's no source access.
thumb_up
23 likes
J
AMD's FSR is the direct opposite: the source code is fully open and as it's a fully software-driven technique and it should run just fine on any modern GPU, whether it's sitting in a PC or a console. This has led to a wide range of support arriving very quickly, with unexpected 'homebrew' support added to various emulators, and even implementation in brand-new console games, such as Myst and (we suspect!) Arkane Studios' Deathloop. All of which leads into this interview with Nick Thibieroz, Director of Game Engineering at AMD.
thumb_up
49 likes
V
We wanted to know more about why AMD chose to go against the grain with a spatial approach to upscaling, how FSR fits into a FidelityFX ecosystem that already includes another beneficial technology for upscaling (CAS), why it hasn't mirrored the approach of Nvidia and Intel in embracing machine learning and what best practises there are for developers in getting the best out of the technology. Also, what of the future?
thumb_up
50 likes
M
FSR is in a 1.0 state, improvements are surely in the pipeline, so what should we expect? Watch on YouTube The DF review of FSR: the technique is intriguing but quality of results is essentially tied closely with the kind of content it is processing. Digital Foundry: Can you give us an initial overview on AMD's objectives for FSR and how you settled specifically on expanding on your existing work with spatial upscaling?
thumb_up
26 likes
K
Nick Thibieroz: The goals we set ourselves for our first upscaling solution were certainly ambitious, and we tried to include as many benefits into the same solution as we could. I would say the three major features we identified as key were: cross-platform support, a great quality/performance balance, and being open source.
thumb_up
12 likes
S
Being cross-platform allows as many players as possible to benefit from the technology, without being restricted to a given generation of graphics cards, GPU vendor or specific hardware platform. We felt this goal was especially important given the current climate around GPU availability.
thumb_up
25 likes
J
Of course, upscaling quality was a major focus, yet this had to be tempered against the cost of executing our solution on a GPU. Obviously, those goals are kind of counter to one another in many ways, as higher quality will typically demand expending more processing effort.
thumb_up
5 likes
D
Clearly, we had to go beyond the current state-of-the-art in spatial upscaling if we were to enable the technology on the broadest set of GPUs possible. Nick Thibieroz, Director of Game Engineering at AMD.
thumb_up
25 likes
L
Open sourcing the source code was also something I personally pushed for, as we know this is what game developers want to see. Not only does open source encourage adoption in a wider variety of contexts, it makes integration easier, allows others to learn, and unlocks the potential for the technique to be expanded or adapted to fit a developer's specific needs. After all, the game development industry has always been about sharing knowledge!
thumb_up
7 likes
A
This sort of openness is really in AMD's DNA, just look at the GPUOpen initiative, we feel it's important for us to play our part. As the process evolved though, it became clear we had something special. So, we set ourselves another objective, which was to achieve wide and rapid adoption.
thumb_up
40 likes
A
Happily, it turns out that the three key objectives of FSR 1.0 - cross-platform, a great quality/performance ratio and open source - really help us achieve this goal too, which is great for gamers everywhere as it gets the technology into their hands that little bit faster! That said we knew we had to go further, and this meant including a few extras in the design which would help speed up adoption. Specifically, we provided support for engines and features that many developers are already using.
thumb_up
1 likes
O
This is why FSR 1.0 is available on both Unreal Engine and Unity and supports arbitrary scaling (a requirement to enable techniques like Dynamic Resolution Scaling). Considering these ambitious objectives and the breakthrough we made with our spatial upscaling algorithm, settling on a pure spatial upscaling solution for FidelityFX Super Resolution 1.0 was the natural choice.
thumb_up
47 likes
H
Here's AMD's launch video for FidelityFX Super Resolution. Digital Foundry: I think it's fair to say that the general direction of travel everywhere else in the industry for upscaling has been towards temporal super-sampling - whether via software or via machine learning. It's a journey that started to really gain traction with PS4 Pro back in 2016 and is effectively the standard in triple-A game development.
thumb_up
41 likes
C
I'm curious why AMD ruled it out? Nick Thibieroz: We did not rule anything out! FSR 1.0 is the result of extensive research at AMD, with multiple groups exploring different solutions using a variety of underlying upscaling technologies.
thumb_up
40 likes
S
Given the goals we had set out, we chose to release FSR 1.0 as we know it would appeal to a large number of developers and gamers who want to be able to enjoy high-quality gaming at faster frame rates on multiple platforms, without being limited by proprietary hardware. So, while I appreciate that the choice of a spatial upscaler surprised many, I think the results speak for themselves in terms of developer reception and adoption. In fact, it's been impressive to see the various ways FSR has been leveraged by professionals and enthusiasts alike so far!
thumb_up
10 likes
I
If you solely focus on just one facet of upscaling - let's talk image quality - then sure, I think it's fair to say some upscaling techniques out there may provide better results (although in some cases "pixel peeping" on still images may be needed to make this claim). I think if you narrow the evaluation of upscalers to just a single criterion then your conclusion will be incomplete.
thumb_up
18 likes
E
FSR was designed to tick many boxes, as we've discussed, and it's the combination of great features that make up the full package. Think of it like buying a new car: I don't think anyone would solely base their purchase on how good the car looks. A smart buyer is going to consider how fast it goes, what options it provides, how smooth the driving experience is, and whether they can afford it in the first place.
thumb_up
34 likes
I
Digital Foundry: The buzz surrounding the announcement of FSR is that it would be an open alternative to DLSS, that an accelerant of some kind was needed with the arrival of hardware-accelerated ray tracing and its high compute requirement. RT is essentially tied to the DX12 Ultimate feature set fully supported by AMD RDNA 2, which includes machine learning - so was there a specific reason not to tap into machine learning in particular?
thumb_up
11 likes
I
Nick Thibieroz: I would say that we should be cautious about thinking of "Machine Learning" as a sort of magic wand which solves all problems. Of course, if it's done right, ML can be a very powerful tool, but it's not the only way to solve problems.
thumb_up
29 likes
D
In a nutshell, Machine Learning works by using a fairly brute force approach to discover the relationship between some inputs and some outputs. The process works by feeding data into an ML framework, with the goal of producing a model which encapsulates the relationship between the set of inputs and the desired outputs in the best way possible.
thumb_up
30 likes
D
We then take the relationship that has been discovered and apply it to inputs the model has never seen before. In some cases, ML is a no-brainer, it is the best and possibly only way to go.
thumb_up
50 likes
N
However, it is often possible to implement similar functionality via conventional algorithms, rather than relying on ML to discover the relationship for you. There are also trade-offs that you're going to need to make to leverage ML, which mean it might not tick some of the other - really important - boxes for a solution. Using ML in a real-time context might mean that we lose portability, performance, and - if not done right - even some quality.
thumb_up
11 likes
K
If we're being objective about ML and upscaling algorithms, I think the first iteration of NVIDIA DLSS is a good illustration of what I'm talking about here. The mere presence of ML in a solution does not imply you are going to get great results. ML clearly shows promise, and AMD is heavily investing in ML R&D on a number of fronts, but just because an algorithm uses ML does not mean it's the overall best solution given a set of goals.
thumb_up
48 likes
A
Watch on YouTube FidelityFX Super Resolution has received a number of high-profile integrations, such as inclusion in Unity's HDRP renderer. Digital Foundry: CAS still sits within the FidelityFX feature set - are there scenarios where AMD would recommend it above FSR, or is FSR effectively a replacement technology? Nick Thibieroz: Contrast Adaptive Sharpening (CAS) is a post-process image sharpening technology that restores detail in an image, typically in cases where Temporal Anti-Aliasing (TAA) has been applied.
thumb_up
1 likes
M
While CAS supports an optional upscale function, CAS by itself is not able to restore enough detail to upscaled content to deliver a close to native resolution gaming experience. Given this, we would recommend that developers who use the CAS upscaling option should replace it with FSR for much improved upscaled image quality. Games that don't use upscaling, or that have FSR turned off in their game options, may choose to use CAS sharpening on their natively rendered image.
thumb_up
38 likes
Z
Digital Foundry: In broad strokes, is there a series of general 'best practise' guidelines for developers in implementing FSR for best results? Nick Thibieroz: FidelityFX Super Resolution can be found on GPUOpen which includes integration steps and information on where to include it in the graphics pipeline. At a high-level, we recommend that FSR be implemented in perceptual space (which typically means after tone mapping), and to use MIP bias on qualifying textures to get an image closer to what native resolution rendering would look like.
thumb_up
3 likes
O
Some form of anti-aliasing is also required for FSR to produce a high-quality result, otherwise hard edges will be detected and rendered as such. Digital Foundry: It's recommended to apply FSR to an anti-aliased image, but most games have various forms of anti-aliasing to choose from.
thumb_up
24 likes
Z
Of course, results may vary on a game-by-game basis, but are there any general recommendations you can provide to users on which AA solution works best with FSR? FSR works well with most forms of AA we've tried: MSAA, TAA, FXAA etc. However, the quality of the underlying AA implementation is critical to the final upscaling quality produced by FSR.
thumb_up
48 likes
G
For example, AA techniques that don't include additional sample information (either via straight multisampling or leveraging previous jittered frames) or do so in a suboptimal manner will show limitations in FSR upscaling quality e.g., on thin features. The best recommendation I can give for a quality FSR implementation is to ensure the underlying AA technique is also of high quality, as temporally stable as possible, and able to depict thin features in a reasonable manner.
thumb_up
13 likes
J
FSR upscaling options are included in Far Cry 6, which can help to offset the additional cost of hardware accelerated ray tracing - or just as a performance accelerant in general. Digital Foundry: With so many games and game engines employing temporal super-sampling, does FSR have any applications as an additive to that technique - or is CAS a better fit for games that use temporal super-sampling?
thumb_up
8 likes
E
Nick Thibieroz: There's been some experiments in mixing temporal techniques with FSR and some of them are even in shipped titles. Since FSR is open-source developers are free to experiment with mixing the two if they feel the results are worth it.
thumb_up
43 likes
C
This is one of the great benefits of the open-source approach! At the end of the day if a game already supports a quality and performant upscaling implementation that all gamers can benefit from regardless of the hardware they play on then FSR may not be needed at all, and that's OK.
thumb_up
3 likes
C
Digital Foundry: On high frequency detail, we found that AMD's current GPU scaler engine preserves more in-surface detail than FSR from the same base resolution - but FSR handles edge artifacting considerably more adeptly. Could your work with FSR roll back into your designs for your next generation GPU hardware scaler? Nick Thibieroz: A correct implementation of FSR presents many advantages compared to just applying the GPU scaler at the back-end.
thumb_up
37 likes
A
First of all, for the best results, FSR should be rendered at a specific part of the frame to play well with post-process operations and to ensure the color space is right. Secondly, FSR offers a built-in sharpening pass which helps recover and preserve some of the high-frequency detail, even those in polygon interiors. Thirdly, we also offer guidance as to how integrations of FSR should leverage MIP bias adjustments, which also help to get the best results possible.
thumb_up
31 likes
V
Given all of that, I would normally expect to see a superior visual result from a successful FSR integration compared to simply using a pure GPU scaler at the end of the frame. Regarding further applications of FSR to different contexts, I'd say we're evaluating all options, but nothing we're ready to talk about at this stage. Digital Foundry: FSR is in its 1.0 state, but can you share any information on where research is heading for the next iteration?
thumb_up
47 likes
T
Right now, the FSR algorithms are based on a 2D input from the game engine - but could FSR stand to benefit from access to depth information, or motion information? Nick Thibieroz: We plan to continue developing and/or enhancing FSR over time, and delivering on our goals of giving the best possible experiences to gamers. With multiple research efforts happening in parallel, several technological directions are being considered.
thumb_up
27 likes
D
I look forward to speaking with Digital Foundry again once we're closer to announcing something. Digital Foundry: FSR's greatest strength is that it's software-based and effectively runs on any GPU - or at least any GPU capable of running the host game. Is this a philosophy you're completely wedded to - or would you consider leveraging DX12 Ultimate features?
thumb_up
32 likes
E
This may reduce compatibility but it would still be open in the sense that the technology would not be proprietary or run on a single brand of GPU. DX12 Ultimate features include DXR 1.1, Variable Rate Shading, Mesh Shader and Sampler Feedback. At this point in time, I don't believe any of those technologies are particularly relevant to upscaling algorithms (VRS could maybe have applications though).
thumb_up
0 likes
I
If you're asking more generally about the concept of leveraging bleeding-edge GPU technologies to produce super resolution algorithms, then all I'd say right now is that our upscaling research spans many directions! Since FSR 1.0 was developed for a broad set of users and platforms there's now a solution in place for virtually everyone, and this allows us to direct some of our focus on solutions possibly leveraging more advanced GPU functionalities or performance levels.
thumb_up
24 likes
H
Will you support the Digital Foundry team? Digital Foundry specialises in technical analysis of gaming hardware and software, using state-of-the-art capture systems and bespoke software to show you how well games and hardware run, visualising precisely what they're capable of. In order to show you what 4K gaming actually looks like we needed to build our own platform to supply high quality 4K video for offline viewing.
thumb_up
13 likes
S
So we did. Our videos are multi-gigabyte files and we've chosen a high quality provider to ensure fast downloads.
thumb_up
32 likes
G
However, that bandwidth isn't free and so we charge a small monthly subscription fee of £4.50. We think it's a small price to pay for unlimited access to top-tier quality encodes of our content. Thank you.
thumb_up
25 likes
A
Support Digital Foundry Find out more about the benefits of our Patreon
More Interviews Interview The Last of Us Part 1 developers on the journey to PlayStation 5 and the endless promise of accessibility "We're all at this very early frontier of what is possible here." 15 Digital Foundry Inside Marvel's Spider-Man Remastered on PC - the Nixxes tech interview How the Insomniac classic was ported from PS5 to PC. 30 Digital Foundry Inside AMD FSR 2.0: the Xbox connection and what could come next AMD's director of game engineering explains how their temporal upscaling tech was developed.
thumb_up
2 likes
A
14 Interview Interior/Night's Caroline Marchal on setting up her inclusive studio after leaving Quantic Dream And creating accessible Xbox narrative game As Dusk Falls. 3
Latest Articles Atari will hold RollerCoaster Tycoon rights for another decade Ups and downs.
thumb_up
13 likes
H
7 Jelly Deals Logitech's G Pro X gaming headset is its lowest-ever price during Amazon's Early Access sale Prime Members can get it for just £52. Jelly Deals Save over £500 off the retail price on this beefy ASUS TUF Dash gaming laptop from Amazon Under £1080 for an RTX 3070 laptop. Lady Dimitrescu will be a tad smaller in Resident Evil Village's Mercenaries DLC Level the playing field.
thumb_up
36 likes
S
1
Supporters Only Premium only Off Topic: Take a minute to appreciate Cookin' with Coolio's incredible scallops recipe. What a great book. Premium only Off Topic: Reading City of Glass in comic form "Where exactly am I going?" Premium only Off Topic: Il Buco is a transporting film about a really big hole Underlands.
thumb_up
10 likes
Z
Off-Topic Netflix handled Sandman brilliantly It was Dreamy. 9 Buy things with globes on them And other lovely Eurogamer merch in our official store!
thumb_up
14 likes
E
Explore our store
thumb_up
21 likes
Similar Discussions
-
Popular AEW tag team reportedly showcase interest in signing with WWE under Triple H s regime after backstage brawl
-
I m not that kind of coach PSG manager Christophe Galtier makes bold claim involving rotation of 2 players
-
I would tell him to stop doing that Erik ten Hag urged to ban Manchester United star Antony from showboating
-
Union Berlin vs Union Saint Gilloise prediction preview team news and more UEFA Europa League 2022 23
-
센텀플랫폼 이용자 APK Download for Android
-
Easy Khata Apka Digital Khata APK Download for Android
-
Highett Youth Club APK Download for Android
-
u00c1lvarez Pallete Europa necesita autonom u00eda estrat u00e9gica para superar esta crisis lvarez Pallete Europa
-
Una mujer embarazada de 23 semanas presume de abdominales Disfruto entrenando m s que nunca Mujer Embarazada
-
David Cameron will not take part in the TV debates unless Green Party does
-
How to Create an Email Signature in iOS Mail
-
How to Use the iPhone App Library iOS 14 and Up
-
How TechGirlz is Paving the Way for Women in Tech
-
Why Apple Watch Mask Unlocking Is the Best Thing Ever
-
Food Delivery App Wars Grubhub vs Uber Eats
-
Bugatti Could Be Working On A Hardcore Chiron SS CarBuzz
-
Leeds vs Fulham live stream and how to watch Premier League game online lineups Tom s Guide
-
Netflix Series Echoes Is 1 and That s Just Not Good Enough IndieWire
-
Jill M Huber M D Doctors and Medical Staff Mayo Clinic
-
Rhinoplasty Care at Mayo Clinic Mayo Clinic
-
Vedder Light Tuck Appendix
-
Texas Find A Grave
-
Ucsf It Jobs
-
Those Who Remain Wiki
-
Sandstone Slab in Minecraft
-
Who is Nango in Naruto
-
Idol and friend Thank you football PSG superstar Neymar hails Lionel Messi with tribute on Instagram
-
5 UFC fighters who went from being fan favorites to heels
-
Mike Florio thinks Odell Beckham Jr could ultimately decide to sign with another team rather than the LA Rams
-
Is Williams Sonoma launching a collaboration with Roblox