Sony's AI Innovation: Custom Hardware and Collaboration with AMD for Next-Gen Gaming Graphics

Sony decided fully four years ago to develop its own AI hardware for the PS5 Pro rather than using AMD tech and the big question is why

Why doesn't AMD have any machine learning acceleration in its GPUs? Graphics upscaling using AI is so important for gaming Sony decided to do its own hardware rather than use AMD technology. That's one of the most important take-homes from the signature deep dive Sony's lead architect for its Playstation consoles, Mark Cerny, has just given. AMD, of course, is responsible for both the CPU cores and the bulk of the GPU technology in PS5 Pro. But when it comes to AI acceleration for upscaling, Sony decided to very much do its own thing. For clarity, what we're talking about here when we refer to "AI acceleration" is the matrix math hardware used to power neural networks, otherwise known as machine learning or ML. Currently, both Nvidia and Intel have dedicated matrix math acceleration hardware in their graphics chips for PCs. But AMD does not. Indeed, the existing AMD-based PS5 non-Pro has no dedicated matrix math hardware. In Nvidia and Intel GPUs, that matrix math hardware is primarily used for various upscaling technologies, including DLSS and XeSS. The lack of matrix math hardware in AMD GPUs is why AMD's FSR upscaling is based on hand-coded algorithms rather than ML. And that, in turn, is a big part of the reason why FSR isn't as good even as XeSS by many measures and is certainly behind DLSS. According to Cerny, Sony had two critical decisions to make for PS5 Pro and ML acceleration. First, it had to decide whether to have a dedicated NPU or Neural Processing Unit or to use an enhanced GPU. Sony chose the latter. Then it had to decide whether to license that technology or build its own. Cerny says the PS5 Pro project started way back in 2020 and from the get go it was decided that Sony didn't want generic ML hardware, but something specific to PS5 Pro's workloads. "Once you're licensing technology, that's what you're doing forever. So, in 2020, despite the degree of effort required, we decided to build our own hardware and software technology," Cerny says. The target was 300 TOPS of ML performance. That's 300 trillion operations per second, or roughly on par with the raw ML performance of an Nvidia RTX 4060 Ti. To achieve that, Sony modified AMD's RDNA 2 GPU architecture. "We made a set of targeted enhancements to the RDNA shader core and the surrounding memory systems. We're calling it custom RDNA," Cerny says. The problem for Sony was that adding matrix math capability to the RDNA 2 shader core as used in the plain old PS5 wasn't going to be enough. Matrix math is incredibly bandwidth hungry to the extent that, without enhancements to the memory system, the new ML hardware would only achieve 3% of its full potential in a worst-case scenario. Yikes. So, Sony came up with a custom on-chip memory solution, plus tiling algorithms, that allow the matrix math to be done entirely on the GPU and without having to access system memory. Designing all this was a "four-year journey" according to Cerny. The result uses the vector registers in the RDNA 2 shaders as RAM. That provides 15 MB of memory at a combined bandwidth of 200 TB/s. That compares with the "mere" 576 GB/s of bandwidth of the PS5 Pros GDDR6 memory. Meanwhile, the tiling algorithms allows the matrix math to be split up into pieces small enough to fit inside that 15 MB. Anyway, it's that new ML hardware that powers Sony's new Playstation Spectral Super Resolution or PSSR upscaling software, which is essentially Sony's answer to Nvidia DLSS and Intel XeSS. Oh and AMD FSR, albeit as explained above FSR is not based on machine learning. "Looking back at the four years since we started this project, I'm so glad that we made the time-intensive decision to build our own technology. Results are good and just as importantly, we've learned so much about how AI can improve game graphics. It can only make our future brighter, say Cerny." So, the big question that follows out of this is why Sony felt the need to do its own ML hardware. Cerny says Sony didn't want to be tied into a permanent licensing arrangement, which implies that Sony views ML upscaling as even more important than GPU architecture. After all, Sony is willing to license AMD's broader GPU architectures. Then again, back in 2020 did AMD even have ML hardware to offer? It's worth noting that AMD's current RDNA 3 based GPUs still lack ML hardware. And some rumors suggest its upcoming RDNA 4 GPUs may not have ML hardware, either. So, maybe the only way Sony could get ML hardware into the PS5 Pro was to do its own thing. If true, it would also mean that we likely won't see an AMD graphics chip with ML hardware until about 2027 when the upcoming RDAN4 chips are replaced with something based on AMD's new CDNA unified architecture. Given even Intel's first-gen Arc GPUs had ML hardware and Nvidia is about to launch its fourth generation of GPUs with ML hardware, that really is a damning indictment of AMD's GPU technology and leaves the company nowhere in the AI upscaling race. Here's hoping that rumors are incorrect and AMD has some GPUs with ML hardware coming soon.

Sony and AMD Developing AI Tech for Better Gaming Graphics, Ray Tracing

Sony is working with AMD to develop machine learning and AI tech for gaming and graphics, according to an announcement on Wednesday. The PlayStation 5 and PS5 Pro consoles already use AMD GPUs, and the newer Pro version has a bigger GPU, better memory, and uses AI to power upscaling. Now, Sony and AMD are working on AI tech development project dubbed "Amethyst," a reference to the purple color you get if you mixed AMD's red branding with PlayStation's blue, to take things a step further. "We've started on another long journey, and are combining our expertise with two goals in mind," said Sony's Mark Cerny, who was the lead architect on the PS5 and PS5 Pro, in a video announcing the collaboration. "The first goal is a more ideal architecture for machine learning." Machine learning, a subfield of AI, could help give a next-generation Sony console the dramatic graphics performance boost needed to justify a new generation like a PS6. "We're combining the lessons AMD has learned from its multigeneration RDNA roadmap and SIE [Sony Interactive Entertainment] has learned from the custom work in PS5 Pro. But ML use in games shouldn't and can't be restricted to graphics libraries," Cerny says. The console architect suggests that Sony's work with AMD isn't to just develop tech that's patented or reserved for their companies alone. He wants to support others' work in the field as well, which may be on other devices besides PlayStation consoles. Cerny also spoke in-depth about Convolutional Neural Networks, or CNNs, which are a type of AI tech that can be used in game hardware. He hopes that AMD and Sony can figure out how to make CNNs better so that they can improve graphics as well as make other game tech like ray tracing and path tracing more prevalent across games. The research and development is expected to take multiple years, according to the executive, who reminded fans that developing a new console is typically a four-year journey for Sony. For him, games have to play "noticeably better" to justify a new piece of hardware. Sony's plans come just a few weeks after its former chairman of PlayStation Worldwide Studios, Shawn Layden, said in an interview that the gaming console industry was hitting a performance plateau. "You're not going to see another PS1 to PS2 jump in performance -- we have sort of maxed out there," Layden predicted.

PS5 Pro Lead Architect Mark Cerny Unveils 'Codename Amethyst' ML Collaboration with AMD

Ahead of the launch of the PS5 Pro last month, Sony invited Wccftech as part of a very select number of media outlets and analysts to get an early look at the console and a soon-to-be-released deep dive into the system architecture by none other than the Lead Architect of the console, Mark Cerny. The event also included an exclusive Q&A with Mr. Cerny himself afterward, which you can read here. Mark Cerny started off the presentation as a sort of 'bits and bytes' talk to principally focus on the GPU for the PlayStation 5 Pro and why it matters. As it's been some seven years since the beginning of the PlayStation 5 journey, it's been more than enough time for the technology advances to need that additional power to make the most of the console generation. With the PlayStation 5 already having been on the market for more than four years, Mark Cerny admitted that it's taken time for game creators to get up to speed (unless you're one of Sony's first-party studios like Naughty Dog or Sony Santa Monica). Much of the PS5 Pro presentation focused on what Mark Cerny called the Big Three: a larger GPU, advanced ray tracing features, and upscaling games through artificial intelligence and PlayStation Super Spectral Resolution, or PSSR. The first focus was on the larger GPU present in the PlayStation 5 Pro which features a unique AMD architecture that Sony was branding as RDNA2.X. This was a hybrid variant of RDNA 2 while featuring both custom and future RDNA versions present in RDNA 3 and beyond. This is combined with a GPU that featured 66% more Work Group Processors, or WGP, compared to the graphics solution present in the existing PlayStation 5. Sony wanted to emphasize that they decided against fully upgrading the system architecture to RDNA 3 because they did not want to have two separate game packages or versions between the PlayStation 5's base and Pro models, not unlike Microsoft's Xbox Series S and X solution. Secondly, Sony made a point of refuting the 335 Teraflop number that was present in the lead-up to the PlayStation 5 Pro's launch. Mark commented that it would have been a nice bonus to have twice the 'flops,' but it wouldn't double the real-world performance and would have instead led to something he dubbed 'Flopflation.' He once again reiterated the increase from 18 WGPs (workgroup processors, or compute units) to 30 WGPs while also introducing 67% higher 'flops'. In terms of real world gains, this meant upwards of a 45% improvement in performance. One example given was how a frame taking 16 milliseconds to render on PlayStation 5 could instead be done in only 11 milliseconds on PlayStation 5 Pro, and those extra 5ms could instead be spent on adding ray tracing. These additional WGPs were primarily focused on ray calculations and the divergence associated. By improving the ease of use, Mark Cerny hoped that ray tracing would continue to be adopted at an increasing rate. Lastly, Sony made a point to touch on the machine learning or AI present in the PlayStation 5 Pro architecture. By utilizing ML, it's possible to give games a graphical boost by having a game render fewer pixels and then using the PSSR neural network to fill in the gaps. This was bookended with the intent that this would not necessarily result in rendering a smaller resolution to begin with. By way of extrapolation and frame generation, the possibility exists to have PSSR insert frames between key frames and reduce the choppiness that's commonly seen in games running at a reduced frame rate. This type of adjusted rendering would not necessarily be the be-all, end-all in terms of graphical processing. Sure, the possibility exists for developers to use the neural processing units to initially render a title at 540P or 720P, but ultimately, it comes down to the matrix math as Cerny called it and integer operations at play, namely the TOPS, or Trillions of integer Operations per Second. These TOPS are the real numbers that Sony wanted to focus on for rendering efficiency and the actual bandwidth of the PS5 Pro hardware. Sony ended their presentation on the PlayStation 5 Pro deep dive by talking about the future potential for what the next PlayStation could achieve and also a unique collaboration with AMD that they're calling Codename Amethyst. While much of the machine learning architecture on the PlayStation 5 Pro pushes the hardware to its limits, there is still a bottleneck at play in terms of system memory access. This is an opportunity for growth and perhaps to achieve a fully fused network with PSSR and the overall potential for growth with machine learning as a whole. Through the PlayStation 5 Pro, Mark Cerny and his team have developed a good understanding of what the current system architecture is capable of and what is to come in a future console. While details are somewhat scarce compared to the rest of the PlayStation 5 Pro deep dive, Amethyst is the culmination of a unique collaboration with AMD to create a democratized library for machine learning, built upon multiple generations of RDNA and Sony Interactive Entertainment's research and development. Mark Cerny likened the potential of this collaboration to developing a sort of neural network focused on graphics for gaming. This is where future hardware architectures for both machine learning and AI would be integrated and allow developers to use these resources and perhaps guide the technology into unanticipated directions. Sony made a key point that this collaboration would be developed in parallel between the two companies, with developers having the ability to draw from libraries. Rather than this being some matter of proprietary technology solely for Sony, Amethyst is intended to bring broad machine learning across a variety of devices. It's definitely an intriguing initiative, though it's unclear when we'll see it bearing fruit in actual games.

Sony Is Working on the Future of Game Graphics for Everyone - IGN

Now that the PS5 Pro is here, we know that it does deliver slightly better gaming performance, especially whenever a 'performance mode' is enabled. This is largely thanks to the RDNA 2 GPU simply having more compute units and faster memory - but Sony also worked in PSSR, or PlayStation Spectral Super Resolution, to bring AI upscaling to consoles for the first time. This is exciting technology, to be sure, but the way Sony got here is fascinating, and could have repercussions for the entire gaming industry, thanks to an open approach to who gets to use the underlying technology behind these new features. Because unlike an Nvidia graphics card, the PS5 Pro does not have a discrete 'Tensor' core to accelerate its AI algorithm - which potentially means anyone can create a similar neural network, even Microsoft with its next piece of hardware. We recently got a chance to sit down with Mark Cerny and talk about the PS5 Pro's GPU on a more granular level, and he confirmed that the new console is still using an RDNA 2 GPU, just like the base PS5, but with small alterations to each of the cores. These alterations were implemented by Sony to enable the neural network behind PSSR to run natively on the shader core itself, rather than, as Cerny put it, "adding a Tensor unit or something to it." I asked Mark Cerny how Sony did this and he said "rather than use a discrete unit for machine learning, the roughly 200 TOPs of processing power was added directly to the WGP (Sony's streaming multiprocessor)". Further, he stated that there's "not much benefit to having a separate unit process the neural network - the rendering pipeline would just stall waiting for the results." It's not a mystery that GPUs are uniquely gifted at powering machine learning technology, but Sony is essentially using the same hardware to render the game, and also to upscale it. I'm not sure what specific improvements Sony has made to the RDNA 2 GPU, but it does mean that PSSR fundamentally can not add quite as much performance as something like DLSS, which does rely on a discrete Tensor unit inside of each Nvidia graphics card. This is likely because adding a discrete Tensor unit or AI accelerator to the GPU would cause issues for developers who ultimately need to code their games to run on both the PS5 and PS5 Pro. In fact, that's the whole reason Sony is sticking with RDNA 2 and adding select features from RDNA 3, instead of just implementing an RDNA 3 GPU entirely. As Cerny explains: "Drivers in the console world tend to be very, very thin. That's viewed as one of the benefits of consoles - that you can take full advantage of all of the hardware features." Trying to have a console generation with two disparate generations of GPUs would complicate game development too much. In fact, you can just look at the problems Larian faced when trying to port Baldur's Gate 3 to both the Xbox Series S and Series X, and those consoles are both using the same generation of graphics - one is just more powerful than the other. It makes sense, then, why Sony would opt for further customizing RDNA 2 instead of just making a generational jump with RDNA 3. Because while RDNA 3 already has AI accelerators - regardless of whether FSR uses them or not - a new generation of GPU would necessitate a new console generation. And Sony isn't ready for that move quite yet. Instead, Sony made some new tweaks, including adding a hidden cache of DDR5 to run the operating system, to let the new neural network have greater access to the 16GB of GDDR6 memory. One of the best things about AMD graphics cards is they use a lot of software that is open to anyone, regardless of what hardware they're running. For instance, FSR or FidelityFX Super Resolution will work on any graphics card, be it from AMD, Nvidia or even Intel. So, even if it isn't quite as powerful as DLSS or XeSS from Intel, developers can implement it and know that their entire install base can use it. This has made upscaling tech the default for PC games over the last few years, making it much easier to play games at higher resolutions and with more pretty effects. And now that's coming to consoles. While PSSR is only in the PS5 Pro - it has PlayStation in the name and everything - the underlying technology isn't exclusive to Sony. Instead, anyone that wants to implement a neural network for upscaling can build off of what Sony has done with the PS5 Pro. In fact, in his latest deep dive into the tech behind PS5 Pro, Cerny revealed that Sony and AMD are entering into a strategic partnership, called Project Amethyst, with the goal of developing and advancing the future of gaming graphics - for everyone. "This is not about creating proprietary technology or hardware for PlayStation," Cerny told IGN. "It can be used by anyone who wants to use it." Given that we still haven't seen a mid-generation refresh of the Xbox Series X, it means Microsoft could theoretically take advantage of this technology, or at least part of it. On PC, we already have access to machine learning-accelerated upscaling through Nvidia's DLSS and Intel's XeSS, however both of those are locked to their respective graphics cards. FSR is still the only upscaling technology that can be enabled on any graphics card on the market, and if the foundations of PSSR are successful, it could mean gamers get access to a much better upscaling technology. I've been waiting for AMD to announce a version of FSR that works with a neural network to boost its accuracy and quality, and because Team Red worked so closely with Sony on PSSR, it's not that much of a jump to assume that FSR 4.0 will take a lot of inspiration from the PS5 Pro's upscaling tech. This would obviously be huge for high-end gaming PCs running AMD graphics cards, but the biggest benefactor would be handheld gaming PCs like the Asus ROG Ally X or Steam Deck. With those, the amount of horsepower is ultimately limited by the small form factor of the device, plus the fact that it needs to perform under battery power. Right now, the only real upscaling tech you can use with one of those handhelds is FSR, which has limits, especially at 1080p. Likewise, there have been rumors for a while that Microsoft is working on some kind of Xbox handheld. That's nowhere near being confirmed, but having a more reliable upscaling tech would certainly eliminate some of the hurdles of creating a handheld able to handle current-generation Xbox games without too much of a visual downgrade. With how ubiquitous FSR, DLSS and XeSS have become, it's hard to argue against the importance of AI upscaling in gaming. Really, it was only a matter of time before either Microsoft or Sony implemented it into their consoles. What's amazing, though, is that Sony is approaching it in a way that will facilitate the growth of the technology across the industry. Because PSSR isn't being run on discrete hardware, it will never have quite as much of a performance uplift - at least with the same quality level - as something like DLSS that does run on a discrete Tensor core. Cerny confirms this performance cost, stating that there's "a tradeoff - render at a slightly higher resolution using less expensive non-ML upscaling, or render at a lower resolution and use PSSR. Of course, rendering resolution should not be the primary concern, the ultimate goal is the highest possible image quality!" No matter how you slice it, rendering a game at 1080p, then upscaling that to 4K is always going to be easier on your hardware than rendering natively at 4K, even if you add an AI upscaling algorithm to the mix. Hopefully, other hardware manufacturers can take what Sony has started and iterate on it, in order to make gaming at higher resolutions more accessible for everyone, no matter the level of hardware they have access to. We don't have any idea what the next generation of consoles are going to look like, but I am now almost certain every next-generation console is going to be using some sort of machine learning to make the games look better than they otherwise would.

PS5 architect mostly interested in using AI for graphics and gameplay performance

TL;DR: Sony is leveraging AI and machine learning to enhance its PlayStation platform, focusing on innovations like PlayStation Spectral Resolution (PSSR) upscaling and hardware-base modifications in service of ML. Sony is all-in on machine learning technology, and wants to use AI and ML to help transform its lucrative PlayStation platform. Although it has a variety of top-tier studios that have sold tens of millions of games, Sony is a platform-holder first and a game-maker second. This means Sony must always innovate and provide new tools to help make developers' jobs easier. It behooves Sony to optimize the game development process, and it does so through a mixture of organic means via its first-party studios who rigorously test new hardware and technologies like PlayStation Spectral Resolution (PSSR) upscaling, and inorganic means through feedback from external third-party publishers, developers, and partners. That's the main impetus for the PlayStation 5 Pro. The mid-gen console is meant to help ease the burden that developers face when making games, namely in the constraints of the age-old struggle of resolution vs performance. Sony did this by adding in a variety of new tools and technologies that devs can use, including PSSR, which allows for unique enhancement opportunities that devs can scale for their specific games/in-game sequences. Then there's the hardware-based modifications like the +67% beefier GPU, new shader core modifications with ML instruction sets, and an extra 1GB of GDDR5 RAM specifically meant to house the PS5 Pro's OS. While Sony went out of its way to facilitate PSSR and new raytracing tech, going so far as to completely customize and design the PS5 Pro's GPU, console architect Mark Cerny says he is mostly interested in using AI to tackle graphics. We shouldn't expect to see big gen AI shifts through Sony's aspirations. Sony and AMD recently partnered together in a collab codenamed Amethyst, which will help bring next-gen AI tech to games. For Sony, this basically means better PSSR iterations, higher quality training sources, and potentially better hardware iterations indirectly with AMD's help. In a recent interview with IGN, Cerny underlines Sony's position on AI and how the company went through great lengths to allow big upgrades like PSSR through hardware-based modifications. Cerny goes on to say that he is confident that ML/AI tech like PSSR will help drive the next generation of gaming. "One very simple way to look at this is: are we taking the next roadmap AMD technology, or are we, in fact, going in and trying to design the circuitry ourselves - and we chose the latter. And we chose the latter because we really wanted to start working in the space ourselves. "It was clear that the future was very ML-driven. And by that, you know, the world's talking about LLMs and generative AI, but I'm really mostly just looking at game graphics and the boost for game graphics we can get. So, based on that, we wanted to be working in that space." During an extensive 37-minute technical deep dive on the PS5 Pro, console architect Mark Cerny also said that Super Resolution upscaling like PSSR can be a huge benefit for developers. "It's a world where internal rendering resolution is not the primary concern," Cerny said. Lower internal rendering targets gives developers more room to use those freed up resources for other things, further optimizing development and the end-user visual/gameplay experience. PSSR isn't perfect, and it's causing issues in some PlayStation games, but Sony is committed on iterating on this new technology and will continue to train its neural network on new games data as time goes by.

Sony and AMD are partnering to launch a project called Amethyst that will change gaming forever

Just weeks after the release of the PS5 Pro, Sony is already looking ahead to its next major technological breakthrough. This time, the company has decided to partner with AMD once again, the tech leader behind the custom chips in both the PS4 and PS5. Together, they're launching a game-changing project called Amethyst, which aims to dramatically improve gaming graphics and overall gameplay. In a recent presentation, Mark Cerny, the mastermind behind the PS5 and PS5 Pro, shared new insights into Sony's latest console. One of the most exciting reveals was the announcement of Amethyst, a collaboration named after the fusion of AMD's signature red and PlayStation's iconic blue. The goal of this initiative is twofold. First, the companies aim to create an architecture that is optimized for machine learning. This technology will enable more efficient processing of neural networks, especially those used in game graphics. The vision is to push the boundaries of what's possible by integrating fully fused networks for a smoother, more immersive gaming experience. However, the impact of machine learning won't be limited to just graphics. Cerny emphasized that one of the main goals of Amethyst is to make machine learning more accessible to game developers, allowing them to directly implement AI and enhance both gameplay and visuals. In fact, the partnership isn't just about creating proprietary PlayStation technology. Instead, it's about fostering a broad shift in how machine learning can be used across devices, revolutionizing the gaming industry as a whole. On the graphics front, the collaboration will focus on developing a high-quality set of neural networks specifically designed for game visuals. With this, both Sony Interactive Entertainment and AMD will have the flexibility to work independently on advancing the technology. The hope is that this will lead to more detailed graphics and greater use of advanced features like Ray Tracing and Path Tracing in upcoming games. In other words, Amethyst is set to take gaming visuals and gameplay to the next level, bringing us closer to the next frontier of interactive entertainment. Meanwhile, the PS5 Pro continues to perform well in the market, offering an incredibly powerful platform for next-gen gaming. However, it hasn't been without its challenges. Some titles, such as Silent Hill 2 Remake have faced visual glitches despite being labeled as "Enhanced for PS5 Pro." Thankfully, developers are already working on fixes through patches, so players can expect those issues to be ironed out soon. Cerny's recent deep dive into the PS5 Pro's technical specs only adds to the excitement surrounding the console. With its improved ray tracing, faster GPU, and the introduction of technologies like Amethyst, the PS5 Pro seems poised to be a major player in the gaming world for years to come. And it doesn't stop there -- this collaboration sets the stage for what's to come in the next generation of consoles, including the highly anticipated PS6. With Amethyst, Sony and AMD are not only improving the quality of game graphics but also reshaping the future of gaming itself. As exciting as that sounds, we have to wonder: how will these advancements change the way we experience games in the near future? What do you think -- are you excited for the next leap in gaming technology?

Sony explains how it modified PS5 Pro's GPU to enable PSSR neural network AI upscaling

TL;DR: Sony's 37-minute technical breakdown of the PlayStation 5 Pro, led by Mark Cerny, details the console's advancements, including the PlayStation Spectral Resolution (PSSR) neural network. The PS5 Pro features a custom GPU redesign with increased Work Group Processors and enhanced Vector Registers, enabling AI-based upscaling without additional AI-specific hardware. Ever wonder how the PlayStation 5 Pro works? Sony gives us a hefty 37-minute technical breakdown on everything you'd want to know about the new mid-gen upgrade. Sony just released a new PS5 Pro video specifically for enthusiasts. The 37-minute "technical seminar" is hosted by console architect Mark Cerny, who dispels the magic of Sony's new $700 console. One of the more fascinating developments was the lengths in which Sony went to enable its proprietary PlayStation Spectral Resolution (PSSR) neural network. PSSR is Sony's own first-party, console-ready analog of DLSS, FSR, or XeSS. It's an AI-based upscaling solution that utilizes a Recurrent unit Neural Network (RNN), but to enable PSSR, Sony first had to create what it calls a "fully fused network." Okay, so let's go over what that is before we move on. A fully fused network essentially means everything related to CNNs--or Convolutional Neural Networks or the series of calculations and processes that go into upscaling an image without harming image quality--is handled on-chip and then shot back to system memory, and that the entire pipeline stream better or best utilizes its bandwidth to ensure minimized waste. Cerny explains how Sony had to tackle the GPU solution before it could enable PSSR, but this required a custom GPU redesign specifically for the PS5 Pro. "What we really want is a fully fused network. That's the holy grail of neural network implementation. "With a fully fused network, you're reading the input image from a game at the very start, processing all the layers of the CNN internally on-chip, and then writing the results back to system memory at the very end. "There's two problems that we need to solve, though. The first relates to the amount of on-chip memory required. "There's 8 million pixels in a 4K image. If each pixel needs 16 bytes, that's about 128 megabytes. In terms of on-chip memory, that's a lot. Luckily we don't have to process the whole scene at once. We can sub-divide this screen and take just a piece of it at a time through the neural network. "The difficulty comes in that as we process a tile, bad data from the edges creeps in. So we have to throw out part of our results. The smaller the tile, the higher the portion of data that has to be discarded. There are effective limits to how small we can make the tile. And correspondingly, there's a certain amount of fast, on-chip memory that's key if we are to achieve that goal of a fully-fused network. "The other problem we need to solve relates to the bandwidth of that on-chip memory. Our targets are incredibly high, we'd like many many terabytes per second. When you think in those terms, everything seems small. "For example, we could increase the size of the GPU's L2 cache and try to use that for the on-chip memory, but unfortunately, the L2 bandwidth is just a few terabytes per second. "This memory problem was the starting point for our custom design. From there it's been almost a four-year journey." Read Also: PS5 Pro GPU explained by architect Mark Cerny: Hybrid, custom graphics processor with multi-generational RDNA tech To solve the memory problem, Sony modified and boosted the PlayStation 5 Pro's GPU, increasing its Work Group Processors (WGPs) by 67% over the base PS5, going from 18 to 30 WGPs. The PS5 Pro architect also confirms that Sony is using WGP Vector Registers as system memory, thus avoiding any need for additional custom AI/ML-specific hardware. "What we're doing on PS5 Pro is using the Vector Registers in the Work Group Processors as that RAM. "Each WGP has four sets of registers, each 128KB in size, and over a terabyte per second. 30 WGPs therefore give us 15MB of memory and a combined bandwidth of 200 terabytes per second, which is to say, several hundred times faster than system memory. "Of course the roadmap RDNA architecture and instruction set required some modifications to take better advantage of that registered RAM. "We ended up adding 44 new Shader Instructions, those instructions take that freer approach to register RAM access and also implement the math needed for the CNNs, which is primarily done in 8-bit precision. "These instructions are specifically designed to operate in a takeover mode where each WGP processes the CNN for a single screen tile." Sony chose to boost the PS5 Pro's GPU instead of going with a dedicated NPU because NPUs aren't so great at pre- and post-processing required for game content streaming. Mark Cerny also reiterated how Sony modified the PS5 Pro's GPU in a recent interview with IGN:

PlayStation, AMD team up to build next-gen AI, ML architectures for PS6, other hardware

Sony and AMD announce project Amethyst, a new collaboration focused on forging next-gen AI and ML solutions for new console hardware like the PS6, as well as other platforms, potentially even PC and rival platforms. Today Sony Interactive Entertainment dropped a surprise video of PlayStation console architect Mark Cerny discussing the PS5 Pro in a deep dive seminar. Cerny goes through everything, including how exactly the PS5 Pro's GPU works, why it was chosen, as well as PSSR details. But towards the end of the video, Cerny teased what's next for PlayStation. Sony and AMD are working closely together to develop new hardware and architecture solutions optimized specifically for ML and AI. Here's what Cerny said during the presentation about Amethyst:

PlayStation's Mark Cerny did a deep-dive on the PS5 Pro and Sony's new partnership with AMD

PlayStation Lead Architect Mark Cerny is back again to explain the nitty-gritty details of how the achieves its various graphical improvements. Cerny first in September and in , he gets into how the Pro's improved GPU uses tech from AMD and announces a "deeper collaboration" between Sony and the chip maker. The PS5 uses AMD's RDNA 2 GPU architecture originally released in 2020, while the PS5 Pro uses what Cerny refers to in the video as RDNA 2.X. The new GPU is a mixture of what was already offered on the , with some cherry-picked features from the more advanced AMD introduced in 2022. That's paired with ray tracing techniques that Cerny says are from future RDNA tech on AMD's roadmap, and custom machine learning features created for the PS5 Pro. Those machine learning components are also apparently a key part of AMD and Sony's future work together. "AMD has been a fantastic partner for SIE for many years now," Cerny says. "And I'm honored to announce that we have begun a deeper collaboration with a focus on machine learning-based technology for graphics and gameplay." "Amethyst," the name the companies chose for their new project together, is primarily concerned with creating "a more ideal architecture for machine learning," according to Cerny. The new hardware architectures the companies are developing could benefit future consoles and AMD's own GPUs, but they're just one part of the plan. Sony and AMD are also working towards the "democratization of machine learning," which sounds like possible software tools to make it easier for developers to implement AI in gameplay and graphics. The whole video is jam-packed with information on the thinking and engineering that went into the PS5 Pro and worth a watch if you're looking for more detail on what "Pro" means in this case. It might not convince you to upgrade to the new $700 console, but it certainly makes the case that Sony didn't take designing it lightly.