I did something similar recently: took a ThinkCentre M90q Gen 5, with an i5-14500 CPU and 32GB RAM, and then rammed a 16GB RTX 2000 Ada into it using a custom cooling solution from a modder in Belgium. Nuked Windows and installed CachyOS Linux on it. Works beautifully. Fun project.
ITG Gear has modified a Lenovo ThinkStation P340 Tiny with a rare AMD Radeon RX 7400 8GB GPU and an i7-10700 CPU. The build is a great exhibit for the gaming potential of this compact workstation, especially for fans of small form-factor machines.