The answer is probably not the motherboard, no. This is what it lame about office prebuilds (unlike the gaming prebuilds you see alongside them on the bestseller charts on Amazon). They aren't built on the universal ATX form factor. For this reason the motherboard likely won't mount properly in a new case you buy. There's not really anything magical about mounting. The idea is simply to fasten the motherboard as securely in place as possible, and depending on how the motherboard is configured, it's usually ideal to orient it in a certain way (so that some ports are nearest to the front, and so that the CPU fan blows from front to back). For this reason, some DIYer's who are comfortable using power tools will measure the non-universal motherboard, drill holes into some cheap case that's big enough to house recycle components, and install their own standoffs to mount the motherboard. That's quite a hassle, and probably not the best or easiest strategy for you moving forward.
Now the good news. Everything else can easily be switched into a new case once you purchase a new motherboard on a universal form factor, or at least establish that the one you have already is. The only downside, here, is that the LGA 1155 motherboards you need for that CPU are out of production, and tend to come at steeper premiums from the traditional retailers in brand new condition. You can see here if you first hit "Edit This Part List" and then go to "Choose a motherboard" what the compatible options are (the cheapest starts at $157):
https://pcpartpicker.com/list/DpH9TW
Your best bet is to scour eBay, where a lot of older hardware sells, which isn't tracked by PC Part Picker, or go look for guys selling/trading LGA 1155 motherboards here:
https://www.reddit.com/r/hardwareswap/
Once you have a new motherboard the most delicate operation you'll encounter is extracting your CPU & CPU Cooler from your old motherboard without damaging them. Either find a friend who knows how to do that, or watch YouTube tutorials. Everything else will be a simple transfer. Your hard drive might have a plastic shell or some other mounting fixture in the Dell case that you must shed for your new case, but that will be pretty straightforward. The GPU will run just fine in an LGA 1155 motherboard, the RAM can be easily transplanted, and the new PSU will fit in an ATX case pretty much no matter what it is (but I suspect it's an ATX).