Most winches are rated at their capacity to pull a rolling load up a 10% incline. Your 4,500 pound winch may only actually lift 450 pounds. The only winch I know of that could actually lift it's rated load was the Megawinch, and I believe they are out of business.
It's pretty easy to test the bigger battery theory, just attach a bigger battery with jumper cables. But first, make sure the vehicle battery is in good shape and fully charged.
I don't have a clue how you use a winch to plow. Are you snow plowing or agricultural plowing? If this plow weighs 450 pounds are you sure the Teryx is enough vehicle for what you want to do?
If you are snow plowing, sub zero weather can weaken a healthy, fully charged battery by as much as 80%! In such cold, you're lucky to even have enough power to crank the engine starter. Forget a winch. There are stick on battery heating pads available, I think in both 12 volt and 110.
You could put your Teryx in a warm garage to see if the winch works better.
If you are indeed overloading your winch, this will cause abnormally high electrical draw as will be evidenced by rapid overheating and damage to the winch.
With anything electrical, there are many other possibilities. Is the winch wiring thick enough? Are the connections good? Is the winch motor damaged in some way?
Is the cable wedged between the drum and the winch body? I had a friend where the very end of the cable passed through the side of the winch drum where it was anchored. Somehow it got out of position and wedged itself between the drum and the rest of the winch, binding the drum. This was hard to spot on a winch that was out in front of the vehicle and would be harder to spot on an ATV winch hidden in the front bodywork. Also, a submarined cable could increase the load on the winch.