As an aside, these starters have a reduction gearbox on the end which if dry make high load on the electric starter motor. When you reassemble the starter ensure the gears are well greased and are meshing correctly. If all is correct you should be able to turn the starter motor over by hand when it is off the bike.
Thanks everyone for the help! Got the armature cleaned up and rebuilt the starter. Found out that one of the brackets holding a brush was almost burnt completely off. Just put it back on the bike and started it without any issues. You won't believe how happy I am right now!