Other I'm right-handed, but I prefer charging handle on the left side. Am I doing it wrong?

You can’t operate charging handle on the right side without coming off your target. With semi-auto rifles it’s not a problem as the only time you need to let go of the grip is when you need to swap a mag and cock it. With single shot you have to do it after every shot. If we are talking about firearms than I must note that I always liked HK delight with forward charging handle on the left side. But I totally understand and respect tat everyone has own preference :).

I'm not sure if I understand what you are trying to say, but that is exactly why the handle is on the right side for right hand shooters and on the left for left hand shooters. You support the rifle with your non-dominant hand and push it into your shoulder (2 points of contact), the dominant hand is only responsible for pulling the trigger and manipulating the controls, it does not bear any weight. If you take your non-dominant hand off, you will come off target, because you just created a huge lever.