Since I've been baligh, I've prayed and fasted on and off and I have no idea how many prayers/fasts I've missed. I'm pretty sure some years I haven't missed anything, and other years I've missed the majority.

How do I determine how many prayers/fasts I have to make up? Also, some of the fasts missed were intentionally broken while some weren't, so how do i determine how many i have to do kaffarah for?

You have to make up the number you are certain about.
That is, if the number of missed days is either 40 or 50 days, you can assume that it is 40.
The same thing applies to the kaffarah.