1) Given that Prophets are leaders for whoever they're sent to, I feel like it makes sense that the verse regarding Ibrahim becoming an Imam makes more sense when you understand it as he went from being a leader for his people, to all of mankind. On what basis would it make more sense to accept the Shia understanding, that this Imamah means a divine rank that's higher than Prophethood? Furthermore, even if we were to accept accept the Shia understanding of this verse, what would be the evidence of accepting which of the descendants of Ibrahim are also infallible Imams? Is it only if they're explicitly referred to as Imams like Ishaaq and Isma'il?

2) Shia scholars use the narrations from Sunni books that mention 12 caliphs to prove their theology to be correct. However, Sunnis will argue that there are variants of the narrations which actually disprove Shi'ism to be true. See the following https://www.twelvershia.net/2015/05/12/hadith-of-twelve-caliphs/
However, let's say for argument sake that we still rely on these narrations. How can we then establish what the identities of each individual caliph is? Furthermore, if the Quran explicitly teaches us about the rank of Imamah using that name, why couldn't the Prophet have just simply taught Muslims there would be 12 infallible Imams after him? For example, Prophet Isa talks about the coming of a Messenger after him, our Prophet in 61:6. It's a clear example without any doubt as far as I'm concerned. Why not specifically have referred to the 12 as Imams, rather than caliphs? Even with Imam Ali, I believe the argument still stands if we still accept that he was supposed to be the rightful leader after the Prophet.

3) I've heard that Shia scholars say the reason why the AhlulBayt are mentioned in the second part of 33:33 is to show the distinction in their superiority compared to the wives of the Prophet. What is the supporting evidence from this that doesn't pre-suppose we can take our understanding of the Quran from Shia sources?
And even if we were to accept the wives aren't part of the AhlulBayt as per some Sunni narrations, what's to stop us from including those outside of Sayeda Fatima and the 12 Imams as the AhlulBayt as Sunni narrations include them too?

1) The verse about Prophet Ibrahim (a) being an Imam is significant in Shia thought because:

A- It indicates Imamate is a divine position. God chooses Imams, not the people. God chooses such religious leaders, not the people. The caliphate or Imamate after the Prophet (s) must be by divine appointment, and not a matter deferred to the people. In 2:30 and 38:26 the Qur’an makes it clear that the position of Khalifah is also by divine appointment. This demonstrates that the 3 caliphs after the Prophet (s) were not rightful caliphs. They were not appointed by God through His messenger. Imam Ali, to the contrary, based on numerous authentic sources, was declared by the Prophet (s) to be his Caliph, Wazir, Imam and the Mawla after him.

B- It indicates that one who is a ظالم, meaning one who is unjust by committing acts of injustice, is not qualified for this position. We believe Imam Ali was infallible and the only caliph qualified for this position. The others were not infallible.

Secondly, in our Sahih hadiths, the Prophet (s) explicitly mentioned who the 12 Imams are. He made references to them, but of course the Umayads and other enemies of the Imams tried to bury such narrations. Just because Bukhari and other Sunni works don’t mention these clear hadiths doesn’t mean the Prophet did not say them. Just as Jesus names his successors, so did the Prophet. This is well established in our hadiths.

Third, the Prophet made it very clear in the famous Hadith of Thiqlayn that his Ahlulbayt or progeny are tied to the Qur’an and will never separate until they meet him at the pool on the Day of Judgment. This hadith confirms they are infallible, for if they were not, then they could not be tied to the Qur’an as such. If they would sin, they’d separate from the Qur’an.

2) As for verse 33:33, though it directly includes the Prophet, Imam Ali, Lady Fatima and Imams Hassan and Hussain, the Prophet and Imam Ali and Imams Hassan and Hussain clearly stated that it also includes the other 9 Imams. This is well established in our hadith sources.

3) Furthermore, we know that 33:33 doesn’t include the wives of the Prophet (s) because in Sura Tahrim (which was revealed after 33:33) the Qur’an condemns two of the wives of the Prophet for their conspiracies against the Prophet, and it is rationally impossible for them to be a part of 33:33 and thoroughly purified by God yet they would do such disturbing things that warranted condemnation from God. How could their hearts stray (66:4) when they are thoroughly purified by God? And there is no doubt Aisha fought Imam Ali at Jamal. How could they both be thoroughly purified when they both fought each other? This creates a discrepancy in 33:33, and there are no discrepancies in the Qur’an.

In sum, our Hadiths indicate the Prophet clearly mentioned the 12 Imams. If you ask why they weren’t clearly mentioned in the Qur’an, then my response would be similar to these responses here: