I was talking to a Sunni and he told me that we don’t have any primary sources for the seera of the prophet which makes their sources hujjah on us. Is that true?

That’s not true. We have the holy Quran and our four major Hadith books

1. Kitab al-Kafi of Kulayni (divided into Usul al-Kafi, Furu al-Kafi and Rawdat al-Kafi)
2. Man La Yahduruhu al-Faqih of Shaikh Saduq.
3. Tahdhib al-Ahkam of Shaikh Tusi.
4. al-Istibsar of Shaikh Tusi.