Regarding this there are numerous verses in the Holy Quran and i will mention one that directly talks about it.

And as for him who fears to stand in the presence of his Lord and forbids his own soul from its whims and caprices then surely Paradise is the abode. (79:40 & 41)
Whims and caprices are none other then sinful instigations set up by ones own vicious self and shaitan.
So, it is a basic fact that we must have self-control in order to refrain from these whims and desires. If we believe that we should just do what we want by satisfying and gratifying ourselves, then there is no point in talking about spiritual direction.
The habit of falling into a sin begins will lack of self control and will power and to build it, there are numerous traditions on how to go about it.
The first step is to awaken ones heart and sincerely repent. Allah loves those who continuously ask for His forgiveness even if they flip and commit some mistakes that they had not intended to along the way.
Great scholars like Ayatullah Bahjat said that if one were to strictly follow the sharia, ie refraining from the forbidden acts and performing all the obligatory actions, one would purify himself and become close to Allah s.w.t.
Whilst doing this, give charity in the way of Allah as much as you can because if you detach the thing you love like property and wealth, then you will begin to taste what purity is all about inshAllah.
Remember, Islam lays for us simple rules to follow and there is nothing complicated on how to go about it. Everything is in our innate disposition to follow and this is but a reminder for us.
The guilt you have is because you innate self wants to be the best infront of your creator, who is pure and wants His servants to also be pure so that they can get closer to Him and attain perfection.