The traditions tell us that if we find a prospective mate who is a believer and has good morals/ethics then we should marry them. Physical attraction is not a criteria. Yet, being human it is hard not to consider it, thus, it is not a sin or a problem if you were not to marry him. However, consider that maybe this is who Allah has meant for you and that if married, you will find not only attraction but fulfillment as well. It is in the bounds of marriage that true attraction develops