This guy comes across as also having low self esteem in the reading I have drawn for you. Although you both have a lot of attraction and admiration for each other, both of you are scared to label this relationship a committed one.
He does think that you are great and the reason you are receiving mixed signals from him, is that he is unsure of what you expect from him. You mention that both of you are in denial to your friends and to yourselves about what is really going on between you. The cards indicate that you now need to relax and enjoy the situation and also to put him in a situation that he has to work harder to be with you on a physical level. Only by doing this will he be able to distinguish the real feelings that are sometimes hidden away and confused by the sexual side of the relationship.
The only reason that you are talking about forgetting this relationship is that you have realised that you are in love with your friend. This aspect scares you. It is important that the subject of where you are going is not approached at this time, instead let the relationship develop. Maybe you should start letting him see a small part of the vulnerable side of you, rather than trying to pretend that you are cool with what has been going on.
Either way this needs more time to develop into a loving, honest and meaningful relationship.
Take care and enjoy yourself!