Steve's Personal Response
A long time ago, I used to go with someone, and everything was cool when we were alone, but when we were out in public with our friends, I did not pay her enough attention. This is is similar to how Dimmesdale does Hester, even though his disregards for her are quite more severe. It is not the way that you treat the person you care about. I now realize if you really do care about someone, that they would be objects of your attention whether you are alone with them or out in public.