There is really no way of knowing whether a man had a vasectomy, other than his
telling you that he did. Although there might be small scars at the top of the
scrotum, they are usually quite hidden by the normal wrinkles of the testicular
sac.
I agree that pregnancy is a huge risk to take if you have any concerns. If you
have any doubts about his honesty in this regard, then perhaps it might be a
good idea for you to be using condoms anyway (even if he can prove he had a
vasectomy) for safer sex purposes.
If you really want to find out whether you can stop using birth control without
directly confronting him about it (or asking him to produce a medical report),
what I would suggest is the following: It is an established fact that in rare
cases a vasectomy can reverse itself after a number of years. Perhaps you could
suggest to him that he have a sperm count done just to be sure. Of course, if
you really don't trust him in this respect, I guess you will have to find a
way to get a copy of the report to make sure.
My other piece of advice is to reflect a little on what is going on in this
relationship. Although you don't have to love someone to have sex with them
(or even necessarily know them that well), you should have some basic trust
that they would not knowingly jeopardize your health or safety. Good luck to you.