Talk to Gynecologist on Male Infertility Issues

I have been married for 3 years and my wife didn't conceive. After 1 year of my marriage I went for semen analysis and got to know that i have low sperm count of 10m. A doc. recommended me evion and surbex z for 3 months and sperm count reached 12m. Then I went for herbal medicine which costs me loss in sperm count which was a mistake. Then i went to urologist and he suggested me testosterone doplar test upon which he told me that i have grade 1 varicocele and i need a surgery and i will be fine. But i was not willing for having surgery then i went to embryologist he told me that grade 1 varicocele is not dangerous you may have another problem and i have Asthenozoospermia and suggested me a 2.5 months course of medicine i.e fertican, gonadil f. After completing this course my semen results remains almost same as low sperm count. Now i am thinking that i should have a varicocele surgery, maybe my veins are having less blood flow. I want to ask you docs that should i go for varicocele su