Low cholesterol low testosterone connection?

You see people like pro athletes with the highest testosterone levels they also tend to be very healthy people who don't drink alcohol much. I used to drink a load of beer and Smoke tobacco im 30 male and have poor sex drive now and again for the last ten years it is now getting worse. Cholesterol is made in the liver you need it to make Testosterone simple as that. I feel pretty good now that I quit drinking but liver damage even if mild scarring occurs is permanent. At the doctors I was told I had low cholesterol and was very healthy but I think lower cholesterol is bad it actually causes hearts disease you need it to make hormones and vitamin d whats your take on this idea?