I've had that same concern, and IMO it is still real. Recycling panels can apparently be done, but not economically yet (happy to be corrected on this if folks know more). The risk is only somewhat mitigated by the long useful life of solar panels. Much work seems to be going into this field though.
One of the folks working on this is SolarCycle.
We offer solar asset owners a low-cost, eco-friendly, comprehensive process for retiring solar panels and repurposing them for new uses.
www.solarcycle.us
General recycling progress overview:
Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News! Lithium battery recycling is an important part of protecting the environment, as it can reduce the amount of raw materials used to create new batteries and decrease the amount of waste generated by...
cleantechnica.com
One specific recycling arrangement:
Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News! Silicon Ranch, a major US-based independent power producer, has obtained an extra 1.5 gigawatts of cutting-edge thin film solar modules from First Solar, Inc. The modules, which belong to the Series 6 Plus...
cleantechnica.com
As far as I can tell, Tesla is letting other folks focus on this particular problem - although to their credit they include the cost of recycling the solar panels in Master Plan Part 3.