They already suffered the brunt of that challenge due to the oversupply situation that existed: it forced panel manufacturers to cut costs a lot quicker than anyone expected. Solar panels are down 99% over the past 30 years, and they are at a point where they can't get much cheaper, since all of the low hanging fruit has already been picked.
That said, panels will get cheaper since companies are still cutting costs; but the rate is more like $0.04/W per year rather than $0.40/W per year like it was two years ago.
Solar panels can come down in price about 10% per year or less going forward. Companies are making them at ~$0.50 - $0.55/W and selling at ~$0.60 - $0.65/W. Next year they might make them at $0.45 - $0.50, but even as costs come down 5% - 10% per year, eventually they will hit a bottom as will residential installation costs.
The reason why I am excited about panel manufacturers in the short run, is because demand is going to outstrip supply this year and could lead to higher ASP's. Here is GTM saying that panel prices in US will exceed $0.80/W this year, when they were selling under $0.70 recently:
http://www.greentechmedia.com/articles/read/chinese-module-price-to-increase-20-percent-in-2014
So even if ASP's stabilize and do not grow, then solar panel makers will see margins go up a lot as they continue to cut costs. I can see long term gross margins around 20% - 30%, since anything under 20% and you are losing money but anything over 30% and you are inviting competition. In the future though ASP's will be coming down as well, but roughly 5%-10% per year and not 20% - 30% like in the past or what I expect in the future for residential system installation costs.
There will be a few big winners in solar manufacturing and it will become an oligopoly IMO as barriers to entry become a lot bigger in the future (mostly due to poly shortages and economies of scale).
So the same applies to panel manufacturers in that as ASP's go down, they have to grow in size just to keep revenues flat. But a 5% annual decline in ASP's is manageable, while a 20% - 30% decline can be catastrophic (as it was in the 2011-2012 period).