I'll agree with the others that its primarily due to lack of exposure.
But if I might elaborate a little... if it weren't for the greed of radio, tv, and record producers pushing lower and lower budget Rock 'n Roll productions fewer people would have been exposed to Rock n Roll than even classical. Its a lot cheaper to pay the salaries of 3 guitarrists, a drummer and a lead singer than an orchestra. And it doesn't help that the traditional symphony orchestra has been growing ever since its inception by Haydn, many of whose early symphonies were little more than a glorified string quartetts with acompaniment by 1 or 2 woodwinds and timpani. Later Hadyn and Mozart incorporated fuller woodwinds, trumpets and horns, beethoven added the trombone. By the time Wagner was perfecting opera, the tuba had been included as well. And many many contemporaries of Wagner expanded the percussion section as well.
In the days before radio and television many free concerts were provided by the wealthy noblemen in Europe to celebrate festivals, coronations, etc etc. The musicians in these orchestras were mostly very poor. Contrast that with today where there are a lot fewer musicians, but even the more successful ones are only slightly better off financially (popular rock band members may earn a hefty sum while they are still popular.... but today its very much like professional sports.... a downturn in the market, a change of taste, etc can end your career early and without a retirement plan...). The exevutives of course keep getting richer and richer.
So the long story is the reason why more people don't like classical music is because classical music is more expensive to produce and influential people in the media industries, wanting to pinch and save every penny don't want the masses to be exposed to classical music. This in turn will lead to a decline in demand for classical music, which means there will be less and less need to hire symphonic orchestras. And so on and so forth.