Because the combination of song writer and recorded music displaced it. A song is much more accessible to the average person than a poem, and technology made songs widely available. Poetry responded to its diminished status by retreating into narcissistic incoherence. Sure the philistines may not recognize the true greatness of modern poetry, but at least poets could take comfort in their own self-declared cultural superiority. When public rejection became a necessary characteristic of great poetry, then there was no longer any hope.
It’s not correct to say a poet is the same as a song writer. It’s easier to write lyrics for a song because the song writer can lean upon the music. The poet has only words (and at one time such archaic concepts as rhyme and rhythm and meaning and significant subject matter) to carry his message. Poetry is a much harder art form. But better Dylan than a modern poet who composes what passes for poetry in this day and age. Better to let the dead decompose undisturbed.