Apart from the giant yellow bird, the red furry monster and the blue Muppet with an insatiable appetite for cookies, “Sesame Street” appears as real as the New York City streets that inspired it.

It may surprise some to hear that “Sesame Street” has a legacy of musical guests that is arguably only rivaled by “Saturday Night Live” — performers over the decades have ranged from Stevie Wonder to ...