Ira Kaplan and Georgia Hubley, a husband/wife duo, formed the band in 1984. They went through several other band members before stabilizing with lead guitarist Dave Schramm and bass player Mike Lewis (founding bass player of Boston garage-punk mainstays DMZ and Lyres, and a member of Brooklyn garage band The A-Bones throughout his tenure in Yo La Tengo) for their debut recording, "The River of Water". In 1986, their first LP Ride the Tiger was released.
Schramm and Lewis left the band, and Stephen Wichnewski joined, with Kaplan taking on the role of lead guitar. New Wave Hot Dogs (1987) did much to establish the band's reputation among rock critics, though it sold poorly. President Yo La Tengo (1989) continued this trend, with rave reviews yet poor sales.
1990 saw the release of Fakebook, an album of mostly acoustic tunes, including covers from Gene Clark, Cat Stevens, Rex Garvin & the Mighty Cravers, The Escorts, The Flamin' Groovies, The Scene Is Now, the Kinks, the Pastels, Daniel Johnston, and several songs by Yo La Tengo themselves. May I Sing with Me (1992) included new bassist James McNew (of Christmas and Dump), who has stayed with the band ever since. Painful (1993), Electr-O-Pura (1995) and I Can Hear the Heart Beating as One (1997) marked a steady progression towards a sprawling, multi-faceted style, which included such disparate elements as folk, punk rock, shoegazing, long instrumental noise-jams and electronica oriented songs. Painful was also the beginning of the band's fruitful creative relationship with producer Roger Moutenot, who has produced every subsequent Yo La Tengo album release. During these years their cult grew and, thanks to almost constant touring and unrelenting critical praise, they became one of the most prominent American indie rock bands. They released And Then Nothing Turned Itself Inside Out in 2000 and Summer Sun in 2003.
The band is renowned for its encyclopedic repertoire of cover songs, and every year plays live on the New Jersey freeform radio station, WFMU, as part of the station's annual fundraising marathon, performing (with guitarist Bruce Bennett of Norton Records The A-Bones, and occasional additional guests like Hamish Kilgour of New Zealand's The Clean and Pacific Northwest folk-punk chanteuse Lois Maffeo) impromptu cover-song requests from listeners who call in to pledge money to the station. In 2006 the band released an album compiling performances from the marathons between 1996 and 2003 titled Yo La Tengo Is Murdering the Classics.
In 1996 Yo La Tengo appeared briefly (along with their friend Tara Key of the band Antietam) as the Velvet Underground in the film I Shot Andy Warhol. In 2001 they recorded an instrumental soundtrack for eight short undersea documentaries of Jean Painlevé, entitled The Sounds of the Sounds of Science. Yo La Tengo also provided the soundtracks for the films Junebug, Game 6, Shortbus, as well as Kelly Reichardt's award-winning 2006 feature film Old Joy. Three songs from May I Sing With Me ("Always Something", "Sleeping Pill" and "Some Kinda Fatigue") were featured on the soundtrack to American indie filmmaker Hal Hartley's 1992 movie Simple Men. Their song "Tears Are In Your Eyes" was also in the "Family" episode of Buffy the Vampire Slayer.
The group collaborated with Yoko Ono as well on the 2003 album Wig in a Box, Songs From and Inspired by Hedwig and the Angry Inch," made as a charity album to support the Harvey Milk High School.
March of 2005 saw the group release a 2-disc greatest hits package, entitled Prisoners of Love. A bonus edition included a 3rd disc of rarities and unreleased tracks.
The album I Am Not Afraid of You and I Will Beat Your Ass followed in September 2006.
In 2007, Yo La Tengo's covers of the Bob Dylan songs "I Wanna Be Your Lover" and "Fourth Time Around" were released on the 2-CD/4-LP soundtrack album for I'm Not There.