Harper was born at the Good Samaritan Hospital in Suffern, Rockland County, New York. Her Canadian-born mother, Iva (née McConnell), was a nurse, and her father, Howard Donald Harper, was a lighting salesman. Harper is of French and British ancestry, and was raised Roman Catholic, although at an early age she "quit" the church. The family moved every two years because of her father's promotions, living in South Orange, New Jersey. St. Andrews in Pasadena, California (where Harper attended the school St. Andrews); Monroe, Michigan, at age 10, where she attended St. Mary's Academy; then Ashland, Oregon "at [age] 11 to 13"; and then Jersey City, New Jersey from eighth grade through age 18. When her family returned to Oregon, Harper remained in the New York City area to study ballet. She has an older sister, Leah; a younger brother, Don (originally Merrill); and a half-sister, Virginia, from her father's second marriage. Harper based her future character Rhoda Morgenstern both on her Italian stepmother, Angela Basilico, and Penny Ann Green (née Joanna Greenberg from Brooklyn), with whom she danced in the Broadway musical Wildcat.
Early TV-comedy influences included Lucille Ball, Jackie Gleason, Joan Davis, Your Show of Shows, and, later, Mary Tyler Moore in The Dick Van Dyke Show. She attended Lincoln High School in Jersey City, and finished high school at the private Young Professionals School on West 56th Street, where classmates included Sal Mineo, Barbara Dana, Tuesday Weld, and Carol Lynley. She has been married to Tony Cacciotti since 1987.
Harper began as a dancer/chorus girl on Broadway in the late 1950s and early 1960s. She began with the musical Li'l Abner and went on to perform in several Broadway shows for Michael Kidd. Her Broadway shows included Wildcat, in which she performed with Lucille Ball; Take Me Along with Jackie Gleason; and Subways Are For Sleeping. In-between she was cast in Destry Rides Again but got sick and had to leave during rehearsals. Her roommate Arlene Golonka introduced her to Second City improvisation theater and to improv performer Dick Schaal, whom Harper later married in 1965. Harper was stepmother to Schaal's daughter, actress Wendy Schaal. They lived in Greenwich Village at the corner of Perry Street and Bleecker Street.
Harper returned to Broadway in February 2010 starring as Tallulah Bankhead in Matthew Lombardo's Looped at the Lyceum Theatre.
She also appeared in a bit part in Li'l Abner (1959) when she was a teenager, playing one of the Yokumberry Tonic wives. She broke into television with an episode of the soap opera The Doctors, "Zip Guns can Kill" and was an extra in Love with the Proper Stranger. She toured with Second City with Schaal, Linda Lavin and others, and with Schaal and Skitch Henderson did a New York City talk show. She appeared in sketches on Playboy After Dark. Harper and Schaal moved to Los Angeles, California in 1968, and co-wrote an episode of Love, American Style.
Harper was doing theater in a small theater on Vermont Street in Los Angeles, where casting agent Ethel Winant spotted her and in 1970 called her in to audition for the role of the wise-cracking Jewish New Yorker Rhoda Morgenstern on the landmark CBS TV sitcom The Mary Tyler Moore Show. She co-starred from 1970–1974 and then starred in the spin-off series, Rhoda (CBS 1974-1978) in which her character returned to New York. She won four Emmy Awards and a Golden Globe Award for her work as Rhoda Morgenstern throughout this period. In 2000, Harper reunited with Mary Tyler Moore in Mary and Rhoda, a TV movie that brought their iconic characters back together again in later life. The first season of Rhoda was released on DVD on April 21, 2009 by Shout! Factory.
She was also nominated for a Golden Globe for "New Star of the Year" for her role in 1974's Freebie and The Bean. Harper was a guest star on The Muppet Show in 1976, its first season.
Harper returned to situation comedy in 1986 when she played family matriarch Valerie Hogan on the NBC series Valerie. Following a salary dispute with NBC and production company Lorimar in 1987, Harper was fired from the series at the end of its second season. Harper sued NBC and Lorimar for breach of contract. Her claims against NBC were dismissed, but the jury found that Lorimar had wrongfully fired her and awarded her $1.4 million plus 12.5 percent of the show's profits. The series continued without her with the explanation that her character had died off-screen. In 1987, it was initially renamed Valerie's Family and then The Hogan Family, as Harper was replaced by actress Sandy Duncan, who played her sister-in-law Sandy Hogan. NBC canceled The Hogan Family in 1990, but it was picked up by CBS for one more season.
Harper has worked almost exclusively in theater and television, but did have key supporting roles in Neil Simon's Chapter Two in 1979 and Stanley Donen's Blame It on Rio (1984) opposite Michael Caine. She has had roles in TV movies and guest spots on a number of series, including Melrose Place in 1998 and Sex and the City in 1999. Also in the 1990s, she advocated hormone replacement therapy for Eli Lilly and Company.
In the 1970s and 1980s, Harper was involved in the Women's Liberation Movement and was an advocate of the Equal Rights Amendment.
Harper is a member of the Screen Actors Guild (SAG) and ran for president in the 2001 election, losing to Melissa Gilbert. However, she serves on the Hollywood Board of Directors of SAG.
In 2005 through 2006, Harper portrayed Golda Meir in a US National tour of the one-woman drama Golda's Balcony. A film of this production was released in 2007.
She played Tallulah Bankhead in the world-premiere production of Matthew Lombardo's Looped at the Pasadena Playhouse in California in a June 27 to August 3, 2008 run, and at Arena Stage in Washington, DC, in 2009. The play had a brief run on Broadway at the Lyceum Theatre, from February 2010 (previews) through April 2010.
Harper played the role of Claire, aunt of Susan Delfino (Teri Hatcher), on ABC's Desperate Housewives in 2011.