From its inception in 2004, Tory Burch  projected an idea of leisure and refinement, outfits that could be worn while pruning the hydrangeas of an estate but were also beloved by suburban moms making midday trips to Kroger. Gallery owner Jeanne Greenberg Rohatyn, a friend for many decades, remembers Burch showing her some of her father’s golf sweaters very early on: “She had this notion of an old-world Americana sportswear that people wore in country clubs. She understood that there was a niche business missing, and she wanted it affordable.”

The languorous aesthetics that the brand radiates can obscure just how much scrappy hustle it took to get it off the ground. The company’s first investors were friends and family, who were asked to put in “only what they could afford to lose,” Burch says, which was sometimes just a few thousand dollars. Even then, Burch had an idea that she wanted a broader purpose for the business. “I’ll never forget, I sat on the couch of one of our first investors and I said, ‘I want to start a global lifestyle brand so I can start a foundation.’ And he was like, ‘Don’t ever say that.’” (The Tory Burch Foundation was officially launched in 2009 and has distributed $2 million in grants to female business owners and many millions more through a loan program; its current goal is to add $1 billion to the economy by 2030.)

As the company grew, Burch desperately wanted to keep her personal life separate from her professional one, and that wariness made her closed off at times, she acknowledges. But she had good reason to desire privacy: She and Chris Burch were going through a protracted divorce. (They’re now friends.) When I ask her what she thinks people misunderstand about her past, it’s the idea that she strolled into success. “I think people have this perception that things were easier,” Burch says. “But just physically, the amount of work, and the hours from eight in the morning to ten at night, every night. Sometimes falling asleep at four in the morning, and doing it all with little babies.”

“The idea that women have to do more with less,” says the businesswoman and financier Mellody Hobson, now a good friend of Burch’s, “it’s just a given. We don’t even talk about that.” When we met in her office, Burch discussed this time in her life with some care, aware that she did not want to complain while sitting on her plush sofa, sipping a Diet Coke, with cloth napkins set out by her assistants. “What I’m saying,” she articulated, “is that it was excruciating, the amount of work it took to build this company. And women are held to different standards.”