Knies scored his 22nd goal at 6:21 of the third, less than three minutes after Will Cuylle tied it for New York with a short-handed goal.
Oliver Ekman-Larsson and William Nylander also scored to help Toronto win its fourth straight National Hockey League game since the 4 Nations Face-off break. The Leafs won seven of eight games in February.
Mika Zibanejad also scored for the Rangers. They had won two in a row.
Ekman-Larsson opened the scoring on a power play 1:47 into the contest. Zibanejad tied it with 6:52 left in the period, deflecting a pass from defenseman Will Borgen.
Nylander scored 51 seconds into the second, jamming a loose puck past goalie Igor Shesterkin for his team-leading 34th goal.
Cuylle tied it at 3:47 of the third with the short-handed score.
Maple Leafs: Toronto moved a point ahead of the idle Florida Panthers for the Atlantic Division lead.
Rangers: New York remained four points behind Ottawa and Detroit, current holders of wild-card playoff berths.
Stolarz made a dazzling glove save just after Rangers tied it at 2. Knies scored the winning goal shortly thereafter.
Stolarz improved to 13-5-2 in a backup role to starter Joseph Woll.
Maple Leafs: Visit the Pittsburgh Penguins on Sunday.
Rangers: Host the Nashville Predators on Sunday.
