The TB-3 was used operationally during the Battle of Khalkhin Gol against Japan and in the Winter War with Finland. Although it was officially withdrawn from service in 1939, at the start of the Great Patriotic War on 22 June 1941, the Soviet Air Force had 516 operational TB-3s, with an additional 25 operated by the Soviet Navy. Stationed far from the USSR's western border, the ТB-3s avoided catastrophic losses during the first German air strikes, after which TB-3s from 3rd TBAP (Heavy Bomber Regiment) began flying night bombing missions on 23 June. A shortage of combat-ready aircraft also required daytime use of TB-3s without fighter escort and in this role the bombers, operating at low-to-medium altitudes, suffered heavy losses to enemy fighters and ground fire. By August 1941, TB-3s made up 25% of the Soviet bomber force and, operated by elite air force crews, were flying up to three combat missions per night. The aircraft participated in all major battles through 1943, including the first Battle of Smolensk, the Battle of Moscow, the Battle of Stalingrad, the Siege of Leningrad, and the Battle of Kursk. On 1 July 1945, 18th Air Army still had ten TB-3s on the active roster.
The TB-3 served extensively as a cargo and paratroop transport, carrying up to 35 soldiers in the latter role. In the first five months of the war, the aircraft transported 2,797 tons of cargo and 2,300 personnel.
The TB-3 was also used in several special projects as a fighter mothership in the Zveno project and for delivering light T-27, T-37, and T-38tanks. On 1 August 1941, a pair of TB-3s in Zveno-SPB configuration, each with two Polikarpov I-16 fighters carrying a pair of 250 kilograms (550 lb) bombs, destroyed an oil depot with no losses in the port of Constanța, Romania. On 11 August and 13 August 1941, Zveno-SPBsuccessfully damaged the King Carol I Bridge over the Danube in Romania. Zveno operations ended in the autumn of 1942 due to high vulnerability of the motherships.
In recognition of the role TB-3 played during the war, three aircraft were included in the first post-war air parade on 18 June 1945.