Just before he jumped, the man, known as Fearless Felix, tweeted "Do you think I will survive this? :)" to his followers. Over 5 million people are following his record-breaking skydive live.
Earlier, the Austrian skydiver passed the 19km point where, without pressurization, liquids boil at body temperature – known as the 'Armstrong line'.
The capsule carrying Baumgartner rose at an average speed of 5 meters per second. He's constantly communicating with mission control in Roswell, New Mexico, as he gets closer and closer to his target altitude of 37 kilometers.
Baumgartner's family and friends were on the ground to see him lift off.
This is the Fearless Felix's second attempt to set a new world record. The first, scheduled for October 9th, was cancelled due to weather conditions.
Millions are following the event, with 'Go Felix' trending worldwide on Twitter.
The man himself managed to post on the microblogging site, saying "Thank you to our sponsor, @redbull & @RedBullStratos for making this possible. Family & friends, see you on the ground."