Olsen was born in Eugene, Oregon, the son of Jeanne (née Donstad), a non-denominational chaplain, and Paul V. Olsen, a professor of English, and head track and cross country coach of Augustana College in Rock Island, Illinois. He has an older brother, David, who acts as his stunt double; David is married to actress Daniela Ruah. Olsen is of Norwegian descent. Olsen spent most of his childhood in Bettendorf, Iowa and attended Bettendorf Middle and High School, where he pursued interests such as sports, Japanese, and Chinese. In addition to many local theatre performances, Olsen trained in improv with ComedySportz Quad Cities and later joined the cast.