Khenpo Karthar Rinpoche was born in Rabshu in the province of Kham in eastern Tibet. He was born at sunrise on Mahakala Day, the twenty-ninth day, of the second month in the Year of the Wood Mouse, 1924. On this day, very early in the morning, Rinpoche's mother went to fetch water from the stream, carrying the full vessel of water home. Rinpoche was then born with no pain to his mother.

According to Tibetan tradition, all of these special circumstances show a very auspicious birth. Rinpoche's father was a devoted Manjushri practitioner who constantly recited the Manjushri Sutra. He would go to sleep reciting the sutra and when he woke up, he simply continued with his recitation. His practice was so strong that he was known to benefit even animals when they would die. When Rinpoche was quite young, his father taught him to read and write, as well as study and memorize Dharma texts. Rinpoche decided at a young age to follow the path of his older brothers, who were both monks. At the age of twelve he entered Thrangu Monastery in eastern Tibet.