Born in Sheffield, England, Varley studied – as did his childhood friend Arthur Lismer – at the Sheffield School of Art and at the Antwerp Academy, in Antwerp, Belgium. From Antwerp, he returned to London, where he almost starved trying to support himself as an illustrator. Four years later, on his return to Yorkshire, he married and fathered 2 children. In 1912, Lismer once again met up with Varley – who this time was depressed and struggling to support his family. Lismer persuaded him to come to Canada, where he found work first at Grip Ltd. and then at Rous and Mann where other Grip artists, such as Tom Thomson and Franklin Carmichael, had already gone.