James was born on 17 June 1906 and lived in London. He studied at Clapham School of Art between 1923–27, and then Royal College of Art 1927-30, where his teachers included William Rothenstein and Allan Gwynne-Jones.
He married the fashion artist, designer and writer Phyllis Stroudley, they had one son. He later married Gillian Stroudley (née Thain), they had two sons and one daughter.
He died in May 1985, at Wandsworth, London.