Price was born on 1 March 1735 in Llandegla, Denbighshire, Wales, where his father was the rector. He was educated in Llangollen (his father having become the vicar there in 1737) and at Jesus College, Oxford, where he matriculated in 1754, obtained a Bachelor of Arts degree in 1757 and a Master of Arts degree in 1760, the year in which he was ordained. He was initially appointed janitor of the Bodleian library by Humphrey Owen in 1757, becoming sub-librarian in 1761. He was acting librarian from 1765 to 1767 when Owen was Principal of Jesus College, succeeding as librarian upon Owen's death in 1768 after an election (in which he defeated William Cleaver, who later became Principal of Brasenose College, Oxford and Bishop of Chester).
He remained librarian until his death in the night of 11 to 12 August 1813, a period of 45 years, but was criticised during this period for neglecting his duty and the manner in which he organised the library. He also held various appointments as a parish priest in England and in Wales, and wrote some works such as a Short Account of Holyhead in the Isle of Anglesea (1783). His collection of books, manuscripts, maps and prints took five days to sell after his death.