Small was born in 1828 in Edinburgh to Margaret (née Brown) and John Small. He was educated at the Edinburgh Academy and the University of Edinburgh where he graduated with an MA in 1847. In the same year he succeeded his father, who was acting librarian of the university library until his death.
In 1854 he obtained the full status of librarian, with an official residence. He held the office, also in succession to his father, of acting librarian to the College of Physicians (Edinburgh), for which he prepared a catalogue in 1863. He also served for many years as assistant clerk to the Senatus Academicus and editor of the University Calendar. He was president of the Library Association in 1880, and on 21 April 1886 the University of Edinburgh awarded him a LLD. He was for some time treasurer of the university musical society.
Small devoted his leisure time to literary work. His first larger publication was a volume, English Metrical Homilies … Edited, with an Introduction and Notes, Edinburgh, 1862. He was the chief associate of Cosmo Innes in editing the Journal of Andrew Halyburton, published in 1867. Thereafter his chief labour was expended on editing, with careful glossaries and indices, the works of early Scottish poets, viz. The Poetical Works of Gavin Douglas, 4 vols. Edinburgh, 1874; Sir David Lyndesay's Monarchie for the Early English Text Society (1865–6), and The Poems of William Dunbar for the Scottish Text Society (1884–1892). In 1885 he re-edited David Laing's Remains of Early Scottish Poetry, prefixing a bibliographical notice of his predecessor. To the British and Foreign Evangelical Review he sent an elaborate article on the authorship of the Ode to the Cuckoo, and he contributed numerous papers to the Transactions of the Royal Society of Edinburgh and the Society of Antiquaries. He also gave assistance to Sir Alexander Grant in writing the History of Edinburgh University (1884).
Small's nephew was Andrew Munro, (1869-1935), a Scottish fellow, lecturer in mathematics and bursar at Queens' College, Cambridge from 1893 to 1935. The Munro scholarships and studentships at Queens' College, Cambridge are named in his honor.
After a long illness John Small died unmarried in Edinburgh on 20 August 1886, and was buried in the Grange cemetery in Edinburgh.
In 1924, the estate of John Small's sister, Jemima, left £5,000 to establish a fund in the name of both her father and her brother for the purchase of books and to subsidize general purpose expenses within the library.
Besides the works mentioned above, Small wrote:
- Some Account of the Original Protest of the Bohemian Nobles, 4to, Edinburgh, 1861.
- Historical Sketch of the Library of the Royal College of Physicians, 4to, Edinburgh, 1863.
- Biographical Sketch of Dr. Adam Fergusson, 4to, Edinburgh, 1864.
- Biographical Sketch of Patrick Fraser Tytler, 8vo, Edinburgh, 1864.
- A Hundred Wonders of the World in Nature and Art, 8vo, Edinburgh, 1876.
- On Serfdom in Scotland, 4to, Edinburgh, 1878.
- The Castles and Mansions of the Lothians, 2 vols. Edinburgh, 4to, 1878.
- Queen Mary at Jedburgh in 1566 … 4to, Edinburgh, 1881.
Small edited the following works:
- The Indian Primer, by John Eliot, 12mo, Edinburgh, 1878
- The Image of Ireland, by John Derricke, 4to, Edinburgh, 1883
- A Description of the Isles of Orkney, by James Wallace, Edinburgh, 1883
|Wikisource has the text of the 1885–1900 Dictionary of National Biography's article about John Small, Librarian.|
- "Edinburgh University". The Scotsman Newspaper, Edinburgh, Scotland. 23 January 1924.
£48,000 Bequest For Research
- Eliot, John; Small, John (1880). The Indian Primer (facsimile reprint ed.). Andrew Eliot. ISBN 978-1-115-43792-9. Retrieved 20 November 2010.
- Stronach, George (1897). . In Lee, Sidney (ed.). Dictionary of National Biography. 52. London: Smith, Elder & Co.
[Obituary Notice in Scotsman, August 1886; notice of his life by Professor W. P. Dickson in Library Chron., December 1887.]
- John Small at Find a Grave
- John Small and Son, University of Edinburgh Portrait
- John Small - Edinburgh University, Gallery of Benefactors