Son of James Fergusson of Badifurrow, who bought the estate of Pitfour in Aberdeenshire in 1700. Married (1733) to Anne Murray (1708-1793), a sister of the 5th Lord Elibank. Dean of the (Scottish) Faculty of Advocates until 1764 when he was raised to the bench as Lord Pitfour. Member of The Poker Club.
Ferguson had 3 sons and 3 daughters, including Patrick Ferguson, who died in the battle of King's Mountain in 1780, James Ferguson, who sat for 30 years as a Member of Parliament, and George Ferguson (1748-1820), who became Lt-Governor of the Caribbean island of Tobago.
Ramsay of Ochtertyre wrote of Lord Pitfour that he
"was one of the greatest and most popular lawyers of that period, and also a man of probity and amiable disposition. . . . Though his small shrill voice and awkward person prevented him from being an elegant speaker, yet so deeply learned was he in the philosophy of the law, and so well acquainted with the springs that actuate the human heart, that few barristers were heard with more satisfaction."1
- 1 Cited in Fergusson, James & Fergusson, Robert Menzies (1895). Records of the clan and name of Fergusson, Ferguson and Fergus; (1895), p. 250
Boswell dined with Lord Pitfour on November 9, 1762, describing him as "the greatest lawyer in Scotland and one of the best of men, possessed of good sense, honesty, and meekness". They also met in London on various occasions throughout the spring of 1763. In various journal entries and letters, he writes about imitating Pitfour in some way or other.
On July 29, 1763, shortly before leaving for Utrecht, Boswell wrote to Pitfour "that his prophecy that I would return to the law will be fulfilled; and I say that if a saint nowadays obtained the gift of prophecy, none can have a better chance for it than Pitfour."