Married (1763) to Catharine Kinloch (1736-?), a daughter of James Kinloch and Anne Marguerite Wild. They had at least one child, their daughter Anne Elizabeth Brown (1764-1842), see also "Life with Boswell" below.1
In 1757 he succeeded his brother, William (1719-1791), as Minister of the English church at Utrecht (1757-1777), when the latter returned to Scotland to become Professor of Theology at the University of St. Andrews.2 He was even British Agent at Utrecht from 1763 until his death in 1777.3
Brown was one of Boswell's closest friends during his stay in Utrecht 1763-64. He is first mentioned in Boswell's memoranda of September 18, 1763, when Boswell was to have tea with Brown. Boswell often visited and dined with Brown and his family during the next year, and he attended the baptism (with Lord Marischal as godfather) of Brown's first child, Anne Elizabeth, on June 17, 1764.